Some links are indexable and some others are not when they are all created equal, why?

by Rosamunda   Last Updated August 13, 2019 02:04 AM - source

I'm trying to create a sitemap. I'm using the screaming frog software, but I think that this can be achieved in different ways with other software as well.

My links are generated with different parameters using php and they all look like mywebsite.com/content.php?id=123. I have a rewrite.php file that takes a clean url alias typed by the user (e.g. mysite.com/mi/nuevo/mensaje) and redirects the user to the proper ugly location.

For example:

mywebsite.com/mi-mensaje is read by the rewrite.php script and is displayed in the browser as mywebsite.com/content.php?id=523, or

mywebsite.com/mi-otro-mensaje is displayed as mywebsite.com/content.php?id=268, or

mywebsite.com/legislacion/mi-otro-mensaje is displayed as mywebsite.com/content.php?id=274

and so on...

PHP reads stuff from a MySQL database in order to know where to redirect the user.

Now, when I analyze the results with screaming frog, I am effectively told, some aliases can be indexed, getting a 200 status result, and some others cannot, getting a 301 or 302 status. The funny thing is that all aliases work the same way. They are all redirections from an ugly ...?id=X link.

Here's as example of what I mean, here are two aliases:

Here are two aliases

In the above screenshot I am comparing the selected (green) link, with the link immediately above it. As you can see the status is different even though these 2 links work normally.

Here is the resultant output for the above highlighted link:

Here is the ok one

And here is the output for other link that I am comparing it against.

the other one non indexable

As can be seen the 2 links are generated in the exact same way, but the second link is non indexable.

In both cases they do have a redirect URL. My question is why is the first link indexable and the second not, when both links are similarly generated. Why status 200 for one link and 302 for the other? I can access both links by their clean alias.

Here is my .htaccess file:

<IfModule mod_rewrite.c>
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{SCRIPT_FILENAME} !-d
RewriteCond %{SCRIPT_FILENAME} !-f
RewriteRule ^.*$ ./rewrite.php
</IfModule>


Related Questions




Forcing the removal of query string on a rewrite rule

Updated February 16, 2016 00:14 AM


Rewrite url dont work

Updated April 05, 2018 20:04 PM