Jump to content

how to increase the crawling of my web? also robot is blocking the following...


cutecat

Recommended Posts

The automatic generated sitemap by default add line to disvow the controller=authentication. You can remove it by editing your robots.txt file.

 

Please be clear about what URL and Why you you want to unblock them as there are some strong reason that backs the point that controller URLs should be blocked.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...