Jump to content

[Solved] Manually add "disallow" to robots.txt file


Recommended Posts

I would like to disallow bots to crawl a certain directory on my server. It is not possible to add this manually to the robot.txt file like adding information to the .htaccess file via de BO. Besides that it's not possible to overwrite the robots.txt file via ftp and even if it would be possible the manually added line will be removed when generating a new robots.txt file via de BO. Does someone know a good solution for this?

 

Additional details why?

The directory that I don't want to be crawled contains a flash banner that shows on virtualy every page of the shop and the text of the flash file is indexed by google. To be precisely the text of the flash banner is shown in the description of any indexed page in google, even though I have a unique meta description for virtually every page. Normally the meta-description of a page should be the page description for indexed pages, however google finds flash text more important.

 

To resolve this I thought that disallowing the directory to be crawled should resolve the problem.

 

I you have any other idea's? They would be welcome too!

 

Thanks in advance for any replies.

Edited by impressed (see edit history)
Link to comment
Share on other sites

Actually protecting the file from writing is a very simple and effective solution. Thanks for that, however I think that some users might be interested in being able to modify the robots.txt file from the BO. This is something that the prestashop developers need to find out as well, maybe they agree and will add this functionality in later versions.

Link to comment
Share on other sites

  • 3 years later...

I have faced pretty much the same problem. My new website has just been crawled by google, and I have unique meta desciptions for any subpage, but my shop information is shown as meta description in every page when I do a search of my website in google. Anybody who has an idea to what to do to get my customized meta descriptions shown in google?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...