Disallow manufacturers from being indexed by search engines in robots.txt

[mallrat]

Hi there

 

I need to stop search engines from indexing all the manufacturers added to a site, but I cannot work out the directory structure to put in the robots.txt file.

 

I am using the Prestashop default theme.

 

I have tried adding this line to robots.txt

 

Disallow: */manufacturers/

 

But when I try the updated robots.txt in Google Webmaster Tools and submit a test for a manufacturer it tells me that it is allowed.

 

Any ideas anyone?!

 

Thank you.

Edited April 14, 2016 by mallrat (see edit history)

[shokinro]

Based on my knowledge, Since regular express is not able to use in robot.txt, you have specify one by one.

you can try as following

 

Assume - PrestaShop SEO friendly is enabled, and URL rewrite rules are default, no other 3rd party SEO related modules installed.  

 

 

For manufacturer list page

 

Try this is your store has only one language installed

Disallow:/manufacturers

Try this is there is multiple languages installed, assume (en,es,fr are installed)

Disallow:/en/manufacturers
Disallow:/es/manufacturers
Disallow:/fr/manufacturers

For each manufacturer, you may have to specify one by one because regular expression does not work.

 

If there is only one language installed

Disallow:/1_*
Disallow:/2_*

If multiple language is installed,, assume (en/es/fr) are installed

Disallow:/en/1_*
Disallow:/es/1_*
Disallow:/fr/1_*
Disallow:/en/2_*
Disallow:/es/2_*
Disallow:/fr/2_*

Where 1 and 2 are ID of manufacturer.

 

 

You need to consider to create a tool to generate this automatically.