Duplicate content

[brih]

Hi guys,

 

I hope someone here can help. I have a prestashop site and was wondering about the double content. I have now found a product on google, with different url's.

 

As an example i have the following:

 

www.mysite.com/this-product.html

 

www.mysite.com/en/this-product.html

 

www.mysite.com/de/this-product.html

 

I really want the products indexed with the language, so I am planning to use either robots.txt to remove all products in teh database, however I was just wondering if I can make a disallow a bit more clever than that ?!

 

E.g. I want to disallow all urls that doesnt contain the /en/ or /de/ part and I want to allow url if they actually have the /en/ or /de/

 

As a suggestion would this work ?

 

Disallow : /*

Allow : /en/*

Allow: /de/*

 

Has anyone any experience with that ? Or any warnings.

 

All input is sincerely appreciated.

 

Thanks in advance.

 

 

[NemoPS]

Hm, I'm no seo [spam-filter], but google shouldn't really spawn a duplicate issue if the page contains different texts. Are they using the same copy?

[brih]

Hi

 

Well actually the default language, in each prestashop installation I guess, will mean that the url without the language snip, will link to a page that can be found with the actual language.

 

E.g. the default language is English, so the page www.thissite.com will be in english as well as www.thissite.com/en/

 

So at least one of languages will be identical.

 

I read at google that they do not recommend to fix this by robots.txt

 

It would be easy though. I tested these and the below would really work:

 

 

Disallow: /*.html

Allow: /en/*.html

 

That will disallow all product pages that ends with a .html - which they will have with the url rewrite that comes with prestashop, unless they have the language snip.

 

I just wonder why google do not recommend this .

 

Anyway, maybe there would be a way to do the same in the .htaccess file ?? WOuld you know ?

 

 Thanks