Jump to content
mountarreat

Attention, PrestaShop may harm Google rankings - action required

Recommended Posts

I have recently updated from 1.6 to 1.7.4.3. You can read about what went wrong, including solutions to all problems encountered, here: https://www.prestashop.com/forums/topic/912572-problem-with-upgrade-from-1620-to-174/#comment-2989412

Now another problem. Today I got a little scared after an index-update because the Google search console suddenly told me that all my pages are no longer mobile-friendly, which they already were with 1.6. The classic Theme 1.7 looks good already, at least better and more mobile than the old 1.6. The problem is this ... by default, PrestaShop blocks important resources in the robots.txt with the lines

Disallow: */cache/

Disallow: */css/

Disallow: */js/

Unfortunately, this also includes '/themes/classic/assets/cache/', which is the resources that Google needs to have access to evaluate the layout. Otherwise, the page is only partially loaded. If the CSS and JS files, among others, can not be loaded, the pure HTML is assessed without styles. Clickable elements are suddenly too close together because they are too small, and the content may be wider than the screen. Boom, the page in the index is marked as not mobile-friendly, although it is actually. This could now have fatal long-term consequences in the Google Mobile-First-Index and should be fixed immediately.

Adding the lines

Allow: */themes/*/cache/*.css
Allow: */themes/*/cache/*.js
Allow: */themes/*/css/*.svg
Allow: */themes/*/css/*.ttf
Allow: */themes/*/css/*.woff
Allow: */themes/*/css/*.woff2
Allow: */themes/*/js/*.js

in the robots.txt again allows access to these files (theme styles, scripts, icons and fonts) and can prevent worse. For me it has restored the original state: Page is mobile-friendly.

If you're concerned, it's best to check with the Google tool itself: https://search.google.com/test/mobile-friendly

Keep in mind that after you uploaded the new robots.txt, it may take a while for Google to internalize it, so results are not immediately visible using the mobile-friendly tool. Give it 30 minutes, and check again later, to verify that your site now is mobile-friendly.

Edited by mountarreat (see edit history)
  • Like 1
  • Thanks 2

Share this post


Link to post
Share on other sites

Hi thanks for these information they are intrusting.

But I would like to ask.

Are you sure that these:

Allow: */themes/*/cache/*.css
Allow: */themes/*/cache/*.js

 

must be also allowed? I thought that the cache files are only for caching system and the search google should not take into account?

 

Best Regards

Share this post


Link to post
Share on other sites

Yes, I am 100% sure, and it's nothing new actually. Google notified about this since 2015, this is why I am so surprised that my robots.txt was overwritten with such rules from version 1.7.4.3. They should know better. See the link below for a reference:

https://searchengineland.com/google-search-console-warnings-issued-for-blocking-javascript-css-226227

The themes' cache folder should be allowed because the minified CSS and JS files sit within that very folder.
Furthermore, I have limited it to allow only these certain types to be crawlable.

Check the official source at Google: https://webmasters.googleblog.com/2014/10/updating-our-technical-webmaster.html

Quote

Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.

 

It's known for about 4 years but didn't make it into the store default robots.txt yet.

Edited by mountarreat (see edit history)

Share this post


Link to post
Share on other sites

Thanks for information. I will try it.

(I was also surprised that our web pages are not mobile friendly)

Share this post


Link to post
Share on other sites

Thank you mountarreat for this information, I changed the robots.txt and Google can now read the css files. A very simple solution to a problem that has bugged me for months.

Phil

   

Share this post


Link to post
Share on other sites

I noticed that updates within version 1.7 also overwrite the robots.txt file. This iis very bad and Prestashop should repair this bug asap.

Share this post


Link to post
Share on other sites

I think the best will be to send a defect to prestashop. This is just a forum not defect management for developers of prestashop.

Did someone already send a defect to prestashop? If yes what they say?

Share this post


Link to post
Share on other sites
On 10/25/2018 at 2:49 PM, vlskrbek said:

Hi thanks for these information they are intrusting.

But I would like to ask.

Are you sure that these:

Allow: */themes/*/cache/*.css
Allow: */themes/*/cache/*.js

 

must be also allowed? I thought that the cache files are only for caching system and the search google should not take into account?

 

Best Regards

Thanks for these information. Are these solutions same for Prestashop 1.6? If not, what is the right ones for 1.6?

Share this post


Link to post
Share on other sites

That's the problem that i faced. I added these 'allow' lines to the robots.txt file but after an update i found that robots.txt was overwritten. When you are aware, you can correct the issue. 

Share this post


Link to post
Share on other sites

you could just add this to your robots.txt , or if you just want google ( User-Agent: Googlebot ) 

User-agent: *
Allow: *.js
Allow: *.css

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Cookies ensure the smooth running of our services. Using these, you accept the use of cookies. Learn More