thezey Posted April 7, 2015 Share Posted April 7, 2015 Hello, I received an email from Google saying that some of my web pages are NOT mobile-friendly. I checked the mentioned pages and tested them. It turns out that robots block some JavaScript and CSS files. As a result, Google can't see my pages properly. For example, Google sees those addresses as blocked by robots: http://www.nizoralshop.com/themes/…f4_all.css http://www.nizoralshop.com/themes/…4b6ca68.js The problem is I don't know which line in the robot.txt file is blocking the javascript and CSS files. Could you please help me find what lines are blocking Google from seeing JavaScript and CSS? Thank you. P.S: below is my robot.txt list # robots.txt automaticaly generated by PrestaShop e-commerce open-source solution # http://www.prestashop.com - http://www.prestashop.com/forums # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # For more information about the robots.txt standard, see: # http://www.robotstxt.org/wc/robots.html User-agent: * Disallow: /*fr/ Disallow: /*es/ Disallow: /*de/ Disallow: /*ru/ User-agent: * # Private pages Disallow: /*orderby= Disallow: /*orderway= Disallow: /*tag= Disallow: /*id_currency= Disallow: /*search_query= Disallow: /*back= Disallow: /*utm_source= Disallow: /*utm_medium= Disallow: /*utm_campaign= Disallow: /*n= Disallow: /*controller=addresses Disallow: /*controller=address Disallow: /*controller=authentication Disallow: /*controller=cart Disallow: /*controller=discount Disallow: /*controller=footer Disallow: /*controller=get-file Disallow: /*controller=header Disallow: /*controller=history Disallow: /*controller=identity Disallow: /*controller=images.inc Disallow: /*controller=init Disallow: /*controller=my-account Disallow: /*controller=order Disallow: /*controller=order-opc Disallow: /*controller=order-slip Disallow: /*controller=order-detail Disallow: /*controller=order-follow Disallow: /*controller=order-return Disallow: /*controller=order-confirmation Disallow: /*controller=pagination Disallow: /*controller=password Disallow: /*controller=pdf-invoice Disallow: /*controller=pdf-order-return Disallow: /*controller=pdf-order-slip Disallow: /*controller=product-sort Disallow: /*controller=search Disallow: /*controller=statistics Disallow: /*controller=attachment Disallow: /*controller=guest-tracking # Directories Disallow: /*fr/ Disallow: /*es/ Disallow: /*de/ Disallow: /*ru/ Disallow: /*classes/ Disallow: /*config/ Disallow: /*download/ Disallow: /*mails/ Disallow: /*modules/ Disallow: /*translations/ Disallow: /*tools/ # Files Disallow: /*en/password-recovery Disallow: /*en/address Disallow: /*en/addresses Disallow: /*en/authentication Disallow: /*en/cart Disallow: /*en/discount Disallow: /*en/order-history Disallow: /*en/identity Disallow: /*en/my-account Disallow: /*en/order-follow Disallow: /*en/order-slip Disallow: /*en/order Disallow: /*en/search Disallow: /*en/quick-order Disallow: /*en/guest-tracking Disallow: /*en/order-confirmation Disallow: /*fr/search Disallow: /*es/search Disallow: /*de/search Disallow: /*ru/search # Sitemap Sitemap: https://www.nizoralshop.com/sitemap.xml Link to comment Share on other sites More sharing options...
Dh42 Posted April 8, 2015 Share Posted April 8, 2015 You can either delete your robots.txt or delete this from it Disallow: /*modules/ Link to comment Share on other sites More sharing options...
thezey Posted April 8, 2015 Author Share Posted April 8, 2015 Hi DH42, I just deleted the line mentioned, but I still have those two addresses that are blocked: http://www.nizoralshop.com/themes/theme686/cache/7973aa189748ef096f5481d4e99088f4_all.css http://www.nizoralshop.com/themes/theme686/cache/6338ba3dc45211de62b73587e4b6ca68.js When I enter both addresses in the Google webmaster tools website robot tester, it strikes in red bar "Disallow: /*es/" How is it possible? Is my theme in the Spanish language folder? Please note that I disallow all languages exept English, I only use En and don't want to have Google indexing duplicate pages in the other languages. Link to comment Share on other sites More sharing options...
Dh42 Posted April 8, 2015 Share Posted April 8, 2015 It might be pulling a false positive. If you look at https://www.google.com/webmasters/tools/mobile-friendly/?url=http%3A%2F%2Fwww.nizoralshop.com%2Fen%2F It does not detect it, which it normally would. Link to comment Share on other sites More sharing options...
thezey Posted April 8, 2015 Author Share Posted April 8, 2015 Thanks for your prompt reply, Dh42. I just entered those two lines as ALLOW in the robot file and now it's fine as the test doesn't display any error anymore. However, I had the bad idea to follow Google Webmasters FAQ yesterday, and enabled the mobile version of the website in the Theme tab, even though my theme was already Responsive. It turns out when I want to disable the smartphone version, an error message is displayed "Can not add configuration PS_ALLOW_MOBILE_DEVICE". Is any idea how to fix this issue? Link to comment Share on other sites More sharing options...
Dh42 Posted April 8, 2015 Share Posted April 8, 2015 I do not, I don't know what theme you are using. I would say that you do not need to use a mobile website if you have a responsive template though, google favors responsive. Link to comment Share on other sites More sharing options...
thezey Posted April 8, 2015 Author Share Posted April 8, 2015 Yes, which is why I want to go back to my previous setting, which had mobile version disabled. Link to comment Share on other sites More sharing options...
Dh42 Posted April 8, 2015 Share Posted April 8, 2015 You might have to contact the person that made your theme. They are all different and they all store different values in the database. It is pretty difficult to debug like this. Link to comment Share on other sites More sharing options...
thezey Posted April 9, 2015 Author Share Posted April 9, 2015 I solved the problem by myself. For Information, I went to the PS_Configuration tab in the phpAdmin, and deleted the PS_ALLOW_MOBILE_DEVICE row. As a result, I was able to disable the mobile version option. I also checked out the website with a smartphone and tested it with Google mobile-friendly tester. Now, it correctly displays my responsive theme, instead of prestashop text version. Link to comment Share on other sites More sharing options...
maximo88 Posted May 5, 2015 Share Posted May 5, 2015 You can either delete your robots.txt or delete this from it Disallow: /*modules/ Why prestashop choose to block modules on robot.txt? I have 13 files blocked Link to comment Share on other sites More sharing options...
sg1_anhell Posted August 1, 2015 Share Posted August 1, 2015 I solve the problem deleting 'modules/', in the function getRobotsContent() from /controllers/admin/AdminMetaController.php and then generating again the file in SEO & URLs in back-office. 1 Link to comment Share on other sites More sharing options...
woo Posted September 13, 2015 Share Posted September 13, 2015 Hi, Googlebot error! redirect how to remove: index.php ex . http://mysite.com -------> redirect http://mysite.com/index.php please help please check: http://odav.eu Thanks Link to comment Share on other sites More sharing options...
Recommended Posts