Jump to content

Recommended Posts

Hi all

 

I've just signed up to Google Webmaster, but I'm getting an 'Unreachable' error message when I run a 'fetch and render' function.

 

Hours later and I'm wondering if it could be related to my robot file, please see below...

 

# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
User-agent: *
# Allow Directives
Allow: */modules/*.css
Allow: */modules/*.js
# Private pages
Disallow: /*?orderby=
Disallow: /*?orderway=
Disallow: /*?tag=
Disallow: /*?id_currency=
Disallow: /*?search_query=
Disallow: /*?back=
Disallow: /*?n=
Disallow: /*&orderby=
Disallow: /*&orderway=
Disallow: /*&tag=
Disallow: /*&id_currency=
Disallow: /*&search_query=
Disallow: /*&back=
Disallow: /*&n=
Disallow: /*controller=addresses
Disallow: /*controller=address
Disallow: /*controller=authentication
Disallow: /*controller=cart
Disallow: /*controller=discount
Disallow: /*controller=footer
Disallow: /*controller=get-file
Disallow: /*controller=header
Disallow: /*controller=history
Disallow: /*controller=identity
Disallow: /*controller=images.inc
Disallow: /*controller=init
Disallow: /*controller=my-account
Disallow: /*controller=order
Disallow: /*controller=order-opc
Disallow: /*controller=order-slip
Disallow: /*controller=order-detail
Disallow: /*controller=order-follow
Disallow: /*controller=order-return
Disallow: /*controller=order-confirmation
Disallow: /*controller=pagination
Disallow: /*controller=password
Disallow: /*controller=pdf-invoice
Disallow: /*controller=pdf-order-return
Disallow: /*controller=pdf-order-slip
Disallow: /*controller=product-sort
Disallow: /*controller=search
Disallow: /*controller=statistics
Disallow: /*controller=attachment
Disallow: /*controller=guest-tracking
# Directories
Disallow: */classes/
Disallow: */config/
Disallow: */download/
Disallow: */mails/
Disallow: */modules/
Disallow: */translations/
Disallow: */tools/
# Files
Disallow: /*password-recovery
Disallow: /*address
Disallow: /*addresses
Disallow: /*login
Disallow: /*cart
Disallow: /*discount
Disallow: /*order-history
Disallow: /*identity
Disallow: /*my-account
Disallow: /*order-follow
Disallow: /*credit-slip
Disallow: /*order
Disallow: /*search
Disallow: /*quick-order
Disallow: /*guest-tracking
Disallow: /*order-confirmation
# Sitemap
 
Anything obvious please?
 
I've contacted my web hosts and they have said nothing is blocking Google their end.
 
Thanks in advance.

Share this post


Link to post
Share on other sites
  • 1 month later...
  • 3 years later...

i know this topic is a little bit older, but if someone need the solution:

You need to insert the following lines manually in the robots.txt:

User-agent: Googlebot
Disallow:

User-agent: Googlebot-image
Disallow:

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

Cookies ensure the smooth running of our services. Using these, you accept the use of cookies. Learn More