Edit History

I had a similar crawl issue and it turned out my hosting was blocking Googlebot through some weird server-level rule. While checking logs, I did some data mining on my access files and spotted the blocked requests. Once I whitelisted Google’s user-agent in my server config, crawling started working again. Double-check if your server’s firewall or security tools are doing something similar.