Google robots.txt update: 29 July 2015

Over the last few days you may have received an email from Google about your websites robots.txt file. The email reported a problem with Google not being able to access the CSS and JS files in robots.txt.

We have just released an update to robots.txt which should help resolve this issue with Google - robots.txt is now set to allow access to CSS and JS (javascript) files.

Google suggested 3 steps that need to be completed to fix this. We have addressed Google's first two steps, and you will now need to complete the 3rd step using Google webmaster tools:

"3: Validate the fix using “Fetch as Google”

Fetch and Render your page with the “Mobile: smartphone” option selected to double-check that Googlebot for smartphones renders your content properly."

The steps below explain how you can fetch and render as Google:

- Log into Google webmaster tools

- Select the website you want to fetch and render

- Click on ‘Crawl’ on the list of options on the left-hand side

- Click ‘Fetch as Google’

- Use the drop down menu which says ‘Desktop’ to select ‘Mobile: smartphone’

- Click ‘Fetch and render’

- Click the ‘Submit to index’ button when it appears in the results and choose ‘Crawl this URL and its direct links’.

Please also note that you may not see an immediate impact on the SEO of your site, however it should allow Google to access the resources they have mentioned from now on.

Powered by Zendesk