July 8, 2022
by Sofia

How to audit non-200 response codes in search engine logs

If your website is perfectly on-page optimized, but the search engines get a lot of non-200 response codes, this can decrease the crawling budget. Search engines will crawl your website less often, pages will be updated less often in the SERP, and new ones will be shown with a lag. To avoid this, we recommend analyzing search engine logs. Analysis of non-200 status codes is one of the most important points.

Using JetOctopus, you can quickly analyze non-200 pages in logs of any search engine.

Where to find a list of all non-200 status codes

To see the general statistics of non-200 status codes, go to the “Logs” menu – the “Bot Dynamics” report. Next, select the desired period, type of search engine and domain.

How to audit non-200 response codes in search engine logs - JetOctopus - 1

Pay attention to the chart “Status codes”: here you can see the ratio of 200 and other status codes for the selected search robot and period. Clicking on a chart segment or status code will take you to the data table.

How to audit non-200 response codes in search engine logs - JetOctopus - 2

On the “Dynamics of status codes” chart, you can find information on how the number of non-200 status codes changed during the period you selected. A sharp jump of non-200 pages with an unchanged number of visited pages may indicate large errors on your website.

How to audit non-200 response codes in search engine logs - JetOctopus - 3

For an in-depth analysis of non-200 status codes in search engine logs, go to the data tables – “Raw logs” – “Non 200 Status”. Here you can configure the necessary filters: click “+Add filter” to set the required URLs, status code, referers, bot IPs, etc.

How to audit non-200 response codes in search engine logs - JetOctopus - 4

To get additional data, configure the columns by clicking the “Setup Columns” button. Next, select the desired ones.

How to audit non-200 response codes in search engine logs - JetOctopus - 6

For bulk export of non-200 status codes for search robots, for example, for GoogleBot, click the “Export” button and select the desired format.

How to audit non-200 response codes in search engine logs - JetOctopus - 7

What types of non-200 status codes should be paid special attention to

1. When analyzing non-200 response codes in search engine logs, pay attention to the dynamics of these status codes and the ratio of non-200 visits to pages.

2. Also, we recommend an in-depth analysis of 404 status codes in the logs. More information: Step-by-step instructions: analysis of 404 URLs in search engine logs. 

3. Pay attention to 304 status codes: How to audit 304 pages visited by GoogleBot.

4. And, of course, keep the blinking status codes in focus. Pages with blinking status codes are those pages that returned to bot at least two different status codes during the selected period. This is usually a 200 and 404 or 5xx response code. More information: How to find blinking status codes in search robots logs.

5. Analyze the 5xx status code separately. They are usually related to temporary web server problems. However, a large number of 5xx crawlers reduces the frequency of crawling of your website.

About Sofia
Technical SEO specialist. Sofia has almost 10 years of experience, of which the last 5 years in JavaScript SEO. She is convinced that SEO is a very technical part of digital marketing. And without logs and in-depth data analysis, you can't do effective SEO.

Categories

Find technical SEO errors at your website at free trial
Get exclusive tech SEO insights
We are tech SEO geeks who believe that SEO is predictable and numeric. Don’t miss our insigths!
Start your free trial
free 7-days trial