Analyzing the status codes in logs can help you understand how bots crawl your website and how the crawling budget is spent. In this article, we will discuss the 410 Gone status response code and what to look for during analysis.
The 410 Gone response code informs the client browser that the requested page is no longer available on the website. However, this is not the same as the 404 Not Found status code. The 410 and 404 status codes are different. The 410 status code indicates that the page was once available but has since been permanently deleted, while the 404 status code indicates that the requested page is currently not found (but it is not known for certain whether permanently or only temporarily).
Analyzing 410 status codes is crucial to understand how many 410 pages search robots are crawling instead of crawling indexable pages that can bring traffic to your website. When a page returns a 410 Gone status code, it means that it has been permanently deleted and is no longer available on the website.
As a result, it is essential to ensure that your website’s crawl budget is not spent on 410 pages, as this could decrease the search engine’s interest in your website over time. Since search engine bots want to provide useful and accessible information to users in the SERP, 410 pages do not contain any information and as a result, it can negatively affect your website’s search engine ranking. Therefore, it is crucial to monitor the number of 410 Gone pages to prevent a decrease in your website’s visibility. Because if there are more 410 pages instead of 200 OK, the less useful your website looks to bots.
You can analyze 410 pages of logs using our JetOctopus log analyzer. First, you need to integrate your log files, which can be done easily using one of several methods. More information on this can be found in the article.
Once you’ve integrated your logs, navigate to the “Logs” section of the analyzer and select “Bot Details”.
From there, you can choose the period you want to analyze and specify which search engine or engines you want to analyze, as well as the relevant domains.
After setting up period, bot and domain, you will see the “Status codes” chart, which displays the distribution of status codes according to the selected settings. Pay attention to the percentage of 410 status codes relative to the others. Ideal ratio being 99% 200 OK responses.
Clicking on the “410” segment will take you to a data table with the full list of pages that returned a 410 status code during the selected period.
From there, you can filter the links by URL segment or apply other necessary filters.
Finally, all of the data can be exported to your desired format if you prefer working with files.
Before fixing a 410 Gone response code, it is important to analyze its feasibility by answering several questions.
After answering these questions, create a plan of actions. Here are some general recommendations that can help reduce the number of 410 pages in logs.
Additionally, customize the content of the 410 page. If the page has been indexed and then deleted, it will be available in the SERP for some time. Also, some users may have saved the link of this page. Therefore, create a 410 page that explains why the page is no longer available and allows the user to return to the home page or encourages them to visit other interesting pages of your website.
There may be other non 200 status codes in the logs. we recommend that you read the information about these status codes:
How to find blinking status codes in search robots logs
Step-by-step instructions: analysis of 404 URLs in search engine logs
How to audit non 200 response codes in search engine logs
401 response codes: a how-to for diagnosing and resolving log file issues