410 Gone: understanding and analyzing status codes in log files
Analyzing the status codes in logs can help you understand how bots crawl your website and how the crawling budget is spent. In this article, we will discuss the 410 Gone status response code and what to look for during analysis.
What is the 410 Gone Response Code?
The 410 Gone response code informs the client browser that the requested page is no longer available on the website. However, this is not the same as the 404 Not Found status code. The 410 and 404 status codes are different. The 410 status code indicates that the page was once available but has since been permanently deleted, while the 404 status code indicates that the requested page is currently not found (but it is not known for certain whether permanently or only temporarily).
Why is it important to check 410 Gone pages in logs?
Analyzing 410 status codes is crucial to understand how many 410 pages search robots are crawling instead of crawling indexable pages that can bring traffic to your website. When a page returns a 410 Gone status code, it means that it has been permanently deleted and is no longer available on the website.
As a result, it is essential to ensure that your website’s crawl budget is not spent on 410 pages, as this could decrease the search engine’s interest in your website over time. Since search engine bots want to provide useful and accessible information to users in the SERP, 410 pages do not contain any information and as a result, it can negatively affect your website’s search engine ranking. Therefore, it is crucial to monitor the number of 410 Gone pages to prevent a decrease in your website’s visibility. Because if there are more 410 pages instead of 200 OK, the less useful your website looks to bots.
How to find 410 Gone pages in logs
You can analyze 410 pages of logs using our JetOctopus log analyzer. First, you need to integrate your log files, which can be done easily using one of several methods. More information on this can be found in the article.
Once you’ve integrated your logs, navigate to the “Logs” section of the analyzer and select “Bot Details”.
From there, you can choose the period you want to analyze and specify which search engine or engines you want to analyze, as well as the relevant domains.
After setting up period, bot and domain, you will see the “Status codes” chart, which displays the distribution of status codes according to the selected settings. Pay attention to the percentage of 410 status codes relative to the others. Ideal ratio being 99% 200 OK responses.
Clicking on the “410” segment will take you to a data table with the full list of pages that returned a 410 status code during the selected period.
From there, you can filter the links by URL segment or apply other necessary filters.
Finally, all of the data can be exported to your desired format if you prefer working with files.
How to fix 410 Gone response code
Before fixing a 410 Gone response code, it is important to analyze its feasibility by answering several questions.
- Is the number of 410 pages critical in the logs? Is the 410 status code the main one that Googlebot receives?
- Do the 410 pages return the correct pages, or are there 410 pages that should return a 200 OK or another status code? Many websites use 410 Gone to remove old ads or offers, and a small number of 410s is normal. However, you need to ensure that the server is not returning 410s by mistake.
- Where does Googlebot come to these pages? Are they in the sitemap? Perhaps a 410 page link is placed on your website’s home page or there are redirects to 410 pages?
- If the 410 status code is returned by the removed offers, do you remove the links to these offers from the HTML code of your website in a timely manner? Maybe search engines are crawling these links because they are placed on your categories?
After answering these questions, create a plan of actions. Here are some general recommendations that can help reduce the number of 410 pages in logs.
- Remove 410 pages from the sitemap as soon as possible.
- Remove 410 pages from the HTML code of your website.
- Fix redirects pointing to 410 pages, as well as canonicals and hreflangs.
Additionally, customize the content of the 410 page. If the page has been indexed and then deleted, it will be available in the SERP for some time. Also, some users may have saved the link of this page. Therefore, create a 410 page that explains why the page is no longer available and allows the user to return to the home page or encourages them to visit other interesting pages of your website.
There may be other non 200 status codes in the logs. we recommend that you read the information about these status codes:
How to find blinking status codes in search robots logs
Step-by-step instructions: analysis of 404 URLs in search engine logs
How to audit non 200 response codes in search engine logs
401 response codes: a how-to for diagnosing and resolving log file issues
- Category:
- How to guides
- Log Analyzer
About Sofia
Technical SEO specialist. Sofia has almost 10 years of experience, of which the last 5 years in JavaScript SEO. She is convinced that SEO is a very technical part of digital marketing. And without logs and in-depth data analysis, you can't do effective SEO.
