Blog
Get exclusive tech SEO insights
Recent Articles
Why is a different number of logs displayed in GSC and in your logs?
You may have noticed that the number of crawled URLs in the Google Search Console “Crawl stats” report may differ from the number of URLs visited by search engines that you see from the JetOctopus logs. Why this is so and how to correctly count the number of visited pages, read in this article. 1. […]
Custom Extraction Guide with multiple use-cases
Custom extraction is needed to find certain elements in the HTML code of the page. Our web scraper will search selected elements while crawling your website. With custom extraction, you can use JetOctopus more efficiently and get non-standard information about your website. With JetOctopus, you can configure up to 50 custom extraction rules during a […]
Step-by-step instructions: analysis of 404 URLs in search engine logs
If search engines receive a lot of 404 response codes, they can reduce the scanning frequency of your website. Also, 404 in the logs can indicate the following problems: Also, 404 response codes in the logs can highlight problems with the site or web pages that broke due to releases, updates, etc. How to find […]
Web page returned non 200 to search bot. Why do I get a different response code while manual checking?
You may meet a situation when you check a page that returned a 4xx or 5xx response code to the search engine (you found this data in the logs). And when checking manually, the page returns a 200 response code. What could be the reason for such a situation? 1. In the logs, you can […]
How to find out the frequency of scanning by search robots
Regular scanning of the website by search robots ensures the updating of data in the index. The regularly crawled pages will also have an up-to-date snippet in the SERP. However, sometimes crawling is excessive and can cause server overload. If your website is popular and constantly updated, extreme crawling by search engines can even lead […]
Product Update. GSC URL Inspection Tool
We are happy to announce a big update. Now with the help of JetOctopus, you will be able to work with Google Search Console data faster and more efficiently. Features of the new GSC URL Inspection tool If you have a large website with millions of pages, it can be difficult to monitor the process […]
How to audit 304 pages visited by GoogleBot/other bots
When the search robot user-agent requests the web page using the header “If-Modified-Since” or “If-None-Match” and the page has not changed since the requested time (usually this is the last visit of the GoogleBot), the server returns a 304 response code. This signals to the search engine that the web page has not changed since […]
How to find the slowest pages visited by search engines
Page load time affects how often your website is crawled by search engines. The slower the pages, the less often the search engine will visit them. That is, pages that respond slowly during the visit of search robots negatively affect the crawling budget of your website. So, we recommend analyzing the slowest pages and optimizing […]
How to analyze what types of search robots visit your website
Analysis of the activity of search robots is an important part of technical SEO. Some robots may visit your website too often and overload your web server. At the same time, the number of sessions and conversions with this search engine may be close to zero. Therefore, the activity of such search robots should be […]
How to analyze crawl results for a separate mobile domain
If you use separate mobile domains, you need a special crawl analysis. In the article “How to crawl separate mobile domains with JetOctopus” we wrote about how to configure crawling for a separate mobile site. In this article, we will tell you what to focus on during the crawl analysis of a separate mobile domain. […]
How to check all images on your website
In this article, we’ll show you how to bulk audit images on your website with JetOctopus. Images can bring organic traffic if they have the proper context and the correct settings. Images can also affect your website’s SERP ranking if the page is slow due to a large number of images. So, it’s a good […]
How to check image count on the page
Pages with lots of images may load slowly. As a result, these URLs will be indexed worse and may rank lower in SERPs. Additionally, on many devices, a large number of images will not display quickly or at all, resulting in a degradation of user behavioral metrics. Browsers and search engines do not have limits […]
How to find blinking status codes in search robots logs
Analysis of search engine logs is an important part of the work of SEO specialists. With the help of log analysis, you can get insights on why your website is displayed in the SERP in a certain way. In addition, you may find issues in the logs that prevent your pages from being crawled and […]
Product update. Alerts for advanced website monitoring
The JetOctopus team has released a long-awaited functionality. Now, you can set your own alerts to monitor the performance of your website. If something breaks, you will receive an instant notification. All our customers can create an unlimited number of custom alerts, as well as use a lot of builtin notifications. We have prepared for […]
How to analyze website performance by countries in SERP
If your website is global/international, it is important to track performance by countries. Consider an example why a general analysis of effectiveness in the SERP is not enough. Your website has visitors from two countries. Each country has a certain number of clicks and impressions and different conversion rates. Country A has a high conversion […]
How to analyze internal linking with JetOctopus
Internal linking is one of the most important features of a website. Without convenient and clear internal linking there is a risk of a high bounce rate and low conversions. Search engines also take into account internal linking when ranking the pages of your website. For this reason, we recommend auditing internal links and improving […]
Webinar: How to Migrate a website to JavaScript. Best practices
JavaScript SEO is one of the most popular topics nowadays. Barely each website is on JS or plan to migrate to JS. Therefore we are opening a series of meetups dedicated to this crucial topic. The first meetup will be about: The webinar is conducted by Sofia Vatulyak, Tech SEO expert, In-house. Link to presentation: […]
Why does the crawl of your website fail?
In this article, we’ll talk about why a crawl of your website can fail. There are several reasons why a crawl of your website may not finish. Some of the reasons may be due to the settings of your web server and the fact that your server perceives our crawler as a potential security threat. […]
How to use segments
Using JetOctopus, you can customize as many segments as you need. We do not limit the number of segments. You can use URL segments when analyzing the results of crawling, logs and Google Search Console. Why you should segment your website Slice your site into as many segments as you need and watch your errors […]
Why the results of crawls differ
If you have crawled the same website twice or more, you may notice that the crawl results are different. In this article, we will explain why this happens and what to keep in focus when analyzing differences in crawls. Next, we will describe in points the possible reasons why the results of crawls differ. Different […]
How does the JetOctopus crawler work?
Understanding how the JetOctopus crawler works will help you better understand the results of your crawling. The main principle of our crawler is very similar to search engines. However, you can make many additional settings with JetOctopus. We want to emphasize that if there are errors in the crawl results or there are not all […]