Category: Crawlers
December 19, 2022
by Sofia
You can manually upload different data to JetOctopus if you need to include it in your crawl results or log analysis.
You can manually import the following file types:
CSV...
November 8, 2022
by Sofia
In this article, we will explain why different crawlers may show different numbers of pages in the results. By the way, we recommend reading the article why there are missing...
October 13, 2022
by Sofia
You may have noticed that the Core Web Vitals metrics check results differ when you check your site with JetOctopus and when you manually check it with Lighthouse or other tools....
October 12, 2022
by Sofia
Why can the number of indexed pages differ in Google’s SERP data and Google Search Console? Why is the number of indexable pages found by JetOctopus also different from these...
October 5, 2022
by Sofia
As you know, you can use a custom robots.txt file when configuring a crawl. This is a useful option if, for example, you want to scan a staging website. But there are other...
September 13, 2022
by Sofia
JetOctopus is a comprehensive tool for SEO that has many advantages and cool features. We would like to remind you about several cool features that will help you use JetOctopus...
September 9, 2022
by Sofia
The last six months have been interesting and exciting for the JetOctopus team. It was difficult at times, but we managed it. And as a result, many useful updates were released....
September 8, 2022
by Sofia
Use the Include and Exclude feature to prevent JetOctopus from scanning unwanted pages. We recommend that you use this option if you need to scan one or more types of pages. You...
August 12, 2022
by Sofia
Security is important to all websites. For JetOctopus, the security of your data is also in the first place. Therefore, we want to tell you which user agents JetOctopus uses when...
July 14, 2022
by Sofia
Custom extraction is needed to find certain elements in the HTML code of the page. Our web scraper will search selected elements while crawling your website. With custom...
