Use the Include and Exclude feature to prevent JetOctopus from scanning unwanted pages. We recommend that you use this option if you need to scan one or more types of pages. You can use this feature if you need to scan a part of the large website. Read about other ways to scan a large […]
Security is important to all websites. For JetOctopus, the security of your data is also in the first place. Therefore, we want to tell you which user agents JetOctopus uses when scanning your website. DevOps or website administrators will also need this information so they don’t block the JetOctopus crawler. We emphasize that JetOctopus is […]
Custom extraction is needed to find certain elements in the HTML code of the page. Our web scraper will search selected elements while crawling your website. With custom extraction, you can use JetOctopus more efficiently and get non-standard information about your website. With JetOctopus, you can configure up to 50 custom extraction rules during a […]
Merging data from different tables is much more convenient if you use the “Join Dataset” tool. You can use this option in all data tables: logs, Google Search Console and crawl. In this way, you can merge the data from the crawl or logs with the most effective pages in the GSC. You will find […]
We offer several options if you want to share a JetOctopus report with your colleagues. Choose the most convenient way depending on what your needs are. 1. Share the PDF file. This is the easiest way to share all charts from any menu with your colleagues. You can save reports from the “Logs”, “Crawler”, “GSC” […]
If you use separate mobile domains, you need a special crawl analysis. In the article “How to crawl separate mobile domains with JetOctopus” we wrote about how to configure crawling for a separate mobile site. In this article, we will tell you what to focus on during the crawl analysis of a separate mobile domain. […]
If your website is global/international, it is important to track performance by countries. Consider an example why a general analysis of effectiveness in the SERP is not enough. Your website has visitors from two countries. Each country has a certain number of clicks and impressions and different conversion rates. Country A has a high conversion […]
Although Google recommends using the responsive version of the website for mobile devices, many websites still use a separate domain for mobile users. This means that the website for desktop and mobile has different URL addresses. For example, your main desktop domain is located at example.com and your mobile domain is located at m.example.com. When […]
In this article, we’ll talk about why a crawl of your website can fail. There are several reasons why a crawl of your website may not finish. Some of the reasons may be due to the settings of your web server and the fact that your server perceives our crawler as a potential security threat. […]
Get exclusive tech SEO insights
