Product Update. Crawler location update

The JetOctopus team is thrilled to announce a new product update for our customers. In our work, we strive to ensure that our crawler works just like a real search engine when visiting websites. However, we understand that our customers have different needs and have released an update that includes new crawler location logic.

Improved crawler location logic

The updated version of JetOctopus allows users to choose the location from which their website will be crawled from the first request. Also, with this new update all your sitemaps also will be scanned from the selected location. This feature is especially important for websites that use geo IP-based redirects. With the new update, all requests will go from the exact location selected by the user, regardless of the crawl mode or user agent chosen.

Benefits of the update

The update allows users to check how geo IP-based redirects work for users or even how they work for Googlebot if it scans sitemaps and so on. This update is crucial for some websites because some indicators in the crawl results of a website may depend on the location of the web server. With the new update, users can be sure that all requests are coming from the exact location they chose.

In April, the JetOctopus team made many useful updates to the platform. New reports and data tables were added for analyzing JavaScript sites. These updates are aimed at making JetOctopus more powerful and user-friendly.

To help users configure the crawl to receive the most interesting and useful data, we have provided some useful links. 

How to configure a crawl of your website

How to configure crawl for JavaScript websites

How To Audit Canonicals

How to use segments

How to use the list mode: 5 tips

By the way, we will be happy to receive feedback from our users on how we can improve JetOctopus even further. We are always open to suggestions and ideas that can make our SEO  crawler more powerful and useful to our customers.