Using JetOctopus, you can find links to broken pages easily. Start a new crawl or check for broken links in the results of the previous crawl. However, keep in mind that the number of broken links may have changed since the last crawl. Links to broken pages are the links that lead to non-200 pages. […]
Orphan pages are URLs that cannot be found on your website during a crawl. They are not included in the structure of internal linking. As a consequence, users will not be able to find these pages. But search engines can follow the orphan pages from external links, from search index (for example, if the orphan […]
Unique content is an essential factor in the ranking of your website in search results. If there is a lot of duplicate content, search engines may incorrectly identify the most relevant page or exclude duplicate content URLs from search results. However, the most important thing is that duplications are wasting your crawling budget. A lot […]
Sitemaps are one of the most important sources of new URLs for search engines. They scan URLs from sitemaps regularly. And if the URLs meet the requirements of search engines, they will be indexed. However, if sitemaps contain broken links, 4xx, 3xx pages, etc., this will negatively affect your crawling budget. Therefore, it is wise […]
If you crawl your website with a regular JetOctopus user agent, it will not be blocked by Cloudflare. However, if you want to check how your site is seen by Google bot using crawling, Cloudflare will block unconfirmed Googlebot. We remind you that you can crawl your website with the Googlebot Mobile or Googlebot Desktop […]
JetOctopus can crawl staging websites and websites closed from scanning by search engines. You can also run a crawl of a website that needs authentication (password/login). To crawl staging websites, just set up special crawl settings. When you actually need to crawl a staging website How to crawl a staging website blocked by robots.txt If […]
Indexability is one of the most important characteristics to pay attention to when analyzing crawling results. We’ve created a separate segment to make work with indexable pages easier. You can use this segment in all JetOctopus reports. Why the page is not displayed in the segment “Indexable” An indexable page has 200 response code and […]
Core Web Vitals is a set of specific features that Google considers important for users. The better your Core Web Vitals, the better your site is at the top in the SERP. Core Web Vitals consist of three main metrics: the largest contentful paint (LCP), first input delay (FID), and cumulative layout shift (CLS). Also, […]
Hreflang attributes show search engines that a website has localized versions of pages. For example, you may have English and Ukrainian versions, or separate English language versions for the United Kingdom and the United States. You can use the hreflang attributes in regular HTML, HTTP headers or sitemaps. Most hreflang problems are with the incorrect […]
If you find errors in crawl results or logs, you can submit the corrected/updated pages for scanning to search engines. This can be done with a custom sitemap. JetOctopus allows you to generate a custom sitemap in two clicks. In which cases do you need to generate a sitemap? For example, due to a technical […]
Happens when you run a crawl and the website returns a 503 response code. “503 Service Unavailable” response code means that your web server cannot process the request at this time. What are the reasons for the 503 status code? There is good news: the 503 response code is temporary, and as soon as the […]
JetOctopus can find redirect chains on your site. What is a redirect chain? This is a sequence of several redirects. For example, from URL1 to URL2, from URL2 to URL3, and so on. You can find the general information about redirect chains in the “Technical” report. Go to the crawl results and select “Technical” – […]
Before you start a new crawl we recommend preparing basic information about your website. It allows you to set up the optimal configuration of the crawl. Of course, the basic settings of JetOctopus crawl are suitable for most websites, but it is worth delving into the settings to get the most useful information about your […]
We want to have clean and accessible code on our website. Also, we want search engines to scan only pages needed in search results because we worry about our crawling budget. But sometimes there are situations when URLs in the code are blocked by the robots.txt file. It means that search engines cannot access the […]
The alt attribute (or alternative text) contains a description of the image. Alt will be displayed on the screen instead of the image if the user has a laggy internet connection. Of course, search engines also “read” alt attributes. Thanks to this HTML attribute, search engines understand what is depicted, and show images in search […]
Alt attribute (alternative text) is a description of the appearance, function, or/and content of images on a web page. Alt attributes are in the HTML code. Users usually don’t see alt text if they have a normal internet connection. But if the Internet is laggy or images are blocked by the browser, then users will […]
If you need to crawl all the images on your website, you can use JetOctopus. By default, we do not crawl images, so you need to configure the crawl specifically for further image analysis. Image crawl settings Go to the desired project and click the “New crawl” button. In the basic settings, enter the domain […]
Using JetOctopus you can bulk export all inlinks to 5xx, 4xx, and 3xx. Why you should pay special attention to those types of response code? Relevant links with 200 response codes are important for your customers. 404, 502, 503 (and others) are not very appealing to users. They just want to get the information they […]
JetOctopus has switched logs quota calculation from pages to usual log lines.What does it mean? Now we are calculating user quota limits by the number of requests made from verified Googlebot, Bing, and Baidu bots to regular HTML pages only.Bot requests to images, CSS/JS objects, and other media files are not counted as usage of […]
Search engines declare that content is one of the most important ranking factors. Pages with high-quality and complete content always have better chances to be in the top positions in SERP. Using JetOctopus, you can easily find pages that don’t have a lot of content. Below you will find a step-by-step guide to checking pages […]
In JetOctopus crawl you can find the pages with a 404 response code. This response code means that the page was not found and the content is not available to your users. To find all 404 pages on your website, use JetOctopus crawler. Among other things, you can use the free 7-days trial after demo […]
Get exclusive tech SEO insights
