May 20, 2022
by Sofia

The easy way to check for duplicate content

Unique content is an essential factor in the ranking of your website in search results. If there is a lot of duplicate content, search engines may incorrectly identify the most relevant page or exclude duplicate content URLs from search results. However, the most important thing is that duplications are wasting your crawling budget. A lot of duplications make the crawling of your website ineffective.

It’s easy to control duplications on a site with a few pages, but how do you check all duplications if you have millions of URLs? JetOctopus knows the easy way.

To check for duplicate content on a website, start a crawl. If you want to know which duplicate pages could be scanned by search robots, activate the “Follow the rules of robots” checkbox in “Basic settings”.

The easy way to check for duplicate content - Step 1 - JetOctopus

After completing the crawl, you can go to the analysis of the results. We know that unique content is important, so we’ve created a separate report “Duplication”, where you can find general information about duplicate pages, duplicate metadata, titles, headings, and more.

The general information is in the “Overview” report. Here you will find the number of pages with the same title, description and headings.

The easy way to check for duplicate content - Step 2 - JetOctopus

In the “Duplications problems list”, we show content issues that were found during the crawl. Click on the number next to the problem to go to the data table.

Note the following points:

  • Duplicated Titles – if two or more pages have a similar title;
  • Duplicated Meta Description – if the description of two or more pages matches 100%;
  • Title low uniq words ratio (<50%) – if the number of unique words in the title is less than 50%, it is a partial duplicate;
  • Meta Description low uniq words ratio (<30%) – meta-descriptions of URLs with index,follow should be unique to increase the number of clicks per page.
The easy way to check for duplicate content - Step 3 - JetOctopus

Duplication – Title

Go to the “Duplication – Title” report. Here you can see how many identical or similar titles you have on your website. Note that you can choose the percentage of duplication.

The easy way to check for duplicate content - Step 4 - JetOctopus

Below the charts, we show the most common titles on your website.

The easy way to check for duplicate content - Step 5 - JetOctopus

Unique titles are important because search engines take titles into account when ranking URLs. However, it is quite normal when non-indexable pages have duplicate titles.

We also created similar reports for descriptions, H1 and H2 headings, and main content. When calculating the uniqueness of content, JetOctopus takes into account all the characters in the corresponding elements of the HTML code.

The easy way to check for duplicate content - Step 6 - JetOctopus

Also, we recommend checking the “Content” report for duplicate pages. For example, to check if you have pages with HTML Full Duplication. This means that these URLs are absolute duplicates: not only the content but also the page code is the same.

How to check for duplicate content - Step 7 - JetOctopus

For your convenience, we have created separate data tables, where you can find a complete list of URLs that contain duplicate titles, meta-descriptions and H1 headings.

How to check for duplicate content - Step 8 - JetOctopus

You can customize additional filters and columns to display data most conveniently.

To export data tables, go to the Export menu and select a needed format: Excel, CSV, Google spreadsheets or Sitemaps.

The easy way to check for duplicate content - JetOctopus - Step 9
About Sofia
Technical SEO specialist. Sofia has almost 10 years of experience, of which the last 5 years in JavaScript SEO. She is convinced that SEO is a very technical part of digital marketing. And without logs and in-depth data analysis, you can't do effective SEO.

Categories

Find technical SEO errors at your website at free trial
Get exclusive tech SEO insights
We are tech SEO geeks who believe that SEO is predictable and numeric. Don’t miss our insigths!
Start your free trial
free 7-days trial