Why is internal linking so important?

For a page on a website to receive even one organic visit from Google SERP, it takes a long journey. A page should be created, crawled by Googlebot, added to Google’s Index, ranked in SERP, and a user should click on it.

By creating many pages on a website, we expect all pages opened for indexation to be crawled and indexed. Unfortunately, google’s crawling bot (Googlebot) has limited resources and has to decide what to crawl and leave off. 

The number of pages crawled on a website is called the Crawl Budget.

The crawl budget usually ranges from 40 to 70 percent for big e-commerce websites, catalogs, classifieds, and job boards.

This means that almost half of the website’s pages remain uncrawled and are not indexed by Google. As a result, there is less organic traffic, especially on low-frequency keywords, than it could have.

How to influence the Googlebot behavior

How can Googlebot crawl the whole website? Unfortunately, Google doesn’t provide any tools or detailed information on how to control Googlebot on your website.

Having analyzed hundreds of different big websites, we found three main areas of influence on Googlebot behavior:
🛠 Technical: load time (essential!), page size, HTML size, and structure
📝 Content: the quality and need of the content
🔗 Internal Linking: how effective pages on the website link (connected) each other

Page load time means the fetching time of pure HTML from a server without rendering it. This is one of the crucial factors. If the average page load time is over 2 seconds, it should be the task for improvement with priority number one. Learn more about page load time in our article.

Content is another essential aspect, but usually, it’s not a big problem for e-commerce, catalog, or classified websites. Because of the nature of such websites, product descriptions are non-unique and usually short.

Internal linking: Google understands a page by its content and how it is connected (linked) with other pages. The number of internal links pointing to a specific page is one of the strongest signals to Googlebot to crawl this page.

If a website has 1M pages, and Page A receives only 1 link from the whole website, but Page Y receives 500 links, which page is more important in Googlebot’s eyes?

Before the introduction of sitemap technology, internal links were the only way a Googlebot could find a page on the website. But even now, just putting all pages in the sitemap without internally linking them will not be enough to get them indexed. 

They will have the “Discovered – currently not indexed” status in the Google Search Console.

The most common problem of Internal Linking on modern websites

Case 1: Pagination

The products are usually placed on listing pages with pagination. Pages after the second are closed from indexation by non-canonical or meta non-index tags. 

If a category has a thousand products, normally linked will be just products from the first page, usually 20-50 items. Less than 5% from the whole category! In this article I tell how to analyze pagination in search engine logs.

Case 2: Overlinked top pages and underlinked the rest

Another common problem on most big e-commerce websites is it has some top categories, brands, and products. Those entities are presented on many different page types in blocks like ‘Bestsellers’, ‘Top products’, etc.

This leads to a situation where such top products and categories are heavily linked, leaving the rest, 80%, with poor or non-existent internal linking.

Case 3: Faceted filters

Faceted filters are one more complicated topic. They often work with a combination of filters and produce infinite non-canonical URLs. Googlebot crawls a fraction of them and reduces the crawl budget on such pages by URL patterns, which may affect other parts of the website catalog.

Case 4: Orphan pages

Orphan pages are the pages that are present only in sitemaps. There is no way to find them through website navigation. Usually, Googlebot crawls a very tiny number of such pages, keeping them out of the Google Index.

The list of the most common problems could have way more examples. The root of the problem is that developers and search engines have different perceptions of website entities. For developers, they are products, categories, tags, reviews, etc., but for search engines, they are just pages with content and connections with other pages.

Internal linking is not only for SEO

Historically, SEO tasks have been perceived as weird, non-user friendly, and ‘only for Google’ tasks. It had some part of the truth about ten years ago. At the current time, this is not the case anymore. 

The same applies to internal linking: it’s not a list of hundreds of links at the bottom of every page, but proper internal linking should also be helpful for a user. Not only does Googlebot crawl internal links, but a user also clicks on them.

Improving and increasing internal linking on a website will not harm UI and UX.

Why did we develop an internal linker?

Having seen many different websites from many locations, we found that internal linking doesn’t usually realize its full potential. Significant parts of the website remain under linked or worse – orphans pages. 

The problem is the tasks for improving internal linking are very complicated and take a lot of time and iterations to implement.

By improving internal linking on underperforming pages, a website could have increased the number of pages in the Google index by 10-15% with an improvement in organic traffic.

In JetOctopus, we have the complete data for technical SEO, which allows us to generate the most effective internal linking for underlinked pages:
✔️ Crawl – gives the full site structure, how many pages are created, their parameters, and the current internal linking state
✔️ Logs – how Googlebot crawls a website, what is the crawl budget, how many pages are uncrawled by the bot (you can check with JO Log Analyzer)
✔️ Google Search Console Keywords – the performance of pages in organic SERP, visits, dynamics, and positions (check with JO Search Console integration).

How JetOctopus’ Internal Linker works

Using data from Crawls, Logs, and GSC, we built a multidimensional dataset with hundreds of parameters for every page. The amount of data allows us to segment pages into chunks by different criteria to find weak and strong parts of the website.

The main pieces are:
✔️ Acceptors – the underperforming pages. They could have existing poor internal linking, be unvisited by Googlebot, not indexed, and rank further than the third page of Google SERPs, etc.
✔️ Donors – the well-performing pages

Those pages are regularly crawled by Googlebot; they have organic impressions, conversions, etc.

By putting links from Donors to Acceptors’ pages, we transfer internal PageRank (link-juice, link authority, you name it).

The problem is pages can’t be linked randomly. It makes no sense for end-users or Googlebot to see internal links from mobile phones to garden tools. But linking mobile phones to phone cases is way more organic.

In JetOctopus Linker, we developed an algorithm that uses attributes from every page, classifies them, applies AI, and produces the best possible matches between donors and acceptors. This leads to true internal semantic linking when we link pages that are closer by their content and semantics.

It helps Googlebot understand and crawl the website much better, producing a higher crawl budget and indexation rate.

The workflow with JetOctopus Linker

Any big e-commerce website is alive; new products, brands, and categories come and gone. Summer and winter may change demand in different categories and products.

It leads to continuous experiments and optimization to get the best results.

What is the workflow with JO Linker:

1. The JetOctopus SEO team, together with the client’s SEOs analyze and makes hypotheses about weak pages and possible improvements in crawl budget, indexation, and organic rankings

2. Select Acceptors and Donors pages

3. Enter parameters into Linker Configurator: how many links are needed, what type of attributes will be used for matching, linking strategy, and many more.

4. Generating

5. Verifying the result, parameters fine tuning, and regenerating

6. Saving interlinking, export to file if needed

7. Implementing on the website

8. After a week or two, assessment of changes in crawl budget, indexation, organic traffic, and conversions.

9. Start again

Deployment

Usually, big websites are quite big and old; a few dev teams work on them, deploying releases once to twice a month. It makes implementing even simple SEO tasks long and complicated, not talking about internal linking.

Understanding these issues, we developed a new way to implement and deploy internal linking on a website: JetOctopus Linker API.

On the website’s backend, during the page render, a website requests the Jetoctopus Linker API for internal links for the specific page. Within a fraction of the time (talking about 10-20ms), the API returns a response with internal links and anchors for the page. The rest of the job is simple: just render the list of URLs at the proper place on a page.

By integrating JetOctopus Linker API, the SEO team can do their job more effectively and produce results in months rather than years. 

Is JetOctopus Internal Linker a replacement for existing internal linking on a website?

No. Our linker is not designed to replace existing internal linking on a website. We are an addition to the existing scheme.

Conclusion

Almost all e-commerce websites have a small number of top-performing categories and products that bring the most revenue. 

On the other hand, low-end pages hide a potential for organic SEO traffic growth, usually with a higher conversion rate.

Internal linking is almost the universal solution for any big website to improve the crawl budget, indexation, and organic traffic on such pages.

Curious how AI-powered internal linking can boost your indexation and organic traffic? Book a personalized demo and see how JetOctopus Internal Linker works in action — from identifying orphan pages to injecting semantically relevant links at scale.