The Future of Indexing. Like a G6.

Updated in 2026: The original article followed a short-lived search trend. The durable lesson is not to predict the next query spike. It is to make important pages accessible, understandable and honest about what is still uncertain.

Search visibility still starts with four separate questions: can a crawler fetch the page, can it see the important content, should the page enter the index, and which URL represents it? AI-assisted search does not remove those controls.

Crawl: make the page reachable

A crawler needs to discover a URL and receive a meaningful response. Important pages should be linked from ordinary HTML links. Check that the server returns the intended status, that authentication is not accidental and that robots.txt is not blocking access to the resources needed to understand the page.

An XML sitemap can help expose canonical URLs, especially on a large or recently changed site. It is a discovery aid, not a command to crawl or index every URL. Google’s description of how Search works makes that distinction explicit.

Render: make the answer visible

Modern pages may need JavaScript before the main content appears. Google documents a crawl, render and index process for JavaScript pages in its JavaScript SEO basics. Rendering is not a reason to hide the answer behind an interaction that a crawler or a person cannot reliably complete.

Put the important text, links, headings and metadata in the server response where practical. If JavaScript changes the page, inspect the rendered HTML rather than assuming that the browser view and crawler view are identical. Use meaningful HTTP status codes for missing, moved or restricted pages.

Index: give the page a reason to exist

A page can be crawlable and still not be indexed. The content may be thin, inaccessible, blocked by noindex, too similar to another page or simply not useful for the queries people make. Structured data can describe visible content; it cannot turn an empty or misleading page into a useful result.

Check the page’s title, main text, links and metadata together. The answer should be specific enough to understand without a hidden script, an undocumented private data source or a promise that the page cannot support.

Canonical: choose one representative URL

Duplicate URLs are a maintenance problem as well as a search problem. Choose the URL that represents the page, link to it consistently and use redirects or rel="canonical" where the site has genuine alternate versions. Google’s canonicalisation guidance describes these signals as preferences, not guarantees: Google may select a different canonical when its systems find another page more complete or useful.

Review protocol variants, parameters, print views, filters, translated versions and copied pages. Do not create alternate URLs merely to catch more wording. A clean canonical decision gives readers, editors and crawlers the same answer to ‘which page should I use?’

The 2010 forecast in the original source should be read as a forecast, not as evidence for the controls described here. The retrospective lesson is that the interfaces changed while crawl access, useful content, index eligibility, and canonical ownership remained separate questions.

AI retrieval is a bounded inference

Google’s current guidance for AI features says that the ordinary technical requirements and SEO best practices still apply. There is no separate file, secret markup or guaranteed route into an AI result. A page must be indexed and eligible to appear in Search, and even then inclusion is not guaranteed.

It is reasonable to infer that clear, self-contained answers, identifiable sources and visible limits can make a page easier for an answer system to interpret. That is an editorial quality argument, not a promise about retrieval. Other systems publish different or incomplete documentation, so claims about how they select or quote pages should remain qualified.

The useful work is therefore concrete: check crawl access, inspect rendered content, control indexing and canonicals, explain the answer with evidence and measure what the available tools actually show. The next interface may change. Those controls still describe whether there is a page worth finding.