How to Implement a Content Scoring System to Prioritize What to Make AI-Ready

Most teams approach AI readiness backwards. Someone signs off on a RAG project or an internal assistant, and the first instinct is to "index everything," so the whole content store gets shoveled into a pipeline. Six weeks later the retrieval is noisy, the assistant confidently cites a deprecated pricing page, and nobody can explain why the wrong document keeps winning. The failure was not the model. It was the absence of a way to decide what content deserved to be made AI-ready first, and what should have been fixed, retired, or left out entirely.

Content scoring solves that. Instead of treating your library as a flat pile of documents, you rank each piece against the dimensions that actually predict AI usefulness: structural quality, freshness, authority, and reuse frequency. The score tells you where to spend effort, what to remediate, and what to quarantine. Sanity, the Content Operating System, makes this practical because scoring is not a spreadsheet exercise bolted onto a static export; it is a computed field that lives with the content in the Content Lake, queryable with GROQ and enforced in editorial workflow.

This guide walks through the scoring dimensions, how to compute and store them, and how to turn a score into a governed pipeline that decides what gets made AI-ready and in what order.

Why indexing everything is the wrong default

The instinct to make all content AI-ready at once comes from a category error: treating retrieval as a storage problem rather than a quality problem. When you embed and index every document, you are not making your knowledge accessible. You are amplifying whatever is already wrong with it. A stale FAQ, a half-migrated help article, and an internal draft all become equally retrievable, and the model has no way to know which one a human would trust.

Consider a support assistant fed 12,000 knowledge-base articles. Roughly a third are duplicates, superseded revisions, or single-use announcements. Once indexed, those low-value pages compete for the same top-k slots as the canonical answer. Retrieval quality does not degrade gracefully; it degrades on exactly the high-stakes queries where two similar documents disagree. The cost is not just wasted compute on embeddings you will never usefully retrieve. It is eroded trust, because every wrong citation trains your users to stop believing the assistant.

The reframe is to make AI-readiness a prioritized program, not a bulk operation. You want to identify the 20 percent of content that answers 80 percent of real questions, get that subset structurally clean and current, and index it first. Everything else waits behind a score. This is a governance decision as much as an engineering one, which is why it belongs in the content backend where editors and pipelines share one source of truth, rather than in a downstream vector job that no editor can see.

The four dimensions of a content readiness score

A useful score is a weighted composite of dimensions that each predict AI usefulness in a different way. Four hold up across most enterprises.

Structural quality measures whether the content is machine-legible. Is the body structured rich text with real headings, typed references, and annotated links, or is it a blob of HTML with inline styles? Structured content chunks cleanly, preserves semantic relationships, and survives translation to other channels. Portable Text scores high here by design, because marks, annotations, and typed blocks give a retrieval pipeline and an AI agent explicit structure to read rather than guess at.

Freshness measures decay. A pricing page, a compliance policy, and a product spec have different half-lives, so freshness is best expressed as time-since-review against a per-type expectation, not a raw timestamp. Content past its review window should lose points even if nothing else is wrong with it.

Authority captures whether this is the canonical source or a derivative. Is it the reference document other pages link to, or a one-off blog post covering the same topic worse? Reference counts and editorial designation both feed this.

Reuse frequency measures pull: how often the content is actually queried, referenced across channels, or cited by existing assistants. High-reuse content earns priority because improving it has leverage. Weight these to your risk profile; a regulated team weights freshness and authority heavily, while a marketing team may weight reuse. The point is an explicit, tunable formula rather than a gut call.

Computing and storing the score where content lives

A score is only operational if it lives with the content and updates as the content changes. Storing it in a separate analytics tool means it drifts the moment an editor publishes an edit, and no workflow can act on it. The better pattern is to treat the readiness score as a computed field on the document itself.

In Sanity this maps cleanly onto existing surface area. Signals that are already structural, such as whether a document has typed references, whether its body is Portable Text with real headings, or how many other documents reference it, are derivable directly from Content Lake with GROQ. A single query can project the shape you need and compute per-document facts in one round trip: count inbound references with a reverse lookup, check `defined()` on required fields, and measure time since the last review date. Because GROQ returns exactly the projection you ask for, the scoring job does not overfetch entire documents just to read three fields.

Signals that need external data or heavier computation, such as query-frequency pull from your assistant's logs or an embedding-based near-duplicate check, fit Functions: serverless jobs that run on content events, write the resulting sub-scores back to the document, and keep the composite current. TypeGen gives the scoring code typed access to the schema, so a change to the content model surfaces as a type error rather than a silent scoring bug. The result is a readiness score that any editor can see in the Studio and any pipeline can filter on, rather than a number trapped in a dashboard nobody consults before shipping.

Turning the score into a prioritized AI-ready pipeline

A score with no action attached is a vanity metric. The payoff comes from wiring thresholds to concrete outcomes, so the number routes each document to one of a few destinations.

Start with three bands. High-scoring content is clean, current, and canonical, so it goes straight into the AI-ready set: chunked, embedded, and exposed to retrieval. Mid-band content is valuable but flawed, usually failing on structure or freshness, so it routes to a remediation queue where editors fix the specific dimension that dropped the score. Low-band content is stale, duplicative, or non-authoritative, so it is quarantined out of the index and flagged for retirement or merge. This turns an unbounded migration into a triaged backlog where every item has a next action.

The ordering matters as much as the bands. Sort the remediation queue by reuse frequency so editors fix high-pull content first, where a single improvement lifts the most real queries. Content Releases and scheduling let you stage a batch of remediated documents and promote them into the AI-ready set together, rather than trickling changes that force constant re-indexing. Because the score is a live field, promotion can be governed: a document only enters the index once it crosses the threshold and passes review, and it automatically drops out if a later edit pushes it stale. That is the difference between indexing everything once and running an AI-ready content operation that stays correct as the underlying content moves.

Keeping scores honest with governance and review

The fastest way to lose trust in a scoring system is to let it become a black box that assigns numbers nobody can interrogate. When a document scores 42, an editor needs to see which dimension dropped it and what to do about it, or they will route around the system entirely. Transparency is not a nice-to-have here; it is what keeps the score from being ignored.

Store sub-scores alongside the composite, not just the final number. A document that fails on freshness needs a different fix than one that fails on structure, and surfacing the breakdown in the Studio turns an opaque score into an actionable diagnosis. Custom input components let you render the score and its components right on the editing screen, so the person who can fix the content is the person who sees why it needs fixing.

Governance also means controlling who can override. A score should be able to gate whether content enters the AI-ready set, and that gate needs an owner. Roles and Permissions decide who can promote a below-threshold document by exception, and Audit logs record when they did, which matters when a regulator or an incident review asks why a particular document was live in the assistant. For teams with data residency obligations, running this inside a backend that offers regional hosting, is SOC 2 Type II compliant, and publishes its sub-processor list means the scoring and review trail lives under the same controls as the content, not scattered across ungoverned tools. A score you can explain, override deliberately, and audit is a score people will actually run their pipeline on.