A search index helps you avoid reading every document. It also decides which data moves together, which references an update invalidates, and what must be fetched before returning a result. Those decisions become expensive when the workload outgrows the query shape that the index was built for.
Suppose a document’s vector moves to a different cluster while its text, tenant and public ID stay the same. Should that move rewrite the document’s text and every posting that points to it? The answer depends on whether the vector’s physical address has become the document’s identity.
The opening primer, A Map of Search Systems, introduces the service’s read and write paths and the shared vocabulary for documents and representations and indexes and candidates. This chapter starts the mechanism work: define the answer, then trace how identity and layout change the work beneath it. Reading time covers the prose; allow extra time to draw and solve the exercises.
Read in two passes. First, begin with the primer and take the mechanism chapters in order: follow each worked table, trace and failure boundary, making the short requested predictions before opening guided answers. Then return to the saved inputs for a worksheet pass: solve the guided and independent variants with answers closed, and compare your work with every stated pass criterion. Extend one decision note as you go. The displayed minutes estimate prose reading; they do not include completing both passes.
Evidence boundary: The workload, keys and numerical exercises are invented teaching models. Company mechanisms are attributed to their public sources, inspected on October 1, 2026. In turbopuffer’s September 30 account, v3 passes CI but has a significant performance regression and tuning is beginning. It is an in-progress migration, not a shipped speedup.
Define the answer before choosing an index
Our service initially has 50,000 documents, 10 queries/s and 20 document updates/s in one region. Each document has a stable doc_id, tenant, text, live/deleted state, version and one 768-dimensional float32 body vector. These rates specify a scenario; they are not measured capacity.
A dense query selects one named vector field. Here it selects body and asks for distinct documents ordered by squared Euclidean distance, ascending, then doc ID ascending to break ties. No vector normalization is assumed. Later we add a title vector; a query still selects either body or title. Combining fields or scoring many vector hits per document needs a separate aggregation contract.
The tenant comes from trusted identity context. At the stipulated search snapshot, eligible records belong to that tenant and are live. Use these saved distances directly:
| doc_id | tenant | state | distance |
|---|---|---|---|
| 1 | A | live | 0.2 |
| 2 | B | live | 0.1 |
| 3 | A | live | 0.2 |
| 4 | A | deleted | 0.0 |
| 5 | A | live | 0.5 |
| 6 | B | live | 0.3 |
For tenant A and k=2, remove ineligible rows first. Sort the remaining rows by (distance, doc_id). The exact eligible answer, or oracle, is [1,3]. Doc4’s excellent distance cannot override deletion; doc2’s distance cannot override tenant eligibility.
An approximate result [1,5] has two eligible documents but misses doc3. Its eligible-neighbor recall is |result ∩ oracle| / |oracle| = 1/2. This measures agreement with exact distance ranking. It does not measure whether the documents answer a person’s question. That requires relevance judgments, as discussed in retrieval fundamentals and the IR evaluation text.
Keep four questions separate:
| Question | What would establish it? |
|---|---|
| Was the result eligible at the search snapshot? | Tenant/state predicate evaluated at that snapshot. |
| Did approximate search find the exact nearest eligible documents? | Comparison with the eligible distance oracle. |
| Is the content relevant? | Judged relevance or task-specific evaluation. |
| Is the content fresh and authorized to disclose? | A specified visibility contract and a trusted authorization decision under the disclosure policy. |
Predict a changed answer
For the guided case, ask for five results from tenant A. For the independent case, delete doc1 and change doc3’s tenant to B in the selected snapshot, then request two results as A. Finally, query as tenant C with k=2. Write the IDs and the metric denominator before opening the answer.
| Exercise | k | Trusted tenant | Snapshot change |
|---|---|---|---|
| Worked | 2 | A | none |
| Guided | 5 | A | none |
| Independent | 2 | A | doc1 deleted; doc3 tenant B |
| Empty | 2 | C | none |
Check the exact oracle and recall convention
| Exercise | Exact eligible doc IDs | Recall for the specified result |
|---|---|---|
| Worked | [1,3] | 1/2 for approximate [1,5] |
| Guided | [1,3,5] | 1 for the exact result |
| Independent | [5] | 1 for the exact result |
| Empty | [] | N/A |
Asking for five does not create five eligible documents. The guided denominator is three. In the independent snapshot, one result is correct despite k=2. An empty oracle has a zero denominator, so report recall as N/A and separately require the empty result. A pass includes IDs, population, tie policy and denominator.
Eligibility does not settle disclosure
An old search snapshot includes doc3 for A. At the application’s current-policy authorization decision point, a trusted policy lookup now assigns doc3 to B. May A’s result send doc3’s content to an external reranker because it was searchable in the old snapshot?
Check the disclosure boundary
Deny doc3 to A at that stipulated decision point. Exclude its content before returning it or forwarding it to reranking/generation. Snapshot membership and nearest-neighbor recall are not permission proofs. This toy defines a trusted check and its timing; it does not design a production permission protocol or promise policy cannot change after that point.
The filtered-search article goes deeper into candidate truncation. Here the contract makes later physical-layout comparisons meaningful: both designs must produce answers under the same ranking, eligibility and disclosure requirements.
How an ANN address became a document key
A cluster-based approximate nearest-neighbor index groups nearby vectors so a query can inspect promising groups. A vector’s location might be represented as a cluster ID plus a local position. That address describes placement; the external document ID describes which logical document the application means.
In RIP, vector database, Dan Harrison describes turbopuffer’s v1/v2 layout as a sorted key-value map. Vector records and document contents are keyed by {ClusterId, LocalId}, called the ANN address. Attribute indexes and full-text postings refer to that address. The account says the architecture worked well for ANN on object storage, then constrained other query shapes.
An inverted index maps a term or attribute value to records containing it. A posting is one such membership, optionally with scoring metadata. A full-text posting can include term count and document length; that metadata is not the document’s full text. A projection fetches the attributes requested in the response. These structures can refer to one logical document while using different storage layouts.
Before reading the trace, predict which references change when doc7 moves from C0L0 to C4L2. Its body and tenant remain identical. The keys below are simplified teaching keys, not a complete production keyspace.
ANN-addressed document
- Before: C0L0 identifies placement
Vector, payload and secondary references use the ANN address.
- Move to C4L2
Install the new vector address; move unchanged payload and replace address references.
- Read at the chosen snapshot
Co-located payload can be fetched with the addressed document block.
Hypothetical stable document identity
- Before: doc7 identifies the document
ANN membership points to doc7; payload and stable postings use doc7.
- Move ANN membership
Keep unchanged payload and doc-ID references. Maintain any cluster-local filter summaries.
- Project the result
Resolve returned doc IDs to payload blocks; batching and cache locality determine the added read cost.
Both layouts maintain ANN membership. Stable identity preserves unchanged document references but can require a separate projection read. Cluster-aware derived metadata may still change.
| Structure | Before in the addressed model | Change on the move | Stable-ID alternative |
|---|---|---|---|
| Vector | Vector(C0L0)=v | Install Vector(C4L2)=v, retire old membership safely. | Move ANN membership pointing to doc7; index work remains. |
| ID mapping | Id(C0L0)=7 | New address maps to the same external ID. | Document identity remains doc7. |
| Payload | Attr(C0L0,body/tenant) | Move unchanged contents to the new address. | Doc(7).body/tenant remain unchanged. |
| Tenant membership | A’s attribute posting contains C0L0 | Replace it with C4L2. | A’s doc-ID posting remains 7. |
| Full-text membership | FTS(puffer) contains (C0L0,tf,len) | Replace the address; tf and length stay the same. | Posting (7,tf,len) remains unchanged. |
| Conditional cluster metadata | Summary/count/bitmap depends on cluster membership. | Update affected derived structures if retained. | It may still change even though doc-ID postings do not. |
The conditional row is important. The older native-filtering account describes cluster-aware attribute structures. If our hypothetical stable layout retains such structures, vector movement can still update them. That source does not reveal v3’s eventual metadata design.
Count logical changes here, not object-store requests. Concurrent readers may still need old keys for their snapshot; safe publication and reclamation require a protocol. A stable ID is not itself that protocol. The later visibility chapter will work through this boundary using a stipulated model.
Add a second vector and count the payload
We now add a title embedding. Each query still selects one field and ranks distinct documents. Adding stored vectors changes the maintenance/storage problem without silently changing the oracle.
Assume two vectors/document, 768 components/vector, 4B/component and a 12KiB non-vector payload/document. One uncompressed vector is 768 × 4 = 3072B = 3KiB; the pair is 6KiB.
| Limited layout model | Uncompressed material per document |
|---|---|
| Repeat the 12KiB payload beside each vector | 2 × (12+3) = 30KiB |
| Store the payload once with stable document identity | 12 + 2×3 = 18KiB |
The difference is 12KiB/document, or 40% of the duplicated model’s total. This is not a physical-index saving prediction. We omitted IDs, ANN metadata, compression, indirection, replicas and alignment. A late-interaction representation with many vectors would also need its own scoring/aggregation rules; two independent fields are sufficient for this lesson.
Stable identity introduces another boundary: the ANN result points to doc IDs and projection may fetch a separate document block. A cache hit can make that inexpensive; a cold small query can make dependent I/O costly. Batching many result IDs can amortize work, while fetching large unused payloads can waste bandwidth.
Falsifying measurement: Hold corpus and answer fidelity fixed. Vary vector-update rate and vectors/document; measure logical changed keys, serialized bytes, compaction bytes and projection reads/bytes in warm and cold conditions. A layout that saves update work can still lose on query latency. The object-store LSM overview and RocksDB compaction account supply accessible examples of why physical maintenance exceeds a changed-key count; neither specifies turbopuffer’s complete compaction policy.
Choose the posting replacement unit
Stable identity lets a posting layout make a different choice from the ANN clusters. A KV entry is the storage map’s key/value unit. A posting block can be one KV entry while the storage engine packs many entries into larger files or objects. A codec frame is another unit again.
For one term, stipulate 1,024 postings, each 8B, plus 32B of key/header overhead per KV entry. No compression, split/merge or alignment is modeled. One update changes one existing posting, with no change in term length or block membership. Compare material serialized for the replaced entry:
| Layout | KV entries | Stored bytes for the term | Entry material replaced for one update |
|---|---|---|---|
| Entire list in one KV | 1 | 32 + 1024×8 = 8224B | 8224B |
| One posting per KV | 1024 | 1024×(32+8) = 40960B | 40B |
| 256 postings per KV | 4 | 4×(32+256×8) = 8320B | 2080B |
The whole-list layout has low overhead but a large replacement unit. The per-posting layout has a small replacement unit but many keys and repeated headers. Blocks balance those pressures. They also change compression, decoding and skip granularity. This arithmetic cannot choose a universal optimum.
Logical and serialized work
- One posting changes
A membership or weight changes under the scoring/snapshot contract.
- Replace its KV material
Our toy counts the key/header and the selected posting block.
Physical engine work
- Buffer, persist and compact
WAL, sorted files, compression and merges add physical work.
- Read object ranges
One range can contain many entries; dependency rounds and transferred bytes need their own accounting.
A logical posting update does not imply one remote request. Engine buffering, file layout and compaction intervene.
Designing inverted indexes in a KV-store on object storage describes turbopuffer FTS v2’s target of about 256 postings per KV block, independent of ANN cluster boundaries. Its implementation uses 128-posting compression frames, splits beyond 512 and merges below 128. Those maintenance rules differ from our fixed-size arithmetic toy. The source reports combined structure/execution improvements for FTS v2; those results do not establish v3 performance or isolate each change’s contribution.
Do not confuse its KV blocks with Lucene 10.3.1’s 128-integer packed codec blocks, a remote object, or an execution batch. Numbers that sound similar can describe different work.
Guided prediction: move, then change the tenant
With two vectors and stable doc identity, move the body vector while title/text/tenant stay fixed. Name two references that can remain unchanged and one structure that still changes. Then change the tenant from A to B. Does stable identity avoid the tenant-index update?
Check the movement and logical-update boundary
The unchanged FTS doc-ID posting and A tenant doc-ID membership can remain. ANN membership still changes, and retained cluster-local metadata may change. A logical tenant change updates payload and removes A membership/adds B membership in both layouts. Stable identity avoids rewriting references solely because placement changes; it does not make real document changes free.
Independent problem: use smaller posting blocks
Keep 1,024 postings, 8B/posting and 32B/KV overhead. Choose 128-posting blocks. Calculate entry count, stored bytes and one existing-block replacement. Move one of the document’s two vectors with unchanged payload, then separately change its tenant. Enumerate the effects in each layout and name an omitted physical cost that could reverse your preference.
Check the 128-posting variant
There are 8 KV entries. Stored bytes are 8 × (32 + 128×8) = 8448B; one replacement is 32 + 128×8 = 1056B. Smaller blocks reduce this replacement unit while increasing header/key overhead. Pure vector movement preserves stable payload/doc-ID postings while maintaining ANN membership and any cluster metadata; the addressed model changes associated address references. Actual handling of multiple memberships is implementation-dependent. Tenant change updates payload/attribute membership in both.
Compression, compaction amplification, splits/merges, snapshot retention and added projection reads are possible missing costs. A pass includes arithmetic, unchanged versus changed references, the tenant exception, and a controlled measurement that could contradict the preference. No benchmark or learner assessment is implied.
Continue below the retrieval pipeline
The primer supplies the whole-domain map; the five mechanism chapters add pressures beneath it. Keep the same document, query and answer contract as requirements evolve, and carry each chapter’s artifact forward:
| Chapter | Question | Artifact to carry forward |
|---|---|---|
| A Map of Search Systems | Where do representations, access paths, ranking, visibility and disclosure fit? | Read/write map and shared vocabulary. |
| This chapter: identity and layout | Which references move with a vector, and which should remain stable? | Reference trace and limited byte table. |
| Doing less work, and doing work faster | When can lexical execution skip safely, and when does batching help? | Bound/tie proof and controlled comparison plan. |
| Finding neighbors inside the eligible corpus | How do predicate placement and geometry change the access path? | Exact eligible baseline, planner choice and fallback limits. |
| From durable write to searchable snapshot | Which versions survive index lag, deletion, crash and retry? | Version/visibility trace and recovery authority. |
| Paying for retrieval across storage and shards | Which bytes and dependencies dominate as the service grows? | Resource budget, failure contract and revised architecture. |
The application-level retrieval-to-RAG chapter explains the downstream bridge from retrieved documents to generated answers.
For deeper study, Database Internals by Alex Petrov (2019) is a useful bridge for physical organization, buffering, log structures and recovery. Designing Data-Intensive Applications, second edition, by Martin Kleppmann and Chris Riccomini (March 2026), supplies requirements, derived data and consistency framing. Full book texts were not accessed for this series; these are topic recommendations. Start with the inspected free sources linked above, then verify selected passages in the edition you obtain.
Your exit artifact is a data/reference trace and a byte table you can defend when an assumption changes. Choosing a product name or repeating a reported speedup does not produce that artifact.
For selected reading, start with RIP, vector database for the ANN-address coupling and migration status; the FTS v2 posting-layout account for block size, replacement and compression tradeoffs; and SlateDB’s design overview for how logical updates become buffered, persisted and compacted state. Read each against the trace you drew, then list the additional costs your byte table omits.