The Staff Engineer Search System Design Interview: Relevance, Freshness, and Permissions

A search-system interview often starts with a familiar prompt: design search for a company's documents, products, or support articles. It is tempting to draw an API, a database, and a vector index immediately. The difficult questions arrive later: why did the right document disappear, why can a former employee still see its title, and why does a successful write take thirty seconds to become searchable?

For senior and staff engineers, those questions are an opportunity to show judgment. Search combines retrieval quality, asynchronous data movement, authorization, and latency. A convincing design connects each component to an observable promise. This guide uses primary research and current documentation to build that argument. The examples and rehearsal schedule are practice tools, not a claim about any company's interview rubric.

1. Define what a correct search result means

Start with one concrete workflow: an authenticated employee searches internal technical documentation. Ask whether users primarily navigate to known documents or explore unfamiliar concepts. Ask which languages, document formats, permissions, and update rates matter. A query for an exact error code and a question about recovering a failed deployment may need different retrieval behavior.

Write four contracts before choosing infrastructure: relevant results appear near the top; changes become visible within an agreed interval; inaccessible content never appears; and responses meet a latency target under a stated workload. Treat the numbers as questions for the interviewer. “Assume a two-second freshness target and a 250-millisecond p95 response target” is a useful proposal, provided you explain what would change if either assumption changes.

Keep search separate from a generated answer. The 2025 FreshStack paper builds retrieval benchmarks from community questions and technical documentation across five domains. It evaluates whether retrieved material supports the information needed for an answer. Your design can apply that distinction: a plausible answer does not prove that retrieval found the right evidence.

Draw the boundary explicitly: authoritative document storage feeds a derived index; the serving API authenticates the caller, retrieves eligible candidates, ranks them, and returns results. Record an index version and retrieval configuration with evaluation runs. This makes “search got worse” a diagnosis you can investigate instead of a disagreement about examples.

2. Build a lexical baseline that handles real documents

Explain an inverted index in plain language: it maps terms to documents containing them, so retrieval can inspect relevant posting lists instead of scanning the whole corpus. Then explain BM25 as a scoring baseline that considers term frequency, term rarity, and document length. The Stanford information-retrieval textbook describes its term-frequency saturation and length-normalization components. You do not need to derive the formula to defend why repeated words should not increase a score without limit.

Separate fields according to their meaning. A title, body, product identifier, language, owner, and access policy should not all become one text blob. Elastic's keyword-field documentation distinguishes exact structured values from full-text fields. That distinction matters for a document named “ERR_CONNECTION_RESET”: users should be able to find the exact identifier without relying on semantic similarity.

Discuss analysis before claiming the index works. Tokenization, case normalization, stemming, and synonyms change which terms match. The custom-analyzer documentation shows how character filters, a tokenizer, and token filters form a pipeline. Propose inspecting actual tokens for filenames, API names, punctuation, and multilingual text. A broken analyzer can look like a ranking problem.

For long documents, choose an explicit retrieval unit: whole document, section, or passage. Passage retrieval can reduce irrelevant surrounding text, but it creates duplicates and loses context across boundaries. Keep parent identifiers and section locations so the interface can group hits and open the original source. Evaluate the baseline before adding embeddings; otherwise you cannot tell whether extra complexity improves anything.

3. Give vectors and reranking a measurable job

Dense retrieval can retrieve paraphrases that share few exact words. It also introduces an embedding model, vector dimensions, approximate-neighbor search, and a re-embedding lifecycle. The HNSW research paper describes a graph-based approximate-nearest-neighbor method. Approximate retrieval is a tradeoff: it can miss neighbors an exhaustive search would find. Distinguish that approximation error from a model that places the wrong documents near the query.

Offer a hybrid baseline when the workload contains both exact identifiers and conceptual questions. Run lexical and vector retrieval, combine candidates by stable document or passage identity, and optionally rerank a bounded set. Elastic's reciprocal rank fusion documentation describes combining ranked lists using rank positions. This avoids directly adding a BM25 score to a vector similarity score as though their numerical scales were interchangeable.

Make the candidate budget concrete. In a hypothetical design, retrieve 100 lexical hits and 100 vector hits, deduplicate them, and rerank at most 50. These are starting assumptions for testing, not published optimums. If the relevant document never enters the candidate pool, the reranker cannot rescue it. Measure candidate recall separately from final ranking quality.

A cross-encoder scores a query and candidate together, rather than comparing independently produced embeddings. The Sentence Transformers cross-encoder documentation explains why this stage is commonly applied to a smaller candidate set. State its latency and cost budget. Explain the fallback if it times out: return the authorized baseline ranking and record which path served the request.

When discussing tuning, separate index construction from query effort. The hnswlib project documentation exposes controls such as graph connectivity and search-time ef. Increase effort only when measured recall improves enough to justify memory or latency. A leaderboard win on unrelated documents is evidence to investigate a model, not permission to skip your own evaluation.

4. Evaluate queries, not attractive demos

Build a judged query set from the workflows you promised to support. Include known-document searches, exact symbols, natural-language questions, misspellings, rare terms, empty-result cases, and different permission groups. Keep a development set for tuning and a separate evaluation set for comparison. Have assessors judge relevance to the actual task; record disagreement instead of treating every label as objective truth.

Choose metrics according to the user journey. Recall at a cutoff measures how much judged relevant material entered the candidate set. Mean reciprocal rank rewards finding a relevant result early. Normalized discounted cumulative gain supports graded relevance and discounts lower-ranked hits. Elastic's ranking-evaluation API documents these distinct measures. A single aggregate score cannot tell you which stage failed.

Use a small calculation to explain the separation. If your labeled set has five relevant passages and retrieval returns three within its first 100 candidates, recall@100 is 3/5, or 60%. A perfect reranker can reorder those three, but cannot recover the two missing passages. The example assumes complete labels; real evaluation pools can miss relevant material, so inspect apparent failures before declaring them model errors.

Report slices as well as the mean. An improvement for conceptual English queries may conceal a regression for exact API names or another language. Include latency, zero-result rate, and permission failures alongside relevance. Treat clicks as behavioral evidence with exposure bias, not automatic relevance labels: an item cannot receive a click if the interface never displays it.

Version the corpus too. The March 2026 preprint Still Fresh? Evaluating Temporal Drift in Retrieval Benchmarks compared October 2024 and October 2025 FreshStack snapshots for LangChain and reported ranking correlation up to 0.978 Kendall tau at Recall@50. Its narrow result does not guarantee stability elsewhere. It illustrates why corpus dates and renewed judgments belong in the experiment record.

5. Trace a write all the way into search

An accepted write is not the same event as a searchable write. Elastic's near-real-time search documentation explains that refresh makes changes available to search. Current documented defaults differ: Elasticsearch Stack refreshes eligible active indices every second; Elastic Cloud Serverless defaults to five seconds. Neither number promises end-to-end freshness through your ingestion, transformation, and embedding pipeline.

Trace the whole route: commit to authoritative storage, emit a durable change, transform the document, generate or reuse embeddings, apply the index operation, refresh, and observe the result. Define freshness from source commit to searchable visibility. Measure a distribution and oldest pending change, not only the mean queue depth.

Handle duplicates and reordering explicitly. Carry a stable document identifier and monotonic source version through every stage. Elastic's indexing API documents external versioning: updates with a version no greater than the stored version are rejected. Your consumer must distinguish an expected stale-event conflict from a transport failure. The version must represent source ordering, not the time a worker happened to finish.

Deletes need their own proof. A delayed upsert must not resurrect a removed document. Elastic's general index settings document retention of deleted-document versions through index.gc_deletes, with a default of 60 seconds. Do not assume that temporary retention protects arbitrary old replays. Propose durable source tombstones or a replay boundary that remains valid throughout the maximum event delay.

For a read-after-write interface, be selective. The refresh-parameter documentation distinguishes forcing a refresh from waiting for one. Explain whether the caller needs its edit searchable before acknowledgment, or whether a documented pending state is acceptable. Forcing refreshes on every background update can trade ingestion efficiency for an unnecessary visibility guarantee.

6. Apply permissions before content escapes

Authentication identifies the caller; it does not determine every document they may see. Derive tenant and group context at a trusted boundary and apply it throughout retrieval. Include snippets, highlights, facets, counts, autocomplete, and cached results in the permission contract. Removing an unauthorized document from the final list is too late if its title already reached a client or external reranker.

Distinguish filtering placement from its semantics. Elastic's vector-filtering explanation distinguishes prefiltering through knn.filter from postfiltering retrieved candidates. A postfilter can leave fewer than the requested number of results. Test highly selective access policies and measure recall over the eligible corpus, rather than celebrating good recall that includes forbidden documents.

Check the engine's security composition rules. Elastic's document-level security documentation explains that document queries from multiple roles combine with OR; an unrestricted role can remove the restriction supplied by another role. Permission tests should include users with multiple roles, service accounts, and recently changed group membership.

Separate document access from strict information isolation. The same documentation warns about aggregate-information inference because relevance scoring still uses global statistics. If the product promises that tenants cannot infer anything about another tenant's corpus, state that stronger requirement and evaluate a separate index boundary. A permitted result list alone does not prove it.

A filtered alias is not a complete authorization boundary. The alias documentation notes that alias filters apply to Query DSL searches but not direct retrieval by document ID. Name the enforcement point for every access path. For urgent revocations, use a current authorization check before exposing content and define how cached snippets are invalidated. If the permission service is unavailable, reject protected searches rather than broaden access.

7. Size the work and defend the latency budget

Estimate storage using the retrieval unit you chose. Suppose the interview assumes two million documents, four passages per document, and 768-dimensional float32 vectors. Eight million vectors times 768 times four bytes is 24.576 billion bytes, about 22.9 GiB. That is vector payload only. Text, metadata, graph structures, replication, and operating headroom add capacity. Present this as arithmetic under explicit assumptions, not a vendor storage quote.

Size query work separately. Suppose peak load is 200 queries per second and every query reranks 50 candidates: that is 10,000 query-passage pairs per second. Batching, hardware, token lengths, and the chosen model determine whether the reranking service can sustain it. Keep indexing and serving capacity budgets separate so a backfill cannot quietly consume all interactive compute.

Break the response target into stages: authorization, query encoding, retrieval, fusion, reranking, and result rendering. Allocate deadlines and cancellation rather than adding independently measured p95 values as though their sum were the end-to-end p95. Google's The Tail at Scale explains why fan-out and variable component latency make the tail harder to control.

Inspect completeness as well as duration. Elastic's search API permits partial search results by default and exposes timeout and shard-failure indicators. Decide whether the workflow can display incomplete results, label them when it does, and avoid recording a partial zero-hit response as evidence that no relevant document exists.

Make fan-out risk visible with another hypothetical calculation. If 100 independent shard requests each have a 1% chance of exceeding a threshold, the chance at least one exceeds it is 1 minus 0.99 to the power of 100, roughly 63.4%. Independence is an illustrative assumption, not a production measurement. It motivates measuring the complete request path and examining shard placement.

Bound queued work and shed optional effort during overload. Google SRE's overload guidance discusses queue limits and load shedding. For this design, dropping reranking or reducing an expensive candidate budget may preserve an authorized baseline. Return a clear failure when no safe baseline exists. Measure degraded-path frequency so reduced quality does not become an invisible permanent operating mode.

8. Rebuild safely and paginate consistently

An analyzer change or embedding-model upgrade can require a new index. Build a versioned replacement from a source snapshot, then replay subsequent changes while the existing index serves traffic. Record the source watermark, transformation version, and embedding model. Validate document counts, sampled content, deletes, permission behavior, relevance slices, and latency before switching.

Elastic's reindex documentation states that the destination must be configured first; mappings and settings are not copied automatically. Reindexing documents also does not magically reconcile writes that happen after the source snapshot. Explain the catch-up mechanism and prove it has reached the chosen cutover point.

Use a serving alias to avoid binding clients to a physical index name. The aliases API supports an atomic remove/add swap, but action lists can report partial failures. Use the appropriate must_exist condition and inspect action results. Pointer replacement does not make the entire migration correct. An old index must receive ongoing changes or catch up before serving a rollback.

Pagination has a related consistency problem. Elastic recommends search_after with a point in time for deep pagination. Preserve the query, sorting, and a stable tie-breaker across pages; bound snapshot lifetime and resource use. A snapshot freezes search state, not the user's permission to view it forever. Recheck current authorization when serving later pages, even if that makes a page shorter after a revocation.

9. Rehearse an answer that survives follow-ups

Use a 45-minute practice session to make the design concrete. Spend the first five minutes clarifying the workflow, corpus, access model, and target load. Spend the next eight on authoritative storage, change ingestion, and a lexical baseline. Then add hybrid retrieval only if the query mix justifies it. Reserve time for evaluation, stale updates, revocations, and a failed rebuild.

Deliver a concise opening: “I will treat the index as a derived view. I will start with field-aware lexical retrieval, measure candidate recall and final ranking quality on judged queries, and add vectors or reranking when they improve the relevant slices within our latency budget. Source versions govern asynchronous updates, and current permissions govern what leaves the service.”

Then test yourself with scenarios that require a decision:

• The exact API name vanished. Inspect field mapping, analyzer tokens, and candidate retrieval before replacing the embedding model.
• A delete was followed by an old update. Identify the durable version or tombstone that rejects resurrection beyond temporary engine retention.
• An employee lost access between pages. Apply current authorization to the snapshot's hits and every derived snippet.
• Reranking consumes the response deadline. Serve the permitted baseline, cancel unnecessary work, and record the degraded path.
• A new index scores better overall. Inspect important query slices and operational behavior before making it the serving index.

The useful staff-level habit is to connect a decision to evidence that could disprove it. Name the user harm, the governing invariant, the measurement, and the fallback. In your next mock interview, keep one search query visible throughout the conversation and follow it through ingestion, retrieval, ranking, permission checks, and pagination. That gives every box in the architecture a reason to exist.