The Staff Engineer Unicode Interview: Text Boundaries, Normalization, and Identity

Two usernames look identical on screen. One account can sign in; the other disappears from lookup. In this illustrative interview scenario, the important question is not which database you chose: it is which definition of “the same text” each component used.

A Unicode interview can connect a small string function to account ownership, storage limits, search, and migrations. For senior and staff candidates, a useful preparation goal is to explain those connections without treating every text field alike. This guide develops one hypothetical service with public handles and editable display names. Its exercises and evaluation criteria are original practice tools, not a description of any employer's interview process.

1. Name the unit before implementing the function

Bytes, code points, and grapheme clusters answer different questions

RFC 3629 defines UTF-8 using one to four bytes for an encoded Unicode scalar value. A Unicode code point is a position in the Unicode codespace; scalar values exclude surrogate code points. An extended grapheme cluster can contain multiple code points. UAX #29 defines default segmentation rules that approximate the units people perceive as characters.

Consider this worked example, with the escape sequences making the representation explicit. The precomposed é, U+00E9, occupies two UTF-8 bytes and one code point. The sequence e\u0301 occupies three bytes and two code points. Both form one extended grapheme cluster under the default rules. Byte counts, code-point counts, and grapheme counts therefore describe different properties of the same-looking text.

Clarify the operation before coding: is the task to bound a network payload, count visible units, move a cursor, or extract a database key? For the hypothetical service, propose a byte limit for incoming requests and a separate grapheme limit for display names. State the actual limits as interview assumptions. UAX #29 permits sequences of extending characters, so a grapheme limit alone cannot bound storage.

In Go, len(s) measures bytes, indexing selects a byte, and range decodes runes with byte offsets. A string can contain arbitrary bytes, as the Go team's explanation of strings emphasizes. Converting to []rune does not turn code points into grapheme clusters. For a user-facing truncation exercise, select complete grapheme boundaries with a suitable segmenter, then verify the resulting encoded size.

Choose malformed-input behavior deliberately

For the handle API, propose rejecting malformed UTF-8 before deriving identity. Go's utf8.ValidString checks validity; decoder behavior is a separate issue. DecodeRuneInString returns RuneError with width one for an invalid encoding, while a valid encoded replacement character also yields RuneError with a different width. Checking only the decoded rune confuses these cases.

Carry that decision across the service boundary. Require import jobs, administrative tools, and signup requests to use the same handle-validation path. Give users a clear validation error and avoid logging raw handles unnecessarily. In a mock interview, trace one rejected request through the API and explain why it never reaches the identity index.

2. Define the equivalence your product needs

NFC and NFKC preserve different distinctions

UAX #15 distinguishes canonical equivalence from compatibility equivalence. NFC applies canonical decomposition and composition; it brings the two accented examples above to the same normalized representation. NFKC additionally applies compatibility decomposition, so distinctions such as a circled digit versus its ordinary digit can disappear. Neither form is a general instruction to remove accents or ignore case.

Make the product decision explicit. For a display name, propose preserving the user's accepted spelling and deriving any search representation separately. For a handle, select a documented identity profile before choosing transformations. Ask whether compatibility variants should reserve the same handle. If the interviewer changes that requirement, explain which stored key and collision checks must change.

A helpful thought experiment is a mathematics product where superscripts carry meaning. Applying compatibility normalization to every stored field would erase distinctions the product may need. Conversely, a handle namespace might intentionally ignore certain compatibility differences. The design argument should name the field and the required equality, rather than promise one universal sanitizer.

Case folding is a comparison operation

The Unicode case-mapping FAQ separates case conversion for text presentation from case folding for caseless comparison. Full case folding can expand a character: sharp s, ß, folds to ss. Lowercasing ß leaves it unchanged. Default folding also differs from language-specific casing conventions. Keep folded values internal and retain the spelling intended for display.

Do not infer that every identifier specification uses folding. RFC 8265's UsernameCaseMapped profile specifies width mapping, permitted-code-point checks, lowercase mapping, NFC, and directionality rules in a defined sequence. Its final comparison is byte-for-byte. Study the complete profile before claiming conformance; replacing its lowercase rule with arbitrary folding changes the contract.

For interview preparation, write down a proposed profile's name, mapping order, rejection rules, and examples of equal and unequal inputs. Treat a library's default behavior as something to verify against that contract. A concise answer is: “I will choose the identifier policy first, then use an implementation of that policy consistently on creation and lookup.”

3. Separate account identity from resemblance and search

Give each representation a specific job

For the hypothetical service, use an opaque account ID for ownership and references. Store an accepted display name, a handle spelling, and a derived handle key with its policy version. Keep optional search tokens and spoofing signals separate. This is a proposed schema, not a Unicode requirement. Its purpose is to make each decision reviewable when the interviewer introduces a new failure.

A search query may intentionally match broadly. That does not authorize signing in as whichever nearby name appears first. Design exact account resolution around the handle policy, with authorization attached to the opaque account ID. When a user changes a display name, demonstrate that memberships, documents, and audit events continue to reference that same ID.

Use a comparison worksheet during practice. Put two input strings in each row and answer four separate questions: do they share an identity key, should search retrieve both, should the interface display them differently, and should registration request extra review? Requiring separate answers exposes ambiguous requirements before they become code.

Normalization does not establish visual trust

A Latin letter and a similar-looking Cyrillic letter can remain different after normalization. UTS #39 provides confusable detection, identifier restrictions, and script-related mechanisms. Its skeleton mapping supports comparison for confusability. A skeleton is not a display name or a universal identity key; resemblance and ownership are different product questions.

For this exercise, propose checking registrations against protected administrative handles and routing suspicious similarities to a documented decision. Explain the tradeoff before suggesting broad script restrictions. Ask which writing systems the product supports and how legitimate users appeal a rejection. “Unicode support” is too vague to tell a user why their name was refused.

Specify operational behavior too: which reason code reaches the client, what policy version appears in internal diagnostics, and who resolves a disputed handle? Keep account recovery anchored to verified account credentials. A support agent should not transfer ownership because two display names resemble each other. Those choices make the security discussion concrete without pretending that a normalization call proves a person's identity.

4. Make storage enforce the same equality as lookup

Collation is another comparison policy

ICU's collation documentation explains comparison levels and language-sensitive ordering. Sorting names for a directory and deciding whether a handle is already owned need separate requirements. Choose a locale and comparison strength for the directory. Keep the handle constraint tied to its explicit identity policy.

PostgreSQL 18 documents nondeterministic collations that can compare different byte sequences as equal, including case-insensitive or normalization-insensitive configurations. The flag controls bytewise tie-breaking; the provider and locale settings determine the comparison behavior. PostgreSQL also documents tradeoffs, including the loss of B-tree deduplication. Inspect the deployed database version and exact operations rather than assuming every text index behaves alike.

One illustrative implementation stores a derived UTF-8 handle key in a byte-oriented column and enforces a unique constraint on that key. This makes the database enforce the key's exact byte equality. An alternative is a carefully configured text collation. Choose one authority for uniqueness and demonstrate agreement between the application, lookup queries, and constraint.

Explain the race, not just the happy path

Walk through two simultaneous registrations whose spellings produce the same key. Both may pass an availability check. Require the unique constraint to select the successful insert, then translate the conflict into a handle-unavailable response. The availability check is a convenience; the write must enforce the invariant. Make retries return a defined result without creating another account.

Apply the same key derivation to handle changes, imports, and login lookup. Avoid a fallback that tries several incompatible transformations and accepts the first match. If a lookup becomes ambiguous during a migration, define an explicit resolution path. The interviewer should be able to point at every entry point and find the same ownership rule.

5. Rehearse the migration that changes text behavior

Version the policy and its dependencies

Unicode normalization has stability guarantees for text assigned in the relevant versions, as UAX #15 explains. Do not imply that every Unicode upgrade arbitrarily changes NFC. Other derived behavior needs separate review: UTS #39 warns that confusable mappings and identifier-status data can change. Record the library, Unicode data, profile, and collation versions actually used.

Propose generating candidate keys beside existing keys before a policy change. Report changed outputs, rejected historical inputs, and groups that collide under the new policy. Resolve collisions through an approved ownership process; never merge accounts automatically. Keep concurrent registrations covered by the selected policy throughout backfill and cutover. State whether conflicting new registrations pause or are checked against both namespaces.

Database upgrades deserve their own check. PostgreSQL's ALTER COLLATION documentation warns that changed collation definitions can invalidate dependent indexes. Rebuild affected objects before refreshing the recorded version. REFRESH VERSION updates catalog metadata and does not verify the rebuild. Include index validation and a rollback plan in the interview answer.

An illustrative practice session and rubric

Rehearse a 40-minute mock: spend five minutes clarifying fields and equality, ten on encoding and boundaries, ten on registration and database races, and fifteen on spoofing and migration. These timings are suggested exercises. Have a partner introduce a decomposed accent, a multi-code-point emoji, malformed UTF-8, and two spellings that collide under your proposed policy.

• Boundary reasoning: identify the unit of each limit and explain the expected truncation result.
• Identity reasoning: derive the same key on every account-resolution path and preserve the display spelling separately.
• Concurrency reasoning: show the conflicting insert and the constraint that protects ownership.
• Migration reasoning: find collisions before cutover and explain how newly created accounts remain protected during rollback.

Build a small regression corpus from those cases and the conformance tests referenced by UAX #15 and UAX #29. Verify expected acceptance, key bytes, boundary positions, and uniqueness outcomes. Prefer a few explained counterexamples to a long list of characters you cannot connect to requirements.

Finish the mock by tracing one handle from input to validation, derived key, constrained write, login, and eventual policy migration. Every stage should preserve the same ownership promise. That gives your Unicode preparation a practical destination: a design whose behavior remains explainable when the text, traffic, and dependencies change.