Design a Resumable File Upload Service: A Staff Engineer Interview Playbook (2026)

A file upload can return HTTP 200 and still fail. Amazon S3 explicitly documents this behavior for multipart completion: an error can appear inside the response body after the success header. That detail is a useful opening for a staff engineer system design interview because it forces you to define what “uploaded” actually means.

Use this playbook to design a service that survives interrupted connections, protects tenant boundaries, and publishes only validated files. The architecture and operating policies below are proposed interview choices. Provider limits and protocol behavior come from the linked official documentation; workload numbers are illustrative assumptions.

1. Define the product contract and its invariants

Start by asking which files matter: profile photos, customer documents, or multi-gigabyte videos. Ask about maximum size, supported clients, geographic distribution, retention, and whether the user must resume after closing a browser. Clarify whether “complete” means bytes stored or safe to download. These answers determine the critical path.

For this exercise, propose authenticated uploads up to 100 GiB, browser and mobile clients, a 24-hour resume window, and asynchronous security scanning. These are product assumptions, not cloud limits. Define separate states for initializing, uploading, finalizing, quarantined, available, rejected, and expired. The interface can report transfer completion while explaining that processing continues.

Separate service objectives for session creation, accepted-part durability, and processing delay. A successful control request should not promise a fixed end-to-end upload time on an unknown connection. Ask whether customers need a processing deadline, a regional data boundary, or an explicit cancellation guarantee. State the unresolved requirements and show which decisions would change when the interviewer answers them.

Write the rules before the boxes

• Every upload session belongs to exactly one tenant and authorized owner.
• A resume reuses acknowledged bytes from the same file and session.
• Finalization selects one immutable manifest of expected parts.
• Only a verified object with an accepted scan result becomes downloadable.
• Expired sessions eventually release reserved quota and stored fragments.

Use these invariants to evaluate every later choice. For example, a random object key reduces naming collisions, but the API must still authorize access to the session. OWASP’s upload guidance recommends authorized uploaders, application-generated filenames, size limits, and layered validation. Translate those recommendations into enforceable service behavior.

2. Separate session control from byte transfer

Choose a control API for authorization and metadata, object storage for bytes, a relational database for session state, and workers for finalization, scanning, and cleanup. The client asks the API to create a session, uploads parts directly to storage, records acknowledgments, and requests finalization. Downloads go through an authorization service that checks availability.

Direct transfer keeps large request bodies out of application servers. The cost is a capability boundary: S3 presigned URLs are bearer tokens, bounded by the signer’s permissions and credential lifetime. Issue narrowly scoped URLs for one session and part. Use short expiry, renew them after authorization, and redact them from logs.

Make the API express the workflow

Operation	Proposed responsibility
POST /uploads	Authorize, reserve quota, allocate a session and object key.
POST /uploads/{id}/parts	Authorize requested part numbers and issue bounded capabilities.
PUT /uploads/{id}/parts/{number}/receipt	Validate and persist a part acknowledgment.
GET /uploads/{id}	Return state, expiry, and the acknowledged manifest.
POST /uploads/{id}/complete	Freeze the manifest and start recoverable finalization.
DELETE /uploads/{id}	Request cancellation and eventual resource cleanup.

Store tenant, owner, expected size, part size, checksum policy, storage upload ID, expiry, state version, and final object identity. Keep part receipts in a child table. Browser integration also needs CORS response-header exposure for ETags and checksums. Test that JavaScript can read them, not merely that a PUT succeeds.

Give creation requests an application idempotency key scoped to the tenant. Persist a fingerprint of the requested file attributes and return the original session when a matching request repeats. Reject conflicting reuse. This prevents a lost creation response from reserving quota twice. Keep storage initialization recoverable as well: a database session can remain initializing until its storage upload ID is durably attached.

3. Calculate part size from explicit limits

Do not repeat the old 5 TB ceiling from memory. The current S3 multipart limits table lists a maximum object size of 48.8 TiB, at most 10,000 parts, and part sizes from 5 MiB to 5 GiB. The final part has no minimum size requirement. Our proposed 100 GiB product limit is deliberately smaller.

For known file size S, choose part size P so ceil(S / P) ≤ 10,000, while respecting storage bounds. Start with a practical default and enlarge it when necessary. Freeze boundaries when creating the session; changing them during resume changes the manifest’s interpretation.

Now estimate service load. Assume 20,000 completed uploads per day averaging 256 MiB: that is about 4.88 TiB daily, or 59.3 MiB/s averaged across a day. At 64 MiB per part, the idealized workload needs 80,000 part writes daily, about 0.93 per second, before retries. A hypothetical tenfold peak needs about 593 MiB/s and 9.3 part writes per second.

These averages hide the largest files and synchronized bursts. Request the size distribution and peak active uploads before selecting capacity. Smaller parts reduce retry work; larger parts reduce request counts. A resume verifier must paginate: S3 returns at most 1,000 parts per list request, so our largest example already exceeds one page.

If the source length is unknown, require a declared upper bound or select another explicit policy before issuing capabilities. A stream cannot keep the smallest part size indefinitely and assume the part budget will suffice. Bound the session’s authorized part numbers and remaining byte reservation, then reject growth beyond the agreed product limit.

4. Resume from durable acknowledgments

There are two useful protocol models. An offset protocol exposes the durable byte position. A multipart protocol exposes a set of completed part numbers. Choose one client contract explicitly; “retry the upload” leaves unanswered which bytes the service has accepted.

With tus 1.0, HEAD returns Upload-Offset; PATCH must present that same offset. A mismatch produces 409 without changing the resource. This is a concrete answer to “the response disappeared after the server accepted bytes”: query the server before continuing.

For direct S3 multipart uploads, persist the session ID and each part’s number, size, checksum, and returned ETag. On reconnect, reconcile expected receipts against storage and upload missing parts. Reusing a part number overwrites the previous part. Treat retry as safe only when the session, byte range, and expected content remain identical.

Handle the acknowledgment gap

A part can reach storage while its receipt never reaches your database. Use ListParts pagination markers to verify storage evidence, compare it with expected boundaries and checksums, and persist validated receipts. Do not blindly complete whatever storage lists: AWS recommends maintaining your own completion manifest.

After browser restart, let the user reselect the file. Name, size, and modification time are convenient hints, not sufficient proof of content identity. Bind resumed parts to the declared checksum policy. Reject a different file rather than silently combining it with earlier parts. Renew URLs independently of session expiry, and make lost client state recoverable through the authenticated API.

Show two progress measures: bytes currently transmitted and bytes durably acknowledged. The first can move backward after a retry; the second should describe recoverable progress. A receipt update must match the session generation and expected part identity. A delayed client acknowledgment cannot replace a newer validated receipt simply because it arrives last.

5. Specify integrity independently of ETags

Keep a storage receipt, a transport checksum, and an application content digest as separate concepts. A completed multipart ETag is not necessarily an MD5 hash of the object. Save it for provider operations; avoid presenting it as a universal file fingerprint.

S3’s integrity documentation distinguishes full-object checksums from composite checksums assembled from part checksums. For multipart upload, CRC64NVME supports full-object checksums; CRC32 and CRC32C support both types; SHA-256 supports composite checksums. Single-part SSE-KMS ETags also are not MD5 digests. Algorithm, encoding, and checksum type belong in the contract.

For this design, validate part checksums during transfer and configure the algorithm at multipart initiation. Use the provider’s supported full-object CRC mode for storage validation. If the product also requires a conventional SHA-256 digest of file bytes, compute and verify that separately. A client’s asserted digest alone cannot establish that the stored bytes match it.

Composite checksum uploads require consecutive part numbers beginning at one, according to S3’s checksum rules. Our manifest validator enforces that sequence. In an offset implementation, the tus checksum extension discards a mismatched chunk and leaves the offset unchanged. Integrity failures should preserve trustworthy progress and produce a specific recovery instruction.

6. Make finalization a recoverable state machine

A storage completion and a metadata transaction are separate operations. Avoid claiming one transaction commits them both. Recommended flow: conditionally move the session from uploading to finalizing, freeze the ordered manifest, persist a finalization attempt, and let a worker complete the storage upload. Stop issuing new part capabilities once the manifest is frozen.

Validate every expected part, total byte count, and checksum policy before calling completion. Already issued URLs remain a race to consider: bind them to expected part content, keep their lifetimes bounded, and reject changed receipts. The completion request must carry the selected part numbers and ETags, as the S3 API requires. Parse its full result using an appropriate SDK.

Recover the ambiguous success

If the worker crashes after storage completion, leave the session finalizing. A reconciler checks the unique object key, version where enabled, size, and configured checksum against the frozen attempt. Matching evidence advances it to quarantined. A missing object triggers recovery of the attempt; contradictory evidence requires investigation. A timeout is not permission to publish or start a conflicting upload.

S3 provides strong read-after-write consistency and atomic updates to a single key. Those guarantees help reconciliation, but do not make your database and object store one transaction. Allocate a unique key per session. Where appropriate, CompleteMultipartUpload supports If-None-Match to prevent overwriting an existing object. For 412, reconcile the existing object against the attempt; its existence alone is insufficient. For a conditional completion 409, S3 requires a new multipart upload and reupload.

Once storage evidence is verified, commit quarantined state and a scan-request outbox record in one database transaction. A dispatcher publishes the record and retries until acknowledged. The transactional outbox pattern addresses the database/event dual-write gap and still requires idempotent consumers. Repeated completion calls return the same workflow status.

Define cancellation during finalization separately from cancellation during transfer. Once completion may have succeeded, aborting a multipart ID is insufficient. Record deletion intent and reconcile the resulting object before removing it. A scan worker must check that intent and the current state version before publishing availability. Retrying either workflow must preserve the cancellation decision.

7. Keep scanning on the availability boundary

Store newly completed objects in private quarantine. The download authorizer must require available state and the exact approved object identity. Do not issue a download URL at transfer completion. This makes asynchronous scanning compatible with a clear user experience: bytes received, processing pending, ready, or rejected.

OWASP warns that client Content-Type can be spoofed and recommends complementary extension, signature, content, and antivirus checks. Apply an allowlist appropriate to the product. Bound decompression output, parser runtime, and scanner memory as proposed operating safeguards; a large permitted object need not be a safe parser input.

Record scan engine and policy versions, object version, result, and timestamp. GuardDuty’s documented scan results distinguish NO_THREATS_FOUND from THREATS_FOUND, UNSUPPORTED, ACCESS_DENIED, and FAILED. Unsupported or failed scans need an explicit product policy. They are not clean verdicts.

Check the chosen scanner’s own size, archive, encryption, and runtime limits before adopting the 100 GiB product assumption. The result vocabulary above illustrates workflow semantics; it does not establish that a particular managed scanner supports every accepted file. When requirements exceed scanner capability, narrow the product contract or design an appropriate scanning path before launch.

Deduplicate work by session and object identity. S3 notifications are designed for at-least-once delivery and may arrive out of order. A late clean event for an older object must never approve a newer replacement. Commit availability with its notification outbox record, and expose retry or review status when processing stalls.

8. Bound admission, concurrency, and retries

Backpressure starts when creating the session. Reserve expected bytes against tenant quota, limit active sessions, and issue part URLs in bounded batches. This constrains authorized in-flight work. Direct uploads still need storage-side limits and final size validation; API request quotas alone do not bound stored bytes.

Give clients a proposed initial concurrency of four and lower it under throttling or memory pressure. For API rate limits, RFC 6585 defines 429 and permits Retry-After. Return a useful retry instruction. Monitor storage throttling separately: AWS recommends tracking S3 503 responses.

Cap retries by attempts and elapsed time, add randomized backoff, and avoid multiplying retries across client, gateway, and worker layers. AWS SDK retry guidance describes jitter and retry quotas; actual behavior depends on SDK version and configuration. Treat those mechanisms as a starting point, then set a workflow-wide budget.

Bound scanner backlog in bytes as well as object count. In our illustrative workload, ten minutes at the hypothetical peak of 593 MiB/s adds roughly 347 GiB of unprocessed data if scanning stops. That calculation motivates admission reduction before quarantine storage becomes the outage. Prioritize existing sessions over new ones where practical, and communicate delays rather than promising instant readiness.

9. Treat abandoned uploads as an operating workload

Track session expiry, reserved quota, multipart fragments, quarantined objects, and available objects separately. Uploaded S3 parts incur storage charges until completion or abort. An expired database row alone does not remove them.

A cleanup worker conditionally marks an eligible session expired, stops issuing capabilities, aborts its storage upload, verifies cleanup, and releases quota. Race it against finalization using the state version. S3 warns that in-flight parts can survive an abort attempt, requiring repeated aborts and verification when necessary.

Add an AbortIncompleteMultipartUpload lifecycle rule as a storage safety net. Its clock runs from initiation, and it does not delete completed objects. Choose its age beyond the promised resume window; separately expire rejected quarantine objects under the product’s retention policy.

Publish the expiry contract. The tus expiration extension uses Upload-Expires and supports a gone response for removed uploads. Monitor abandoned bytes, cleanup age, and quota reconciliation failures so cleanup remains observable.

10. Rehearse the failures that test your design

Use a proposed 45-minute rehearsal: five minutes for requirements, seven for sizing, eight for architecture, fifteen for failure handling, and ten for tradeoffs. Spend the deep dive on the object/metadata boundary. Then ask a practice partner to interrupt the happy path.

Close with concrete verification: inject lost acknowledgments, duplicate scan events, a worker crash after storage success, and aborts during active transfers. Measure resume recovery, finalization age, time to availability, checksum rejection, and abandoned storage. Explain which invariant each experiment protects.

Offer an incremental rollout: first prove bounded uploads and private processing, then enable resume and tune concurrency using measured client behavior. Keep operational ownership explicit for stuck finalizations, scan failures, and orphaned storage. A design is easier to defend when each exceptional state has a repair procedure and a team responsible for executing it.

Your strongest ending is a precise promise: acknowledged bytes survive interruption, finalization is recoverable, and downloading requires verified availability. Every component should earn its place by helping uphold that promise. Bring the same clarity to your next mock interview and let the failure cases reveal where your explanation needs work.