Skip to content

Upload Protocol

Draft

The upload protocol is a custom resumable-upload protocol modeled on TUS but trimmed to Capsule’s needs: no per-request capability negotiation, no metadata smuggled in headers, ciphertext-only payloads. Compatibility is gated once, up front, via the universal protocol handshake. (We follow TUS v1’s offset/PATCH model, not the TUS v2 draft — see the media-type note in Chunk Rules.)

This protocol is the most fragile contract between client and server: a client that misunderstands chunk alignment, offset semantics, or finalization can silently corrupt or orphan data. The endpoint table, the chunk rules, the session state machine, and the finalization steps below are the contract — every implementation MUST conform exactly. The client implementation lives in capsule-sdk::upload; the server in capsule-api-upload. The two are tested independently against the protocol surface.

The protocol is strict by construction: any request that violates the contract is rejected with a machine-readable error, never silently corrected or partially honored. Leniency here would hide client bugs until they corrupt data; strictness surfaces them as loud, attributable rejections on the first bad request. The strictness table enumerates every tolerated-vs-rejected behavior, and the error taxonomy names the code a client receives for each.

Every upload is idempotent but stateful. Uploads can complete partially and are identified by an upload ID.

An asset is never uploaded as a single plaintext file. Because Capsule is end-to-end encrypted, the client encrypts and signs everything before transmission, and the server only ever stores opaque, content-addressed ciphertext blobs. A single imported asset produces a bundle of blobs:

  • The original blob — the source asset, encrypted under the bulk AEAD with the STREAM construction.
  • Derivative blobs — thumbnails and previews, generated client-side during import (see Thumbnails), each encrypted independently.
  • The metadata blob — the CBOR metadata document (capture date, dimensions, EXIF-derived fields, the LQIP, provenance), encrypted under the bulk AEAD (see Metadata).

Each blob is its own upload with its own upload ID; the protocol does not couple them and imposes no wire ordering. Every session declares its blob’s role at creation — a closed enum original | derivative | metadata | provenance | backup in the POST /upload body, recorded on the pending row — so the server can reason about bundle completeness without reading plaintext. The client may transfer the bundle in any order — decoupling lets small derivatives land while a large original is still in flight — and the server gates visibility on the pending-asset row: the asset becomes visible to other devices once its manifest and metadata blob are finalized. Whether the original is also held yet travels as a per-asset completeness fact on the sync feed (original_held), which is what makes staged uploads possible; an asset whose original has not landed is in the derived awaiting-original state, and fetching its original returns the transient error.blob.pending_upload rather than 410. Every blob in the bundle — original, derivatives, metadata, provenance — counts toward the uploader’s storage quota.

“Blob” is defined once, in Filesystem — Server: Uniform, Opaque Blobs; this protocol is its transport, not its definition. Every asset and derivative blob carries a signed manifest envelope: at POST /upload the server validates the envelope’s created_by_device against the uploader’s device directory (invariant 7), and the client verifies the full write-tier signature on download via verify_asset. Backup artifacts are the one exception — they carry no per-asset provenance of their own (the exporting device is not the original author); their integrity rides the library-level backup MANIFEST instead (see Backup and Recovery).

The server performs no decoding, no metadata extraction, and no thumbnail generation — it cannot, since it never holds a decryption key. All such work happens client-side during import.

The upload protocol guarantees the following, and every endpoint upholds them:

  • Content-addressed. Every blob is identified by its ciphertext content hash. The plaintext hash is never transmitted to the server.
  • Idempotent. Re-creating a session for a blob with an active session returns that session; re-creating one for a hash already finalized is rejected with a conflict that names the existing asset so the client resolves it as a merge. Re-sending a chunk with the same idempotency tuple returns the same response.
  • Resumable. A session is guaranteed to survive at least the survival floor and may survive up to the 24-hour cap. A client resumes by querying the authoritative offset and continuing from there — no bytes are re-sent unnecessarily.
  • Strict. Unknown request fields, empty chunks, wrong media types, and malformed headers are rejected outright. A buggy client is stopped at its first bad request, not accommodated.
  • Strictly bounded. The total ciphertext size is declared at session creation and immutable thereafter. The cumulative received bytes may never exceed it, nor exceed the server’s per-file limit.
  • Verified. No upload is marked complete until the server has recomputed the ciphertext hash and confirmed it matches the declared value.
  • Recoverable. Every session is either driven to a terminal state or garbage-collected. There are no permanently orphaned upload files or pending asset rows.

All endpoints are authenticated with a bearer JWT.

MethodPathPurpose
POST/uploadCreate a session. The JSON body declares ciphertext size, hash (lowercase hex; digest length fixed by crypto_suite_id), content_type (closed enum), crypto_suite_id, protocol_version, blob_role (closed enum), manifest_envelope (the unencrypted manifest fields the server validates per Threat Model — Server-Side Validation Invariants), optional album_id, optional owner_id, optional intent_id (required only during an album upgrade). Unknown fields are rejected. Returns 201 with Location: /upload/{id} and X-Capsule-Suggested-Chunk-Size. Rejects per the error taxonomy.
HEAD/upload/{id}Query progress. Returns X-Capsule-Offset (next expected byte), X-Capsule-Content-Length (declared total), and X-Capsule-Upload-Status (the session state). No body — HTTP forbids one on HEAD, which is why status rides a header. This is the resumption primitive.
PATCH/upload/{id}Append a chunk at X-Capsule-Offset, with Content-Type: application/octet-stream and the required per-chunk X-Capsule-Checksum. Returns 204 and the new offset.
DELETE/upload/{id}Cancel the session — removes the upload file, the session record, and the pending asset row. Rejected with 409 while finalization is running.
GET/upload/sessionsList the uploader’s active sessions (keyed by upload_user_id — the resuming party — not owner_id), so a client can resume across app restarts.
GET/upload/{id}/receiptFetch the session’s CustodyReceipt once Completed. 409 error.upload.receipt_not_available before then; receipts are permanent, so this also works after the session record itself expires — durably via GET /assets/{asset_id}/receipts.

Creating a session writes a pending asset row to Postgres (uploaded = false) and a session record to Valkey (see Filesystem — Server: Required Services). The pending row reserves the asset ID that derivative and metadata blobs reference. The session record carries everything finalization needs — declared size and hash, content_type, crypto_suite_id, protocol_version, blob_role, the manifest envelope, album_id/owner_id/intent_id, and the uploader — so a session is finalizable from its own record with no further client input.

Per-endpoint header census (the X-Capsule-* namespace itself is owned by the universal header census):

HeaderDirectionWhereRequired
X-Capsule-Protocol (+ -Min/-Max on responses)bothevery request/responseyes
X-Capsule-Suggested-Chunk-SizeresponsePOST /uploadyes
X-Capsule-OffsetbothPATCH request; HEAD/PATCH/409 responsesyes
X-Capsule-Content-LengthresponseHEADyes
X-Capsule-Upload-StatusresponseHEADyes
X-Capsule-ChecksumrequestPATCHyes

Enforced strictly; a violation fails the request rather than being silently corrected:

  • Media type. A PATCH body MUST be Content-Type: application/octet-stream (parameters ignored) — the payload is literally opaque ciphertext bytes. Anything else, or a missing Content-Type, is rejected with 415. We deliberately do not use the TUS v2 draft’s application/partial-upload: we implement TUS v1 semantics with our own headers, and advertising v2’s media type would claim a compatibility we do not have.
  • Non-empty. An empty PATCH body is rejected with 400. (An empty chunk can only be a client bug; accepting it as a no-op would mask that bug.)
  • Alignment. Every chunk except the final one MUST be a multiple of 4 KiB (4096 bytes). This keeps server-side writes page/block-aligned — friendly to direct and batched I/O on the append path — and doubles as a cheap tripwire for client offset-arithmetic bugs: a client whose chunking drifts mis-aligns long before it corrupts anything. The final chunk is exempt because the blob’s total size is arbitrary. A non-aligned, non-final chunk is rejected with 400. (The encryption layer’s 65,536-byte ciphertext stride is a separate concern; 65,536 is a clean multiple of 4,096, so crypto-chunked ciphertext is transport-alignable by construction.)
  • Bounds. A chunk is at least 4 KiB (except the final chunk) and at most 16 MiB; a larger body is rejected with 413. These bounds are protocol surface (see Protocol Versioning); the suggested sizes and adaptation tiers are not.
  • Sequential offsets. A PATCH must arrive at exactly the current received-byte count; an out-of-order or gapped write is rejected with 409 carrying the authoritative offset, and the client recovers by re-aligning (or issuing HEAD).
  • Required checksum + idempotency tuple. Every PATCH carries X-Capsule-Checksum: the SHA-256 of the chunk bytes as bare lowercase hex. Missing or malformed → 400. The server verifies the body against the header before any write; a mismatch is 400 with nothing persisted (the transit-corruption defense). The server keys each accepted PATCH by (upload_id, offset, chunk_hash): a duplicate PATCH with the same tuple returns the same response — a re-send after a lost ACK is a no-op — while a PATCH at an already-acknowledged offset with a different chunk_hash is rejected with 409 + a corruption error: the structural defense against a faulty client that retries with garbage. The checksum is required precisely because this tuple is undefined without it. The complete idempotency contract is owned by Threat Model — Idempotency Invariants.
  • Cumulative bounds. Cumulative size may never exceed the declared size nor the server’s max_file_size. The server checks the cumulative count at every chunk arrival, not only at finalization — a buggy client that streams past the declared size is cut off before more bytes are persisted. Either ceiling is rejected (400 / 413) and the session is moved to FailedProcessing.
  • The upload completes exactly when received bytes equal the declared size; finalization then runs automatically.

Strictness applies to the transport layer — the JSON request bodies (unknown fields rejected) and the header set above. It does not extend into the CBOR interiors (manifests, sidecars), where the schema rules deliberately require clients to preserve unknown keys for forward compatibility; that asymmetry is Postel’s law as scoped by Core Principles: be strict on the wire you own, tolerant inside the documents that outlive you.

Candidate leniencyBehaviorRejection
Unknown JSON field on POST /uploadrejected400 error.upload.malformed_request
Empty PATCH bodyrejected400 error.upload.empty_chunk
Wrong / missing Content-Type on PATCHrejected415 error.upload.unsupported_media_type
Missing / malformed X-Capsule-Checksumrejected400 error.upload.missing_checksum
Checksum ≠ body hashrejected400 error.upload.checksum_mismatch (no write)
Unaligned non-final chunkrejected400 error.upload.chunk_not_aligned
Chunk > 16 MiBrejected413 error.upload.chunk_too_large
Stale / gapped offsetrejected409 error.upload.offset_mismatch + offset
Replay, same idempotency tupleacceptedsame response as the original (no-op)
Same offset, different checksumrejected409 error.upload.chunk_conflict
Bytes past declared sizerejected400 error.upload.size_exceeded → failed
PATCH/DELETE on a finalizing sessionrejected409 error.upload.session_not_active

The upload session is gated by Capsule’s universal protocol handshake, so a client never begins a transfer against a server it is not known to be compatible with. This section names the upload-specific specializations.

Versioning is date-based (YYYY-MM-DD — the day a protocol revision is frozen), not integer or semver. An integer version conveys nothing about ordering granularity and invites a bump for every change; semver implies a minor/patch backward-compatibility contract finer than we are willing to maintain on a hot path. A date is unambiguously ordered, human-readable, and maps directly onto a release.

  • Every client sends X-Capsule-Protocol: <date> on every request. (The old upload-specific alias X-Capsule-Upload-Protocol is gone, not deprecated — nothing shipped speaks it, and an unregistered header kept alive ‘just in case’ is exactly the hidden behavior this protocol forbids.) The server advertises the inclusive range it accepts via X-Capsule-Protocol-Min and -Max on every response, errors included.
  • A POST /upload whose version falls outside the accepted range is rejected with 426 Upgrade Required before any session or pending asset row is created. The response names the supported range so the client can show an actionable message (“update Capsule to keep uploading”). Per Threat Model, the same rule applies to every other write surface.
  • This is a one-shot compatibility gate, not negotiation: there is no back-and-forth to settle on a shared version, and the protocol carries no capability flags. A client either speaks a version the server accepts, or it does not upload.
  • The server supports a window of past protocol versions, not only the newest, so a staggered client rollout keeps working. A version leaves the window only after the deprecation period defined in Threat Model — Min-Supported-Client Deprecation Policy; dropping one is a breaking change announced ahead of time.
  • The date is bumped only for an incompatible wire change — offset semantics, alignment rules, the hard chunk bounds (raising the 16 MiB maximum is additive; lowering it or raising the 4 KiB minimum is breaking), finalization, the state machine. Purely additive, safely-ignorable changes do not bump it, and server-tunable parameters such as suggested chunk sizes and adaptive-sizing tiers are not protocol surface at all.

A session moves through a strict state machine:

Pending ─▶ Uploading ─▶ WaitingForProcessing ─▶ Completed
└─▶ FailedProcessing
FromToOn
PendingUploadingfirst accepted chunk (rejected chunks do not transition)
UploadingWaitingForProcessingreceived bytes == declared size
Pending/Uploading(discarded)TTL expiry or pressure eviction (see below)
WaitingForProcessingCompletedrecomputed hash matches; pending row marked uploaded
WaitingForProcessingFailedProcessinghash mismatch, size mismatch, or envelope re-validation failure
  • Pending — session created, no bytes received.
  • Uploading — at least one chunk accepted, transfer in progress. The transition is observable: a HEAD reflects it in X-Capsule-Upload-Status.
  • WaitingForProcessing — all declared bytes received; finalization (hash verification) is running. Not evictable, not cancellable.
  • Completed — hash verified, asset marked uploaded. Terminal.
  • FailedProcessing — terminal failure. The upload file and the pending asset row are removed at the transition; the session status record is retained as a receipt. Terminal.

Session records live in Valkey — keys upload:session:{id} with atomic HINCRBY/HSET and native TTL — with an uploader-keyed index for listing. This split is intentional: the session store holds only volatile transfer state, so the hot path — offset increments and status transitions — never touches the durable Postgres asset row. Postgres’s durable asset record is written exactly twice per upload: once at session creation (the pending row) and once at finalization (mark uploaded).

A session’s lifetime is a floor and a cap, not a flat promise:

  • Survival floor (guaranteed). A session in Pending or Uploading survives at least 1 hour after its last accepted chunk (after creation, if none yet). Within the floor it is never evicted, under any pressure — a slow but live upload keeps its session as long as it keeps making progress.
  • Cap. No session outlives 24 hours from creation, progressing or not.
  • Discard window (server’s right). Between the floor and the cap, the server MAY discard sessions to reclaim space, least-recently-progressed first (ties broken toward the most on-disk bytes — reclaiming the most space from the most-stalled uploads first).
  • After discard: uniform 404. A discarded, expired, or never-existent session is indistinguishable — HEAD/PATCH/DELETE all return 404 error.upload.session_not_found. No tombstones, no 410: this matches the platform-wide indistinguishable-404 rule (invariant 26), and the client recovery is identical either way — re-create the session and resume from zero for that blob, retrying with backoff and halting after a bounded number of attempts.
  • Terminal receipts are exempt from pressure eviction — their upload bytes are already gone (Completed renamed the file into the blob store; FailedProcessing deleted it), so evicting them reclaims nothing while destroying the lost-ACK receipt. They are removed only at the 24-hour cap.
  • Finalization is never interrupted. A session in WaitingForProcessing is not evicted out from under the finalizer; it is driven to Completed or FailedProcessing.
  • Startup scrub. On boot the server reconciles disk against the session store: an upload file with no session is deleted; a session whose file is shorter than its recorded received-byte count is moved to FailedProcessing (the file is authoritative — an ACK the disk cannot back must not stand). Discard, expiry, and cancellation all delete the upload file and the pending asset row.

The receipt behavior is why a client whose finalization ACK was lost re-queries (HEAD) and observes the terminal outcome — learning the upload already succeeded or failed — instead of seeing a vanished session and blindly re-uploading.

Each session owns exactly one file, incoming/{upload_id}.bin, and every accepted chunk is appended to it in order. There is no per-chunk file and no assembly step — sequential offsets are already a hard protocol rule, so append-only storage is correct by construction, and finalization has one less failure scenario (no reflink/concatenate stage that can half-complete).

  • Append with cross-check. Before writing, the server compares the file’s current length against the session’s expected offset; a divergence is a 500 error.upload.storage_inconsistent (server-side inconsistency, never the client’s fault) and the write is refused. The file length is the on-disk truth; the session counter is its cache.
  • Durability before ACK. A chunk is flushed to disk before its 204 is sent — an acknowledged byte is a byte on disk.
  • Offloaded blocking work. Hashing runs on a blocking thread pool, never on the async reactor. (The write path uses plain aligned buffered writes today; io_uring is an optimization slot, not contract.)
  • Into the blob store. Only after finalization verifies the hash is the file renamed to its content address under blobs/ — the atomic-rename discipline is owned by Filesystem — Server.
  • Backpressure. max_cache_size bounds the total in-flight upload bytes held on disk; max_file_size bounds any single blob. The configuration asserts max_file_size < max_cache_size and warns if fewer than ~10 concurrent maximum-size uploads would fit. When the bound is hit, the discard window is the relief valve.

When received bytes reach the declared size, the server finalizes:

  1. Session transitions to WaitingForProcessing via an atomic compare-and-set on the status — two racing finalizers cannot both proceed; the loser observes the transition and returns a conflict.
  2. The server recomputes the content hash over the upload file on the blocking pool and compares it to the declared hash.
  3. The manifest envelope is re-validated (invariant 15) — the same checks as at session creation, inside the finalization transaction, so nothing that changed since creation (a revoked device, a closed album) slips through.
  4. On match — the file is renamed to its content address; then, inside one Postgres transaction, the pending asset is marked uploaded and the write’s CustodyReceipt is signed and persisted — the receipt and the uploaded flip commit or fail together. The session transitions to Completed.
  5. On mismatch — the upload file and the pending asset row are deleted, the session transitions to FailedProcessing, and error.upload.content_hash_mismatch is returned. A mismatch is always treated as corruption or tampering and is never silently retried server-side.

The server verifies only the ciphertext hash — it has no other option. The client independently verifies the plaintext on download via the STREAM construction’s per-chunk authentication tags, which detect truncation, reordering, and chunk deletion. The two checks are complementary: the server guarantees “the bytes I stored are the bytes you declared,” and the AEAD guarantees “the plaintext I decrypted is authentic.”

Completed is a one-time transfer acknowledgement, not a standing durability guarantee a client can re-query later — but it is now accompanied by a durable, server-signed CustodyReceipt: the transferable evidence that this server accepted custody of exactly these bytes, fetched via GET /upload/{id}/receipt and persisted client-side. After finalization, a client confirms an asset remains durably stored, indexed, and retrievable — the precondition for releasing its local copy — through the separate storage-verification endpoint, which re-checks state that server-side GC, migration, or corruption could change after Completed.

  • Duplicate session creation is keyed by (owner_id, hash, album_id) and split by state: if an active session for the tuple exists, POST /upload returns it (200, Location + current X-Capsule-Offset) — no second session is created; if the hash is already finalized, the request is rejected with 409 error.upload.duplicate_blob carrying the existing asset reference, which is the client’s trigger to resolve the situation as a merge instead of a transfer. The dedup check and the pending-row insert run inside a single PostgreSQL transaction (SELECT ... FOR UPDATE + INSERT ... ON CONFLICT), closing the TOCTOU race at the database layer.
  • Chunk replay is governed by the idempotency tuple in Chunk Rules: same tuple → same response; same offset with different bytes → 409 corruption.
  • Resumption is HEAD-driven: the response carries the authoritative offset, the declared length, and the session status. An upload is not expected to run to completion in a single connection; the server tolerates arbitrarily long pauses within the session lifetime, and auto syncing explicitly assumes interrupted transfers are normal.
  • Lost finalization ACK: re-query HEAD and read the terminal receipt (see Session Lifetime).
  • Every critical step — session creation, each chunk, finalization — is logged with the upload ID so an interrupted or failed upload can be reconstructed after the fact.
  • Streaming import-upload mode for storage-constrained devices uses these same sessions unchanged: it creates, uploads, and finalizes them one bounded window at a time and releases each local original after a durable storage-verification check. The wire protocol is identical — only the pipeline’s drive pattern differs — and a server connection loss simply leaves the in-flight sessions resumable via HEAD.

Chunk-size selection is split cleanly between the two sides:

  • The server enforces, by rejection only, the protocol bounds: 4 KiB alignment, the [4 KiB, 16 MiB] chunk range, and the cumulative ceilings. It never adapts, negotiates, or corrects.
  • capsule-sdk owns adaptation. The X-Capsule-Suggested-Chunk-Size returned at session creation is a starting point only, from non-normative file-size tiers (currently < 10 MB → 256 KiB, < 100 MB → 1 MiB, ≥ 100 MB → 4 MiB — server-tunable, not protocol surface).

The client algorithm is normative for capsule-sdk (another client may adapt differently, within the protocol bounds):

  • Measure throughput over a sliding 30-second window.
  • Warm-up: no adjustment until ≥ 5 chunks or ≥ 8 MiB have been sent at the current size — decisions on a cold window oscillate.
  • Double the chunk size when sustained throughput exceeds 5 MB/s; halve it below 1 MB/s.
  • Clamp to the tier’s range; every tier range sits inside the protocol bounds [256 KiB, 16 MiB] for non-final chunks.
  • Every candidate size is a 4 KiB multiple by construction (the candidate set is doublings/halvings of 4 KiB-aligned tier bounds), so alignment never depends on a runtime check.

The rationale is a direct trade-off — chunks that are too small waste round-trips, while chunks that are too large waste re-transmission on a flaky link and pin more memory per in-flight request. Under an adverse connection, the conservative choice is the tier minimum; the client must never let adaptation regress effective throughput.

We deliberately do not expose per-blob upload ordering as a protocol concern. Concurrent sessions plus the OS and TCP stack settle ordering naturally; see Pipeline — Upload Prioritization for the client-side heuristics that decide which assets to start.

Every rejection carries a machine-readable error.* code (the i18n error-code contract); clients switch on codes, never on bare HTTP statuses. The full upload domain:

ConditionHTTPCodeInvariant
Protocol version outside window426error.protocol.version_unsupported1
Unknown crypto suite400error.upload.unknown_crypto_suite2
Hash length ≠ suite digest length / not hex400error.upload.invalid_hash3
size(0, max_file_size]400/413error.upload.invalid_size / error.upload.file_too_large4
content_type outside closed enum400error.upload.unsupported_content_type5
Album missing / no write capability / version drift403error.upload.album_access_denied6
created_by_device not in device directory403error.upload.device_not_authorized7
Envelope timestamp outside sanity window400error.upload.timestamp_out_of_range8
Top-level fields contradict manifest_envelope400error.upload.envelope_mismatch15 family
Malformed body / unknown JSON fields400error.upload.malformed_request
On-behalf upload without verified relationship403error.upload.owner_not_permitted
Quota would be exceeded by declared size403error.quota.exceeded
Hash already finalized (dup create)409error.upload.duplicate_blob
Wrong/missing Content-Type on PATCH415error.upload.unsupported_media_type10
Empty chunk400error.upload.empty_chunk10
Unaligned non-final chunk400error.upload.chunk_not_aligned10
Chunk > 16 MiB413error.upload.chunk_too_large10
Missing/invalid X-Capsule-Offset header400error.upload.missing_offset9
Missing/malformed X-Capsule-Checksum400error.upload.missing_checksum12
Checksum ≠ received bytes400error.upload.checksum_mismatch12
Receipt requested before Completed409error.upload.receipt_not_available33
Offset ≠ current received count409error.upload.offset_mismatch9
Same offset, different chunk hash409error.upload.chunk_conflict12
Cumulative bytes exceed declared size400error.upload.size_exceeded11
Unknown / expired / discarded session404error.upload.session_not_found
Session already terminal or finalizing409error.upload.session_not_active
Concurrent finalization lost the CAS409error.upload.finalize_in_progress
Recomputed hash ≠ declared (finalization)400error.upload.content_hash_mismatch14
Envelope re-validation failed (finalization)400error.upload.envelope_rejected15
Caller is neither uploader nor owner403error.upload.forbidden
Upload-file / session-counter divergence500error.upload.storage_inconsistent

The stateful surface, enumerated — each of these is a test in the Validation section:

  • PATCH after finalization began or completed409 error.upload.session_not_active; the terminal outcome is read via HEAD.
  • DELETE during WaitingForProcessing409 — finalization is not interruptible; cancel before completion or observe the receipt after.
  • Duplicate POST /upload → active session returned / finalized hash 409 duplicate_blob (see Idempotency and Resumption).
  • Offset beyond current EOF (client skipped ahead) → 409 offset_mismatch with the authoritative offset — indistinguishable from any other stale offset, deliberately.
  • Offset correct but offset + len > size400 size_exceeded, session → FailedProcessing (the declaration was violated; the session is unsalvageable by definition).
  • Checksum mismatch mid-stream400, nothing persisted, offset unchanged; the client re-sends the same chunk.
  • PATCH against an expired/discarded session → uniform 404; the client re-creates and re-uploads that blob.
  • Zero-length blob — impossible by construction: invariant 4 requires size > 0 at creation, so no chunk sequence can validly be empty.

Because blobs are addressed by their ciphertext content hash, the protocol avoids redundant transfers:

  • At session creation, the server checks for an asset with the same content hash already owned by the user; the split behavior (active session returned; finalized hash → 409 duplicate_blob + existing asset reference) is defined in Idempotency and Resumption. Nothing is ever re-uploaded.
  • The import pipeline treats already-uploaded local assets as non-importable. But because encryption and hashing are deferred until upload, an asset may already exist remotely under a different ciphertext (for example, re-encrypted under a newer album key). Import still admits such an asset, and the upload then resolves to a merge: the server links the existing stored blob to the new asset and album reference rather than storing a second copy. The original blob’s upload short-circuits, and only the new metadata blob is transferred.
  • Merge is strictly additive on the server. A merge never deletes an existing blob or rewrites an existing manifest — it only adds a new reference. The blob’s reference count goes up, never down, on merge. Reference removal happens only through an explicit delete lifecycle action signed by a current writer (see Authorization), and the underlying blob is hard-purged only after every reference is provably gone.

These checks deduplicate at upload time. Byte-identical assets that still slip into a client library — for example through overlapping folder imports or a restore over an existing library — are collapsed separately by client-side intra-library deduplication.

  • An upload is attributed to upload_user_id (the authenticated uploader) for storage-quota accounting, which is distinct from owner_id (the asset’s owner). Uploading on behalf of a different owner requires a verified relationship and is permission-checked at session creation. The quota accounting model is owned by Quota.
  • Adding an asset to an album requires write-tier album access (AMK_write; see Cryptography — Keys); the server validates album write permission before creating the session.
  • For an ordinary asset bundle the client resolves a concrete container album_id — the user’s choice or the default albumbefore encryption, since the bytes are encrypted under that album’s AMK. So album_id is effectively always present for asset uploads; the optional marking on POST /upload covers only non-asset/owner-scoped kinds and the intent_id-bearing album upgrade. This is why invariant 6 can require album_id to exist and be writable.
  • Only the uploader may append chunks. The uploader or the owner may query (HEAD) or cancel (DELETE) a session; anyone else receives 403. Session listing is uploader-scoped — the uploader is the party that resumes.

The wire protocol is the boundary across two modules, so both sides have rich isolated test surfaces:

  • Server protocol conformance (smoke). Exercise the full state machine against the real server against testcontainer Postgres + Valkey: create session → PATCH chunks → finalize → verify Completed. Mock the client at the HTTP layer using a generated request fixture set.
  • Server strictness rejection (unit). Every row of the strictness table and every row of the error taxonomy has a unit test asserting the exact HTTP status and error.* code.
  • Server idempotency (unit). Replay each idempotent endpoint with identical input; assert byte-identical response. Duplicate create returns the active session; finalized-hash create returns duplicate_blob.
  • Server lifetime (unit). A session with an accepted chunk in the last hour is never evicted under injected pressure; a stalled session past the floor is; terminal receipts survive pressure and die at the cap.
  • Server append-only crash safety (smoke). Kill the server between the append and the counter increment; on restart the scrub reconciles (file length is truth) and HEAD reports the on-disk offset. Kill between rename and commit; assert the atomicity invariants recover.
  • Server finalization integrity (smoke). Upload; recompute hash; assert match. Inject a corrupted chunk stream; assert FailedProcessing and full cleanup of the pending row + upload file.
  • Client protocol conformance (smoke). The client capsule-sdk::upload runs against a mocked HTTP layer that replays every taxonomy code; assert the client’s recovery matrix (re-align on offset_mismatch, re-create on session_not_found, merge on duplicate_blob, abort-with-upgrade on 426, re-send on checksum_mismatch).
  • Client resume semantics (smoke). Start an upload, interrupt at random offset, resume; assert no bytes re-sent that the server already has.

The cross-module case — real client → real server full upload — is bounded E2E surface listed in Module Map. Because both sides have rich smoke coverage, the E2E case can be a single happy-path round-trip rather than the full rejection matrix.