TransparencyKit — Specification v0.1

Status: draft · Date: 2026-10-14 · Licence: Apache-2.0

TransparencyKit is a single static binary that lets a cloud, SaaS, PaaS or IaaS provider keep a tamper-evident ledger of transparency-relevant events in S3-compatible object storage and publish signed, verifiable transparency outputs from it.

"Proof, not prose": every published number is derived from a hash-chained private ledger, and every published file is covered by a signed manifest that anyone can check.

1. Goals and non-goals #

Goals #

Non-goals (v0.1) #

• Submitting statements of reasons to the DSA Transparency Database. Only online platforms owe these; hosting services do not.
• Content-moderation workflow, case management or ticketing. TransparencyKit records outcomes; it does not run the process.
• Legal advice. Templates and categorisations must be reviewed by counsel before filing.
• Witness co-signing / public transparency-log anchoring (planned v0.3, see §11).

2. Regulatory mapping #

Template fidelity caveat. The DSA CSV column headers are aligned with published hosting-provider filings of the Annex I template (tabs and column semantics), but v0.1 is not validated byte-for-byte against the official XLSX. Operators must map/verify against the Commission's current template before filing. Median times are expressed in hours.

3. Architecture #

            operator workstation / CI                     object storage (SOS)
 ┌──────────────────────────────────────┐        ┌───────────────────────────────────┐
 │ transparencykit record add  ───────────────► │──PUT──►│ private bucket (object lock ON)   │
 │ transparencykit ledger verify ◄───────────── │◄──GET──│   records/000000000001.json …     │
 │ transparencykit publish ──┐                  │        │   (record objects only)           │
 │   aggregate+band  │  sign (Ed25519)  │        └───────────────────────────────────┘
 │   render          ▼                  │        ┌───────────────────────────────────┐
 │                manifest.json+.sig ───│──PUT──►│ public bucket (public-read ACL)   │
 └──────────────────────────────────────┘        │   reports/2026/v1/…  latest/…     │
                                                 │   latest.json  index.html  .pub   │
 anyone: transparencykit verify https://…  ◄────GET──────│                                   │
 browser: direct object URLs or `transparencykit serve`  └───────────────────────────────────┘

SOS has no static website hosting and no event notifications. Public files are read through direct object URLs (https://<bucket>.sos-<zone>.exo.io/<key>) or through transparencykit serve. The operator runs publish on a schedule; nothing reacts to events.

4. Ledger format #

4.1 Canonical JSON #

All hashing uses canonical JSON: UTF-8, object keys sorted lexicographically by byte, no insignificant whitespace, no HTML escaping, integers only (floats are rejected in ledger records), strings escaped per Go encoding/json. This matches RFC 8785 for the value types TransparencyKit allows.

4.2 Entry envelope #

Each ledger entry is one object at records/<seq zero-padded to 12>.json:

{
  "v": 1,
  "seq": 42,
  "type": "gov_request",
  "recorded_at": "2026-10-14T20:15:03Z",
  "recorded_by": "legal@example.ch",
  "amends": 0,
  "prev": "sha256:<64 hex of entry 41's hash>",
  "body": { "...": "type-specific, see §5" },
  "hash": "sha256:<64 hex>"
}

• hash = "sha256:" + hex(SHA-256(canonical(envelope without "hash"))).
• Genesis prev is sha256: followed by 64 zeros.
• seq starts at 1 and is contiguous.
• amends (optional, omitted when 0) is the seq of an earlier entry of the same type that this entry supersedes. It must point to an original (non-amending) entry. The latest amendment wins.
• recorded_by is optional free text (operator identity).
• Unknown envelope or body fields are rejected.

4.3 Append protocol #

1. List records/ and find the highest seq n.
2. Fetch entry n, recompute and check its hash.
3. Build entry n+1 with prev = hash(n) and validate the body.
4. PUT records/<n+1> with If-None-Match: *. On 412 Precondition Failed (another writer won), restart from 1, up to 5 attempts.

Concurrency caveat. Exoscale does not document SOS support for conditional writes. Without it, two simultaneous writers can race. Mitigations: run a single writer (recommended), keep bucket versioning on (object lock implies it) so an overwritten entry survives as a prior version, and run ledger verify after writes. v0.2 adds ledger verify --versions (ListObjectVersions).

4.4 Immutability #

Bucket and API prerequisites are listed in Storage requirements.

Create the private bucket with object lock (it cannot be enabled later on SOS) and a default retention (e.g. GOVERNANCE, 10 years). Optionally TransparencyKit sets per-object retention headers (x-amz-object-lock-mode, x-amz-object-lock-retain-until-date, plus Content-MD5) when ledger_retention is configured. Scope a dedicated IAM key to the bucket.

4.5 Verification (transparencykit ledger verify) #

Fails if any of these hold:

• a gap or duplicate in seq;
• a stored hash that doesn't match the recomputed hash;
• a prev that doesn't match the previous hash;
• a body that fails schema validation;
• an amends target that is missing, of another type, or itself an amendment.

It prints the head seq and hash. --expect-head sha256:… additionally asserts that the hash committed in a published manifest (ledger.hash) is an entry of the verified chain. With it, an auditor who has access to the ledger can prove a report was derived from a prefix of exactly this ledger, even after later appends.

5. Record types (body schemas) #

Dates are YYYY-MM-DD, timestamps are RFC 3339 UTC. Countries are ISO 3166-1 alpha-2 (GB not UK, GR not EL). Every body may carry a private object ({"case_ref": "...", "notes": "..."}), which is never read by publish.

5.1 gov_request: request or order from a public authority #

A request is third-country when authority_country is not in the configured home_jurisdictions (default ["CH","EU","EEA"]). This is derived at publish time, and the set used is written into report.json.

5.2 abuse_notice: notice or own-initiative detection (DSA Art. 16 / own initiative) #

5.3 subprocessor_change #

5.4 infrastructure and measures #

infrastructure (fold key zone; latest effective ≤ period end wins; status: retired removes it): effective, zone, country, city, operator, operator_country, legal_entity, legal_entity_country, services[], status (active|retired).

measures (latest effective ≤ period end wins): effective, statement, technical[], organisational[], contractual[].

5.5 Category taxonomy #

These are the DSA Transparency Database STATEMENT_CATEGORY_* constants, which the Commission aligned with the Annex I template: ANIMAL_WELFARE, CONSUMER_INFORMATION, CYBER_VIOLENCE, CYBER_VIOLENCE_AGAINST_WOMEN, DATA_PROTECTION_AND_PRIVACY_VIOLATIONS, ILLEGAL_OR_HARMFUL_SPEECH, INTELLECTUAL_PROPERTY_INFRINGEMENTS, NEGATIVE_EFFECTS_ON_CIVIC_DISCOURSE_OR_ELECTIONS, NOT_SPECIFIED_NOTICE, OTHER_VIOLATION_TC, PROTECTION_OF_MINORS, RISK_FOR_PUBLIC_SECURITY, SCAMS_AND_FRAUD, SELF_HARM, UNSAFE_AND_PROHIBITED_PRODUCTS, VIOLENCE. These are stored with the STATEMENT_CATEGORY_ prefix. Keyword sub-categories (KEYWORD_*) are out of scope for v0.1.

5.6 retraction #

{"seq": N, "reason": "..."} excludes entry N (and its amendments) from aggregation. The original entry stays in the ledger.

6. Configuration (transparencykit.json) #

{
  "provider": { "name": "Example Cloud SA", "legal_entity": "Example Cloud SA",
                "country": "CH", "contact": "transparency@example.ch",
                "website": "https://example.ch", "service_type": "provider of hosting services" },
  "home_jurisdictions": ["CH", "EU", "EEA"],
  "ledger": "sos://example-ledger/transparencykit?zone=ch-gva-2",
  "public": "sos://example-transparency/?zone=ch-gva-2",
  "public_base_url": "https://example-transparency.sos-ch-gva-2.exo.io",
  "signing_key": "transparencykit.key",
  "min_count": 3,
  "derivable_counts": "withhold",
  "dsa_qualitative": "dsa-qualitative.json",
  "ledger_retention": { "mode": "GOVERNANCE", "days": 3650 }
}

derivable_counts (withhold, the default, or publish) controls breakdowns whose banded cells could be calculated (§7.2). dsa_qualitative names a JSON file with the DSA qualitative answers, keys own_initiative, automated_means, training and other; they fill dsa/tab11_qualitative.csv.

Store URL schemes:

Credentials are never read from the config file. Each store looks up, in order:

1. a per-store pair: TRANSPARENCYKIT_LEDGER_ACCESS_KEY / TRANSPARENCYKIT_LEDGER_SECRET_KEY for the ledger, TRANSPARENCYKIT_PUBLIC_ACCESS_KEY / TRANSPARENCYKIT_PUBLIC_SECRET_KEY for the public store;
2. EXOSCALE_API_KEY / EXOSCALE_API_SECRET;
3. AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY.

One publish run therefore can use two bucket-scoped keys. Local file:// stores create the ledger with mode 0700/0600 and public files with 0755/0644, and refuse to read or write through symbolic links.

7. Publication #

transparencykit publish --period 2026 | 2026-H1 | 2026-H2 | 2026-01-01..2026-03-31 [--as-of date]

7.1 Aggregation rules #

• An entry is in the period when its received_at / effective falls in it (inclusive dates, UTC).
• Amendments are applied, then retractions.
• Counts cover the whole period. State-like types (subprocessor_change, infrastructure, measures) are folded up to the as-of date: the earlier of the publication date and the period end, or --as-of. A full-year report published in October therefore shows the subprocessors in force in October; changes recorded with a later effective date are listed as upcoming (announced) and appear in the Atom feed. period.as_of in report.json and as_of in the Data Act and subprocessor files record the date used.
• Medians use the lower median for even counts and are expressed in hours with 1 decimal. Pairs missing an end timestamp are excluded and counted as pending.

7.2 Small-count banding (privacy) #

In index.html and report.json, a breakdown cell whose count c satisfies 0 < c < min_count is published as the string "1-<min_count-1>". Zero is always published exactly, so explicit "0 third-country disclosures" claims stay possible. Overall totals are exact.

Banding alone does not hide a count that can be calculated from other published numbers. publish therefore applies these rules:

• Breakdowns that add up to an exact total. If a single cell is banded, total minus the other cells gives its value. With several banded cells, their sum is known and each lies in [1, min_count-1]; a cell is determined when only one value fits. Such a breakdown is withheld (published as null, listed in report.json withheld with the cells it would reveal). With "derivable_counts": "publish" it is published and still listed, as a warning.
• Country rows. A row shows its exact total, disclosed and rejected; other outcomes are not shown. If the published cells leave no room for a banded value, the row's outcome cells are replaced by "withheld".
• Disclosure rate. The rate is computed over every assignment of the banded outcome cells consistent with the published total. One possible value is published as a number (it reveals nothing new); otherwise the range "lo–hi" is published. No rate is published when the outcome breakdown is withheld.
• Medians over fewer than min_count observations are omitted.

Residual risk: cross-table inference (combining, say, the country table with the authority-type breakdown, or a report with an earlier version or another period) is not checked. Choose min_count and the breakdown granularity accordingly, and review the withheld list of a draft (§7.5). DSA CSVs are never banded, because the DSA requires exact figures.

7.3 Output layout (public store) #

transparencykit.pub                         Ed25519 public key (PEM), convenience copy only
latest.json                         unsigned pointer, see below
index.html                          list of all published reports (convenience)
reports/<label>/v<N>/
    index.html                      human-readable report
    report.json                     schema transparencykit/report/v1
    data-act-art28.html|json        as of the as-of date (§7.1)
    subprocessors.json              current list as of the as-of date, history, upcoming
    subprocessors.atom              change history including announced changes
    dsa/tab1_identification.csv … tab11_qualitative.csv
    manifest.json                   signed manifest
    manifest.sig                    base64(Ed25519(manifest.json bytes))
latest/…                            byte-identical copy of the publication latest.json names

latest.json holds label, version, manifest, manifest_sha256 and period_end for the publication with the latest period end. It also holds chain_head ({path, sha256}), the most recently published manifest of any period. Publishing an older period (e.g. a correction to 2025) extends the chain but does not move latest/.

Before linking, publish walks the chain from chain_head, re-hashing every manifest, and lists all signed version folders (manifest.sig present):

• A signed publication outside the chain whose previous is the current head and whose signature verifies with the signing key was published after latest.json was last written (an interrupted run, or an older latest.json put back). It is linked back in as the new head, repeatedly, so the next publication continues the real history.
• Any other signed publication outside the chain makes publish refuse: publishing would fork history. The operator restores latest.json (OPERATIONS.md).

Versions are immutable: republishing a period creates v<N+1>. This satisfies the DSA rule that report versions be explicitly marked. Files are uploaded first, then manifest.json, then manifest.sig. A folder without manifest.sig is an abandoned upload: the next run takes the next free version number for the label (folders without a signature count as used), logs the abandoned folder and leaves it in place; the root index lists only signed versions. Abandoned folders are safe to delete from the public bucket.

7.4 Manifest #

{
  "schema": "transparencykit/manifest/v1",
  "provider": "Example Cloud SA",
  "label": "2026", "version": 1, "sequence": 7,
  "period": { "start": "2026-01-01", "end": "2026-12-31" },
  "generated_at": "2027-02-10T09:00:00Z",
  "ledger": { "seq": 1234, "hash": "sha256:…" },
  "previous": { "path": "reports/2025/v2/manifest.json", "sha256": "…" },
  "key_id": "ed25519:<16 hex of SHA-256(public key DER)>",
  "files": [ { "path": "index.html", "sha256": "…", "size": 12345 } ]
}

previous links to the chain head from before this publication, which forms a hash chain over all publications in publication order. sequence numbers publications 1, 2, 3… in that order, so publication n must be the n-th from the start of the history it links to.

7.5 Drafts and review #

publish --period P --draft dir renders every file into a local directory together with draft.json (label, version, sequence, as-of date, generation time, ledger head, previous link, key id, file hashes). Nothing is signed or uploaded. Reviewers read index.html, check the withheld list and may edit the Answer column of dsa/tab11_qualitative.csv (the only editable file; questions, rows and headers must stay unchanged).

publish --from-draft dir re-renders from the ledger state recorded in the draft, even if records were appended since, compares every file byte for byte with the draft (except the answers), and then signs and uploads. It refuses when the public store has moved on (another publication since the draft) or the key differs; make a new draft in that case.

8. Public verification (transparencykit verify) #

transparencykit verify --pubkey key.pub [--pubkey old.pub] [--chain] [--checkpoint file] <base-url | store-url> [reports/<label>/v<N>]

1. Without a path, read latest.json and verify the publication it names, requiring its manifest hash to match manifest_sha256. Then verify latest/manifest.json and every latest/ file against the same manifest bytes, so the stable URLs readers open are covered.
2. For each manifest: fetch manifest.json and manifest.sig, pick the trusted key whose key id matches key_id, and verify the Ed25519 signature. Keys are given with --pubkey (repeatable, and one file may hold several PEM blocks) and must come out-of-band. The key served next to the reports is only a convenience, and using it emits a warning.
3. Fetch every listed file and compare SHA-256 and size.
4. With --chain, start at chain_head (the newest publication of any period, including back-filled ones), follow previous links to the first publication, verify each manifest and its SHA-256 against the link, check that every sequence matches its position, and check that the displayed publication is part of that history.
5. With --checkpoint file (implies --chain), every publication recorded in the file by earlier runs must still be in the history with the same hash; the publications seen now are added.

Exit code 0 = verified; 1 = verification failure; 2 = usage or I/O error.

Limits. Everything a verifier fetches comes from the provider. Without a checkpoint, a provider (or an attacker with the signing key and write access) can present a different but self-consistent history to a new verifier. Checkpoints detect a rewrite only for verifiers that saw the earlier state. Detecting split views across verifiers needs external witnesses or a public transparency log (roadmap).

9. CLI #

10. Security model #

Out of scope: a provider that never records a request in the first place. TransparencyKit makes recorded history tamper-evident; it cannot prove completeness. Witness co-signing and external auditors address this partially.

11. Roadmap #

• v0.2: ledger verify --versions; authenticated key rotation; canary.json; XLSX export; official-template validation.
• v0.3: publish manifest checksums to a public transparency log (Sigsum or Tessera with C2SP witnesses).
• v0.4: importers (ticketing CSV, Lumen export); multi-signer approval (two-person rule).

12. Acceptance criteria (v0.1) #

1. A tampered, deleted or reordered ledger entry makes ledger verify exit non-zero, naming the seq.
2. publish output passes verify over HTTP. Modifying any published byte makes it fail.
3. Republishing a period creates v2, whose manifest links v1 by hash; verify --chain passes. A tampered latest/ file or a corrupted back-filled publication fails the default check.
4. Counts 1..min_count−1 never appear in HTML/JSON breakdowns and cannot be calculated from a breakdown and its total (§7.2); zeros do appear; DSA CSVs are exact.
5. private fields never appear in any published file.
6. The S3 client signs requests correctly (AWS SigV4 reference vectors) and works against an S3-compatible endpoint, including pagination and If-None-Match conflicts.
7. CGO_ENABLED=0 go build produces a static binary with no third-party modules in go.mod.