james/carol

Fork 0

Publish cosign signatures to Sigstore public-good Rekor #81

New issue

Closed

opened 2026-06-17 17:58:13 +00:00 by james · 1 comment

james commented

2026-06-17 17:58:13 +00:00

Owner

First successful release build (after #79, #76, #16 all landed) produces a signed image, but the verification command from the release notes fails:

Error: no matching signatures: no known key found for this signature in database

That message comes from cosign querying Sigstore's public-good Rekor transparency log and finding no entry matching the signature's public key + image digest. Most likely the sign step in our workflow isn't actually uploading to Rekor — cosign 2.5.x changed the default for --tlog-upload on keyed signing, and we're relying on the (now-different) default.

Decision

Publish to Rekor explicitly. The verification command stays the default-shape (cosign verify --key ... <image>@<digest>); we just need to make sure the sign step actually uploads the entry that verify is going to look for.

This gives us:

A public, append-only attestation that the signing event happened at time T.
Verification that works with no special flags — a self-hoster's verify command is the canonical cosign shape.
A small but real defense against silent key compromise: if the key leaks and someone signs a different image with it, the original (legitimate) Rekor entry stays visible alongside the malicious one.

Trade-off: a public log entry per release reveals the existence and timing of every release to anyone watching https://rekor.sigstore.dev. For an open-source project that is acceptable (everything is public anyway). The runtime dependency on Sigstore's public infrastructure is a real one; if rekor.sigstore.dev is down during a release, the sign step fails the release. The mitigation if that becomes painful is self-hosting Rekor — explicitly out of scope (see "Out of scope" below).

Scope

Add --tlog-upload=true explicitly to cosign sign and cosign attest in .forgejo/workflows/release.yml. Today's defaults may already do this in some cosign versions, but the explicit flag survives future default flips and documents intent.
Leave the release-notes verify commands at the canonical shape (no --insecure-ignore-tlog, no --rekor-url override). Default verify is what self-hosters will type.
Update docs/ci.md "Release pipeline → Verifying a published image" with a one-paragraph note explaining that signatures are published to Sigstore's public Rekor and what that means for the trust chain (timestamping + reproducibility check).
Update ADR-0014 Consequences to capture both the property gained (public timestamping) and the runtime dependency taken on (Sigstore public-good infrastructure).

Acceptance criteria

Next tag push (v0.0.1-rc.2 or similar) produces an image whose signature shows up in Rekor — cosign tree forge.wynning.tech/james/carol@<digest> reports both the signature and the SLSA attestation, each with a Rekor entry.
The verify command from the release notes (no extra flags) succeeds with Verified OK on both cosign verify and cosign verify-attestation.
docs/ci.md and ADR-0014 reflect the explicit Rekor dependency.

Out of scope

Self-hosting Rekor (Trillian + rekor-server + persistent storage + TLS + monitoring — multi-week infra for a personal project; revisit only if Sigstore public-good becomes unacceptable for trust or availability reasons).
Opting out of Rekor (--tlog-upload=false / --insecure-ignore-tlog) — considered and rejected in favor of the timestamping property.

Part of epic #2. Follow-up from #16.

First successful release build (after #79, #76, #16 all landed) produces a signed image, but the verification command from the release notes fails: ``` Error: no matching signatures: no known key found for this signature in database ``` That message comes from cosign querying Sigstore's public-good [Rekor](https://docs.sigstore.dev/logging/overview/) transparency log and finding no entry matching the signature's public key + image digest. Most likely the sign step in our workflow isn't actually uploading to Rekor — cosign 2.5.x changed the default for `--tlog-upload` on keyed signing, and we're relying on the (now-different) default. ## Decision Publish to Rekor explicitly. The verification command stays the default-shape (`cosign verify --key ... <image>@<digest>`); we just need to make sure the sign step actually uploads the entry that verify is going to look for. This gives us: - A public, append-only attestation that the signing event happened at time T. - Verification that works with no special flags — a self-hoster's verify command is the canonical cosign shape. - A small but real defense against silent key compromise: if the key leaks and someone signs a different image with it, the original (legitimate) Rekor entry stays visible alongside the malicious one. Trade-off: a public log entry per release reveals the existence and timing of every release to anyone watching `https://rekor.sigstore.dev`. For an open-source project that is acceptable (everything is public anyway). The runtime dependency on Sigstore's public infrastructure is a real one; if `rekor.sigstore.dev` is down during a release, the sign step fails the release. The mitigation if that becomes painful is self-hosting Rekor — explicitly out of scope (see "Out of scope" below). ## Scope - Add `--tlog-upload=true` explicitly to `cosign sign` and `cosign attest` in `.forgejo/workflows/release.yml`. Today's defaults may already do this in some cosign versions, but the explicit flag survives future default flips and documents intent. - Leave the release-notes verify commands at the canonical shape (no `--insecure-ignore-tlog`, no `--rekor-url` override). Default verify is what self-hosters will type. - Update `docs/ci.md` "Release pipeline → Verifying a published image" with a one-paragraph note explaining that signatures are published to Sigstore's public Rekor and what that means for the trust chain (timestamping + reproducibility check). - Update ADR-0014 **Consequences** to capture both the property gained (public timestamping) and the runtime dependency taken on (Sigstore public-good infrastructure). ## Acceptance criteria - [ ] Next tag push (`v0.0.1-rc.2` or similar) produces an image whose signature shows up in Rekor — `cosign tree forge.wynning.tech/james/carol@<digest>` reports both the signature and the SLSA attestation, each with a Rekor entry. - [ ] The verify command from the release notes (no extra flags) succeeds with `Verified OK` on both `cosign verify` and `cosign verify-attestation`. - [ ] `docs/ci.md` and ADR-0014 reflect the explicit Rekor dependency. ## Out of scope - Self-hosting Rekor (Trillian + rekor-server + persistent storage + TLS + monitoring — multi-week infra for a personal project; revisit only if Sigstore public-good becomes unacceptable for trust or availability reasons). - Opting out of Rekor (`--tlog-upload=false` / `--insecure-ignore-tlog`) — considered and rejected in favor of the timestamping property. Part of epic #2. Follow-up from #16.

james changed title from ~~cosign verify fails against Rekor: opt out of the transparency log~~ to Publish cosign signatures to Sigstore public-good Rekor

2026-06-17 19:36:59 +00:00

james referenced this issue from a commit

2026-06-17 19:41:05 +00:00

fix(release): explicitly publish cosign sigs to Rekor (#81)

james added the

area:ci

label

2026-06-17 19:41:19 +00:00

james referenced this issue from a pull request that will close it,

2026-06-17 19:41:46 +00:00

fix(release): publish cosign sigs to Sigstore Rekor (#81) #82

james closed this issue

2026-06-17 19:46:18 +00:00

james referenced this issue from a commit

2026-06-17 19:46:18 +00:00

Merge pull request 'fix(release): publish cosign sigs to Sigstore Rekor (#81)' (#82) from 81-cosign-no-rekor into main

james referenced this issue

2026-06-17 21:07:18 +00:00

Document cosign.pub URL must be anonymously fetchable; clarify "verified offline" Rekor behaviour #83

james referenced this issue from a commit

2026-06-17 21:43:46 +00:00

docs(release): cosign.pub URL must be anonymously fetchable + clarify verify "offline" (#83)

james referenced this issue

2026-06-17 21:47:50 +00:00

docs(release): cosign.pub URL must be anonymously fetchable + clarify verify "offline" (#83) #84

james commented

2026-06-17 21:48:50 +00:00

Author

Owner

Root-cause correction for future readers:

The user-visible Error: no matching signatures: no known key found for this signature in database chased here was not caused by the missing --tlog-upload=true flag — it was caused by repo visibility. The repo was private at debug time, and cosign's HTTP client does an anonymous GET on --key <URL> without carrying a Forgejo session, so the URL returned a login page (HTML) instead of the PEM. The error message is what cosign emits when it can't load any usable key.

The fix made in #82 (--tlog-upload=true on sign and attest) is still a real improvement — release signatures are now genuinely uploaded to Sigstore's public-good Rekor (confirmed logIndex: 1854384861 on the rc.2 sign step), and verify reports Existence of the claims in the transparency log was verified offline against the embedded SignedEntryTimestamp bundle. So this issue's outcome was correct, just for a different reason than the symptom.

The missing prerequisite (cosign.pub URL must be anonymously fetchable) and a troubleshooting entry for the misleading error message are documented in #83 / PR #84.

Root-cause correction for future readers: The user-visible `Error: no matching signatures: no known key found for this signature in database` chased here was **not** caused by the missing `--tlog-upload=true` flag — it was caused by repo visibility. The repo was private at debug time, and cosign's HTTP client does an anonymous GET on `--key <URL>` without carrying a Forgejo session, so the URL returned a login page (HTML) instead of the PEM. The error message is what cosign emits when it can't load any usable key. The fix made in #82 (`--tlog-upload=true` on sign and attest) is still a real improvement — release signatures are now genuinely uploaded to Sigstore's public-good Rekor (confirmed `logIndex: 1854384861` on the rc.2 sign step), and verify reports `Existence of the claims in the transparency log was verified offline` against the embedded `SignedEntryTimestamp` bundle. So this issue's outcome was correct, just for a different reason than the symptom. The missing prerequisite (cosign.pub URL must be anonymously fetchable) and a troubleshooting entry for the misleading error message are documented in #83 / PR #84.