Kysely-based DB abstraction over SQLite + Postgres (#8) #33

Merged

james merged 3 commits from 8-db-abstraction into main

2026-06-12 21:57:17 +00:00

james commented

2026-06-12 21:55:28 +00:00

Owner

Closes #8.

This branch also introduces the ADR process (concurrent work, but rolled in so ADR-0003 doesn't reference files that don't exist on main).

Three commits

Add Kysely-based DB abstraction over SQLite + Postgres (#8) — the DB abstraction itself plus ADR-0003.
Adopt Architecture Decision Records — the ADR process foundation: ADR-0001 (format + conventions), ADR-0002 (dual-engine commitment), CLAUDE.md callout, docs/adr/README.md index.
Add at-rest encryption guidance doc — crypto.md: envelope-encryption pattern for future sensitive-data features. Not implementing, just recording the approach.

Summary of #8

Query builder: Kysely. One TypeScript interface describes the schema; the query builder emits dialect-specific SQL at runtime. Drizzle would require parallel pg-core and sqlite-core schemas — the failure mode ADR-0002's abstraction exists to prevent. Reasoning + known gap (no migration codegen) in ADR-0003.
SQLite driver: @libsql/client. better-sqlite3 doesn't build on Node 26 (the C++ trips -Wimplicit-fallthrough); libsql ships prebuilt binaries and has an official Kysely dialect. Driver choice is reversible — it sits behind the Kysely dialect interface — so no ADR.
Postgres driver: pg.
Connection: one env var DATABASE_URL. Protocol scheme picks the driver (sqlite: / postgres:). Defaults to sqlite:./data/carol.db so zero-config self-host works. SQLite parent directory auto-created on first connect.
Migrations: hardcoded TypeScript registry in db/migrator.ts (not filesystem-glob, so the production bundle works). Applied on Next.js startup via instrumentation.ts. TEXT primary keys via UUID dodge the INTEGER/SERIAL autoincrement portability problem entirely.
Entities vs DTOs are separated structurally. db/entities/ holds snake_case DB row types (Selectable/Insertable/Updateable). lib/dto/ holds camelCase API shapes plus parser/validator. Example entity + repository + DTO trio is the reference pattern in db/README.md.
Test harness: describePerEngine(suite, body) in tests/db/_engines.ts. SQLite (in-process :memory:) always runs. Postgres runs when TEST_POSTGRES_URL is set and skips cleanly otherwise.
Webpack/native bindings: marked @libsql, libsql, and pg as serverExternalPackages in next.config.mjs so they're loaded at runtime instead of statically bundled.

Test plan

npm run typecheck — clean
npm run lint — clean
npm run build — clean (Compiled successfully in 3.7s)
npm test (SQLite only, no TEST_POSTGRES_URL) — 13 pass, 5 skip
npm test with Postgres 16 via podman — 18/18 pass on both engines
CI matrix once #13 lands

Known dep audit findings — not addressed here

npm audit reports 8 vulnerabilities (1 critical, 2 high, 5 moderate) in the transitive dep tree. Same as for #7 — out of scope, this is #14 (CI security scanning) territory. Flagging so it isn't overlooked.

Closes #8. This branch also introduces the ADR process (concurrent work, but rolled in so ADR-0003 doesn't reference files that don't exist on `main`). ## Three commits 1. **Add Kysely-based DB abstraction over SQLite + Postgres (#8)** — the DB abstraction itself plus ADR-0003. 2. **Adopt Architecture Decision Records** — the ADR process foundation: ADR-0001 (format + conventions), ADR-0002 (dual-engine commitment), `CLAUDE.md` callout, `docs/adr/README.md` index. 3. **Add at-rest encryption guidance doc** — `crypto.md`: envelope-encryption pattern for future sensitive-data features. Not implementing, just recording the approach. ## Summary of #8 - **Query builder: Kysely.** One TypeScript interface describes the schema; the query builder emits dialect-specific SQL at runtime. Drizzle would require parallel `pg-core` and `sqlite-core` schemas — the failure mode ADR-0002's abstraction exists to prevent. Reasoning + known gap (no migration codegen) in [ADR-0003](docs/adr/0003-kysely-as-query-builder.md). - **SQLite driver: `@libsql/client`.** `better-sqlite3` doesn't build on Node 26 (the C++ trips `-Wimplicit-fallthrough`); libsql ships prebuilt binaries and has an official Kysely dialect. Driver choice is reversible — it sits behind the Kysely dialect interface — so no ADR. - **Postgres driver: `pg`.** - **Connection: one env var `DATABASE_URL`.** Protocol scheme picks the driver (`sqlite:` / `postgres:`). Defaults to `sqlite:./data/carol.db` so zero-config self-host works. SQLite parent directory auto-created on first connect. - **Migrations: hardcoded TypeScript registry** in `db/migrator.ts` (not filesystem-glob, so the production bundle works). Applied on Next.js startup via `instrumentation.ts`. TEXT primary keys via UUID dodge the INTEGER/SERIAL autoincrement portability problem entirely. - **Entities vs DTOs are separated structurally.** `db/entities/` holds snake_case DB row types (`Selectable`/`Insertable`/`Updateable`). `lib/dto/` holds camelCase API shapes plus parser/validator. Example entity + repository + DTO trio is the reference pattern in `db/README.md`. - **Test harness: `describePerEngine(suite, body)`** in `tests/db/_engines.ts`. SQLite (in-process `:memory:`) always runs. Postgres runs when `TEST_POSTGRES_URL` is set and skips cleanly otherwise. - **Webpack/native bindings:** marked `@libsql`, `libsql`, and `pg` as `serverExternalPackages` in `next.config.mjs` so they're loaded at runtime instead of statically bundled. ## Test plan - [x] `npm run typecheck` — clean - [x] `npm run lint` — clean - [x] `npm run build` — clean (`Compiled successfully in 3.7s`) - [x] `npm test` (SQLite only, no `TEST_POSTGRES_URL`) — 13 pass, 5 skip - [x] `npm test` with Postgres 16 via podman — **18/18 pass on both engines** - [ ] CI matrix once #13 lands ## Known dep audit findings — not addressed here `npm audit` reports 8 vulnerabilities (1 critical, 2 high, 5 moderate) in the transitive dep tree. Same as for #7 — out of scope, this is #14 (CI security scanning) territory. Flagging so it isn't overlooked.

james added 3 commits

2026-06-12 21:55:28 +00:00

Add Kysely-based DB abstraction over SQLite + Postgres (#8 ) b402709579

Kysely over Drizzle, with the reasoning recorded in ADR-0003. Short
version: dual-engine parity is a load-bearing invariant per ADR-0002,
and Kysely's single-schema model enforces it structurally where
Drizzle's parallel-per-engine schemas leave drift as a discipline
problem. The known gap — Kysely has no schema-diff codegen for
migrations — is acceptable at Carol's scale and called out
explicitly in ADR-0003's Consequences.

SQLite driver: @libsql/client over better-sqlite3. better-sqlite3
won't build on Node 26 (the bundled C++ trips
-Wimplicit-fallthrough); libsql ships prebuilt binaries and has an
official Kysely dialect. Driver choice is reversible — it sits
behind the Kysely dialect interface — so it didn't warrant its own
ADR.

Connection: one env var, DATABASE_URL. Protocol scheme picks the
driver (sqlite: / postgres:). Defaults to sqlite:./data/carol.db so a
zero-config self-host works. SQLite parent directory is created on
first connect.

Migrations: hardcoded TypeScript registry in db/migrator.ts (not
filesystem-glob, so the production bundle works). Applied on Next.js
startup via instrumentation.ts. TEXT primary keys via UUID dodge the
INTEGER/SERIAL autoincrement portability problem entirely.

Entities vs DTOs: db/entities/ holds snake_case DB row types
(Selectable/Insertable/Updateable). lib/dto/ holds camelCase API
shapes plus a parser/validator. The example entity + repository +
DTO trio is the reference pattern documented in db/README.md.

Test harness: describePerEngine(suite, body) in tests/db/_engines.ts.
SQLite (in-process :memory:) always runs. Postgres runs when
TEST_POSTGRES_URL is set and skips cleanly otherwise. Verified
locally against postgres:16 — 18/18 tests pass on both engines.

Build: @libsql, libsql, and pg ship native bindings webpack can't
bundle; marked them as serverExternalPackages in next.config.mjs so
they're loaded at runtime instead.

Note: The ADR process foundation (CLAUDE.md edit + ADR-0001 + ADR-0002
+ docs/adr/README.md) is being added concurrently in a separate
commit. Once that lands, docs/adr/README.md needs one line added for
ADR-0003.

Closes #8

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Adopt Architecture Decision Records 4ace80e198

Introduces a lightweight ADR process for capturing the *why* behind
load-bearing technical choices, separate from CLAUDE.md's *what*.

- ADR-0001 establishes the format (Michael Nygard) and conventions:
  immutable once accepted, superseded rather than edited, indexed in
  docs/adr/README.md.
- ADR-0002 records the dual SQLite/Postgres commitment, the
  abstraction requirement, and the rejected alternatives (Postgres-
  only, SQLite-only, MariaDB, defer-the-abstraction).
- CLAUDE.md gains a pointer to docs/adr/, with the rule that ADRs win
  when state and reasoning disagree.

ADR-0003 already landed in the previous commit alongside the DB
abstraction it documents.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Add at-rest encryption guidance doc 2a24897151

crypto.md captures the envelope-encryption pattern for any future
feature that stores sensitive user data. The threat model assumes the
DB (and its backups) can be read by someone who is not the user —
including the instance operator. This isn't implementing encryption,
just recording the approach so the first feature that needs it has a
starting point rather than relitigating the threat model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

james merged commit d39c64d507 into main

2026-06-12 21:57:17 +00:00

james referenced this pull request from a commit

2026-06-12 21:57:18 +00:00

Merge pull request 'Kysely-based DB abstraction over SQLite + Postgres (#8)' (#33) from 8-db-abstraction into main