Persist material consumption snapshots

This commit is contained in:
2026-09-05 11:35:25 +02:00
parent 7c029759f5
commit ae13c3c35a
9 changed files with 303 additions and 17 deletions
+43 -6
View File
@@ -9,8 +9,8 @@ production-order context, and calculation state.
This repository includes an ENLYZE exploration CLI, a production-run/timeseries
gateway, and a configured continuous material polling runner with atomic JSON
checkpoints. The live material-consumption MVP remains in progress: Grafana-oriented
derived-total persistence/exposure and database migrations are still pending.
checkpoints and PostgreSQL cumulative material-consumption snapshots for Grafana.
The live material-consumption MVP remains in progress.
There are no HTTP endpoints.
## Intended flow
@@ -174,8 +174,46 @@ calculation parameters or running a different calculation for the same machine
and order: checkpoints are not namespaced by calculation id/version. These files
are integration checkpoints, not a Grafana metric store.
This runner exposes no Grafana/TimescaleDB metric yet. Grafana-oriented derived-total
persistence/exposure remains the next milestone; the MVP is not complete.
Each successful non-empty poll writes one cumulative snapshot to PostgreSQL before
printing its result. JSON remains the restart checkpoint; PostgreSQL stores derived
time-series snapshots only. ENLYZE remains the raw-data source of truth. Database
write failures emit a concise error and polling continues with the next cycle;
missed snapshots are not retried or backfilled and JSON state is not rolled back.
The next successful snapshot includes the continuing cumulative total.
A new order's first snapshot is **not guaranteed to be zero**: the service processes
available samples from the run start before returning, so its first total may
already be non-zero. A returned zero is stored normally. Totals continue across
runs for the same order; existing checkpoints for a previously seen order resume.
### PostgreSQL setup
Set all five runtime settings: `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_DB`,
`POSTGRES_USER`, and `POSTGRES_PASSWORD`, using exported environment variables or
the existing secrets file (file values override the environment). The runner does
not automatically load `.env`. Missing or invalid settings fail at startup;
connectivity and schema errors are reported during polling. For the existing
Compose instance, use host `localhost`, the published port (default `5432`), and
`production_analytics` for both database and user. Set the same password in
Compose's `.env` and the runner environment/secrets file.
Start the database and apply the repeatable schema from the repository root:
```bash
docker compose up -d timescaledb
docker compose exec -T timescaledb psql -U production_analytics -d production_analytics \
-v ON_ERROR_STOP=1 < db/schema.sql
```
Wait until PostgreSQL is ready before applying the schema. Configure Grafana's
PostgreSQL data source to query `material_consumption_snapshots`, selecting
`timestamp` as time and `consumption_kg` as the cumulative value, filtered by
`calculation_id`, `machine_id`, and `production_order`. These are ordinary
PostgreSQL tables/indexes; no Timescale-specific features or hypertables are used.
The primary key deduplicates calculation/machine/order/timestamp (first write wins).
A short transaction opens and closes one synchronous connection per snapshot,
with 10-second connection and statement timeouts. Snapshot timestamps are poll
start times, not source-sample timestamps. Schema application is manual.
Bento 1 bentonite source and gate selection remain intentionally undefined pending
process validation; the generic integration core is unchanged and reusable.
@@ -215,8 +253,7 @@ completed peaks, including a legitimate approximately 1195 kg cycle. Run
available. Neither capture should be added to version control or automated
tests.
`docker compose up -d timescaledb` is an optional local database design for a
future persistence milestone. It is not required for the bootstrap tests.
The normal unit test suite mocks PostgreSQL and requires no live database.
See [PROJECT_KNOWLEDGE.md](PROJECT_KNOWLEDGE.md) for durable project context
and [docs/roadmap.md](docs/roadmap.md) for the implementation sequence.