Persist material consumption snapshots
This commit is contained in:
@@ -9,8 +9,8 @@ production-order context, and calculation state.
|
||||
|
||||
This repository includes an ENLYZE exploration CLI, a production-run/timeseries
|
||||
gateway, and a configured continuous material polling runner with atomic JSON
|
||||
checkpoints. The live material-consumption MVP remains in progress: Grafana-oriented
|
||||
derived-total persistence/exposure and database migrations are still pending.
|
||||
checkpoints and PostgreSQL cumulative material-consumption snapshots for Grafana.
|
||||
The live material-consumption MVP remains in progress.
|
||||
There are no HTTP endpoints.
|
||||
|
||||
## Intended flow
|
||||
@@ -174,8 +174,46 @@ calculation parameters or running a different calculation for the same machine
|
||||
and order: checkpoints are not namespaced by calculation id/version. These files
|
||||
are integration checkpoints, not a Grafana metric store.
|
||||
|
||||
This runner exposes no Grafana/TimescaleDB metric yet. Grafana-oriented derived-total
|
||||
persistence/exposure remains the next milestone; the MVP is not complete.
|
||||
Each successful non-empty poll writes one cumulative snapshot to PostgreSQL before
|
||||
printing its result. JSON remains the restart checkpoint; PostgreSQL stores derived
|
||||
time-series snapshots only. ENLYZE remains the raw-data source of truth. Database
|
||||
write failures emit a concise error and polling continues with the next cycle;
|
||||
missed snapshots are not retried or backfilled and JSON state is not rolled back.
|
||||
The next successful snapshot includes the continuing cumulative total.
|
||||
|
||||
A new order's first snapshot is **not guaranteed to be zero**: the service processes
|
||||
available samples from the run start before returning, so its first total may
|
||||
already be non-zero. A returned zero is stored normally. Totals continue across
|
||||
runs for the same order; existing checkpoints for a previously seen order resume.
|
||||
|
||||
### PostgreSQL setup
|
||||
|
||||
Set all five runtime settings: `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_DB`,
|
||||
`POSTGRES_USER`, and `POSTGRES_PASSWORD`, using exported environment variables or
|
||||
the existing secrets file (file values override the environment). The runner does
|
||||
not automatically load `.env`. Missing or invalid settings fail at startup;
|
||||
connectivity and schema errors are reported during polling. For the existing
|
||||
Compose instance, use host `localhost`, the published port (default `5432`), and
|
||||
`production_analytics` for both database and user. Set the same password in
|
||||
Compose's `.env` and the runner environment/secrets file.
|
||||
|
||||
Start the database and apply the repeatable schema from the repository root:
|
||||
|
||||
```bash
|
||||
docker compose up -d timescaledb
|
||||
docker compose exec -T timescaledb psql -U production_analytics -d production_analytics \
|
||||
-v ON_ERROR_STOP=1 < db/schema.sql
|
||||
```
|
||||
|
||||
Wait until PostgreSQL is ready before applying the schema. Configure Grafana's
|
||||
PostgreSQL data source to query `material_consumption_snapshots`, selecting
|
||||
`timestamp` as time and `consumption_kg` as the cumulative value, filtered by
|
||||
`calculation_id`, `machine_id`, and `production_order`. These are ordinary
|
||||
PostgreSQL tables/indexes; no Timescale-specific features or hypertables are used.
|
||||
The primary key deduplicates calculation/machine/order/timestamp (first write wins).
|
||||
A short transaction opens and closes one synchronous connection per snapshot,
|
||||
with 10-second connection and statement timeouts. Snapshot timestamps are poll
|
||||
start times, not source-sample timestamps. Schema application is manual.
|
||||
Bento 1 bentonite source and gate selection remain intentionally undefined pending
|
||||
process validation; the generic integration core is unchanged and reusable.
|
||||
|
||||
@@ -215,8 +253,7 @@ completed peaks, including a legitimate approximately 1195 kg cycle. Run
|
||||
available. Neither capture should be added to version control or automated
|
||||
tests.
|
||||
|
||||
`docker compose up -d timescaledb` is an optional local database design for a
|
||||
future persistence milestone. It is not required for the bootstrap tests.
|
||||
The normal unit test suite mocks PostgreSQL and requires no live database.
|
||||
|
||||
See [PROJECT_KNOWLEDGE.md](PROJECT_KNOWLEDGE.md) for durable project context
|
||||
and [docs/roadmap.md](docs/roadmap.md) for the implementation sequence.
|
||||
|
||||
Reference in New Issue
Block a user