Compare commits
18
Commits
363be0a47e
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ea940ad0fa | ||
|
|
640ec53330 | ||
|
|
5452dc035b | ||
|
|
e9f9cf6f3c | ||
|
|
4a3ed625ce | ||
|
|
73a6e60450 | ||
|
|
0210ced9e3 | ||
|
|
be8d76507a | ||
|
|
ed0e20a04b | ||
|
|
c411581a1a | ||
|
|
0856629e77 | ||
|
|
9fea9cb6ba | ||
|
|
b89d57d1c1 | ||
|
|
ae13c3c35a | ||
|
|
7c029759f5 | ||
|
|
6ddb76fd03 | ||
|
|
efc563512f | ||
|
|
98a92c3739 |
@@ -29,3 +29,6 @@ secrets/*
|
||||
.DS_Store
|
||||
|
||||
activate.sh
|
||||
|
||||
# Live polling checkpoints
|
||||
data/state/
|
||||
|
||||
@@ -158,3 +158,16 @@ The exploration deliverable is verified API observations and sanitized
|
||||
fixtures. The next product deliverable is live incremental material
|
||||
integration, using a generic material-rate integrator rather than a separate
|
||||
historical-only calculation path.
|
||||
|
||||
## Power-meter pipeline
|
||||
|
||||
The generic `power_meter` package collects configured ENLYZE meters into
|
||||
TimescaleDB/PostgreSQL. The B2 example uses only verified UUIDs for machine,
|
||||
energy total, active power total and grid frequency. Power-outage counting is
|
||||
supported by schema/repository but remains unset until the real UUID is present
|
||||
in local raw exploration data. Counter decreases start a new epoch and record
|
||||
the new counter value as the increment; the first observation establishes a
|
||||
baseline with zero increment. Optional phase power is stored as JSONB and does
|
||||
not require L2. `report monthly` reads the database directly, writes
|
||||
`<mount>/<meter>/<year>/<meter>_Powermeter_<YYYY-MM>.pdf`, and persists report
|
||||
metadata independently of Grafana.
|
||||
|
||||
@@ -7,8 +7,11 @@ production-order context, and calculation state.
|
||||
|
||||
## Status
|
||||
|
||||
This repository includes a read-only ENLYZE API exploration CLI. It contains
|
||||
no verified ENLYZE operation wrappers, database migrations, or HTTP endpoints.
|
||||
This repository includes an ENLYZE exploration CLI, a production-run/timeseries
|
||||
gateway, and a configured continuous material polling runner with atomic JSON
|
||||
checkpoints and PostgreSQL cumulative material-consumption snapshots for Grafana.
|
||||
The live material-consumption MVP remains in progress.
|
||||
There are no HTTP endpoints.
|
||||
|
||||
## Intended flow
|
||||
|
||||
@@ -53,6 +56,10 @@ The second command saves a sanitized, reviewable fixture under
|
||||
`fixtures/enlyze/` by default. Raw captures belong in the ignored
|
||||
`data/raw/enlyze/` directory and must not be committed.
|
||||
|
||||
The `/v2/variables` exploration path automatically follows cursor pagination,
|
||||
so the emitted response contains all variable pages. Non-2xx responses retain a
|
||||
bounded, sanitized response body in the error for troubleshooting.
|
||||
|
||||
The OpenAPI server URL is `https://app.enlyze.com/api/`; its operation paths
|
||||
begin with `/v2/`. Set `ENLYZE_BASE_URL` to that server URL, not to an
|
||||
operation path. The documented read-only time-series operation is exposed for
|
||||
@@ -76,7 +83,7 @@ Call `process(sample)` or `process_many(samples)` on the same instance for live
|
||||
input, replay, or successive chunks. Both use the same calculation. The immutable
|
||||
`state` snapshot exposes cumulative kg, integrated running seconds, last timestamp
|
||||
(UTC), last rate, last gate, and whether the latest gate is active. State restoration
|
||||
and persistence are not implemented yet. Naive timestamps and non-finite values
|
||||
and JSON persistence are supported. Naive timestamps and non-finite values
|
||||
are rejected; backwards timestamps raise without changing state. Duplicate
|
||||
timestamps add no consumption but replace the baseline in arrival order.
|
||||
Finite negative material rates are currently accepted and decrease cumulative
|
||||
@@ -88,13 +95,320 @@ Run `python scripts/validate_k7_material_consumption.py` from the repository roo
|
||||
with the ignored `k7-00842-throughput.raw.json` and `k7-00842-speed.raw.json`
|
||||
captures in `data/raw/enlyze/`. The utility joins common non-null timestamps
|
||||
without filling values and rejects unordered or duplicate capture records.
|
||||
K7-specific signal UUIDs are confined to the utility: `Stundenleistung Anlage`
|
||||
is gated by `Geschwindigkeit Gesamtanlage > 0.5 m/min`. With the explicit
|
||||
K7-specific signal UUIDs are declared in the runner configuration and validation
|
||||
utility: `Stundenleistung Anlage` is gated by `Geschwindigkeit Gesamtanlage > 0.5 m/min`. With the explicit
|
||||
20-second validation gap limit (`--max-sample-gap-seconds` to override),
|
||||
2,672 common samples yield **5.205555556 h** and **5,180.127150811 kg** for run
|
||||
00842, matching the previous manual calculation's rounded results. Synthetic
|
||||
tests verify equivalence to manual interval integration without local captures
|
||||
or live ENLYZE access. Production polling, Grafana, and Bento 1 are not implemented.
|
||||
or live ENLYZE access. Grafana and Bento 1 remain pending.
|
||||
|
||||
`MaterialPollingService.poll_once(now=...)` starts at the open run's start or
|
||||
resumes inclusively at its last processed timestamp. Duplicate boundary samples
|
||||
add no time. A new run UUID for the same order retains cumulative kg and running
|
||||
seconds but resets the temporal baseline; different orders and machines have
|
||||
separate state. Empty responses add no consumption. Naive `now` is rejected;
|
||||
no open run or a window beginning after `now` returns `None` without saving.
|
||||
|
||||
`JsonMaterialStateStore` uses deterministic SHA-256 filenames derived from the
|
||||
machine/order pair, preventing path traversal and sanitized-name collisions.
|
||||
It flushes and fsyncs a temporary file in the state directory before atomic
|
||||
replacement; a failed write leaves the previous primary file intact. Missing
|
||||
state returns `None`; malformed JSON or state raises `ValueError` with file
|
||||
context. Use an ignored runtime directory and one polling writer per state key;
|
||||
atomic replacement does not coordinate concurrent read-modify-write cycles.
|
||||
Files from the earlier uncommitted sanitized-filename prototype are not loaded
|
||||
under the new names. The gateway rejects ambiguous open runs, naive windows,
|
||||
and missing columns or malformed records instead of silently skipping them.
|
||||
|
||||
## Continuous material polling
|
||||
|
||||
The committed [K7 configuration](config/k7-material-consumption.yaml) declares
|
||||
`k7-fiber-consumption`, a `material_consumption` calculation at version `"1"`.
|
||||
It selects `Stundenleistung Anlage` (kg/h) as the rate and `Geschwindigkeit
|
||||
Gesamtanlage` (m/min) as the gate, integrating only when the gate is **strictly
|
||||
above 0.5**, with a maximum sample gap of 20 seconds. `Anlage läuft` is not the
|
||||
primary gate. The output is `material_consumption` in `kg`, grouped by
|
||||
`production_order`; production orders remain opaque strings, including leading
|
||||
zeros and whitespace.
|
||||
|
||||
Install the declared dependencies in the repository venv, then run from the
|
||||
repository root:
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m pip install -e '.[dev]'
|
||||
.venv/bin/python -m production_analytics.cli run material-poll \
|
||||
--config config/k7-material-consumption.yaml \
|
||||
--calculation-id k7-fiber-consumption \
|
||||
--poll-interval-seconds 10 \
|
||||
--state-directory data/state/material \
|
||||
--secrets-file secrets/enlyze.env
|
||||
```
|
||||
|
||||
The installed `production-analytics run material-poll` command is equivalent.
|
||||
The interval, state directory, and secrets path above are defaults. Paths are
|
||||
relative to the current working directory. Existing `ENLYZE_BASE_URL`,
|
||||
`ENLYZE_API_KEY`, and `ENLYZE_HTTP_TIMEOUT_SECONDS` handling is reused; values
|
||||
in the secrets file override environment values. Keys are never CLI arguments.
|
||||
|
||||
Keep calculation fields in YAML and technical runtime settings in CLI options.
|
||||
The loader requires every field shown in the K7 file, rejects unknown fields,
|
||||
duplicate keys/ids, unsupported types/versions/outputs, and invalid or non-finite
|
||||
numbers. Numeric fields must be YAML numbers, and `version` must be quoted.
|
||||
Multiple material calculations may share a file, but then `--calculation-id` is
|
||||
required. All entries are validated before selection. The unrelated
|
||||
`config/calculations.example.yaml` remains illustrative and is not executable
|
||||
by this material-only runner.
|
||||
|
||||
The foreground runner polls once immediately using UTC, then sleeps for the
|
||||
configured positive interval after each completed cycle, including failures.
|
||||
A slow poll delays the next cycle; there is no overlap or catch-up scheduling.
|
||||
Ctrl-C/SIGINT exits cleanly. Successful cycles print machine, opaque order,
|
||||
cumulative kg, integrated running seconds, and cycle-start timestamp. No open
|
||||
run (or no eligible window) is normal and visible. Errors on stderr identify
|
||||
state loading/saving or ENLYZE/gateway/polling and the exception class; arbitrary
|
||||
exception text and response bodies are withheld to avoid leaking credentials.
|
||||
Startup/configuration failures return non-zero; cycle failures retry after the
|
||||
delay without replacing checkpoints with empty state.
|
||||
|
||||
Checkpoints live in ignored `data/state/material/` by default, with one hashed
|
||||
JSON filename per machine/order pair. Run **one polling writer per state key**;
|
||||
there is no multi-process locking. Use a separate state directory when changing
|
||||
calculation parameters or running a different calculation for the same machine
|
||||
and order: checkpoints are not namespaced by calculation id/version. These files
|
||||
are integration checkpoints, not a Grafana metric store.
|
||||
|
||||
Each successful non-empty poll writes one cumulative snapshot to PostgreSQL before
|
||||
printing its result. JSON remains the restart checkpoint; PostgreSQL stores derived
|
||||
time-series snapshots only. ENLYZE remains the raw-data source of truth. Database
|
||||
write failures emit a concise error and polling continues with the next cycle;
|
||||
missed snapshots are not retried or backfilled and JSON state is not rolled back.
|
||||
The next successful snapshot includes the continuing cumulative total.
|
||||
|
||||
A new order's first snapshot is **not guaranteed to be zero**: the service processes
|
||||
available samples from the run start before returning, so its first total may
|
||||
already be non-zero. A returned zero is stored normally. Totals continue across
|
||||
runs for the same order; existing checkpoints for a previously seen order resume.
|
||||
|
||||
### PostgreSQL setup
|
||||
|
||||
Set all five runtime settings: `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_DB`,
|
||||
`POSTGRES_USER`, and `POSTGRES_PASSWORD`, using exported environment variables or
|
||||
the existing secrets file (file values override the environment). The runner does
|
||||
not automatically load `.env`. Missing or invalid settings fail at startup;
|
||||
connectivity and schema errors are reported during polling. For the existing
|
||||
Compose instance, use host `localhost`, the published port (default `5432`), and
|
||||
`production_analytics` for both database and user. Set the same password in
|
||||
Compose's `.env` and the runner environment/secrets file.
|
||||
|
||||
Start the database and apply the repeatable schema from the repository root:
|
||||
|
||||
```bash
|
||||
docker compose up -d timescaledb
|
||||
docker compose exec -T timescaledb psql -U production_analytics -d production_analytics \
|
||||
-v ON_ERROR_STOP=1 < db/schema.sql
|
||||
```
|
||||
|
||||
Wait until PostgreSQL is ready before applying the schema. Configure Grafana's
|
||||
PostgreSQL data source to query `material_consumption_snapshots`, selecting
|
||||
`timestamp` as time and `consumption_kg` as the cumulative value, filtered by
|
||||
`calculation_id`, `machine_id`, and `production_order`. These are ordinary
|
||||
PostgreSQL tables/indexes; no Timescale-specific features or hypertables are used.
|
||||
The primary key deduplicates calculation/machine/order/timestamp (first write wins).
|
||||
A short transaction opens and closes one synchronous connection per snapshot,
|
||||
with 10-second connection and statement timeouts. Snapshot timestamps are poll
|
||||
start times, not source-sample timestamps. Schema application is manual.
|
||||
Bento 1 bentonite source and gate selection remain intentionally undefined pending
|
||||
process validation; the generic integration core is unchanged and reusable.
|
||||
|
||||
## Generic ENLYZE power meters
|
||||
|
||||
`config/power-meters.example.yaml` is the configuration-only B2 starting point.
|
||||
It contains the verified machine, energy, active-power and frequency UUIDs; K1
|
||||
and K6 are added as more `meters` entries without new code. Optional phase
|
||||
channels are a mapping and may omit L2. `power_outages` is intentionally absent
|
||||
until its real UUID is verified from the raw second `/v2/variables` page/register
|
||||
1828; sanitized placeholders are never accepted.
|
||||
|
||||
Apply `db/schema.sql`, then run a bounded collection/backfill with ENLYZE and
|
||||
PostgreSQL values from the environment or secret files:
|
||||
|
||||
```bash
|
||||
production-analytics run power-meter --config config/power-meters.example.yaml \
|
||||
--meter B2 --start 2026-09-01T00:00:00Z --end 2026-10-01T00:00:00Z
|
||||
```
|
||||
|
||||
Generate the independent monthly PDF (under the configured mounted SMB path):
|
||||
|
||||
```bash
|
||||
production-analytics report monthly --config config/power-meters.example.yaml \
|
||||
--meter B2 --month 2026-09
|
||||
```
|
||||
|
||||
The collector stores raw counters plus reset-safe increments in
|
||||
`power_meter_readings`. Monthly reports preserve exact source timestamps for
|
||||
the first/last reading, consumption, outage delta, frequency extrema and their
|
||||
timestamps. PDF rendering needs WeasyPrint or `wkhtmltopdf`; report metadata is
|
||||
stored in `power_meter_monthly_reports`. Grafana query templates are in
|
||||
`db/power_meter_grafana.sql`.
|
||||
|
||||
For continuous collection, add `--follow`; without it the same command is a
|
||||
bounded collector/backfill over `--start` and `--end` (or the latest polling
|
||||
window when omitted).
|
||||
|
||||
## ERP current workplace status
|
||||
|
||||
The read-only ERP adapter reads production-order/article context from MSSQL
|
||||
`NV_DWH.dbo.GRAFANA_WORKPLACE_STATUS`, for later live material KPIs. Credentials
|
||||
belong in ignored `secrets/erp.env`, using `ERP_DB_HOST`, `ERP_DB_PORT`,
|
||||
`ERP_DB_NAME`, `ERP_DB_USER`, and `ERP_DB_PASSWORD`. The verified endpoint is
|
||||
`192.168.111.24:49601`, database `NV_DWH`, using SQL Server authentication and
|
||||
a read-only account. All settings are required; ports must be integers 1–65535.
|
||||
|
||||
```python
|
||||
from production_analytics.erp import ErpSettings, ErpWorkplaceStatusGateway
|
||||
|
||||
settings = ErpSettings.from_secret_file() # File values override environment values.
|
||||
status = ErpWorkplaceStatusGateway(settings).get_current_workplace_status("K7")
|
||||
```
|
||||
|
||||
Alternatively, use `ErpSettings.from_environment(mapping)`. The existing simple
|
||||
dotenv loader is reused; no shell content is executed. Each call opens and closes
|
||||
one connection, with 10-second login/query timeouts, and performs a parameterized
|
||||
SELECT filtered by workplace. Zero rows returns `None`, one row returns an
|
||||
immutable `CurrentWorkplaceStatus`, and multiple rows raise `ErpReadError`.
|
||||
Driver failures and malformed rows also raise `ErpReadError` with safe messages.
|
||||
|
||||
Only trailing whitespace is removed from workplace, production order, article
|
||||
number, and description; identifiers otherwise remain unchanged. Nullable fields
|
||||
stay `None`. Decimal/integer quantities are explicitly converted to finite floats
|
||||
(with normal floating-point precision); non-finite/overflowing values are rejected.
|
||||
Quantity fields use m² and remaining time uses hours, following the supplied source
|
||||
semantics. `feedback_timestamp` is preserved exactly, including a naive timezone.
|
||||
It represents the latest ERP feedback, and the ERP export is delayed relative to
|
||||
live process data; the adapter makes no freshness inference.
|
||||
|
||||
Automated ERP tests use mocks and require no live connectivity.
|
||||
|
||||
## ERP context normalization
|
||||
|
||||
Two pure helpers in `production_analytics.context` prepare context for later KPIs:
|
||||
|
||||
- `build_enlyze_production_order(erp_production_order, format_template) -> str`
|
||||
uses an explicit template configured by the caller per machine/workplace.
|
||||
The template requires exactly one literal `{production_order}` placeholder and
|
||||
no other braces; invalid templates raise `ValueError`.
|
||||
Outer ERP-order whitespace is stripped and the
|
||||
remaining order must contain only ASCII digits. Leading zeros are preserved.
|
||||
Empty/invalid orders raise `ValueError`. There is no default machine rule.
|
||||
- `extract_nominal_width_m(article_description: str | None) -> float | None`
|
||||
is generic across machines and conservatively reads a single
|
||||
`<width> x <length> m` pair, accepting decimal
|
||||
comma/point and variable whitespace. For example, `Stex R 1501 C (PR) 5,80 x 50 m`
|
||||
yields `5.8`. Earlier article numbers and trailing descriptive text are ignored.
|
||||
Missing, malformed, multiple or chained dimension patterns return `None`, as do
|
||||
non-positive/non-finite dimensions. Signed/scientific notation and other units
|
||||
are deliberately unsupported; both dimensions must be positive plain numbers.
|
||||
|
||||
K7 currently uses the explicit template `K 7-{production_order}`:
|
||||
|
||||
```python
|
||||
k7_order_template = "K 7-{production_order}"
|
||||
enlyze_order = build_enlyze_production_order("12026000815", k7_order_template)
|
||||
# "K 7-12026000815"
|
||||
```
|
||||
|
||||
Future machines may supply different verified templates without changing the
|
||||
generic helper. Template selection belongs to machine/workplace configuration
|
||||
or adapters; the helper contains no workplace lookup or machine-specific branch.
|
||||
No other machine rule is introduced here.
|
||||
|
||||
ENLYZE production-order identifiers remain opaque everywhere else, with exact
|
||||
comparison. These helpers never reverse-parse or split ENLYZE orders, including
|
||||
combined identifiers such as `K 7-12025000074-K 7-12025000075`; passing such an
|
||||
identifier to the ERP mapping function is rejected.
|
||||
|
||||
Nominal finished-product width currently comes from ERP article-description
|
||||
parsing because neither verified DWH view (`dbo.GRAFANA_WORKPLACE_STATUS` and
|
||||
`dbo.GRAFANA_PRODUCTION_CONFIRMATION`) has a dedicated width column. ENLYZE
|
||||
`wLg1MeasuringWidth` (`Produktbreite`) is explicitly not used as K7 nominal
|
||||
finished-product width:
|
||||
it belongs to the MAHLO measurement system and observed values differ from nominal
|
||||
article widths. Structured ERP product master data would be preferred when available.
|
||||
|
||||
These helpers are independent of database access. The service below composes them
|
||||
for material efficiency; no background processing is attached.
|
||||
|
||||
## Feedback-aligned material efficiency
|
||||
|
||||
`MaterialEfficiencyService` in `service.material_efficiency` combines ERP good area
|
||||
with cumulative material consumption for one exact production order. Its
|
||||
`evaluate_current()` reads the existing ERP workplace adapter once; `evaluate(status)`
|
||||
evaluates an already supplied `CurrentWorkplaceStatus`. Both return an immutable
|
||||
`MaterialEfficiencySnapshot` or `None` when inputs are unavailable.
|
||||
|
||||
`PostgresMaterialSnapshotRepository(settings).latest_at_or_before(...)` in
|
||||
`service.postgres_material` takes keyword arguments `calculation_id`, `machine_id`,
|
||||
`production_order`, and an aware `timestamp`. A parameterized query selects the
|
||||
latest snapshot **at or before ERP feedback time**, matching calculation, machine,
|
||||
and mapped order exactly. It returns `MaterialConsumptionSnapshot` or `None`.
|
||||
Run IDs do not restrict this lookup: stored consumption is already cumulative
|
||||
across the order's runs. The service neither reintegrates nor sums snapshots and
|
||||
does not bridge run boundaries.
|
||||
|
||||
The result includes workplace/machine/calculation identifiers, both order identifiers,
|
||||
article context, nominal width, the UTC ERP feedback instant and selected material timestamp, good area, aligned
|
||||
consumption, `material_consumption_kg_per_m2 = consumption_kg / good_quantity_m2`,
|
||||
and `material_consumption_g_per_m2 = material_consumption_kg_per_m2 * 1000`.
|
||||
Nominal width comes from generic ERP description parsing and is context only;
|
||||
ERP good square metres remain authoritative even when width cannot be parsed.
|
||||
|
||||
Missing ERP status, missing aligned material, missing/non-positive/non-finite good
|
||||
area, invalid consumption (negative or non-finite), or non-finite computed ratios
|
||||
produce `None`. Zero consumption is valid. Configuration errors, invalid timestamp
|
||||
alignment, and database failures raise rather than masquerading as missing data.
|
||||
|
||||
All machine/workplace settings are supplied explicitly. Example using the existing
|
||||
K7 material calculation (with previously constructed ERP gateway and PG settings):
|
||||
|
||||
```python
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencyService
|
||||
from production_analytics.service.postgres_material import PostgresMaterialSnapshotRepository
|
||||
|
||||
service = MaterialEfficiencyService(
|
||||
erp_gateway, PostgresMaterialSnapshotRepository(postgres_settings),
|
||||
workplace="K7",
|
||||
machine_id="c220f95c-a65e-4cb7-99b7-0626d6c7508c",
|
||||
calculation_id="k7-fiber-consumption",
|
||||
format_template="K 7-{production_order}",
|
||||
# Supply only after verifying the ERP timestamp's source timezone:
|
||||
erp_timezone=ZoneInfo("Europe/Berlin"),
|
||||
)
|
||||
result = service.evaluate_current()
|
||||
```
|
||||
|
||||
Aware ERP timestamps are compared as UTC instants. Naive ERP timestamps require
|
||||
explicit `erp_timezone`; there is no inferred system/database timezone. Ambiguous
|
||||
or nonexistent local times during DST transitions are rejected. The result carries the
|
||||
ERP feedback instant normalized to UTC and the selected aware material timestamp.
|
||||
The original ERP status object remains unchanged.
|
||||
|
||||
This alignment avoids knowingly including future material consumption, but does
|
||||
not eliminate ERP roll-feedback timing uncertainty (feedback may be roughly one
|
||||
roll ahead or otherwise offset). Live values are plausibility indicators; final
|
||||
production-order values are more meaningful as relative timing error diminishes.
|
||||
There is no interpolation, lag correction, smoothing, freshness threshold, or
|
||||
estimated timestamp. Historical bootstrap snapshots are usable only when their
|
||||
stored timestamps satisfy the cutoff; a later cumulative total cannot reconstruct
|
||||
an earlier value. Reliable final evaluation requires retaining final ERP feedback;
|
||||
the current-workplace adapter alone does not provide historical completed orders.
|
||||
|
||||
The service itself returns results in memory without modifying material snapshots.
|
||||
The standalone persistence runner is described below. Daily per-machine 24h reporting
|
||||
and aggregation remain future work. Repository SQL tests use an in-memory SQLite
|
||||
fixture with driver transport adaptation; they require no live PostgreSQL.
|
||||
|
||||
## Peak-cycle detection
|
||||
|
||||
@@ -132,8 +446,62 @@ completed peaks, including a legitimate approximately 1195 kg cycle. Run
|
||||
available. Neither capture should be added to version control or automated
|
||||
tests.
|
||||
|
||||
`docker compose up -d timescaledb` is an optional local database design for a
|
||||
future persistence milestone. It is not required for the bootstrap tests.
|
||||
The normal unit test suite mocks PostgreSQL and requires no live database.
|
||||
|
||||
See [PROJECT_KNOWLEDGE.md](PROJECT_KNOWLEDGE.md) for durable project context
|
||||
and [docs/roadmap.md](docs/roadmap.md) for the implementation sequence.
|
||||
|
||||
### Feedback-driven material-efficiency persistence
|
||||
|
||||
Apply the updated `db/schema.sql` using the PostgreSQL setup command above, then run
|
||||
this independent foreground process:
|
||||
|
||||
```bash
|
||||
production-analytics run material-efficiency --config config/k7-material-efficiency.yaml
|
||||
```
|
||||
|
||||
`poll_interval_seconds` in YAML defaults to 60 seconds (fixed delay after each
|
||||
cycle). Workplace, machine, calculation, order format and `erp_timezone` are also
|
||||
configured in YAML; K7 uses `Europe/Berlin`. PostgreSQL settings use the existing
|
||||
exported `POSTGRES_*` variables and `--secrets-file` (default `secrets/enlyze.env`);
|
||||
ERP uses `--erp-secrets-file` (default `secrets/erp.env`). No additional secrets are
|
||||
needed, and `.env` is not loaded automatically.
|
||||
|
||||
`PostgresMaterialEfficiencyWriter.write(snapshot) -> bool` validates aware
|
||||
timestamps and finite numbers, then inserts all KPI fields in a short transaction.
|
||||
`material_efficiency_snapshots` retains one immutable point per
|
||||
`(calculation_id, machine_id, enlyze_production_order, erp_feedback_timestamp)`.
|
||||
The writer uses `RETURNING 1` to return `True` for a new insert and `False` for a
|
||||
duplicate. Only new inserts produce a flushed stdout line with feedback time,
|
||||
workplace, order, good m², material kg and g/m². Unavailable evaluations and
|
||||
duplicates remain silent.
|
||||
Repeated evaluations attempt `ON CONFLICT DO NOTHING`; they never update history,
|
||||
including after a restart. Missing/invalid KPI inputs produce no row and can be
|
||||
retried on a later cycle. Nullable article context remains SQL NULL.
|
||||
|
||||
Persistence frequency follows ERP feedback changes, not ENLYZE sample frequency.
|
||||
This runner is independent of the 10-second material polling process. Live values
|
||||
are plausibility indicators; final production-order (FA) values are more meaningful.
|
||||
Grafana can read PostgreSQL alone:
|
||||
|
||||
```sql
|
||||
SELECT erp_feedback_timestamp AS "time", material_consumption_g_per_m2
|
||||
FROM material_efficiency_snapshots
|
||||
WHERE $__timeFilter(erp_feedback_timestamp)
|
||||
AND calculation_id = 'k7-fiber-consumption'
|
||||
AND machine_id = 'c220f95c-a65e-4cb7-99b7-0626d6c7508c'
|
||||
ORDER BY erp_feedback_timestamp;
|
||||
```
|
||||
|
||||
ERP read errors and transient PostgreSQL connection/operational errors are reported
|
||||
using error classes only and retried after the configured delay. Each database
|
||||
operation opens a fresh connection. Configuration, schema/programming and invalid
|
||||
timestamp contract errors terminate with a nonzero CLI exit; ambiguous/nonexistent
|
||||
DST feedback remains rejected. Ctrl-C stops the foreground runner cleanly.
|
||||
|
||||
The current-status ERP source cannot backfill feedback events missed between polls
|
||||
or during outages, nor guarantee observation of final FA feedback before the order
|
||||
changes. Daily 24h reports and final order summary tables remain future work.
|
||||
No dashboard, aggregation or lag correction is added here.
|
||||
|
||||
Bento 1: [fresh bentonite consumption configuration and scope](docs/bento1-fresh-bentonite.md).
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
services:
|
||||
timescaledb:
|
||||
image: timescale/timescaledb:latest-pg16
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: production_analytics
|
||||
POSTGRES_USER: production_analytics
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
machine_id: 5f42a4f6-9ca0-4f6f-9786-40d50a35b230
|
||||
erp_workplace: Bento 1
|
||||
production_order_format: "Bento 1-{production_order}"
|
||||
poll_interval_seconds: 300
|
||||
lookback_hours: 48
|
||||
full_reconciliation_hours: 24
|
||||
completion_tolerance_m2: 0.001
|
||||
@@ -0,0 +1,21 @@
|
||||
# Fresh bentonite only: the recycled/recovered third spreader is excluded.
|
||||
calculations:
|
||||
- id: bento1-fresh-bentonite-consumption
|
||||
type: material_consumption
|
||||
version: "1"
|
||||
machine_ref: 5f42a4f6-9ca0-4f6f-9786-40d50a35b230
|
||||
process_application_calculation_id: bento1-fresh-bentonite-application
|
||||
source_mode: rotational_discharge
|
||||
rotational_speed_signal_refs:
|
||||
- 6e5d2d94-98f9-4cc1-8a88-7987c6282525
|
||||
- cd7385c4-337b-4759-ab32-45d65beaf190
|
||||
calibration_ref: bento1-spreader-1-2
|
||||
# Transport Auszug Geschwindigkeit Istwert, m/min (production gate only).
|
||||
gate_signal_ref: fef41976-1103-4090-b780-eaeecc02fdfa
|
||||
gate_threshold: 0.3
|
||||
max_sample_gap_seconds: 20.0
|
||||
erp_workplace: Bento 1
|
||||
production_order_format: "Bento 1-{production_order}"
|
||||
output_metric: material_consumption
|
||||
output_unit: kg
|
||||
group_by: production_order
|
||||
@@ -0,0 +1,7 @@
|
||||
workplace: BENTO 1
|
||||
machine_id: 5f42a4f6-9ca0-4f6f-9786-40d50a35b230
|
||||
calculation_id: bento1-fresh-bentonite-consumption
|
||||
output_calculation_id: bento1-fresh-bentonite-efficiency
|
||||
production_order_format: "Bento 1-{production_order}"
|
||||
erp_timezone: Europe/Berlin
|
||||
poll_interval_seconds: 60
|
||||
@@ -0,0 +1,9 @@
|
||||
# Generic ENLYZE downtime reconciliation configuration; one runner per machine.
|
||||
machine_id: machine-uuid
|
||||
erp_workplace: WORKPLACE
|
||||
production_order_format: "Machine-{production_order}"
|
||||
poll_interval_seconds: 300
|
||||
lookback_hours: 48
|
||||
# Full source scan keeps very old open/UNKNOWN events eligible for delayed edits.
|
||||
full_reconciliation_hours: 24
|
||||
completion_tolerance_m2: 0.001
|
||||
@@ -0,0 +1,30 @@
|
||||
# Supply E1's ENLYZE machine UUID before operating. Signal references are PLC
|
||||
# origins, resolved uniquely by the ENLYZE gateway at poll time.
|
||||
calculations:
|
||||
- id: e1-channel-material-consumption
|
||||
type: channel_material_consumption
|
||||
version: "1"
|
||||
machine_ref: 8302e3d1-b1e5-42f1-8540-615eb6c73e08
|
||||
max_sample_gap_seconds: 20.0
|
||||
percentage_tolerance: 1.0
|
||||
extruders:
|
||||
- id: Ex1
|
||||
total_rate_signal_ref: DB107:18
|
||||
channels:
|
||||
- {id: C1, status_signal_ref: DB107:122.0, percentage_signal_ref: DB107:126, screw_speed_signal_ref: DB107:170}
|
||||
- {id: C2, status_signal_ref: DB107:222.0, percentage_signal_ref: DB107:226, screw_speed_signal_ref: DB107:270}
|
||||
- {id: C3, status_signal_ref: DB107:322.0, percentage_signal_ref: DB107:326, screw_speed_signal_ref: DB107:370}
|
||||
- {id: C4, status_signal_ref: DB107:422.0, percentage_signal_ref: DB107:426, screw_speed_signal_ref: DB107:470}
|
||||
- {id: C5, status_signal_ref: DB107:522.0, percentage_signal_ref: DB107:526, screw_speed_signal_ref: DB107:570}
|
||||
- {id: C6, status_signal_ref: DB107:622.0, percentage_signal_ref: DB107:626, screw_speed_signal_ref: DB107:670}
|
||||
- {id: C7, status_signal_ref: DB107:722.0, percentage_signal_ref: DB107:726, screw_speed_signal_ref: DB107:770}
|
||||
- id: Ex2
|
||||
total_rate_signal_ref: DB207:18
|
||||
channels:
|
||||
- {id: C1, status_signal_ref: DB207:122.0, percentage_signal_ref: DB207:126, screw_speed_signal_ref: DB207:170}
|
||||
- {id: C2, status_signal_ref: DB207:222.0, percentage_signal_ref: DB207:226, screw_speed_signal_ref: DB207:270}
|
||||
- {id: C3, status_signal_ref: DB207:322.0, percentage_signal_ref: DB207:326, screw_speed_signal_ref: DB207:370}
|
||||
- {id: C4, status_signal_ref: DB207:422.0, percentage_signal_ref: DB207:426, screw_speed_signal_ref: DB207:470}
|
||||
- {id: C5, status_signal_ref: DB207:522.0, percentage_signal_ref: DB207:526, screw_speed_signal_ref: DB207:570}
|
||||
- {id: C6, status_signal_ref: DB207:622.0, percentage_signal_ref: DB207:626, screw_speed_signal_ref: DB207:670}
|
||||
- {id: C7, status_signal_ref: DB207:722.0, percentage_signal_ref: DB207:726, screw_speed_signal_ref: DB207:770}
|
||||
@@ -0,0 +1,14 @@
|
||||
# Validated K7: Stundenleistung Anlage (kg/h), gated by
|
||||
# Geschwindigkeit Gesamtanlage (m/min) > 0.5. Not Anlage läuft.
|
||||
calculations:
|
||||
- id: k7-fiber-consumption
|
||||
type: material_consumption
|
||||
version: "1"
|
||||
machine_ref: c220f95c-a65e-4cb7-99b7-0626d6c7508c
|
||||
rate_signal_ref: c9d06af5-f6d6-4ede-b6c4-5a98bac77129
|
||||
gate_signal_ref: 823867bb-f5d2-40eb-b875-657155addfd0
|
||||
gate_threshold: 0.5
|
||||
max_sample_gap_seconds: 20.0
|
||||
output_metric: material_consumption
|
||||
output_unit: kg
|
||||
group_by: production_order
|
||||
@@ -0,0 +1,6 @@
|
||||
workplace: K7
|
||||
machine_id: c220f95c-a65e-4cb7-99b7-0626d6c7508c
|
||||
calculation_id: k7-fiber-consumption
|
||||
production_order_format: "K 7-{production_order}"
|
||||
erp_timezone: Europe/Berlin
|
||||
poll_interval_seconds: 60
|
||||
@@ -0,0 +1,14 @@
|
||||
calibrations:
|
||||
bento1-spreader-1-2:
|
||||
type: rotational_discharge
|
||||
value: 2.75
|
||||
unit: kg_per_rev_m
|
||||
calibrated_at: 2026-09-08
|
||||
method: gravimetric_tray
|
||||
description: Bento 1 fresh-bentonite spreaders 1 and 2
|
||||
reference:
|
||||
measured_application_g_m2: 4068
|
||||
line_speed_m_min: 2.3
|
||||
signal_values:
|
||||
left: 1.65
|
||||
right: 1.75
|
||||
@@ -0,0 +1,15 @@
|
||||
meters:
|
||||
B2:
|
||||
machine_uuid: 0d7955e9-5cde-4e14-9af7-5b041dc796f0
|
||||
signals:
|
||||
energy_total: 0f5c0853-1aa7-4fa3-b267-ea2669024e46
|
||||
active_power_total: 41e5729e-b724-49d4-ad24-666d959e121d
|
||||
grid_frequency: 127ddf56-9358-43ce-84fe-f7631486cfc9
|
||||
# Add only after verifying the real UUID in the raw /v2/variables page.
|
||||
# power_outages: <verified-uuid>
|
||||
# Optional; L2 is not required when it is unavailable.
|
||||
# phase_active_power:
|
||||
# L1: <verified-uuid>
|
||||
# L3: <verified-uuid>
|
||||
poll_interval_seconds: 60
|
||||
report_mount_path: /mnt/reports/energy
|
||||
@@ -0,0 +1,14 @@
|
||||
-- Set :meter, :from and :to from Grafana variables.
|
||||
-- Current power:
|
||||
SELECT timestamp AS "time", active_power_total_kw AS "value"
|
||||
FROM power_meter_readings WHERE meter = '${meter}' AND $__timeFilter(timestamp);
|
||||
|
||||
-- Energy today / rolling 24h (counter increments are reset-safe).
|
||||
SELECT COALESCE(sum(energy_delta_kwh), 0) AS kwh
|
||||
FROM power_meter_readings
|
||||
WHERE meter = '${meter}' AND timestamp >= now() - interval '24 hours';
|
||||
|
||||
-- Frequency and outages in selected period.
|
||||
SELECT min(grid_frequency_hz) AS frequency_min_hz, max(grid_frequency_hz) AS frequency_max_hz,
|
||||
COALESCE(sum(outage_delta), 0) AS outages
|
||||
FROM power_meter_readings WHERE meter = '${meter}' AND $__timeFilter(timestamp);
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
CREATE TABLE IF NOT EXISTS material_consumption_snapshots (
|
||||
timestamp timestamptz NOT NULL,
|
||||
calculation_id text NOT NULL,
|
||||
machine_id text NOT NULL,
|
||||
production_order text NOT NULL,
|
||||
run_id text NOT NULL,
|
||||
consumption_kg double precision NOT NULL,
|
||||
PRIMARY KEY (calculation_id, machine_id, production_order, timestamp)
|
||||
);
|
||||
|
||||
-- Machine time series across orders; the primary key covers a specific order.
|
||||
CREATE INDEX IF NOT EXISTS material_consumption_snapshots_machine_time_idx
|
||||
ON material_consumption_snapshots (calculation_id, machine_id, timestamp);
|
||||
|
||||
-- Channel identities are intentionally independent of material assignment.
|
||||
CREATE TABLE IF NOT EXISTS channel_material_consumption_snapshots (
|
||||
timestamp timestamptz NOT NULL,
|
||||
calculation_id text NOT NULL,
|
||||
machine_id text NOT NULL,
|
||||
extruder text NOT NULL,
|
||||
doser_channel text NOT NULL,
|
||||
production_order text NOT NULL,
|
||||
run_id text NOT NULL,
|
||||
cumulative_consumption_kg double precision NOT NULL,
|
||||
material_number text NULL,
|
||||
material_name text NULL,
|
||||
material_mapping_status text NOT NULL CHECK (material_mapping_status IN ('ASSIGNED', 'UNAVAILABLE', 'AMBIGUOUS', 'UNMAPPED')),
|
||||
percentage_sum_valid boolean NOT NULL,
|
||||
PRIMARY KEY (calculation_id, machine_id, extruder, doser_channel, production_order, timestamp)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS material_efficiency_snapshots (
|
||||
erp_feedback_timestamp timestamptz NOT NULL,
|
||||
material_snapshot_timestamp timestamptz NOT NULL,
|
||||
calculation_id text NOT NULL,
|
||||
machine_id text NOT NULL,
|
||||
workplace text NOT NULL,
|
||||
erp_production_order text NOT NULL,
|
||||
enlyze_production_order text NOT NULL,
|
||||
article_number text NULL,
|
||||
article_description text NULL,
|
||||
nominal_width_m double precision NULL,
|
||||
good_quantity_m2 double precision NOT NULL,
|
||||
material_consumption_kg double precision NOT NULL,
|
||||
material_consumption_kg_per_m2 double precision NOT NULL,
|
||||
material_consumption_g_per_m2 double precision NOT NULL,
|
||||
PRIMARY KEY (calculation_id, machine_id, enlyze_production_order, erp_feedback_timestamp)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS material_efficiency_snapshots_machine_time_idx
|
||||
ON material_efficiency_snapshots (calculation_id, machine_id, erp_feedback_timestamp);
|
||||
|
||||
-- Instantaneous process measurements; no ERP good-area denominator.
|
||||
CREATE TABLE IF NOT EXISTS material_application_snapshots (
|
||||
timestamp timestamptz NOT NULL,
|
||||
calculation_id text NOT NULL,
|
||||
machine_id text NOT NULL,
|
||||
production_order text NOT NULL,
|
||||
run_id text NOT NULL,
|
||||
application_g_m2 double precision NOT NULL,
|
||||
PRIMARY KEY (calculation_id, machine_id, production_order, timestamp)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS material_application_snapshots_machine_time_idx
|
||||
ON material_application_snapshots (calculation_id, machine_id, timestamp);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS power_meter_readings (
|
||||
meter text NOT NULL,
|
||||
timestamp timestamptz NOT NULL,
|
||||
energy_total_kwh double precision NOT NULL,
|
||||
energy_delta_kwh double precision NOT NULL,
|
||||
active_power_total_kw double precision NOT NULL,
|
||||
grid_frequency_hz double precision NOT NULL,
|
||||
power_outages double precision NULL,
|
||||
outage_delta double precision NULL,
|
||||
phase_active_power_kw jsonb NOT NULL DEFAULT '{}'::jsonb,
|
||||
PRIMARY KEY (meter, timestamp)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS power_meter_readings_time_idx
|
||||
ON power_meter_readings (meter, timestamp DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS power_meter_monthly_reports (
|
||||
meter text NOT NULL,
|
||||
month text NOT NULL,
|
||||
report_path text NOT NULL,
|
||||
summary jsonb NOT NULL,
|
||||
generated_at timestamptz NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (meter, month)
|
||||
);
|
||||
|
||||
-- Mutable ENLYZE downtime source records. external_id is the stable ENLYZE UUID.
|
||||
CREATE TABLE IF NOT EXISTS production_downtime_events (
|
||||
external_id text PRIMARY KEY,
|
||||
machine_id text NOT NULL,
|
||||
production_order text NULL,
|
||||
source_type text NOT NULL,
|
||||
source_start timestamptz NOT NULL,
|
||||
source_end timestamptz NULL,
|
||||
attributed_start timestamptz NULL,
|
||||
attributed_end timestamptz NULL,
|
||||
attributed_duration_seconds double precision NULL,
|
||||
reason_id text NULL,
|
||||
reason_name text NULL,
|
||||
reason_description text NULL,
|
||||
reason_group text NULL,
|
||||
category text NOT NULL CHECK (category IN ('PLANNED', 'UNPLANNED', 'UNKNOWN')),
|
||||
comment text NULL,
|
||||
source_updated_at timestamptz NULL,
|
||||
last_reconciled_at timestamptz NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS production_downtime_events_machine_attributed_time_idx
|
||||
ON production_downtime_events (machine_id, attributed_start);
|
||||
CREATE INDEX IF NOT EXISTS production_downtime_events_production_order_idx
|
||||
ON production_downtime_events (production_order);
|
||||
CREATE INDEX IF NOT EXISTS production_downtime_events_category_idx
|
||||
ON production_downtime_events (category);
|
||||
CREATE INDEX IF NOT EXISTS production_downtime_events_unresolved_idx
|
||||
ON production_downtime_events (machine_id, source_start)
|
||||
WHERE source_end IS NULL OR category = 'UNKNOWN';
|
||||
|
||||
-- One observed ERP order boundary per machine, used to clip open ENLYZE downtimes.
|
||||
CREATE TABLE IF NOT EXISTS production_order_attribution_state (
|
||||
machine_id text PRIMARY KEY,
|
||||
production_order text NOT NULL,
|
||||
started_at timestamptz NOT NULL,
|
||||
ended_at timestamptz NULL
|
||||
);
|
||||
@@ -0,0 +1,250 @@
|
||||
# Bento 1 fresh bentonite KPIs
|
||||
|
||||
|
||||
## Three distinct quantities
|
||||
|
||||
All **fresh bentonite** values exclude the third recycled/recovered-material
|
||||
scatterer. Only the two actual rpm signals listed below are used; no SET signals
|
||||
are introduced. The specific discharge is central calibration
|
||||
data: **2.75 kg/(revolution × metre product width)**.
|
||||
|
||||
A) **Instantaneous process application [g/m²]**
|
||||
|
||||
`application_g_m2 = (rpm_left + rpm_right) × 2.75 × 1000 / line_speed_m_min`
|
||||
|
||||
Emitted only for observed samples with line speed **strictly greater than
|
||||
0.3 m/min**. Zero, near-zero, negative and threshold-equal speeds emit no point.
|
||||
Width cancels between kg/min and m²/min; neither nominal width nor ERP good area
|
||||
enters this formula. For 3.739 + 3.956 rpm at 8 m/min the result is 2645.15625 g/m².
|
||||
This is fresh-roll process application, not FA material efficiency.
|
||||
|
||||
B) **Cumulative fresh consumption [kg]**, unchanged
|
||||
|
||||
`kg/min = (rpm_left + rpm_right) × nominal_width_m × 2.75`
|
||||
|
||||
The existing previous-value integration sums `kg/min × elapsed_seconds / 60`
|
||||
for eligible intervals, using the preceding sample's production gate. Width
|
||||
continues to come from the ERP nominal-width parser. The integration algorithm,
|
||||
gap handling, order checkpoint format and disjoint-run accumulation are unchanged.
|
||||
|
||||
C) **FA material efficiency [g/m²]**
|
||||
|
||||
`efficiency_g_m2 = cumulative_fresh_bentonite_kg × 1000 / cumulative_good_area_m2`
|
||||
|
||||
The generic material-efficiency service reads the latest exact-order cumulative
|
||||
snapshot **at or before the ERP feedback timestamp**, then divides by that
|
||||
feedback's cumulative good area. Missing/nonpositive good area emits no point.
|
||||
This ratio includes losses captured by the existing consumption model (startup
|
||||
material, rejects and other consumed material not represented in good area).
|
||||
It does not add consumption during intervals excluded by the validated gate or
|
||||
gap rules, including stopped-line periods. Disjoint runs of one FA share the
|
||||
existing order total; their cumulative snapshots must not be summed again.
|
||||
|
||||
## Configuration, PostgreSQL and Grafana
|
||||
|
||||
| Calculation ID | Table | Value column | Time column |
|
||||
| --- | --- | --- | --- |
|
||||
| `bento1-fresh-bentonite-application` | `material_application_snapshots` | `application_g_m2` | `timestamp` |
|
||||
| `bento1-fresh-bentonite-consumption` | `material_consumption_snapshots` | `consumption_kg` | `timestamp` |
|
||||
| `bento1-fresh-bentonite-efficiency` | `material_efficiency_snapshots` | `material_consumption_g_per_m2` | `erp_feedback_timestamp` |
|
||||
|
||||
Scope Grafana queries by calculation ID and
|
||||
`machine_id = '5f42a4f6-9ca0-4f6f-9786-40d50a35b230'`; optionally filter by
|
||||
`production_order` (application/consumption) or `enlyze_production_order`
|
||||
(efficiency). The efficiency table also stores `good_quantity_m2`,
|
||||
`material_consumption_kg`, `material_consumption_kg_per_m2` and
|
||||
`material_snapshot_timestamp` for auditability.
|
||||
|
||||
`config/bento1-material-consumption.yaml` opts into process snapshots through
|
||||
`process_application_calculation_id`. This generic option requires rotational
|
||||
sources and a positive speed gate. Both outputs reuse the same configured ACT
|
||||
signals, discharge factor and samples. Process points use source timestamps,
|
||||
including bootstrap samples from disjoint runs; no interpolation or hold-forward
|
||||
points are stored for inactive periods. Configure Grafana to leave missing
|
||||
periods as gaps rather than carrying the last active value forward.
|
||||
|
||||
The shared consumption poller still requires valid ERP width/order context to
|
||||
complete a cycle, although the application formula itself has no width input.
|
||||
Application rows are committed before the existing checkpoint advances; a failed
|
||||
application write retries the window. Duplicate source timestamps are ignored by
|
||||
the primary key. Existing checkpoints are retained, so earlier application
|
||||
history is not automatically backfilled. K7 does not enable this output.
|
||||
|
||||
`config/bento1-material-efficiency.yaml` uses the existing generic runner.
|
||||
`calculation_id` selects the cumulative input; optional `output_calculation_id`
|
||||
sets the distinct persisted KPI ID. Omitting it preserves existing behavior,
|
||||
including K7. ERP workplace `strip().casefold()` normalization is unchanged.
|
||||
|
||||
Before deploying, apply the additive, idempotent `db/schema.sql` to the existing
|
||||
PostgreSQL database to create `material_application_snapshots` and its index.
|
||||
The existing consumption and efficiency tables need no column migration. For
|
||||
example, on the deployment host:
|
||||
|
||||
```sh
|
||||
docker compose exec -T timescaledb psql -U production_analytics -d production_analytics \
|
||||
-v ON_ERROR_STOP=1 < db/schema.sql
|
||||
```
|
||||
|
||||
Restart the configured Bento consumption runner to enable process persistence,
|
||||
and supervise a separate generic efficiency runner:
|
||||
|
||||
```sh
|
||||
production-analytics run material-efficiency \
|
||||
--config config/bento1-material-efficiency.yaml \
|
||||
--secrets-file secrets/enlyze.env --erp-secrets-file secrets/erp.env
|
||||
```
|
||||
|
||||
No cleanup/reset of validated rpm consumption state or snapshots is required for
|
||||
these additions. No database migration, live service restart or historical data
|
||||
cleanup was performed as part of this implementation.
|
||||
|
||||
## Validated cumulative model details
|
||||
|
||||
`config/bento1-material-consumption.yaml` defines
|
||||
`bento1-fresh-bentonite-consumption`, an estimate of **fresh bentonite consumption**
|
||||
from actual spreader roll speeds. Both needle rolls apply sequentially across the
|
||||
full web width. The recycled/recovered third spreader is excluded.
|
||||
|
||||
The generic `rotational_discharge` source converts one or more rpm signals to the
|
||||
existing integrator's kg/h: `sum(rpm) × nominal_width_m × specific_discharge_kg_per_rev_m × 60`.
|
||||
Bento config sets the factor to **2.75 kg/(rev·m)** and uses:
|
||||
|
||||
- Right ACT: `6e5d2d94-98f9-4cc1-8a88-7987c6282525`
|
||||
- Left ACT: `cd7385c4-337b-4759-ab32-45d65beaf190`
|
||||
|
||||
ENLYZE now returns physical rpm directly (scaling factor 1.0); no division by
|
||||
1000 is applied. For 3.739 + 3.956 rpm and 5 m width, the rate is 105.80625 kg/min
|
||||
(6348.375 kg/h), giving 17.634375 kg in ten seconds.
|
||||
|
||||
For cumulative consumption, transport speed
|
||||
`fef41976-1103-4090-b780-eaeecc02fdfa` remains exclusively the production gate:
|
||||
speed must be strictly greater than 0.3 m/min. It does not multiply the mass rate.
|
||||
For instantaneous application, this same speed is also the denominator. The generic rotational mode also supports omitting
|
||||
`gate_signal_ref` and `gate_threshold`, in which case all valid intervals are
|
||||
active. Bento retains its gate.
|
||||
|
||||
The two old SET sources `19ea65d2-bd35-4d02-89de-4495583d9026` and
|
||||
`d9f47615-69cf-4f97-88df-199585764491` are no longer requested by Bento.
|
||||
`direct_mass_rate` remains the default for K7; `area_application` remains
|
||||
available for other configurations. There is no Bento branch in the integrator.
|
||||
|
||||
The previous-value integration, JSON state format and order bootstrap/resume
|
||||
remain unchanged. Totals accumulate across disjoint runs, with a new baseline
|
||||
at each run boundary, even if the inter-run gap is under 20 seconds. Intervals
|
||||
up to 20 seconds use the preceding rate and gate; longer sample gaps and
|
||||
unobserved run tails are not integrated. Missing/nonfinite source values reject
|
||||
the cycle. Finite negative speeds retain the existing generic signed-rate
|
||||
semantics; no clamping is introduced.
|
||||
|
||||
Width comes from the existing ERP nominal-width parser, with workplace
|
||||
`Bento 1` and order format `Bento 1-{production_order}`. Workplace comparisons
|
||||
use `strip().casefold()`; order matching remains strict. Missing/mismatched ERP
|
||||
context or an unparseable width rejects the cycle without saving state. The
|
||||
current ERP context supplies width for all runs of the same order, assuming
|
||||
constant article width. Historical orders no longer in the ERP workplace view
|
||||
need a historical width source for replay.
|
||||
|
||||
The current factor is **2.75 kg/(rev*m)**, independently derived on
|
||||
**2026-09-08** by the **gravimetric tray method**: measured application
|
||||
**4068 g/m²**, line speed **2.3 m/min**, actual rotational signals **left 1.65**
|
||||
and **right 1.75**. The derivation is `4.068 × 2.3 / (1.65 + 1.75) ≈ 2.75188`,
|
||||
rounded to the configured 2.75. This supersedes the earlier 3.12 estimate.
|
||||
See [central calibration configuration](material-calibrations.md) for updates
|
||||
and persistence semantics.
|
||||
|
||||
Tests use realistic rpm values and synthetic samples, including disjoint-run
|
||||
bootstrap and persisted resume. They are not a new replay of recorded ENLYZE data.
|
||||
|
||||
## Historical migration from SET to rpm: review before execution
|
||||
|
||||
The following records the earlier source-model migration, **not a prerequisite
|
||||
cleanup for adding these KPIs**. Its deployment observations are historical.
|
||||
If the validated rpm model is already deployed, retain its state and snapshots;
|
||||
do not execute this historical cleanup for the KPI extension. Recheck deployment
|
||||
provenance separately if that earlier migration is still outstanding.
|
||||
|
||||
No production state or database rows were changed during this implementation,
|
||||
and no service start was requested. The operator reports Bento stopped;
|
||||
sandbox systemd access could not independently confirm that state.
|
||||
The deployed `/etc/systemd/system/production-analytics-bento1.service` specifies
|
||||
`/opt/git-projects/production-analytics/data/state/material` as its state directory.
|
||||
|
||||
The existing state file for machine `5f42a4f6-9ca0-4f6f-9786-40d50a35b230`
|
||||
and order `Bento 1-12026000857` is exactly:
|
||||
|
||||
```text
|
||||
/opt/git-projects/production-analytics/data/state/material/4d6cc209e35bc58887e5b4a3816835006425429e810e0d957f651971723e8c2f.json
|
||||
```
|
||||
|
||||
This identity was verified using the store's SHA-256 of the JSON machine/order
|
||||
pair. The file exists but its contents are not readable by the sandbox user.
|
||||
The other state file, `6d11088159b8e4fd21543560a54a31a0f86faf22bbf8f16873b228d1903c23f9.json`,
|
||||
has not been attributed here and must be left untouched.
|
||||
|
||||
Old PostgreSQL snapshots are in `material_consumption_snapshots`, scoped by both
|
||||
`calculation_id = 'bento1-fresh-bentonite-consumption'` and
|
||||
`machine_id = '5f42a4f6-9ca0-4f6f-9786-40d50a35b230'`.
|
||||
The table has no calculation-source/version provenance column. Given the stopped
|
||||
service and no rpm deployment yet, existing rows in this scope belong to the old
|
||||
calculation. PostgreSQL was unreachable from the sandbox, so exact row counts,
|
||||
order/run membership and timestamp bounds remain unverified. Do not mistake
|
||||
this predicate for a completed row inventory. Review the following output first.
|
||||
|
||||
Run these **read-only inventory commands** on the deployment host:
|
||||
|
||||
```sh
|
||||
cd /opt/git-projects/production-analytics
|
||||
sudo systemctl is-active production-analytics-bento1.service
|
||||
sudo cat data/state/material/4d6cc209e35bc58887e5b4a3816835006425429e810e0d957f651971723e8c2f.json
|
||||
sudo docker compose exec -T timescaledb psql -U production_analytics -d production_analytics -v ON_ERROR_STOP=1 <<'SQL'
|
||||
BEGIN READ ONLY;
|
||||
SELECT production_order, run_id, count(*) AS rows,
|
||||
min(timestamp) AS first_snapshot, max(timestamp) AS last_snapshot,
|
||||
min(consumption_kg), max(consumption_kg)
|
||||
FROM material_consumption_snapshots
|
||||
WHERE calculation_id = 'bento1-fresh-bentonite-consumption'
|
||||
AND machine_id = '5f42a4f6-9ca0-4f6f-9786-40d50a35b230'
|
||||
GROUP BY production_order, run_id ORDER BY first_snapshot;
|
||||
SELECT * FROM material_consumption_snapshots
|
||||
WHERE calculation_id = 'bento1-fresh-bentonite-consumption'
|
||||
AND machine_id = '5f42a4f6-9ca0-4f6f-9786-40d50a35b230'
|
||||
ORDER BY production_order, timestamp;
|
||||
COMMIT;
|
||||
SQL
|
||||
```
|
||||
|
||||
After reviewing the inventory, and **only with cleanup authorization**, keep the
|
||||
service stopped and execute this backup-and-delete transaction. The backup table
|
||||
intentionally has no `IF NOT EXISTS`: a repeated invocation fails rather than
|
||||
reusing an older backup. Only rows with the backed-up primary keys are deleted.
|
||||
K7 rows are outside the predicate.
|
||||
|
||||
```sh
|
||||
cd /opt/git-projects/production-analytics
|
||||
sudo docker compose exec -T timescaledb psql -U production_analytics -d production_analytics -v ON_ERROR_STOP=1 <<'SQL'
|
||||
BEGIN;
|
||||
CREATE TABLE bento1_material_snapshots_before_rpm_20260907 AS
|
||||
SELECT * FROM material_consumption_snapshots
|
||||
WHERE calculation_id = 'bento1-fresh-bentonite-consumption'
|
||||
AND machine_id = '5f42a4f6-9ca0-4f6f-9786-40d50a35b230';
|
||||
DELETE FROM material_consumption_snapshots AS s
|
||||
USING bento1_material_snapshots_before_rpm_20260907 AS old
|
||||
WHERE s.calculation_id = old.calculation_id AND s.machine_id = old.machine_id
|
||||
AND s.production_order = old.production_order AND s.timestamp = old.timestamp;
|
||||
COMMIT;
|
||||
SQL
|
||||
sudo mv -n -- \
|
||||
data/state/material/4d6cc209e35bc58887e5b4a3816835006425429e810e0d957f651971723e8c2f.json \
|
||||
data/state/material/4d6cc209e35bc58887e5b4a3816835006425429e810e0d957f651971723e8c2f.json.before-rpm-20260907
|
||||
```
|
||||
|
||||
Verify the original JSON path is absent before restarting; `mv -n` preserves an
|
||||
existing backup and will not overwrite it. If inventory reveals other Bento
|
||||
orders, derive their paths with `JsonMaterialStateStore._path(machine, order)`
|
||||
and review any existing files before moving them too. Do not clear the shared
|
||||
directory. Complete both state and snapshot cleanup before starting the new
|
||||
code: the state schema deliberately does not detect a changed source model.
|
||||
|
||||
A later start bootstraps only the currently open order using its current ERP
|
||||
width. It does not automatically rebuild snapshots for closed historical
|
||||
orders. Reconstructing those requires a separately reviewed historical replay.
|
||||
@@ -0,0 +1,36 @@
|
||||
# ENLYZE downtime architecture
|
||||
|
||||
`GET /v2/downtimes` is the source of downtime events. Its UUID is the durable external
|
||||
identity. `end: null` means the source event is currently open. `reason: null` is valid
|
||||
and is persisted as `UNKNOWN`; it is never treated as `UNPLANNED`. ENLYZE's
|
||||
`updated.timestamp` describes source metadata/reason editing, not finalization.
|
||||
|
||||
The reconciliation runner polls each configured machine with a configurable source-start
|
||||
lookback (48 hours by default), follows pagination, and upserts by UUID. Every 24 hours by
|
||||
default it also scans the full machine source so an old open or UNKNOWN event remains eligible
|
||||
for a delayed classification update. It always replaces
|
||||
the end time, comment, reason metadata, category, source update timestamp, and attributed
|
||||
timing. This is intentionally not append-only because supervisors can classify an event much
|
||||
later. Operators can enlarge `lookback_hours` where delayed classification exceeds the normal
|
||||
window.
|
||||
|
||||
`production_downtime_events` retains ENLYZE source timing separately from FA-attributed
|
||||
timing. An event only receives an order while the persisted ERP order boundary is active.
|
||||
When ERP reports a new production order, the preceding boundary ends at the new feedback
|
||||
timestamp. A boundary also ends when `remaining_quantity_m2 <= completion_tolerance_m2`, or
|
||||
when `good_quantity_m2 + tolerance >= order_quantity_m2`. These are the exact fields from
|
||||
`CurrentWorkplaceStatus`; `feedback_timestamp` supplies the boundary instant. No equality on
|
||||
floating values is required. If neither signal proves completion, attribution stays open.
|
||||
|
||||
This protects a completed FA from time in an ENLYZE downtime that remains open after work has
|
||||
finished. It does not infer completion from downtime. The durable
|
||||
`production_order_attribution_state` makes this clipping survive runner restarts.
|
||||
|
||||
There is no machine-readable schedule today. No clock-time, shift, overnight, weekend, or Excel
|
||||
planning inference occurs. ENLYZE `reason.category` is the only planned/unplanned source;
|
||||
anything absent or unsupported remains `UNKNOWN`. A future SPA schedule provider can refine
|
||||
classification or boundaries as a separate input without changing stored source facts.
|
||||
|
||||
The resulting fields support totals per order, category and reason, individual source events,
|
||||
and open events. Source duration is ENLYZE's `source_start/source_end`; attributed duration is
|
||||
only the interval inside the proven FA boundary.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Material calibrations
|
||||
|
||||
Process calibration values live in the version-controlled
|
||||
`config/material-calibrations.yaml`. Calculations refer to opaque IDs; IDs have
|
||||
no special meaning to the parser. The current entry is:
|
||||
|
||||
```yaml
|
||||
calibrations:
|
||||
bento1-spreader-1-2:
|
||||
type: rotational_discharge
|
||||
value: 2.75
|
||||
unit: kg_per_rev_m
|
||||
calibrated_at: 2026-09-08
|
||||
method: gravimetric_tray
|
||||
description: Bento 1 fresh-bentonite spreaders 1 and 2
|
||||
reference:
|
||||
measured_application_g_m2: 4068
|
||||
line_speed_m_min: 2.3
|
||||
signal_values:
|
||||
left: 1.65
|
||||
right: 1.75
|
||||
```
|
||||
|
||||
All entry fields are required. Type, unit, method and description are non-empty
|
||||
strings; value must be a finite number (booleans are rejected); calibrated_at is
|
||||
a calendar date in YYYY-MM-DD format, quoted or unquoted. Reference is a non-empty
|
||||
mapping whose contents preserve source-specific measurement provenance. Unknown
|
||||
entry fields and duplicate YAML keys are rejected. The generic registry permits
|
||||
other types and units; each consuming calculation checks its own compatibility.
|
||||
|
||||
`config/bento1-material-consumption.yaml` contains
|
||||
`calibration_ref: bento1-spreader-1-2`. During configuration loading, references
|
||||
resolve from `material-calibrations.yaml` in the calculation file's directory,
|
||||
independently of the working directory. The entire registry is validated when a
|
||||
reference is used. Rotational discharge requires type `rotational_discharge`,
|
||||
unit `kg_per_rev_m` and a positive value. Resolution supplies the existing
|
||||
`specific_discharge_kg_per_rev_m` runtime field for both application and
|
||||
consumption; the calculation algorithms are unchanged. There is no unit conversion.
|
||||
|
||||
Missing/unreadable files, missing IDs, invalid metadata and incompatible type or
|
||||
unit raise `CalculationConfigError` before a runner starts. Existing direct numeric
|
||||
configurations remain supported; specifying both a reference and a direct factor
|
||||
is rejected. References are currently supported by rotational discharge consumers.
|
||||
K7 uses direct mass rate and does not load or require a calibration file.
|
||||
|
||||
## Updating a calibration
|
||||
|
||||
After a physical measurement, edit only the central entry's value, date, method
|
||||
and reference details, and keep its ID stable. Review and version-control the
|
||||
change. Additional machines can add new IDs using the same structure. A new
|
||||
calculation type needs an explicit consumer compatibility contract.
|
||||
|
||||
Ship the central YAML alongside the calculation YAML and restart the consuming
|
||||
process to load changes. No new CLI flag, environment variable, database migration
|
||||
or UI is required. There is no hot reload or historical date-based selection.
|
||||
Changes affect future calculations and explicitly rebuilt calculations. Historical
|
||||
persisted snapshots are **not automatically recalculated**. Existing checkpoints
|
||||
retain accumulated totals, so subsequent increments use the newly loaded factor;
|
||||
a complete historical rebuild requires a separately planned replay. Persisted
|
||||
snapshots do not gain calibration-version provenance through this change.
|
||||
|
||||
## Current Bento provenance
|
||||
|
||||
On 2026-09-08, an independent gravimetric tray measurement found 4068 g/m² at
|
||||
2.3 m/min with actual rotational signals left 1.65 and right 1.75.
|
||||
`(4068 / 1000) × 2.3 / (1.65 + 1.75) ≈ 2.75188 kg/(rev*m)` gives the configured
|
||||
rounded factor **2.75 kg/(rev*m)**. The original measurement remains recorded,
|
||||
without fitting or adjusting the runtime value to reproduce it exactly.
|
||||
|
||||
Only fresh-bentonite spreaders 1 and 2 are included. Before adding spreader 3,
|
||||
confirm its material scope, actual signal units, independent calibration and
|
||||
whether its output belongs in the fresh-material KPIs. No spreader 3 support is
|
||||
included here.
|
||||
+17
-1
@@ -13,7 +13,23 @@ Exploration established a K7 candidate mass-flow signal and speed gate; the
|
||||
evidence and remaining uncertainties are recorded in `docs/enlyze-api.md`.
|
||||
Continue to record sanitized fixtures for newly verified semantics.
|
||||
|
||||
## 2. Live material-consumption MVP (next implementation milestone)
|
||||
## 2. Live material-consumption MVP (in progress)
|
||||
|
||||
The integration engine, ENLYZE gateway, single-cycle polling service, atomic
|
||||
JSON checkpoint store, and configured continuous foreground polling runner are
|
||||
implemented and covered by synthetic tests. Configure the validated K7 instance
|
||||
in `config/k7-material-consumption.yaml`; start it with
|
||||
`production-analytics run material-poll --config config/k7-material-consumption.yaml`.
|
||||
Runtime options select the polling interval, state directory (default: ignored
|
||||
`data/state/material/`), and existing secrets-file handling. Only one writer may
|
||||
poll each machine/order state key.
|
||||
|
||||
The runner now persists cumulative material-consumption snapshots for Grafana in
|
||||
PostgreSQL using `db/schema.sql`. JSON remains the restart checkpoint; PostgreSQL
|
||||
stores derived time-series snapshots only. No Timescale-specific features are used.
|
||||
A new order may first appear with a non-zero total after initial sample integration.
|
||||
Grafana dashboard validation remains pending. No HTTP endpoints, application
|
||||
containers, daemonization, or schedulers have been added.
|
||||
|
||||
Detect the currently active Production Run and production order, continuously
|
||||
ingest new ENLYZE samples, maintain persistent incremental integration state,
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@ readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = { text = "Proprietary" }
|
||||
authors = [{ name = "Production Analytics Team" }]
|
||||
dependencies = []
|
||||
dependencies = ["PyYAML>=6.0", "psycopg[binary]>=3.2,<4", "pymssql>=2.4.0,<3"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
"""Generic, version-controlled calibration values and measurement provenance."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
_UniqueLoader,
|
||||
finite_number,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Calibration:
|
||||
type: str
|
||||
value: float
|
||||
unit: str
|
||||
calibrated_at: date
|
||||
method: str
|
||||
description: str
|
||||
reference: dict[str, Any]
|
||||
|
||||
|
||||
def load_calibrations(path: str | Path) -> dict[str, Calibration]:
|
||||
"""Validate every entry, retaining opaque IDs and free-form reference metadata."""
|
||||
try:
|
||||
with Path(path).open(encoding="utf-8") as stream:
|
||||
document = yaml.load(stream, Loader=_UniqueLoader)
|
||||
except (OSError, UnicodeError) as exc:
|
||||
raise CalculationConfigError(f"Cannot read calibration configuration: {path}") from exc
|
||||
except yaml.YAMLError as exc:
|
||||
raise CalculationConfigError(f"Invalid YAML in calibration configuration: {path}") from exc
|
||||
if not isinstance(document, dict) or set(document) != {"calibrations"}:
|
||||
raise CalculationConfigError("Calibration configuration must contain only 'calibrations'")
|
||||
entries = document["calibrations"]
|
||||
if not isinstance(entries, dict) or not entries:
|
||||
raise CalculationConfigError("calibrations must be a non-empty mapping")
|
||||
result = {}
|
||||
required = {"type", "value", "unit", "calibrated_at", "method", "description", "reference"}
|
||||
for identifier, entry in entries.items():
|
||||
prefix = f"calibrations[{identifier!r}]"
|
||||
if not identifier.strip():
|
||||
raise CalculationConfigError("Calibration IDs must be non-empty strings")
|
||||
if not isinstance(entry, dict) or set(entry) != required:
|
||||
raise CalculationConfigError(
|
||||
f"{prefix}: required fields are {', '.join(sorted(required))}"
|
||||
)
|
||||
for name in ("type", "unit", "method", "description"):
|
||||
if not isinstance(entry[name], str) or not entry[name].strip():
|
||||
raise CalculationConfigError(f"{prefix}.{name} must be a non-empty string")
|
||||
calibrated_at = entry["calibrated_at"]
|
||||
if isinstance(calibrated_at, str):
|
||||
try:
|
||||
parsed = date.fromisoformat(calibrated_at)
|
||||
if parsed.isoformat() != calibrated_at:
|
||||
raise ValueError
|
||||
calibrated_at = parsed
|
||||
except ValueError as exc:
|
||||
raise CalculationConfigError(f"{prefix}.calibrated_at must be YYYY-MM-DD") from exc
|
||||
if type(calibrated_at) is not date:
|
||||
raise CalculationConfigError(f"{prefix}.calibrated_at must be YYYY-MM-DD")
|
||||
if not isinstance(entry["reference"], dict) or not entry["reference"]:
|
||||
raise CalculationConfigError(f"{prefix}.reference must be a non-empty mapping")
|
||||
result[identifier] = Calibration(
|
||||
**{**entry, "calibrated_at": calibrated_at,
|
||||
"value": finite_number(entry["value"], f"{prefix}.value")},
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def resolve_calibration(
|
||||
calibrations: dict[str, Calibration], reference: str, *, expected_type: str, expected_unit: str,
|
||||
) -> Calibration:
|
||||
"""Require explicit compatibility; no implicit conversion or ID interpretation."""
|
||||
if not isinstance(reference, str) or not reference.strip():
|
||||
raise CalculationConfigError("calibration_ref must be a non-empty string")
|
||||
if reference not in calibrations:
|
||||
raise CalculationConfigError(f"Missing calibration ID: {reference!r}")
|
||||
calibration = calibrations[reference]
|
||||
for field, expected in (("type", expected_type), ("unit", expected_unit)):
|
||||
if getattr(calibration, field) != expected:
|
||||
raise CalculationConfigError(
|
||||
f"Calibration {reference!r}: incompatible {field}; expected {expected!r}"
|
||||
)
|
||||
return calibration
|
||||
@@ -0,0 +1,127 @@
|
||||
"""Configuration for reusable channel-level extruder consumption."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import CalculationConfigError, finite_number
|
||||
from production_analytics.service.channel_material import ChannelDefinition
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChannelMaterialCalculationConfig:
|
||||
id: str
|
||||
machine_ref: str
|
||||
max_sample_gap_seconds: float
|
||||
percentage_tolerance: float
|
||||
channels: tuple[ChannelDefinition, ...]
|
||||
material_names: dict[str, str]
|
||||
|
||||
|
||||
def load_channel_material_calculation(path: str | Path) -> ChannelMaterialCalculationConfig:
|
||||
try:
|
||||
document = yaml.safe_load(Path(path).read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeError, yaml.YAMLError) as exc:
|
||||
raise CalculationConfigError("Cannot read channel material configuration") from exc
|
||||
if (
|
||||
not isinstance(document, dict)
|
||||
or set(document) != {"calculations"}
|
||||
or not isinstance(document["calculations"], list)
|
||||
):
|
||||
raise CalculationConfigError("Configuration must contain only calculations")
|
||||
if len(document["calculations"]) != 1:
|
||||
raise CalculationConfigError("Channel configuration requires one calculation")
|
||||
entry = document["calculations"][0]
|
||||
required = {
|
||||
"id",
|
||||
"type",
|
||||
"version",
|
||||
"machine_ref",
|
||||
"max_sample_gap_seconds",
|
||||
"percentage_tolerance",
|
||||
"extruders",
|
||||
}
|
||||
if (
|
||||
not isinstance(entry, dict)
|
||||
or set(entry) - (required | {"material_names"})
|
||||
or required - set(entry)
|
||||
):
|
||||
raise CalculationConfigError("Invalid channel calculation fields")
|
||||
if entry["type"] != "channel_material_consumption" or entry["version"] != "1":
|
||||
raise CalculationConfigError("Unsupported channel calculation type or version")
|
||||
if not all(isinstance(entry[x], str) and entry[x].strip() for x in ("id", "machine_ref")):
|
||||
raise CalculationConfigError("id and machine_ref must be non-empty strings")
|
||||
channels = []
|
||||
extruders = entry["extruders"]
|
||||
if not isinstance(extruders, list) or not extruders:
|
||||
raise CalculationConfigError("extruders must be a non-empty list")
|
||||
for extruder in extruders:
|
||||
if not isinstance(extruder, dict) or set(extruder) != {
|
||||
"id",
|
||||
"total_rate_signal_ref",
|
||||
"channels",
|
||||
}:
|
||||
raise CalculationConfigError("Invalid extruder fields")
|
||||
if not all(
|
||||
isinstance(extruder[x], str) and extruder[x].strip()
|
||||
for x in ("id", "total_rate_signal_ref")
|
||||
):
|
||||
raise CalculationConfigError("Extruder id and total signal must be strings")
|
||||
rows = extruder["channels"]
|
||||
if not isinstance(rows, list) or len(rows) != 7:
|
||||
raise CalculationConfigError("Each extruder requires exactly seven channels")
|
||||
for row in rows:
|
||||
allowed = {
|
||||
"id",
|
||||
"status_signal_ref",
|
||||
"percentage_signal_ref",
|
||||
"screw_speed_signal_ref",
|
||||
"material_number_signal_ref",
|
||||
}
|
||||
if (
|
||||
not isinstance(row, dict)
|
||||
or set(row) - allowed
|
||||
or {"id", "status_signal_ref", "percentage_signal_ref", "screw_speed_signal_ref"}
|
||||
- set(row)
|
||||
):
|
||||
raise CalculationConfigError("Invalid dosing channel fields")
|
||||
if not all(
|
||||
isinstance(row[x], str) and row[x].strip()
|
||||
for x in (
|
||||
"id",
|
||||
"status_signal_ref",
|
||||
"percentage_signal_ref",
|
||||
"screw_speed_signal_ref",
|
||||
)
|
||||
):
|
||||
raise CalculationConfigError("Dosing signal references must be non-empty strings")
|
||||
material = row.get("material_number_signal_ref")
|
||||
if material is not None and (not isinstance(material, str) or not material.strip()):
|
||||
raise CalculationConfigError("material signal must be a string or null")
|
||||
channels.append(
|
||||
ChannelDefinition(
|
||||
extruder["id"],
|
||||
row["id"],
|
||||
extruder["total_rate_signal_ref"],
|
||||
row["status_signal_ref"],
|
||||
row["percentage_signal_ref"],
|
||||
row["screw_speed_signal_ref"],
|
||||
material,
|
||||
)
|
||||
)
|
||||
if len({(c.extruder, c.channel) for c in channels}) != len(channels):
|
||||
raise CalculationConfigError("Extruder/channel identities must be unique")
|
||||
names = entry.get("material_names", {})
|
||||
if not isinstance(names, dict) or any(
|
||||
not isinstance(k, str) or not isinstance(v, str) for k, v in names.items()
|
||||
):
|
||||
raise CalculationConfigError("material_names must map strings to strings")
|
||||
return ChannelMaterialCalculationConfig(
|
||||
entry["id"],
|
||||
entry["machine_ref"],
|
||||
finite_number(entry["max_sample_gap_seconds"], "max_sample_gap_seconds", positive=True),
|
||||
finite_number(entry["percentage_tolerance"], "percentage_tolerance"),
|
||||
tuple(channels),
|
||||
names,
|
||||
)
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Pure channel-level dosing calculations, shared by extruder implementations."""
|
||||
|
||||
from collections.abc import Iterable
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
|
||||
from production_analytics.calculations.material_consumption import MaterialSample
|
||||
|
||||
MATERIAL_ASSIGNED = "ASSIGNED"
|
||||
MATERIAL_UNAVAILABLE = "UNAVAILABLE"
|
||||
MATERIAL_AMBIGUOUS = "AMBIGUOUS"
|
||||
MATERIAL_UNMAPPED = "UNMAPPED"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DosingChannelSample:
|
||||
timestamp: datetime
|
||||
total_rate_kg_per_hour: float
|
||||
channel_status: bool
|
||||
dosing_percentage: float
|
||||
screw_speed: float
|
||||
material_number: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChannelRate:
|
||||
sample: MaterialSample
|
||||
material_number: str | None
|
||||
material_name: str | None
|
||||
material_mapping_status: str
|
||||
|
||||
|
||||
def validate_active_percentage_sum(
|
||||
samples: Iterable[DosingChannelSample], *, tolerance: float
|
||||
) -> bool:
|
||||
"""Return whether active channel percentages close to 100; never normalize them."""
|
||||
values = tuple(samples)
|
||||
if not isfinite(tolerance) or tolerance < 0:
|
||||
raise ValueError("percentage tolerance must be finite and non-negative")
|
||||
total = 0.0
|
||||
for value in values:
|
||||
if not isfinite(value.dosing_percentage):
|
||||
raise ValueError("dosing percentage must be finite")
|
||||
if value.channel_status:
|
||||
total += value.dosing_percentage
|
||||
return abs(total - 100.0) <= tolerance
|
||||
|
||||
|
||||
def channel_rates(
|
||||
samples: Iterable[DosingChannelSample],
|
||||
*,
|
||||
material_names: dict[str, str] | None = None,
|
||||
percentage_tolerance: float = 1.0,
|
||||
) -> tuple[ChannelRate, ...]:
|
||||
"""Derive rates and conservative material identity for one aligned timestamp.
|
||||
|
||||
A material value which occurs on more than one channel is deliberately not
|
||||
attributed to either channel: PLC assignments are not reliable enough to
|
||||
resolve that ambiguity from a previous sample.
|
||||
"""
|
||||
values = tuple(samples)
|
||||
validate_active_percentage_sum(values, tolerance=percentage_tolerance)
|
||||
counts = {
|
||||
material_number: sum(sample.material_number == material_number for sample in values)
|
||||
for material_number in (sample.material_number for sample in values)
|
||||
if material_number is not None
|
||||
}
|
||||
names = material_names or {}
|
||||
result = []
|
||||
for value in values:
|
||||
if not isfinite(value.total_rate_kg_per_hour) or not isfinite(value.screw_speed):
|
||||
raise ValueError("total rate and screw speed must be finite")
|
||||
if value.material_number is None:
|
||||
number, name, status = None, None, MATERIAL_UNAVAILABLE
|
||||
elif counts[value.material_number] > 1:
|
||||
number, name, status = None, None, MATERIAL_AMBIGUOUS
|
||||
elif value.material_number not in names and names:
|
||||
number, name, status = value.material_number, None, MATERIAL_UNMAPPED
|
||||
else:
|
||||
number, name, status = (
|
||||
value.material_number,
|
||||
names.get(value.material_number),
|
||||
MATERIAL_ASSIGNED,
|
||||
)
|
||||
rate = (
|
||||
value.total_rate_kg_per_hour * value.dosing_percentage / 100
|
||||
if value.channel_status
|
||||
else 0.0
|
||||
)
|
||||
result.append(
|
||||
ChannelRate(
|
||||
sample=MaterialSample(value.timestamp, rate, 1.0 if value.channel_status else 0.0),
|
||||
material_number=number,
|
||||
material_name=name,
|
||||
material_mapping_status=status,
|
||||
)
|
||||
)
|
||||
return tuple(result)
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Strict configuration for executable material-consumption instances."""
|
||||
|
||||
from dataclasses import MISSING, dataclass, fields
|
||||
from math import isfinite
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
class CalculationConfigError(ValueError):
|
||||
"""An operator-readable configuration failure, without input values."""
|
||||
|
||||
|
||||
class _UniqueLoader(yaml.SafeLoader):
|
||||
pass
|
||||
|
||||
|
||||
def _mapping(loader, node):
|
||||
result = {}
|
||||
for key_node, value_node in node.value:
|
||||
key = loader.construct_object(key_node)
|
||||
if not isinstance(key, str) or key in result:
|
||||
raise CalculationConfigError("YAML mapping keys must be unique strings")
|
||||
result[key] = loader.construct_object(value_node)
|
||||
return result
|
||||
|
||||
|
||||
_UniqueLoader.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _mapping)
|
||||
|
||||
|
||||
def finite_number(value: object, name: str, *, positive: bool = False) -> float:
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise CalculationConfigError(f"{name} must be a finite number")
|
||||
try:
|
||||
number = float(value)
|
||||
except (OverflowError, ValueError) as exc:
|
||||
raise CalculationConfigError(f"{name} must be a finite number") from exc
|
||||
if not isfinite(number) or (positive and number <= 0):
|
||||
requirement = "finite and greater than zero" if positive else "finite"
|
||||
raise CalculationConfigError(f"{name} must be {requirement}")
|
||||
return number
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaterialCalculationConfig:
|
||||
id: str
|
||||
type: str
|
||||
version: str
|
||||
machine_ref: str
|
||||
rate_signal_ref: str
|
||||
gate_signal_ref: str | None
|
||||
gate_threshold: float
|
||||
max_sample_gap_seconds: float
|
||||
output_metric: str
|
||||
output_unit: str
|
||||
group_by: str
|
||||
source_mode: str = "direct_mass_rate"
|
||||
application_signal_refs: tuple[str, ...] = ()
|
||||
rotational_speed_signal_refs: tuple[str, ...] = ()
|
||||
specific_discharge_kg_per_rev_m: float | None = None
|
||||
calibration_ref: str | None = None
|
||||
erp_workplace: str = ""
|
||||
production_order_format: str = ""
|
||||
process_application_calculation_id: str | None = None
|
||||
|
||||
|
||||
def load_material_calculation(
|
||||
path: str | Path, calculation_id: str | None = None,
|
||||
) -> MaterialCalculationConfig:
|
||||
try:
|
||||
with Path(path).open(encoding="utf-8") as stream:
|
||||
document = yaml.load(stream, Loader=_UniqueLoader)
|
||||
except (OSError, UnicodeError) as exc:
|
||||
raise CalculationConfigError("Cannot read calculation configuration file") from exc
|
||||
except yaml.YAMLError as exc:
|
||||
raise CalculationConfigError("Invalid YAML in calculation configuration") from exc
|
||||
if not isinstance(document, dict) or set(document) != {"calculations"}:
|
||||
raise CalculationConfigError("Configuration must contain only 'calculations'")
|
||||
entries = document["calculations"]
|
||||
if not isinstance(entries, list) or not entries:
|
||||
raise CalculationConfigError("calculations must be a non-empty list")
|
||||
allowed = {field.name for field in fields(MaterialCalculationConfig)}
|
||||
required = {field.name for field in fields(MaterialCalculationConfig)
|
||||
if field.default is MISSING}
|
||||
instances = {}
|
||||
calibrations = None
|
||||
for index, entry in enumerate(entries):
|
||||
prefix = f"calculations[{index}]"
|
||||
if not isinstance(entry, dict):
|
||||
raise CalculationConfigError(f"{prefix} must be a mapping")
|
||||
if "type" in entry and entry["type"] != "material_consumption":
|
||||
raise CalculationConfigError(f"{prefix}.type: only 'material_consumption' is supported")
|
||||
area = entry.get("source_mode", "direct_mass_rate") == "area_application"
|
||||
rotational = entry.get("source_mode") == "rotational_discharge"
|
||||
if rotational:
|
||||
entry.setdefault("gate_signal_ref", None)
|
||||
entry.setdefault("gate_threshold", 0.0)
|
||||
if area or rotational:
|
||||
entry.setdefault("rate_signal_ref", "unused")
|
||||
missing = required - entry.keys()
|
||||
if missing:
|
||||
raise CalculationConfigError(f"{prefix}: missing fields: {', '.join(sorted(missing))}")
|
||||
if entry.keys() - allowed:
|
||||
raise CalculationConfigError(f"{prefix}: unknown fields (check spelling)")
|
||||
for name in sorted(required - {"gate_threshold", "max_sample_gap_seconds"}):
|
||||
if name == "gate_signal_ref" and rotational and entry[name] is None:
|
||||
continue
|
||||
if not isinstance(entry[name], str) or not entry[name].strip():
|
||||
raise CalculationConfigError(f"{prefix}.{name} must be a non-empty string")
|
||||
for name, expected in (
|
||||
("type", "material_consumption"), ("version", "1"),
|
||||
("output_metric", "material_consumption"), ("output_unit", "kg"),
|
||||
("group_by", "production_order"),
|
||||
):
|
||||
if entry[name] != expected:
|
||||
raise CalculationConfigError(f"{prefix}.{name}: only {expected!r} is supported")
|
||||
entry["gate_threshold"] = finite_number(entry["gate_threshold"], f"{prefix}.gate_threshold")
|
||||
entry["max_sample_gap_seconds"] = finite_number(
|
||||
entry["max_sample_gap_seconds"], f"{prefix}.max_sample_gap_seconds", positive=True,
|
||||
)
|
||||
mode = entry.get("source_mode", "direct_mass_rate")
|
||||
if not isinstance(mode, str) or mode not in {
|
||||
"direct_mass_rate", "area_application", "rotational_discharge",
|
||||
}:
|
||||
raise CalculationConfigError("Unsupported material source_mode")
|
||||
if area or rotational:
|
||||
refs_name = "rotational_speed_signal_refs" if rotational else "application_signal_refs"
|
||||
other = "application_signal_refs" if rotational else "rotational_speed_signal_refs"
|
||||
if other in entry:
|
||||
raise CalculationConfigError(f"{mode} cannot specify {other}")
|
||||
refs = entry.get(refs_name)
|
||||
if (not isinstance(refs, list) or not refs
|
||||
or any(not isinstance(ref, str) or not ref.strip() for ref in refs)
|
||||
or len(set(refs)) != len(refs)):
|
||||
raise CalculationConfigError(
|
||||
f"{refs_name} must be unique signal strings"
|
||||
)
|
||||
if entry["rate_signal_ref"] != "unused":
|
||||
raise CalculationConfigError(f"{mode} cannot specify rate_signal_ref")
|
||||
for name in ("erp_workplace", "production_order_format"):
|
||||
if not isinstance(entry.get(name), str) or not entry[name].strip():
|
||||
raise CalculationConfigError(f"{mode} requires {name}")
|
||||
from production_analytics.context import build_enlyze_production_order
|
||||
try:
|
||||
build_enlyze_production_order("0", entry["production_order_format"])
|
||||
except ValueError as exc:
|
||||
raise CalculationConfigError("Invalid production_order_format") from exc
|
||||
entry[refs_name] = tuple(refs)
|
||||
elif any(name in entry for name in (
|
||||
"application_signal_refs", "rotational_speed_signal_refs",
|
||||
"erp_workplace", "production_order_format",
|
||||
)):
|
||||
raise CalculationConfigError("Width context fields require a width-based source mode")
|
||||
if "calibration_ref" in entry:
|
||||
if not rotational:
|
||||
raise CalculationConfigError("calibration_ref requires rotational_discharge")
|
||||
if "specific_discharge_kg_per_rev_m" in entry:
|
||||
raise CalculationConfigError(
|
||||
"Specify calibration_ref or direct specific discharge, not both"
|
||||
)
|
||||
from production_analytics.calculations.calibrations import (
|
||||
load_calibrations,
|
||||
resolve_calibration,
|
||||
)
|
||||
if calibrations is None:
|
||||
calibrations = load_calibrations(Path(path).parent / "material-calibrations.yaml")
|
||||
calibration = resolve_calibration(
|
||||
calibrations, entry["calibration_ref"],
|
||||
expected_type="rotational_discharge", expected_unit="kg_per_rev_m",
|
||||
)
|
||||
entry["specific_discharge_kg_per_rev_m"] = calibration.value
|
||||
if rotational:
|
||||
entry["specific_discharge_kg_per_rev_m"] = finite_number(
|
||||
entry.get("specific_discharge_kg_per_rev_m"),
|
||||
"specific_discharge_kg_per_rev_m", positive=True,
|
||||
)
|
||||
elif "specific_discharge_kg_per_rev_m" in entry:
|
||||
raise CalculationConfigError("Specific discharge requires rotational_discharge")
|
||||
if "process_application_calculation_id" in entry:
|
||||
process_id = entry["process_application_calculation_id"]
|
||||
if (not isinstance(process_id, str) or not process_id.strip()
|
||||
or "\x00" in process_id or process_id == entry["id"]):
|
||||
raise CalculationConfigError(
|
||||
"process application ID must be a distinct non-empty string"
|
||||
)
|
||||
if not rotational or not entry["gate_signal_ref"] or entry["gate_threshold"] <= 0:
|
||||
raise CalculationConfigError(
|
||||
"Process application requires rotational_discharge and a positive speed gate"
|
||||
)
|
||||
if entry["id"] in instances:
|
||||
raise CalculationConfigError("Calculation ids must be unique")
|
||||
instances[entry["id"]] = MaterialCalculationConfig(**entry)
|
||||
process_ids = [c.process_application_calculation_id for c in instances.values()
|
||||
if c.process_application_calculation_id is not None]
|
||||
if len(set(process_ids)) != len(process_ids) or set(process_ids) & instances.keys():
|
||||
raise CalculationConfigError("Calculation ids must be unique across all outputs")
|
||||
if calculation_id is not None:
|
||||
if calculation_id not in instances:
|
||||
raise CalculationConfigError("Requested calculation id was not found")
|
||||
return instances[calculation_id]
|
||||
if len(instances) != 1:
|
||||
raise CalculationConfigError("Multiple calculations: specify --calculation-id")
|
||||
return next(iter(instances.values()))
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Width-independent process application from actual rotational speeds."""
|
||||
|
||||
from collections.abc import Iterable
|
||||
from math import isfinite
|
||||
|
||||
|
||||
def rotational_application_g_m2(
|
||||
rotational_speeds_rpm: Iterable[float], specific_discharge_kg_per_rev_m: float,
|
||||
line_speed_m_min: float, gate_threshold: float,
|
||||
) -> float | None:
|
||||
speeds = tuple(rotational_speeds_rpm)
|
||||
if not speeds or not all(isfinite(value) for value in speeds):
|
||||
raise ValueError("rotational speeds must be non-empty and finite")
|
||||
if not isfinite(specific_discharge_kg_per_rev_m) or specific_discharge_kg_per_rev_m <= 0:
|
||||
raise ValueError("specific discharge must be finite and positive")
|
||||
if not isfinite(gate_threshold) or gate_threshold <= 0:
|
||||
raise ValueError("speed gate threshold must be finite and positive")
|
||||
if not isfinite(line_speed_m_min):
|
||||
raise ValueError("line speed must be finite")
|
||||
if line_speed_m_min <= gate_threshold:
|
||||
return None
|
||||
value = sum(speeds) * specific_discharge_kg_per_rev_m * 1000 / line_speed_m_min
|
||||
if not isfinite(value):
|
||||
raise ValueError("derived application must be finite")
|
||||
return value
|
||||
@@ -13,6 +13,7 @@ class MaterialSample:
|
||||
timestamp: datetime
|
||||
material_rate_kg_per_hour: float
|
||||
gate_value: float
|
||||
application_g_m2: float | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -102,3 +103,38 @@ class MaterialConsumptionIntegrator:
|
||||
for sample in samples:
|
||||
self.process(sample)
|
||||
return self.state
|
||||
|
||||
|
||||
def area_application_rate_kg_per_hour(
|
||||
applications_g_m2: Iterable[float], nominal_width_m: float, line_speed_m_min: float,
|
||||
) -> float:
|
||||
"""Normalize summed area application to the integrator's canonical kg/h."""
|
||||
applications = tuple(applications_g_m2)
|
||||
if not applications or not all(isfinite(value) for value in applications):
|
||||
raise ValueError("application values must be non-empty and finite")
|
||||
if not isfinite(nominal_width_m) or nominal_width_m <= 0:
|
||||
raise ValueError("nominal width must be finite and positive")
|
||||
if not isfinite(line_speed_m_min):
|
||||
raise ValueError("line speed must be finite")
|
||||
rate = sum(applications) * nominal_width_m * line_speed_m_min * 60 / 1000
|
||||
if not isfinite(rate):
|
||||
raise ValueError("derived material rate must be finite")
|
||||
return rate
|
||||
|
||||
|
||||
def rotational_discharge_rate_kg_per_hour(
|
||||
rotational_speeds_rpm: Iterable[float], nominal_width_m: float,
|
||||
specific_discharge_kg_per_rev_m: float,
|
||||
) -> float:
|
||||
"""Convert full-width roll speeds and specific discharge to canonical kg/h."""
|
||||
speeds = tuple(rotational_speeds_rpm)
|
||||
if not speeds or not all(isfinite(value) for value in speeds):
|
||||
raise ValueError("rotational speeds must be non-empty and finite")
|
||||
if not isfinite(nominal_width_m) or nominal_width_m <= 0:
|
||||
raise ValueError("nominal width must be finite and positive")
|
||||
if not isfinite(specific_discharge_kg_per_rev_m) or specific_discharge_kg_per_rev_m <= 0:
|
||||
raise ValueError("specific discharge must be finite and positive")
|
||||
rate = sum(speeds) * nominal_width_m * specific_discharge_kg_per_rev_m * 60
|
||||
if not isfinite(rate):
|
||||
raise ValueError("derived material rate must be finite")
|
||||
return rate
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Serialization helpers for persistent material-integration state."""
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from math import isfinite
|
||||
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialIntegrationState,
|
||||
@@ -9,6 +10,11 @@ from production_analytics.calculations.material_consumption import (
|
||||
|
||||
def material_state_to_dict(state: MaterialIntegrationState) -> dict[str, object]:
|
||||
"""Convert integration state to a JSON-serializable dictionary."""
|
||||
if state.last_processed_timestamp is not None and (
|
||||
state.last_processed_timestamp.tzinfo is None
|
||||
or state.last_processed_timestamp.utcoffset() is None
|
||||
):
|
||||
raise ValueError("last_processed_timestamp must be timezone-aware")
|
||||
return {
|
||||
"cumulative_consumption_kg": state.cumulative_consumption_kg,
|
||||
"integrated_running_seconds": state.integrated_running_seconds,
|
||||
@@ -25,7 +31,33 @@ def material_state_to_dict(state: MaterialIntegrationState) -> dict[str, object]
|
||||
|
||||
def material_state_from_dict(data: dict[str, object]) -> MaterialIntegrationState:
|
||||
"""Restore integration state from a JSON-compatible dictionary."""
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("integration_state must be an object")
|
||||
for key in (
|
||||
"cumulative_consumption_kg", "integrated_running_seconds",
|
||||
"last_material_rate_kg_per_hour", "last_gate_value",
|
||||
):
|
||||
value = data[key]
|
||||
if value is None and key.startswith("last_"):
|
||||
continue
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)) or not isfinite(value):
|
||||
raise ValueError(f"{key} must be a finite number")
|
||||
if data["integrated_running_seconds"] < 0:
|
||||
raise ValueError("integrated_running_seconds must not be negative")
|
||||
if not isinstance(data["integration_active"], bool):
|
||||
raise ValueError("integration_active must be a boolean")
|
||||
raw_timestamp = data["last_processed_timestamp"]
|
||||
if raw_timestamp is not None:
|
||||
if not isinstance(raw_timestamp, str):
|
||||
raise ValueError("last_processed_timestamp must be an ISO timestamp")
|
||||
parsed = datetime.fromisoformat(raw_timestamp)
|
||||
if parsed.tzinfo is None or parsed.utcoffset() is None:
|
||||
raise ValueError("last_processed_timestamp must be timezone-aware")
|
||||
if data["integration_active"] and (
|
||||
raw_timestamp is None or data["last_material_rate_kg_per_hour"] is None
|
||||
or data["last_gate_value"] is None
|
||||
):
|
||||
raise ValueError("active integration requires a complete temporal baseline")
|
||||
|
||||
timestamp = (
|
||||
datetime.fromisoformat(str(raw_timestamp)).astimezone(UTC)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Command-line entry point for safe, read-only API exploration."""
|
||||
"""Command-line entry point for exploration and production polling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -37,10 +37,18 @@ def _parser() -> argparse.ArgumentParser:
|
||||
raw.add_argument("--query", action="append", type=_query_item, default=[], metavar="KEY=VALUE")
|
||||
raw.add_argument("--pretty", action="store_true", help="Pretty-print sanitized JSON")
|
||||
raw.add_argument("--verbose", action="store_true", help="Show safe request progress on stderr")
|
||||
raw.add_argument("--save-fixture", metavar="NAME", help="Save sanitized JSON under fixtures/enlyze/")
|
||||
raw.add_argument("--save-raw", metavar="NAME", help="Save unmodified response under ignored data/raw/enlyze/")
|
||||
raw.add_argument("--fixture-dir", type=Path, default=Path("fixtures/enlyze"), help=argparse.SUPPRESS)
|
||||
raw.add_argument("--raw-dir", type=Path, default=Path("data/raw/enlyze"), help=argparse.SUPPRESS)
|
||||
raw.add_argument(
|
||||
"--save-fixture", metavar="NAME", help="Save sanitized JSON under fixtures/enlyze/"
|
||||
)
|
||||
raw.add_argument(
|
||||
"--save-raw", metavar="NAME", help="Save unmodified response under ignored data/raw/enlyze/"
|
||||
)
|
||||
raw.add_argument(
|
||||
"--fixture-dir", type=Path, default=Path("fixtures/enlyze"), help=argparse.SUPPRESS
|
||||
)
|
||||
raw.add_argument(
|
||||
"--raw-dir", type=Path, default=Path("data/raw/enlyze"), help=argparse.SUPPRESS
|
||||
)
|
||||
raw.add_argument(
|
||||
"--secrets-file",
|
||||
type=Path,
|
||||
@@ -52,24 +60,93 @@ def _parser() -> argparse.ArgumentParser:
|
||||
timeseries.add_argument("--start", required=True, help="ISO 8601 datetime with timezone")
|
||||
timeseries.add_argument("--end", required=True, help="ISO 8601 datetime with timezone")
|
||||
timeseries.add_argument("--variable", required=True, help="Variable UUID")
|
||||
timeseries.add_argument("--resampling-interval", type=int, help="Seconds; schema range is 10..604800")
|
||||
timeseries.add_argument(
|
||||
"--resampling-interval", type=int, help="Seconds; schema range is 10..604800"
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--resampling-method",
|
||||
choices=["first", "last", "max", "min", "count", "sum", "avg", "median", "std", "q5", "q25", "q75", "q95"],
|
||||
choices=[
|
||||
"first",
|
||||
"last",
|
||||
"max",
|
||||
"min",
|
||||
"count",
|
||||
"sum",
|
||||
"avg",
|
||||
"median",
|
||||
"std",
|
||||
"q5",
|
||||
"q25",
|
||||
"q75",
|
||||
"q95",
|
||||
],
|
||||
help="Optional schema-defined method for this variable",
|
||||
)
|
||||
timeseries.add_argument("--pretty", action="store_true", help="Pretty-print sanitized JSON")
|
||||
timeseries.add_argument("--verbose", action="store_true", help="Show safe request progress on stderr")
|
||||
timeseries.add_argument("--save-fixture", metavar="NAME", help="Save sanitized JSON under fixtures/enlyze/")
|
||||
timeseries.add_argument("--save-raw", metavar="NAME", help="Save unmodified response under ignored data/raw/enlyze/")
|
||||
timeseries.add_argument("--fixture-dir", type=Path, default=Path("fixtures/enlyze"), help=argparse.SUPPRESS)
|
||||
timeseries.add_argument("--raw-dir", type=Path, default=Path("data/raw/enlyze"), help=argparse.SUPPRESS)
|
||||
timeseries.add_argument(
|
||||
"--verbose", action="store_true", help="Show safe request progress on stderr"
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--save-fixture", metavar="NAME", help="Save sanitized JSON under fixtures/enlyze/"
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--save-raw", metavar="NAME", help="Save unmodified response under ignored data/raw/enlyze/"
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--fixture-dir", type=Path, default=Path("fixtures/enlyze"), help=argparse.SUPPRESS
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--raw-dir", type=Path, default=Path("data/raw/enlyze"), help=argparse.SUPPRESS
|
||||
)
|
||||
timeseries.add_argument(
|
||||
"--secrets-file",
|
||||
type=Path,
|
||||
default=Path("secrets/enlyze.env"),
|
||||
help="Local dotenv-style secret file (default: secrets/enlyze.env)",
|
||||
)
|
||||
run = namespaces.add_parser("run", help="Foreground production runners")
|
||||
runners = run.add_subparsers(dest="command", required=True)
|
||||
material = runners.add_parser("material-poll", help="Continuously poll material consumption")
|
||||
material.add_argument("--config", type=Path, required=True)
|
||||
material.add_argument("--calculation-id")
|
||||
material.add_argument("--poll-interval-seconds", type=float, default=10.0)
|
||||
material.add_argument("--state-directory", type=Path, default=Path("data/state/material"))
|
||||
material.add_argument("--secrets-file", type=Path, default=Path("secrets/enlyze.env"))
|
||||
material.add_argument("--erp-secrets-file", type=Path, default=Path("secrets/erp.env"))
|
||||
power = runners.add_parser("power-meter", help="Collect configured ENLYZE power-meter telemetry")
|
||||
power.add_argument("--config", type=Path, required=True)
|
||||
power.add_argument("--meter")
|
||||
power.add_argument("--start", help="ISO timestamp; defaults to one polling window ago")
|
||||
power.add_argument("--end", help="ISO timestamp; defaults to now")
|
||||
power.add_argument("--follow", action="store_true", help="Keep polling until interrupted")
|
||||
power.add_argument("--secrets-file", type=Path, default=Path("secrets/enlyze.env"))
|
||||
power.add_argument("--db-secrets-file", type=Path, default=Path("secrets/postgres.env"))
|
||||
report = namespaces.add_parser("report", help="Generate independent reports")
|
||||
report_commands = report.add_subparsers(dest="command", required=True)
|
||||
monthly = report_commands.add_parser("monthly", help="Render one configured monthly PDF")
|
||||
monthly.add_argument("--config", type=Path, required=True)
|
||||
monthly.add_argument("--meter")
|
||||
monthly.add_argument("--month", required=True, help="YYYY-MM")
|
||||
monthly.add_argument("--db-secrets-file", type=Path, default=Path("secrets/postgres.env"))
|
||||
|
||||
channel_material = runners.add_parser(
|
||||
"channel-material-poll", help="Continuously poll channel-level material consumption"
|
||||
)
|
||||
channel_material.add_argument("--config", type=Path, required=True)
|
||||
channel_material.add_argument("--poll-interval-seconds", type=float, default=10.0)
|
||||
channel_material.add_argument(
|
||||
"--state-directory", type=Path, default=Path("data/state/channel-material")
|
||||
)
|
||||
channel_material.add_argument("--secrets-file", type=Path, default=Path("secrets/enlyze.env"))
|
||||
channel_material.add_argument("--once", action="store_true", help="Run one poll cycle and exit")
|
||||
efficiency = runners.add_parser("material-efficiency", help="Persist ERP feedback KPI points")
|
||||
efficiency.add_argument("--config", type=Path, required=True)
|
||||
efficiency.add_argument("--secrets-file", type=Path, default=Path("secrets/enlyze.env"))
|
||||
efficiency.add_argument("--erp-secrets-file", type=Path, default=Path("secrets/erp.env"))
|
||||
downtime = runners.add_parser("downtime-reconcile", help="Reconcile ENLYZE downtime events")
|
||||
downtime.add_argument("--config", type=Path, required=True)
|
||||
downtime.add_argument("--secrets-file", type=Path, default=Path("secrets/enlyze.env"))
|
||||
downtime.add_argument("--erp-secrets-file", type=Path, default=Path("secrets/erp.env"))
|
||||
return parser
|
||||
|
||||
|
||||
@@ -83,8 +160,183 @@ def _write_fixture(directory: Path, name: str, payload: object) -> Path:
|
||||
return destination
|
||||
|
||||
|
||||
def _run_material(args: argparse.Namespace) -> int:
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.service.material_runtime import build_material_runner
|
||||
|
||||
try:
|
||||
calculation = load_material_calculation(args.config, args.calculation_id)
|
||||
runner = build_material_runner(
|
||||
calculation,
|
||||
poll_interval_seconds=args.poll_interval_seconds,
|
||||
state_directory=args.state_directory,
|
||||
erp_secrets_file=args.erp_secrets_file,
|
||||
secrets_file=args.secrets_file,
|
||||
)
|
||||
runner.run()
|
||||
except (CalculationConfigError, ConfigurationError) as exc:
|
||||
print(f"Config/startup error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
except Exception as exc:
|
||||
print(f"Startup/runtime error: {type(exc).__name__}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
return 0
|
||||
return 0
|
||||
|
||||
|
||||
def _run_channel_material(args: argparse.Namespace) -> int:
|
||||
from production_analytics.calculations.channel_config import load_channel_material_calculation
|
||||
from production_analytics.calculations.config import CalculationConfigError
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.service.channel_material_runtime import build_channel_material_runner
|
||||
|
||||
try:
|
||||
calculation = load_channel_material_calculation(args.config)
|
||||
runner = build_channel_material_runner(
|
||||
calculation,
|
||||
poll_interval_seconds=args.poll_interval_seconds,
|
||||
state_directory=args.state_directory,
|
||||
secrets_file=args.secrets_file,
|
||||
)
|
||||
if args.once:
|
||||
runner.run_once()
|
||||
else:
|
||||
runner.run()
|
||||
except (CalculationConfigError, ConfigurationError) as exc:
|
||||
print(f"Config/startup error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
except KeyboardInterrupt:
|
||||
return 0
|
||||
except Exception as exc:
|
||||
print(f"Startup/runtime error: {type(exc).__name__}", file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def _run_material_efficiency(args: argparse.Namespace) -> int:
|
||||
from production_analytics.calculations.config import CalculationConfigError
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.service.material_efficiency_runtime import (
|
||||
build_material_efficiency_runner,
|
||||
)
|
||||
|
||||
try:
|
||||
build_material_efficiency_runner(
|
||||
args.config,
|
||||
secrets_file=args.secrets_file,
|
||||
erp_secrets_file=args.erp_secrets_file,
|
||||
).run()
|
||||
except (CalculationConfigError, ConfigurationError) as exc:
|
||||
print(f"Config/startup error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
except Exception as exc:
|
||||
print(f"Startup/runtime error: {type(exc).__name__}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
return 0
|
||||
return 0
|
||||
|
||||
|
||||
def _runtime_environment(*paths: Path) -> dict[str, str]:
|
||||
environment = dict(os.environ)
|
||||
for path in paths:
|
||||
environment.update(load_secret_file(path))
|
||||
return environment
|
||||
|
||||
|
||||
def _parse_cli_timestamp(value: str) -> object:
|
||||
from datetime import datetime
|
||||
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
if parsed.tzinfo is None:
|
||||
raise ExplorationError("timestamps must include a timezone")
|
||||
return parsed
|
||||
|
||||
|
||||
def _run_power_meter(args: argparse.Namespace) -> int:
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import time
|
||||
from production_analytics.calculations.config import CalculationConfigError
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.power_meter.config import load_power_meter_config
|
||||
from production_analytics.power_meter.gateway import PowerMeterGateway
|
||||
from production_analytics.power_meter.repository import PostgresPowerMeterRepository
|
||||
from production_analytics.power_meter.runtime import PowerMeterCollector
|
||||
try:
|
||||
config = load_power_meter_config(args.config, args.meter)
|
||||
environment = _runtime_environment(args.secrets_file, args.db_secrets_file)
|
||||
settings = ExplorationSettings.from_environment(environment)
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
db = PostgresSettings.from_environment(environment)
|
||||
collector = PowerMeterCollector(config, PowerMeterGateway(ExplorationClient(settings)),
|
||||
PostgresPowerMeterRepository(db))
|
||||
if args.follow:
|
||||
while True:
|
||||
count = collector.poll_once()
|
||||
print(f"Collected {count} {config.name} power-meter readings.", flush=True)
|
||||
time.sleep(config.poll_interval_seconds)
|
||||
now = datetime.now(UTC)
|
||||
end = _parse_cli_timestamp(args.end) if args.end else now
|
||||
start = (_parse_cli_timestamp(args.start) if args.start else
|
||||
end - timedelta(seconds=config.poll_interval_seconds * 2))
|
||||
count = collector.backfill(start, end)
|
||||
print(f"Collected {count} {config.name} power-meter readings.")
|
||||
return 0
|
||||
except (CalculationConfigError, ConfigurationError, ExplorationError, ValueError) as exc:
|
||||
print(f"Config/startup error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
def _run_monthly_report(args: argparse.Namespace) -> int:
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.power_meter.config import load_power_meter_config
|
||||
from production_analytics.power_meter.repository import PostgresPowerMeterRepository
|
||||
from production_analytics.power_meter.runtime import monthly_report
|
||||
try:
|
||||
config = load_power_meter_config(args.config, args.meter)
|
||||
environment = _runtime_environment(args.db_secrets_file)
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
path = monthly_report(config, PostgresPowerMeterRepository(PostgresSettings.from_environment(environment)), args.month)
|
||||
print(path)
|
||||
return 0
|
||||
except (CalculationConfigError, ConfigurationError, ValueError, RuntimeError) as exc:
|
||||
print(f"Report error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
def _run_downtime(args: argparse.Namespace) -> int:
|
||||
from production_analytics.service.downtime_runtime import build_downtime_runner
|
||||
|
||||
try:
|
||||
build_downtime_runner(
|
||||
args.config,
|
||||
secrets_file=args.secrets_file,
|
||||
erp_secrets_file=args.erp_secrets_file,
|
||||
).run()
|
||||
except Exception as exc:
|
||||
print(f"Config/startup/runtime error: {type(exc).__name__}", file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv: Sequence[str] | None = None) -> int:
|
||||
args = _parser().parse_args(argv)
|
||||
if args.namespace == "run":
|
||||
if args.command == "channel-material-poll":
|
||||
return _run_channel_material(args)
|
||||
if args.command == "material-efficiency":
|
||||
return _run_material_efficiency(args)
|
||||
if args.command == "power-meter":
|
||||
return _run_power_meter(args)
|
||||
if args.command == "downtime-reconcile":
|
||||
return _run_downtime(args)
|
||||
return _run_material(args)
|
||||
if args.namespace == "report":
|
||||
return _run_monthly_report(args)
|
||||
if args.namespace != "enlyze" or args.command not in {"raw", "timeseries"}:
|
||||
return 2
|
||||
|
||||
@@ -97,16 +349,33 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
operation = "GET" if args.command == "raw" else "POST"
|
||||
target = args.path if args.command == "raw" else "/v2/timeseries"
|
||||
print(
|
||||
f"Requesting {operation} {target} (timeout={settings.timeout_seconds:g}s; authentication {authentication}).",
|
||||
f"Requesting {operation} {target} (timeout={settings.timeout_seconds:g}s; "
|
||||
f"authentication {authentication}).",
|
||||
file=sys.stderr,
|
||||
)
|
||||
client = ExplorationClient(settings)
|
||||
if args.command == "raw":
|
||||
response = client.get(args.path, dict(args.query))
|
||||
query = dict(args.query)
|
||||
if args.path == "/v2/variables":
|
||||
pages = client.get_all_pages(args.path, query)
|
||||
response = pages[0]
|
||||
if len(pages) > 1:
|
||||
merged = dict(response.body)
|
||||
merged["data"] = [item for page in pages for item in page.body.get("data", [])]
|
||||
merged["metadata"] = {"next_cursor": None, "pages": len(pages)}
|
||||
response = type(response)(response.status_code, response.path,
|
||||
response.headers, merged)
|
||||
else:
|
||||
response = client.get(args.path, query)
|
||||
request = {"method": "GET", "path": response.path}
|
||||
else:
|
||||
if args.resampling_interval is not None and not 10 <= args.resampling_interval <= 604800:
|
||||
raise ExplorationError("--resampling-interval must be between 10 and 604800 seconds.")
|
||||
if (
|
||||
args.resampling_interval is not None
|
||||
and not 10 <= args.resampling_interval <= 604800
|
||||
):
|
||||
raise ExplorationError(
|
||||
"--resampling-interval must be between 10 and 604800 seconds."
|
||||
)
|
||||
variable: dict[str, str] = {"uuid": args.variable}
|
||||
if args.resampling_method:
|
||||
variable["resampling_method"] = args.resampling_method
|
||||
@@ -134,9 +403,15 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
indent = 2 if args.pretty or args.save_fixture else None
|
||||
print(json.dumps(payload, indent=indent, sort_keys=True))
|
||||
if args.save_fixture:
|
||||
print(f"Sanitized fixture written to {_write_fixture(args.fixture_dir, args.save_fixture, payload)}")
|
||||
print(
|
||||
"Sanitized fixture written to "
|
||||
f"{_write_fixture(args.fixture_dir, args.save_fixture, payload)}"
|
||||
)
|
||||
if args.save_raw:
|
||||
print(f"Raw response written to {_write_fixture(args.raw_dir, args.save_raw, raw_payload)}")
|
||||
print(
|
||||
"Raw response written to "
|
||||
f"{_write_fixture(args.raw_dir, args.save_raw, raw_payload)}"
|
||||
)
|
||||
except ExplorationError as error:
|
||||
print(f"ENLYZE exploration error: {error}")
|
||||
return 1
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Deterministic ERP context helpers, independent of database and timeseries access."""
|
||||
|
||||
import math
|
||||
import re
|
||||
|
||||
__all__ = ["build_enlyze_production_order", "extract_nominal_width_m"]
|
||||
|
||||
# Capture malformed numeric tokens too, so a valid-looking suffix cannot become width.
|
||||
_NUMBER_TOKEN = r"[+-]?(?:[0-9]+(?:[.,][0-9]+)*|inf(?:inity)?|nan)(?:[eE][+-]?[0-9]+)?"
|
||||
_DIMENSIONS = re.compile(
|
||||
rf"(?<![\w.,+\-/])(?P<width>{_NUMBER_TOKEN})\s*x\s*"
|
||||
rf"(?P<length>{_NUMBER_TOKEN})\s*m(?![\w/²³^])",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_PLAIN_NUMBER = re.compile(r"[0-9]+(?:[.,][0-9]+)?")
|
||||
|
||||
|
||||
def build_enlyze_production_order(erp_production_order: str, format_template: str) -> str:
|
||||
"""Format a single ERP order using explicit caller-supplied configuration.
|
||||
|
||||
The template must contain exactly one literal {production_order} placeholder
|
||||
and no other braces. Outer order whitespace is stripped; the order
|
||||
must otherwise contain ASCII digits only (leading zeros are preserved).
|
||||
This is not a parser for ENLYZE identifiers, combined or otherwise.
|
||||
"""
|
||||
order = erp_production_order.strip()
|
||||
if not order or not order.isascii() or not order.isdigit():
|
||||
raise ValueError("ERP production order must be a non-empty ASCII digit string")
|
||||
placeholder = "{production_order}"
|
||||
literal = format_template.replace(placeholder, "")
|
||||
if format_template.count(placeholder) != 1 or "{" in literal or "}" in literal:
|
||||
raise ValueError("Format template must contain exactly one {production_order} placeholder")
|
||||
return format_template.replace(placeholder, order)
|
||||
|
||||
|
||||
def extract_nominal_width_m(article_description: str | None) -> float | None:
|
||||
"""Read one unambiguous positive '<width> x <length> m' pair from ERP text.
|
||||
|
||||
Decimal comma/point and variable whitespace are supported. Multiple pairs,
|
||||
dimension chains, signed/scientific notation and invalid dimensions fail
|
||||
closed.
|
||||
"""
|
||||
if article_description is None:
|
||||
return None
|
||||
matches = list(_DIMENSIONS.finditer(article_description))
|
||||
if len(matches) != 1:
|
||||
return None
|
||||
match = matches[0]
|
||||
# Do not mistake the tail of a three-dimensional expression for a width pair.
|
||||
if re.search(r"[x×]\s*$", article_description[:match.start()], re.IGNORECASE):
|
||||
return None
|
||||
if re.match(r"\s*[x×]", article_description[match.end():], re.IGNORECASE):
|
||||
return None
|
||||
values = []
|
||||
for token in (match["width"], match["length"]):
|
||||
if _PLAIN_NUMBER.fullmatch(token) is None:
|
||||
return None
|
||||
value = float(token.replace(",", "."))
|
||||
if not math.isfinite(value) or value <= 0:
|
||||
return None
|
||||
values.append(value)
|
||||
return values[0]
|
||||
@@ -21,12 +21,20 @@ class ConfigurationError(ExplorationError):
|
||||
|
||||
|
||||
class AuthenticationError(ExplorationError):
|
||||
"""Raised for HTTP 401/403 without exposing credentials or response bodies."""
|
||||
"""Raised for HTTP 401/403 with a bounded, sanitized response body."""
|
||||
|
||||
def __init__(self, message: str, *, response_body: object | None = None) -> None:
|
||||
super().__init__(message)
|
||||
self.response_body = response_body
|
||||
|
||||
|
||||
class HttpResponseError(ExplorationError):
|
||||
"""Raised for a non-successful HTTP response."""
|
||||
|
||||
def __init__(self, message: str, *, response_body: object | None = None) -> None:
|
||||
super().__init__(message)
|
||||
self.response_body = response_body
|
||||
|
||||
|
||||
class NonJsonResponseError(ExplorationError):
|
||||
"""Raised when a successful response cannot be decoded as JSON."""
|
||||
@@ -82,6 +90,30 @@ class ExplorationClient:
|
||||
"""Issue a documented read-only POST operation with a JSON body."""
|
||||
return self._request_json("POST", path, payload=payload)
|
||||
|
||||
def get_all_pages(
|
||||
self, path: str, query: Mapping[str, str] | None = None,
|
||||
) -> list[ExplorationResponse]:
|
||||
"""Follow the ENLYZE cursor contract and return every response page."""
|
||||
request_query = dict(query or {})
|
||||
pages: list[ExplorationResponse] = []
|
||||
followed: set[str] = set()
|
||||
while True:
|
||||
response = self.get(path, request_query)
|
||||
pages.append(response)
|
||||
body = response.body
|
||||
if not isinstance(body, dict) or "metadata" not in body:
|
||||
return pages
|
||||
metadata = body["metadata"]
|
||||
if not isinstance(metadata, dict) or "next_cursor" not in metadata:
|
||||
raise ExplorationError("Invalid paginated response: metadata.next_cursor is missing.")
|
||||
cursor = metadata["next_cursor"]
|
||||
if cursor is None:
|
||||
return pages
|
||||
if not isinstance(cursor, str) or not cursor or cursor in followed:
|
||||
raise ExplorationError("Invalid paginated response: repeated or invalid cursor.")
|
||||
followed.add(cursor)
|
||||
request_query["cursor"] = cursor
|
||||
|
||||
def _request_json(
|
||||
self,
|
||||
method: str,
|
||||
@@ -118,9 +150,27 @@ class ExplorationClient:
|
||||
status_code = response.status
|
||||
response_headers = dict(response.headers.items())
|
||||
except HTTPError as error:
|
||||
response_body: object | None = None
|
||||
try:
|
||||
raw_error_body = error.read(8192)
|
||||
try:
|
||||
from production_analytics.enlyze.sanitize import sanitize
|
||||
response_body = sanitize(json.loads(raw_error_body))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||
response_body = raw_error_body.decode("utf-8", errors="replace")[:2000]
|
||||
except OSError:
|
||||
response_body = None
|
||||
if error.code in {401, 403}:
|
||||
raise AuthenticationError(f"Authentication or authorization failed (HTTP {error.code}).") from error
|
||||
raise HttpResponseError(f"HTTP request failed with status {error.code}.") from error
|
||||
suffix = f" Response: {response_body!r}" if response_body is not None else ""
|
||||
raise AuthenticationError(
|
||||
f"Authentication or authorization failed (HTTP {error.code}).{suffix}",
|
||||
response_body=response_body,
|
||||
) from error
|
||||
suffix = f" Response: {response_body!r}" if response_body is not None else ""
|
||||
raise HttpResponseError(
|
||||
f"HTTP request failed with status {error.code}.{suffix}",
|
||||
response_body=response_body,
|
||||
) from error
|
||||
except URLError as error:
|
||||
raise HttpResponseError("HTTP request could not be completed.") from error
|
||||
|
||||
|
||||
@@ -2,10 +2,20 @@
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from math import isfinite
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from production_analytics.calculations.material_consumption import MaterialSample
|
||||
from production_analytics.calculations.channel_material import DosingChannelSample
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialSample,
|
||||
area_application_rate_kg_per_hour,
|
||||
rotational_discharge_rate_kg_per_hour,
|
||||
)
|
||||
from production_analytics.enlyze.exploration import ExplorationClient
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from production_analytics.service.channel_material import ChannelDefinition
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class EnlyzeProductionRun:
|
||||
@@ -17,73 +27,514 @@ class EnlyzeProductionRun:
|
||||
end: datetime | None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class EnlyzeDowntime:
|
||||
"""ENLYZE downtime source record. ``end`` and ``reason`` are deliberately nullable."""
|
||||
|
||||
uuid: str
|
||||
machine_id: str
|
||||
source_type: str
|
||||
start: datetime
|
||||
end: datetime | None
|
||||
comment: str | None
|
||||
reason_id: str | None
|
||||
reason_name: str | None
|
||||
reason_description: str | None
|
||||
reason_group: str | None
|
||||
reason_category: str | None
|
||||
source_updated_at: datetime | None
|
||||
|
||||
|
||||
class EnlyzeApiGateway:
|
||||
def __init__(self, client: ExplorationClient) -> None:
|
||||
self._client = client
|
||||
|
||||
def get_open_production_run(self, machine_id: str) -> EnlyzeProductionRun | None:
|
||||
response = self._client.get(
|
||||
"/v2/production-runs",
|
||||
{"machine": machine_id},
|
||||
)
|
||||
|
||||
for item in response.body.get("data", []):
|
||||
if item.get("end") is not None:
|
||||
continue
|
||||
|
||||
return EnlyzeProductionRun(
|
||||
uuid=str(item["uuid"]),
|
||||
machine_id=str(item["machine"]),
|
||||
product_id=(
|
||||
str(item["product"])
|
||||
if item.get("product") is not None
|
||||
else None
|
||||
),
|
||||
production_order=str(item["production_order"]),
|
||||
start=_parse_timestamp(item["start"]),
|
||||
end=None,
|
||||
runs = [run for run in self.get_production_runs(machine_id) if run.end is None]
|
||||
if len(runs) > 1:
|
||||
raise ValueError(
|
||||
"Invalid production-run response: multiple open Production Runs returned"
|
||||
)
|
||||
return runs[0] if runs else None
|
||||
|
||||
return None
|
||||
def get_production_runs(self, machine_id: str) -> list[EnlyzeProductionRun]:
|
||||
"""Retrieve all machine runs, following validated cursor pagination."""
|
||||
params = {"machine": machine_id}
|
||||
runs = []
|
||||
followed_cursors: set[str] = set()
|
||||
page = 1
|
||||
while True:
|
||||
response = self._client.get("/v2/production-runs", params)
|
||||
try:
|
||||
if not isinstance(response.body, dict):
|
||||
raise ValueError("body must be an object")
|
||||
items = response.body["data"]
|
||||
if not isinstance(items, list):
|
||||
raise ValueError("data must be a list")
|
||||
for item in items:
|
||||
if not isinstance(item, dict) or "end" not in item:
|
||||
raise ValueError("run must be an object with an end field")
|
||||
for key in ("uuid", "machine", "production_order"):
|
||||
if not isinstance(item[key], str) or not item[key]:
|
||||
raise ValueError(f"{key} must be a non-empty string")
|
||||
if item["machine"] != machine_id:
|
||||
raise ValueError("run machine does not match requested machine")
|
||||
start = _parse_timestamp(item["start"])
|
||||
end = _parse_timestamp(item["end"]) if item["end"] is not None else None
|
||||
if end is not None and end < start:
|
||||
raise ValueError("run end must not precede start")
|
||||
runs.append(
|
||||
EnlyzeProductionRun(
|
||||
uuid=item["uuid"],
|
||||
machine_id=item["machine"],
|
||||
product_id=item.get("product"),
|
||||
production_order=item["production_order"],
|
||||
start=start,
|
||||
end=end,
|
||||
)
|
||||
)
|
||||
next_cursor = None
|
||||
if "metadata" in response.body:
|
||||
metadata = response.body["metadata"]
|
||||
if not isinstance(metadata, dict) or "next_cursor" not in metadata:
|
||||
raise ValueError("metadata must be an object with a next_cursor field")
|
||||
next_cursor = metadata["next_cursor"]
|
||||
if next_cursor is not None and (
|
||||
not isinstance(next_cursor, str) or not next_cursor
|
||||
):
|
||||
raise ValueError("next_cursor must be null or a non-empty string")
|
||||
if next_cursor is None:
|
||||
return runs
|
||||
if next_cursor in followed_cursors:
|
||||
raise ValueError("next_cursor has already been followed")
|
||||
followed_cursors.add(next_cursor)
|
||||
params = {**params, "cursor": next_cursor}
|
||||
page += 1
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Invalid production-run response on page {page}: {exc}") from exc
|
||||
|
||||
def get_downtimes(
|
||||
self,
|
||||
machine_id: str,
|
||||
*,
|
||||
start: datetime | None = None,
|
||||
) -> list[EnlyzeDowntime]:
|
||||
"""Retrieve machine downtimes and follow ENLYZE cursor pagination.
|
||||
|
||||
``start`` is an inclusive source-time lookback; callers use it for reconciliation,
|
||||
not as a claim that records before it cannot subsequently be edited.
|
||||
"""
|
||||
if start is not None and (start.tzinfo is None or start.utcoffset() is None):
|
||||
raise ValueError("start must be timezone-aware")
|
||||
params = {"machine": machine_id}
|
||||
if start is not None:
|
||||
params["start"] = start.astimezone(UTC).isoformat()
|
||||
result: list[EnlyzeDowntime] = []
|
||||
cursors: set[str] = set()
|
||||
page = 1
|
||||
while True:
|
||||
response = self._client.get("/v2/downtimes", params)
|
||||
try:
|
||||
if not isinstance(response.body, dict) or not isinstance(
|
||||
response.body["data"], list
|
||||
):
|
||||
raise ValueError("body data must be a list")
|
||||
for item in response.body["data"]:
|
||||
result.append(_parse_downtime(item, machine_id))
|
||||
cursor = _next_cursor(response.body)
|
||||
if cursor is None:
|
||||
return result
|
||||
if cursor in cursors:
|
||||
raise ValueError("next_cursor has already been followed")
|
||||
cursors.add(cursor)
|
||||
params = {**params, "cursor": cursor}
|
||||
page += 1
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Invalid downtime response on page {page}: {exc}") from exc
|
||||
|
||||
def get_material_samples(
|
||||
self,
|
||||
*,
|
||||
machine_id: str,
|
||||
rate_variable_id: str,
|
||||
gate_variable_id: str,
|
||||
gate_variable_id: str | None,
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
application_variable_ids: tuple[str, ...] = (),
|
||||
nominal_width_m: float | None = None,
|
||||
rotational_speed_variable_ids: tuple[str, ...] = (),
|
||||
specific_discharge_kg_per_rev_m: float | None = None,
|
||||
process_application_gate_threshold: float | None = None,
|
||||
) -> list[MaterialSample]:
|
||||
response = self._client.post_json(
|
||||
"/v2/timeseries",
|
||||
{
|
||||
"machine": machine_id,
|
||||
"start": start.astimezone(UTC).isoformat(),
|
||||
"end": end.astimezone(UTC).isoformat(),
|
||||
"variables": [
|
||||
{"uuid": rate_variable_id},
|
||||
{"uuid": gate_variable_id},
|
||||
],
|
||||
},
|
||||
for name, value in (("start", start), ("end", end)):
|
||||
if value.tzinfo is None or value.utcoffset() is None:
|
||||
raise ValueError(f"{name} must be timezone-aware")
|
||||
if end < start:
|
||||
raise ValueError("end must not precede start")
|
||||
if application_variable_ids and rotational_speed_variable_ids:
|
||||
raise ValueError("Material source modes are mutually exclusive")
|
||||
if gate_variable_id is None and not rotational_speed_variable_ids:
|
||||
raise ValueError("This source requires a gate signal")
|
||||
source_ids = (
|
||||
rotational_speed_variable_ids or application_variable_ids or (rate_variable_id,)
|
||||
)
|
||||
gate_ids = (gate_variable_id,) if gate_variable_id else ()
|
||||
variable_ids = list(dict.fromkeys((*source_ids, *gate_ids)))
|
||||
request_body = {
|
||||
"machine": machine_id,
|
||||
"start": start.astimezone(UTC).isoformat(),
|
||||
"end": end.astimezone(UTC).isoformat(),
|
||||
"variables": [{"uuid": variable_id} for variable_id in variable_ids],
|
||||
}
|
||||
samples = []
|
||||
followed_cursors: set[str] = set()
|
||||
page = 1
|
||||
while True:
|
||||
response = self._client.post_json("/v2/timeseries", request_body)
|
||||
try:
|
||||
data = response.body["data"]
|
||||
columns = data["columns"]
|
||||
if not isinstance(columns, list):
|
||||
raise ValueError("columns must be a list")
|
||||
for required in ("time", *variable_ids):
|
||||
if columns.count(required) != 1:
|
||||
raise ValueError(f"required column {required!r} must occur exactly once")
|
||||
time_index = columns.index("time")
|
||||
source_indices = [columns.index(ref) for ref in source_ids]
|
||||
gate_index = columns.index(gate_variable_id) if gate_variable_id else None
|
||||
if not isinstance(data["records"], list):
|
||||
raise ValueError("records must be a list")
|
||||
for index, record in enumerate(data["records"]):
|
||||
try:
|
||||
if not isinstance(record, list) or len(record) != len(columns):
|
||||
raise ValueError("record must match columns")
|
||||
sources = [record[i] for i in source_indices]
|
||||
gate = record[gate_index] if gate_index is not None else 1.0
|
||||
for value in (*sources, gate):
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ValueError("rate and gate must be numeric")
|
||||
if not isfinite(value):
|
||||
raise ValueError("rate and gate must be finite")
|
||||
if rotational_speed_variable_ids:
|
||||
if nominal_width_m is None or specific_discharge_kg_per_rev_m is None:
|
||||
raise ValueError("rotational discharge requires width and factor")
|
||||
rate = rotational_discharge_rate_kg_per_hour(
|
||||
sources,
|
||||
nominal_width_m,
|
||||
specific_discharge_kg_per_rev_m,
|
||||
)
|
||||
elif application_variable_ids:
|
||||
if nominal_width_m is None:
|
||||
raise ValueError("area application requires nominal width")
|
||||
rate = area_application_rate_kg_per_hour(sources, nominal_width_m, gate)
|
||||
else:
|
||||
rate = sources[0]
|
||||
application = None
|
||||
if process_application_gate_threshold is not None:
|
||||
if not rotational_speed_variable_ids or gate_variable_id is None:
|
||||
raise ValueError("Process application requires rpm and speed gate")
|
||||
from production_analytics.calculations.material_application import (
|
||||
rotational_application_g_m2,
|
||||
)
|
||||
|
||||
application = rotational_application_g_m2(
|
||||
sources,
|
||||
specific_discharge_kg_per_rev_m,
|
||||
gate,
|
||||
process_application_gate_threshold,
|
||||
)
|
||||
samples.append(
|
||||
MaterialSample(
|
||||
timestamp=_parse_timestamp(record[time_index]),
|
||||
material_rate_kg_per_hour=float(rate),
|
||||
gate_value=float(gate),
|
||||
application_g_m2=application,
|
||||
)
|
||||
)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"malformed record {index}: {exc}") from exc
|
||||
next_cursor = None
|
||||
if "metadata" in response.body:
|
||||
metadata = response.body["metadata"]
|
||||
if not isinstance(metadata, dict) or "next_cursor" not in metadata:
|
||||
raise ValueError("metadata must be an object with a next_cursor field")
|
||||
next_cursor = metadata["next_cursor"]
|
||||
if next_cursor is not None and (
|
||||
not isinstance(next_cursor, str) or not next_cursor
|
||||
):
|
||||
raise ValueError("next_cursor must be null or a non-empty string")
|
||||
if next_cursor is None:
|
||||
return samples
|
||||
if next_cursor in followed_cursors:
|
||||
raise ValueError("next_cursor has already been followed")
|
||||
followed_cursors.add(next_cursor)
|
||||
request_body = {**request_body, "cursor": next_cursor}
|
||||
page += 1
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Invalid timeseries response on page {page}: {exc}") from exc
|
||||
|
||||
def get_dosing_channel_samples(
|
||||
self,
|
||||
*,
|
||||
machine_id: str,
|
||||
channels: tuple["ChannelDefinition", ...],
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
) -> dict[str, list[DosingChannelSample]]:
|
||||
"""Read configured dosing channels through the gateway only.
|
||||
|
||||
Configurations name stable PLC origins; ENLYZE UUIDs are resolved at poll
|
||||
time and must be unique, preventing a silently stale variable mapping.
|
||||
"""
|
||||
if (
|
||||
start.tzinfo is None
|
||||
or start.utcoffset() is None
|
||||
or end.tzinfo is None
|
||||
or end.utcoffset() is None
|
||||
):
|
||||
raise ValueError("dosing channel window must be timezone-aware")
|
||||
if end < start:
|
||||
raise ValueError("end must not precede start")
|
||||
resolved = self.resolve_dosing_channel_signal_refs(
|
||||
machine_id=machine_id,
|
||||
channels=channels,
|
||||
)
|
||||
variable_ids = list(
|
||||
dict.fromkeys(
|
||||
variable_id
|
||||
for references in resolved.values()
|
||||
for variable_id in references
|
||||
if variable_id is not None
|
||||
)
|
||||
)
|
||||
body = {
|
||||
"machine": machine_id,
|
||||
"start": start.astimezone(UTC).isoformat(),
|
||||
"end": end.astimezone(UTC).isoformat(),
|
||||
"variables": [{"uuid": x} for x in variable_ids],
|
||||
}
|
||||
result = {key: [] for key in resolved}
|
||||
cursors: set[str] = set()
|
||||
while True:
|
||||
response = self._client.post_json("/v2/timeseries", body)
|
||||
try:
|
||||
data = response.body["data"]
|
||||
columns = data["columns"]
|
||||
if not isinstance(columns, list) or any(
|
||||
columns.count(x) != 1 for x in ("time", *variable_ids)
|
||||
):
|
||||
raise ValueError("required columns must occur exactly once")
|
||||
if not isinstance(data["records"], list):
|
||||
raise ValueError("records must be a list")
|
||||
for row in data["records"]:
|
||||
if not isinstance(row, list) or len(row) != len(columns):
|
||||
raise ValueError("record must match columns")
|
||||
timestamp = _parse_timestamp(row[columns.index("time")])
|
||||
for key, (total, status, percentage, screw, material) in resolved.items():
|
||||
sample = self._dosing_sample(
|
||||
row, columns, timestamp, total, status, percentage, screw, material
|
||||
)
|
||||
if sample is not None:
|
||||
result[key].append(sample)
|
||||
cursor = _next_cursor(response.body)
|
||||
if cursor is None:
|
||||
return result
|
||||
if cursor in cursors:
|
||||
raise ValueError("next_cursor has already been followed")
|
||||
cursors.add(cursor)
|
||||
body = {**body, "cursor": cursor}
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Invalid dosing timeseries response: {exc}") from exc
|
||||
|
||||
def resolve_dosing_channel_signal_refs(
|
||||
self,
|
||||
*,
|
||||
machine_id: str,
|
||||
channels: tuple["ChannelDefinition", ...],
|
||||
) -> dict[str, tuple[str, str, str, str, str | None]]:
|
||||
"""Resolve configured PLC origins without reading samples or writing state."""
|
||||
origins = self._variables_by_origin(machine_id)
|
||||
resolved: dict[str, tuple[str, str, str, str, str | None]] = {}
|
||||
for channel in channels:
|
||||
refs = (
|
||||
channel.total_rate_signal_ref,
|
||||
channel.status_signal_ref,
|
||||
channel.percentage_signal_ref,
|
||||
channel.screw_speed_signal_ref,
|
||||
)
|
||||
ids = tuple(self._unique_origin(origins, ref) for ref in refs)
|
||||
material = (
|
||||
self._unique_origin(origins, channel.material_number_signal_ref)
|
||||
if channel.material_number_signal_ref
|
||||
else None
|
||||
)
|
||||
key = channel.channel + "\0" + channel.extruder
|
||||
resolved[key] = (*ids, material)
|
||||
return resolved
|
||||
|
||||
@staticmethod
|
||||
def _dosing_sample(
|
||||
row: list[object],
|
||||
columns: list[object],
|
||||
timestamp: datetime,
|
||||
total: str,
|
||||
status: str,
|
||||
percentage: str,
|
||||
screw: str,
|
||||
material: str | None,
|
||||
) -> DosingChannelSample | None:
|
||||
def numeric(variable: str) -> float:
|
||||
value = row[columns.index(variable)]
|
||||
if (
|
||||
isinstance(value, bool)
|
||||
or not isinstance(value, (int, float))
|
||||
or not isfinite(value)
|
||||
):
|
||||
raise ValueError("dosing values must be finite numeric")
|
||||
return float(value)
|
||||
|
||||
for variable in (total, percentage, screw):
|
||||
if row[columns.index(variable)] is None:
|
||||
return None
|
||||
|
||||
status_value = row[columns.index(status)]
|
||||
if status_value is None:
|
||||
enabled = False
|
||||
elif isinstance(status_value, bool):
|
||||
enabled = status_value
|
||||
elif (
|
||||
isinstance(status_value, (int, float))
|
||||
and not isinstance(status_value, bool)
|
||||
and isfinite(status_value)
|
||||
):
|
||||
enabled = status_value != 0
|
||||
else:
|
||||
raise ValueError("dosing status must be boolean or numeric")
|
||||
material_value = None if material is None else row[columns.index(material)]
|
||||
if material_value is not None and not isinstance(material_value, (str, int, float)):
|
||||
raise ValueError("material assignment must be scalar or null")
|
||||
return DosingChannelSample(
|
||||
timestamp,
|
||||
numeric(total),
|
||||
enabled,
|
||||
numeric(percentage),
|
||||
numeric(screw),
|
||||
None if material_value is None else str(material_value),
|
||||
)
|
||||
|
||||
data = response.body["data"]
|
||||
columns = data["columns"]
|
||||
def _variables_by_origin(self, machine_id: str) -> dict[str, list[str]]:
|
||||
params = {"machine": machine_id}
|
||||
result: dict[str, list[str]] = {}
|
||||
cursors: set[str] = set()
|
||||
while True:
|
||||
response = self._client.get("/v2/variables", params)
|
||||
try:
|
||||
items = response.body["data"]
|
||||
if not isinstance(items, list):
|
||||
raise ValueError("data must be a list")
|
||||
for item in items:
|
||||
uuid = item["uuid"]
|
||||
if not isinstance(uuid, str):
|
||||
raise ValueError("variable uuid must be a string")
|
||||
details = item.get("details")
|
||||
if not isinstance(details, dict):
|
||||
raise ValueError("variable details must be an object")
|
||||
origin_identifier = details.get("origin_identifier")
|
||||
if origin_identifier is None:
|
||||
# Derived ENLYZE variables do not have a PLC origin.
|
||||
continue
|
||||
if not isinstance(origin_identifier, dict):
|
||||
raise ValueError("variable origin_identifier must be an object")
|
||||
code = origin_identifier.get("code")
|
||||
if not isinstance(code, str):
|
||||
raise ValueError("variable origin must be a string")
|
||||
result.setdefault(code, []).append(uuid)
|
||||
cursor = _next_cursor(response.body)
|
||||
if cursor is None:
|
||||
return result
|
||||
if cursor in cursors:
|
||||
raise ValueError("next_cursor has already been followed")
|
||||
cursors.add(cursor)
|
||||
params = {**params, "cursor": cursor}
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Invalid variable response: {exc}") from exc
|
||||
|
||||
time_index = columns.index("time")
|
||||
rate_index = columns.index(rate_variable_id)
|
||||
gate_index = columns.index(gate_variable_id)
|
||||
|
||||
return [
|
||||
MaterialSample(
|
||||
timestamp=_parse_timestamp(record[time_index]),
|
||||
material_rate_kg_per_hour=float(record[rate_index]),
|
||||
gate_value=float(record[gate_index]),
|
||||
)
|
||||
for record in data["records"]
|
||||
]
|
||||
@staticmethod
|
||||
def _unique_origin(origins: dict[str, list[str]], origin: str) -> str:
|
||||
matches = origins.get(origin, [])
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"Configured PLC origin {origin!r} is unavailable or ambiguous")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def _parse_timestamp(value: str) -> datetime:
|
||||
return datetime.fromisoformat(value.replace("Z", "+00:00")).astimezone(UTC)
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("timestamp must be an ISO string")
|
||||
timestamp = datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
if timestamp.tzinfo is None or timestamp.utcoffset() is None:
|
||||
raise ValueError("timestamp must be timezone-aware")
|
||||
return timestamp.astimezone(UTC)
|
||||
|
||||
|
||||
def _next_cursor(body: dict[object, object]) -> str | None:
|
||||
if "metadata" not in body:
|
||||
return None
|
||||
metadata = body["metadata"]
|
||||
if not isinstance(metadata, dict) or "next_cursor" not in metadata:
|
||||
raise ValueError("metadata must be an object with a next_cursor field")
|
||||
cursor = metadata["next_cursor"]
|
||||
if cursor is not None and (not isinstance(cursor, str) or not cursor):
|
||||
raise ValueError("next_cursor must be null or a non-empty string")
|
||||
return cursor
|
||||
|
||||
|
||||
def _nullable_text(value: object, name: str) -> str | None:
|
||||
if value is not None and not isinstance(value, str):
|
||||
raise ValueError(f"{name} must be a string or null")
|
||||
return value
|
||||
|
||||
|
||||
def _parse_downtime(item: object, machine_id: str) -> EnlyzeDowntime:
|
||||
if not isinstance(item, dict):
|
||||
raise ValueError("downtime must be an object")
|
||||
for key in ("uuid", "machine", "type", "start"):
|
||||
if not isinstance(item.get(key), str) or not item[key]:
|
||||
raise ValueError(f"{key} must be a non-empty string")
|
||||
if item["machine"] != machine_id:
|
||||
raise ValueError("downtime machine does not match requested machine")
|
||||
start = _parse_timestamp(item["start"])
|
||||
end_value = item.get("end")
|
||||
end = _parse_timestamp(end_value) if end_value is not None else None
|
||||
if end is not None and end < start:
|
||||
raise ValueError("downtime end must not precede start")
|
||||
reason = item.get("reason")
|
||||
if reason is not None and not isinstance(reason, dict):
|
||||
raise ValueError("reason must be an object or null")
|
||||
if reason is not None:
|
||||
for key in ("uuid", "name", "category"):
|
||||
if not isinstance(reason.get(key), str) or not reason[key]:
|
||||
raise ValueError(f"reason {key} must be a non-empty string")
|
||||
updated = item.get("updated")
|
||||
if updated is not None and not isinstance(updated, dict):
|
||||
raise ValueError("updated must be an object or null")
|
||||
updated_at = None
|
||||
if updated is not None:
|
||||
updated_at = _parse_timestamp(updated.get("timestamp"))
|
||||
return EnlyzeDowntime(
|
||||
uuid=item["uuid"],
|
||||
machine_id=machine_id,
|
||||
source_type=item["type"],
|
||||
start=start,
|
||||
end=end,
|
||||
comment=_nullable_text(item.get("comment"), "comment"),
|
||||
reason_id=None if reason is None else reason["uuid"],
|
||||
reason_name=None if reason is None else reason["name"],
|
||||
reason_description=None
|
||||
if reason is None
|
||||
else _nullable_text(reason.get("description"), "reason description"),
|
||||
reason_group=None
|
||||
if reason is None
|
||||
else _nullable_text(reason.get("group"), "reason group"),
|
||||
reason_category=None if reason is None else reason["category"],
|
||||
source_updated_at=updated_at,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
"""Read-only ERP current workplace context."""
|
||||
|
||||
from production_analytics.erp.gateway import (
|
||||
CurrentWorkplaceStatus,
|
||||
ErpReadError,
|
||||
ErpSettings,
|
||||
ErpWorkplaceStatusGateway,
|
||||
)
|
||||
|
||||
__all__ = ["CurrentWorkplaceStatus", "ErpReadError", "ErpSettings", "ErpWorkplaceStatusGateway"]
|
||||
@@ -0,0 +1,148 @@
|
||||
"""Focused MSSQL boundary for dbo.GRAFANA_WORKPLACE_STATUS."""
|
||||
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from math import isfinite
|
||||
|
||||
import pymssql
|
||||
|
||||
from production_analytics.enlyze.exploration import ConfigurationError, load_secret_file
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ErpSettings:
|
||||
host: str
|
||||
port: int
|
||||
database: str
|
||||
user: str = field(repr=False)
|
||||
password: str = field(repr=False)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for attribute, suffix in (
|
||||
("host", "HOST"), ("database", "NAME"), ("user", "USER"), ("password", "PASSWORD"),
|
||||
):
|
||||
value = getattr(self, attribute)
|
||||
if not isinstance(value, str) or not value.strip() or "\x00" in value:
|
||||
raise ConfigurationError(f"ERP_DB_{suffix} must be non-empty without NUL bytes")
|
||||
if type(self.port) is not int or not 1 <= self.port <= 65535:
|
||||
raise ConfigurationError("ERP_DB_PORT must be an integer from 1 to 65535")
|
||||
|
||||
@classmethod
|
||||
def from_environment(cls, environment: Mapping[str, str]) -> "ErpSettings":
|
||||
raw_port = environment.get("ERP_DB_PORT", "")
|
||||
try:
|
||||
if not isinstance(raw_port, str):
|
||||
raise ValueError
|
||||
port = int(raw_port)
|
||||
except ValueError:
|
||||
raise ConfigurationError("ERP_DB_PORT must be an integer from 1 to 65535") from None
|
||||
return cls(
|
||||
host=environment.get("ERP_DB_HOST", ""), port=port,
|
||||
database=environment.get("ERP_DB_NAME", ""),
|
||||
user=environment.get("ERP_DB_USER", ""),
|
||||
password=environment.get("ERP_DB_PASSWORD", ""),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_secret_file(
|
||||
cls, path: str | os.PathLike[str] = "secrets/erp.env",
|
||||
*, environment: Mapping[str, str] | None = None,
|
||||
) -> "ErpSettings":
|
||||
"""Load dotenv values over the environment, following the existing convention."""
|
||||
try:
|
||||
values = load_secret_file(path)
|
||||
except Exception:
|
||||
raise ConfigurationError("ERP secret file could not be loaded") from None
|
||||
base = os.environ if environment is None else environment
|
||||
return cls.from_environment({**base, **values})
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CurrentWorkplaceStatus:
|
||||
"""Latest ERP feedback; timestamp and timezone are preserved, with no freshness claim."""
|
||||
|
||||
workplace: str
|
||||
production_order: str
|
||||
article_number: str | None
|
||||
article_description: str | None
|
||||
feedback_timestamp: datetime
|
||||
order_quantity_m2: float | None
|
||||
good_quantity_m2: float | None
|
||||
remaining_quantity_m2: float | None
|
||||
remaining_time_hours: float | None
|
||||
remaining_rolls: float | None
|
||||
|
||||
|
||||
class ErpReadError(RuntimeError):
|
||||
"""Safe operator-facing ERP read or result error."""
|
||||
|
||||
|
||||
_CURRENT_STATUS_SQL = """SELECT
|
||||
[Arbeitsplatz], [Fertigungsauftragsnummer], [Artikelnummer], [Artikelbezeichnung],
|
||||
[Zeitstempel], [Auftragsmenge], [Gutmenge], [Restmenge],
|
||||
[Verbleibende Zeit], [Verbleibende Rollen]
|
||||
FROM [dbo].[GRAFANA_WORKPLACE_STATUS]
|
||||
WHERE [Arbeitsplatz] = %s"""
|
||||
|
||||
|
||||
class ErpWorkplaceStatusGateway:
|
||||
def __init__(self, settings: ErpSettings) -> None:
|
||||
self._settings = settings
|
||||
|
||||
def get_current_workplace_status(self, workplace: str) -> CurrentWorkplaceStatus | None:
|
||||
if not isinstance(workplace, str) or not workplace.strip() or "\x00" in workplace:
|
||||
raise ValueError("workplace must be a non-empty string without NUL bytes")
|
||||
try:
|
||||
with pymssql.connect(
|
||||
server=self._settings.host, port=self._settings.port,
|
||||
database=self._settings.database, user=self._settings.user,
|
||||
password=self._settings.password, login_timeout=10, timeout=10,
|
||||
) as connection:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.execute(_CURRENT_STATUS_SQL, (workplace,))
|
||||
# Two rows suffice to detect a violation without selecting an arbitrary row.
|
||||
rows = cursor.fetchmany(2)
|
||||
except Exception:
|
||||
# Driver messages may include credentials; suppress their traceback chain too.
|
||||
raise ErpReadError("ERP current workplace status read failed") from None
|
||||
if not rows:
|
||||
return None
|
||||
if len(rows) > 1:
|
||||
raise ErpReadError("ERP current workplace status returned multiple rows")
|
||||
try:
|
||||
row = rows[0]
|
||||
if len(row) != 10 or not isinstance(row[4], datetime):
|
||||
raise ValueError
|
||||
return CurrentWorkplaceStatus(
|
||||
workplace=_text(row[0]), production_order=_text(row[1]),
|
||||
article_number=_text(row[2], nullable=True),
|
||||
article_description=_text(row[3], nullable=True),
|
||||
feedback_timestamp=row[4],
|
||||
order_quantity_m2=_number(row[5]), good_quantity_m2=_number(row[6]),
|
||||
remaining_quantity_m2=_number(row[7]), remaining_time_hours=_number(row[8]),
|
||||
remaining_rolls=_number(row[9]),
|
||||
)
|
||||
except Exception:
|
||||
raise ErpReadError("ERP current workplace status returned an invalid row") from None
|
||||
|
||||
|
||||
def _text(value: object, *, nullable: bool = False) -> str | None:
|
||||
if value is None and nullable:
|
||||
return None
|
||||
if not isinstance(value, str):
|
||||
raise ValueError
|
||||
return value.rstrip()
|
||||
|
||||
|
||||
def _number(value: object) -> float | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, (Decimal, int, float)):
|
||||
raise ValueError
|
||||
result = float(value)
|
||||
if not isfinite(result):
|
||||
raise ValueError
|
||||
return result
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Strict, meter-generic configuration for ENLYZE power meters."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import CalculationConfigError, _UniqueLoader
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PowerMeterSignals:
|
||||
energy_total: str
|
||||
active_power_total: str
|
||||
grid_frequency: str
|
||||
power_outages: str | None = None
|
||||
phase_active_power: tuple[tuple[str, str], ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PowerMeterConfig:
|
||||
name: str
|
||||
machine_uuid: str
|
||||
signals: PowerMeterSignals
|
||||
poll_interval_seconds: float = 60.0
|
||||
report_mount_path: str = "/mnt/reports/energy"
|
||||
|
||||
|
||||
def _text(value: Any, label: str) -> str:
|
||||
if not isinstance(value, str) or not value.strip() or "\x00" in value:
|
||||
raise CalculationConfigError(f"{label} must be a non-empty string")
|
||||
if value.startswith("redacted-") or value.startswith("<"):
|
||||
raise CalculationConfigError(f"{label} must contain a real local identifier")
|
||||
return value.strip()
|
||||
|
||||
|
||||
def load_power_meter_config(path: str | Path, meter_name: str | None = None) -> PowerMeterConfig:
|
||||
try:
|
||||
with Path(path).open(encoding="utf-8") as stream:
|
||||
document = yaml.load(stream, Loader=_UniqueLoader)
|
||||
except (OSError, UnicodeError, yaml.YAMLError) as exc:
|
||||
raise CalculationConfigError("Cannot read power-meter configuration") from exc
|
||||
if not isinstance(document, dict) or set(document) != {"meters"}:
|
||||
raise CalculationConfigError("Configuration must contain only 'meters'")
|
||||
meters = document["meters"]
|
||||
if not isinstance(meters, dict) or not meters:
|
||||
raise CalculationConfigError("meters must be a non-empty mapping")
|
||||
if meter_name is None:
|
||||
if len(meters) != 1:
|
||||
raise CalculationConfigError("Multiple meters: specify --meter")
|
||||
meter_name = next(iter(meters))
|
||||
if meter_name not in meters or not isinstance(meters[meter_name], dict):
|
||||
raise CalculationConfigError("Requested meter was not found")
|
||||
entry = meters[meter_name]
|
||||
if set(entry) - {"machine_uuid", "signals", "poll_interval_seconds", "report_mount_path"}:
|
||||
raise CalculationConfigError("Unknown power-meter configuration field")
|
||||
machine = _text(entry.get("machine_uuid"), f"meters.{meter_name}.machine_uuid")
|
||||
raw = entry.get("signals")
|
||||
if not isinstance(raw, dict):
|
||||
raise CalculationConfigError("signals must be a mapping")
|
||||
required = {"energy_total", "active_power_total", "grid_frequency"}
|
||||
if not required <= raw.keys():
|
||||
raise CalculationConfigError("signals requires energy_total, active_power_total, grid_frequency")
|
||||
phase = raw.get("phase_active_power", {})
|
||||
if not isinstance(phase, dict) or any(not isinstance(k, str) for k in phase):
|
||||
raise CalculationConfigError("phase_active_power must be a mapping")
|
||||
phase_items = tuple((_text(k, "phase name"), _text(v, f"phase_active_power.{k}"))
|
||||
for k, v in phase.items())
|
||||
if len({k for k, _ in phase_items}) != len(phase_items):
|
||||
raise CalculationConfigError("phase names must be unique")
|
||||
interval = entry.get("poll_interval_seconds", 60.0)
|
||||
if isinstance(interval, bool) or not isinstance(interval, (int, float)) or interval <= 0:
|
||||
raise CalculationConfigError("poll_interval_seconds must be greater than zero")
|
||||
return PowerMeterConfig(
|
||||
name=_text(meter_name, "meter name"), machine_uuid=machine,
|
||||
signals=PowerMeterSignals(
|
||||
energy_total=_text(raw["energy_total"], "energy_total"),
|
||||
active_power_total=_text(raw["active_power_total"], "active_power_total"),
|
||||
grid_frequency=_text(raw["grid_frequency"], "grid_frequency"),
|
||||
power_outages=_text(raw["power_outages"], "power_outages") if raw.get("power_outages") else None,
|
||||
phase_active_power=phase_items,
|
||||
),
|
||||
poll_interval_seconds=float(interval),
|
||||
report_mount_path=_text(entry.get("report_mount_path", "/mnt/reports/energy"), "report_mount_path"),
|
||||
)
|
||||
@@ -0,0 +1,85 @@
|
||||
"""ENLYZE adapter for generic power-meter telemetry."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from math import isfinite
|
||||
from production_analytics.enlyze.exploration import ExplorationClient
|
||||
from .config import PowerMeterConfig
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PowerMeterReading:
|
||||
timestamp: datetime
|
||||
energy_total_kwh: float
|
||||
active_power_total_kw: float
|
||||
grid_frequency_hz: float
|
||||
power_outages: float | None
|
||||
phase_active_power_kw: tuple[tuple[str, float], ...]
|
||||
|
||||
|
||||
def _number(value: object, label: str) -> float:
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)) or not isfinite(float(value)):
|
||||
raise ValueError(f"{label} must be a finite number")
|
||||
return float(value)
|
||||
|
||||
|
||||
class PowerMeterGateway:
|
||||
def __init__(self, client: ExplorationClient) -> None:
|
||||
self.client = client
|
||||
|
||||
def read(self, config: PowerMeterConfig, start: datetime, end: datetime) -> list[PowerMeterReading]:
|
||||
if start.tzinfo is None or end.tzinfo is None:
|
||||
raise ValueError("power-meter window must be timezone-aware")
|
||||
if end < start:
|
||||
raise ValueError("power-meter end must not precede start")
|
||||
ids = [config.signals.energy_total, config.signals.active_power_total,
|
||||
config.signals.grid_frequency]
|
||||
ids += [value for _, value in config.signals.phase_active_power]
|
||||
if config.signals.power_outages:
|
||||
ids.append(config.signals.power_outages)
|
||||
body = {"machine": config.machine_uuid, "start": start.astimezone(UTC).isoformat(),
|
||||
"end": end.astimezone(UTC).isoformat(),
|
||||
"variables": [{"uuid": value} for value in dict.fromkeys(ids)]}
|
||||
response = self.client.post_json("/v2/timeseries", body)
|
||||
result: list[PowerMeterReading] = []
|
||||
page_body = response.body
|
||||
while True:
|
||||
if not isinstance(page_body, dict) or not isinstance(page_body.get("data"), dict):
|
||||
raise ValueError("Invalid power-meter timeseries response")
|
||||
data = page_body["data"]
|
||||
columns, records = data.get("columns"), data.get("records")
|
||||
if not isinstance(columns, list) or not isinstance(records, list):
|
||||
raise ValueError("Invalid power-meter columns or records")
|
||||
index = {column: i for i, column in enumerate(columns)}
|
||||
required = ["time", config.signals.energy_total, config.signals.active_power_total,
|
||||
config.signals.grid_frequency]
|
||||
if any(name not in index for name in required):
|
||||
raise ValueError("Power-meter response is missing a required column")
|
||||
for row_number, row in enumerate(records):
|
||||
if not isinstance(row, list) or len(row) < len(columns):
|
||||
raise ValueError(f"Malformed power-meter record {row_number}")
|
||||
try:
|
||||
timestamp = datetime.fromisoformat(str(row[index["time"]]).replace("Z", "+00:00"))
|
||||
if timestamp.tzinfo is None:
|
||||
raise ValueError("timestamp is not timezone-aware")
|
||||
phase = tuple((name, _number(row[index[signal]], name))
|
||||
for name, signal in config.signals.phase_active_power
|
||||
if signal in index and row[index[signal]] is not None)
|
||||
result.append(PowerMeterReading(
|
||||
timestamp=timestamp.astimezone(UTC),
|
||||
energy_total_kwh=_number(row[index[config.signals.energy_total]], "energy_total"),
|
||||
active_power_total_kw=_number(row[index[config.signals.active_power_total]], "active_power_total"),
|
||||
grid_frequency_hz=_number(row[index[config.signals.grid_frequency]], "grid_frequency"),
|
||||
power_outages=(None if not config.signals.power_outages or
|
||||
row[index[config.signals.power_outages]] is None else
|
||||
_number(row[index[config.signals.power_outages]], "power_outages")),
|
||||
phase_active_power_kw=phase,
|
||||
))
|
||||
except (IndexError, KeyError, TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Malformed power-meter record {row_number}") from exc
|
||||
metadata = page_body.get("metadata", {})
|
||||
cursor = metadata.get("next_cursor") if isinstance(metadata, dict) else None
|
||||
if cursor is None:
|
||||
return sorted(result, key=lambda item: item.timestamp)
|
||||
body["cursor"] = cursor
|
||||
page_body = self.client.post_json("/v2/timeseries", body).body
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Independent HTML-to-PDF monthly power-meter reporting."""
|
||||
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
import html
|
||||
import subprocess
|
||||
from .repository import MonthlySummary
|
||||
|
||||
|
||||
def render_html(summary: MonthlySummary) -> str:
|
||||
def value(item, suffix=""):
|
||||
return "—" if item is None else f"{item}{suffix}"
|
||||
rows = [
|
||||
("Source start", value(summary.first_timestamp)),
|
||||
("Source end", value(summary.last_timestamp)),
|
||||
("Start meter reading", value(summary.first_energy_kwh, " kWh")),
|
||||
("End meter reading", value(summary.last_energy_kwh, " kWh")),
|
||||
("Monthly consumption", value(round(summary.consumption_kwh, 3), " kWh")),
|
||||
("Outage counter delta", value(summary.outages)),
|
||||
("Frequency minimum", value(summary.frequency_min_hz, " Hz") + (f" ({summary.frequency_min_timestamp})" if summary.frequency_min_timestamp else "")),
|
||||
("Frequency maximum", value(summary.frequency_max_hz, " Hz") + (f" ({summary.frequency_max_timestamp})" if summary.frequency_max_timestamp else "")),
|
||||
("Deviation from 50 Hz", value(None if summary.frequency_min_hz is None else round(summary.frequency_min_hz - 50, 3), " Hz") + " min; " + value(None if summary.frequency_max_hz is None else round(summary.frequency_max_hz - 50, 3), " Hz") + " max"),
|
||||
]
|
||||
body = "".join(f"<tr><th>{html.escape(str(k))}</th><td>{html.escape(str(v))}</td></tr>" for k, v in rows)
|
||||
return f"""<!doctype html><html><head><meta charset='utf-8'><style>
|
||||
body {{ font-family: sans-serif; margin: 28mm 20mm; color: #17202a; }} h1 {{ font-size: 20pt; }}
|
||||
table {{ border-collapse: collapse; width: 100%; font-size: 10pt; }} th,td {{ border-bottom: 1px solid #d5d8dc; padding: 7px; text-align:left; }} th {{ width: 42%; background:#f3f5f7; }}
|
||||
footer {{ margin-top:25px; color:#657; font-size:8pt; }}
|
||||
</style></head><body><h1>Powermeter report — {html.escape(summary.meter)} — {html.escape(summary.month)}</h1>
|
||||
<table>{body}</table><footer>Source: TimescaleDB power_meter_readings. Source timestamps are preserved exactly.</footer></body></html>"""
|
||||
|
||||
|
||||
def write_pdf(summary: MonthlySummary, output: Path) -> None:
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
html_path = output.with_suffix(".html")
|
||||
html_path.write_text(render_html(summary), encoding="utf-8")
|
||||
try:
|
||||
from weasyprint import HTML
|
||||
HTML(filename=str(html_path)).write_pdf(str(output))
|
||||
except ImportError:
|
||||
try:
|
||||
subprocess.run(["wkhtmltopdf", str(html_path), str(output)], check=True,
|
||||
capture_output=True, text=True)
|
||||
except (FileNotFoundError, subprocess.CalledProcessError) as exc:
|
||||
raise RuntimeError("PDF requires WeasyPrint or wkhtmltopdf") from exc
|
||||
finally:
|
||||
html_path.unlink(missing_ok=True)
|
||||
@@ -0,0 +1,102 @@
|
||||
"""TimescaleDB persistence and monthly aggregation for power meters."""
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import datetime
|
||||
import json
|
||||
from .gateway import PowerMeterReading
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MonthlySummary:
|
||||
meter: str
|
||||
month: str
|
||||
first_timestamp: datetime | None
|
||||
last_timestamp: datetime | None
|
||||
first_energy_kwh: float | None
|
||||
last_energy_kwh: float | None
|
||||
consumption_kwh: float
|
||||
outages: float
|
||||
frequency_min_hz: float | None
|
||||
frequency_min_timestamp: datetime | None
|
||||
frequency_max_hz: float | None
|
||||
frequency_max_timestamp: datetime | None
|
||||
|
||||
|
||||
def counter_delta(previous: float | None, current: float) -> float:
|
||||
"""Return a non-negative increment; a lower value starts a new counter epoch."""
|
||||
if previous is None:
|
||||
return 0.0
|
||||
return current - previous if current >= previous else current
|
||||
|
||||
|
||||
class PostgresPowerMeterRepository:
|
||||
def __init__(self, settings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def _connect(self):
|
||||
import psycopg
|
||||
return psycopg.connect(host=self.settings.host, port=self.settings.port,
|
||||
dbname=self.settings.dbname, user=self.settings.user,
|
||||
password=self.settings.password, connect_timeout=10,
|
||||
options="-c statement_timeout=10000")
|
||||
|
||||
def latest(self, meter: str, before: datetime):
|
||||
with self._connect() as connection:
|
||||
return connection.execute(
|
||||
"SELECT energy_total_kwh, power_outages FROM power_meter_readings "
|
||||
"WHERE meter=%s AND timestamp < %s ORDER BY timestamp DESC LIMIT 1",
|
||||
(meter, before)).fetchone()
|
||||
|
||||
def write(self, meter: str, reading: PowerMeterReading) -> None:
|
||||
previous = self.latest(meter, reading.timestamp)
|
||||
energy_delta = counter_delta(None if previous is None else previous[0], reading.energy_total_kwh)
|
||||
outage_delta = None if reading.power_outages is None else counter_delta(
|
||||
None if previous is None or previous[1] is None else previous[1], reading.power_outages)
|
||||
phases = dict(reading.phase_active_power_kw)
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO power_meter_readings
|
||||
(meter,timestamp,energy_total_kwh,energy_delta_kwh,active_power_total_kw,
|
||||
grid_frequency_hz,power_outages,outage_delta,phase_active_power_kw)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)
|
||||
ON CONFLICT (meter,timestamp) DO UPDATE SET
|
||||
energy_total_kwh=EXCLUDED.energy_total_kwh,
|
||||
energy_delta_kwh=EXCLUDED.energy_delta_kwh,
|
||||
active_power_total_kw=EXCLUDED.active_power_total_kw,
|
||||
grid_frequency_hz=EXCLUDED.grid_frequency_hz,
|
||||
power_outages=EXCLUDED.power_outages,
|
||||
outage_delta=EXCLUDED.outage_delta,
|
||||
phase_active_power_kw=EXCLUDED.phase_active_power_kw""",
|
||||
(meter, reading.timestamp, reading.energy_total_kwh, energy_delta,
|
||||
reading.active_power_total_kw, reading.grid_frequency_hz,
|
||||
reading.power_outages, outage_delta, json.dumps(phases)),
|
||||
)
|
||||
|
||||
def monthly(self, meter: str, start: datetime, end: datetime, month: str) -> MonthlySummary:
|
||||
query = """WITH points AS (
|
||||
SELECT * FROM power_meter_readings WHERE meter=%s AND timestamp >= %s AND timestamp < %s
|
||||
), extrema AS (
|
||||
SELECT min(grid_frequency_hz) AS min_hz, max(grid_frequency_hz) AS max_hz,
|
||||
(array_agg(timestamp ORDER BY grid_frequency_hz ASC, timestamp ASC))[1] AS min_at,
|
||||
(array_agg(timestamp ORDER BY grid_frequency_hz DESC, timestamp ASC))[1] AS max_at
|
||||
FROM points
|
||||
) SELECT (SELECT timestamp FROM points ORDER BY timestamp LIMIT 1),
|
||||
(SELECT timestamp FROM points ORDER BY timestamp DESC LIMIT 1),
|
||||
(SELECT energy_total_kwh FROM points ORDER BY timestamp LIMIT 1),
|
||||
(SELECT energy_total_kwh FROM points ORDER BY timestamp DESC LIMIT 1),
|
||||
COALESCE((SELECT sum(energy_delta_kwh) FROM points),0),
|
||||
COALESCE((SELECT sum(outage_delta) FROM points),0),
|
||||
min_hz, min_at, max_hz, max_at FROM extrema"""
|
||||
with self._connect() as connection:
|
||||
row = connection.execute(query, (meter, start, end)).fetchone()
|
||||
return MonthlySummary(meter, month, *row)
|
||||
|
||||
def save_report_metadata(self, summary: MonthlySummary, path: str) -> None:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO power_meter_monthly_reports
|
||||
(meter,month,report_path,summary) VALUES (%s,%s,%s,%s::jsonb)
|
||||
ON CONFLICT (meter,month) DO UPDATE SET report_path=EXCLUDED.report_path,
|
||||
summary=EXCLUDED.summary, generated_at=now()""",
|
||||
(summary.meter, summary.month, path, json.dumps(asdict(summary), default=str)),
|
||||
)
|
||||
@@ -0,0 +1,40 @@
|
||||
"""One-shot collection/backfill and monthly report orchestration."""
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from .config import PowerMeterConfig
|
||||
from .gateway import PowerMeterGateway
|
||||
from .repository import PostgresPowerMeterRepository
|
||||
from .report import write_pdf
|
||||
|
||||
|
||||
class PowerMeterCollector:
|
||||
def __init__(self, config: PowerMeterConfig, gateway: PowerMeterGateway,
|
||||
repository: PostgresPowerMeterRepository) -> None:
|
||||
self.config, self.gateway, self.repository = config, gateway, repository
|
||||
|
||||
def backfill(self, start: datetime, end: datetime) -> int:
|
||||
readings = self.gateway.read(self.config, start, end)
|
||||
for reading in readings:
|
||||
self.repository.write(self.config.name, reading)
|
||||
return len(readings)
|
||||
|
||||
def poll_once(self, now: datetime | None = None) -> int:
|
||||
now = now or datetime.now(UTC)
|
||||
start = now - timedelta(seconds=self.config.poll_interval_seconds * 2)
|
||||
return self.backfill(start, now)
|
||||
|
||||
|
||||
def monthly_report(config: PowerMeterConfig, repository: PostgresPowerMeterRepository,
|
||||
month: str) -> Path:
|
||||
start = datetime.fromisoformat(month + "-01").replace(tzinfo=UTC)
|
||||
end = datetime.fromisoformat(month + "-01").replace(tzinfo=UTC)
|
||||
if start.month == 12:
|
||||
end = end.replace(year=start.year + 1, month=1)
|
||||
else:
|
||||
end = end.replace(month=start.month + 1)
|
||||
summary = repository.monthly(config.name, start, end, month)
|
||||
output = Path(config.report_mount_path) / config.name / month[:4] / f"{config.name}_Powermeter_{month}.pdf"
|
||||
write_pdf(summary, output)
|
||||
repository.save_report_metadata(summary, str(output))
|
||||
return output
|
||||
@@ -0,0 +1,246 @@
|
||||
"""Stateful polling for configured extruder dosing channels."""
|
||||
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Protocol
|
||||
|
||||
from production_analytics.calculations.channel_material import (
|
||||
ChannelRate,
|
||||
DosingChannelSample,
|
||||
channel_rates,
|
||||
validate_active_percentage_sum,
|
||||
)
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialConsumptionIntegrator,
|
||||
MaterialIntegrationState,
|
||||
MaterialIntegratorConfig,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeProductionRun
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChannelDefinition:
|
||||
extruder: str
|
||||
channel: str
|
||||
total_rate_signal_ref: str
|
||||
status_signal_ref: str
|
||||
percentage_signal_ref: str
|
||||
screw_speed_signal_ref: str
|
||||
material_number_signal_ref: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChannelPollingState:
|
||||
run_id: str
|
||||
integration_state: MaterialIntegrationState
|
||||
|
||||
|
||||
class ChannelStateStore(Protocol):
|
||||
def load(
|
||||
self, machine_id: str, extruder: str, channel: str, production_order: str
|
||||
) -> ChannelPollingState | None: ...
|
||||
def save(
|
||||
self,
|
||||
machine_id: str,
|
||||
extruder: str,
|
||||
channel: str,
|
||||
production_order: str,
|
||||
state: ChannelPollingState,
|
||||
) -> None: ...
|
||||
|
||||
|
||||
class ChannelGateway(Protocol):
|
||||
def get_open_production_run(self, machine_id: str) -> EnlyzeProductionRun | None: ...
|
||||
def get_production_runs(self, machine_id: str) -> list[EnlyzeProductionRun]: ...
|
||||
def get_dosing_channel_samples(
|
||||
self,
|
||||
*,
|
||||
machine_id: str,
|
||||
channels: tuple[ChannelDefinition, ...],
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
) -> dict[str, list[DosingChannelSample]]: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChannelPollResult:
|
||||
run: EnlyzeProductionRun
|
||||
extruder: str
|
||||
channel: str
|
||||
state: MaterialIntegrationState
|
||||
material_number: str | None
|
||||
material_name: str | None
|
||||
material_mapping_status: str
|
||||
percentage_sum_valid: bool
|
||||
|
||||
|
||||
class ChannelMaterialPollingService:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
gateway: ChannelGateway,
|
||||
state_store: ChannelStateStore,
|
||||
machine_id: str,
|
||||
channels: tuple[ChannelDefinition, ...],
|
||||
max_sample_gap_seconds: float,
|
||||
percentage_tolerance: float = 1.0,
|
||||
material_names: dict[str, str] | None = None,
|
||||
) -> None:
|
||||
if not channels or len({(c.extruder, c.channel) for c in channels}) != len(channels):
|
||||
raise ValueError("channels must be non-empty and uniquely identified")
|
||||
self.gateway, self.state_store, self.machine_id, self.channels = (
|
||||
gateway,
|
||||
state_store,
|
||||
machine_id,
|
||||
channels,
|
||||
)
|
||||
self.config = MaterialIntegratorConfig(0.5, max_sample_gap_seconds)
|
||||
self.percentage_tolerance, self.material_names = percentage_tolerance, material_names or {}
|
||||
|
||||
def poll_once(self, *, now: datetime) -> tuple[ChannelPollResult, ...] | None:
|
||||
if now.tzinfo is None or now.utcoffset() is None:
|
||||
raise ValueError("now must be timezone-aware")
|
||||
run = self.gateway.get_open_production_run(self.machine_id)
|
||||
if run is None or now < run.start:
|
||||
return None
|
||||
# Each channel is separately checkpointed so an E4-style extra extruder
|
||||
# can be introduced without changing state identity.
|
||||
states: dict[str, ChannelPollingState | None] = {
|
||||
c.channel + "\0" + c.extruder: self.state_store.load(
|
||||
self.machine_id, c.extruder, c.channel, run.production_order
|
||||
)
|
||||
for c in self.channels
|
||||
}
|
||||
# A missing checkpoint is bootstrapped over each closed run separately.
|
||||
# Resetting the temporal baseline at every run boundary is essential:
|
||||
# a short wall-clock gap between ENLYZE runs must not become material.
|
||||
if all(state is None for state in states.values()):
|
||||
totals = {key: MaterialIntegrationState() for key in states}
|
||||
previous_runs = sorted(
|
||||
(
|
||||
candidate
|
||||
for candidate in self.gateway.get_production_runs(self.machine_id)
|
||||
if candidate.production_order == run.production_order
|
||||
and candidate.uuid != run.uuid
|
||||
and candidate.end is not None
|
||||
and candidate.end <= run.start
|
||||
),
|
||||
key=lambda candidate: candidate.start,
|
||||
)
|
||||
for previous in previous_runs:
|
||||
historical = self._derive(
|
||||
self.gateway.get_dosing_channel_samples(
|
||||
machine_id=self.machine_id,
|
||||
channels=self.channels,
|
||||
start=previous.start,
|
||||
end=previous.end,
|
||||
)
|
||||
)
|
||||
for key, samples in historical.items():
|
||||
integrator = MaterialConsumptionIntegrator(self.config, totals[key])
|
||||
integrator.process_many(rate.sample for rate, _ in samples)
|
||||
state = integrator.state
|
||||
totals[key] = MaterialIntegrationState(
|
||||
cumulative_consumption_kg=state.cumulative_consumption_kg,
|
||||
integrated_running_seconds=state.integrated_running_seconds,
|
||||
)
|
||||
states = {key: ChannelPollingState("bootstrap", state) for key, state in totals.items()}
|
||||
start = min(
|
||||
(
|
||||
s.integration_state.last_processed_timestamp
|
||||
for s in states.values()
|
||||
if s is not None
|
||||
and s.run_id == run.uuid
|
||||
and s.integration_state.last_processed_timestamp is not None
|
||||
),
|
||||
default=run.start,
|
||||
)
|
||||
if now < start:
|
||||
return None
|
||||
raw = self.gateway.get_dosing_channel_samples(
|
||||
machine_id=self.machine_id, channels=self.channels, start=start, end=now
|
||||
)
|
||||
derived = self._derive(raw)
|
||||
results = []
|
||||
for definition in self.channels:
|
||||
key = definition.channel + "\0" + definition.extruder
|
||||
saved = states[key]
|
||||
initial = (
|
||||
saved.integration_state
|
||||
if saved is not None and saved.run_id == run.uuid
|
||||
else MaterialIntegrationState(
|
||||
cumulative_consumption_kg=0.0
|
||||
if saved is None
|
||||
else saved.integration_state.cumulative_consumption_kg,
|
||||
integrated_running_seconds=0.0
|
||||
if saved is None
|
||||
else saved.integration_state.integrated_running_seconds,
|
||||
)
|
||||
)
|
||||
integrator = MaterialConsumptionIntegrator(self.config, initial)
|
||||
samples = derived[key]
|
||||
latest: ChannelRate | None = None
|
||||
quality = True
|
||||
for rate, sample_quality in samples:
|
||||
if (
|
||||
initial.last_processed_timestamp is not None
|
||||
and rate.sample.timestamp < initial.last_processed_timestamp
|
||||
):
|
||||
continue
|
||||
integrator.process(rate.sample)
|
||||
latest = rate
|
||||
quality = quality and sample_quality
|
||||
state = integrator.state
|
||||
self.state_store.save(
|
||||
self.machine_id,
|
||||
definition.extruder,
|
||||
definition.channel,
|
||||
run.production_order,
|
||||
ChannelPollingState(run.uuid, state),
|
||||
)
|
||||
results.append(
|
||||
ChannelPollResult(
|
||||
run,
|
||||
definition.extruder,
|
||||
definition.channel,
|
||||
state,
|
||||
None if latest is None else latest.material_number,
|
||||
None if latest is None else latest.material_name,
|
||||
"UNAVAILABLE" if latest is None else latest.material_mapping_status,
|
||||
quality,
|
||||
)
|
||||
)
|
||||
return tuple(results)
|
||||
|
||||
def _derive(
|
||||
self, raw: dict[str, list[DosingChannelSample]]
|
||||
) -> dict[str, list[tuple[ChannelRate, bool]]]:
|
||||
"""Evaluate aligned channel records with extruder-scoped quality rules."""
|
||||
by_timestamp: dict[datetime, dict[str, DosingChannelSample]] = defaultdict(dict)
|
||||
for definition in self.channels:
|
||||
key = definition.channel + "\0" + definition.extruder
|
||||
for sample in raw[key]:
|
||||
by_timestamp[sample.timestamp][key] = sample
|
||||
derived: dict[str, list[tuple[ChannelRate, bool]]] = defaultdict(list)
|
||||
for timestamp, values in sorted(by_timestamp.items()):
|
||||
if set(values) != set(raw):
|
||||
raise ValueError(f"unaligned channel samples at {timestamp.isoformat()}")
|
||||
grouped: dict[str, list[tuple[str, DosingChannelSample]]] = defaultdict(list)
|
||||
for key, sample in values.items():
|
||||
grouped[key.split("\0", 1)[1]].append((key, sample))
|
||||
for siblings in grouped.values():
|
||||
quality = validate_active_percentage_sum(
|
||||
(sample for _, sample in siblings),
|
||||
tolerance=self.percentage_tolerance,
|
||||
)
|
||||
for (key, _), rate in zip(
|
||||
siblings,
|
||||
channel_rates(
|
||||
(sample for _, sample in siblings),
|
||||
material_names=self.material_names,
|
||||
percentage_tolerance=self.percentage_tolerance,
|
||||
),
|
||||
):
|
||||
derived[key].append((rate, quality))
|
||||
return derived
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Sequential foreground runner for channel-level material consumption."""
|
||||
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from datetime import UTC, datetime
|
||||
from typing import TextIO
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.enlyze.exploration import ConfigurationError, ExplorationError
|
||||
from production_analytics.service.channel_material import ChannelMaterialPollingService
|
||||
from production_analytics.service.postgres_channel_material import (
|
||||
PostgresChannelMaterialSnapshotWriter,
|
||||
)
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
class ChannelMaterialPollingRunner:
|
||||
def __init__(
|
||||
self,
|
||||
service: ChannelMaterialPollingService,
|
||||
*,
|
||||
machine_id: str,
|
||||
calculation_id: str,
|
||||
snapshot_writer: PostgresChannelMaterialSnapshotWriter,
|
||||
poll_interval_seconds: float,
|
||||
clock: Callable[[], datetime] = utc_now,
|
||||
sleep: Callable[[float], None] = time.sleep,
|
||||
stdout: TextIO | None = None,
|
||||
stderr: TextIO | None = None,
|
||||
) -> None:
|
||||
self.service = service
|
||||
self.machine_id = machine_id
|
||||
self.calculation_id = calculation_id
|
||||
self.snapshot_writer = snapshot_writer
|
||||
self.interval = finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
self.clock = clock
|
||||
self.sleep = sleep
|
||||
self.stdout = stdout if stdout is not None else sys.stdout
|
||||
self.stderr = stderr if stderr is not None else sys.stderr
|
||||
|
||||
def run_once(self) -> bool:
|
||||
"""Run and report one cycle; return whether channel snapshots were written."""
|
||||
now = self.clock()
|
||||
if now.tzinfo is None or now.utcoffset() is None:
|
||||
raise ConfigurationError("Runner clock must return timezone-aware timestamps")
|
||||
now = now.astimezone(UTC)
|
||||
prefix = f"{now.isoformat()} machine={self.machine_id!r}"
|
||||
results = self.service.poll_once(now=now)
|
||||
if results is None:
|
||||
print(
|
||||
f"{prefix} no open Production Run / no eligible polling window; snapshots=0",
|
||||
file=self.stdout,
|
||||
flush=True,
|
||||
)
|
||||
return False
|
||||
for result in results:
|
||||
self.snapshot_writer.write(
|
||||
timestamp=now,
|
||||
calculation_id=self.calculation_id,
|
||||
machine_id=self.machine_id,
|
||||
extruder=result.extruder,
|
||||
channel=result.channel,
|
||||
production_order=result.run.production_order,
|
||||
run_id=result.run.uuid,
|
||||
cumulative_consumption_kg=result.state.cumulative_consumption_kg,
|
||||
material_number=result.material_number,
|
||||
material_name=result.material_name,
|
||||
material_mapping_status=result.material_mapping_status,
|
||||
percentage_sum_valid=result.percentage_sum_valid,
|
||||
)
|
||||
print(
|
||||
f"{prefix} production_order={results[0].run.production_order!r} "
|
||||
f"channel snapshots={len(results)} written",
|
||||
file=self.stdout,
|
||||
flush=True,
|
||||
)
|
||||
return True
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
self.run_once()
|
||||
except ConfigurationError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
if isinstance(exc, ExplorationError):
|
||||
detail = f"ENLYZE/gateway: {type(exc).__name__}"
|
||||
else:
|
||||
detail = f"poll cycle: {type(exc).__name__}"
|
||||
print(f"channel material error {detail}", file=self.stderr, flush=True)
|
||||
self.sleep(self.interval)
|
||||
except KeyboardInterrupt:
|
||||
print("Channel material polling stopped", file=self.stdout, flush=True)
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Compose the live channel-level material polling application."""
|
||||
|
||||
import math
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from production_analytics.calculations.channel_config import ChannelMaterialCalculationConfig
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.enlyze.exploration import (
|
||||
ConfigurationError,
|
||||
ExplorationClient,
|
||||
ExplorationSettings,
|
||||
load_secret_file,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
from production_analytics.service.channel_material import ChannelMaterialPollingService
|
||||
from production_analytics.service.channel_material_runner import ChannelMaterialPollingRunner
|
||||
from production_analytics.service.channel_material_state_store import JsonChannelMaterialStateStore
|
||||
from production_analytics.service.postgres_channel_material import (
|
||||
PostgresChannelMaterialSnapshotWriter,
|
||||
)
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
def build_channel_material_runner(
|
||||
calculation: ChannelMaterialCalculationConfig,
|
||||
*,
|
||||
poll_interval_seconds: float,
|
||||
state_directory: Path,
|
||||
secrets_file: Path,
|
||||
) -> ChannelMaterialPollingRunner:
|
||||
finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
environment = dict(os.environ)
|
||||
environment.update(load_secret_file(secrets_file))
|
||||
settings = ExplorationSettings.from_environment(environment)
|
||||
parsed = urlsplit(settings.base_url)
|
||||
if parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username:
|
||||
raise ConfigurationError(
|
||||
"ENLYZE_BASE_URL must be an HTTP(S) server URL without credentials"
|
||||
)
|
||||
if not math.isfinite(settings.timeout_seconds):
|
||||
raise ConfigurationError("ENLYZE_HTTP_TIMEOUT_SECONDS must be finite")
|
||||
try:
|
||||
state_directory.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryFile(dir=state_directory):
|
||||
pass
|
||||
except OSError as exc:
|
||||
raise ConfigurationError(
|
||||
f"State directory startup check failed: {type(exc).__name__}"
|
||||
) from exc
|
||||
postgres_settings = PostgresSettings.from_environment(environment)
|
||||
service = ChannelMaterialPollingService(
|
||||
gateway=EnlyzeApiGateway(ExplorationClient(settings)),
|
||||
state_store=JsonChannelMaterialStateStore(state_directory),
|
||||
machine_id=calculation.machine_ref,
|
||||
channels=calculation.channels,
|
||||
max_sample_gap_seconds=calculation.max_sample_gap_seconds,
|
||||
percentage_tolerance=calculation.percentage_tolerance,
|
||||
material_names=calculation.material_names,
|
||||
)
|
||||
return ChannelMaterialPollingRunner(
|
||||
service,
|
||||
machine_id=calculation.machine_ref,
|
||||
calculation_id=calculation.id,
|
||||
snapshot_writer=PostgresChannelMaterialSnapshotWriter(postgres_settings),
|
||||
poll_interval_seconds=poll_interval_seconds,
|
||||
)
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Restart-safe JSON state storage keyed by channel identity."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from production_analytics.calculations.material_state import (
|
||||
material_state_from_dict,
|
||||
material_state_to_dict,
|
||||
)
|
||||
from production_analytics.service.channel_material import ChannelPollingState
|
||||
|
||||
|
||||
class JsonChannelMaterialStateStore:
|
||||
def __init__(self, directory: str | Path) -> None:
|
||||
self.directory = Path(directory)
|
||||
|
||||
def _path(self, *identity: str) -> Path:
|
||||
digest = hashlib.sha256(json.dumps(identity).encode()).hexdigest()
|
||||
return self.directory / f"channel-material-{digest}.json"
|
||||
|
||||
def load(
|
||||
self, machine_id: str, extruder: str, channel: str, production_order: str
|
||||
) -> ChannelPollingState | None:
|
||||
path = self._path(machine_id, extruder, channel, production_order)
|
||||
if not path.exists():
|
||||
return None
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(data, dict) or not isinstance(data.get("run_id"), str):
|
||||
raise ValueError("Invalid channel material state")
|
||||
return ChannelPollingState(
|
||||
data["run_id"], material_state_from_dict(data["integration_state"])
|
||||
)
|
||||
|
||||
def save(
|
||||
self,
|
||||
machine_id: str,
|
||||
extruder: str,
|
||||
channel: str,
|
||||
production_order: str,
|
||||
state: ChannelPollingState,
|
||||
) -> None:
|
||||
self.directory.mkdir(parents=True, exist_ok=True)
|
||||
payload = {
|
||||
"run_id": state.run_id,
|
||||
"integration_state": material_state_to_dict(state.integration_state),
|
||||
}
|
||||
material_state_from_dict(payload["integration_state"])
|
||||
destination = self._path(machine_id, extruder, channel, production_order)
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w", encoding="utf-8", dir=self.directory, delete=False
|
||||
) as handle:
|
||||
temporary = Path(handle.name)
|
||||
json.dump(payload, handle, allow_nan=False)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(temporary, destination)
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Generic ENLYZE downtime reconciliation and ERP order attribution."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from enum import StrEnum
|
||||
from math import isfinite
|
||||
from typing import Protocol
|
||||
|
||||
from production_analytics.context import build_enlyze_production_order
|
||||
from production_analytics.enlyze.gateway import EnlyzeDowntime
|
||||
from production_analytics.erp import CurrentWorkplaceStatus
|
||||
|
||||
|
||||
class DowntimeCategory(StrEnum):
|
||||
PLANNED = "PLANNED"
|
||||
UNPLANNED = "UNPLANNED"
|
||||
UNKNOWN = "UNKNOWN"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProductionOrderBoundary:
|
||||
machine_id: str
|
||||
production_order: str
|
||||
started_at: datetime
|
||||
ended_at: datetime | None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProductionDowntimeEvent:
|
||||
external_id: str
|
||||
machine_id: str
|
||||
production_order: str | None
|
||||
source_type: str
|
||||
source_start: datetime
|
||||
source_end: datetime | None
|
||||
attributed_start: datetime | None
|
||||
attributed_end: datetime | None
|
||||
attributed_duration_seconds: float | None
|
||||
reason_id: str | None
|
||||
reason_name: str | None
|
||||
reason_description: str | None
|
||||
reason_group: str | None
|
||||
category: DowntimeCategory
|
||||
comment: str | None
|
||||
source_updated_at: datetime | None
|
||||
last_reconciled_at: datetime
|
||||
|
||||
|
||||
class DowntimeGateway(Protocol):
|
||||
def get_downtimes(
|
||||
self, machine_id: str, *, start: datetime | None = None
|
||||
) -> list[EnlyzeDowntime]: ...
|
||||
|
||||
|
||||
class WorkplaceStatusReader(Protocol):
|
||||
def get_current_workplace_status(self, workplace: str) -> CurrentWorkplaceStatus | None: ...
|
||||
|
||||
|
||||
class DowntimeRepository(Protocol):
|
||||
def current_boundary(self, machine_id: str) -> ProductionOrderBoundary | None: ...
|
||||
def save_boundary(self, boundary: ProductionOrderBoundary) -> None: ...
|
||||
def close_order_attribution(
|
||||
self, machine_id: str, production_order: str, ended_at: datetime
|
||||
) -> None: ...
|
||||
def upsert(self, event: ProductionDowntimeEvent) -> None: ...
|
||||
|
||||
|
||||
class DowntimeReconciliationService:
|
||||
"""Reconciles mutable source events without deriving a shift schedule."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
gateway: DowntimeGateway,
|
||||
erp: WorkplaceStatusReader,
|
||||
repository: DowntimeRepository,
|
||||
*,
|
||||
machine_id: str,
|
||||
workplace: str,
|
||||
production_order_format: str,
|
||||
lookback_hours: float = 48,
|
||||
full_reconciliation_hours: float = 24,
|
||||
completion_tolerance_m2: float = 0.001,
|
||||
) -> None:
|
||||
if not isfinite(lookback_hours) or lookback_hours <= 0:
|
||||
raise ValueError("lookback_hours must be positive")
|
||||
if not isfinite(full_reconciliation_hours) or full_reconciliation_hours <= 0:
|
||||
raise ValueError("full_reconciliation_hours must be positive")
|
||||
if not isfinite(completion_tolerance_m2) or completion_tolerance_m2 < 0:
|
||||
raise ValueError("completion_tolerance_m2 must be non-negative")
|
||||
self.gateway, self.erp, self.repository = gateway, erp, repository
|
||||
self.machine_id, self.workplace = machine_id, workplace
|
||||
self.production_order_format = production_order_format
|
||||
self.lookback = timedelta(hours=lookback_hours)
|
||||
self.full_reconciliation_interval = timedelta(hours=full_reconciliation_hours)
|
||||
self._last_full_reconciliation: datetime | None = None
|
||||
self.tolerance = completion_tolerance_m2
|
||||
|
||||
def reconcile_once(self, now: datetime) -> int:
|
||||
if now.tzinfo is None or now.utcoffset() is None:
|
||||
raise ValueError("now must be timezone-aware")
|
||||
now = now.astimezone(UTC)
|
||||
boundary = self._refresh_boundary(now)
|
||||
events = self.gateway.get_downtimes(self.machine_id, start=now - self.lookback)
|
||||
if (
|
||||
self._last_full_reconciliation is None
|
||||
or now - self._last_full_reconciliation >= self.full_reconciliation_interval
|
||||
):
|
||||
# ENLYZE does not guarantee that a delayed reason edit remains in a source-start
|
||||
# lookback window. A periodic full scan makes old open/UNKNOWN UUIDs mutable too.
|
||||
events = list(
|
||||
{
|
||||
event.uuid: event
|
||||
for event in [
|
||||
*events,
|
||||
*self.gateway.get_downtimes(self.machine_id),
|
||||
]
|
||||
}.values()
|
||||
)
|
||||
self._last_full_reconciliation = now
|
||||
for source in events:
|
||||
self.repository.upsert(self._event(source, boundary, now))
|
||||
return len(events)
|
||||
|
||||
def _refresh_boundary(self, now: datetime) -> ProductionOrderBoundary | None:
|
||||
status = self.erp.get_current_workplace_status(self.workplace)
|
||||
old = self.repository.current_boundary(self.machine_id)
|
||||
if status is None:
|
||||
return old
|
||||
order = build_enlyze_production_order(status.production_order, self.production_order_format)
|
||||
feedback_at = _aware_feedback(status.feedback_timestamp)
|
||||
if old is not None and old.production_order != order and old.ended_at is None:
|
||||
self.repository.close_order_attribution(
|
||||
self.machine_id, old.production_order, feedback_at
|
||||
)
|
||||
if old is None or old.production_order != order:
|
||||
current = ProductionOrderBoundary(self.machine_id, order, feedback_at, None)
|
||||
else:
|
||||
current = old
|
||||
if _is_complete(status, self.tolerance) and current.ended_at is None:
|
||||
current = ProductionOrderBoundary(
|
||||
current.machine_id,
|
||||
current.production_order,
|
||||
current.started_at,
|
||||
feedback_at,
|
||||
)
|
||||
self.repository.close_order_attribution(
|
||||
self.machine_id, current.production_order, feedback_at
|
||||
)
|
||||
self.repository.save_boundary(current)
|
||||
return current
|
||||
|
||||
def _event(
|
||||
self,
|
||||
source: EnlyzeDowntime,
|
||||
boundary: ProductionOrderBoundary | None,
|
||||
now: datetime,
|
||||
) -> ProductionDowntimeEvent:
|
||||
category = (
|
||||
DowntimeCategory(source.reason_category)
|
||||
if source.reason_category
|
||||
in {
|
||||
"PLANNED",
|
||||
"UNPLANNED",
|
||||
}
|
||||
else DowntimeCategory.UNKNOWN
|
||||
)
|
||||
attributed_start = attributed_end = None
|
||||
order = None
|
||||
if boundary is not None:
|
||||
lower = max(source.start, boundary.started_at)
|
||||
upper = source.end
|
||||
if boundary.ended_at is not None:
|
||||
upper = boundary.ended_at if upper is None else min(upper, boundary.ended_at)
|
||||
if upper is None or lower < upper:
|
||||
order, attributed_start, attributed_end = boundary.production_order, lower, upper
|
||||
duration = (
|
||||
None
|
||||
if attributed_start is None or attributed_end is None
|
||||
else (attributed_end - attributed_start).total_seconds()
|
||||
)
|
||||
return ProductionDowntimeEvent(
|
||||
source.uuid,
|
||||
source.machine_id,
|
||||
order,
|
||||
source.source_type,
|
||||
source.start,
|
||||
source.end,
|
||||
attributed_start,
|
||||
attributed_end,
|
||||
duration,
|
||||
source.reason_id,
|
||||
source.reason_name,
|
||||
source.reason_description,
|
||||
source.reason_group,
|
||||
category,
|
||||
source.comment,
|
||||
source.source_updated_at,
|
||||
now,
|
||||
)
|
||||
|
||||
|
||||
def _aware_feedback(value: datetime) -> datetime:
|
||||
if value.tzinfo is None or value.utcoffset() is None:
|
||||
raise ValueError("ERP feedback timestamp must be timezone-aware for downtime attribution")
|
||||
return value.astimezone(UTC)
|
||||
|
||||
|
||||
def _is_complete(status: CurrentWorkplaceStatus, tolerance: float) -> bool:
|
||||
remaining = status.remaining_quantity_m2
|
||||
if remaining is not None and remaining <= tolerance:
|
||||
return True
|
||||
if status.order_quantity_m2 is None or status.good_quantity_m2 is None:
|
||||
return False
|
||||
return status.good_quantity_m2 + tolerance >= status.order_quantity_m2
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Foreground reconciliation runner suitable for a systemd service."""
|
||||
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from datetime import UTC, datetime
|
||||
from typing import TextIO
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.service.downtime import DowntimeReconciliationService
|
||||
|
||||
|
||||
class DowntimeReconciliationRunner:
|
||||
def __init__(
|
||||
self,
|
||||
service: DowntimeReconciliationService,
|
||||
*,
|
||||
machine_id: str,
|
||||
poll_interval_seconds: float = 300,
|
||||
clock: Callable[[], datetime] | None = None,
|
||||
sleep: Callable[[float], None] = time.sleep,
|
||||
stdout: TextIO | None = None,
|
||||
stderr: TextIO | None = None,
|
||||
) -> None:
|
||||
self.service, self.machine_id = service, machine_id
|
||||
self.interval = finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
self.clock, self.sleep = clock or (lambda: datetime.now(UTC)), sleep
|
||||
self.stdout, self.stderr = stdout or sys.stdout, stderr or sys.stderr
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
now = self.clock()
|
||||
try:
|
||||
count = self.service.reconcile_once(now)
|
||||
print(
|
||||
f"{now.isoformat()} machine={self.machine_id!r} reconciled={count}",
|
||||
file=self.stdout,
|
||||
flush=True,
|
||||
)
|
||||
except Exception as exc:
|
||||
print(
|
||||
f"{now.isoformat()} machine={self.machine_id!r} reconciliation failed: "
|
||||
f"{type(exc).__name__}",
|
||||
file=self.stderr,
|
||||
flush=True,
|
||||
)
|
||||
self.sleep(self.interval)
|
||||
except KeyboardInterrupt:
|
||||
return
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Build the generic downtime runner from a small, explicit YAML configuration."""
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from production_analytics.enlyze.exploration import (
|
||||
ExplorationClient,
|
||||
ExplorationSettings,
|
||||
load_secret_file,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
from production_analytics.erp import ErpSettings, ErpWorkplaceStatusGateway
|
||||
from production_analytics.service.downtime import DowntimeReconciliationService
|
||||
from production_analytics.service.downtime_runner import DowntimeReconciliationRunner
|
||||
from production_analytics.service.postgres_downtime import PostgresDowntimeRepository
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DowntimeConfig:
|
||||
machine_id: str
|
||||
erp_workplace: str
|
||||
production_order_format: str
|
||||
poll_interval_seconds: float = 300
|
||||
lookback_hours: float = 48
|
||||
full_reconciliation_hours: float = 24
|
||||
completion_tolerance_m2: float = 0.001
|
||||
|
||||
|
||||
def load_downtime_config(path: Path) -> DowntimeConfig:
|
||||
raw = yaml.safe_load(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError("Downtime config must be a mapping")
|
||||
allowed = {field.name for field in DowntimeConfig.__dataclass_fields__.values()}
|
||||
if set(raw) - allowed or not {"machine_id", "erp_workplace", "production_order_format"} <= set(
|
||||
raw
|
||||
):
|
||||
raise ValueError("Downtime config has unknown or missing fields")
|
||||
for name in ("machine_id", "erp_workplace", "production_order_format"):
|
||||
if not isinstance(raw[name], str) or not raw[name].strip():
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
return DowntimeConfig(**raw)
|
||||
|
||||
|
||||
def build_downtime_runner(
|
||||
config_path: Path, *, secrets_file: Path, erp_secrets_file: Path
|
||||
) -> DowntimeReconciliationRunner:
|
||||
config = load_downtime_config(config_path)
|
||||
environment = {**os.environ, **load_secret_file(secrets_file)}
|
||||
erp_environment = {**os.environ, **load_secret_file(erp_secrets_file)}
|
||||
service = DowntimeReconciliationService(
|
||||
EnlyzeApiGateway(ExplorationClient(ExplorationSettings.from_environment(environment))),
|
||||
ErpWorkplaceStatusGateway(ErpSettings.from_environment(erp_environment)),
|
||||
PostgresDowntimeRepository(PostgresSettings.from_environment(os.environ)),
|
||||
machine_id=config.machine_id,
|
||||
workplace=config.erp_workplace,
|
||||
production_order_format=config.production_order_format,
|
||||
lookback_hours=config.lookback_hours,
|
||||
full_reconciliation_hours=config.full_reconciliation_hours,
|
||||
completion_tolerance_m2=config.completion_tolerance_m2,
|
||||
)
|
||||
return DowntimeReconciliationRunner(
|
||||
service, machine_id=config.machine_id, poll_interval_seconds=config.poll_interval_seconds
|
||||
)
|
||||
@@ -0,0 +1,25 @@
|
||||
"""ERP nominal-width resolution for an explicitly matched production order."""
|
||||
|
||||
from production_analytics.context import build_enlyze_production_order, extract_nominal_width_m
|
||||
from production_analytics.erp import ErpWorkplaceStatusGateway
|
||||
|
||||
|
||||
class ErpNominalWidthProvider:
|
||||
def __init__(
|
||||
self, gateway: ErpWorkplaceStatusGateway, *, workplace: str, format_template: str,
|
||||
) -> None:
|
||||
self.gateway = gateway
|
||||
self.workplace = workplace
|
||||
self.format_template = format_template
|
||||
|
||||
def __call__(self, production_order: str) -> float:
|
||||
status = self.gateway.get_current_workplace_status(self.workplace)
|
||||
if (status is None
|
||||
or status.workplace.strip().casefold() != self.workplace.strip().casefold()
|
||||
or build_enlyze_production_order(status.production_order, self.format_template)
|
||||
!= production_order):
|
||||
raise ValueError("ERP context does not match the production order")
|
||||
width = extract_nominal_width_m(status.article_description)
|
||||
if width is None:
|
||||
raise ValueError("ERP article description has no unambiguous nominal width")
|
||||
return width
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Feedback-aligned, machine-independent production-order material efficiency."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from math import isfinite
|
||||
from typing import Protocol
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from production_analytics.context import build_enlyze_production_order, extract_nominal_width_m
|
||||
from production_analytics.erp import CurrentWorkplaceStatus
|
||||
from production_analytics.service.postgres_material import MaterialSnapshotRepository
|
||||
|
||||
|
||||
class WorkplaceStatusReader(Protocol):
|
||||
def get_current_workplace_status(self, workplace: str) -> CurrentWorkplaceStatus | None: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaterialEfficiencySnapshot:
|
||||
workplace: str
|
||||
machine_id: str
|
||||
calculation_id: str
|
||||
erp_production_order: str
|
||||
enlyze_production_order: str
|
||||
article_number: str | None
|
||||
article_description: str | None
|
||||
nominal_width_m: float | None
|
||||
erp_feedback_timestamp: datetime
|
||||
material_snapshot_timestamp: datetime
|
||||
good_quantity_m2: float
|
||||
material_consumption_kg: float
|
||||
material_consumption_kg_per_m2: float
|
||||
material_consumption_g_per_m2: float
|
||||
|
||||
|
||||
class MaterialEfficiencyService:
|
||||
def __init__(
|
||||
self, erp: WorkplaceStatusReader, materials: MaterialSnapshotRepository, *,
|
||||
workplace: str, machine_id: str, calculation_id: str, format_template: str,
|
||||
erp_timezone: ZoneInfo | None = None,
|
||||
output_calculation_id: str | None = None,
|
||||
) -> None:
|
||||
self.erp = erp
|
||||
self.materials = materials
|
||||
self.workplace = workplace
|
||||
self.machine_id = machine_id
|
||||
self.calculation_id = calculation_id
|
||||
self.output_calculation_id = output_calculation_id or calculation_id
|
||||
self.format_template = format_template
|
||||
self.erp_timezone = erp_timezone
|
||||
|
||||
def evaluate_current(self) -> MaterialEfficiencySnapshot | None:
|
||||
"""Read current ERP feedback once; no scheduling or mutable KPI history."""
|
||||
status = self.erp.get_current_workplace_status(self.workplace)
|
||||
return None if status is None else self.evaluate(status)
|
||||
|
||||
def evaluate(self, status: CurrentWorkplaceStatus) -> MaterialEfficiencySnapshot | None:
|
||||
"""Evaluate supplied feedback; missing/invalid numeric inputs yield None.
|
||||
|
||||
Configuration, timestamp and adapter contract errors raise ValueError.
|
||||
Database failures propagate, rather than being treated as missing data.
|
||||
"""
|
||||
if status.workplace.strip().casefold() != self.workplace.strip().casefold():
|
||||
raise ValueError("ERP workplace does not match configured workplace")
|
||||
order = build_enlyze_production_order(status.production_order, self.format_template)
|
||||
quantity = status.good_quantity_m2
|
||||
if quantity is None or not isfinite(quantity) or quantity <= 0:
|
||||
return None
|
||||
cutoff = _feedback_instant(status.feedback_timestamp, self.erp_timezone)
|
||||
material = self.materials.latest_at_or_before(
|
||||
calculation_id=self.calculation_id, machine_id=self.machine_id,
|
||||
production_order=order, timestamp=cutoff,
|
||||
)
|
||||
if material is None:
|
||||
return None
|
||||
if (
|
||||
material.calculation_id != self.calculation_id
|
||||
or material.machine_id != self.machine_id
|
||||
or material.production_order != order
|
||||
or material.timestamp.utcoffset() is None
|
||||
or material.timestamp > cutoff
|
||||
):
|
||||
raise ValueError("Material repository returned an unaligned snapshot")
|
||||
consumption = material.consumption_kg
|
||||
if not isfinite(consumption) or consumption < 0:
|
||||
return None
|
||||
kg_per_m2 = consumption / quantity
|
||||
g_per_m2 = kg_per_m2 * 1000
|
||||
if not isfinite(kg_per_m2) or not isfinite(g_per_m2):
|
||||
return None
|
||||
return MaterialEfficiencySnapshot(
|
||||
workplace=status.workplace, machine_id=self.machine_id,
|
||||
calculation_id=self.output_calculation_id,
|
||||
erp_production_order=status.production_order,
|
||||
enlyze_production_order=order, article_number=status.article_number,
|
||||
article_description=status.article_description,
|
||||
nominal_width_m=extract_nominal_width_m(status.article_description),
|
||||
erp_feedback_timestamp=cutoff,
|
||||
material_snapshot_timestamp=material.timestamp,
|
||||
good_quantity_m2=quantity, material_consumption_kg=consumption,
|
||||
material_consumption_kg_per_m2=kg_per_m2,
|
||||
material_consumption_g_per_m2=g_per_m2,
|
||||
)
|
||||
|
||||
|
||||
def _feedback_instant(timestamp: datetime, timezone: ZoneInfo | None) -> datetime:
|
||||
if timestamp.utcoffset() is not None:
|
||||
return timestamp.astimezone(UTC)
|
||||
if timezone is None:
|
||||
raise ValueError("Naive ERP feedback timestamp requires explicit erp_timezone")
|
||||
# Round trips reject DST gaps; two distinct instants indicate a DST overlap.
|
||||
instants = set()
|
||||
for fold in (0, 1):
|
||||
instant = timestamp.replace(tzinfo=timezone, fold=fold).astimezone(UTC)
|
||||
if instant.astimezone(timezone).replace(tzinfo=None) == timestamp:
|
||||
instants.add(instant)
|
||||
if len(instants) != 1:
|
||||
raise ValueError("ERP feedback timestamp is ambiguous or nonexistent in erp_timezone")
|
||||
return instants.pop()
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Independent foreground ERP feedback polling; PostgreSQL owns deduplication."""
|
||||
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from typing import TextIO
|
||||
|
||||
import psycopg
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.erp import ErpReadError
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencyService
|
||||
from production_analytics.service.postgres_material_efficiency import MaterialEfficiencyWriter
|
||||
|
||||
|
||||
class MaterialEfficiencyRunner:
|
||||
def __init__(
|
||||
self,
|
||||
service: MaterialEfficiencyService,
|
||||
writer: MaterialEfficiencyWriter,
|
||||
*,
|
||||
poll_interval_seconds: float = 60,
|
||||
sleep: Callable[[float], None] = time.sleep,
|
||||
stderr: TextIO | None = None,
|
||||
stdout: TextIO | None = None,
|
||||
) -> None:
|
||||
self.service = service
|
||||
self.writer = writer
|
||||
self.interval = finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
self.sleep = sleep
|
||||
self.stderr = stderr if stderr is not None else sys.stderr
|
||||
self.stdout = stdout if stdout is not None else sys.stdout
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
snapshot = self.service.evaluate_current()
|
||||
if snapshot is not None and self.writer.write(snapshot):
|
||||
print(
|
||||
f"{snapshot.erp_feedback_timestamp.isoformat()} "
|
||||
f"workplace={snapshot.workplace!r} "
|
||||
f"production_order={snapshot.enlyze_production_order!r} "
|
||||
f"good_m2={snapshot.good_quantity_m2:.3f} "
|
||||
f"material_kg={snapshot.material_consumption_kg:.3f} "
|
||||
f"g_per_m2={snapshot.material_consumption_g_per_m2:.3f}",
|
||||
file=self.stdout,
|
||||
flush=True,
|
||||
)
|
||||
except (ErpReadError, psycopg.OperationalError, psycopg.InterfaceError) as exc:
|
||||
# Never include driver messages, SQL, credentials or response bodies.
|
||||
print(
|
||||
f"Material efficiency cycle failed: {type(exc).__name__}",
|
||||
file=self.stderr,
|
||||
flush=True,
|
||||
)
|
||||
# Fixed delay even after unavailable evaluations or transient failures.
|
||||
self.sleep(self.interval)
|
||||
except KeyboardInterrupt:
|
||||
return
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Small strict YAML configuration and composition for KPI persistence."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
_UniqueLoader,
|
||||
finite_number,
|
||||
)
|
||||
from production_analytics.context import build_enlyze_production_order
|
||||
from production_analytics.enlyze.exploration import ConfigurationError, load_secret_file
|
||||
from production_analytics.erp import ErpSettings, ErpWorkplaceStatusGateway
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencyService
|
||||
from production_analytics.service.material_efficiency_runner import MaterialEfficiencyRunner
|
||||
from production_analytics.service.postgres_material import (
|
||||
PostgresMaterialSnapshotRepository,
|
||||
PostgresSettings,
|
||||
)
|
||||
from production_analytics.service.postgres_material_efficiency import (
|
||||
PostgresMaterialEfficiencyWriter,
|
||||
)
|
||||
|
||||
|
||||
def build_material_efficiency_runner(
|
||||
config: Path,
|
||||
*,
|
||||
secrets_file: Path = Path("secrets/enlyze.env"),
|
||||
erp_secrets_file: Path = Path("secrets/erp.env"),
|
||||
) -> MaterialEfficiencyRunner:
|
||||
try:
|
||||
document = yaml.load(config.read_text(encoding="utf-8"), Loader=_UniqueLoader)
|
||||
except (OSError, UnicodeError, yaml.YAMLError):
|
||||
raise CalculationConfigError("Cannot read material efficiency YAML configuration") from None
|
||||
required = {
|
||||
"workplace",
|
||||
"machine_id",
|
||||
"calculation_id",
|
||||
"production_order_format",
|
||||
"erp_timezone",
|
||||
}
|
||||
if (
|
||||
not isinstance(document, dict)
|
||||
or not required <= document.keys()
|
||||
or document.keys() - required - {"poll_interval_seconds", "output_calculation_id"}
|
||||
):
|
||||
raise CalculationConfigError("Invalid material efficiency configuration fields")
|
||||
for name in required | ({"output_calculation_id"} & document.keys()):
|
||||
value = document[name]
|
||||
if not isinstance(value, str) or not value.strip() or "\x00" in value:
|
||||
raise CalculationConfigError(f"{name} must be a non-empty string without NUL bytes")
|
||||
try:
|
||||
timezone = ZoneInfo(document["erp_timezone"])
|
||||
build_enlyze_production_order("0", document["production_order_format"])
|
||||
except (ValueError, ZoneInfoNotFoundError):
|
||||
raise CalculationConfigError("Invalid ERP timezone or production order format") from None
|
||||
interval = finite_number(
|
||||
document.get("poll_interval_seconds", 60), "poll interval", positive=True
|
||||
)
|
||||
environment = dict(os.environ)
|
||||
try:
|
||||
environment.update(load_secret_file(secrets_file))
|
||||
except Exception:
|
||||
raise ConfigurationError("PostgreSQL settings file could not be loaded") from None
|
||||
postgres = PostgresSettings.from_environment(environment)
|
||||
erp = ErpSettings.from_secret_file(erp_secrets_file)
|
||||
return MaterialEfficiencyRunner(
|
||||
MaterialEfficiencyService(
|
||||
ErpWorkplaceStatusGateway(erp),
|
||||
PostgresMaterialSnapshotRepository(postgres),
|
||||
workplace=document["workplace"],
|
||||
machine_id=document["machine_id"],
|
||||
calculation_id=document["calculation_id"],
|
||||
format_template=document["production_order_format"],
|
||||
erp_timezone=timezone,
|
||||
output_calculation_id=document.get("output_calculation_id"),
|
||||
),
|
||||
PostgresMaterialEfficiencyWriter(postgres),
|
||||
poll_interval_seconds=interval,
|
||||
)
|
||||
@@ -0,0 +1,198 @@
|
||||
"""Single-cycle live material polling orchestration."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Protocol
|
||||
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialConsumptionIntegrator,
|
||||
MaterialIntegrationState,
|
||||
MaterialIntegratorConfig,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import (
|
||||
EnlyzeApiGateway,
|
||||
EnlyzeProductionRun,
|
||||
)
|
||||
from production_analytics.service.postgres_material_application import MaterialApplicationWriter
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaterialPollingState:
|
||||
run_id: str
|
||||
integration_state: MaterialIntegrationState
|
||||
|
||||
|
||||
class MaterialStateStore(Protocol):
|
||||
def load(
|
||||
self,
|
||||
machine_id: str,
|
||||
production_order: str,
|
||||
) -> MaterialPollingState | None: ...
|
||||
|
||||
def save(
|
||||
self,
|
||||
machine_id: str,
|
||||
production_order: str,
|
||||
state: MaterialPollingState,
|
||||
) -> None: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaterialPollResult:
|
||||
run: EnlyzeProductionRun
|
||||
state: MaterialIntegrationState
|
||||
|
||||
|
||||
class MaterialPollingService:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
gateway: EnlyzeApiGateway,
|
||||
state_store: MaterialStateStore,
|
||||
machine_id: str,
|
||||
rate_variable_id: str,
|
||||
gate_variable_id: str | None,
|
||||
gate_threshold: float,
|
||||
max_sample_gap_seconds: float,
|
||||
application_variable_ids: tuple[str, ...] = (),
|
||||
rotational_speed_variable_ids: tuple[str, ...] = (),
|
||||
specific_discharge_kg_per_rev_m: float | None = None,
|
||||
nominal_width_provider: Callable[[str], float] | None = None,
|
||||
process_application_calculation_id: str | None = None,
|
||||
application_writer: MaterialApplicationWriter | None = None,
|
||||
) -> None:
|
||||
if application_variable_ids and rotational_speed_variable_ids:
|
||||
raise ValueError("Material source modes are mutually exclusive")
|
||||
if process_application_calculation_id is not None and (
|
||||
not rotational_speed_variable_ids or not gate_variable_id
|
||||
or gate_threshold <= 0 or application_writer is None
|
||||
):
|
||||
raise ValueError(
|
||||
"Process application requires rotational source, positive gate and writer"
|
||||
)
|
||||
self._process_application_calculation_id = process_application_calculation_id
|
||||
self._application_writer = application_writer
|
||||
self._rotational_speed_variable_ids = rotational_speed_variable_ids
|
||||
self._specific_discharge = specific_discharge_kg_per_rev_m
|
||||
self._application_variable_ids = application_variable_ids
|
||||
self._nominal_width_provider = nominal_width_provider
|
||||
if ((application_variable_ids or rotational_speed_variable_ids)
|
||||
and nominal_width_provider is None):
|
||||
raise ValueError("width-based source requires a nominal width provider")
|
||||
self._gateway = gateway
|
||||
self._state_store = state_store
|
||||
self._machine_id = machine_id
|
||||
self._rate_variable_id = rate_variable_id
|
||||
self._gate_variable_id = gate_variable_id
|
||||
self._config = MaterialIntegratorConfig(
|
||||
gate_threshold=gate_threshold if gate_variable_id is not None else 0.0,
|
||||
max_sample_gap_seconds=max_sample_gap_seconds,
|
||||
)
|
||||
|
||||
def poll_once(self, *, now: datetime) -> MaterialPollResult | None:
|
||||
if now.tzinfo is None or now.utcoffset() is None:
|
||||
raise ValueError("now must be timezone-aware")
|
||||
run = self._gateway.get_open_production_run(self._machine_id)
|
||||
if run is None or now < run.start:
|
||||
return None
|
||||
|
||||
width = (self._nominal_width_provider(run.production_order)
|
||||
if self._application_variable_ids or self._rotational_speed_variable_ids else None)
|
||||
saved_polling_state = self._state_store.load(
|
||||
self._machine_id,
|
||||
run.production_order,
|
||||
)
|
||||
|
||||
if saved_polling_state is None:
|
||||
initial_state = MaterialIntegrationState()
|
||||
historical_runs = sorted(
|
||||
(
|
||||
previous for previous in self._gateway.get_production_runs(self._machine_id)
|
||||
if previous.production_order == run.production_order
|
||||
and previous.uuid != run.uuid
|
||||
and previous.start <= run.start
|
||||
),
|
||||
key=lambda previous: previous.start,
|
||||
)
|
||||
for previous in historical_runs:
|
||||
if previous.end is None or previous.end > run.start:
|
||||
raise ValueError("Historical production run overlaps the current open run")
|
||||
integrated = self._integrate(
|
||||
initial_state, start=previous.start, end=previous.end,
|
||||
width=width, run=previous,
|
||||
)
|
||||
initial_state = MaterialIntegrationState(
|
||||
cumulative_consumption_kg=integrated.cumulative_consumption_kg,
|
||||
integrated_running_seconds=integrated.integrated_running_seconds,
|
||||
)
|
||||
start = run.start
|
||||
elif saved_polling_state.run_id == run.uuid:
|
||||
initial_state = saved_polling_state.integration_state
|
||||
start = (
|
||||
initial_state.last_processed_timestamp
|
||||
if initial_state.last_processed_timestamp is not None
|
||||
else run.start
|
||||
)
|
||||
else:
|
||||
previous = saved_polling_state.integration_state
|
||||
initial_state = MaterialIntegrationState(
|
||||
cumulative_consumption_kg=previous.cumulative_consumption_kg,
|
||||
integrated_running_seconds=previous.integrated_running_seconds,
|
||||
)
|
||||
start = run.start
|
||||
|
||||
if now < start:
|
||||
return None
|
||||
|
||||
state = self._integrate(initial_state, start=start, end=now, width=width, run=run)
|
||||
|
||||
self._state_store.save(
|
||||
self._machine_id,
|
||||
run.production_order,
|
||||
MaterialPollingState(
|
||||
run_id=run.uuid,
|
||||
integration_state=state,
|
||||
),
|
||||
)
|
||||
|
||||
return MaterialPollResult(
|
||||
run=run,
|
||||
state=state,
|
||||
)
|
||||
|
||||
def _integrate(
|
||||
self, initial_state: MaterialIntegrationState, *, start: datetime, end: datetime,
|
||||
width: float | None = None, run: EnlyzeProductionRun | None = None,
|
||||
) -> MaterialIntegrationState:
|
||||
integrator = MaterialConsumptionIntegrator(self._config, initial_state=initial_state)
|
||||
source_options = {}
|
||||
if self._application_variable_ids:
|
||||
source_options = dict(
|
||||
application_variable_ids=self._application_variable_ids, nominal_width_m=width,
|
||||
)
|
||||
if self._rotational_speed_variable_ids:
|
||||
source_options = dict(
|
||||
rotational_speed_variable_ids=self._rotational_speed_variable_ids,
|
||||
specific_discharge_kg_per_rev_m=self._specific_discharge, nominal_width_m=width,
|
||||
)
|
||||
if self._process_application_calculation_id is not None:
|
||||
source_options["process_application_gate_threshold"] = self._config.gate_threshold
|
||||
samples = self._gateway.get_material_samples(
|
||||
machine_id=self._machine_id,
|
||||
rate_variable_id=self._rate_variable_id,
|
||||
gate_variable_id=self._gate_variable_id,
|
||||
start=start,
|
||||
end=end,
|
||||
**source_options,
|
||||
)
|
||||
state = integrator.process_many(samples)
|
||||
if self._process_application_calculation_id is not None:
|
||||
assert self._application_writer is not None and run is not None
|
||||
# Persist before advancing the checkpoint, so failed writes can be retried.
|
||||
self._application_writer.write(
|
||||
calculation_id=self._process_application_calculation_id,
|
||||
machine_id=self._machine_id, production_order=run.production_order,
|
||||
run_id=run.uuid, samples=samples,
|
||||
)
|
||||
return state
|
||||
@@ -0,0 +1,88 @@
|
||||
"""Sequential foreground polling with injectable time and output."""
|
||||
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from datetime import UTC, datetime
|
||||
from typing import TextIO
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.enlyze.exploration import ConfigurationError, ExplorationError
|
||||
from production_analytics.service.material_polling import MaterialPollingService
|
||||
from production_analytics.service.postgres_material import MaterialSnapshotWriter
|
||||
|
||||
|
||||
class MaterialStateError(RuntimeError):
|
||||
"""State operation failed; message contains only the operation and error class."""
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
class MaterialPollingRunner:
|
||||
def __init__(
|
||||
self, service: MaterialPollingService, *, machine_id: str,
|
||||
calculation_id: str, snapshot_writer: MaterialSnapshotWriter,
|
||||
poll_interval_seconds: float, clock: Callable[[], datetime] = utc_now,
|
||||
sleep: Callable[[float], None] = time.sleep,
|
||||
stdout: TextIO | None = None, stderr: TextIO | None = None,
|
||||
) -> None:
|
||||
self.service = service
|
||||
self.machine_id = machine_id
|
||||
self.calculation_id = calculation_id
|
||||
self.snapshot_writer = snapshot_writer
|
||||
self.interval = finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
self.clock = clock
|
||||
self.sleep = sleep
|
||||
self.stdout = stdout if stdout is not None else sys.stdout
|
||||
self.stderr = stderr if stderr is not None else sys.stderr
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
now = self.clock()
|
||||
if now.tzinfo is None or now.utcoffset() is None:
|
||||
raise ConfigurationError("Runner clock must return timezone-aware timestamps")
|
||||
now = now.astimezone(UTC)
|
||||
prefix = f"{now.isoformat()} machine={self.machine_id!r}"
|
||||
try:
|
||||
result = self.service.poll_once(now=now)
|
||||
except ConfigurationError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
if isinstance(exc, MaterialStateError):
|
||||
detail = str(exc)
|
||||
elif isinstance(exc, ExplorationError):
|
||||
detail = f"ENLYZE/gateway: {type(exc).__name__}"
|
||||
else:
|
||||
detail = f"gateway/poll cycle: {type(exc).__name__}"
|
||||
# Do not echo arbitrary exception messages or response bodies.
|
||||
print(f"{prefix} error {detail}", file=self.stderr, flush=True)
|
||||
else:
|
||||
if result is None:
|
||||
detail = "no open Production Run / no eligible polling window"
|
||||
else:
|
||||
try:
|
||||
self.snapshot_writer.write(
|
||||
timestamp=now, calculation_id=self.calculation_id,
|
||||
machine_id=self.machine_id,
|
||||
production_order=result.run.production_order,
|
||||
run_id=result.run.uuid,
|
||||
consumption_kg=result.state.cumulative_consumption_kg,
|
||||
)
|
||||
except Exception:
|
||||
print(
|
||||
f"{prefix} error PostgreSQL snapshot write failed",
|
||||
file=self.stderr, flush=True,
|
||||
)
|
||||
detail = (
|
||||
f"production_order={result.run.production_order!r} "
|
||||
f"consumption_kg={result.state.cumulative_consumption_kg:.9f} "
|
||||
f"running_seconds={result.state.integrated_running_seconds:.3f}"
|
||||
)
|
||||
print(f"{prefix} {detail}", file=self.stdout, flush=True)
|
||||
# Fixed delay after completion, including errors; never catch up or overlap.
|
||||
self.sleep(self.interval)
|
||||
except KeyboardInterrupt:
|
||||
print("Material polling stopped", file=self.stdout, flush=True)
|
||||
@@ -0,0 +1,113 @@
|
||||
"""Compose the configured live material polling application."""
|
||||
|
||||
import math
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from production_analytics.calculations.config import MaterialCalculationConfig, finite_number
|
||||
from production_analytics.enlyze.exploration import (
|
||||
ConfigurationError,
|
||||
ExplorationClient,
|
||||
ExplorationSettings,
|
||||
load_secret_file,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
from production_analytics.erp import ErpSettings, ErpWorkplaceStatusGateway
|
||||
from production_analytics.service.material_context import ErpNominalWidthProvider
|
||||
from production_analytics.service.material_polling import (
|
||||
MaterialPollingService,
|
||||
MaterialPollingState,
|
||||
)
|
||||
from production_analytics.service.material_runner import MaterialPollingRunner, MaterialStateError
|
||||
from production_analytics.service.material_state_store import JsonMaterialStateStore
|
||||
from production_analytics.service.postgres_material import (
|
||||
PostgresMaterialSnapshotWriter,
|
||||
PostgresSettings,
|
||||
)
|
||||
from production_analytics.service.postgres_material_application import (
|
||||
PostgresMaterialApplicationWriter,
|
||||
)
|
||||
|
||||
|
||||
class _ReportingStateStore:
|
||||
def __init__(self, store: JsonMaterialStateStore) -> None:
|
||||
self.store = store
|
||||
|
||||
def load(self, machine_id: str, production_order: str) -> MaterialPollingState | None:
|
||||
try:
|
||||
return self.store.load(machine_id, production_order)
|
||||
except Exception as exc:
|
||||
raise MaterialStateError(f"state loading: {type(exc).__name__}") from exc
|
||||
|
||||
def save(
|
||||
self, machine_id: str, production_order: str, state: MaterialPollingState,
|
||||
) -> None:
|
||||
try:
|
||||
self.store.save(machine_id, production_order, state)
|
||||
except Exception as exc:
|
||||
raise MaterialStateError(f"state saving: {type(exc).__name__}") from exc
|
||||
|
||||
|
||||
def build_material_runner(
|
||||
calculation: MaterialCalculationConfig, *, poll_interval_seconds: float,
|
||||
state_directory: Path, secrets_file: Path,
|
||||
erp_secrets_file: Path = Path("secrets/erp.env"),
|
||||
) -> MaterialPollingRunner:
|
||||
finite_number(poll_interval_seconds, "poll interval", positive=True)
|
||||
environment = dict(os.environ)
|
||||
environment.update(load_secret_file(secrets_file))
|
||||
settings = ExplorationSettings.from_environment(environment)
|
||||
parsed = urlsplit(settings.base_url)
|
||||
if parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username:
|
||||
raise ConfigurationError(
|
||||
"ENLYZE_BASE_URL must be an HTTP(S) server URL without credentials"
|
||||
)
|
||||
if not math.isfinite(settings.timeout_seconds):
|
||||
raise ConfigurationError("ENLYZE_HTTP_TIMEOUT_SECONDS must be finite")
|
||||
try:
|
||||
state_directory.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryFile(dir=state_directory):
|
||||
pass
|
||||
except OSError as exc:
|
||||
raise ConfigurationError(
|
||||
f"State directory startup check failed: {type(exc).__name__}"
|
||||
) from exc
|
||||
postgres_settings = PostgresSettings.from_environment(environment)
|
||||
source_options = {}
|
||||
if calculation.source_mode in {"area_application", "rotational_discharge"}:
|
||||
source_options = dict(
|
||||
application_variable_ids=calculation.application_signal_refs,
|
||||
nominal_width_provider=ErpNominalWidthProvider(
|
||||
ErpWorkplaceStatusGateway(ErpSettings.from_secret_file(erp_secrets_file)),
|
||||
workplace=calculation.erp_workplace,
|
||||
format_template=calculation.production_order_format,
|
||||
),
|
||||
)
|
||||
if calculation.source_mode == "rotational_discharge":
|
||||
source_options.pop("application_variable_ids")
|
||||
source_options.update(
|
||||
rotational_speed_variable_ids=calculation.rotational_speed_signal_refs,
|
||||
specific_discharge_kg_per_rev_m=calculation.specific_discharge_kg_per_rev_m,
|
||||
)
|
||||
if calculation.process_application_calculation_id is not None:
|
||||
source_options.update(
|
||||
process_application_calculation_id=calculation.process_application_calculation_id,
|
||||
application_writer=PostgresMaterialApplicationWriter(postgres_settings),
|
||||
)
|
||||
service = MaterialPollingService(
|
||||
gateway=EnlyzeApiGateway(ExplorationClient(settings)),
|
||||
state_store=_ReportingStateStore(JsonMaterialStateStore(state_directory)),
|
||||
machine_id=calculation.machine_ref,
|
||||
rate_variable_id=calculation.rate_signal_ref,
|
||||
gate_variable_id=calculation.gate_signal_ref,
|
||||
gate_threshold=calculation.gate_threshold,
|
||||
max_sample_gap_seconds=calculation.max_sample_gap_seconds,
|
||||
**source_options,
|
||||
)
|
||||
return MaterialPollingRunner(
|
||||
service, calculation_id=calculation.id,
|
||||
snapshot_writer=PostgresMaterialSnapshotWriter(postgres_settings),
|
||||
machine_id=calculation.machine_ref, poll_interval_seconds=poll_interval_seconds,
|
||||
)
|
||||
@@ -0,0 +1,80 @@
|
||||
"""JSON-backed persistence for material polling state."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from production_analytics.calculations.material_state import (
|
||||
material_state_from_dict,
|
||||
material_state_to_dict,
|
||||
)
|
||||
from production_analytics.service.material_polling import MaterialPollingState
|
||||
|
||||
|
||||
class JsonMaterialStateStore:
|
||||
def __init__(self, directory: str | Path) -> None:
|
||||
self._directory = Path(directory)
|
||||
|
||||
def load(
|
||||
self,
|
||||
machine_id: str,
|
||||
production_order: str,
|
||||
) -> MaterialPollingState | None:
|
||||
path = self._path(machine_id, production_order)
|
||||
if not path.exists():
|
||||
return None
|
||||
|
||||
try:
|
||||
with path.open(encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict) or not isinstance(data.get("run_id"), str):
|
||||
raise ValueError("run_id must be a string")
|
||||
if not data["run_id"]:
|
||||
raise ValueError("run_id must not be empty")
|
||||
return MaterialPollingState(
|
||||
run_id=data["run_id"],
|
||||
integration_state=material_state_from_dict(data["integration_state"]),
|
||||
)
|
||||
except (ValueError, KeyError, TypeError) as exc:
|
||||
raise ValueError(f"Invalid material polling state in {path}: {exc}") from exc
|
||||
|
||||
def save(
|
||||
self,
|
||||
machine_id: str,
|
||||
production_order: str,
|
||||
state: MaterialPollingState,
|
||||
) -> None:
|
||||
self._directory.mkdir(parents=True, exist_ok=True)
|
||||
path = self._path(machine_id, production_order)
|
||||
|
||||
payload = {
|
||||
"run_id": state.run_id,
|
||||
"integration_state": material_state_to_dict(state.integration_state),
|
||||
}
|
||||
|
||||
# Validate before touching the primary file, including non-finite values.
|
||||
material_state_from_dict(payload["integration_state"])
|
||||
if not isinstance(state.run_id, str) or not state.run_id:
|
||||
raise ValueError("run_id must be a non-empty string")
|
||||
temporary_path = None
|
||||
try:
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w", encoding="utf-8", dir=self._directory,
|
||||
prefix=".material-state-", suffix=".tmp", delete=False,
|
||||
) as f:
|
||||
temporary_path = Path(f.name)
|
||||
json.dump(payload, f, indent=2, allow_nan=False)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(temporary_path, path)
|
||||
finally:
|
||||
if temporary_path is not None:
|
||||
temporary_path.unlink(missing_ok=True)
|
||||
|
||||
def _path(self, machine_id: str, production_order: str) -> Path:
|
||||
# Hash the structured pair: sanitizing alone aliases distinct identifiers.
|
||||
identity = json.dumps([machine_id, production_order], ensure_ascii=True)
|
||||
digest = hashlib.sha256(identity.encode("utf-8")).hexdigest()
|
||||
return self._directory / f"{digest}.json"
|
||||
@@ -0,0 +1,63 @@
|
||||
"""PostgreSQL writer for channel-level cumulative consumption snapshots."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
class PostgresChannelMaterialSnapshotWriter:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def write(
|
||||
self,
|
||||
*,
|
||||
timestamp: datetime,
|
||||
calculation_id: str,
|
||||
machine_id: str,
|
||||
extruder: str,
|
||||
channel: str,
|
||||
production_order: str,
|
||||
run_id: str,
|
||||
cumulative_consumption_kg: float,
|
||||
material_number: str | None,
|
||||
material_name: str | None,
|
||||
material_mapping_status: str,
|
||||
percentage_sum_valid: bool,
|
||||
) -> None:
|
||||
import psycopg
|
||||
|
||||
with psycopg.connect(
|
||||
host=self.settings.host,
|
||||
port=self.settings.port,
|
||||
dbname=self.settings.dbname,
|
||||
user=self.settings.user,
|
||||
password=self.settings.password,
|
||||
connect_timeout=10,
|
||||
options="-c statement_timeout=10000",
|
||||
) as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO channel_material_consumption_snapshots
|
||||
(timestamp, calculation_id, machine_id, extruder, doser_channel,
|
||||
production_order, run_id, cumulative_consumption_kg,
|
||||
material_number, material_name, material_mapping_status,
|
||||
percentage_sum_valid)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (calculation_id, machine_id, extruder, doser_channel,
|
||||
production_order, timestamp)
|
||||
DO NOTHING""",
|
||||
(
|
||||
timestamp,
|
||||
calculation_id,
|
||||
machine_id,
|
||||
extruder,
|
||||
channel,
|
||||
production_order,
|
||||
run_id,
|
||||
cumulative_consumption_kg,
|
||||
material_number,
|
||||
material_name,
|
||||
material_mapping_status,
|
||||
percentage_sum_valid,
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,112 @@
|
||||
"""PostgreSQL persistence for reconciled ENLYZE downtime events."""
|
||||
|
||||
from production_analytics.service.downtime import ProductionDowntimeEvent, ProductionOrderBoundary
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
class PostgresDowntimeRepository:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def _connect(self):
|
||||
import psycopg
|
||||
|
||||
return psycopg.connect(
|
||||
host=self.settings.host,
|
||||
port=self.settings.port,
|
||||
dbname=self.settings.dbname,
|
||||
user=self.settings.user,
|
||||
password=self.settings.password,
|
||||
connect_timeout=10,
|
||||
options="-c statement_timeout=10000",
|
||||
)
|
||||
|
||||
def current_boundary(self, machine_id: str) -> ProductionOrderBoundary | None:
|
||||
with self._connect() as connection:
|
||||
row = connection.execute(
|
||||
"""SELECT machine_id, production_order, started_at, ended_at
|
||||
FROM production_order_attribution_state WHERE machine_id = %s""",
|
||||
(machine_id,),
|
||||
).fetchone()
|
||||
return None if row is None else ProductionOrderBoundary(*row)
|
||||
|
||||
def save_boundary(self, boundary: ProductionOrderBoundary) -> None:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO production_order_attribution_state
|
||||
(machine_id, production_order, started_at, ended_at)
|
||||
VALUES (%s, %s, %s, %s)
|
||||
ON CONFLICT (machine_id) DO UPDATE SET
|
||||
production_order = EXCLUDED.production_order,
|
||||
started_at = EXCLUDED.started_at, ended_at = EXCLUDED.ended_at""",
|
||||
(
|
||||
boundary.machine_id,
|
||||
boundary.production_order,
|
||||
boundary.started_at,
|
||||
boundary.ended_at,
|
||||
),
|
||||
)
|
||||
|
||||
def close_order_attribution(self, machine_id: str, production_order: str, ended_at) -> None:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""UPDATE production_downtime_events
|
||||
SET attributed_end = CASE WHEN source_end IS NULL OR source_end > %s THEN %s
|
||||
ELSE source_end END,
|
||||
attributed_duration_seconds = EXTRACT(EPOCH FROM
|
||||
(CASE WHEN source_end IS NULL OR source_end > %s
|
||||
THEN %s ELSE source_end END
|
||||
- attributed_start))
|
||||
WHERE machine_id = %s AND production_order = %s
|
||||
AND attributed_start IS NOT NULL AND attributed_end IS NULL""",
|
||||
(ended_at, ended_at, ended_at, ended_at, machine_id, production_order),
|
||||
)
|
||||
|
||||
def upsert(self, event: ProductionDowntimeEvent) -> None:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO production_downtime_events
|
||||
(external_id, machine_id, production_order, source_type, source_start,
|
||||
source_end, attributed_start, attributed_end, attributed_duration_seconds,
|
||||
reason_id, reason_name, reason_description, reason_group, category, comment,
|
||||
source_updated_at, last_reconciled_at)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (external_id) DO UPDATE SET
|
||||
machine_id = EXCLUDED.machine_id,
|
||||
production_order = COALESCE(
|
||||
EXCLUDED.production_order, production_downtime_events.production_order),
|
||||
source_type = EXCLUDED.source_type, source_start = EXCLUDED.source_start,
|
||||
source_end = EXCLUDED.source_end,
|
||||
attributed_start = COALESCE(
|
||||
EXCLUDED.attributed_start, production_downtime_events.attributed_start),
|
||||
attributed_end = COALESCE(
|
||||
EXCLUDED.attributed_end, production_downtime_events.attributed_end),
|
||||
attributed_duration_seconds = COALESCE(
|
||||
EXCLUDED.attributed_duration_seconds,
|
||||
production_downtime_events.attributed_duration_seconds),
|
||||
reason_id = EXCLUDED.reason_id, reason_name = EXCLUDED.reason_name,
|
||||
reason_description = EXCLUDED.reason_description,
|
||||
reason_group = EXCLUDED.reason_group,
|
||||
category = EXCLUDED.category, comment = EXCLUDED.comment,
|
||||
source_updated_at = EXCLUDED.source_updated_at,
|
||||
last_reconciled_at = EXCLUDED.last_reconciled_at""",
|
||||
(
|
||||
event.external_id,
|
||||
event.machine_id,
|
||||
event.production_order,
|
||||
event.source_type,
|
||||
event.source_start,
|
||||
event.source_end,
|
||||
event.attributed_start,
|
||||
event.attributed_end,
|
||||
event.attributed_duration_seconds,
|
||||
event.reason_id,
|
||||
event.reason_name,
|
||||
event.reason_description,
|
||||
event.reason_group,
|
||||
event.category.value,
|
||||
event.comment,
|
||||
event.source_updated_at,
|
||||
event.last_reconciled_at,
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,116 @@
|
||||
"""PostgreSQL settings and derived material snapshot reads/writes."""
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from typing import Protocol
|
||||
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PostgresSettings:
|
||||
host: str
|
||||
port: int
|
||||
dbname: str
|
||||
user: str
|
||||
password: str = field(repr=False)
|
||||
|
||||
@classmethod
|
||||
def from_environment(cls, environment: Mapping[str, str]) -> "PostgresSettings":
|
||||
values = {}
|
||||
for key in ('HOST', 'PORT', 'DB', 'USER', 'PASSWORD'):
|
||||
name = f'POSTGRES_{key}'
|
||||
value = environment.get(name)
|
||||
if not isinstance(value, str) or not value.strip() or '\x00' in value:
|
||||
raise ConfigurationError(f'{name} must be a non-empty value without NUL bytes')
|
||||
values[key] = value
|
||||
try:
|
||||
port = int(values['PORT'])
|
||||
except ValueError:
|
||||
raise ConfigurationError('POSTGRES_PORT must be an integer from 1 to 65535') from None
|
||||
if not 1 <= port <= 65535:
|
||||
raise ConfigurationError('POSTGRES_PORT must be an integer from 1 to 65535')
|
||||
return cls(values['HOST'], port, values['DB'], values['USER'], values['PASSWORD'])
|
||||
|
||||
|
||||
class MaterialSnapshotWriter(Protocol):
|
||||
def write(
|
||||
self, *, timestamp: datetime, calculation_id: str, machine_id: str,
|
||||
production_order: str, run_id: str, consumption_kg: float,
|
||||
) -> None: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaterialConsumptionSnapshot:
|
||||
timestamp: datetime
|
||||
calculation_id: str
|
||||
machine_id: str
|
||||
production_order: str
|
||||
run_id: str
|
||||
consumption_kg: float
|
||||
|
||||
|
||||
class MaterialSnapshotRepository(Protocol):
|
||||
def latest_at_or_before(
|
||||
self, *, calculation_id: str, machine_id: str, production_order: str,
|
||||
timestamp: datetime,
|
||||
) -> MaterialConsumptionSnapshot | None: ...
|
||||
|
||||
|
||||
class PostgresMaterialSnapshotRepository:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def latest_at_or_before(
|
||||
self, *, calculation_id: str, machine_id: str, production_order: str,
|
||||
timestamp: datetime,
|
||||
) -> MaterialConsumptionSnapshot | None:
|
||||
"""Read one exact-order cumulative snapshot, never later than the aware cutoff."""
|
||||
if timestamp.utcoffset() is None:
|
||||
raise ValueError("Material snapshot cutoff must be timezone-aware")
|
||||
import psycopg
|
||||
|
||||
with psycopg.connect(
|
||||
host=self.settings.host, port=self.settings.port, dbname=self.settings.dbname,
|
||||
user=self.settings.user, password=self.settings.password, connect_timeout=10,
|
||||
options='-c statement_timeout=10000',
|
||||
) as connection:
|
||||
row = connection.execute(
|
||||
"""SELECT timestamp, calculation_id, machine_id, production_order,
|
||||
run_id, consumption_kg
|
||||
FROM material_consumption_snapshots
|
||||
WHERE calculation_id = %s AND machine_id = %s
|
||||
AND production_order = %s AND timestamp <= %s
|
||||
ORDER BY timestamp DESC
|
||||
LIMIT 1""",
|
||||
(calculation_id, machine_id, production_order, timestamp),
|
||||
).fetchone()
|
||||
return None if row is None else MaterialConsumptionSnapshot(*row)
|
||||
|
||||
|
||||
class PostgresMaterialSnapshotWriter:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def write(
|
||||
self, *, timestamp: datetime, calculation_id: str, machine_id: str,
|
||||
production_order: str, run_id: str, consumption_kg: float,
|
||||
) -> None:
|
||||
import psycopg
|
||||
|
||||
# One short transaction per cycle; context exit commits or rolls back and closes.
|
||||
# A fresh connection lets the next cycle recover after a database outage.
|
||||
with psycopg.connect(
|
||||
host=self.settings.host, port=self.settings.port, dbname=self.settings.dbname,
|
||||
user=self.settings.user, password=self.settings.password, connect_timeout=10,
|
||||
options='-c statement_timeout=10000',
|
||||
) as connection:
|
||||
connection.execute(
|
||||
"""INSERT INTO material_consumption_snapshots
|
||||
(timestamp, calculation_id, machine_id, production_order, run_id, consumption_kg)
|
||||
VALUES (%s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (calculation_id, machine_id, production_order, timestamp)
|
||||
DO NOTHING""",
|
||||
(timestamp, calculation_id, machine_id, production_order, run_id, consumption_kg),
|
||||
)
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Generic process application snapshots, timestamped at the source sample."""
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Protocol
|
||||
|
||||
import psycopg
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.calculations.material_consumption import MaterialSample
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
class MaterialApplicationWriter(Protocol):
|
||||
def write(
|
||||
self, *, calculation_id: str, machine_id: str, production_order: str,
|
||||
run_id: str, samples: Sequence[MaterialSample],
|
||||
) -> None: ...
|
||||
|
||||
|
||||
class PostgresMaterialApplicationWriter:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def write(
|
||||
self, *, calculation_id: str, machine_id: str, production_order: str,
|
||||
run_id: str, samples: Sequence[MaterialSample],
|
||||
) -> None:
|
||||
rows = []
|
||||
for sample in samples:
|
||||
if sample.application_g_m2 is None:
|
||||
continue
|
||||
if sample.timestamp.utcoffset() is None:
|
||||
raise ValueError("Application timestamp must be timezone-aware")
|
||||
finite_number(sample.application_g_m2, "application_g_m2")
|
||||
rows.append((sample.timestamp, calculation_id, machine_id, production_order,
|
||||
run_id, sample.application_g_m2))
|
||||
if not rows:
|
||||
return
|
||||
with psycopg.connect(
|
||||
host=self.settings.host, port=self.settings.port, dbname=self.settings.dbname,
|
||||
user=self.settings.user, password=self.settings.password, connect_timeout=10,
|
||||
options="-c statement_timeout=10000",
|
||||
) as connection:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.executemany(
|
||||
"""INSERT INTO material_application_snapshots
|
||||
(timestamp, calculation_id, machine_id, production_order,
|
||||
run_id, application_g_m2)
|
||||
VALUES (%s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (calculation_id, machine_id, production_order, timestamp)
|
||||
DO NOTHING""",
|
||||
rows,
|
||||
)
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Immutable PostgreSQL history of feedback-aligned KPI points."""
|
||||
|
||||
from typing import Protocol
|
||||
|
||||
import psycopg
|
||||
|
||||
from production_analytics.calculations.config import finite_number
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencySnapshot
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
|
||||
|
||||
class MaterialEfficiencyWriter(Protocol):
|
||||
def write(self, snapshot: MaterialEfficiencySnapshot) -> bool: ...
|
||||
|
||||
|
||||
class PostgresMaterialEfficiencyWriter:
|
||||
def __init__(self, settings: PostgresSettings) -> None:
|
||||
self.settings = settings
|
||||
|
||||
def write(self, snapshot: MaterialEfficiencySnapshot) -> bool:
|
||||
for name in ("erp_feedback_timestamp", "material_snapshot_timestamp"):
|
||||
if getattr(snapshot, name).utcoffset() is None:
|
||||
raise ValueError(f"{name} must be timezone-aware")
|
||||
for name in (
|
||||
"good_quantity_m2",
|
||||
"material_consumption_kg",
|
||||
"material_consumption_kg_per_m2",
|
||||
"material_consumption_g_per_m2",
|
||||
):
|
||||
finite_number(getattr(snapshot, name), name)
|
||||
if snapshot.nominal_width_m is not None:
|
||||
finite_number(snapshot.nominal_width_m, "nominal_width_m")
|
||||
# Context exit commits/rolls back and closes; later cycles reconnect after outages.
|
||||
with psycopg.connect(
|
||||
host=self.settings.host,
|
||||
port=self.settings.port,
|
||||
dbname=self.settings.dbname,
|
||||
user=self.settings.user,
|
||||
password=self.settings.password,
|
||||
connect_timeout=10,
|
||||
options="-c statement_timeout=10000",
|
||||
) as connection:
|
||||
row = connection.execute(
|
||||
"""INSERT INTO material_efficiency_snapshots (
|
||||
erp_feedback_timestamp, material_snapshot_timestamp, calculation_id,
|
||||
machine_id, workplace, erp_production_order, enlyze_production_order,
|
||||
article_number, article_description, nominal_width_m, good_quantity_m2,
|
||||
material_consumption_kg, material_consumption_kg_per_m2,
|
||||
material_consumption_g_per_m2
|
||||
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (calculation_id, machine_id, enlyze_production_order,
|
||||
erp_feedback_timestamp) DO NOTHING
|
||||
RETURNING 1""",
|
||||
(
|
||||
snapshot.erp_feedback_timestamp,
|
||||
snapshot.material_snapshot_timestamp,
|
||||
snapshot.calculation_id,
|
||||
snapshot.machine_id,
|
||||
snapshot.workplace,
|
||||
snapshot.erp_production_order,
|
||||
snapshot.enlyze_production_order,
|
||||
snapshot.article_number,
|
||||
snapshot.article_description,
|
||||
snapshot.nominal_width_m,
|
||||
snapshot.good_quantity_m2,
|
||||
snapshot.material_consumption_kg,
|
||||
snapshot.material_consumption_kg_per_m2,
|
||||
snapshot.material_consumption_g_per_m2,
|
||||
),
|
||||
).fetchone()
|
||||
return row is not None
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Shared test adaptations for the production PostgreSQL schema."""
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def sqlite_schema() -> str:
|
||||
"""Keep PostgreSQL DDL authoritative; adapt JSON storage and clock for SQLite.
|
||||
|
||||
SQLite stores serialized JSON as text. These tests exercise schema and INSERT
|
||||
semantics, not PostgreSQL JSONB operators or TimescaleDB behavior.
|
||||
"""
|
||||
schema = (Path(__file__).resolve().parents[1] / "db" / "schema.sql").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
schema = re.sub(r"::jsonb\b", "", schema)
|
||||
schema = re.sub(r"\bjsonb\b", "text", schema)
|
||||
return schema.replace("DEFAULT now()", "DEFAULT CURRENT_TIMESTAMP")
|
||||
@@ -0,0 +1,257 @@
|
||||
from dataclasses import replace
|
||||
from datetime import datetime, timedelta
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialConsumptionIntegrator,
|
||||
MaterialIntegratorConfig,
|
||||
area_application_rate_kg_per_hour,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway, EnlyzeProductionRun
|
||||
from production_analytics.service.material_context import ErpNominalWidthProvider
|
||||
from production_analytics.service.material_polling import MaterialPollingService
|
||||
from production_analytics.service.material_state_store import JsonMaterialStateStore
|
||||
|
||||
|
||||
def timestamp(value):
|
||||
return datetime.fromisoformat(value.replace('Z', '+00:00'))
|
||||
|
||||
|
||||
def test_area_units_and_strict_gate():
|
||||
client = Mock()
|
||||
start = timestamp('2026-08-25T05:27:18Z')
|
||||
client.post_json.return_value.body = {'data': {
|
||||
'columns': ['speed', 's2', 'time', 's1'],
|
||||
'records': [[speed, 2000, (start + timedelta(minutes=i)).isoformat(), 3000]
|
||||
for i, speed in enumerate([2, 0.3, 0, 2, 2])],
|
||||
}}
|
||||
samples = EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id='machine', rate_variable_id='unused', gate_variable_id='speed',
|
||||
application_variable_ids=('s1', 's2'), nominal_width_m=5,
|
||||
start=start, end=start + timedelta(minutes=4),
|
||||
)
|
||||
state = MaterialConsumptionIntegrator(MaterialIntegratorConfig(0.3, 60)).process_many(samples)
|
||||
assert state.cumulative_consumption_kg == 100
|
||||
assert state.integrated_running_seconds == 120
|
||||
assert client.post_json.call_args.args[1]['variables'] == [
|
||||
{'uuid': 's1'}, {'uuid': 's2'}, {'uuid': 'speed'},
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('sources,width,speed', [
|
||||
([], 5, 2), ([float('nan')], 5, 2), ([1], 0, 2), ([1], 5, float('inf')),
|
||||
])
|
||||
def test_invalid_area_inputs(sources, width, speed):
|
||||
with pytest.raises(ValueError):
|
||||
area_application_rate_kg_per_hour(sources, width, speed)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('bad', [None, True, float('nan')])
|
||||
def test_missing_or_invalid_spreader_fails(bad):
|
||||
client = Mock()
|
||||
now = timestamp('2026-08-25T05:27:18Z')
|
||||
client.post_json.return_value.body = {'data': {
|
||||
'columns': ['time', 's1', 's2', 'speed'],
|
||||
'records': [[now.isoformat(), 3000, bad, 2]],
|
||||
}}
|
||||
with pytest.raises(ValueError):
|
||||
EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id='machine', rate_variable_id='unused', gate_variable_id='speed',
|
||||
application_variable_ids=('s1', 's2'), nominal_width_m=5, start=now, end=now,
|
||||
)
|
||||
|
||||
|
||||
def width_provider():
|
||||
erp = Mock()
|
||||
erp.get_current_workplace_status.return_value = SimpleNamespace(
|
||||
workplace='Bento 1', production_order='12026000814', article_number='180305',
|
||||
article_description='Bfix NSP 5300, 5,00 x 40 m',
|
||||
)
|
||||
return ErpNominalWidthProvider(
|
||||
erp, workplace='Bento 1', format_template='Bento 1-{production_order}',
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('configured,returned', [
|
||||
('Bento 1', 'BENTO 1'),
|
||||
('Bento 1', ' \tBENTO 1\n'),
|
||||
(' \tBento 1\n', 'BENTO 1'),
|
||||
('K7', ' k7 '),
|
||||
])
|
||||
def test_width_context_normalizes_workplace(configured, returned):
|
||||
provider = width_provider()
|
||||
provider.workplace = configured
|
||||
provider.gateway.get_current_workplace_status.return_value.workplace = returned
|
||||
assert provider('Bento 1-12026000814') == 5.0
|
||||
|
||||
|
||||
@pytest.mark.parametrize('production_order', [
|
||||
'Bento 1-12026000815',
|
||||
'BENTO 1-12026000814',
|
||||
' Bento 1-12026000814 ',
|
||||
])
|
||||
def test_width_context_keeps_production_order_strict(production_order):
|
||||
provider = width_provider()
|
||||
provider.gateway.get_current_workplace_status.return_value.workplace = 'BENTO 1'
|
||||
with pytest.raises(ValueError, match='ERP context does not match the production order'):
|
||||
provider(production_order)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [
|
||||
('production_order', '12026000815'), ('workplace', 'K7'),
|
||||
('article_description', None), ('article_description', '180305'),
|
||||
])
|
||||
def test_width_context_fails_closed(field, value):
|
||||
provider = width_provider()
|
||||
setattr(provider.gateway.get_current_workplace_status.return_value, field, value)
|
||||
with pytest.raises(ValueError):
|
||||
provider('Bento 1-12026000814')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('inter_run_gap_seconds', [5, 3761])
|
||||
def test_bento_rpm_bootstrap_and_persistent_resume(tmp_path, inter_run_gap_seconds):
|
||||
"""Synthetic aggregate fixture, not recorded ENLYZE samples.
|
||||
|
||||
Uses realistic rpm values and synthetic active time within actual run boundaries.
|
||||
"""
|
||||
config = load_material_calculation('config/bento1-material-consumption.yaml')
|
||||
order = 'Bento 1-12026000814'
|
||||
runs = [EnlyzeProductionRun(
|
||||
uuid, config.machine_ref, 'external-product', order, timestamp(start), timestamp(end),
|
||||
) for uuid, start, end in [
|
||||
('b675818c-4636-4976-be6c-3e0e24da9e8a',
|
||||
'2026-08-25T05:27:18Z', '2026-08-26T04:40:58Z'),
|
||||
('25d67424-8346-42d4-b844-75b4097382bd',
|
||||
'2026-08-26T05:43:39Z', '2026-08-26T05:54:05Z'),
|
||||
]]
|
||||
runs[1] = replace(
|
||||
runs[1], start=runs[0].end + timedelta(seconds=inter_run_gap_seconds),
|
||||
end=runs[0].end + timedelta(seconds=inter_run_gap_seconds + 626),
|
||||
)
|
||||
speed = 2.0
|
||||
client = Mock()
|
||||
|
||||
def response(path, body):
|
||||
start, end = timestamp(body['start']), timestamp(body['end'])
|
||||
run = next(run for run in runs if run.start <= start <= run.end)
|
||||
active_seconds = 1256 * 60 - 626 if run == runs[0] else 626
|
||||
stop = run.start + timedelta(seconds=active_seconds)
|
||||
times = {start, end}
|
||||
t = start
|
||||
while t < end:
|
||||
times.add(t)
|
||||
t += timedelta(seconds=10)
|
||||
if start <= stop <= end:
|
||||
times.add(stop)
|
||||
return SimpleNamespace(body={'data': {
|
||||
'columns': ['time', *config.rotational_speed_signal_refs, config.gate_signal_ref],
|
||||
'records': [[t.isoformat(), 3.739, 3.956, speed if t < stop else 0]
|
||||
for t in sorted(times)],
|
||||
}})
|
||||
|
||||
client.post_json.side_effect = response
|
||||
gateway = EnlyzeApiGateway(client)
|
||||
gateway.get_open_production_run = Mock(return_value=replace(runs[1], end=None))
|
||||
gateway.get_production_runs = Mock(return_value=runs)
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
|
||||
def service():
|
||||
return MaterialPollingService(
|
||||
gateway=gateway, state_store=store, machine_id=config.machine_ref,
|
||||
rate_variable_id=config.rate_signal_ref, gate_variable_id=config.gate_signal_ref,
|
||||
gate_threshold=config.gate_threshold,
|
||||
max_sample_gap_seconds=config.max_sample_gap_seconds,
|
||||
rotational_speed_variable_ids=config.rotational_speed_signal_refs,
|
||||
specific_discharge_kg_per_rev_m=config.specific_discharge_kg_per_rev_m,
|
||||
nominal_width_provider=width_provider(),
|
||||
)
|
||||
|
||||
partial = service().poll_once(now=runs[1].start + timedelta(seconds=300))
|
||||
result = service().poll_once(now=runs[1].end)
|
||||
assert result.state.cumulative_consumption_kg - partial.state.cumulative_consumption_kg == (
|
||||
pytest.approx((3.739 + 3.956) * 5 * 2.75 * 326 / 60)
|
||||
)
|
||||
assert result.state.integrated_running_seconds / 60 == pytest.approx(1256)
|
||||
assert result.state.cumulative_consumption_kg == pytest.approx(
|
||||
(3.739 + 3.956) * 5 * 2.75 * 1256,
|
||||
)
|
||||
assert service().poll_once(now=runs[1].end).state == result.state
|
||||
assert gateway.get_production_runs.call_count == 1
|
||||
assert [(timestamp(call.args[1]['start']), timestamp(call.args[1]['end']))
|
||||
for call in client.post_json.call_args_list[:2]] == [
|
||||
(runs[0].start, runs[0].end),
|
||||
(runs[1].start, runs[1].start + timedelta(seconds=300)),
|
||||
]
|
||||
|
||||
|
||||
def test_k7_config_defaults_remain_direct():
|
||||
config = load_material_calculation('config/k7-material-consumption.yaml')
|
||||
assert config.source_mode == 'direct_mass_rate'
|
||||
assert config.rotational_speed_signal_refs == ()
|
||||
assert config.application_signal_refs == ()
|
||||
assert config.gate_threshold == 0.5
|
||||
assert config.rate_signal_ref == 'c9d06af5-f6d6-4ede-b6c4-5a98bac77129'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [
|
||||
('source_mode', 'unknown'), ('rotational_speed_signal_refs', []),
|
||||
('rotational_speed_signal_refs', ['same', 'same']), ('erp_workplace', ''),
|
||||
('production_order_format', '{wrong}'), ('rate_signal_ref', 'direct'),
|
||||
])
|
||||
def test_invalid_rotational_configuration(tmp_path, field, value):
|
||||
with open('config/bento1-material-consumption.yaml') as stream:
|
||||
document = yaml.safe_load(stream)
|
||||
document['calculations'][0][field] = value
|
||||
path = tmp_path / 'config.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_rotational_context_failure_does_not_touch_persistent_state(tmp_path):
|
||||
config = load_material_calculation('config/bento1-material-consumption.yaml')
|
||||
gateway = Mock()
|
||||
now = timestamp('2026-08-25T05:27:18Z')
|
||||
gateway.get_open_production_run.return_value = EnlyzeProductionRun(
|
||||
'run', config.machine_ref, None, 'Bento 1-12026000814', now, None,
|
||||
)
|
||||
provider = width_provider()
|
||||
provider.gateway.get_current_workplace_status.return_value = None
|
||||
store = Mock()
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway, state_store=store, machine_id=config.machine_ref,
|
||||
rate_variable_id=config.rate_signal_ref, gate_variable_id=config.gate_signal_ref,
|
||||
gate_threshold=config.gate_threshold, max_sample_gap_seconds=20,
|
||||
rotational_speed_variable_ids=config.rotational_speed_signal_refs,
|
||||
specific_discharge_kg_per_rev_m=config.specific_discharge_kg_per_rev_m,
|
||||
nominal_width_provider=provider,
|
||||
)
|
||||
with pytest.raises(ValueError, match='context'):
|
||||
service.poll_once(now=now)
|
||||
gateway.get_material_samples.assert_not_called()
|
||||
store.save.assert_not_called()
|
||||
|
||||
|
||||
def test_area_gateway_pagination():
|
||||
client = Mock()
|
||||
now = timestamp('2026-08-25T05:27:18Z')
|
||||
client.post_json.side_effect = [SimpleNamespace(body={
|
||||
'metadata': {'next_cursor': cursor},
|
||||
'data': {'columns': ['time', 's1', 's2', 'speed'],
|
||||
'records': [[(now + timedelta(seconds=i * 10)).isoformat(), 3000, 2000, 2]]},
|
||||
}) for i, cursor in enumerate(['next', None])]
|
||||
samples = EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id='machine', rate_variable_id='unused', gate_variable_id='speed',
|
||||
application_variable_ids=('s1', 's2'), nominal_width_m=5,
|
||||
start=now, end=now + timedelta(seconds=10),
|
||||
)
|
||||
assert [sample.material_rate_kg_per_hour for sample in samples] == [3000, 3000]
|
||||
assert client.post_json.call_args.args[1]['cursor'] == 'next'
|
||||
@@ -0,0 +1,143 @@
|
||||
from dataclasses import replace
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.calibrations import load_calibrations, resolve_calibration
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.calculations.material_application import rotational_application_g_m2
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialConsumptionIntegrator,
|
||||
MaterialIntegratorConfig,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
|
||||
CALIBRATIONS = Path('config/material-calibrations.yaml')
|
||||
BENTO = Path('config/bento1-material-consumption.yaml')
|
||||
ID = 'bento1-spreader-1-2'
|
||||
|
||||
|
||||
def resolve(entries, identifier=ID):
|
||||
return resolve_calibration(entries, identifier, expected_type='rotational_discharge',
|
||||
expected_unit='kg_per_rev_m')
|
||||
|
||||
|
||||
def test_load_and_resolve():
|
||||
calibration = resolve(load_calibrations(CALIBRATIONS))
|
||||
assert calibration.value == 2.75
|
||||
assert calibration.calibrated_at == date(2026, 9, 8)
|
||||
assert calibration.method == 'gravimetric_tray'
|
||||
assert calibration.reference == {
|
||||
'measured_application_g_m2': 4068, 'line_speed_m_min': 2.3,
|
||||
'signal_values': {'left': 1.65, 'right': 1.75},
|
||||
}
|
||||
assert load_material_calculation(BENTO).specific_discharge_kg_per_rev_m == 2.75
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field', ['type', 'value', 'unit', 'calibrated_at', 'method',
|
||||
'description', 'reference'])
|
||||
def test_missing_metadata(tmp_path, field):
|
||||
document = yaml.safe_load(CALIBRATIONS.read_text())
|
||||
del document['calibrations'][ID][field]
|
||||
path = tmp_path / 'material-calibrations.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError, match='required fields'):
|
||||
load_calibrations(path)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [
|
||||
('type', ''), ('unit', None), ('method', 12), ('description', ' '),
|
||||
('calibrated_at', '2026-02-30'), ('calibrated_at', 123),
|
||||
('reference', []), ('reference', {}), ('value', True), ('value', float('nan')),
|
||||
])
|
||||
def test_invalid_metadata(tmp_path, field, value):
|
||||
document = yaml.safe_load(CALIBRATIONS.read_text())
|
||||
document['calibrations'][ID][field] = value
|
||||
path = tmp_path / 'material-calibrations.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError, match=field):
|
||||
load_calibrations(path)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value,error', [
|
||||
('type', 'other', 'incompatible type'), ('unit', 'kg_per_rev', 'incompatible unit'),
|
||||
('value', 0, 'greater than zero'),
|
||||
])
|
||||
def test_incompatible_at_config_load(tmp_path, field, value, error):
|
||||
document = yaml.safe_load(CALIBRATIONS.read_text())
|
||||
document['calibrations'][ID][field] = value
|
||||
(tmp_path / CALIBRATIONS.name).write_text(yaml.safe_dump(document))
|
||||
path = tmp_path / BENTO.name
|
||||
path.write_text(BENTO.read_text())
|
||||
with pytest.raises(CalculationConfigError, match=error):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_missing_id_and_file(tmp_path):
|
||||
with pytest.raises(CalculationConfigError, match='Missing calibration ID'):
|
||||
resolve(load_calibrations(CALIBRATIONS), 'unknown')
|
||||
path = tmp_path / BENTO.name
|
||||
path.write_text(BENTO.read_text())
|
||||
with pytest.raises(CalculationConfigError, match='Cannot read calibration'):
|
||||
load_material_calculation(path)
|
||||
(tmp_path / CALIBRATIONS.name).write_text(CALIBRATIONS.read_text())
|
||||
path.write_text(BENTO.read_text().replace(ID, 'unknown'))
|
||||
with pytest.raises(CalculationConfigError, match='Missing calibration ID'):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_exact_direct_value_equivalence(tmp_path):
|
||||
referenced = load_material_calculation(BENTO)
|
||||
document = yaml.safe_load(BENTO.read_text())
|
||||
entry = document['calculations'][0]
|
||||
del entry['calibration_ref']
|
||||
entry['specific_discharge_kg_per_rev_m'] = 2.75
|
||||
path = tmp_path / 'direct.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
direct = load_material_calculation(path)
|
||||
assert replace(referenced, calibration_ref=None) == direct
|
||||
start = datetime(2026, 9, 8, tzinfo=UTC)
|
||||
outputs = []
|
||||
for config in (referenced, direct):
|
||||
factor = config.specific_discharge_kg_per_rev_m
|
||||
application = rotational_application_g_m2([1.65, 1.75], factor, 2.3, 0.3)
|
||||
client = Mock()
|
||||
client.post_json.return_value.body = {'data': {
|
||||
'columns': ['time', 'left', 'right', 'speed'],
|
||||
'records': [[(start + timedelta(seconds=t)).isoformat(), 1.65, 1.75, speed]
|
||||
for t, speed in [(0, 2.3), (10, 0), (20, 2.3), (50, 2.3), (60, 2.3)]],
|
||||
}}
|
||||
samples = EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id='m', rate_variable_id='unused', gate_variable_id='speed',
|
||||
rotational_speed_variable_ids=('left', 'right'), nominal_width_m=5,
|
||||
specific_discharge_kg_per_rev_m=factor, start=start,
|
||||
end=start + timedelta(seconds=60),
|
||||
)
|
||||
integrator = MaterialConsumptionIntegrator(MaterialIntegratorConfig(0.3, 20))
|
||||
state = integrator.process_many(samples)
|
||||
assert samples[0].material_rate_kg_per_hour == 2805.0
|
||||
assert state.integrated_running_seconds == 20
|
||||
outputs.append((application, samples, state))
|
||||
assert outputs[0] == outputs[1]
|
||||
assert outputs[0][0] == pytest.approx(4065.217391304348)
|
||||
|
||||
|
||||
def test_update_only_central_file(tmp_path):
|
||||
path = tmp_path / BENTO.name
|
||||
path.write_text(BENTO.read_text())
|
||||
(tmp_path / CALIBRATIONS.name).write_text(CALIBRATIONS.read_text().replace('2.75', '2.8'))
|
||||
assert load_material_calculation(path).specific_discharge_kg_per_rev_m == 2.8
|
||||
|
||||
|
||||
def test_k7_without_calibration_file(tmp_path):
|
||||
original = Path('config/k7-material-consumption.yaml')
|
||||
path = tmp_path / original.name
|
||||
path.write_text(original.read_text())
|
||||
assert load_material_calculation(path) == load_material_calculation(original)
|
||||
assert load_material_calculation(path).calibration_ref is None
|
||||
@@ -0,0 +1,158 @@
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.calculations.channel_config import load_channel_material_calculation
|
||||
from production_analytics.calculations.channel_material import (
|
||||
MATERIAL_AMBIGUOUS,
|
||||
MATERIAL_UNAVAILABLE,
|
||||
DosingChannelSample,
|
||||
channel_rates,
|
||||
validate_active_percentage_sum,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway, EnlyzeProductionRun
|
||||
from production_analytics.service.channel_material import (
|
||||
ChannelDefinition,
|
||||
ChannelMaterialPollingService,
|
||||
)
|
||||
from production_analytics.service.channel_material_state_store import JsonChannelMaterialStateStore
|
||||
|
||||
NOW = datetime(2026, 9, 1, tzinfo=UTC)
|
||||
E1_CONFIG = (
|
||||
Path(__file__).resolve().parents[1] / "config/e1-channel-material-consumption.example.yaml"
|
||||
)
|
||||
|
||||
|
||||
def sample(*, on=True, percent=100, material=None, timestamp=NOW):
|
||||
return DosingChannelSample(timestamp, 120, on, percent, 10, material)
|
||||
|
||||
|
||||
def test_off_channel_with_nonzero_displayed_percentage_contributes_zero():
|
||||
assert channel_rates((sample(on=False, percent=40),))[0].sample.material_rate_kg_per_hour == 0
|
||||
|
||||
|
||||
def test_e1_config_dry_run_resolves_every_signal_through_gateway_without_sampling():
|
||||
config = load_channel_material_calculation(E1_CONFIG)
|
||||
origins = {
|
||||
reference
|
||||
for channel in config.channels
|
||||
for reference in (
|
||||
channel.total_rate_signal_ref,
|
||||
channel.status_signal_ref,
|
||||
channel.percentage_signal_ref,
|
||||
channel.screw_speed_signal_ref,
|
||||
)
|
||||
}
|
||||
client = Mock()
|
||||
client.get.return_value.body = {
|
||||
"data": [
|
||||
{
|
||||
"uuid": f"variable-{index}",
|
||||
"details": {"origin_identifier": {"code": origin}},
|
||||
}
|
||||
for index, origin in enumerate(sorted(origins))
|
||||
]
|
||||
}
|
||||
|
||||
resolved = EnlyzeApiGateway(client).resolve_dosing_channel_signal_refs(
|
||||
machine_id=config.machine_ref,
|
||||
channels=config.channels,
|
||||
)
|
||||
|
||||
assert config.machine_ref == "8302e3d1-b1e5-42f1-8540-615eb6c73e08"
|
||||
assert len(config.channels) == 14
|
||||
assert len(origins) == 44
|
||||
assert len(resolved) == 14
|
||||
assert all(len(refs) == 5 and refs[-1] is None for refs in resolved.values())
|
||||
client.get.assert_called_once_with("/v2/variables", {"machine": config.machine_ref})
|
||||
client.post_json.assert_not_called()
|
||||
|
||||
|
||||
def test_active_percentages_sum_to_100_and_deviation_is_reported_not_normalized():
|
||||
active = (sample(percent=40), sample(percent=60))
|
||||
assert validate_active_percentage_sum(active, tolerance=0.1)
|
||||
assert not validate_active_percentage_sum(
|
||||
(sample(percent=40), sample(percent=50)), tolerance=0.1
|
||||
)
|
||||
assert [x.sample.material_rate_kg_per_hour for x in channel_rates(active)] == [48, 72]
|
||||
|
||||
|
||||
def test_null_and_duplicate_material_assignments_are_not_inferred():
|
||||
assert (
|
||||
channel_rates((sample(material=None),))[0].material_mapping_status == MATERIAL_UNAVAILABLE
|
||||
)
|
||||
rates = channel_rates((sample(material="123"), sample(material="123")))
|
||||
assert all(rate.material_mapping_status == MATERIAL_AMBIGUOUS for rate in rates)
|
||||
assert all(rate.material_number is None and rate.material_name is None for rate in rates)
|
||||
|
||||
|
||||
def test_run_change_and_json_restart_preserve_channel_total(tmp_path):
|
||||
channels = (ChannelDefinition("Ex1", "C1", "rate", "on", "pct", "rpm"),)
|
||||
store, gateway = JsonChannelMaterialStateStore(tmp_path), Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
gateway.get_open_production_run.return_value = EnlyzeProductionRun(
|
||||
"r1", "m", None, "order", NOW, None
|
||||
)
|
||||
key = "C1\0Ex1"
|
||||
gateway.get_dosing_channel_samples.return_value = {
|
||||
key: [sample(timestamp=NOW), sample(timestamp=NOW + timedelta(seconds=10))]
|
||||
}
|
||||
first = ChannelMaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=store,
|
||||
machine_id="m",
|
||||
channels=channels,
|
||||
max_sample_gap_seconds=20,
|
||||
)
|
||||
assert first.poll_once(now=NOW + timedelta(seconds=10))[
|
||||
0
|
||||
].state.cumulative_consumption_kg == pytest.approx(10 / 30)
|
||||
gateway.get_open_production_run.return_value = EnlyzeProductionRun(
|
||||
"r2", "m", None, "order", NOW + timedelta(seconds=20), None
|
||||
)
|
||||
gateway.get_dosing_channel_samples.return_value = {
|
||||
key: [
|
||||
sample(timestamp=NOW + timedelta(seconds=20)),
|
||||
sample(timestamp=NOW + timedelta(seconds=30)),
|
||||
]
|
||||
}
|
||||
restarted = ChannelMaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=store,
|
||||
machine_id="m",
|
||||
channels=channels,
|
||||
max_sample_gap_seconds=20,
|
||||
)
|
||||
assert restarted.poll_once(now=NOW + timedelta(seconds=30))[
|
||||
0
|
||||
].state.cumulative_consumption_kg == pytest.approx(20 / 30)
|
||||
|
||||
|
||||
def test_bootstrap_replays_closed_runs_without_integrating_the_inter_run_gap(tmp_path):
|
||||
channels = (ChannelDefinition("Ex1", "C1", "rate", "on", "pct", "rpm"),)
|
||||
closed = EnlyzeProductionRun("r1", "m", None, "order", NOW, NOW + timedelta(seconds=10))
|
||||
current = EnlyzeProductionRun("r2", "m", None, "order", NOW + timedelta(seconds=20), None)
|
||||
gateway = Mock()
|
||||
gateway.get_open_production_run.return_value = current
|
||||
gateway.get_production_runs.return_value = [closed, current]
|
||||
key = "C1\0Ex1"
|
||||
gateway.get_dosing_channel_samples.side_effect = [
|
||||
{key: [sample(timestamp=NOW), sample(timestamp=NOW + timedelta(seconds=10))]},
|
||||
{
|
||||
key: [
|
||||
sample(timestamp=NOW + timedelta(seconds=20)),
|
||||
sample(timestamp=NOW + timedelta(seconds=30)),
|
||||
]
|
||||
},
|
||||
]
|
||||
result = ChannelMaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=JsonChannelMaterialStateStore(tmp_path),
|
||||
machine_id="m",
|
||||
channels=channels,
|
||||
max_sample_gap_seconds=20,
|
||||
).poll_once(now=NOW + timedelta(seconds=30))
|
||||
assert result is not None
|
||||
assert result[0].state.cumulative_consumption_kg == pytest.approx(20 / 30)
|
||||
@@ -0,0 +1,141 @@
|
||||
import io
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from production_analytics.calculations.channel_config import load_channel_material_calculation
|
||||
from production_analytics.calculations.material_consumption import MaterialIntegrationState
|
||||
from production_analytics.cli.__main__ import _parser, main
|
||||
from production_analytics.enlyze.gateway import EnlyzeProductionRun
|
||||
from production_analytics.service.channel_material import ChannelPollResult
|
||||
from production_analytics.service.channel_material_runner import ChannelMaterialPollingRunner
|
||||
from production_analytics.service.channel_material_runtime import build_channel_material_runner
|
||||
|
||||
EXAMPLE = (
|
||||
Path(__file__).resolve().parents[1] / "config/e1-channel-material-consumption.example.yaml"
|
||||
)
|
||||
NOW = datetime(2026, 9, 1, tzinfo=UTC)
|
||||
|
||||
|
||||
def test_cli_channel_material_arguments():
|
||||
args = _parser().parse_args(
|
||||
[
|
||||
"run",
|
||||
"channel-material-poll",
|
||||
"--config",
|
||||
str(EXAMPLE),
|
||||
"--poll-interval-seconds",
|
||||
"15",
|
||||
"--state-directory",
|
||||
"state",
|
||||
"--secrets-file",
|
||||
"secrets.env",
|
||||
"--once",
|
||||
]
|
||||
)
|
||||
assert args.command == "channel-material-poll"
|
||||
assert args.config == EXAMPLE
|
||||
assert args.poll_interval_seconds == 15
|
||||
assert args.state_directory == Path("state")
|
||||
assert args.secrets_file == Path("secrets.env")
|
||||
assert args.once
|
||||
|
||||
|
||||
def test_runtime_constructs_channel_service(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("ENLYZE_BASE_URL", "https://example.invalid/api/")
|
||||
for key, value in {
|
||||
"HOST": "localhost",
|
||||
"PORT": "5432",
|
||||
"DB": "analytics",
|
||||
"USER": "user",
|
||||
"PASSWORD": "environment-password",
|
||||
}.items():
|
||||
monkeypatch.setenv(f"POSTGRES_{key}", value)
|
||||
secrets = tmp_path / "secrets.env"
|
||||
secrets.write_text("ENLYZE_API_KEY=secret-from-file\nPOSTGRES_PASSWORD=file-password\n")
|
||||
calculation = load_channel_material_calculation(EXAMPLE)
|
||||
with patch(
|
||||
"production_analytics.service.channel_material_runtime.ChannelMaterialPollingService"
|
||||
) as service:
|
||||
runner = build_channel_material_runner(
|
||||
calculation,
|
||||
poll_interval_seconds=7,
|
||||
state_directory=tmp_path / "state",
|
||||
secrets_file=secrets,
|
||||
)
|
||||
kwargs = service.call_args.kwargs
|
||||
assert kwargs["machine_id"] == calculation.machine_ref
|
||||
assert kwargs["channels"] == calculation.channels
|
||||
assert kwargs["max_sample_gap_seconds"] == calculation.max_sample_gap_seconds
|
||||
assert kwargs["percentage_tolerance"] == calculation.percentage_tolerance
|
||||
assert kwargs["material_names"] == calculation.material_names
|
||||
assert kwargs["gateway"]._client._settings.api_key == "secret-from-file"
|
||||
assert kwargs["state_store"].directory == tmp_path / "state"
|
||||
assert runner.service is service.return_value
|
||||
assert runner.interval == 7
|
||||
assert runner.snapshot_writer.settings.password == "file-password"
|
||||
|
||||
|
||||
def test_once_executes_one_cycle_and_exits(capsys):
|
||||
with patch(
|
||||
"production_analytics.service.channel_material_runtime.build_channel_material_runner"
|
||||
) as build:
|
||||
assert main(["run", "channel-material-poll", "--config", str(EXAMPLE), "--once"]) == 0
|
||||
build.return_value.run_once.assert_called_once_with()
|
||||
build.return_value.run.assert_not_called()
|
||||
assert capsys.readouterr().err == ""
|
||||
|
||||
|
||||
def test_no_snapshots_are_written_without_eligible_production_run():
|
||||
writer = Mock()
|
||||
output = io.StringIO()
|
||||
runner = ChannelMaterialPollingRunner(
|
||||
Mock(poll_once=Mock(return_value=None)),
|
||||
machine_id="machine",
|
||||
calculation_id="calculation",
|
||||
snapshot_writer=writer,
|
||||
poll_interval_seconds=10,
|
||||
clock=lambda: NOW,
|
||||
stdout=output,
|
||||
)
|
||||
assert not runner.run_once()
|
||||
writer.write.assert_not_called()
|
||||
assert "snapshots=0" in output.getvalue()
|
||||
|
||||
|
||||
def test_one_cycle_writes_each_channel_snapshot():
|
||||
run = EnlyzeProductionRun("run", "machine", None, "order", NOW, None)
|
||||
first = ChannelPollResult(
|
||||
run, "Ex1", "C1", MaterialIntegrationState(1.5, 10), "42", "Material", "MAPPED", True
|
||||
)
|
||||
second = ChannelPollResult(
|
||||
run, "Ex1", "C2", MaterialIntegrationState(2.5, 10), None, None, "AMBIGUOUS", False
|
||||
)
|
||||
writer = Mock()
|
||||
output = io.StringIO()
|
||||
runner = ChannelMaterialPollingRunner(
|
||||
Mock(poll_once=Mock(return_value=(first, second))),
|
||||
machine_id="machine",
|
||||
calculation_id="calculation",
|
||||
snapshot_writer=writer,
|
||||
poll_interval_seconds=10,
|
||||
clock=lambda: NOW,
|
||||
stdout=output,
|
||||
)
|
||||
assert runner.run_once()
|
||||
assert writer.write.call_count == 2
|
||||
assert writer.write.call_args_list[0].kwargs == {
|
||||
"timestamp": NOW,
|
||||
"calculation_id": "calculation",
|
||||
"machine_id": "machine",
|
||||
"extruder": "Ex1",
|
||||
"channel": "C1",
|
||||
"production_order": "order",
|
||||
"run_id": "run",
|
||||
"cumulative_consumption_kg": 1.5,
|
||||
"material_number": "42",
|
||||
"material_name": "Material",
|
||||
"material_mapping_status": "MAPPED",
|
||||
"percentage_sum_valid": True,
|
||||
}
|
||||
assert "channel snapshots=2 written" in output.getvalue()
|
||||
+16
-1
@@ -3,7 +3,7 @@ import json
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from contextlib import redirect_stdout
|
||||
from contextlib import redirect_stderr, redirect_stdout
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -12,6 +12,21 @@ from production_analytics.enlyze.exploration import ExplorationResponse
|
||||
|
||||
|
||||
class CliTests(unittest.TestCase):
|
||||
def test_power_meter_config_error_returns_controlled_exit_code(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary_directory:
|
||||
missing_config = Path(temporary_directory) / "missing.yaml"
|
||||
error_output = io.StringIO()
|
||||
with redirect_stderr(error_output):
|
||||
result = main([
|
||||
"run", "power-meter", "--config", str(missing_config), "--meter", "B2",
|
||||
"--start", "2026-10-01T10:00:00Z", "--end", "2026-10-01T10:10:00Z",
|
||||
])
|
||||
|
||||
self.assertEqual(result, 2)
|
||||
self.assertIn(
|
||||
"Config/startup error: Cannot read power-meter configuration", error_output.getvalue()
|
||||
)
|
||||
|
||||
@patch.dict(os.environ, {"ENLYZE_BASE_URL": "https://enlyze.example"}, clear=True)
|
||||
@patch("production_analytics.cli.__main__.ExplorationClient.get")
|
||||
def test_raw_prints_and_saves_sanitized_response(self, get_mock: object) -> None:
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
import pytest
|
||||
|
||||
from production_analytics.context import (
|
||||
build_enlyze_production_order,
|
||||
extract_nominal_width_m,
|
||||
)
|
||||
|
||||
K7_ORDER_TEMPLATE = "K 7-{production_order}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("order, expected", [
|
||||
("12026000815", "K 7-12026000815"),
|
||||
(" \t12026000815\n", "K 7-12026000815"),
|
||||
("00123", "K 7-00123"),
|
||||
])
|
||||
def test_k7_mapping(order, expected):
|
||||
assert build_enlyze_production_order(order, K7_ORDER_TEMPLATE) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("order", [
|
||||
"", " \t", "K 7-12026000815", "K 7-12025000074-K 7-12025000075",
|
||||
"12025000074-12025000075", "prefix12026000815", "12026000815suffix",
|
||||
"12026 000815", "123", "-123",
|
||||
])
|
||||
def test_mapping_rejects_non_erp_identifiers(order):
|
||||
with pytest.raises(ValueError, match="ERP production order"):
|
||||
build_enlyze_production_order(order, K7_ORDER_TEMPLATE)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("template, expected", [
|
||||
("EXAMPLE/{production_order}/finished", "EXAMPLE/00123/finished"),
|
||||
("{production_order}", "00123"),
|
||||
("{production_order}-example", "00123-example"),
|
||||
])
|
||||
def test_mapping_uses_explicit_template(template, expected):
|
||||
assert build_enlyze_production_order(" 00123 ", template) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("template", [
|
||||
"", "fixed-order", "{order}", "{production_order}-{production_order}",
|
||||
"{production_order:>20}", "{production_order!r}", "{production_order}{unknown}",
|
||||
"{{production_order}}",
|
||||
])
|
||||
def test_mapping_rejects_invalid_template(template):
|
||||
with pytest.raises(ValueError, match="Format template"):
|
||||
build_enlyze_production_order("12026000815", template)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("description, expected", [
|
||||
("Stex R 1501 C (PR) 5,80 x 50 m", 5.8),
|
||||
("Stex R 401, 6,00 x 90 m", 6.0),
|
||||
("Stex R 1501 5.80 x 50 m", 5.8),
|
||||
("5,80x50 m", 5.8),
|
||||
("5,80x50m", 5.8),
|
||||
("Article 212520 grade 1501 5,80\t x\t50 m", 5.8),
|
||||
("5,80 x 50 m coated (batch 42)", 5.8),
|
||||
("(5,80 x 50 m), coated", 5.8),
|
||||
("6 x 90 m", 6.0),
|
||||
])
|
||||
def test_width(description, expected):
|
||||
assert extract_nominal_width_m(description) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("description", [
|
||||
None, "", "Stex R 1501", "5,80 m", "5,80 x 50", "5,80 x 50 cm",
|
||||
"5,80 x 50 mm", "5,80 x 50 m²", "5,80 x 50 m2", "5,80 x 50 m/min",
|
||||
"5,80 x 50 metres", "5,80,2 x 50 m", "5.80.2 x 50 m",
|
||||
"5,80 x ? m", "5,80 x 50 m or 6,00 x 90 m", "2 x 5,80 x 50 m",
|
||||
"5,80 x 50 m x 2", "0 x 50 m", "-5,80 x 50 m", "+5,80 x 50 m",
|
||||
"nan x 50 m", "inf x 50 m", "Infinity x 50 m", "1e309 x 50 m",
|
||||
"9" * 400 + " x 50 m", "5,80 x 0 m", "5,80 x -50 m", "5,80 x nan m",
|
||||
"1/5,80 x 50 m", "abc5,80 x 50 m",
|
||||
])
|
||||
def test_width_fails_safely(description):
|
||||
assert extract_nominal_width_m(description) is None
|
||||
@@ -0,0 +1,140 @@
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from production_analytics.enlyze.gateway import EnlyzeDowntime
|
||||
from production_analytics.erp import CurrentWorkplaceStatus
|
||||
from production_analytics.service.downtime import (
|
||||
DowntimeCategory,
|
||||
DowntimeReconciliationService,
|
||||
ProductionOrderBoundary,
|
||||
)
|
||||
|
||||
NOW = datetime(2026, 9, 16, 12, tzinfo=UTC)
|
||||
|
||||
|
||||
def source(*, identifier="d", end=NOW, category=None):
|
||||
return EnlyzeDowntime(
|
||||
identifier,
|
||||
"machine",
|
||||
"THRESHOLD",
|
||||
datetime(2026, 9, 16, 10, tzinfo=UTC),
|
||||
end,
|
||||
None,
|
||||
"reason" if category else None,
|
||||
"reason" if category else None,
|
||||
None,
|
||||
None,
|
||||
category,
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def status(order="1", *, remaining=3, good=7, target=10, feedback=NOW):
|
||||
return CurrentWorkplaceStatus(
|
||||
"WP", order, None, None, feedback, target, good, remaining, None, None
|
||||
)
|
||||
|
||||
|
||||
class Gateway:
|
||||
def __init__(self, events):
|
||||
self.events = events
|
||||
|
||||
def get_downtimes(self, machine_id, *, start=None):
|
||||
return self.events
|
||||
|
||||
|
||||
class Erp:
|
||||
def __init__(self, current):
|
||||
self.current = current
|
||||
|
||||
def get_current_workplace_status(self, workplace):
|
||||
return self.current
|
||||
|
||||
|
||||
class Repo:
|
||||
def __init__(self):
|
||||
self.boundary = None
|
||||
self.events = {}
|
||||
self.closures = []
|
||||
|
||||
def current_boundary(self, machine_id):
|
||||
return self.boundary
|
||||
|
||||
def save_boundary(self, boundary):
|
||||
self.boundary = boundary
|
||||
|
||||
def close_order_attribution(self, machine_id, production_order, ended_at):
|
||||
self.closures.append((production_order, ended_at))
|
||||
for event in self.events.values():
|
||||
if event.production_order == production_order and event.attributed_end is None:
|
||||
event # database-specific clipping is covered by SQL contract
|
||||
|
||||
def upsert(self, event):
|
||||
self.events[event.external_id] = event
|
||||
|
||||
|
||||
def service(repo, events, current):
|
||||
return DowntimeReconciliationService(
|
||||
Gateway(events),
|
||||
Erp(current),
|
||||
repo,
|
||||
machine_id="machine",
|
||||
workplace="WP",
|
||||
production_order_format="FA-{production_order}",
|
||||
)
|
||||
|
||||
|
||||
def test_categories_open_and_reconciliation_upsert() -> None:
|
||||
repo = Repo()
|
||||
runner = service(
|
||||
repo,
|
||||
[
|
||||
source(identifier="p", category="PLANNED"),
|
||||
source(identifier="u", category="UNPLANNED"),
|
||||
source(identifier="x", end=None),
|
||||
],
|
||||
status(),
|
||||
)
|
||||
assert runner.reconcile_once(NOW) == 3
|
||||
assert {key: event.category for key, event in repo.events.items()} == {
|
||||
"p": DowntimeCategory.PLANNED,
|
||||
"u": DowntimeCategory.UNPLANNED,
|
||||
"x": DowntimeCategory.UNKNOWN,
|
||||
}
|
||||
assert repo.events["x"].source_end is None
|
||||
# Same UUID is overwritten, never duplicated; delayed classification is accepted.
|
||||
runner.gateway.events = [source(identifier="x", end=NOW, category="PLANNED")]
|
||||
runner.reconcile_once(NOW)
|
||||
assert len(repo.events) == 3
|
||||
assert repo.events["x"].category is DowntimeCategory.PLANNED
|
||||
assert repo.events["x"].source_end == NOW
|
||||
|
||||
|
||||
def test_completion_clips_attribution_and_new_order_closes_previous() -> None:
|
||||
repo = Repo()
|
||||
repo.boundary = ProductionOrderBoundary(
|
||||
"machine",
|
||||
"FA-1",
|
||||
datetime(2026, 9, 16, 9, tzinfo=UTC),
|
||||
None,
|
||||
)
|
||||
event = source(identifier="open", end=None)
|
||||
first = service(repo, [event], status(remaining=0, feedback=NOW))
|
||||
first.reconcile_once(NOW)
|
||||
assert repo.boundary is not None and repo.boundary.ended_at == NOW
|
||||
assert repo.events["open"].production_order == "FA-1"
|
||||
assert repo.events["open"].attributed_end == NOW
|
||||
repo.boundary = ProductionOrderBoundary(
|
||||
"machine",
|
||||
"FA-1",
|
||||
datetime(2026, 9, 16, 8, tzinfo=UTC),
|
||||
None,
|
||||
)
|
||||
service(repo, [], status(order="2", feedback=NOW)).reconcile_once(NOW)
|
||||
assert repo.closures == [("FA-1", NOW), ("FA-1", NOW)]
|
||||
assert repo.boundary.production_order == "FA-2"
|
||||
|
||||
|
||||
def test_unknown_is_not_unplanned_and_no_schedule_is_used() -> None:
|
||||
repo = Repo()
|
||||
service(repo, [source(end=datetime(2026, 9, 20, tzinfo=UTC))], status()).reconcile_once(NOW)
|
||||
assert repo.events["d"].category is DowntimeCategory.UNKNOWN
|
||||
@@ -9,6 +9,7 @@ from production_analytics.enlyze.exploration import (
|
||||
AuthenticationError,
|
||||
ExplorationClient,
|
||||
ExplorationSettings,
|
||||
HttpResponseError,
|
||||
NonJsonResponseError,
|
||||
load_secret_file,
|
||||
)
|
||||
@@ -78,6 +79,28 @@ class ExplorationClientTests(unittest.TestCase):
|
||||
with self.assertRaisesRegex(AuthenticationError, "HTTP 401"):
|
||||
ExplorationClient(self.settings).get("/observed/path")
|
||||
|
||||
@patch("production_analytics.enlyze.exploration.urlopen")
|
||||
def test_http_error_keeps_sanitized_bounded_body(self, urlopen_mock: object) -> None:
|
||||
urlopen_mock.side_effect = HTTPError(
|
||||
"https://example.invalid", 422, "Unprocessable", {},
|
||||
io.BytesIO(b'{"message":"bad variable","token":"must not echo"}'),
|
||||
) # type: ignore[attr-defined]
|
||||
|
||||
with self.assertRaises(HttpResponseError) as caught:
|
||||
ExplorationClient(self.settings).get("/observed/path")
|
||||
self.assertEqual(caught.exception.response_body["message"], "bad variable")
|
||||
self.assertEqual(caught.exception.response_body["token"], "<redacted-secret>")
|
||||
|
||||
@patch("production_analytics.enlyze.exploration.urlopen")
|
||||
def test_get_all_pages_follows_cursor(self, urlopen_mock: object) -> None:
|
||||
urlopen_mock.side_effect = [
|
||||
_Response(b'{"data":[1],"metadata":{"next_cursor":"next"}}'),
|
||||
_Response(b'{"data":[2],"metadata":{"next_cursor":null}}'),
|
||||
]
|
||||
|
||||
pages = ExplorationClient(self.settings).get_all_pages("/v2/variables", {"machine": "m"})
|
||||
self.assertEqual([page.body["data"] for page in pages], [[1], [2]])
|
||||
|
||||
|
||||
class SanitizationTests(unittest.TestCase):
|
||||
def test_sanitization_preserves_shape_and_redacts_sensitive_values(self) -> None:
|
||||
|
||||
@@ -1,9 +1,57 @@
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
|
||||
|
||||
def test_downtime_pagination_and_nullable_reason() -> None:
|
||||
client = Mock()
|
||||
base = {
|
||||
"machine": "machine-1",
|
||||
"type": "THRESHOLD",
|
||||
"comment": None,
|
||||
"start": "2026-09-03T04:00:00Z",
|
||||
"end": None,
|
||||
"reason": None,
|
||||
"updated": None,
|
||||
}
|
||||
client.get.side_effect = [
|
||||
Mock(body={"data": [{**base, "uuid": "d1"}], "metadata": {"next_cursor": "next"}}),
|
||||
Mock(
|
||||
body={
|
||||
"data": [
|
||||
{
|
||||
**base,
|
||||
"uuid": "d2",
|
||||
"end": "2026-09-03T05:00:00Z",
|
||||
"reason": {
|
||||
"uuid": "r",
|
||||
"name": "Break",
|
||||
"description": None,
|
||||
"group": "General",
|
||||
"category": "PLANNED",
|
||||
},
|
||||
}
|
||||
],
|
||||
"metadata": {"next_cursor": None},
|
||||
}
|
||||
),
|
||||
]
|
||||
result = EnlyzeApiGateway(client).get_downtimes(
|
||||
"machine-1",
|
||||
start=datetime(2026, 9, 3, tzinfo=UTC),
|
||||
)
|
||||
assert [item.uuid for item in result] == ["d1", "d2"]
|
||||
assert result[0].end is None and result[0].reason_category is None
|
||||
assert result[1].reason_category == "PLANNED"
|
||||
assert client.get.call_args_list[1].args == (
|
||||
"/v2/downtimes",
|
||||
{"machine": "machine-1", "start": "2026-09-03T00:00:00+00:00", "cursor": "next"},
|
||||
)
|
||||
|
||||
|
||||
def test_get_open_production_run_returns_current_run() -> None:
|
||||
client = Mock()
|
||||
client.get.return_value.body = {
|
||||
@@ -131,3 +179,322 @@ def test_get_material_samples_uses_column_names_not_fixed_positions() -> None:
|
||||
|
||||
assert samples[0].material_rate_kg_per_hour == 1028.5
|
||||
assert samples[0].gate_value == 2.1
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def timeseries():
|
||||
client = Mock()
|
||||
client.post_json.return_value.body = {
|
||||
"data": {
|
||||
"columns": ["time", "rate", "gate"],
|
||||
"records": [],
|
||||
}
|
||||
}
|
||||
args = dict(
|
||||
machine_id="m",
|
||||
rate_variable_id="rate",
|
||||
gate_variable_id="gate",
|
||||
start=datetime(2026, 9, 3, tzinfo=UTC),
|
||||
end=datetime(2026, 9, 4, tzinfo=UTC),
|
||||
)
|
||||
return EnlyzeApiGateway(client), client, args
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["start", "end"])
|
||||
def test_naive_window_rejected(timeseries, name) -> None:
|
||||
gateway, client, args = timeseries
|
||||
args[name] = args[name].replace(tzinfo=None)
|
||||
with pytest.raises(ValueError, match="timezone-aware"):
|
||||
gateway.get_material_samples(**args)
|
||||
client.post_json.assert_not_called()
|
||||
|
||||
|
||||
def test_reversed_window_rejected(timeseries) -> None:
|
||||
gateway, client, args = timeseries
|
||||
args["start"], args["end"] = args["end"], args["start"]
|
||||
with pytest.raises(ValueError, match="precede"):
|
||||
gateway.get_material_samples(**args)
|
||||
client.post_json.assert_not_called()
|
||||
|
||||
|
||||
def test_empty_timeseries(timeseries) -> None:
|
||||
gateway, client, args = timeseries
|
||||
assert gateway.get_material_samples(**args) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("column", ["time", "rate", "gate"])
|
||||
def test_missing_required_column(timeseries, column) -> None:
|
||||
gateway, client, args = timeseries
|
||||
client.post_json.return_value.body["data"]["columns"].remove(column)
|
||||
with pytest.raises(ValueError, match="required column"):
|
||||
gateway.get_material_samples(**args)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"record",
|
||||
[
|
||||
[],
|
||||
{},
|
||||
["bad", 1, 1],
|
||||
[None, 1, 1],
|
||||
["2026-09-03T00:00:00", 1, 1],
|
||||
["2026-09-03T00:00:00Z", None, 1],
|
||||
["2026-09-03T00:00:00Z", 1, float("inf")],
|
||||
["2026-09-03T00:00:00Z", True, 1],
|
||||
],
|
||||
)
|
||||
def test_malformed_records_fail_clearly(timeseries, record) -> None:
|
||||
gateway, client, args = timeseries
|
||||
client.post_json.return_value.body["data"]["records"] = [record]
|
||||
with pytest.raises(ValueError, match="malformed record 0"):
|
||||
gateway.get_material_samples(**args)
|
||||
|
||||
|
||||
def test_multiple_open_runs_rejected() -> None:
|
||||
client = Mock()
|
||||
run = dict(uuid="r", machine="m", production_order="o", start="2026-09-03T00:00:00Z", end=None)
|
||||
client.get.return_value.body = {"data": [run, {**run, "uuid": "r2"}]}
|
||||
with pytest.raises(ValueError, match="multiple open Production Runs"):
|
||||
EnlyzeApiGateway(client).get_open_production_run("m")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"item",
|
||||
[
|
||||
{},
|
||||
None,
|
||||
{"end": None},
|
||||
dict(uuid="r", machine="m", production_order="o", start="bad", end=None),
|
||||
],
|
||||
)
|
||||
def test_malformed_run_fails_clearly(item) -> None:
|
||||
client = Mock()
|
||||
client.get.return_value.body = {"data": [item]}
|
||||
with pytest.raises(ValueError, match="Invalid production-run response"):
|
||||
EnlyzeApiGateway(client).get_open_production_run("m")
|
||||
|
||||
|
||||
def test_timeseries_with_null_cursor(timeseries) -> None:
|
||||
gateway, client, args = timeseries
|
||||
client.post_json.return_value.body["metadata"] = {"next_cursor": None}
|
||||
client.post_json.return_value.body["data"]["records"] = [
|
||||
["2026-09-03T00:00:00Z", 10, 2],
|
||||
]
|
||||
samples = gateway.get_material_samples(**args)
|
||||
assert len(samples) == 1
|
||||
assert samples[0].material_rate_kg_per_hour == 10
|
||||
client.post_json.assert_called_once()
|
||||
|
||||
|
||||
def test_timeseries_pagination_resolves_columns_per_page(timeseries) -> None:
|
||||
gateway, client, args = timeseries
|
||||
client.post_json.side_effect = [
|
||||
Mock(
|
||||
body={
|
||||
"data": {
|
||||
"columns": ["time", "rate", "gate"],
|
||||
"records": [["2026-09-03T00:00:00Z", 10, 2]],
|
||||
},
|
||||
"metadata": {"next_cursor": "continuation"},
|
||||
}
|
||||
),
|
||||
Mock(
|
||||
body={
|
||||
"data": {
|
||||
"columns": ["gate", "time", "rate"],
|
||||
"records": [[3, "2026-09-03T01:00:00Z", 20]],
|
||||
},
|
||||
"metadata": {"next_cursor": None},
|
||||
}
|
||||
),
|
||||
]
|
||||
samples = gateway.get_material_samples(**args)
|
||||
assert [(s.timestamp, s.material_rate_kg_per_hour, s.gate_value) for s in samples] == [
|
||||
(datetime(2026, 9, 3, tzinfo=UTC), 10, 2),
|
||||
(datetime(2026, 9, 3, 1, tzinfo=UTC), 20, 3),
|
||||
]
|
||||
assert client.post_json.call_count == 2
|
||||
first, second = client.post_json.call_args_list
|
||||
assert first.args == (
|
||||
"/v2/timeseries",
|
||||
{
|
||||
"machine": "m",
|
||||
"start": args["start"].isoformat(),
|
||||
"end": args["end"].isoformat(),
|
||||
"variables": [{"uuid": "rate"}, {"uuid": "gate"}],
|
||||
},
|
||||
)
|
||||
assert second.args == ("/v2/timeseries", {**first.args[1], "cursor": "continuation"})
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"metadata",
|
||||
[
|
||||
None,
|
||||
[],
|
||||
"bad",
|
||||
{},
|
||||
{"next_cursor": ""},
|
||||
{"next_cursor": 1},
|
||||
{"next_cursor": False},
|
||||
{"next_cursor": []},
|
||||
{"next_cursor": {}},
|
||||
],
|
||||
)
|
||||
def test_invalid_pagination_metadata(timeseries, metadata) -> None:
|
||||
gateway, client, args = timeseries
|
||||
client.post_json.return_value.body["metadata"] = metadata
|
||||
with pytest.raises(ValueError, match="Invalid timeseries response.*next_cursor"):
|
||||
gateway.get_material_samples(**args)
|
||||
client.post_json.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cursors", [["a", "a"], ["a", "b", "a"]])
|
||||
def test_repeated_pagination_cursor(timeseries, cursors) -> None:
|
||||
gateway, client, args = timeseries
|
||||
data = client.post_json.return_value.body["data"]
|
||||
client.post_json.side_effect = [
|
||||
Mock(body={"data": data, "metadata": {"next_cursor": cursor}}) for cursor in cursors
|
||||
]
|
||||
with pytest.raises(ValueError, match="next_cursor has already been followed"):
|
||||
gateway.get_material_samples(**args)
|
||||
assert client.post_json.call_count == len(cursors)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"data, error",
|
||||
[
|
||||
({"columns": ["time", "rate", "gate"], "records": [[]]}, "malformed record 0"),
|
||||
({"columns": ["time", "rate", "rate", "gate"], "records": []}, "required column"),
|
||||
({"columns": ["time", "rate"], "records": []}, "required column"),
|
||||
({"columns": None, "records": []}, "columns must be a list"),
|
||||
({"columns": ["time", "rate", "gate"], "records": None}, "records must be a list"),
|
||||
(
|
||||
{"columns": ["time", "rate", "gate"], "records": [["2026-09-03T01:00:00", 10, 2]]},
|
||||
"timezone-aware",
|
||||
),
|
||||
(
|
||||
{
|
||||
"columns": ["time", "rate", "gate"],
|
||||
"records": [["2026-09-03T01:00:00Z", float("nan"), 2]],
|
||||
},
|
||||
"finite",
|
||||
),
|
||||
(
|
||||
{"columns": ["time", "rate", "gate"], "records": [["2026-09-03T01:00:00Z", 10, True]]},
|
||||
"numeric",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_malformed_later_page(timeseries, data, error) -> None:
|
||||
gateway, client, args = timeseries
|
||||
first_body = client.post_json.return_value.body
|
||||
first_body["metadata"] = {"next_cursor": "continuation"}
|
||||
client.post_json.side_effect = [
|
||||
Mock(body=first_body),
|
||||
Mock(body={"data": data, "metadata": {"next_cursor": None}}),
|
||||
]
|
||||
with pytest.raises(ValueError, match=f"Invalid timeseries response on page 2:.*{error}"):
|
||||
gateway.get_material_samples(**args)
|
||||
assert client.post_json.call_count == 2
|
||||
|
||||
|
||||
def test_production_runs_follow_pages_and_find_open_run() -> None:
|
||||
client = Mock()
|
||||
closed = dict(
|
||||
uuid="closed",
|
||||
machine="m",
|
||||
production_order="opaque-order",
|
||||
start="2026-09-01T00:00:00Z",
|
||||
end="2026-09-01T01:00:00Z",
|
||||
)
|
||||
opened = {**closed, "uuid": "open", "end": None}
|
||||
pages = [
|
||||
Mock(body={"data": [closed], "metadata": {"next_cursor": "a"}}),
|
||||
Mock(body={"data": [opened], "metadata": {"next_cursor": None}}),
|
||||
]
|
||||
client.get.side_effect = pages
|
||||
gateway = EnlyzeApiGateway(client)
|
||||
runs = gateway.get_production_runs("m")
|
||||
assert [run.uuid for run in runs] == ["closed", "open"]
|
||||
assert runs[0].end == datetime(2026, 9, 1, 1, tzinfo=UTC)
|
||||
assert [call.args for call in client.get.call_args_list] == [
|
||||
("/v2/production-runs", {"machine": "m"}),
|
||||
("/v2/production-runs", {"machine": "m", "cursor": "a"}),
|
||||
]
|
||||
client.get.side_effect = pages
|
||||
assert gateway.get_open_production_run("m") == runs[1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"metadata",
|
||||
[
|
||||
None,
|
||||
[],
|
||||
"bad",
|
||||
{},
|
||||
{"next_cursor": ""},
|
||||
{"next_cursor": 1},
|
||||
{"next_cursor": False},
|
||||
{"next_cursor": []},
|
||||
{"next_cursor": {}},
|
||||
],
|
||||
)
|
||||
def test_production_run_invalid_pagination(metadata) -> None:
|
||||
client = Mock()
|
||||
client.get.return_value.body = {"data": [], "metadata": metadata}
|
||||
with pytest.raises(ValueError, match="Invalid production-run response.*next_cursor"):
|
||||
EnlyzeApiGateway(client).get_production_runs("m")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cursors", [["a", "a"], ["a", "b", "a"]])
|
||||
def test_production_run_cyclic_pagination(cursors) -> None:
|
||||
client = Mock()
|
||||
client.get.side_effect = [
|
||||
Mock(body={"data": [], "metadata": {"next_cursor": cursor}}) for cursor in cursors
|
||||
]
|
||||
with pytest.raises(ValueError, match="next_cursor has already been followed"):
|
||||
EnlyzeApiGateway(client).get_production_runs("m")
|
||||
assert client.get.call_count == len(cursors)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"body",
|
||||
[
|
||||
None,
|
||||
[],
|
||||
{},
|
||||
{"data": {}},
|
||||
{"data": [None]},
|
||||
{
|
||||
"data": [
|
||||
dict(
|
||||
uuid="r",
|
||||
machine="wrong",
|
||||
production_order="o",
|
||||
start="2026-09-03T00:00:00Z",
|
||||
end=None,
|
||||
)
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": [
|
||||
dict(
|
||||
uuid="r",
|
||||
machine="m",
|
||||
production_order="o",
|
||||
start="2026-09-03T00:00:00Z",
|
||||
end="2026-09-02T00:00:00Z",
|
||||
)
|
||||
]
|
||||
},
|
||||
],
|
||||
)
|
||||
def test_production_run_malformed_later_page(body) -> None:
|
||||
client = Mock()
|
||||
client.get.side_effect = [
|
||||
Mock(body={"data": [], "metadata": {"next_cursor": "a"}}),
|
||||
Mock(body=body),
|
||||
]
|
||||
with pytest.raises(ValueError, match="Invalid production-run response on page 2"):
|
||||
EnlyzeApiGateway(client).get_production_runs("m")
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
import re
|
||||
import traceback
|
||||
from dataclasses import FrozenInstanceError, replace
|
||||
from datetime import UTC, datetime
|
||||
from decimal import Decimal
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.erp import ErpReadError, ErpSettings, ErpWorkplaceStatusGateway
|
||||
|
||||
ENV = dict(ERP_DB_HOST='localhost', ERP_DB_PORT='49601', ERP_DB_NAME='NV_DWH',
|
||||
ERP_DB_USER='private-user', ERP_DB_PASSWORD='private-password')
|
||||
NOW = datetime(2026, 9, 4, 21, 20, 50)
|
||||
ROW = ('K7 ', '12026000815 ', '212520 ',
|
||||
'Stex R 1501 C (PR) 5,80 x 50 m ', NOW,
|
||||
Decimal('80040.000'), Decimal('69281.000'), Decimal('10759.000'), 17, 38)
|
||||
|
||||
|
||||
def test_valid_settings():
|
||||
settings = ErpSettings.from_environment(ENV)
|
||||
assert (settings.host, settings.port, settings.database) == ('localhost', 49601, 'NV_DWH')
|
||||
assert settings.user == ENV['ERP_DB_USER']
|
||||
assert settings.password == ENV['ERP_DB_PASSWORD']
|
||||
assert settings.user not in repr(settings)
|
||||
assert settings.password not in repr(settings)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('key', ENV)
|
||||
@pytest.mark.parametrize('value', [None, '', ' ', '\x00'])
|
||||
def test_required_settings(key, value):
|
||||
environment = dict(ENV)
|
||||
if value is None:
|
||||
del environment[key]
|
||||
else:
|
||||
environment[key] = value
|
||||
with pytest.raises(ConfigurationError, match=key):
|
||||
ErpSettings.from_environment(environment)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('port', ['0', '-1', '65536', '1.5', 'private-password'])
|
||||
def test_invalid_port(port):
|
||||
with pytest.raises(ConfigurationError, match='ERP_DB_PORT') as error:
|
||||
ErpSettings.from_environment(dict(ENV, ERP_DB_PORT=port))
|
||||
assert 'private-password' not in ''.join(traceback.format_exception(error.value))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('port', [1, 65535])
|
||||
def test_port_boundaries(port):
|
||||
assert ErpSettings.from_environment(dict(ENV, ERP_DB_PORT=str(port))).port == port
|
||||
|
||||
|
||||
@pytest.mark.parametrize('changes', [{'port': True}, {'port': 1.5}, {'host': ''},
|
||||
{'database': ''}, {'user': ''}, {'password': ''}])
|
||||
def test_direct_settings_validation(changes):
|
||||
with pytest.raises(ConfigurationError):
|
||||
replace(ErpSettings.from_environment(ENV), **changes)
|
||||
|
||||
|
||||
def test_secret_file(tmp_path):
|
||||
path = tmp_path / 'erp.env'
|
||||
path.write_text("ERP_DB_PASSWORD='file password'\n")
|
||||
assert ErpSettings.from_secret_file(path, environment=ENV).password == 'file password'
|
||||
assert ErpSettings.from_secret_file(tmp_path / 'missing', environment=ENV).port == 49601
|
||||
path.write_text("ERP_DB_PASSWORD='private-password\n")
|
||||
with pytest.raises(ConfigurationError) as error:
|
||||
ErpSettings.from_secret_file(path, environment=ENV)
|
||||
assert 'private-password' not in ''.join(traceback.format_exception(error.value))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(monkeypatch):
|
||||
connect = MagicMock()
|
||||
monkeypatch.setattr('production_analytics.erp.gateway.pymssql.connect', connect)
|
||||
connection = connect.return_value.__enter__.return_value
|
||||
cursor = connection.cursor.return_value.__enter__.return_value
|
||||
cursor.fetchmany.return_value = [ROW]
|
||||
return connect, connection, cursor
|
||||
|
||||
|
||||
def read():
|
||||
return ErpWorkplaceStatusGateway(ErpSettings.from_environment(ENV))
|
||||
|
||||
|
||||
def test_valid_row_and_resource_lifecycle(db):
|
||||
connect, connection, cursor = db
|
||||
status = read().get_current_workplace_status('K7')
|
||||
assert status.workplace == 'K7'
|
||||
assert status.production_order == '12026000815'
|
||||
assert status.article_number == '212520'
|
||||
assert status.article_description == 'Stex R 1501 C (PR) 5,80 x 50 m'
|
||||
assert status.feedback_timestamp is NOW
|
||||
assert status.feedback_timestamp.tzinfo is None
|
||||
assert (status.order_quantity_m2, status.good_quantity_m2, status.remaining_quantity_m2,
|
||||
status.remaining_time_hours, status.remaining_rolls) == (
|
||||
80040., 69281., 10759., 17., 38.,
|
||||
)
|
||||
assert type(status.order_quantity_m2) is float
|
||||
with pytest.raises(FrozenInstanceError):
|
||||
status.workplace = 'other'
|
||||
cursor.fetchmany.assert_called_once_with(2)
|
||||
connection.cursor.return_value.__exit__.assert_called_once()
|
||||
connect.return_value.__exit__.assert_called_once()
|
||||
connect.assert_called_once_with(server='localhost', port=49601, database='NV_DWH',
|
||||
user='private-user', password='private-password',
|
||||
login_timeout=10, timeout=10)
|
||||
connection.commit.assert_not_called()
|
||||
|
||||
|
||||
def test_leading_whitespace_and_timezone_preserved(db):
|
||||
row = list(ROW)
|
||||
row[:4] = [' K7 ', ' K 7-001 ', ' 001 ', ' description ']
|
||||
row[4] = NOW.replace(tzinfo=UTC)
|
||||
db[2].fetchmany.return_value = [row]
|
||||
status = read().get_current_workplace_status(' K7')
|
||||
assert (status.workplace, status.production_order, status.article_number,
|
||||
status.article_description) == (' K7', ' K 7-001', ' 001', ' description')
|
||||
assert status.feedback_timestamp is row[4]
|
||||
|
||||
|
||||
def test_nullable_fields(db):
|
||||
db[2].fetchmany.return_value = [('K7', '001', None, None, NOW, *([None] * 5))]
|
||||
status = read().get_current_workplace_status('K7')
|
||||
assert all(getattr(status, key) is None for key in (
|
||||
'article_number', 'article_description', 'order_quantity_m2', 'good_quantity_m2',
|
||||
'remaining_quantity_m2', 'remaining_time_hours', 'remaining_rolls',
|
||||
))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('value', [Decimal('NaN'), Decimal('Infinity'), Decimal('1e999'),
|
||||
float('nan'), True, 'private-password'])
|
||||
def test_invalid_numbers(db, value):
|
||||
row = list(ROW)
|
||||
row[5] = value
|
||||
db[2].fetchmany.return_value = [row]
|
||||
with pytest.raises(ErpReadError, match='invalid row') as error:
|
||||
read().get_current_workplace_status('K7')
|
||||
assert 'private-password' not in ''.join(traceback.format_exception(error.value))
|
||||
|
||||
|
||||
def test_fractional_decimal(db):
|
||||
row = list(ROW)
|
||||
row[5:] = [Decimal('12.125')] * 5
|
||||
db[2].fetchmany.return_value = [row]
|
||||
status = read().get_current_workplace_status('K7')
|
||||
assert status.order_quantity_m2 == status.remaining_rolls == 12.125
|
||||
|
||||
|
||||
def test_zero_rows(db):
|
||||
db[2].fetchmany.return_value = []
|
||||
assert read().get_current_workplace_status('K7') is None
|
||||
|
||||
|
||||
def test_multiple_rows(db):
|
||||
db[2].fetchmany.return_value = [ROW, ROW]
|
||||
with pytest.raises(ErpReadError, match='multiple rows'):
|
||||
read().get_current_workplace_status('K7')
|
||||
|
||||
|
||||
def test_parameterized_read_only_query(db):
|
||||
workplace = "K7'; DELETE FROM anything; --"
|
||||
read().get_current_workplace_status(workplace)
|
||||
db[2].execute.assert_called_once()
|
||||
sql, params = db[2].execute.call_args.args
|
||||
assert params == (workplace,)
|
||||
assert workplace not in sql
|
||||
assert 'WHERE [Arbeitsplatz] = %s' in sql
|
||||
assert re.findall(r'FROM\s+(\S+)', sql) == ['[dbo].[GRAFANA_WORKPLACE_STATUS]']
|
||||
assert not re.search(r'\b(INSERT|UPDATE|DELETE|MERGE|EXEC|TOP|ORDER BY)\b', sql, re.I)
|
||||
assert sql.lstrip().startswith('SELECT')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('stage', ['connect', 'execute', 'fetch', 'cursor_close', 'close'])
|
||||
def test_driver_errors_are_safe(db, stage):
|
||||
connect, connection, cursor = db
|
||||
target = {'connect': connect, 'execute': cursor.execute, 'fetch': cursor.fetchmany,
|
||||
'cursor_close': connection.cursor.return_value.__exit__,
|
||||
'close': connect.return_value.__exit__}[stage]
|
||||
target.side_effect = RuntimeError('private-user private-password')
|
||||
with pytest.raises(ErpReadError, match='read failed') as error:
|
||||
read().get_current_workplace_status('K7')
|
||||
rendered = ''.join(traceback.format_exception(error.value))
|
||||
assert 'private-user' not in rendered
|
||||
assert 'private-password' not in rendered
|
||||
if stage != 'connect':
|
||||
connect.return_value.__exit__.assert_called_once()
|
||||
@@ -0,0 +1,158 @@
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, Mock
|
||||
|
||||
import psycopg
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.calculations.material_application import rotational_application_g_m2
|
||||
from production_analytics.calculations.material_consumption import MaterialSample
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway, EnlyzeProductionRun
|
||||
from production_analytics.service.material_polling import MaterialPollingService
|
||||
from production_analytics.service.postgres_material import PostgresSettings
|
||||
from production_analytics.service.postgres_material_application import (
|
||||
PostgresMaterialApplicationWriter,
|
||||
)
|
||||
|
||||
NOW = datetime(2026, 9, 7, tzinfo=UTC)
|
||||
|
||||
|
||||
def test_realistic_application_and_generic_factor():
|
||||
assert rotational_application_g_m2([3.739, 3.956], 3.12, 8, 0.3) == pytest.approx(3001.05)
|
||||
assert rotational_application_g_m2([2, 3], 1.5, 10, 0.5) == 750
|
||||
|
||||
|
||||
@pytest.mark.parametrize('speed', [-10, 0, 1e-300, 0.299999999, 0.3])
|
||||
def test_inactive_and_near_zero(speed):
|
||||
assert rotational_application_g_m2([3.739, 3.956], 3.12, speed, 0.3) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize('speeds,factor,speed,threshold', [
|
||||
([], 3.12, 8, 0.3), ([float('nan')], 3.12, 8, 0.3),
|
||||
([1], 0, 8, 0.3), ([1], float('inf'), 8, 0.3),
|
||||
([1], 3.12, float('nan'), 0.3), ([1], 3.12, 8, 0),
|
||||
([1], 3.12, 8, float('inf')), ([1e308], 3.12, 8, 0.3),
|
||||
])
|
||||
def test_invalid_inputs(speeds, factor, speed, threshold):
|
||||
with pytest.raises(ValueError):
|
||||
rotational_application_g_m2(speeds, factor, speed, threshold)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('width', [1, 4.85, 5, 100])
|
||||
def test_gateway_width_cancels_and_only_act_signals_requested(width):
|
||||
config = load_material_calculation('config/bento1-material-consumption.yaml')
|
||||
refs = [*config.rotational_speed_signal_refs, config.gate_signal_ref]
|
||||
client = Mock()
|
||||
client.post_json.return_value.body = {'data': {
|
||||
'columns': ['time', *refs],
|
||||
'records': [[NOW.isoformat(), 3.739, 3.956, 8],
|
||||
[(NOW + timedelta(seconds=10)).isoformat(), 3.739, 3.956, 0]],
|
||||
}}
|
||||
samples = EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id=config.machine_ref, rate_variable_id='unused',
|
||||
gate_variable_id=config.gate_signal_ref, start=NOW, end=NOW + timedelta(seconds=10),
|
||||
rotational_speed_variable_ids=config.rotational_speed_signal_refs,
|
||||
nominal_width_m=width, specific_discharge_kg_per_rev_m=3.12,
|
||||
process_application_gate_threshold=0.3,
|
||||
)
|
||||
assert samples[0].application_g_m2 == pytest.approx(3001.05)
|
||||
assert samples[1].application_g_m2 is None
|
||||
assert samples[0].material_rate_kg_per_hour == pytest.approx(7.695 * width * 3.12 * 60)
|
||||
assert client.post_json.call_args.args[1]['variables'] == [{'uuid': ref} for ref in refs]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('changes', [
|
||||
{'process_application_calculation_id': ''}, {'process_application_calculation_id': None},
|
||||
{'process_application_calculation_id': 3}, {'process_application_calculation_id': 'x\x00'},
|
||||
{'process_application_calculation_id': 'bento1-fresh-bentonite-consumption'},
|
||||
{'gate_signal_ref': None}, {'gate_threshold': 0}, {'gate_threshold': -0.3},
|
||||
])
|
||||
def test_process_config_validation(tmp_path, changes):
|
||||
document = yaml.safe_load(Path('config/bento1-material-consumption.yaml').read_text())
|
||||
document['calculations'][0].update(changes)
|
||||
path = tmp_path / 'config.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_persistence_before_checkpoint_retries_and_disjoint_runs():
|
||||
old = EnlyzeProductionRun('old', 'machine', None, 'order', NOW, NOW + timedelta(seconds=10))
|
||||
current = EnlyzeProductionRun(
|
||||
'new', 'machine', None, 'order', NOW + timedelta(seconds=15), None,
|
||||
)
|
||||
gateway = Mock()
|
||||
gateway.get_open_production_run.return_value = current
|
||||
gateway.get_production_runs.return_value = [old, current]
|
||||
|
||||
def samples(**kwargs):
|
||||
start = kwargs['start']
|
||||
assert kwargs['process_application_gate_threshold'] == 0.3
|
||||
return [MaterialSample(start, 7202.52, 8, 3001.05),
|
||||
MaterialSample(start + timedelta(seconds=10), 7202.52, 8, 3001.05)]
|
||||
|
||||
gateway.get_material_samples.side_effect = samples
|
||||
store = Mock(load=Mock(return_value=None))
|
||||
writer = Mock()
|
||||
writer.write.side_effect = [None, psycopg.OperationalError(), None, None]
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway, state_store=store, machine_id='machine', rate_variable_id='unused',
|
||||
gate_variable_id='speed', gate_threshold=0.3, max_sample_gap_seconds=20,
|
||||
rotational_speed_variable_ids=('right', 'left'), specific_discharge_kg_per_rev_m=3.12,
|
||||
nominal_width_provider=lambda order: 5, process_application_calculation_id='application',
|
||||
application_writer=writer,
|
||||
)
|
||||
with pytest.raises(psycopg.OperationalError):
|
||||
service.poll_once(now=NOW + timedelta(seconds=25))
|
||||
store.save.assert_not_called()
|
||||
result = service.poll_once(now=NOW + timedelta(seconds=25))
|
||||
assert result.state.cumulative_consumption_kg == pytest.approx(40.014)
|
||||
assert result.state.integrated_running_seconds == 20
|
||||
assert [c.kwargs['run_id'] for c in writer.write.call_args_list] == ['old', 'new', 'old', 'new']
|
||||
store.save.assert_called_once()
|
||||
|
||||
|
||||
def test_sql_persistence_idempotent_and_inactive_omitted(monkeypatch, sqlite_schema):
|
||||
import sqlite3
|
||||
|
||||
database = sqlite3.connect(':memory:')
|
||||
database.executescript(sqlite_schema)
|
||||
connection = MagicMock()
|
||||
cursor = connection.__enter__.return_value.cursor.return_value.__enter__.return_value
|
||||
cursor.executemany.side_effect = lambda sql, rows: database.executemany(
|
||||
sql.replace('%s', '?'),
|
||||
[(row[0].isoformat(), *row[1:]) for row in rows],
|
||||
)
|
||||
connect = Mock(return_value=connection)
|
||||
monkeypatch.setattr(psycopg, 'connect', connect)
|
||||
writer = PostgresMaterialApplicationWriter(PostgresSettings('host', 5432, 'db', 'u', 'p'))
|
||||
kwargs = dict(calculation_id='application', machine_id='machine', production_order='order',
|
||||
run_id='run', samples=[MaterialSample(NOW, 7202.52, 8, 3001.05),
|
||||
MaterialSample(NOW + timedelta(seconds=10), 7202.52, 0)])
|
||||
writer.write(**kwargs)
|
||||
writer.write(**kwargs)
|
||||
assert database.execute('SELECT * FROM material_application_snapshots').fetchall() == [
|
||||
(NOW.isoformat(), 'application', 'machine', 'order', 'run', 3001.05),
|
||||
]
|
||||
writer.write(**(kwargs | {'samples': [MaterialSample(NOW, 1, 0)]}))
|
||||
assert connect.call_count == 2
|
||||
database.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('collision', ['output', 'consumption'])
|
||||
def test_output_ids_unique_across_config_entries(tmp_path, collision):
|
||||
document = yaml.safe_load(Path('config/bento1-material-consumption.yaml').read_text())
|
||||
first = document['calculations'][0]
|
||||
second = dict(first, id='second-consumption')
|
||||
if collision == 'consumption':
|
||||
second['process_application_calculation_id'] = first['id']
|
||||
document['calculations'].append(second)
|
||||
path = tmp_path / 'config.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError, match='unique'):
|
||||
load_material_calculation(path, first['id'])
|
||||
@@ -0,0 +1,186 @@
|
||||
from dataclasses import FrozenInstanceError, replace
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import Mock
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.erp import CurrentWorkplaceStatus
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencyService
|
||||
from production_analytics.service.postgres_material import MaterialConsumptionSnapshot
|
||||
|
||||
CALC = 'k7-fiber-consumption'
|
||||
MACHINE = 'c220f95c-a65e-4cb7-99b7-0626d6c7508c'
|
||||
START = datetime(2026, 9, 4, 10, tzinfo=UTC)
|
||||
FEEDBACK = START.replace(minute=5)
|
||||
STATUS = CurrentWorkplaceStatus(
|
||||
'K7', '12026000815', '212520', 'Stex R 1501 C (PR) 5,80 x 50 m',
|
||||
FEEDBACK, None, 800, None, None, None,
|
||||
)
|
||||
MATERIAL = MaterialConsumptionSnapshot(START, CALC, MACHINE, 'K 7-12026000815', 'run', 1000)
|
||||
|
||||
|
||||
def service(repository=None, **config):
|
||||
return MaterialEfficiencyService(
|
||||
Mock(get_current_workplace_status=Mock(return_value=STATUS)),
|
||||
repository if repository is not None else Mock(latest_at_or_before=Mock(
|
||||
return_value=MATERIAL,
|
||||
)),
|
||||
**dict(workplace='K7', machine_id=MACHINE, calculation_id=CALC,
|
||||
format_template='K 7-{production_order}', **config),
|
||||
)
|
||||
|
||||
|
||||
def test_valid_current_feedback():
|
||||
subject = service()
|
||||
result = subject.evaluate_current()
|
||||
assert result.material_consumption_kg_per_m2 == 1.25
|
||||
assert result.material_consumption_g_per_m2 == 1250
|
||||
assert result.nominal_width_m == 5.8
|
||||
assert result.good_quantity_m2 == 800
|
||||
assert result.material_consumption_kg == 1000
|
||||
assert result.erp_feedback_timestamp == FEEDBACK
|
||||
assert result.material_snapshot_timestamp == START
|
||||
assert (result.workplace, result.machine_id, result.calculation_id) == ('K7', MACHINE, CALC)
|
||||
assert (result.erp_production_order, result.enlyze_production_order) == (
|
||||
'12026000815', 'K 7-12026000815',
|
||||
)
|
||||
assert (result.article_number, result.article_description) == (
|
||||
STATUS.article_number, STATUS.article_description,
|
||||
)
|
||||
subject.erp.get_current_workplace_status.assert_called_once_with('K7')
|
||||
subject.materials.latest_at_or_before.assert_called_once_with(
|
||||
calculation_id=CALC, machine_id=MACHINE, production_order='K 7-12026000815',
|
||||
timestamp=FEEDBACK,
|
||||
)
|
||||
with pytest.raises(FrozenInstanceError):
|
||||
result.good_quantity_m2 = 5
|
||||
|
||||
|
||||
@pytest.mark.parametrize('quantity', [None, 0, -1, float('nan'), float('inf'), -float('inf')])
|
||||
def test_invalid_quantity(quantity):
|
||||
subject = service()
|
||||
assert subject.evaluate(replace(STATUS, good_quantity_m2=quantity)) is None
|
||||
subject.materials.latest_at_or_before.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('description', [None, 'unstructured article'])
|
||||
def test_width_is_optional_context(description):
|
||||
result = service().evaluate(replace(STATUS, article_description=description))
|
||||
assert result.nominal_width_m is None
|
||||
assert result.material_consumption_kg_per_m2 == 1.25
|
||||
|
||||
|
||||
def test_unavailable_sources():
|
||||
subject = service()
|
||||
subject.erp.get_current_workplace_status.return_value = None
|
||||
assert subject.evaluate_current() is None
|
||||
subject.materials.latest_at_or_before.return_value = None
|
||||
assert subject.evaluate(STATUS) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize('consumption, quantity', [
|
||||
(float('nan'), 800), (float('inf'), 800), (-1, 800),
|
||||
(1e308, 1e-308), (1e308, 1),
|
||||
])
|
||||
def test_invalid_consumption_and_overflow(consumption, quantity):
|
||||
subject = service()
|
||||
subject.materials.latest_at_or_before.return_value = replace(
|
||||
MATERIAL, consumption_kg=consumption,
|
||||
)
|
||||
assert subject.evaluate(replace(STATUS, good_quantity_m2=quantity)) is None
|
||||
|
||||
|
||||
def test_zero_consumption_is_valid():
|
||||
subject = service()
|
||||
subject.materials.latest_at_or_before.return_value = replace(MATERIAL, consumption_kg=0)
|
||||
assert subject.evaluate(STATUS).material_consumption_kg_per_m2 == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize('changes', [
|
||||
dict(timestamp=START.replace(minute=10)), dict(timestamp=START.replace(tzinfo=None)),
|
||||
dict(production_order='K 7-12026000815-K 7-12026000816'),
|
||||
dict(machine_id='other'), dict(calculation_id='other'),
|
||||
])
|
||||
def test_repository_contract_is_checked(changes):
|
||||
subject = service()
|
||||
subject.materials.latest_at_or_before.return_value = replace(MATERIAL, **changes)
|
||||
with pytest.raises(ValueError, match='unaligned'):
|
||||
subject.evaluate(STATUS)
|
||||
|
||||
|
||||
def test_other_explicit_configuration():
|
||||
repository = Mock()
|
||||
repository.latest_at_or_before.return_value = replace(
|
||||
MATERIAL, machine_id='example-machine', production_order='ORDER/00123',
|
||||
)
|
||||
subject = MaterialEfficiencyService(
|
||||
Mock(), repository, workplace='example', machine_id='example-machine',
|
||||
calculation_id=CALC, format_template='ORDER/{production_order}',
|
||||
)
|
||||
result = subject.evaluate(replace(STATUS, workplace='example', production_order=' 00123 '))
|
||||
assert result.enlyze_production_order == 'ORDER/00123'
|
||||
|
||||
|
||||
def test_wrong_workplace_and_combined_order_rejected():
|
||||
with pytest.raises(ValueError, match='workplace'):
|
||||
service().evaluate(replace(STATUS, workplace='other'))
|
||||
with pytest.raises(ValueError, match='ERP production order'):
|
||||
service().evaluate(replace(STATUS, production_order='K 7-123-K 7-456'))
|
||||
|
||||
|
||||
def test_temporal_regression_and_successive_feedback():
|
||||
later = replace(
|
||||
MATERIAL, timestamp=START.replace(minute=10), consumption_kg=1100, run_id='run2',
|
||||
)
|
||||
repository = Mock()
|
||||
repository.latest_at_or_before.side_effect = lambda **kw: max(
|
||||
(row for row in [MATERIAL, later] if row.timestamp <= kw['timestamp']),
|
||||
key=lambda row: row.timestamp, default=None,
|
||||
)
|
||||
subject = service(repository)
|
||||
first = subject.evaluate(STATUS)
|
||||
second = subject.evaluate(replace(
|
||||
STATUS, feedback_timestamp=later.timestamp, good_quantity_m2=1000,
|
||||
))
|
||||
assert first.material_snapshot_timestamp == START
|
||||
assert first.material_consumption_kg == 1000
|
||||
assert first.material_consumption_kg_per_m2 == 1.25
|
||||
assert second.material_snapshot_timestamp == later.timestamp
|
||||
assert second.material_consumption_kg == 1100
|
||||
assert second.material_consumption_kg_per_m2 == 1.1
|
||||
assert first.good_quantity_m2 == 800
|
||||
|
||||
|
||||
def test_naive_feedback_requires_explicit_timezone_and_returns_instant():
|
||||
naive = datetime(2026, 9, 4, 12, 5)
|
||||
status = replace(STATUS, feedback_timestamp=naive)
|
||||
with pytest.raises(ValueError, match='explicit erp_timezone'):
|
||||
service().evaluate(status)
|
||||
subject = service(erp_timezone=ZoneInfo('Europe/Berlin'))
|
||||
result = subject.evaluate(status)
|
||||
assert result.erp_feedback_timestamp == FEEDBACK
|
||||
assert status.feedback_timestamp == naive
|
||||
assert subject.materials.latest_at_or_before.call_args.kwargs['timestamp'] == FEEDBACK
|
||||
|
||||
|
||||
@pytest.mark.parametrize('timestamp', [datetime(2026, 3, 29, 2, 30), datetime(2026, 10, 25, 2, 30)])
|
||||
def test_dst_gap_and_overlap_rejected(timestamp):
|
||||
with pytest.raises(ValueError, match='ambiguous or nonexistent'):
|
||||
service(erp_timezone=ZoneInfo('Europe/Berlin')).evaluate(
|
||||
replace(STATUS, feedback_timestamp=timestamp),
|
||||
)
|
||||
|
||||
|
||||
def test_aware_feedback_returns_utc():
|
||||
timestamp = FEEDBACK.astimezone(ZoneInfo('Europe/Berlin'))
|
||||
result = service().evaluate(replace(STATUS, feedback_timestamp=timestamp))
|
||||
assert result.erp_feedback_timestamp == timestamp
|
||||
assert result.erp_feedback_timestamp.tzinfo is UTC
|
||||
|
||||
|
||||
def test_database_failure_propagates():
|
||||
subject = service()
|
||||
subject.materials.latest_at_or_before.side_effect = RuntimeError('unavailable')
|
||||
with pytest.raises(RuntimeError, match='unavailable'):
|
||||
subject.evaluate(STATUS)
|
||||
@@ -0,0 +1,488 @@
|
||||
import io
|
||||
import sqlite3
|
||||
from dataclasses import replace
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, Mock
|
||||
|
||||
import psycopg
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import CalculationConfigError
|
||||
from production_analytics.cli.__main__ import main
|
||||
from production_analytics.erp import CurrentWorkplaceStatus, ErpReadError
|
||||
from production_analytics.service.material_efficiency import (
|
||||
MaterialEfficiencySnapshot,
|
||||
)
|
||||
from production_analytics.service.material_efficiency_runner import MaterialEfficiencyRunner
|
||||
from production_analytics.service.material_efficiency_runtime import (
|
||||
build_material_efficiency_runner,
|
||||
)
|
||||
from production_analytics.service.postgres_material import (
|
||||
MaterialConsumptionSnapshot,
|
||||
PostgresSettings,
|
||||
)
|
||||
from production_analytics.service.postgres_material_efficiency import (
|
||||
PostgresMaterialEfficiencyWriter,
|
||||
)
|
||||
|
||||
NOW = datetime(2026, 9, 4, 10, tzinfo=UTC)
|
||||
SNAPSHOT = MaterialEfficiencySnapshot(
|
||||
"example",
|
||||
"machine",
|
||||
"calc",
|
||||
"00123",
|
||||
" ORDER/00123'; -- ",
|
||||
"article",
|
||||
"fabric 5 x 50 m",
|
||||
5,
|
||||
NOW,
|
||||
NOW,
|
||||
800,
|
||||
1000,
|
||||
1.25,
|
||||
1250,
|
||||
)
|
||||
SETTINGS = PostgresSettings("localhost", 5432, "analytics", "writer", "secret")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(monkeypatch, sqlite_schema):
|
||||
# Production schema and INSERT semantics, adapted for the SQLite dialect/transport.
|
||||
database = sqlite3.connect(":memory:")
|
||||
database.executescript(sqlite_schema)
|
||||
connection = MagicMock()
|
||||
|
||||
def execute(sql, params):
|
||||
assert sql.lstrip().startswith("INSERT INTO")
|
||||
assert "SELECT" not in sql.upper()
|
||||
assert "RETURNING 1" in sql
|
||||
assert sql.count("%s") == 14
|
||||
assert SNAPSHOT.enlyze_production_order not in sql
|
||||
return database.execute(
|
||||
sql.replace("%s", "?"),
|
||||
tuple(value.isoformat() if isinstance(value, datetime) else value for value in params),
|
||||
)
|
||||
|
||||
connection.__enter__.return_value.execute.side_effect = execute
|
||||
connect = Mock(return_value=connection)
|
||||
monkeypatch.setattr(psycopg, "connect", connect)
|
||||
yield database, connect, connection
|
||||
database.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("nullable", [False, True])
|
||||
def test_all_fields_and_immutable_event(storage, nullable):
|
||||
database, connect, connection = storage
|
||||
snapshot = (
|
||||
replace(SNAPSHOT, article_number=None, article_description=None, nominal_width_m=None)
|
||||
if nullable
|
||||
else SNAPSHOT
|
||||
)
|
||||
writer = PostgresMaterialEfficiencyWriter(SETTINGS)
|
||||
assert writer.write(snapshot) is True
|
||||
inserted = writer.write(
|
||||
replace(
|
||||
snapshot,
|
||||
material_consumption_kg=9999,
|
||||
material_snapshot_timestamp=NOW.replace(minute=1),
|
||||
)
|
||||
)
|
||||
assert inserted is False
|
||||
assert connection.__enter__.return_value.execute.call_count == 2
|
||||
assert database.execute("SELECT * FROM material_efficiency_snapshots").fetchall() == [
|
||||
(
|
||||
NOW.isoformat(),
|
||||
NOW.isoformat(),
|
||||
"calc",
|
||||
"machine",
|
||||
"example",
|
||||
"00123",
|
||||
SNAPSHOT.enlyze_production_order,
|
||||
snapshot.article_number,
|
||||
snapshot.article_description,
|
||||
snapshot.nominal_width_m,
|
||||
800,
|
||||
1000,
|
||||
1.25,
|
||||
1250,
|
||||
)
|
||||
]
|
||||
assert connect.call_count == 2
|
||||
assert connect.call_args.kwargs["connect_timeout"] == 10
|
||||
connection.__exit__.assert_called_with(None, None, None)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field,value",
|
||||
[
|
||||
("calculation_id", "other"),
|
||||
("machine_id", "other"),
|
||||
("enlyze_production_order", SNAPSHOT.enlyze_production_order.strip()),
|
||||
("enlyze_production_order", SNAPSHOT.enlyze_production_order + "-combined"),
|
||||
("erp_feedback_timestamp", NOW.replace(minute=1)),
|
||||
],
|
||||
)
|
||||
def test_exact_identity(storage, field, value):
|
||||
writer = PostgresMaterialEfficiencyWriter(SETTINGS)
|
||||
writer.write(SNAPSHOT)
|
||||
writer.write(replace(SNAPSHOT, **{field: value}))
|
||||
assert storage[0].execute("SELECT count(*) FROM material_efficiency_snapshots").fetchone() == (
|
||||
2,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["erp_feedback_timestamp", "material_snapshot_timestamp"])
|
||||
def test_naive_rejected(storage, field):
|
||||
with pytest.raises(ValueError, match="timezone-aware"):
|
||||
PostgresMaterialEfficiencyWriter(SETTINGS).write(
|
||||
replace(SNAPSHOT, **{field: NOW.replace(tzinfo=None)}),
|
||||
)
|
||||
storage[1].assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field",
|
||||
[
|
||||
"nominal_width_m",
|
||||
"good_quantity_m2",
|
||||
"material_consumption_kg",
|
||||
"material_consumption_kg_per_m2",
|
||||
"material_consumption_g_per_m2",
|
||||
],
|
||||
)
|
||||
@pytest.mark.parametrize("value", [float("nan"), float("inf"), -float("inf")])
|
||||
def test_nonfinite_rejected(storage, field, value):
|
||||
with pytest.raises(ValueError, match="finite"):
|
||||
PostgresMaterialEfficiencyWriter(SETTINGS).write(replace(SNAPSHOT, **{field: value}))
|
||||
storage[1].assert_not_called()
|
||||
|
||||
|
||||
def run_cycles(service, writer, cycles, interval=60, stdout=None):
|
||||
sleep = Mock(side_effect=[None] * (cycles - 1) + [KeyboardInterrupt])
|
||||
errors = io.StringIO()
|
||||
MaterialEfficiencyRunner(
|
||||
service, writer, poll_interval_seconds=interval, sleep=sleep, stderr=errors, stdout=stdout
|
||||
).run()
|
||||
assert sleep.call_args_list == [((interval,),)] * cycles
|
||||
return errors.getvalue()
|
||||
|
||||
|
||||
def test_runner_history_and_none(storage):
|
||||
service = Mock(
|
||||
evaluate_current=Mock(
|
||||
side_effect=[
|
||||
None,
|
||||
SNAPSHOT,
|
||||
SNAPSHOT,
|
||||
replace(SNAPSHOT, erp_feedback_timestamp=NOW.replace(minute=15)),
|
||||
]
|
||||
)
|
||||
)
|
||||
output = io.StringIO()
|
||||
run_cycles(service, PostgresMaterialEfficiencyWriter(SETTINGS), 4, 73, stdout=output)
|
||||
lines = output.getvalue().splitlines()
|
||||
assert len(lines) == 2
|
||||
assert lines[0].startswith(NOW.isoformat())
|
||||
assert lines[1].startswith(NOW.replace(minute=15).isoformat())
|
||||
assert storage[1].call_count == 3
|
||||
assert storage[0].execute("SELECT count(*) FROM material_efficiency_snapshots").fetchone() == (
|
||||
2,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("error", [ErpReadError, psycopg.OperationalError, psycopg.InterfaceError])
|
||||
def test_read_recovers(error):
|
||||
service = Mock(evaluate_current=Mock(side_effect=[error("secret"), SNAPSHOT]))
|
||||
writer = Mock()
|
||||
errors = run_cycles(service, writer, 2)
|
||||
writer.write.assert_called_once_with(SNAPSHOT)
|
||||
assert error.__name__ in errors and "secret" not in errors
|
||||
|
||||
|
||||
def test_write_recovers_with_fresh_transaction(storage):
|
||||
database, connect, connection = storage
|
||||
connect.side_effect = [psycopg.OperationalError("secret"), connection]
|
||||
errors = run_cycles(
|
||||
Mock(evaluate_current=Mock(return_value=SNAPSHOT)),
|
||||
PostgresMaterialEfficiencyWriter(SETTINGS),
|
||||
2,
|
||||
)
|
||||
assert "OperationalError" in errors and "secret" not in errors
|
||||
assert connect.call_count == 2
|
||||
assert database.execute("SELECT count(*) FROM material_efficiency_snapshots").fetchone() == (1,)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("error", [ValueError, TypeError, psycopg.ProgrammingError])
|
||||
def test_programming_errors_propagate(error):
|
||||
with pytest.raises(error):
|
||||
run_cycles(Mock(evaluate_current=Mock(side_effect=error("bad contract"))), Mock(), 1)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("interval", [0, -1, float("nan"), float("inf"), True])
|
||||
def test_invalid_interval(interval):
|
||||
with pytest.raises(CalculationConfigError):
|
||||
MaterialEfficiencyRunner(Mock(), Mock(), poll_interval_seconds=interval)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def runtime_files(tmp_path):
|
||||
config = tmp_path / "config.yaml"
|
||||
config.write_text(
|
||||
yaml.safe_dump(
|
||||
dict(
|
||||
workplace="example",
|
||||
machine_id="machine",
|
||||
calculation_id="calc",
|
||||
production_order_format="ORDER/{production_order}",
|
||||
erp_timezone="Europe/Berlin",
|
||||
)
|
||||
)
|
||||
)
|
||||
postgres = tmp_path / "postgres.env"
|
||||
postgres.write_text(
|
||||
"POSTGRES_HOST=localhost\nPOSTGRES_PORT=5432\nPOSTGRES_DB=analytics\n"
|
||||
"POSTGRES_USER=writer\nPOSTGRES_PASSWORD=secret\n"
|
||||
)
|
||||
erp = tmp_path / "erp.env"
|
||||
erp.write_text(
|
||||
"ERP_DB_HOST=localhost\nERP_DB_PORT=1433\nERP_DB_NAME=erp\n"
|
||||
"ERP_DB_USER=reader\nERP_DB_PASSWORD=secret\n"
|
||||
)
|
||||
return config, postgres, erp
|
||||
|
||||
|
||||
def test_configured_runtime_naive_feedback_to_persistence(runtime_files, storage):
|
||||
config, postgres, erp = runtime_files
|
||||
runner = build_material_efficiency_runner(config, secrets_file=postgres, erp_secrets_file=erp)
|
||||
assert runner.interval == 60
|
||||
runner.service.erp = Mock(
|
||||
get_current_workplace_status=Mock(
|
||||
return_value=CurrentWorkplaceStatus(
|
||||
"example",
|
||||
"00123",
|
||||
None,
|
||||
None,
|
||||
datetime(2026, 9, 4, 12),
|
||||
None,
|
||||
800,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
)
|
||||
)
|
||||
runner.service.materials = Mock(
|
||||
latest_at_or_before=Mock(
|
||||
return_value=MaterialConsumptionSnapshot(
|
||||
NOW,
|
||||
"calc",
|
||||
"machine",
|
||||
"ORDER/00123",
|
||||
"run",
|
||||
1000,
|
||||
)
|
||||
)
|
||||
)
|
||||
runner.sleep = Mock(side_effect=KeyboardInterrupt)
|
||||
runner.run()
|
||||
row = storage[0].execute("SELECT * FROM material_efficiency_snapshots").fetchone()
|
||||
assert row[:7] == (
|
||||
NOW.isoformat(),
|
||||
NOW.isoformat(),
|
||||
"calc",
|
||||
"machine",
|
||||
"example",
|
||||
"00123",
|
||||
"ORDER/00123",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"changes",
|
||||
[
|
||||
{"erp_timezone": "invalid"},
|
||||
{"production_order_format": "missing"},
|
||||
{"workplace": ""},
|
||||
{"machine_id": None},
|
||||
{"poll_interval_seconds": 0},
|
||||
{"unknown": 1},
|
||||
],
|
||||
)
|
||||
def test_startup_config_errors(runtime_files, changes, capsys):
|
||||
config, postgres, erp = runtime_files
|
||||
config.write_text(yaml.safe_dump(yaml.safe_load(config.read_text()) | changes))
|
||||
assert (
|
||||
main(
|
||||
[
|
||||
"run",
|
||||
"material-efficiency",
|
||||
"--config",
|
||||
str(config),
|
||||
"--secrets-file",
|
||||
str(postgres),
|
||||
"--erp-secrets-file",
|
||||
str(erp),
|
||||
]
|
||||
)
|
||||
== 2
|
||||
)
|
||||
assert "Config/startup error" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_k7_config_and_cli(runtime_files, monkeypatch):
|
||||
_, postgres, erp = runtime_files
|
||||
runner = build_material_efficiency_runner(
|
||||
Path("config/k7-material-efficiency.yaml"), secrets_file=postgres, erp_secrets_file=erp
|
||||
)
|
||||
assert runner.service.workplace == "K7"
|
||||
assert runner.service.machine_id == "c220f95c-a65e-4cb7-99b7-0626d6c7508c"
|
||||
assert runner.service.calculation_id == "k7-fiber-consumption"
|
||||
assert runner.service.format_template == "K 7-{production_order}"
|
||||
assert runner.service.erp_timezone.key == "Europe/Berlin"
|
||||
run = Mock()
|
||||
monkeypatch.setattr(MaterialEfficiencyRunner, "run", run)
|
||||
assert (
|
||||
main(
|
||||
[
|
||||
"run",
|
||||
"material-efficiency",
|
||||
"--config",
|
||||
"config/k7-material-efficiency.yaml",
|
||||
"--secrets-file",
|
||||
str(postgres),
|
||||
"--erp-secrets-file",
|
||||
str(erp),
|
||||
]
|
||||
)
|
||||
== 0
|
||||
)
|
||||
run.assert_called_once()
|
||||
|
||||
|
||||
def test_failed_insert_rolls_back_and_later_cycle_reconnects(storage):
|
||||
database, connect, connection = storage
|
||||
execute = connection.__enter__.return_value.execute
|
||||
original = execute.side_effect
|
||||
attempts = 0
|
||||
|
||||
def fail_once(sql, params):
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
if attempts == 1:
|
||||
raise psycopg.OperationalError("secret SQL")
|
||||
return original(sql, params)
|
||||
|
||||
execute.side_effect = fail_once
|
||||
errors = run_cycles(
|
||||
Mock(evaluate_current=Mock(return_value=SNAPSHOT)),
|
||||
PostgresMaterialEfficiencyWriter(SETTINGS),
|
||||
2,
|
||||
)
|
||||
assert "secret" not in errors
|
||||
assert connect.call_count == 2
|
||||
assert connection.__exit__.call_args_list[0].args[0] is psycopg.OperationalError
|
||||
assert database.execute("SELECT count(*) FROM material_efficiency_snapshots").fetchone() == (1,)
|
||||
|
||||
|
||||
def test_configured_interval_and_missing_settings(runtime_files, monkeypatch):
|
||||
config, postgres, erp = runtime_files
|
||||
config.write_text(
|
||||
yaml.safe_dump(yaml.safe_load(config.read_text()) | {"poll_interval_seconds": 91})
|
||||
)
|
||||
runner = build_material_efficiency_runner(config, secrets_file=postgres, erp_secrets_file=erp)
|
||||
assert runner.interval == 91
|
||||
postgres.write_text("")
|
||||
for key in ("HOST", "PORT", "DB", "USER", "PASSWORD"):
|
||||
monkeypatch.delenv(f"POSTGRES_{key}", raising=False)
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
|
||||
with pytest.raises(ConfigurationError, match="POSTGRES_HOST"):
|
||||
build_material_efficiency_runner(config, secrets_file=postgres, erp_secrets_file=erp)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("result", [None, False, True])
|
||||
def test_runner_success_output(result):
|
||||
snapshot = replace(SNAPSHOT, enlyze_production_order="ORDER/00123")
|
||||
service = Mock(evaluate_current=Mock(return_value=None if result is None else snapshot))
|
||||
writer = Mock(write=Mock(return_value=result))
|
||||
output = Mock(wraps=io.StringIO())
|
||||
errors = run_cycles(service, writer, 1, stdout=output)
|
||||
assert errors == ""
|
||||
if result is None:
|
||||
writer.write.assert_not_called()
|
||||
else:
|
||||
writer.write.assert_called_once_with(snapshot)
|
||||
if result is True:
|
||||
assert output.getvalue() == (
|
||||
"2026-09-04T10:00:00+00:00 workplace='example' production_order='ORDER/00123' "
|
||||
"good_m2=800.000 material_kg=1000.000 g_per_m2=1250.000\n"
|
||||
)
|
||||
output.flush.assert_called_once_with()
|
||||
else:
|
||||
assert output.getvalue() == ""
|
||||
output.flush.assert_not_called()
|
||||
|
||||
|
||||
def test_repeated_duplicates_are_silent(storage):
|
||||
writer = PostgresMaterialEfficiencyWriter(SETTINGS)
|
||||
assert writer.write(SNAPSHOT) is True
|
||||
output = io.StringIO()
|
||||
run_cycles(Mock(evaluate_current=Mock(return_value=SNAPSHOT)), writer, 3, stdout=output)
|
||||
assert output.getvalue() == ""
|
||||
|
||||
|
||||
def test_commit_failure_does_not_log_success(storage):
|
||||
_, _, connection = storage
|
||||
connection.__exit__.side_effect = [psycopg.OperationalError("secret driver SQL"), None]
|
||||
output = io.StringIO()
|
||||
errors = run_cycles(
|
||||
Mock(evaluate_current=Mock(return_value=SNAPSHOT)),
|
||||
PostgresMaterialEfficiencyWriter(SETTINGS),
|
||||
1,
|
||||
stdout=output,
|
||||
)
|
||||
assert output.getvalue() == ""
|
||||
assert errors == "Material efficiency cycle failed: OperationalError\n"
|
||||
|
||||
|
||||
def test_bento_uses_generic_efficiency_and_distinct_output_id(runtime_files, storage):
|
||||
_, postgres, erp = runtime_files
|
||||
runner = build_material_efficiency_runner(
|
||||
Path('config/bento1-material-efficiency.yaml'),
|
||||
secrets_file=postgres, erp_secrets_file=erp,
|
||||
)
|
||||
from production_analytics.service.material_efficiency import MaterialEfficiencyService
|
||||
|
||||
assert type(runner.service) is MaterialEfficiencyService
|
||||
service = runner.service
|
||||
status = CurrentWorkplaceStatus(
|
||||
'Bento 1', '00123', None, None, NOW, None, 800, None, None, None,
|
||||
)
|
||||
service.erp = Mock(get_current_workplace_status=Mock(return_value=status))
|
||||
service.materials = Mock(latest_at_or_before=Mock(return_value=MaterialConsumptionSnapshot(
|
||||
NOW, 'bento1-fresh-bentonite-consumption', service.machine_id, 'Bento 1-00123', 'run', 2400,
|
||||
)))
|
||||
snapshot = service.evaluate_current()
|
||||
assert snapshot.calculation_id == 'bento1-fresh-bentonite-efficiency'
|
||||
assert snapshot.material_consumption_g_per_m2 == 3000
|
||||
assert snapshot.nominal_width_m is None
|
||||
service.materials.latest_at_or_before.assert_called_once_with(
|
||||
calculation_id='bento1-fresh-bentonite-consumption', machine_id=service.machine_id,
|
||||
production_order='Bento 1-00123', timestamp=NOW,
|
||||
)
|
||||
runner.writer.write(snapshot)
|
||||
row = storage[0].execute(
|
||||
'SELECT calculation_id, material_consumption_g_per_m2 FROM material_efficiency_snapshots'
|
||||
).fetchone()
|
||||
assert row == ('bento1-fresh-bentonite-efficiency', 3000)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('value', ['', None, 12, 'bad\x00id'])
|
||||
def test_invalid_output_calculation_id(runtime_files, value):
|
||||
config, postgres, erp = runtime_files
|
||||
document = yaml.safe_load(config.read_text()) | {'output_calculation_id': value}
|
||||
config.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError, match='output_calculation_id'):
|
||||
build_material_efficiency_runner(config, secrets_file=postgres, erp_secrets_file=erp)
|
||||
@@ -0,0 +1,409 @@
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialIntegrationState,
|
||||
MaterialSample,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeProductionRun
|
||||
from production_analytics.service.material_polling import (
|
||||
MaterialPollingService,
|
||||
MaterialPollingState,
|
||||
)
|
||||
|
||||
|
||||
def test_poll_once_starts_at_run_start_without_saved_state() -> None:
|
||||
gateway = Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
state_store = Mock()
|
||||
state_store.load.return_value = None
|
||||
|
||||
run = EnlyzeProductionRun(
|
||||
uuid="run-1",
|
||||
machine_id="machine-1",
|
||||
product_id="product-1",
|
||||
production_order="ORDER-1",
|
||||
start=datetime(2026, 9, 3, 5, 7, 47, tzinfo=UTC),
|
||||
end=None,
|
||||
)
|
||||
gateway.get_open_production_run.return_value = run
|
||||
gateway.get_production_runs.return_value = [run]
|
||||
|
||||
samples = [
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 5, 7, 50, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 5, 8, 0, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
]
|
||||
gateway.get_material_samples.return_value = samples
|
||||
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=state_store,
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
gate_threshold=0.5,
|
||||
max_sample_gap_seconds=20.0,
|
||||
)
|
||||
|
||||
result = service.poll_once(
|
||||
now=datetime(2026, 9, 3, 5, 8, 10, tzinfo=UTC),
|
||||
)
|
||||
|
||||
assert result is not None
|
||||
assert result.run == run
|
||||
assert result.state.cumulative_consumption_kg == 10.0
|
||||
assert result.state.integrated_running_seconds == 10.0
|
||||
|
||||
gateway.get_material_samples.assert_called_once_with(
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
start=run.start,
|
||||
end=datetime(2026, 9, 3, 5, 8, 10, tzinfo=UTC),
|
||||
)
|
||||
|
||||
state_store.save.assert_called_once_with(
|
||||
"machine-1",
|
||||
run.production_order,
|
||||
MaterialPollingState(
|
||||
run_id=run.uuid,
|
||||
integration_state=result.state,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_poll_once_resumes_from_saved_timestamp_without_double_counting() -> None:
|
||||
gateway = Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
state_store = Mock()
|
||||
|
||||
run = EnlyzeProductionRun(
|
||||
uuid="run-1",
|
||||
machine_id="machine-1",
|
||||
product_id="product-1",
|
||||
production_order="ORDER-1",
|
||||
start=datetime(2026, 9, 3, 5, 7, 47, tzinfo=UTC),
|
||||
end=None,
|
||||
)
|
||||
gateway.get_open_production_run.return_value = run
|
||||
|
||||
saved_state = MaterialIntegrationState(
|
||||
cumulative_consumption_kg=10.0,
|
||||
integrated_running_seconds=10.0,
|
||||
last_processed_timestamp=datetime(2026, 9, 3, 5, 8, 0, tzinfo=UTC),
|
||||
last_material_rate_kg_per_hour=3600.0,
|
||||
last_gate_value=1.0,
|
||||
integration_active=True,
|
||||
)
|
||||
state_store.load.return_value = MaterialPollingState(
|
||||
run_id=run.uuid,
|
||||
integration_state=saved_state,
|
||||
)
|
||||
|
||||
gateway.get_material_samples.return_value = [
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 5, 8, 0, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 5, 8, 10, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
]
|
||||
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=state_store,
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
gate_threshold=0.5,
|
||||
max_sample_gap_seconds=20.0,
|
||||
)
|
||||
|
||||
result = service.poll_once(
|
||||
now=datetime(2026, 9, 3, 5, 8, 20, tzinfo=UTC),
|
||||
)
|
||||
|
||||
assert result is not None
|
||||
gateway.get_production_runs.assert_not_called()
|
||||
assert result.state.cumulative_consumption_kg == 20.0
|
||||
assert result.state.integrated_running_seconds == 20.0
|
||||
|
||||
gateway.get_material_samples.assert_called_once_with(
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
start=saved_state.last_processed_timestamp,
|
||||
end=datetime(2026, 9, 3, 5, 8, 20, tzinfo=UTC),
|
||||
)
|
||||
|
||||
|
||||
def test_poll_once_without_open_run_does_nothing() -> None:
|
||||
gateway = Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
state_store = Mock()
|
||||
gateway.get_open_production_run.return_value = None
|
||||
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=state_store,
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
gate_threshold=0.5,
|
||||
max_sample_gap_seconds=20.0,
|
||||
)
|
||||
|
||||
result = service.poll_once(
|
||||
now=datetime(2026, 9, 3, 5, 8, 20, tzinfo=UTC),
|
||||
)
|
||||
|
||||
assert result is None
|
||||
gateway.get_material_samples.assert_not_called()
|
||||
state_store.load.assert_not_called()
|
||||
state_store.save.assert_not_called()
|
||||
|
||||
|
||||
def test_new_run_of_same_order_keeps_total_but_restarts_integration_at_run_start() -> None:
|
||||
gateway = Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
state_store = Mock()
|
||||
|
||||
run = EnlyzeProductionRun(
|
||||
uuid="run-2",
|
||||
machine_id="machine-1",
|
||||
product_id="product-1",
|
||||
production_order="ORDER-1",
|
||||
start=datetime(2026, 9, 3, 8, 0, 0, tzinfo=UTC),
|
||||
end=None,
|
||||
)
|
||||
gateway.get_open_production_run.return_value = run
|
||||
|
||||
previous_integration_state = MaterialIntegrationState(
|
||||
cumulative_consumption_kg=100.0,
|
||||
integrated_running_seconds=360.0,
|
||||
last_processed_timestamp=datetime(2026, 9, 3, 6, 0, 0, tzinfo=UTC),
|
||||
last_material_rate_kg_per_hour=3600.0,
|
||||
last_gate_value=1.0,
|
||||
integration_active=True,
|
||||
)
|
||||
|
||||
state_store.load.return_value = MaterialPollingState(
|
||||
run_id="run-1",
|
||||
integration_state=previous_integration_state,
|
||||
)
|
||||
|
||||
gateway.get_material_samples.return_value = [
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 8, 0, 10, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
MaterialSample(
|
||||
timestamp=datetime(2026, 9, 3, 8, 0, 20, tzinfo=UTC),
|
||||
material_rate_kg_per_hour=3600.0,
|
||||
gate_value=1.0,
|
||||
),
|
||||
]
|
||||
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway,
|
||||
state_store=state_store,
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
gate_threshold=0.5,
|
||||
max_sample_gap_seconds=20.0,
|
||||
)
|
||||
|
||||
result = service.poll_once(
|
||||
now=datetime(2026, 9, 3, 8, 0, 30, tzinfo=UTC),
|
||||
)
|
||||
|
||||
assert result is not None
|
||||
gateway.get_production_runs.assert_not_called()
|
||||
assert result.state.cumulative_consumption_kg == 110.0
|
||||
assert result.state.integrated_running_seconds == 370.0
|
||||
|
||||
state_store.load.assert_called_once_with(
|
||||
"machine-1",
|
||||
"ORDER-1",
|
||||
)
|
||||
|
||||
gateway.get_material_samples.assert_called_once_with(
|
||||
machine_id="machine-1",
|
||||
rate_variable_id="rate-variable",
|
||||
gate_variable_id="gate-variable",
|
||||
start=run.start,
|
||||
end=datetime(2026, 9, 3, 8, 0, 30, tzinfo=UTC),
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def polling(tmp_path):
|
||||
from production_analytics.service.material_state_store import JsonMaterialStateStore
|
||||
|
||||
gateway = Mock()
|
||||
gateway.get_production_runs.return_value = []
|
||||
gateway.get_open_production_run.return_value = EnlyzeProductionRun(
|
||||
"run", "machine", None, "order", datetime(2026, 9, 3, tzinfo=UTC), None,
|
||||
)
|
||||
gateway.get_material_samples.return_value = []
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway, state_store=store, machine_id="machine",
|
||||
rate_variable_id="rate", gate_variable_id="gate", gate_threshold=0.5,
|
||||
max_sample_gap_seconds=20,
|
||||
)
|
||||
return service, gateway, store
|
||||
|
||||
|
||||
def test_empty_response_persists_state_without_inferred_consumption(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
now = datetime(2026, 9, 3, 1, tzinfo=UTC)
|
||||
assert service.poll_once(now=now).state == MaterialIntegrationState()
|
||||
saved = MaterialPollingState("run", MaterialIntegrationState(
|
||||
cumulative_consumption_kg=10, integrated_running_seconds=10,
|
||||
last_processed_timestamp=now, last_material_rate_kg_per_hour=3600,
|
||||
last_gate_value=1, integration_active=True,
|
||||
))
|
||||
store.save("machine", "order", saved)
|
||||
assert service.poll_once(now=now).state == saved.integration_state
|
||||
assert store.load("machine", "order") == saved
|
||||
|
||||
|
||||
def test_different_order_does_not_reuse_state(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
store.save("machine", "other-order", MaterialPollingState(
|
||||
"old", MaterialIntegrationState(cumulative_consumption_kg=100),
|
||||
))
|
||||
result = service.poll_once(now=datetime(2026, 9, 3, 1, tzinfo=UTC))
|
||||
assert result.state == MaterialIntegrationState()
|
||||
assert gateway.get_material_samples.call_args.kwargs["start"] == result.run.start
|
||||
assert store.load("machine", "other-order").integration_state.cumulative_consumption_kg == 100
|
||||
|
||||
|
||||
def test_now_before_run_start_does_not_query_or_save(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
assert service.poll_once(now=datetime(2026, 9, 2, tzinfo=UTC)) is None
|
||||
gateway.get_production_runs.assert_not_called()
|
||||
gateway.get_material_samples.assert_not_called()
|
||||
assert store.load("machine", "order") is None
|
||||
|
||||
|
||||
def test_naive_now_is_rejected_before_io(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
with pytest.raises(ValueError, match="now must be timezone-aware"):
|
||||
service.poll_once(now=datetime(2026, 9, 3))
|
||||
gateway.get_open_production_run.assert_not_called()
|
||||
|
||||
|
||||
def test_new_run_empty_response_resets_baseline_and_retains_totals(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
store.save("machine", "order", MaterialPollingState("previous", MaterialIntegrationState(
|
||||
cumulative_consumption_kg=100, integrated_running_seconds=50,
|
||||
last_processed_timestamp=datetime(2026, 9, 2, 23, 59, 59, tzinfo=UTC),
|
||||
last_material_rate_kg_per_hour=3600, last_gate_value=1, integration_active=True,
|
||||
)))
|
||||
result = service.poll_once(now=datetime(2026, 9, 3, 1, tzinfo=UTC))
|
||||
assert result.state == MaterialIntegrationState(
|
||||
cumulative_consumption_kg=100, integrated_running_seconds=50,
|
||||
)
|
||||
assert store.load("machine", "order") == MaterialPollingState("run", result.state)
|
||||
|
||||
|
||||
def test_now_before_saved_timestamp_leaves_state_unchanged(polling) -> None:
|
||||
service, gateway, store = polling
|
||||
saved = MaterialPollingState("run", MaterialIntegrationState(
|
||||
last_processed_timestamp=datetime(2026, 9, 3, 2, tzinfo=UTC),
|
||||
))
|
||||
store.save("machine", "order", saved)
|
||||
assert service.poll_once(now=datetime(2026, 9, 3, 1, tzinfo=UTC)) is None
|
||||
gateway.get_material_samples.assert_not_called()
|
||||
assert store.load("machine", "order") == saved
|
||||
|
||||
|
||||
@pytest.mark.parametrize("order", [
|
||||
"K 7-12026000815", "K 7-12025000074-K 7-12025000075",
|
||||
])
|
||||
@pytest.mark.parametrize("gap_seconds", [5, 86400])
|
||||
def test_bootstrap_replays_exact_order_segments_and_then_resumes(
|
||||
polling, order, gap_seconds,
|
||||
) -> None:
|
||||
from dataclasses import replace
|
||||
from datetime import timedelta
|
||||
|
||||
service, gateway, store = polling
|
||||
base = datetime(2026, 9, 3, tzinfo=UTC)
|
||||
runs = [
|
||||
EnlyzeProductionRun(
|
||||
f"run-{i}", "machine", None, order,
|
||||
base + timedelta(seconds=i * (10 + gap_seconds)),
|
||||
base + timedelta(seconds=i * (10 + gap_seconds) + 10) if i < 4 else None,
|
||||
)
|
||||
for i in range(5)
|
||||
]
|
||||
current = runs[-1]
|
||||
now = current.start + timedelta(seconds=10)
|
||||
gateway.get_open_production_run.return_value = current
|
||||
gateway.get_production_runs.return_value = [
|
||||
current, runs[2], runs[0], runs[3], runs[1],
|
||||
replace(runs[0], uuid="unrelated", production_order=order + "-suffix"),
|
||||
replace(runs[0], uuid="component", production_order="K 7-12025000074"),
|
||||
replace(runs[0], uuid="whitespace", production_order=order + " "),
|
||||
replace(current, uuid="future", start=now + timedelta(days=1)),
|
||||
]
|
||||
|
||||
def samples(**kwargs):
|
||||
return [
|
||||
MaterialSample(kwargs["start"], 3600, 1),
|
||||
MaterialSample(kwargs["end"], 3600, 1),
|
||||
]
|
||||
|
||||
gateway.get_material_samples.side_effect = samples
|
||||
result = service.poll_once(now=now)
|
||||
assert result.state.cumulative_consumption_kg == 50
|
||||
assert result.state.integrated_running_seconds == 50
|
||||
assert [
|
||||
(call.kwargs["start"], call.kwargs["end"])
|
||||
for call in gateway.get_material_samples.call_args_list
|
||||
] == [(run.start, run.end or now) for run in runs]
|
||||
assert store.load("machine", order) == MaterialPollingState(current.uuid, result.state)
|
||||
|
||||
gateway.get_production_runs.reset_mock()
|
||||
gateway.get_material_samples.reset_mock()
|
||||
result = service.poll_once(now=now + timedelta(seconds=10))
|
||||
gateway.get_production_runs.assert_not_called()
|
||||
assert gateway.get_material_samples.call_args.kwargs["start"] == now
|
||||
assert result.state.cumulative_consumption_kg == 60
|
||||
assert result.state.integrated_running_seconds == 60
|
||||
|
||||
|
||||
def test_bootstrap_failure_does_not_persist_partial_totals(polling) -> None:
|
||||
from dataclasses import replace
|
||||
from datetime import timedelta
|
||||
|
||||
service, gateway, store = polling
|
||||
current = gateway.get_open_production_run.return_value
|
||||
gateway.get_production_runs.return_value = [
|
||||
replace(current, uuid="old", start=current.start - timedelta(hours=1),
|
||||
end=current.start - timedelta(minutes=30)),
|
||||
current,
|
||||
]
|
||||
gateway.get_material_samples.side_effect = [[], ValueError("failed page")]
|
||||
with pytest.raises(ValueError, match="failed page"):
|
||||
service.poll_once(now=current.start + timedelta(seconds=10))
|
||||
assert store.load("machine", "order") is None
|
||||
@@ -0,0 +1,291 @@
|
||||
import io
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.calculations.material_consumption import MaterialIntegrationState
|
||||
from production_analytics.cli.__main__ import main
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.enlyze.gateway import EnlyzeProductionRun
|
||||
from production_analytics.service.material_polling import MaterialPollResult
|
||||
from production_analytics.service.material_runner import MaterialPollingRunner, MaterialStateError
|
||||
from production_analytics.service.material_runtime import (
|
||||
_ReportingStateStore,
|
||||
build_material_runner,
|
||||
)
|
||||
|
||||
EXAMPLE = Path(__file__).resolve().parents[1] / 'config/k7-material-consumption.yaml'
|
||||
NOW = datetime(2026, 1, 1, tzinfo=UTC)
|
||||
|
||||
|
||||
def write_config(tmp_path, entries):
|
||||
path = tmp_path / 'calculations.yaml'
|
||||
path.write_text(yaml.safe_dump({'calculations': entries}))
|
||||
return path
|
||||
|
||||
|
||||
def entry():
|
||||
return yaml.safe_load(EXAMPLE.read_text())['calculations'][0]
|
||||
|
||||
|
||||
def test_valid_k7_config():
|
||||
config = load_material_calculation(EXAMPLE)
|
||||
assert config.id == 'k7-fiber-consumption'
|
||||
assert config.machine_ref == 'c220f95c-a65e-4cb7-99b7-0626d6c7508c'
|
||||
assert config.rate_signal_ref == 'c9d06af5-f6d6-4ede-b6c4-5a98bac77129'
|
||||
assert config.gate_signal_ref == '823867bb-f5d2-40eb-b875-657155addfd0'
|
||||
assert config.gate_threshold == 0.5
|
||||
assert config.max_sample_gap_seconds == 20.0
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field', list(entry()))
|
||||
def test_missing_fields(tmp_path, field):
|
||||
data = entry()
|
||||
del data[field]
|
||||
with pytest.raises(CalculationConfigError, match=f'missing fields: {field}'):
|
||||
load_material_calculation(write_config(tmp_path, [data]))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(('field', 'value'), [
|
||||
('type', 'integration'), ('version', 1), ('version', '2'),
|
||||
('machine_ref', ''), ('rate_signal_ref', ' '), ('gate_signal_ref', None), ('id', ''),
|
||||
('gate_threshold', True), ('gate_threshold', '0.5'), ('gate_threshold', float('nan')),
|
||||
('max_sample_gap_seconds', float('inf')), ('max_sample_gap_seconds', 0),
|
||||
('max_sample_gap_seconds', -1), ('output_unit', 'tonnes'), ('group_by', 'run'),
|
||||
('gate_treshold', 1),
|
||||
])
|
||||
def test_invalid_fields(tmp_path, field, value):
|
||||
data = entry()
|
||||
data[field] = value
|
||||
with pytest.raises(CalculationConfigError):
|
||||
load_material_calculation(write_config(tmp_path, [data]))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('content', [
|
||||
'calculations: [', 'calculations: []', 'calculations: {}', 'other: []',
|
||||
'calculations: []\ncalculations: []', 'calculations:\n - id: x\n id: y',
|
||||
])
|
||||
def test_malformed_documents(tmp_path, content):
|
||||
path = tmp_path / 'bad.yaml'
|
||||
path.write_text(content)
|
||||
with pytest.raises(CalculationConfigError):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_selection(tmp_path):
|
||||
first, second = entry(), dict(entry(), id='second')
|
||||
path = write_config(tmp_path, [first, second])
|
||||
with pytest.raises(CalculationConfigError, match='specify --calculation-id'):
|
||||
load_material_calculation(path)
|
||||
assert load_material_calculation(path, 'second').id == 'second'
|
||||
with pytest.raises(CalculationConfigError, match='not found'):
|
||||
load_material_calculation(path, 'absent')
|
||||
with pytest.raises(CalculationConfigError, match='unique'):
|
||||
load_material_calculation(write_config(tmp_path, [first, first]), first['id'])
|
||||
|
||||
|
||||
def test_repeated_sequential_polls_and_long_cycle():
|
||||
events = []
|
||||
elapsed = 0
|
||||
|
||||
def poll(*, now):
|
||||
nonlocal elapsed
|
||||
assert now == NOW + timedelta(seconds=elapsed)
|
||||
events.append('poll-start')
|
||||
elapsed += 25 # Longer than the interval; no real sleep or concurrent work.
|
||||
events.append('poll-end')
|
||||
|
||||
def sleep(seconds):
|
||||
nonlocal elapsed
|
||||
events.append(('sleep', seconds))
|
||||
elapsed += seconds
|
||||
if elapsed >= 70:
|
||||
raise KeyboardInterrupt
|
||||
|
||||
output = io.StringIO()
|
||||
service = Mock(poll_once=Mock(side_effect=poll))
|
||||
MaterialPollingRunner(
|
||||
service, calculation_id='test-calculation', snapshot_writer=Mock(),
|
||||
machine_id='machine', poll_interval_seconds=10,
|
||||
clock=lambda: NOW + timedelta(seconds=elapsed), sleep=sleep, stdout=output,
|
||||
).run()
|
||||
assert events == ['poll-start', 'poll-end', ('sleep', 10)] * 2
|
||||
assert service.poll_once.call_count == 2
|
||||
assert output.getvalue().count('no open Production Run') == 2
|
||||
assert 'stopped' in output.getvalue()
|
||||
|
||||
|
||||
def test_error_then_success_preserves_opaque_order_and_reports_totals():
|
||||
run = EnlyzeProductionRun('run', 'machine', None, ' 00842/ABC ', NOW, None)
|
||||
result = MaterialPollResult(run, MaterialIntegrationState(12.5, 20))
|
||||
service = Mock(poll_once=Mock(side_effect=[ValueError('SECRET'), result]))
|
||||
output, errors = io.StringIO(), io.StringIO()
|
||||
sleep = Mock(side_effect=[None, KeyboardInterrupt])
|
||||
MaterialPollingRunner(
|
||||
service, calculation_id='test-calculation', snapshot_writer=Mock(),
|
||||
machine_id='machine', poll_interval_seconds=5, clock=lambda: NOW,
|
||||
sleep=sleep, stdout=output, stderr=errors,
|
||||
).run()
|
||||
assert service.poll_once.call_count == 2
|
||||
assert sleep.call_count == 2
|
||||
assert 'gateway/poll cycle: ValueError' in errors.getvalue()
|
||||
assert 'SECRET' not in errors.getvalue()
|
||||
assert "production_order=' 00842/ABC '" in output.getvalue()
|
||||
assert 'consumption_kg=12.500000000 running_seconds=20.000' in output.getvalue()
|
||||
assert NOW.isoformat() in output.getvalue()
|
||||
|
||||
|
||||
def test_interrupt_during_poll():
|
||||
sleep, output = Mock(), io.StringIO()
|
||||
MaterialPollingRunner(
|
||||
Mock(poll_once=Mock(side_effect=KeyboardInterrupt)),
|
||||
calculation_id='test-calculation', snapshot_writer=Mock(), machine_id='m',
|
||||
poll_interval_seconds=1, sleep=sleep, stdout=output,
|
||||
).run()
|
||||
sleep.assert_not_called()
|
||||
assert 'stopped' in output.getvalue()
|
||||
|
||||
|
||||
def test_cycle_configuration_failure_is_fatal():
|
||||
sleep = Mock()
|
||||
runner = MaterialPollingRunner(
|
||||
Mock(poll_once=Mock(side_effect=ConfigurationError('bad settings'))),
|
||||
calculation_id='test-calculation', snapshot_writer=Mock(),
|
||||
machine_id='m', poll_interval_seconds=1, sleep=sleep,
|
||||
)
|
||||
with pytest.raises(ConfigurationError):
|
||||
runner.run()
|
||||
sleep.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('interval', [0, -1, float('nan'), float('inf'), True])
|
||||
def test_invalid_interval(interval):
|
||||
with pytest.raises(CalculationConfigError):
|
||||
MaterialPollingRunner(
|
||||
Mock(), machine_id='m', calculation_id='test', snapshot_writer=Mock(),
|
||||
poll_interval_seconds=interval,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('operation', ['load', 'save'])
|
||||
def test_state_error_classification(operation):
|
||||
store = Mock()
|
||||
getattr(store, operation).side_effect = OSError('SECRET')
|
||||
wrapped = _ReportingStateStore(store)
|
||||
arguments = ('machine', ' 00842 ') if operation == 'load' else ('machine', ' 00842 ', Mock())
|
||||
phase = 'loading' if operation == 'load' else 'saving'
|
||||
with pytest.raises(MaterialStateError, match=f'state {phase}: OSError') as error:
|
||||
getattr(wrapped, operation)(*arguments)
|
||||
assert 'SECRET' not in str(error.value)
|
||||
|
||||
|
||||
def test_runtime_wiring(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv('ENLYZE_BASE_URL', 'https://example.invalid/api/')
|
||||
secrets = tmp_path / 'secret.env'
|
||||
for key, value in dict(HOST='localhost', PORT='5432', DB='analytics',
|
||||
USER='user', PASSWORD='environment-password').items():
|
||||
monkeypatch.setenv(f'POSTGRES_{key}', value)
|
||||
secrets.write_text('ENLYZE_API_KEY=secret-from-file\nPOSTGRES_PASSWORD=file-password\n')
|
||||
with patch('production_analytics.service.material_runtime.MaterialPollingService') as service:
|
||||
runner = build_material_runner(
|
||||
load_material_calculation(EXAMPLE), poll_interval_seconds=7,
|
||||
state_directory=tmp_path / 'state', secrets_file=secrets,
|
||||
)
|
||||
kwargs = service.call_args.kwargs
|
||||
assert kwargs['machine_id'] == entry()['machine_ref']
|
||||
assert kwargs['rate_variable_id'] == entry()['rate_signal_ref']
|
||||
assert kwargs['gate_variable_id'] == entry()['gate_signal_ref']
|
||||
assert kwargs['gate_threshold'] == 0.5
|
||||
assert kwargs['max_sample_gap_seconds'] == 20.0
|
||||
assert kwargs['gateway']._client._settings.api_key == 'secret-from-file'
|
||||
assert kwargs['state_store'].store._directory == tmp_path / 'state'
|
||||
assert runner.service is service.return_value
|
||||
assert runner.interval == 7
|
||||
assert runner.calculation_id == 'k7-fiber-consumption'
|
||||
assert runner.snapshot_writer.settings.password == 'file-password'
|
||||
assert runner.snapshot_writer.settings.dbname == 'analytics'
|
||||
|
||||
|
||||
def test_cli_bad_config(tmp_path, capsys):
|
||||
assert main(['run', 'material-poll', '--config', str(tmp_path / 'missing')]) == 2
|
||||
assert 'Config/startup' in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_cli_startup_and_success(tmp_path, capsys):
|
||||
args = ['run', 'material-poll', '--config', str(EXAMPLE),
|
||||
'--calculation-id', 'k7-fiber-consumption']
|
||||
with patch('production_analytics.service.material_runtime.build_material_runner') as build:
|
||||
assert main(args) == 0
|
||||
build.return_value.run.assert_called_once_with()
|
||||
assert build.call_args.kwargs['state_directory'] == Path('data/state/material')
|
||||
build.side_effect = ConfigurationError('bad settings')
|
||||
assert main(args) == 2
|
||||
assert 'bad settings' in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_failed_cycle_keeps_checkpoint_and_recovers(tmp_path):
|
||||
from production_analytics.calculations.material_consumption import MaterialSample
|
||||
from production_analytics.service.material_polling import (
|
||||
MaterialPollingService,
|
||||
MaterialPollingState,
|
||||
)
|
||||
from production_analytics.service.material_state_store import JsonMaterialStateStore
|
||||
|
||||
order = ' 00842 '
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
store.save('machine', order, MaterialPollingState('run', MaterialIntegrationState(
|
||||
cumulative_consumption_kg=1, integrated_running_seconds=10,
|
||||
last_processed_timestamp=NOW, last_material_rate_kg_per_hour=3600,
|
||||
last_gate_value=1, integration_active=True,
|
||||
)))
|
||||
checkpoint = next(tmp_path.glob('*.json'))
|
||||
original = checkpoint.read_bytes()
|
||||
gateway = Mock()
|
||||
gateway.get_open_production_run.return_value = EnlyzeProductionRun(
|
||||
'run', 'machine', None, order, NOW, None,
|
||||
)
|
||||
gateway.get_material_samples.side_effect = [
|
||||
ValueError('bad response'), [MaterialSample(NOW + timedelta(seconds=10), 3600, 1)],
|
||||
]
|
||||
service = MaterialPollingService(
|
||||
gateway=gateway, state_store=_ReportingStateStore(store), machine_id='machine',
|
||||
rate_variable_id='rate', gate_variable_id='gate', gate_threshold=.5,
|
||||
max_sample_gap_seconds=20,
|
||||
)
|
||||
sleeps = 0
|
||||
|
||||
def sleep(seconds):
|
||||
nonlocal sleeps
|
||||
sleeps += 1
|
||||
if sleeps == 1:
|
||||
assert checkpoint.read_bytes() == original
|
||||
else:
|
||||
raise KeyboardInterrupt
|
||||
|
||||
MaterialPollingRunner(
|
||||
service, calculation_id='test-calculation', snapshot_writer=Mock(),
|
||||
machine_id='machine', poll_interval_seconds=1,
|
||||
clock=lambda: NOW + timedelta(seconds=10), sleep=sleep,
|
||||
stdout=io.StringIO(), stderr=io.StringIO(),
|
||||
).run()
|
||||
restored = store.load('machine', order)
|
||||
assert restored.integration_state.cumulative_consumption_kg == 11
|
||||
assert restored.integration_state.integrated_running_seconds == 20
|
||||
|
||||
|
||||
def test_unwritable_state_location_is_startup_failure(tmp_path, monkeypatch, capsys):
|
||||
monkeypatch.setenv('ENLYZE_BASE_URL', 'https://example.invalid/api/')
|
||||
blocked = tmp_path / 'file'
|
||||
blocked.write_text('not a directory')
|
||||
assert main([
|
||||
'run', 'material-poll', '--config', str(EXAMPLE),
|
||||
'--state-directory', str(blocked), '--secrets-file', str(tmp_path / 'absent'),
|
||||
]) == 2
|
||||
assert 'State directory startup check failed' in capsys.readouterr().err
|
||||
@@ -0,0 +1,101 @@
|
||||
import sqlite3
|
||||
import sys
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.service.postgres_material import (
|
||||
PostgresMaterialSnapshotRepository,
|
||||
PostgresSettings,
|
||||
)
|
||||
|
||||
START = datetime(2026, 9, 4, 10, tzinfo=UTC)
|
||||
ORDER = " K 7-123'; -- "
|
||||
SETTINGS = PostgresSettings('localhost', 5432, 'analytics', 'reader', 'secret')
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def database():
|
||||
# Execute the actual portable SELECT in SQLite; only adapt driver placeholders
|
||||
# and datetime transport. This tests SQL semantics without a live PostgreSQL server.
|
||||
with sqlite3.connect(':memory:') as database:
|
||||
database.execute('''CREATE TABLE material_consumption_snapshots (
|
||||
timestamp TEXT, calculation_id TEXT, machine_id TEXT, production_order TEXT,
|
||||
run_id TEXT, consumption_kg REAL
|
||||
)''')
|
||||
rows = [
|
||||
(START.isoformat(), 'calc', 'machine', ORDER, 'run1', 1000),
|
||||
(START.replace(minute=10).isoformat(), 'calc', 'machine', ORDER, 'run2', 1100),
|
||||
(START.replace(minute=4).isoformat(), 'other', 'machine', ORDER, 'run', 9999),
|
||||
(START.replace(minute=4).isoformat(), 'calc', 'other', ORDER, 'run', 9999),
|
||||
(START.replace(minute=4).isoformat(), 'calc', 'machine', ORDER.strip(), 'run', 9999),
|
||||
(START.replace(minute=4).isoformat(), 'calc', 'machine', ORDER + '-other', 'run', 9999),
|
||||
]
|
||||
database.executemany(
|
||||
'INSERT INTO material_consumption_snapshots VALUES (?,?,?,?,?,?)', rows,
|
||||
)
|
||||
yield database
|
||||
|
||||
|
||||
@pytest.mark.parametrize('minute, expected_minute, consumption', [
|
||||
(-1, None, None), (0, 0, 1000), (5, 0, 1000), (10, 10, 1100), (15, 10, 1100),
|
||||
])
|
||||
def test_aligned_lookup_sql(database, minute, expected_minute, consumption):
|
||||
cutoff = START.replace(minute=minute) if minute >= 0 else START.replace(hour=9, minute=59)
|
||||
driver = MagicMock()
|
||||
connection = driver.connect.return_value.__enter__.return_value
|
||||
|
||||
def execute(sql, parameters):
|
||||
assert parameters == ('calc', 'machine', ORDER, cutoff)
|
||||
assert sql.count('%s') == 4
|
||||
assert ORDER not in sql
|
||||
assert 'ORDER BY timestamp DESC' in sql
|
||||
assert 'LIMIT 1' in sql
|
||||
row = database.execute(
|
||||
sql.replace('%s', '?'), (*parameters[:3], parameters[3].isoformat()),
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
row = (datetime.fromisoformat(row[0]), *row[1:])
|
||||
return MagicMock(fetchone=MagicMock(return_value=row))
|
||||
|
||||
connection.execute.side_effect = execute
|
||||
with patch.dict(sys.modules, psycopg=driver):
|
||||
result = PostgresMaterialSnapshotRepository(SETTINGS).latest_at_or_before(
|
||||
calculation_id='calc', machine_id='machine', production_order=ORDER, timestamp=cutoff,
|
||||
)
|
||||
if expected_minute is None:
|
||||
assert result is None
|
||||
else:
|
||||
assert result.timestamp == START.replace(minute=expected_minute)
|
||||
assert result.timestamp <= cutoff
|
||||
assert result.consumption_kg == consumption
|
||||
assert (result.calculation_id, result.machine_id, result.production_order) == (
|
||||
'calc', 'machine', ORDER,
|
||||
)
|
||||
assert result.run_id == ('run1' if expected_minute == 0 else 'run2')
|
||||
connection.execute.assert_called_once()
|
||||
driver.connect.assert_called_once_with(
|
||||
host='localhost', port=5432, dbname='analytics', user='reader', password='secret',
|
||||
connect_timeout=10, options='-c statement_timeout=10000',
|
||||
)
|
||||
driver.connect.return_value.__exit__.assert_called_once_with(None, None, None)
|
||||
|
||||
|
||||
def test_naive_cutoff_rejected_before_connection():
|
||||
driver = MagicMock()
|
||||
with patch.dict(sys.modules, psycopg=driver), pytest.raises(ValueError, match='timezone-aware'):
|
||||
PostgresMaterialSnapshotRepository(SETTINGS).latest_at_or_before(
|
||||
calculation_id='calc', machine_id='machine', production_order=ORDER,
|
||||
timestamp=START.replace(tzinfo=None),
|
||||
)
|
||||
driver.connect.assert_not_called()
|
||||
|
||||
|
||||
def test_database_failure_propagates():
|
||||
driver = MagicMock()
|
||||
driver.connect.side_effect = RuntimeError('connection unavailable')
|
||||
with patch.dict(sys.modules, psycopg=driver), pytest.raises(RuntimeError):
|
||||
PostgresMaterialSnapshotRepository(SETTINGS).latest_at_or_before(
|
||||
calculation_id='calc', machine_id='machine', production_order=ORDER, timestamp=START,
|
||||
)
|
||||
@@ -0,0 +1,141 @@
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialIntegrationState,
|
||||
)
|
||||
from production_analytics.service.material_polling import MaterialPollingState
|
||||
from production_analytics.service.material_state_store import JsonMaterialStateStore
|
||||
|
||||
|
||||
def test_json_state_store_returns_none_for_missing_state(tmp_path) -> None:
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
|
||||
assert store.load("machine-1", "ORDER-1") is None
|
||||
|
||||
|
||||
def test_json_state_store_roundtrip(tmp_path) -> None:
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
|
||||
original = MaterialPollingState(
|
||||
run_id="run-1",
|
||||
integration_state=MaterialIntegrationState(
|
||||
cumulative_consumption_kg=123.4,
|
||||
integrated_running_seconds=456.0,
|
||||
last_processed_timestamp=datetime(
|
||||
2026, 9, 3, 5, 10, 50, tzinfo=UTC
|
||||
),
|
||||
last_material_rate_kg_per_hour=1028.5,
|
||||
last_gate_value=2.1,
|
||||
integration_active=True,
|
||||
),
|
||||
)
|
||||
|
||||
store.save("machine-1", "ORDER-1", original)
|
||||
restored = store.load("machine-1", "ORDER-1")
|
||||
|
||||
assert restored == original
|
||||
|
||||
|
||||
def test_json_state_store_separates_orders(tmp_path) -> None:
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
|
||||
state_a = MaterialPollingState(
|
||||
run_id="run-a",
|
||||
integration_state=MaterialIntegrationState(
|
||||
cumulative_consumption_kg=10.0,
|
||||
),
|
||||
)
|
||||
state_b = MaterialPollingState(
|
||||
run_id="run-b",
|
||||
integration_state=MaterialIntegrationState(
|
||||
cumulative_consumption_kg=20.0,
|
||||
),
|
||||
)
|
||||
|
||||
store.save("machine-1", "ORDER-1", state_a)
|
||||
store.save("machine-1", "ORDER-2", state_b)
|
||||
|
||||
assert store.load("machine-1", "ORDER-1") == state_a
|
||||
assert store.load("machine-1", "ORDER-2") == state_b
|
||||
|
||||
|
||||
def test_identifiers_are_isolated_safe_and_deterministic(tmp_path) -> None:
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
pairs = [("a/b", "x"), ("a_b", "x"), ("a", "b__x"), ("a__b", "x"),
|
||||
("../..", "/tmp/escape"), ("", ""), ("a/b", "y")]
|
||||
for index, pair in enumerate(pairs):
|
||||
store.save(*pair, MaterialPollingState(str(index), MaterialIntegrationState()))
|
||||
assert len(list(tmp_path.iterdir())) == len(pairs)
|
||||
for index, pair in enumerate(pairs):
|
||||
assert JsonMaterialStateStore(tmp_path).load(*pair).run_id == str(index)
|
||||
assert all(path.parent == tmp_path and len(path.name) == 69 for path in tmp_path.iterdir())
|
||||
|
||||
|
||||
def test_failed_write_preserves_primary_and_cleans_temporary_file(tmp_path, monkeypatch) -> None:
|
||||
import production_analytics.service.material_state_store as module
|
||||
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
state = MaterialPollingState("old", MaterialIntegrationState())
|
||||
store.save("machine", "order", state)
|
||||
path = next(tmp_path.iterdir())
|
||||
original = path.read_bytes()
|
||||
|
||||
def fail_dump(payload, file, **kwargs):
|
||||
file.write('{"partial":')
|
||||
raise OSError("disk full")
|
||||
|
||||
monkeypatch.setattr(module.json, "dump", fail_dump)
|
||||
with pytest.raises(OSError, match="disk full"):
|
||||
store.save("machine", "order", MaterialPollingState("new", MaterialIntegrationState()))
|
||||
assert path.read_bytes() == original
|
||||
assert list(tmp_path.iterdir()) == [path]
|
||||
|
||||
|
||||
def test_replace_failure_preserves_primary(tmp_path, monkeypatch) -> None:
|
||||
import production_analytics.service.material_state_store as module
|
||||
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
state = MaterialPollingState("old", MaterialIntegrationState())
|
||||
store.save("m", "o", state)
|
||||
path = next(tmp_path.iterdir())
|
||||
|
||||
def fail_replace(source, target):
|
||||
assert source.parent == target.parent == tmp_path
|
||||
assert store.load("m", "o") == state
|
||||
raise OSError("replace failed")
|
||||
|
||||
monkeypatch.setattr(module.os, "replace", fail_replace)
|
||||
with pytest.raises(OSError, match="replace failed"):
|
||||
store.save("m", "o", MaterialPollingState("new", MaterialIntegrationState()))
|
||||
assert store.load("m", "o") == state
|
||||
assert list(tmp_path.iterdir()) == [path]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("contents", ['{', '[]', '{}', '{"run_id": null}',
|
||||
'{"run_id": "r", "integration_state": {}}'])
|
||||
def test_corrupt_state_fails_clearly(tmp_path, contents) -> None:
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
store.save("m", "o", MaterialPollingState("r", MaterialIntegrationState()))
|
||||
next(tmp_path.iterdir()).write_text(contents)
|
||||
with pytest.raises(ValueError, match="Invalid material polling state"):
|
||||
store.load("m", "o")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field,value", [
|
||||
("integration_active", "false"), ("cumulative_consumption_kg", float("nan")),
|
||||
("integrated_running_seconds", -1), ("last_processed_timestamp", "2026-09-03T05:00:00"),
|
||||
("integration_active", True), ("last_gate_value", True),
|
||||
])
|
||||
def test_malformed_integration_state_fails(tmp_path, field, value) -> None:
|
||||
import json
|
||||
|
||||
store = JsonMaterialStateStore(tmp_path)
|
||||
store.save("m", "o", MaterialPollingState("r", MaterialIntegrationState()))
|
||||
path = next(tmp_path.iterdir())
|
||||
payload = json.loads(path.read_text())
|
||||
payload["integration_state"][field] = value
|
||||
path.write_text(json.dumps(payload))
|
||||
with pytest.raises(ValueError, match="Invalid material polling state"):
|
||||
store.load("m", "o")
|
||||
@@ -0,0 +1,128 @@
|
||||
import io
|
||||
import sys
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import MagicMock, Mock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.calculations.material_consumption import MaterialIntegrationState
|
||||
from production_analytics.enlyze.exploration import ConfigurationError
|
||||
from production_analytics.enlyze.gateway import EnlyzeProductionRun
|
||||
from production_analytics.service.material_polling import MaterialPollResult
|
||||
from production_analytics.service.material_runner import MaterialPollingRunner
|
||||
from production_analytics.service.postgres_material import (
|
||||
PostgresMaterialSnapshotWriter,
|
||||
PostgresSettings,
|
||||
)
|
||||
|
||||
ENV = dict(POSTGRES_HOST='localhost', POSTGRES_PORT='5432', POSTGRES_DB='analytics',
|
||||
POSTGRES_USER='writer', POSTGRES_PASSWORD='secret-password')
|
||||
NOW = datetime(2026, 1, 1, tzinfo=UTC)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('key', ENV)
|
||||
@pytest.mark.parametrize('value', [None, '', ' ', '\x00'])
|
||||
def test_required_settings(key, value):
|
||||
environment = dict(ENV)
|
||||
if value is None:
|
||||
del environment[key]
|
||||
else:
|
||||
environment[key] = value
|
||||
with pytest.raises(ConfigurationError, match=key) as error:
|
||||
PostgresSettings.from_environment(environment)
|
||||
assert 'secret-password' not in str(error.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('port', ['0', '-1', '65536', '1.5', 'secret-password'])
|
||||
def test_invalid_port(port):
|
||||
with pytest.raises(ConfigurationError, match='POSTGRES_PORT') as error:
|
||||
PostgresSettings.from_environment(dict(ENV, POSTGRES_PORT=port))
|
||||
assert str(error.value) == 'POSTGRES_PORT must be an integer from 1 to 65535'
|
||||
|
||||
|
||||
def test_valid_settings_and_password_repr():
|
||||
settings = PostgresSettings.from_environment(ENV)
|
||||
assert (settings.host, settings.port, settings.dbname, settings.user) == (
|
||||
'localhost', 5432, 'analytics', 'writer',
|
||||
)
|
||||
assert settings.password == 'secret-password'
|
||||
assert settings.password not in repr(settings)
|
||||
|
||||
|
||||
def test_parameterized_insert_and_connection_lifecycle():
|
||||
driver = MagicMock()
|
||||
connection = driver.connect.return_value.__enter__.return_value
|
||||
writer = PostgresMaterialSnapshotWriter(PostgresSettings.from_environment(ENV))
|
||||
fields = dict(timestamp=NOW, calculation_id='calc', machine_id='machine',
|
||||
production_order=" 00842'; DROP TABLE x; -- ", run_id='run', consumption_kg=12.5)
|
||||
with patch.dict(sys.modules, psycopg=driver):
|
||||
writer.write(**fields)
|
||||
sql, parameters = connection.execute.call_args.args
|
||||
assert sql.count('%s') == 6
|
||||
assert fields['production_order'] not in sql
|
||||
assert parameters == tuple(fields.values())
|
||||
assert 'ON CONFLICT' in sql
|
||||
connection.execute.assert_called_once()
|
||||
driver.connect.assert_called_once_with(
|
||||
host='localhost', port=5432, dbname='analytics', user='writer',
|
||||
password='secret-password', connect_timeout=10, options='-c statement_timeout=10000',
|
||||
)
|
||||
driver.connect.return_value.__exit__.assert_called_once_with(None, None, None)
|
||||
|
||||
|
||||
def test_writer_failure_propagates_and_next_write_reconnects():
|
||||
driver = MagicMock()
|
||||
driver.connect.return_value.__enter__.return_value.execute.side_effect = [
|
||||
RuntimeError('secret'), None,
|
||||
]
|
||||
writer = PostgresMaterialSnapshotWriter(PostgresSettings.from_environment(ENV))
|
||||
fields = dict(timestamp=NOW, calculation_id='calc', machine_id='machine',
|
||||
production_order='order', run_id='run', consumption_kg=0)
|
||||
with patch.dict(sys.modules, psycopg=driver):
|
||||
with pytest.raises(RuntimeError):
|
||||
writer.write(**fields)
|
||||
writer.write(**fields)
|
||||
assert driver.connect.call_count == 2
|
||||
assert driver.connect.return_value.__exit__.call_count == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize('consumption', [0, 12.5])
|
||||
def test_runner_snapshot_fields_and_none(consumption):
|
||||
result = MaterialPollResult(
|
||||
EnlyzeProductionRun('run', 'machine', None, ' 00842 ', NOW, None),
|
||||
MaterialIntegrationState(consumption, 20),
|
||||
)
|
||||
writer = Mock()
|
||||
output = io.StringIO()
|
||||
MaterialPollingRunner(
|
||||
Mock(poll_once=Mock(side_effect=[None, result])), machine_id='machine',
|
||||
calculation_id='configured-calculation', snapshot_writer=writer,
|
||||
poll_interval_seconds=10, clock=lambda: NOW,
|
||||
sleep=Mock(side_effect=[None, KeyboardInterrupt]), stdout=output,
|
||||
).run()
|
||||
writer.write.assert_called_once_with(
|
||||
timestamp=NOW, calculation_id='configured-calculation', machine_id='machine',
|
||||
production_order=' 00842 ', run_id='run', consumption_kg=consumption,
|
||||
)
|
||||
assert 'no open Production Run' in output.getvalue()
|
||||
assert f'consumption_kg={consumption:.9f}' in output.getvalue()
|
||||
|
||||
|
||||
def test_snapshot_failure_reports_safely_and_continues():
|
||||
result = MaterialPollResult(
|
||||
EnlyzeProductionRun('run', 'machine', None, 'order', NOW, None),
|
||||
MaterialIntegrationState(12.5, 20),
|
||||
)
|
||||
service = Mock(poll_once=Mock(return_value=result))
|
||||
writer = Mock(write=Mock(side_effect=[RuntimeError('secret-password arbitrary SQL'), None]))
|
||||
output, errors = io.StringIO(), io.StringIO()
|
||||
MaterialPollingRunner(
|
||||
service, machine_id='machine', calculation_id='calc', snapshot_writer=writer,
|
||||
poll_interval_seconds=10, clock=lambda: NOW,
|
||||
sleep=Mock(side_effect=[None, KeyboardInterrupt]), stdout=output, stderr=errors,
|
||||
).run()
|
||||
assert service.poll_once.call_count == writer.write.call_count == 2
|
||||
assert errors.getvalue().count('PostgreSQL snapshot write failed') == 1
|
||||
assert 'secret-password' not in errors.getvalue()
|
||||
assert 'arbitrary SQL' not in errors.getvalue()
|
||||
assert output.getvalue().count('consumption_kg=12.500000000') == 2
|
||||
@@ -0,0 +1,44 @@
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from production_analytics.power_meter.config import load_power_meter_config
|
||||
from production_analytics.power_meter.gateway import PowerMeterGateway
|
||||
from production_analytics.power_meter.repository import counter_delta
|
||||
|
||||
|
||||
def test_b2_config_has_only_verified_signals() -> None:
|
||||
config = load_power_meter_config("config/power-meters.example.yaml", "B2")
|
||||
assert config.machine_uuid == "0d7955e9-5cde-4e14-9af7-5b041dc796f0"
|
||||
assert config.signals.energy_total == "0f5c0853-1aa7-4fa3-b267-ea2669024e46"
|
||||
assert config.signals.power_outages is None
|
||||
|
||||
|
||||
def test_counter_reset_starts_new_epoch() -> None:
|
||||
assert counter_delta(None, 7) == 0
|
||||
assert counter_delta(100, 105) == 5
|
||||
assert counter_delta(100, 4) == 4
|
||||
|
||||
|
||||
def test_gateway_keeps_optional_phases_and_handles_pagination() -> None:
|
||||
config = load_meter()
|
||||
client = Mock()
|
||||
client.post_json.side_effect = [
|
||||
Mock(body={"data": {"columns": ["time", "e", "p", "f", "l1"],
|
||||
"records": [["2026-09-01T00:00:00Z", 10, 4, 50, 2]]},
|
||||
"metadata": {"next_cursor": "next"}}),
|
||||
Mock(body={"data": {"columns": ["time", "e", "p", "f", "l1"],
|
||||
"records": [["2026-09-01T00:01:00Z", 11, 5, 49.9, 3]]},
|
||||
"metadata": {"next_cursor": None}}),
|
||||
]
|
||||
readings = PowerMeterGateway(client).read(config, datetime(2026, 9, 1, tzinfo=UTC),
|
||||
datetime(2026, 9, 2, tzinfo=UTC))
|
||||
assert [item.energy_total_kwh for item in readings] == [10, 11]
|
||||
assert readings[0].phase_active_power_kw == (("L1", 2.0),)
|
||||
assert client.post_json.call_args_list[1].args[1]["cursor"] == "next"
|
||||
|
||||
|
||||
def load_meter():
|
||||
from production_analytics.power_meter.config import PowerMeterConfig, PowerMeterSignals
|
||||
return PowerMeterConfig("B2", "machine", PowerMeterSignals("e", "p", "f", phase_active_power=(("L1", "l1"),)))
|
||||
@@ -0,0 +1,122 @@
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from production_analytics.calculations.config import (
|
||||
CalculationConfigError,
|
||||
load_material_calculation,
|
||||
)
|
||||
from production_analytics.calculations.material_consumption import (
|
||||
MaterialConsumptionIntegrator,
|
||||
MaterialIntegratorConfig,
|
||||
rotational_discharge_rate_kg_per_hour,
|
||||
)
|
||||
from production_analytics.enlyze.gateway import EnlyzeApiGateway
|
||||
from production_analytics.service.material_runtime import build_material_runner
|
||||
|
||||
|
||||
@pytest.mark.parametrize('speeds,width,factor,expected', [
|
||||
([3.739, 3.956], 5, 3.12, 7202.52),
|
||||
([3.739, 3.956], 4.85, 3.12, 6986.4444),
|
||||
([2], 4, 1.5, 720),
|
||||
([2, 3, 4], 4, 1.5, 3240),
|
||||
])
|
||||
def test_conversion(speeds, width, factor, expected):
|
||||
assert rotational_discharge_rate_kg_per_hour(speeds, width, factor) == pytest.approx(expected)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('speeds,width,factor', [
|
||||
([], 5, 3.12), ([float('nan')], 5, 3.12), ([1], 0, 3.12),
|
||||
([1], float('inf'), 3.12), ([1], 5, 0), ([1], 5, float('nan')),
|
||||
([1e308], 5, 3.12),
|
||||
])
|
||||
def test_invalid_conversion(speeds, width, factor):
|
||||
with pytest.raises(ValueError):
|
||||
rotational_discharge_rate_kg_per_hour(speeds, width, factor)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('gate_id', ['speed', None])
|
||||
def test_gate_and_gap(gate_id):
|
||||
start = datetime(2026, 8, 25, tzinfo=UTC)
|
||||
times = [0, 10, 20, 40, 61, 71, 81]
|
||||
gates = [2, 0.3, 0, 8, 1, 1, 1]
|
||||
client = Mock()
|
||||
client.post_json.return_value.body = {'data': {
|
||||
'columns': ['time', 'right', 'left'] + ([gate_id] if gate_id else []),
|
||||
'records': [[(start + timedelta(seconds=t)).isoformat(), 3.739, 3.956]
|
||||
+ ([gate] if gate_id else []) for t, gate in zip(times, gates, strict=True)],
|
||||
}}
|
||||
samples = EnlyzeApiGateway(client).get_material_samples(
|
||||
machine_id='m', rate_variable_id='unused', gate_variable_id=gate_id,
|
||||
rotational_speed_variable_ids=('right', 'left'), nominal_width_m=5,
|
||||
specific_discharge_kg_per_rev_m=3.12, start=start, end=start + timedelta(seconds=81),
|
||||
)
|
||||
assert all(s.material_rate_kg_per_hour == pytest.approx(7202.52) for s in samples)
|
||||
state = MaterialConsumptionIntegrator(MaterialIntegratorConfig(0.3, 20)).process_many(samples)
|
||||
seconds = 30 if gate_id else 60
|
||||
assert state.integrated_running_seconds == seconds
|
||||
assert state.cumulative_consumption_kg == pytest.approx(120.042 * seconds / 60)
|
||||
assert client.post_json.call_args.args[1]['variables'] == [
|
||||
{'uuid': ref} for ref in ['right', 'left'] + ([gate_id] if gate_id else [])]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [
|
||||
('specific_discharge_kg_per_rev_m', None), ('specific_discharge_kg_per_rev_m', 0),
|
||||
('specific_discharge_kg_per_rev_m', True), ('specific_discharge_kg_per_rev_m', float('inf')),
|
||||
('application_signal_refs', ['set']), ('rotational_speed_signal_refs', ['']),
|
||||
])
|
||||
def test_invalid_config(tmp_path, field, value):
|
||||
document = yaml.safe_load(open('config/bento1-material-consumption.yaml'))
|
||||
document['calculations'][0][field] = value
|
||||
path = tmp_path / 'config.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
with pytest.raises(CalculationConfigError):
|
||||
load_material_calculation(path)
|
||||
|
||||
|
||||
def test_optional_gate_config(tmp_path):
|
||||
document = yaml.safe_load(open('config/bento1-material-consumption.yaml'))
|
||||
del document['calculations'][0]['process_application_calculation_id']
|
||||
del document['calculations'][0]['gate_signal_ref']
|
||||
del document['calculations'][0]['gate_threshold']
|
||||
path = tmp_path / 'config.yaml'
|
||||
path.write_text(yaml.safe_dump(document))
|
||||
(tmp_path / 'material-calibrations.yaml').write_text(
|
||||
Path('config/material-calibrations.yaml').read_text())
|
||||
config = load_material_calculation(path)
|
||||
assert config.gate_signal_ref is None
|
||||
assert config.gate_threshold == 0
|
||||
|
||||
|
||||
def test_bento_runtime_wiring(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv('ENLYZE_BASE_URL', 'https://example.invalid/api/')
|
||||
for key, value in dict(HOST='localhost', PORT='5432', DB='analytics',
|
||||
USER='user', PASSWORD='test').items():
|
||||
monkeypatch.setenv(f'POSTGRES_{key}', value)
|
||||
secrets = tmp_path / 'secret.env'
|
||||
secrets.write_text('ENLYZE_API_KEY=test\n')
|
||||
config = load_material_calculation('config/bento1-material-consumption.yaml')
|
||||
with patch('production_analytics.service.material_runtime.ErpSettings') as erp_settings:
|
||||
runner = build_material_runner(
|
||||
config, poll_interval_seconds=10, state_directory=tmp_path / 'state',
|
||||
secrets_file=secrets, erp_secrets_file=tmp_path / 'erp.env',
|
||||
)
|
||||
service = runner.service
|
||||
assert config.source_mode == 'rotational_discharge'
|
||||
assert service._process_application_calculation_id == 'bento1-fresh-bentonite-application'
|
||||
from production_analytics.service.postgres_material_application import (
|
||||
PostgresMaterialApplicationWriter,
|
||||
)
|
||||
assert isinstance(service._application_writer, PostgresMaterialApplicationWriter)
|
||||
assert service._rotational_speed_variable_ids == (
|
||||
'6e5d2d94-98f9-4cc1-8a88-7987c6282525', 'cd7385c4-337b-4759-ab32-45d65beaf190',
|
||||
)
|
||||
assert service._application_variable_ids == ()
|
||||
assert service._specific_discharge == 2.75
|
||||
assert service._gate_variable_id == 'fef41976-1103-4090-b780-eaeecc02fdfa'
|
||||
assert service._config == MaterialIntegratorConfig(0.3, 20)
|
||||
assert service._nominal_width_provider.workplace == 'Bento 1'
|
||||
erp_settings.from_secret_file.assert_called_once_with(tmp_path / 'erp.env')
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Regression coverage for SQLite initialization from production DDL."""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_schema_initialization_and_power_meter_json(sqlite_schema):
|
||||
with sqlite3.connect(":memory:") as database:
|
||||
database.executescript(sqlite_schema)
|
||||
database.executescript(sqlite_schema)
|
||||
tables = {
|
||||
row[0] for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
}
|
||||
assert {
|
||||
"material_consumption_snapshots",
|
||||
"material_efficiency_snapshots",
|
||||
"material_application_snapshots",
|
||||
"channel_material_consumption_snapshots",
|
||||
"production_downtime_events",
|
||||
"production_order_attribution_state",
|
||||
"power_meter_readings",
|
||||
"power_meter_monthly_reports",
|
||||
} <= tables
|
||||
|
||||
insert = """INSERT INTO power_meter_readings
|
||||
(meter, timestamp, energy_total_kwh, energy_delta_kwh,
|
||||
active_power_total_kw, grid_frequency_hz)
|
||||
VALUES (?, ?, 100, 1, 4, 50)"""
|
||||
database.execute(insert, ("B2", "2026-09-01T00:00:00Z"))
|
||||
phases = {"L1": 1.25, "L2": 2.75}
|
||||
database.execute(insert, ("B2", "2026-09-01T00:01:00Z"))
|
||||
database.execute(
|
||||
"UPDATE power_meter_readings SET phase_active_power_kw=? WHERE timestamp=?",
|
||||
(json.dumps(phases), "2026-09-01T00:01:00Z"),
|
||||
)
|
||||
rows = database.execute(
|
||||
"SELECT phase_active_power_kw FROM power_meter_readings ORDER BY timestamp"
|
||||
).fetchall()
|
||||
assert [json.loads(row[0]) for row in rows] == [{}, phases]
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
database.execute(insert, ("B2", "2026-09-01T00:00:00Z"))
|
||||
|
||||
summary = {"consumption_kwh": 42.5, "first_timestamp": None, "outages": 0}
|
||||
database.execute(
|
||||
"""INSERT INTO power_meter_monthly_reports (meter, month, report_path, summary)
|
||||
VALUES (?, ?, ?, ?)""",
|
||||
("B2", "2026-09", "/reports/B2.pdf", json.dumps(summary)),
|
||||
)
|
||||
path, stored_summary, generated_at = database.execute(
|
||||
"SELECT report_path, summary, generated_at FROM power_meter_monthly_reports"
|
||||
).fetchone()
|
||||
assert path == "/reports/B2.pdf"
|
||||
assert json.loads(stored_summary) == summary
|
||||
assert generated_at is not None
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
database.execute(
|
||||
"""INSERT INTO power_meter_monthly_reports (meter, month, report_path, summary)
|
||||
VALUES ('B2', '2026-10', '/reports/B2.pdf', NULL)"""
|
||||
)
|
||||
Reference in New Issue
Block a user