From 3b2f51e6b80eafbdbad3ae08dc110e979dbb84c0 Mon Sep 17 00:00:00 2001 From: Matt Date: Thu, 11 Jun 2026 14:55:33 -0400 Subject: [PATCH] Forecast improvements --- FORECAST_FIX_PLAN.md | 343 +++++++++++ .../forecast_engine.cpython-312.pyc | Bin 0 -> 81621 bytes .../scripts/forecast/forecast_engine.py | 544 ++++++++++++++---- inventory-server/src/routes/dashboard.js | 57 +- .../components/overview/ForecastAccuracy.tsx | 81 ++- 5 files changed, 887 insertions(+), 138 deletions(-) create mode 100644 FORECAST_FIX_PLAN.md create mode 100644 inventory-server/scripts/forecast/__pycache__/forecast_engine.cpython-312.pyc diff --git a/FORECAST_FIX_PLAN.md b/FORECAST_FIX_PLAN.md new file mode 100644 index 0000000..95e1f09 --- /dev/null +++ b/FORECAST_FIX_PLAN.md @@ -0,0 +1,343 @@ +# Forecast Accuracy Fix Plan + +**Written:** 2026-06-10, from a code + live-data review of the forecasting pipeline. +**Goal:** eliminate the systematic ~1.7–2x over-forecast bias, recover demand the model currently ignores, and fix the accuracy measurement so improvements are visible and long-lead forecasts are validated. + +Read this whole document before starting. Fixes are grouped into phases; each phase is independently deployable and has its own validation step. Line numbers are as of 2026-06-10 — re-locate by function name if the file has drifted. + +--- + +## 1. Diagnosis summary (measured 2026-06-10) + +The dashboard headline is **202% WMAPE**. Decomposition of that number, all measured against `forecast_accuracy` run 129 and ad-hoc queries: + +| Finding | Evidence | +|---|---| +| Daily-grain WMAPE has a ~190% *floor* for this catalog | Avg demand ≈ 0.11 units/product/day. A perfect rate forecast of intermittent demand scores ≈ 2e^−λ ≈ 190%. A trivial trailing-30d-average naive forecast scores **204%** on the same products/days; the engine scores 221% (slightly *worse than naive*). | +| Same forecasts at 21-day-per-product grain: **109%**; bias-corrected: **75%** | Half the headline is metric grain, most of the rest is bias. | +| Aggregate over-forecast **+70%** (227,690 forecast vs 133,861 actual units) | Portfolio daily ratio is 1.5–2.5x on most days. | +| Decay phase 2.47x over (fc 51,675 / act 20,915) | Root cause F1: velocity inflated **4.07x** (measured: 1.353 vs true 0.332 units/day) by averaging over sparse snapshot rows. | +| Preorder phase 2.15x over (fc 67,212 / act 31,189) | Root cause F4: launch curve applied at age=0 starting *today*, ignoring that the product hasn't arrived. | +| Mature phase 1.69x over (fc 57,857 / act 34,313) | Root causes F2 (history edge truncation) + F3 (seasonal double-count). | +| Dormant products sold **16,180 units** (~11% of demand) against zero forecasts | Root cause F5; also excluded from the headline metric, so invisible. | +| All 879,800 accuracy samples are in the **1–7d lead bucket** | Root cause F7: archiving design only ever saves yesterday's slice. 30–90d forecasts (what purchasing uses) are never validated. | +| Launch phase is healthy: WMAPE 100%, bias −6%, beats naive | The lifecycle-curve concept works; its calibration inputs are broken. Don't redesign it. | + +**Key data fact** underlying several fixes: `daily_product_snapshots` is **activity-based and sparse** — only ~500–1,800 of ~38K products have a row on a given day. Verified: every pid-day with an order DOES have a snapshot row and units match (5,234/5,234 pid-days, 8,980 vs 8,984 units over 7 days). So *missing row = zero sales*, and any query that aggregates over only the rows that exist is averaging over sold-days. + +--- + +## 2. Environment & operational notes + +- **Files:** engine is `inventory-server/scripts/forecast/forecast_engine.py`; orchestrator `run_forecast.js` in the same dir; consumer endpoints in `inventory-server/src/routes/dashboard.js` (`/forecast/metrics` ~line 308, `/forecast/accuracy` ~line 647); overview UI in `inventory/src/components/overview/ForecastMetrics.tsx` and `ForecastAccuracy.tsx`. +- **Local `inventory-server/` is NFS-mounted to `/var/www/inventory/` on the netcup server.** Edits made locally appear on the server immediately — no copy step. Do NOT run bulk `grep`/`find`/`node --check` over `inventory-server/` locally (the mount hangs); `ssh netcup` and run them there. +- **Avoid the glob tool** for search in this repo; use bash (`grep`/`rg` via ssh for server-side trees). +- **Scheduling:** the engine runs daily at **09:30:01 server time** (runs table is conclusive), but the cron entry is NOT in matt's crontab, `/etc/cron.d`, or pm2. Likely root's crontab (`sudo crontab -l` to confirm). You do not need to touch the schedule for these fixes; just know a run fires at 09:30 daily and occasionally skips days (e.g. 2026-06-07/08). +- **Manual test runs:** `ssh netcup`, then `cd /var/www/inventory/scripts/forecast && node run_forecast.js`. Takes ~3.5–4 min. Safe to run any time: the engine TRUNCATEs and rebuilds `product_forecasts`, archives prior past-dated rows, and records a new `forecast_runs` row. Python deps live in the server venv (`venv/`); `run_forecast.js` handles env + venv automatically. +- **DB access for validation:** `ssh netcup`, then `PGPASSWORD=6D3GUkxuFgi2UghwgnUd psql -h localhost -U inventory_readonly -d inventory_db`. The engine itself connects with the write user via env vars loaded from `/var/www/inventory/.env` — schema changes should be made idempotently *inside the engine code* (the file already uses `CREATE TABLE IF NOT EXISTS` / `CREATE INDEX IF NOT EXISTS`; use `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` the same way) so no manual migration is needed. +- **Python gotchas already handled in this file (don't regress):** numpy types must go through the registered psycopg2 adapters; `pd.Series.combine_first()` keeps zeros over real data — use `reindex(..., fill_value=0.0)`. +- Engine runtime budget: currently ~212–227s. Phases 1–2 shouldn't move it meaningfully; Phase 3's extra archiving adds one INSERT…SELECT. If runtime balloons past ~6 min, investigate before shipping. +- `--backfill` mode (`backfill_accuracy_data`) is an in-sample backtest using the *old* formulas. **Do not run it anymore**; there is enough real out-of-sample history. Updating it to match the new logic is optional/low priority (F11). + +--- + +## Phase 1 — Bias bugs in the engine (no schema changes) + +### F1. Decay velocity: stop averaging over sparse snapshot rows + +**Where:** `forecast_engine.py`, `batch_load_product_data()`, the decay query (~lines 697–710). + +**Problem:** `AVG(COALESCE(dps.units_sold, 0))` runs over only the snapshot rows that exist — mostly sold-days. Measured inflation on the current 975 decay products: **4.07x** (1.353 vs 0.332 true units/day). This feeds `compute_scale_factor()` for the decay phase and is the single largest bias source. + +**Fix:** divide the sum by calendar days in the window, clipped to the product's age (decay products are 14–60 days old, so a 20-day-old product's window is 20 days, not 30): + +```sql +SELECT dps.pid, + SUM(COALESCE(dps.units_sold, 0))::float + / GREATEST(LEAST(30, (CURRENT_DATE - pm.date_first_received::date)), 1) AS avg_daily +FROM daily_product_snapshots dps +JOIN product_metrics pm ON pm.pid = dps.pid +WHERE dps.pid = ANY(%s) + AND dps.snapshot_date >= CURRENT_DATE - INTERVAL '30 days' + AND dps.snapshot_date >= pm.date_first_received::date +GROUP BY dps.pid, pm.date_first_received +``` + +No Python-side changes needed; `data['decay_velocity']` keeps the same shape. Products with zero snapshot rows in the window still get no entry → existing `scale = 1.0` fallback applies (acceptable: decay classification requires `sales_velocity_daily > 0`, so truly dead products don't reach this path). + +### F2. Mature history: reindex over the full calendar window + +**Where:** `forecast_engine.py`, `forecast_mature()` (~lines 833–836). + +**Problem:** `hist.set_index('snapshot_date').resample('D').sum()` only spans first-snapshot → last-snapshot. Interior gaps correctly become zeros, but **leading and trailing quiet periods are absent**, so the Holt level is fitted on the product's busy span. A marginal mature product whose activity clusters in 2 of the last 8 weeks gets a level ~4x too high. + +**Fix:** replace the resample with an explicit reindex over the full `EXP_SMOOTHING_WINDOW` ending yesterday: + +```python +hist = history_df.copy() +hist['snapshot_date'] = pd.to_datetime(hist['snapshot_date']) +hist = hist.set_index('snapshot_date')['units_sold'] +full_index = pd.date_range( + end=pd.Timestamp(date.today() - timedelta(days=1)), + periods=EXP_SMOOTHING_WINDOW, freq='D') +series = hist.reindex(full_index, fill_value=0.0).values.astype(float) +``` + +Notes: (pid, snapshot_date) is unique in `daily_product_snapshots`, so no duplicate-index risk. `observed_mean` and the `cap` recompute over the full window automatically (intended — the cap gets correspondingly tighter). Mature products are by definition >60 days old, so the 60-day window never predates first receipt. Do NOT use `combine_first` (see gotchas above). + +### F3. Stop double-applying the monthly seasonal index + +**Where:** `forecast_engine.py`, `generate_all_forecasts()` — the `seasonal_multipliers` pre-compute (~lines 959–961) and application (~line 1050). + +**Problem:** every per-product calibration (decay velocity, mature Holt level, launch first-week scale, preorder rate, slow-mover velocity) is fitted on *raw recent actuals*, which already embed the current month's seasonal level. The forecast then multiplies by the **absolute** monthly index of the target date. Example from the live indices (`forecast_runs.phase_counts` for run 129): May = 1.224 (sale month), June = 0.982. Early-June forecasts were calibrated on May-sale-inflated velocities and barely discounted — a structural ~25% over-forecast at that transition, and it'll be worse around November (1.316). + +**Fix:** apply the seasonal index *relative to the calibration period*. Compute a calibration index as the average monthly index over the trailing 30 calendar days (robust at month boundaries), then divide: + +```python +today = date.today() +trailing = [today - timedelta(days=i) for i in range(1, 31)] +calibration_index = float(np.mean([monthly_indices.get(d.month, 1.0) for d in trailing])) +seasonal_multipliers = [ + monthly_indices.get(d.month, 1.0) / max(calibration_index, 0.1) + for d in forecast_dates +] +``` + +Leave the DOW multipliers absolute — every calibration is a multi-week average and therefore DOW-neutral, so reshaping by absolute DOW indices is correct. + +**Optional sub-fix (same area, low priority):** the monthly indices are computed from a single trailing 365-day window, so each month appears once and YoY growth contaminates "seasonality". A cheap improvement is widening `SEASONAL_LOOKBACK_DAYS` to 730 and averaging the two observations of each month. Do this only after the main fixes are validated. + +### Phase 1 validation + +Deploy (edit locally; NFS propagates), run the engine manually once, wait for 3–5 daily cycles, then: + +```sql +-- Portfolio ratio per day (target: drifts from ~2.0 toward 0.8–1.3) +WITH ranked AS ( + SELECT pfh.pid, pfh.forecast_date, pfh.forecast_units, pfh.lifecycle_phase, + ROW_NUMBER() OVER (PARTITION BY pfh.pid, pfh.forecast_date ORDER BY fr.started_at DESC) rn + FROM product_forecasts_history pfh + JOIN forecast_runs fr ON fr.id = pfh.run_id + WHERE pfh.forecast_date >= CURRENT_DATE - 7) +SELECT r.forecast_date, round(SUM(r.forecast_units),0) AS fc, + SUM(COALESCE(dps.units_sold,0)) AS act, + round(SUM(r.forecast_units)/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS ratio +FROM ranked r +LEFT JOIN daily_product_snapshots dps ON dps.pid = r.pid AND dps.snapshot_date = r.forecast_date +WHERE r.rn = 1 AND r.lifecycle_phase != 'dormant' +GROUP BY 1 ORDER BY 1; +``` + +Also check `forecast_accuracy` `by_phase` rows for the newest run: decay bias should fall from +0.35 toward ~0, mature from +0.17 toward ~0. (Accuracy lags ~1 day behind each fix since it evaluates yesterday's forecasts.) + +--- + +## Phase 2 — Demand the model currently ignores or mistimes + +### F4. Preorder: forecast the preorder rate until arrival, launch curve after + +**Where:** `forecast_engine.py` — `batch_load_product_data()` (add arrival dates), `generate_all_forecasts()` preorder branch (~lines 1005–1009), and `forecast_from_curve()` (or a small wrapper). + +**Problem:** preorder products run the launch curve from `age=0` starting **today**, i.e. full first-week launch sales while the product is still weeks from arriving. Actual preorder-period sales are a much slower trickle. + +**Fix:** + +1. Batch-load each preorder product's expected arrival from `purchase_orders` (line-item grain: it has `pid` and `expected_date` directly). Open statuses verified against live data: `created`, `ordered`, `electronically_sent`, `receiving_started` (~705 open line items currently have a future `expected_date`): + +```sql +SELECT pid, MIN(expected_date) AS expected_arrival +FROM purchase_orders +WHERE pid = ANY(%s) + AND status IN ('created', 'ordered', 'electronically_sent', 'receiving_started') + AND expected_date IS NOT NULL + AND expected_date >= CURRENT_DATE +GROUP BY pid +``` + +Fallbacks, in order: (a) an open PO with a *past* `expected_date` → assume arrival in 7 days; (b) no PO at all → arrival in 14 days (and log a counter of how many hit this default). + +2. In the preorder branch, build the daily array piecewise. Let `days_until_arrival = (expected_arrival - today).days`: + - Days `0 .. days_until_arrival-1`: flat observed preorder daily rate = `preorder_sales[pid] / max(preorder_days[pid], 1)` (both already batch-loaded), clamped to ≤ the curve's scaled week-0 daily value. + - Days `days_until_arrival .. horizon`: `forecast_from_curve(curve_info, scale, age_days=0, ...)` shifted so the curve's day 0 lands on the arrival date (i.e. pass `horizon_days - days_until_arrival` and offset into the output array). + - Keep the existing `compute_scale_factor('preorder', ...)` for the post-arrival curve; the pre-arrival segment doesn't use it. + +This is consistent with how the reference curves were built: historical preorder units were recorded on their **order dates** (pre-arrival), so week-0 of the fitted curves reflects post-receipt orders, not the backlog. + +### F5. Dormant products: small positive rate instead of hard zero, and count them + +**Where:** `forecast_engine.py` — `generate_all_forecasts()` dormant branch (~lines 1040–1042), `batch_load_product_data()`, and `compute_accuracy()`. + +**Problem:** all ~28K dormant products are forecast at exactly 0, yet they sold 16,180 units in the eval window (~11% of all demand) — restocks, promos, long-tail. Worse, dormant is *excluded* from the headline accuracy filter, so this miss is invisible. + +**Fix (cheap version, do this now):** + +1. Batch-load a trailing-180-day order rate for dormant products (11,362 of them have ≥1 sale in 180d — verified): + +```sql +SELECT o.pid, SUM(o.quantity) / 180.0 AS rate +FROM orders o +WHERE o.pid = ANY(%s) + AND o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - INTERVAL '180 days' +GROUP BY o.pid +``` + +2. Dormant branch: if the product has a rate > 0, forecast it flat with `method = 'velocity'`; else keep zeros with `method = 'zero'`. Apply the same DOW/seasonal multipliers as everything else (automatic — they're applied after the branch). +3. In `compute_accuracy()`, add a second overall row: `metric_type='overall', dimension_value='all_incl_dormant'` with no dormant filter (keep the existing `'all'` row unchanged for trend continuity). One extra entry in the `dimensions`/`filter_clauses` dicts. + +**Upgrade path (optional, Phase 4):** replace flat rates for `slow_mover` + dormant-with-sales with TSB (Teunter–Syntetos–Babai), the standard intermittent-demand method with obsolescence handling. Per product over a daily series `d_t` (build it from snapshots the F2 way — full calendar reindex): + +``` +if d_t > 0: p_t = p_{t-1} + β·(1 − p_{t-1}); z_t = z_{t-1} + α·(d_t − z_{t-1}) +else: p_t = p_{t-1}·(1 − β); z_t = z_{t-1} +forecast = p_T · z_T (flat across horizon) +``` + +Start with α=0.1, β=0.05, initialize p = (nonzero days / total days), z = mean of nonzero demands. Scope: slow_mover (~6K) + dormant with 180d sales (~11K); series from up to 180 days of snapshots (sparse rows → ~manageable volume). Only do this after Phase 3 measurement exists to prove it beats the flat rates. + +### Phase 2 validation + +After 3–5 cycles: preorder `by_phase` bias should drop from +0.85 toward < +0.3; the new `all_incl_dormant` row should appear and its `total_actual_units` minus `'all'`'s should be largely *covered* rather than all-miss (dormant `bias` rising from −1.36 toward ~−0.3 or better). + +--- + +## Phase 3 — Fix the measurement (schema + engine + API + UI) + +> Without this phase you cannot see whether Phases 1–2 worked except by ad-hoc SQL, the lead-time chart stays a single bucket forever, and the dashboard keeps displaying a number with a 190% floor in red. + +### F7. Archive long-lead forecasts so 15/30/60/90d accuracy exists + +**Where:** `forecast_engine.py` — `archive_forecasts()` (~lines 1086–1154), `compute_accuracy()` CTE (~lines 1201–1228). + +**Problem:** the current design archives only *past-dated* rows of the previous run before truncation. With daily runs, that's only ever the 1-day-ahead slice — all 879,800 accuracy samples sit in the '1-7d' bucket and the longer buckets in the UI chart can never populate. Purchasing decisions ride on 30–60d forecasts that are never validated. + +**Fix:** + +1. Keep the existing past-date archiving exactly as is (it provides dense short-lead coverage). +2. After `generate_all_forecasts()` completes, additionally archive a **sampled set of future leads** from the new run, non-dormant only, attributed to the *current* run id (correct attribution, unlike the past-date path which attributes to the previous run): + +```sql +INSERT INTO product_forecasts_history + (run_id, pid, forecast_date, forecast_units, forecast_revenue, + lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at) +SELECT %(run_id)s, pid, forecast_date, forecast_units, forecast_revenue, + lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at +FROM product_forecasts +WHERE lifecycle_phase != 'dormant' + AND forecast_date - CURRENT_DATE IN (7, 14, 30, 60, 89) +ON CONFLICT (run_id, pid, forecast_date) DO NOTHING +``` + +Volume: ~10K non-dormant products × 5 leads ≈ 50K rows/day; the existing 90-day prune (`forecast_date < CURRENT_DATE - 90`) bounds steady state at a few million rows. Note future-dated rows survive until their date passes + 90 days — that's intended. + +3. **CRITICAL companion change** in `compute_accuracy()`: the accuracy CTE must now exclude not-yet-realized rows, or future-dated archives get scored against actual=0: + +```sql +FROM product_forecasts_history pfh +JOIN forecast_runs fr ON fr.id = pfh.run_id +WHERE pfh.forecast_date < CURRENT_DATE -- ADD THIS +``` + +4. **Dedup semantics change.** Today's `ROW_NUMBER() OVER (PARTITION BY pid, forecast_date ORDER BY started_at DESC)` keeps only the latest (= shortest-lead) row per pid/date, which would silently discard all the new long-lead rows. Restructure: + - Compute `lead_days = forecast_date - started_at::date` and the lead bucket *inside* `ranked_history`. + - For `by_lead_time`: dedup `PARTITION BY pid, forecast_date, lead_bucket` (one sample per pid/date/bucket, latest run wins within a bucket). + - For everything else (`overall`, `by_phase`, `by_method`, `daily`, and the new weekly metric below): restrict to `lead_days BETWEEN 0 AND 6` and keep the existing per-(pid, date) dedup. This preserves the current meaning of the headline metrics (short-lead) while the lead-time table becomes real. + +### F8. Track a naive baseline (forecast value-added) + +**Where:** `archive_forecasts()` (both INSERT paths), `compute_accuracy()`, `forecast_accuracy` schema, `/forecast/accuracy` endpoint. + +**Problem:** the engine currently *loses* to a trailing-average naive forecast (221% vs 204% daily WMAPE) and nothing on the dashboard would ever reveal that. Every accuracy improvement should be judged as value-over-naive. + +**Fix:** + +1. Schema (idempotent, in the ensure blocks): `ALTER TABLE product_forecasts_history ADD COLUMN IF NOT EXISTS naive_units NUMERIC(10,2);` and `ALTER TABLE forecast_accuracy ADD COLUMN IF NOT EXISTS naive_wmape NUMERIC(10,4), ADD COLUMN IF NOT EXISTS fva NUMERIC(10,4);` +2. Populate `naive_units` during both archive INSERTs via a join — naive = flat trailing-28-day average daily units as of archive time (28 days = DOW-balanced; information available at generation; same value at every lead, which is exactly what a naive baseline means): + +```sql +LEFT JOIN ( + SELECT o.pid, SUM(o.quantity) / 28.0 AS naive_daily + FROM orders o + WHERE o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - INTERVAL '28 days' AND o.date < CURRENT_DATE + GROUP BY o.pid +) nv ON nv.pid = pf.pid +-- select COALESCE(nv.naive_daily, 0) AS naive_units +``` + +3. In `compute_accuracy()`, add to each dimension's aggregate: `SUM(ABS(naive_units - actual_units)) / NULLIF(SUM(actual_units),0) AS naive_wmape` and store `fva = 1 - wmape / naive_wmape` (NULL-safe). Rows archived before this change have `naive_units` NULL — treat NULL as excluded (`FILTER (WHERE naive_units IS NOT NULL)` on the naive sums) rather than as zero. +4. Endpoint: include `naiveWmape` and `fva` in the `overall` (and per-phase) payload of `/dashboard/forecast/accuracy` in `dashboard.js`. + +### F9. Weekly-grain headline metric + bias as a percentage + +**Where:** `compute_accuracy()`, `/forecast/accuracy` endpoint, `ForecastAccuracy.tsx`. + +**Problem:** daily-grain WMAPE on this catalog has a ~190% floor — as a headline it's noise. The informative numbers are (a) weekly-per-product WMAPE (currently ~109%, target ~70–85% post-fix) and (b) aggregate bias, which the UI currently renders as `+0.108 units` — indistinguishable from zero while the reality is +70%. + +**Fix:** + +1. New metric in `compute_accuracy()`: `metric_type='overall_weekly', dimension_value='all'`. Definition: using the short-lead deduped rows (lead ≤ 6, non-dormant), aggregate per `(pid, date_trunc('week', forecast_date))` keeping only complete weeks (`COUNT(*) = 7`), then `WMAPE = SUM(ABS(fc_week − act_week)) / SUM(act_week)`, excluding pid-weeks where both are 0. Store sample_size = number of pid-weeks. Compute `naive_wmape`/`fva` the same way from `naive_units`. +2. Endpoint: expose as `overallWeekly`; also add a weekly variant to the `accuracyTrend` query (`metric_type='overall_weekly'`). The trend will start empty (old runs lack the row) — that's fine; don't backfill. +3. `ForecastAccuracy.tsx`: + - Headline WMAPE → `overallWeekly.wmape`, labeled "WMAPE (weekly)". Keep daily WMAPE available in a tooltip if desired. + - Color thresholds for weekly grain: green ≤ 60, yellow ≤ 90, red above (tunable; document that they're calibrated for intermittent retail demand). + - Replace the bias row: show `(totalForecast / totalActual − 1)` as a signed percentage labeled "Forecast vs actual" (both totals already arrive in `overall`). Keep MAE. + - Add a "vs naive" line: naive weekly WMAPE and FVA. FVA > 0 = engine adds value. + - The lead-time chart needs no code change — buckets will populate as F7 rows mature (7d lead evaluable after 7 days, 30d after 30, etc.). +4. `confidenceLevel` in `/forecast/metrics` ([dashboard.js ~line 360]) is "share of products forecast via lifecycle curves", not confidence. It only feeds a per-day tooltip field — rename the JSON field to `curveCoverage` and update the one consumer in `ForecastMetrics.tsx`, or leave it and add a comment; low priority. + +### Phase 3 validation + +- Next run after deploy: `forecast_accuracy` contains `overall_weekly` and `fva` values; `/dashboard/forecast/accuracy` returns them; the overview popover renders weekly WMAPE, bias %, and the naive comparison. +- After 7/14/30 days: `by_lead_time` rows appear for '8-14d', '15-30d', '31-60d' buckets respectively (61-90d after ~60 days). +- Confirm engine runtime still < ~5 min and `product_forecasts_history` growth ≈ 50–70K rows/day. + +--- + +## Phase 4 — Optional / after the above is proven + +- **F6. TSB for slow movers + dormant** (spec in F5). Gate on Phase 3 measurement: ship only if weekly FVA improves on those phases. +- **F10. Confidence-margin source:** `load_accuracy_margins()` feeds daily-grain per-phase WMAPE (clamped to 1.0) into the intervals, so every interval is ±100% — uninformative. Once `overall_weekly` exists, add per-phase weekly rows (`by_phase_weekly`) and source margins from those instead. +- **F11.** Update or delete `backfill_accuracy_data()` (it encodes the old formulas). Until then, just don't run `--backfill`. +- **F12.** `compute_dow_indices()` weights by revenue but the multipliers are applied to units — switch `SUM(o.price * o.quantity)` to `SUM(o.quantity)`. Tiny effect. +- **F13.** Longer term: for reorder decisions the right target is P(lead-time demand > stock), not a point forecast. Evaluate quantile (pinball) loss at lead-time horizons using the existing confidence-interval columns. Design separately. + +--- + +## 4. Success criteria + +1. Rolling-14-day portfolio forecast/actual ratio within **0.8–1.25** (currently 1.5–2.5). +2. Weekly-grain WMAPE ≤ **90%** and **FVA > 0** (engine beats naive) sustained for 2+ weeks. +3. Decay/preorder/mature per-phase bias within ±0.1 units/day (currently +0.35 / +0.85 / +0.17). +4. `all_incl_dormant` actuals covered: dormant bias better than −0.4 (currently −1.36, i.e. 100% miss). +5. Lead-time buckets through 31–60d populated with ≥10K samples each within ~6 weeks. +6. Launch phase stays healthy (bias within ±0.15, WMAPE not degraded) — regression guard for F3/F4 changes. + +## 5. Re-measurement appendix + +The naive-vs-engine comparison used in the diagnosis (rerun any time; adjust dates): + +```sql +WITH ranked AS ( + SELECT pfh.pid, pfh.forecast_date, pfh.forecast_units, pfh.lifecycle_phase, + ROW_NUMBER() OVER (PARTITION BY pfh.pid, pfh.forecast_date ORDER BY fr.started_at DESC) rn + FROM product_forecasts_history pfh + JOIN forecast_runs fr ON fr.id = pfh.run_id + WHERE pfh.forecast_date BETWEEN CURRENT_DATE - 9 AND CURRENT_DATE - 1), +eng AS (SELECT * FROM ranked WHERE rn = 1 AND lifecycle_phase != 'dormant'), +naive AS ( + SELECT o.pid, SUM(o.quantity)/30.0 AS naive_daily FROM orders o + WHERE o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - 39 AND o.date < CURRENT_DATE - 9 + GROUP BY o.pid) +SELECT e.lifecycle_phase, COUNT(*) AS n, SUM(COALESCE(dps.units_sold,0)) AS actual, + round(SUM(e.forecast_units),0) AS engine_fc, round(SUM(COALESCE(nv.naive_daily,0)),0) AS naive_fc, + round(SUM(ABS(e.forecast_units - COALESCE(dps.units_sold,0)))/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS engine_wmape, + round(SUM(ABS(COALESCE(nv.naive_daily,0) - COALESCE(dps.units_sold,0)))/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS naive_wmape +FROM eng e +LEFT JOIN naive nv ON nv.pid = e.pid +LEFT JOIN daily_product_snapshots dps ON dps.pid = e.pid AND dps.snapshot_date = e.forecast_date +GROUP BY ROLLUP(e.lifecycle_phase) ORDER BY 1; +``` + +Baseline numbers to beat (June 1–9, 2026): engine 221% / naive 204% daily WMAPE; engine_fc/actual = 1.82; per-phase table in §1. diff --git a/inventory-server/scripts/forecast/__pycache__/forecast_engine.cpython-312.pyc b/inventory-server/scripts/forecast/__pycache__/forecast_engine.cpython-312.pyc new file mode 100644 index 0000000000000000000000000000000000000000..ba931b43c993db08d8e8bd12b9367325119420ee GIT binary patch literal 81621 zcmd?S3s_uddL~#!RZ&+ciYo3xItU~v?g(8Gx?n&T3xNc>*vK|r;uNS56!=sjBwV=Z zo}Mw-N(8YxLa}?APJ1>u_9XOl?*u>Tov72DjkD?9t)l2a*YfP6O!`UU*_~(9ZEw2c z%p!%Pc*7 zbjqO9-PQ5BULCLJ(}wlEdUj9iO=EXMuYui-y+(F7^_p-u45yEz_ok1Sd(9)3UW;B# zYwfkMIQCu#yJz%fuzRL-ce1;2IBUe!>tbn4!`UP5UiXNn*Q3|@GlnYV>Z$jk8ueLz z^4o@L9d{OMc>W_}usM-o>b)ORr<8d&OVckk5ZtFaGYW#gGt^Mzm0_tx`8 z_|?E~L%4@8M%c)2N4S^Yfp8!4+drt|x!=|+=@0NF_;QfniSQ6#im-_N+$Ko}k2t_4EjQ6bRp8|JPGz93)MkJ1K5gw4jT`fY!+s$cs{j^NXizD&&(G>xJ9g{IU%8r^tG@p01A*aTuCd-`YY*NC2&2Ife=yA5@CgCm zRccEGt68l-c*84}*7j82sji+2)f^r!_Oy4ihjT4GJ(s(>+t`oJmb2~R+pg{lTm|nR z@Qn|LxqW*Z_jqj|66`y@hJ=MjJBq#K3d^gj zE~1l1v3&2*y`%4Z$C$8;g>Lnajt%Zf*!(xc{$MCD8Vn_jEuqt)glS-S)E9n7pU~e7 zQN_4VvgLuGcF8A}=3(E+Ro-`0a3jtR{C%_?!L;tX2iFgG#twI`AMS}A?peF=Y_#`E z{K9kb!_QCaK0*xLKV@qL!a;(af-D4LdxUKIP1R-rM-LtfauEEJ*X|vo;Gn`YD9A*R zFa(0($5*!mQDji(uOI4&9qL#=bUt?I{95z_9+18vh}UdYa!i)*s_=#S zX{vcc+?8Dw!~Pro;h>LxKF{r{81aW_UGat{ExS%N?K<1Es|Qon#9}4VF`P$y;e?sy z>%d4jVUfpoKeARX>J>H-!zdLKk>&!sQcc9LQTduKKdJX2tNJWI5pg0D44)&lfj9mN zsZD(PuaMfzTa?r^x7hVL+F<2vaz63`^`8A*X-VtRc^#8Yt%4t&#QC^$qoMGi;O{x# z!S#;@gMRcq;R-@GSkoU}@M{xhX@yT%hev^mt|0^Vju!hwW*~S22m&~%Z#?7|5_UwE z7ypEg#iQjv;ZWo8SJ~@xKn8(>7{6Xq!pJf;vK%a9zEEglRNxco{8bu>3FFwP5ca0C z>76i3wN9jsh7x9}$_c}uKP(gge(#7=f-OynOuX$=TPe^iXvJS>9Kp2iCwBL`y(nfc z+RSj?>7DJp^TO;43#S)fif1$?y@{3|h-Vz!bh+;q&KKU@KEHi2V@Zg+_G~&`cP3{i z@4P(w@Z(BAJ1s`*q-i4f2d10rHe}=%?A0RQO;(^dw2i| zPvpC9$`COOX=6#TWkO{iNH)YsfwXvx=l4wVXwd6aBsOG`i&XCseMA>YdoNuk1pCu? z_D@aUqbt=lV7;G6)9ao^sYD{(G(4-D(0k330_MmU9!ofNAB!L;tLjP4DqYu(DltX$l>^QXEr z)8P$E*1Ba!%(7$A65Vq-YT2=7c?N@H-O0tA+@gDtU*5mwZ2GwgQGZVHe@4M4A)4}E zt!l9uzGpMFxXnozQyCV2jYxMvfeb0jCSFLFIvl3-5q+uw!z=VR28@14SzP5Dbh@cW zx*TOGl#d!uE=#ZbVj6F_-bM&0tPH6kYRi_3H%8Ll)bXaT8RSJtq_4LQsN0960hK=4 zan>hX$0)kR3FE;K7YdK}U*`frt{gw9*URzB7Y5you`>Vy8-#L&zjm-zT))PIz^K6W z3%<}bf5=R0)?c&@GGLc2;BQz1zadfG3@B^=?(#S+S^xq(11fIRs=Uzj_> zb#`5@@Kz_KqXPVbFe>y(V;0fMd%__8sUaNp5I!R6$Hy?@vCPu;0rLcH5yIt38C^cr z-rcULg52(;NF3MF*+!9suvn*Nu})kKce?XJd-tW54z9d`yEZy5sJVBtNjPb5RqHaz zQ6{LP3SJq3Q?)BX-gnXy>7cwrqXL#%aaJTO!stZ*=y))UPopCvfpEfr%}l}=2o8*T z4Z;bGf`kEyuu_Xfg=irIfW+CA&EAfW^|7M+s0)N1M4+u}=sJRF-2-dJoub*I*NPwN zET$a`r5hIe9s8^u3uunxV_mv6<6)-GoqM->zIw*|a~GmK%+{H#ublbPnXjDPFlWxW zzWmZAl2~RfbB7lu76Q@olTp{HA6ZU6H0UhupXX2t%2}L&^V?g>4c{v_wbYyYwey1( zSGF)8#{*`d{>g+(B+ZUlI*C;8N;8$7BZf(&1`5_sR-L0Y3}`CQ&-rhTjRt`>1K8Q| z*i_!)CIaDW+*M!5Ka8uj1toq1!SGF+6LjePo)$*K!G<&46# z4`3JYby~y_F-44n2C!GmzpalLcnb|Q-Wo9s8V1sM8z?^s-p`?gsq`sxB>g$`+>~X? zI%NyH)Vv1uk@Vlz|LgQ$*G<_q?WbMcmJ6lYCjnF5G>N`aYU=p3RGRO!G2C*gJ(42t zj7WxrW-PoT;)rBS0>x0gpq|f|%2c5rvB>q&ytAfwXG`%eQ+_vKBuXcoD_1d08!&ba zq?PG}om0**?YG!9R4hNKcRn-Xd{g%|WsDKE#2craFFS7;#ruSQLMI_Q=j4B|5z06b z2y@|UerX9{vy}^va$tJ|d_iu)@4v3Fkg&z_fy+awT7A2wd$W_1K74z+8gqtSo3T4r$7OTh>ooZ#7s^$dH zR^#A}@m>y?w-aj`7Z?D#3^VL3;#N>{CO&)%H7EtRL{jiWA_)JckzMg)z#8oLOP=35 z`InPlImUho#0B{q5x}0~AI^ZyfSqUBSbZY>DzFb9N~DkYZVvcwd`Qc!6cHmNyOkJe zN*(-s6X$gb7ZE3sjs-Y4=m);?2L+m42@``>!hoC*1cCz19JXMBL>wPU7)OA(6OL|R zJM56#3BxCBmwdzH;Pdn z#Xi&U&u+}=zdZRy_uWhLmlj%He|E9tH(uCqy7ATKye-_hF?(aq_u8$vwP53t7 z<}JfQ%bV$o`g_&|+uTUpSr)hMntp1-o;lt2(3oxVY-Sg2I`S5B7WErh*>i0RJ7-=_ zc(}zqOZso@Uus-Vi|%ZUy7z9l^FFqk+_qWsjB&>I(4niUi&@HNtaD8ZH&KD?;)RP* zSLNb`RafOqI_=4F=W`ca3zzN{F6m?W-gwgMx4OT6`QGKl^KbUv+rHEi%dd&MYG=|n zDr%QqziXR$aqh&zrTMN^XW5Ej#`qJbXYSd!bI02`i<9w^J>O|xsrYu+4=#SN_xo3( z&T~=AxnFEL^53eA+IJE8?jZ`pI%|!3b}zoT*dNWVT(eYdI^F;95LCYBlMwAi|9Ib1 z2it3Pe^To_vBU5uNAgb;8vgr26P}YOeOizidZp!>Hh@~xi0zp^Op}9MgL-+RHkB4h zOGV9xvNi{sQ(mtT|=jbiIK!E-pDK4$u`{`Lw{O8*Pr%0P!4TU`SjP*ZkvsG zGiB((R+2YU!-o{qC1rw}ynmQ7MvOpNSj?wP0iY+eLqSQ8#%|L+-=xzqL}s3{go)S4 zu84)VQc3diJ!OrUcpKZ5xWdE}V;5qv`pB^&CMp*QBprw_Pl;vY?J`*sNsm~GOK`i{ zsCyg?o(Gbrwnx%$=tKJ37Cs%R3@lY9QYpmD5LIx?1yq*K8|ed|@um?-Bb|3Dzq0Vl zrTogqFT3)~jb9$+R}OyVD!=mZD^33OH3g+%$CuuNp3$@3nb3LjeY>z2FdWi4Ixl6ghm03Pqw3mKPRKu%SqjLWosd zkdYI%Io!lGY#3?Vb46 z-O_rY0d z9X-OeUhM8}@4V2*_Pfd3Li9^HYO+hj&L_LOE}r9_dX{To`9NgT-mU)pkQf1z$4>`C z;{yYM{(w3Zh#sMhG)GX7MohIN-WnLjK3NM>VF1~DNE1WC(+`g&%u?H>H`|UU41gO$ z{U6dcY|^F4xrxItI`WC3wsBz6#9bKo9}!-lI4170pFhF~3wnQT{0MF*guoF-J>a>= z7e4YKA$58l4<7MmFn&&gGVkl?>N@jOOY0dx>Diuy$v-j{z9rC3NyO`BiItQvF|t7z z!Iy+303H)|z#+zMl&A=S2om8OzI*L#11y}RxM}=ALeD2mZ1^P%z8ixHvji&P7DX^1 zBjI1LrvQJG@q;v|UBv8eY_e&p_5nJ5a_YNC^A`R>L|{+r9%fnXmg&}yb9J`7x$r{I zTIt^C&V6y~{^_S4n60ne{PN8=8s~)94=kMjjl+buu$hi}F0ENEZ*sd9eM_!y+W*2? zz0feHpKF{?Uv*YTE!7VknKQR$&#pU)V~*muW5;y+hRr!wI$Jf}`XDoF`s7Bo=Wf+} z)$27s(-};=X47Z%GmV>>uDQ}V|9s_Zqcet$n)>C2-@Sa>Ki9foo$riimM-?MW_njj zXAGNp+ZK8jOJ1LvG0z!Zbv!g9hmRe)O!r&^=!z?ZUW;e+8+OOs$yIyNM!~k3lXESv zK24al_+yLCWO?P(mrg~&S<72;eWP%xHOkd5o0bPxeBTMmW}x0JW+dZ%!?HCl0S#k4Yb-}nB|2kz*RvmacH z_Vh-tJQwSEKKg<$+I{t*L7&O%Ki26@e*FV`_H^gp|7@2ot0qKa>U-I?Hm@-WnbUk> z$o$LyKSAc2J(ogmOksa2qS~o+pm_6VM)BC8YEZnDw-8c$7ekz`DI3t7^$`?r-GbuN z6%=pRpm@6~A&HnHw$F^>%?gS)Yf$_Xh@(Lwj_{ob1Ta7CD9yl7v*S(U*9`M%x1C1a zlmpy@CzA;oeCAM**lukYamdg}-fKq8T12lw`Oe9|Wb3+&??*<1;cK+1^!q}i&_a-x z?i9#gTA!bc>{))&HWJvrreV(`D4tW%JFAL};$^fBq+UzQd-z&N zR|~}95dIkjUqO&CF*u3)@>lTU6a5ik2KUKA)^RBYR1^0}S~cg*6wz&NPkT#GS7%E{ zGR_rVqY@O1E8M1^zeWM!TH)6zpvl9~tZ;{(2+1alL!*HpQ27aZL#S4mqu2To;Wy}R z3HGu2hJ@GY+Zz-Rz7@Vo!95Co6G6hvh{a$yMDtmLcx6^K1f(MBWw{OC>XU(};Nfo| z;h*C#)Po@iJe;`|4HbCKM4(9^Q0j&+U0_ zV)_&zufF!& zr=jGA{mZ_U>~A~%BJ;>n!(7Rnd#+_}aKSe}wCIlJ>{@m1j%FU2K8aw{Xcm#Tcd6|6 zz3+IJPsS^oquwLYjH7GDV;drZE!Sn${{7E(>YPVI#7`wOd%*bprtG$6qt}`+wfTL+ zp-IP6!%$WtVh;rEWcp=dXEjLoJ<`2#GJVfMu4`amGIJl-Haa})1HH;pI~(egUszB0 zp!U=QX-E=2r>!Kz=fAM(bgvY@vi+4ENvcHjZOf$r?Qq1+PYZiAL*|i`Q=5mCg4M@r$-x?@GUWv*bxu~E9{68IT{!Z3?_1}U`JSqJs_9{u?ZJh3Up_1E0t5J_pml6 zw&|C1Le36Kl@ud`-pDZqG<|>joKeSTg1Mm`b2fF%X&6ZMh=-cc*D-a!+|88IkOP zw5gm(PQ;0^ zRBpuiDXni4ThGRN2I`l`V8AHyNCv?`hR9H(TAKV6$>Uj#_{t@klP8H;I*QD);sYFx8&8rp*@12h=a}6JH?Lgz|avN;0Q^9$hG}C6CFk5c8kGTmylFa%t*4 zVi`KD{*s^gBDps5T}b0y!K_b<{kh7r%Vo%SR#ryRx|OuDeXdfr$+^q-6s2sVktTl! z{EF!}?%SpN4(ZNG_YlkT6ZR!+Gz`Tj&WE3rRUSy^%!^8JZrDX|9}>IyigzkSR| z%JMGyr@qIiKO&DE^)8Qk8qtvgktIEJNc}E9$?*t^Lr3Me>Rnv@riwJVw5VU@r_Y$n zHqCraMZJ`2s262Uf+{JCY(uy!QiO1K66Jwe|A=DJtR%0k7~jMKW}_{_Q{#bQUJ}?c zNgLDd=lnis<1-z7@T;K&CqCZf6oq|oM8RH%irh7Dy?sLeHR$k@s;yVrU!Ne&C2*0M zcE6Z$HQ3i2RLFTTcge7z90iVpR1IMV3@NXY21=7quOnrBk+ECh>la2tV38y4mNG-$ zYT}MV>l1=nR@3j_MLvB2o|Bq1am`QrZju>u*RT_3P4@>!#|L3^AR0J?Bu4yTfSB!r zV3Cd!c`aoy9Juc1cJ4jIGy+Mf3$^|JU9Mrz&4xYPIUk9m&-wV!PL?r@A)wF~;QiQe z`2?`k1v?^PNs>6?kYC}Qa zSm@d)B;}ZC$)8)hS~}W$TH7lq-gqz&4)ujbhtW^)o|G-%g8rZawm`0_ ziC(hY(9FKkfdNcmB{SCjTgQ(utbEsYzh-!_VZcTdx>*E3auN}M^0<;~8X|B|T57F0 zh_)ma2hk%yqwarF^OVBHG4yEq<|iiSs**GqNqnb~Re4$N(mWEon4`I*6nX*@o$d-1 zx?CN1Ko!eTZhg>rQR97~u zW-$~I+hb!)dOJsD3oncrlCwd+#H4Lo7pG6m9Y7G6mXk%Yuv45Y7)p@YNwQ~xzcAPGBEb9f|;d|KG(6+N?;43)`m*$yB#zAs)(n$*(s~$eKbUs_LD##oI9RSCq-5$ zucug@#06Se>eX>a>yfhHR7l_!1;wg(m;sNKAj^X(S@I=90^(&653su^2>whmGxS0y#L0l6R0I(^P+`5z9?a;LDxlkd?%OPSFIq<1h53HJJPj^C4&elj2(ZFV- zQUoCe270Qbc35FBDepha8p&{`lA@#i#08EelL1X5?^lH9XMmwn0cj|2!PTrAJ~xK` ze-5Q9iyMQYIvNqi56J=Q!STH zcb>#La1kx9^b!hbWFHbaDI1pXY<*2t>g-ooeb_<~!`$6?M22W?(9d8C4Jx>IF=phzc3g~(@nOd}3Ue6RD?^FPj~jMq%H}Z?t`+_jK-GBy2451{ zmT+ikGl9gA!XN?|Slh^Ez3&F;#yFU>@ElQy9YU&^t04W4mk-sc+T6 zwJIvj344h2pZ$FyXr?9dq{%1lhFF~#p(Xq-)q}e60yKclK}>s?nk909&HB!DcePz? zz0lKlqN}?P8d;ax6NQhwf2te!qOY~(Lc+{Ia`hGzo0&2<6O59c6tsT)Hp6e1rMtaNl%o>!6w1#G%mUOYbzV=x!V2r- z2NV zfvhk^8Kb9YH%YM*RvDxd%*Y6AgoI03%V}-vgGyn-p}fMckO!F%36OM6Pftr)(BvSX z4`ZrNZ0uy+Rbtm*^D1rW@W&##N+*TwT2 zW->M!4lHl~QC`hjd*6)XfjxhrdDULN;m&{HDp+@w#9Sqdm2p?iQZVK^vhHe&x!U5c z6VOq%XU^Ps?eK!@)ngPlr|9kr^Dn&JzE;^BFFW!<>D&wPoX#2BM$V4)oZYdU-O=)< zwTh-_&fyu`Ph8w$Vcb;>jb*!Q<|w7E-ygHoh|17Qo`>l=N7frPtM(l%it+vc8q%R}F;yWg_9r~Qrg zh0689s#sxFys&1?Q~N>VjP-#f^G?QW#zs!zM)9tV+-)1%%Qp&kZWNYn>Nln`I9Y&=khCaZB!cmFm1*|F>}_%fbRRo0ZQ38h*YSM~ z-Dpc1C-`x?6GqFlDY($}m~@XMp)4w-y;{WpAY%*wUqBqRcRwbZC{R^+48+@}^2{2y6>4wu7y;3lV>r zKgKgk&QACi_4vP`AlVQ?_%^-!4h8=$f=|*f3!~6go^(ITo2!^C75lMDUe-Ac-;-_I zP|-<;7^#NrjlEt&XJ@CkgeiG4eG*YeMOs==&k8b9o(Y@ymg&tV&+~-gy8l+fMe)UnPAf9zx7acFsRp9&mk{Bq z^YmNFjQUdCoiqJf;m@cx{{sbojv&M#P={@br{l+ui&!H!HIfki6LQcILijHr2tR=h zO|GnZn$fmqY=3Aec3GxRe|%I_x_x%ddwA_gSG3|>+{e zue}_#6#qn2gqfLmBebyR^@+JabVt+5o)!Q5ht>~w#141F4|hhJx}sfQi2C}Y{Pmb` zc-m7*)`K= z9;TU{mg$opXX!GF7WP76_V9O2-yMw~?2J}+#WT)LpLpPOL6ayu|8CuUT|B$=XF8+l ziawP(uRU|_TFkzEW83zbQ*%ADXJPX`=bp`2w`_}9w!PKASh{%a8#VFm^~+4^5arFx?zOfgUL}!-Fv2FFsRnemtV|y;e^DnQtpNUS~n(o-(N7woBRet;}<6Aw8 zMN5}sCHt0Nincx*JJ1_1zOt78T=b=>s4?e3hI9JF&yAHP&%<85Zrk>+H{EMm9E=y$ z{tUXj-dX32Y3AmSEqM!;#a%zLR6xzwpU!ZsZgbijL>N(8p?On6=IrW-e^x zEu( z%4bf@omvP0NiXc4tD9-vaAq&m#+(%!jfX$*u08*S`14m|yZh%d7ecYD(hW!9LSxLa zV=)~G*Bwv9i7QjrXs_j-UTKgWl1jf8Z!ybj2Jyp#x6+R|=IdSX?>g9SfJ@S!L6wH_TbD zeDTX)R7QE-nx%fjnIE;}V^rEQWh-uv$*EXzM?L%2Ec-WdcdX}D#BwX*xs@O3vaQb9 z6EiI{p$&)g&Y9UW4|V2(tcO`TmuK$eykj%J@ayJ#=0(Hrn7?6Oa>luXYx#%fjVqx) zns|TW{ukrNE+@yZ#JPsG{5|u=?;2KG?_1tK8$bATQVb}4$GJmm`Azf29~kbpMo&Ex zYkM|+ymtfIa%W;$9LDl+Y-i(g&r0F9`ZO;`VmtTZXVg;@%c|Xy8Cfc3KqPJr(%UQ>xGT6 z!p3;vzF5}&^{nPt7DdTAHho%ByPLTI^FQzFLFnW~cO6_QSg}W22cm;7MhAqbpspwz zc2Csk`T0Joi4MUhOhNL8ruLmZ^@e!8spn9#uL@ElA|dQY-^FVAiY1i|n=E7|Zxc{{ zMm+I?eVZYy2yj)t!@E)pjlz-h3XIUi$Njb5dznqhNhjPtPBdNr&a#HayW-J4G-t-kkO^MWJ zsN67=mBcHR9UO1g2(q!1=K%G@@JR``9vEkmP^YpbiGM{9ZD&|MqE+cP3BE7 zz=FA0YGF0Yj6C5%&H=|CJ_vY6>5Dzkdul)v!$_<1@%Y1?EFfWIqg# zj$R)hlT^q59mW~UUSz_vM*@H-QdOyGi5aVTF;a`{ASVW$L$+Nhzo5?j*OR`8G#TXn zqy4Y}gB%i00#ih2tk+1jQgh6cQW4i2VrT5Enb1;g1`N+6MkU+HtBDlx16t+_K|BvK zS!CYz=Udyl7$Szp{TAn?4`AnWEi!+#9lf_ESCTE`} z|7Ui4qRdK^@<>7}_JgHKs+Tkx|Ix%#pIZjS7+-8RoTQ0NY+lCNB?it{^>=l%`cy7~ z)WuZwzcHvdRPE6*M}FQkaq~dy9bAp*$3WMM#}-DMF;O`-^GaO4T3LyMRxT6D=J?m1|~|2ATNVppV**l{ZO z0AE0C>qw>WBhn!{0|K15*w2Gwn`9TQ@Fsa3nFK*)Fq_x^rv9!TD{-C?POAR30WPXU z#{!I((CO8){TjJesjW}w`^9F8UTHqDHV?v?=gl$U=pP62g_z;=fwX5OmVrv~WL2m-H_?eRo2LyNpP3x7%9|2+k_DEKM`w22cwLJ%T! zNNXEi($*&U`>c$q!mTuzdgQ{bSK?KbiLpGV{J`Fz<6ucRlL< zhNu%p-5dvG9_NWm<@MjxhhRHQZq#U8!Y@}!&t0m*J}miD$55mya*i-5OC8hnt~7sO zTdx-~BgQY;BWYi%(EHZjY zcv2zMgqz{RWW^QzEP27lfc8IFW^oR-ZpcJaDM9+;`3M=4gIJ zJj=V9Rl9V4xg_RnB+HX5FO4q*S0x}@7hP>?s`t1{0}G3iP+bR{Ft9w19K z|0hWXS^G#j`WQ&DQZ1ieD)tDL@omV25l|fblMt1p zws|oGRiI=7UEUGM`kzfuT_lQ(3Xl>*|5YOM72Q`>7dDfiGV2?L)~Hk`IrCdgxG3qC zs)U}b1&dur2u_V)O%@g%FEDN;V-`zFu5j#(1q0g{@-_Gw)LOZKs1txC2a+e+n$RMX zBBiJb+bZ}5!l9k;PytO<-wkXE$k(<6rk?8o=;}#Q5Jp9G|M9<(%wUs*eH?Vz1o)4m zv?O+g3q%GP*-yu(yK|BSa*B-Z5fQ;7FBp}tcz+TLWIW+c#>G#f?1)dmNZ2+uyR}>Z zahA$>Ea?mLY?AUz(w;sk4cntjcvHcSDrua`a}MUVS)OU?9qxTmSGwf8keO6clGzvJ z?z+!ailPajl;TV(CCQ7;0KB^JItsm(&aI*o5|}W0R~1YHhmizW0Gihc|0@Md2CQdj zN#n^oVd;YpbHUdio-AjMb4i1`54(Gr7*+{?Ibs`Dg3#BIC7kIx$oa!{;y^n}H`$Nj zSaxaLQ6~Nv0)N@DlRZas>tfmUtJ(XOTjGxWn~nSa;NZ6ouC)Hq$@fo2+pff$pId8u z{&xFZX*9PcmR-B%sEhUwZtmIp2NmC{SZVyD1MeRIw>N&ccWuuVF;;afyC&|aeVAd+ zvVnDIwZUuDRsDcoeId#p!7tZ|Vn$j8ftLvmqIl=u(bEP6oAk*_w!uu?E&T8Fk_kB+ zl4ye|mJ8V!7lM)_ETn1a6aFJ&ClT7>{vwf+5}-mZGN~dM{s*K3y`8Vi@DKxZ!5nvR z)9nw;j_Iid>(`yX<%Ih!AY)JQhBbTcK+IaSFdnm(fl)Vi1tj#_&&TY0ernKL_x{vi z@T5<7JhbR6o^^Aw2d=Pftd%VQ3mET3f>H*Ckqg*RWnT#J4m43lR&*7S0lDb zWGuypFNRNrLs^{ z6_GHlP2+Vkij~ipkj|69Y!uI+U`7c(E(C|%jPXT2L>Z2du*L;oO!g7B6#ZbG0V6SQ zNddw|4ILKpiG|2E}6F@itVhMhZiAhF%A%p2Lt{4@s zj*Mfd#KyTC(0LMRjZV?O;y=d_V|r0=nEq?@e361n6cAHExK6COjP*${qYWY0CnGdNo5>fu)i%b9P!d9g8;y?e$CLDbtje`ohMb}v;f6)jK1yhr0Z zkHvD2-LH=2c1F+l#d7;*>>C!>x+Nc+iMQNvTvQb=sQr$}q8j|J5cv7lypq_SY?OCN`mW^s3ME&!XO zi9m=*e}4_*@hbx#eoi^&W^D8-bN9w;;*eD0I!Iu0+Br;%8JmL=KBsjMYYdJ(z$%4# zhlvHVCTS-?lU-U-MJ|B2q>9dQXj0c}R@bB$6XcK_)^u=ocMgZ))Q}}lU~G%uJgD%D zl4EW1WDafz%a+wGJc?z2s}LR3Q6)5$4WSY6T}N7WQaz}~$iYiLgW(!cz(TrsEHAbX z5*_+UHM}u8JPwzAR0*62bbTZ&x-c#GK z?;l@!)W}n`mV>*QFr4g@rJ($h}O{b8%YF0nmnu(}~fu0;-Pc>j5zJ1XfmkiYo3gdi*$$sjC~DmYxfJCr&@pcdnzQlRL3j>Y!vFx!Ps!v!Vyt zoQ9=kHK%1tyF@l6o%f{rG$c5%HjLqEFU^zgCSZaLt6j)*huY@Vgj82rw4f$ zQ!?bhFugcJ#1X9wUnzl-m)u37Scy67ZQ@c)5T=@5$A+9VS5#!EUGSm2E0Y`2YuKy? z2gBDs7m}heB_;>28TJ~cXtbKb1yDY2LIHwA z_M@OoxKn^cD5NA?G|q}C2LnIuOW0|u)r%lxJVVjbx_CYpkrh-7}UAJG|k(dTcX~)H<%j^BQJO zZ59?U^e-Na7gj@Yv}~?%#z|~F`^@np~brptk zWhkh3)v2A(s)?De#*m-Xn5lFnlon*D_7d>0n}JOrU_Z1?&PBb;^b8=*+}cF!)Z4dT zCGf_tCJZqU|e}IM(vvMqBy>g(=eBT9^NJYP8FBRiDdHG%*9FO?W#{5AMT{_ zAv)oaMrb6H+V_M};)!J9Yfh53hdMWNWv=vRn48XsQ@F+FJu)+$ztqg!r5QiknVGMh zcYt02{nkJ))yyjYRLB)3L5}75n3<*6-2hre+L@(U$+rCx(8`Kr0b0e6Kr8DX1zOCV ziQE!Ei@7!Vm7&EYL5t-nm$7ApDXvBEzT`2`+WAXB3+|MJm-*61pydj0muIZjSr^v- z68XJ)S8dTSQ_G}rW}nKAWDk|9vE(Ot#Q^lmwv07pROi5x51N%|)^Iv@<9CYDz~+S7ny zr6VoGv(3n7iL@`1HW{*Dg}Faxk{Rf-YtGM*$w1LGqltS7N;XZ{mO?Z$#j$M{Vn`cX z_7-*~eySX{lCY9u?nKDvhDx1_J32W(E=7)F$_CMu29exMK+al4P-5)9K3V=OR|%e} zQ6aHPv~vY3t5H1Bma%Ifl42g%m0U#mIV|=`Desn?OPg#LF~!;@CS}FTj^o+RsHeS$ zW#lz$g!?QSnVm`6YDCK?a+IIaPKPni5)LgCztlJO={kWR9A zrWxb=J$d|4`z0kZX8*0K%F!1^U2cn&zN|E-)RgsOa@xn+!pn+u5aazartfSYu5}8a znz2g*RHJ3bH0opzA*^LeH;P(yvbB4%8wG0gFX>5uqF1EW31n_&;9`dB>Xx0Evz%d>o_}#B(DnFBg_KcC@s}+H2U$3uff)BLxqo4$>T#9N$zY z&un_j1{KqzRXklV10(iI$%esPWM<2AF3Gu$5>cz*`nYK+RYpl8_L=nd5f3Lc9B8i5 zcezY982?F~^JtwlQH_)i-`bdjO$DD&i>{d5p47P#mr|XhlUcI!8+mdH7V5_T?Frqe zfR3s{YqYXpQ+3oXrS%#ff!N91NohFv3Q`v84tT4+P|;Xl4T)3oK^m!-H_5e=;jdVFMKJ>x#A=STjkV{YsCfcKD;Zn6}R;mkuBj6xj^8lSs27JUVp9%a1ykl&XH_-u)FO_G5dhfjUa#C5vW& z;x$E@jF5wFT`ZK!3-tutf}}YqQFbC!vq-HAct@swLqa;G_*lsyiELuCPkYQw9oKNM z9_RP~bczsw>d1ih2q_~%=pU6P`7Ed69NaqGmEpRD_%Ko%ga!gYSrW)3GE+J!iDHS+ z15y$-OBL{WvxQYcbWHi1seCi_9NJiI)%7MR8J!=49#++N64nbij1DMwWQ<93VMR>T zvZwu-bA3H$ySm7I&PmBdglJZbLyyO9c|F1jD&jv=o=6h*Y3OZW>5%G4UsWGZ*k#h2 ztbbvQJRlrI+ZpYgNE`4a%mY4#d&u~>563t^pxmV+N-5KXU82X?4-@k`q0;2^GPO?! z3=mNooOhKS^Q=`=rviP#Db(Zlh_}8DC^6lcqT-E$qK%T$jr_um!fo_FzwoC{iz6Ke z(B84v419QBRBu<_4(_skH8bY^}JrRR@~fwc%?OZ{9LU0e0+cROj{IAARIg2Zd?p4 z?fb^$H;=^}jhp+LRvK5n7(3h*-*;}NZEi=*QSvrU!dcq&jiGPWtvdGLP#fe2uD{hm zC-LC`yzlg^c%nx;V}~e7bnp4-`R=uzOVP`GyvHBy9*7RyjJj^EStcJkta*+ZJ5Kg< zZd+(utXj@k`eHou7>?J;^4zt}+g^7-gD1D(?(qEZ>%pHvE$V<&>+GDl{)JuhL$8x5 zs>=goz?f_I%u{5Yd~^2ZLfY-fX5qH4SKq5%6yk+&4Z&3H3mfLloBIx~xK=LTAAi4Z zy}3Kq+#TQ7GiQwERL8Pvmikw-_B`0PKk4^FCrTwd&TLf0O#*mp(M&5HV9@y5&4qBZ z>~L&H^QyCXqp4X=_g?>U>ATlrSqC<{-hBM(;OwtU4NCV1UR!5sxE|x*NVzy3DT}jqQIrzUNG| zq2q&=ANEJPpO1IG5Ix%$?Hi4{#?~w^!pGb?sE^+W4K;6(i0> ziXH5X@9l~s-!8-QM4yd}s3Ah_=GtJhq!}|YzPdzUlreb z;&CJ3qzcsouu5*bzdhD`E`IQQ^gwsCyJzjf<>)hf{DME)gXujHb=_RE+*# zqFfqQJ!q4p!Dp4`C}mmV_AVAjx}dY3_8fG{Nri3-Cn(Yp!0=ur9)SGk&`f8ts#GV( zL*Y(2_YtxjNr(M0G0n0h1^OJR&!JcBUS+LZIKX#k^zgBe3&l8Iz{C!C%IEdkBYq(r zQ{_yV>QPQ|sc_gXr^G?UL470x?)}v_j-W@P7TC!H4w{# zyMD1XnK-5(k7~|mk%Ht@ek4C4p6i(z{3>h7Z?NzV+T$XAwlIVCWPhgi6#Rap;%DJj#m}iCO)cEWiS?5LwQZ52$Tl`wU2uyo!o&gR=4P=( zlF3Diq@K-CVi*`qOe{>JP??Q5x*PiAIa9@&Qj^Agq*yLnzJmjvJNZ|P4wn!3V%kF* z-ZxWNk##_cZ9);4BJ-2gMf-hYMDvrfCo~nu;gWR(jOjW!SR`uk0B1 zMCU-Wx=8vxVkr{u7>Yt+sPm9TBdMTDQN-<2aR zqhcB1%9^o7|KtNI!@0#xQ^JK8iZThP-diD|k+LUdr|>kDVPgH8$l(3ZCgtI(ZV-pM z!E+tEvlJk<#czyRmBD1f0u_PCV*zQj9IvXkY?)!BidFf&TEgZ z&n`QoE#1((pR=u+3pWZ%qh)(z1^d>xL(9rhU_$I~RJZ zbcyL|V`{1rH=%c)+?X;{|BGxy;1Pgo|r`09~` zD@zT()wj8A$D(1e`^}D-vm4l}ytwM*HYysIM`9JN_g{>apNeL|uUz%gtyuoSdDDaZ zA_Q{b<*U{LXd^}8XRdJndSO$n5c{e7`gq|}GtMWJo}IH`nD;W^LszPFe)PTU-_Ly~ zce!xczA_lAZ;yLV#Bxr|*s$-)$zL!o1eVHT+iT;Vx*03H`Yc#i?Inw&0RHYn8#!f* zHe>#Q5!Ni^rHkO^qk8xV}gQKe(|DhHLr5suxxKGym5PF)L8kbCKdFjME&5uW)z*O4Ghe{Y!?+;jx2>0{T`%}E?)5qB4Ks|bg1-S zDg+&X9JgLcCxok3^rq`XffcG)Kb4U}SizmpLZamt7icg2lh6BpvfdHs%p@Bs`pvXs zjHwO|Q|+-ty)8(+U1B{zC8f|U(tJ&!Po(MzrPL0RG7j&MOW-ZEoLk*bcH!~ zL%`eKvnz+W;TY6ZS_edvyr&#UHB=_2RqsRP>a+Y51_dQv%6QOtxEZRHzpD3$V!Os? ze*WGKL*GffG-IP$ey85m(v9qDggDtsIq=+@>M$mCv??(*dRs#W4!xeI8;`#HqdJ{ zuRiNqK;`qsDaXqWu_og4gnmLN9;74Ry}7;>nCItVVJgHdxGL7}OJNb_c)|MwX%w zD>O!j)<{YutdY>efci)X`V!O#BEeNlDLMs6wvEzSB10RYhoa(`9@tcIGU-IR@B^U5 zK%7g~wv3+C1D=c`?qtMmA|pvMn6Rl*3q~!!qZgWxBB5{d8WILT5eARYe;EbC);J*e zUt|_U!tY~{u!GhUR`8nx=o>$O0S;7h0?=Uz_=W?M%-s`-HPHj0G{KSv#mWHd)hDc@ zV@L$=$vk|oTtjh_+uCoA^^CwNGcyvWW=JYIUP(j0OcW<WXOa!`oI@V7_e=l!+LrDut0`-u2>|)#94( zxR-msz2knvYD3#kz*2_iiM71yrE78LfvDxc!)%=+_s+4|V{!Yo>9$R{W$e7yxwLoL z8LK)T-_|nSHnSsUEZDF+?;M>yx=;t2X{kA8KZvOCJ6Jqlya{Woj@gd4JYO%oSNP_( z_pHk=u2m3a9(OctW|b{9ehhwZ{q)nI$TG9D_`9GW?kGAn+~4J?+;1mc5nraR~Ef%yYqN}pdcU=slKeP`Z`^B0?T&(Cx=Q}aWk#oD~-*}Zt< z{*^V)`I+{O?7X|)dG8`W=Z$AqfqkD>csDX1St~mf$421PW?sSFiTMe5ym2kH#`EfD zPQlpi)vjM`I`c_x(Yy)wil=8!NAp3#FSoB`ue7W*Msts?IgU#;Su3hvu86x2uR9L^ z14$;DKY8dvJw6E$0{N4}Eg2`Rx<9pAPuCm%)ZOAZwKMI59eVuuV5jwTwdsSZy{F4f ze}*^Jra$9Mc>ibRCVH+m;<+C=C>O?;Nb~N+0}j;1m=mv57H}$;P{!lnd*~-C+P28m z*rDKr7iwsdlJt!&)3xvd#Wv_WX+n%pKxZW~LR{QEX30_eBvG9(jadH+O#(Vh0JiP+ z46^ZCOCsX`=C|(A_2XlM1MK>XV?rNd|Bcv?2ghK4$#@lv^}u)yB#jnK6p#RQu#sKg zKgO6FNsbbfEx{hlK`nSxhdweWrt~03X#HXrZ5b3|1kO=XIrI?n)RTvYz`omhsoM#h z_@~D?!DOb%u6Lz7wijtnBKg~N&tqX@CtJdH8{{r?)Bqc9I74B|@mVTaj;~4P5-`>v z7ZL0Vdz2?~F2*j{a4Uqo+*0`tHEnV0U>GWrXd+BOqH%J~Kvb5(aD)1sU8%MVJY2VcIMd_;Ld!a3dD%ieBtEHR zme)4o5-!Q2*ihc-+7yvOBx(2chM5b~O7UebB4Db627>ISL z`lhKyr%-203U}cD1_y9}Bb}Nk_D+Z~LK7wi=E)s25<%+8xz&)RI4E~EbV5y>@Fs?S z(h^8Z8tn={+Jivx0tjM^Eh~DmF2kOM{GRIHG=9w>dH!>0JpcU`5>6Uw_Y8Eh^&VK@ z=XjIQUr*?7ga`qliERBB{+?o1FbxB_g3_oCsV+|x2Z0%&8$+c?BD();z7{)o0g};Z z&xQN?mC*f?`z`nT?;rT?3$f;Fe;*YRbjwzETItuwlATrK{kNP~q6M>F${N(*nBwwC6F)7=kvwo_5=8 zgzY7M8K<4972%__7dUQ^CyQ4qk7JM;qi5*?2f;2r!Mn0zPYJ)wPfnk%5*#YZ##}dl zO|g6g5=~DgGz z6&oW!N+ED&(KeZ&0a-v5X4g=m{G{HMnrYnNgty7x<$gea+xpW6slPE+$i7qMXhv-B zDJM5gBH?`6Z8ye&9kJ|42R;rF6g;OejBU?*NYqqo27#`D_6r9wgR&Yi@ za!P@k3}wH8mPY4P5w=l>)SBSzGx^Il$nKeqtni1CZ7|EC z%_F<`+(=$z8~DY9XxePNIsvhpyV|?T>3=@c%^nIM%+)XkV$?KIoEc8DF2ZeZjTQ-n!4*Yr)?}y-QfV zOEE@wK7NeKV`gesxLxU=yvQ!iC@7sO4NfAzGM3+-e}dY#qRzV>UuRyOfx9ESr4e5I znPIhC(>9FopyP494+&u^oLa!EIh2P_Do|4>2C!J(lh#lhFtbfVGwqb&eI4^3-YOLg z8QB@JQH$m8&^9Wcs)$sOhXi_0(zr;L`aI?3jr?xpt>7bdKJjTJ|4d$EC_T{-Q_O10nBdhuzIDpnt**QoT= zxw5z@IyuIGqMr!n(C@*kZg$FkDOJY>Ab?yV*^yF4Te|TZ^!F63vBT%&mQ8|Q#Ml8n z5=xsh+=eHZNRGU|f#!QMHCY{TgX*o(^sFM!#rxXXI)rsvpAIMiHUFi`sApv~L_ElA z56iJpo8za{aBn0#nckhPi?}1^0YeWcBt20{rV07UraU>6r&RCowN%#tfBqB5;aTh< z^tw6Y?M_jC2)0>=3|p|x&P-KnYDlt_Ef-%I$w$koBGt)~&4@?pEf3Lb&x2VAit4c= zmCph_QNVAfd(t>5Hll{m;j4bRdQN|vwE6I)VZ-ek|7%o8b$1RwUb1CUHP~c|y*ja0 zs=>Vu!pa(EBXlMzjzjVLMrtMm90;dyvSk;Fm+39oPO==5j#VR#Iw^B<*~&QIA#y_4 ze{DPn*J1eVyD&Jo0V7EwZM0z@0T&58GT0Lja4aVz8u3Hh4<5-H51f&64RPPU z%N-KUWWcy;tZh8YqDvZjoubp(Vl%nWNSRPilTaV~DKzG!8#*f@pOsoy;g5cHfUCD;M#WY18`=jmNJV>Fkh@{+4bw z?MaT9u!+#A@`hBZM5#QXMs%}=S!^p*e)fB5?_x2g3tJBMPhQZmwPeM+D z13@C4`9uySGW|FG{o{}zi@qxpwx?Py;6Q+$)4lCNC&dmRfKTD`EYa2R0do9(f_~Vb z3E&sR;}#`bSej>|vwxgjJ1sKI90SASp=*6&%E@YZIp_mhU7gV?KT;dJ7dw#9b(Hms zY29Bsb2c2=8yT)US7xug_WVPfB$m0*zv=MY>6-0YcT~n4m2{+8?_%yo!QsuqvWI3} zM%BkUV@BrB4TuU832O$it8Qm(Sh8_k*zVV_$1ElA3FdOorvKbUQ7Fq#LUxq*Rc}kV z;d|w#mU^>j5|#n!fqE#CsBFumV9dySCz1$Nn3ha~HQ^i%4EXzR_2Z0X<{eMa(iVL%V)?y`qCwV)6>Wy{aFY|B6CD?OPW824YR8$X z@`2didNwndZ*91Q#UUKfYQP~{A*evZrR6pF7?59NZRN-d|B?#$&j@fdDRX!r0z^1V zRU+fWggyXA1KOoo@S5Y`FS>B3u7jayVlQs@wsI7lZvVPfZmF{NptgFgM=?R{%hn`eHfgd~ta z5)wi}h?~INWG*%a+xQw|3^q0f+u+2GlQ1#@8^-}h!j7#G3E7=)aK^KNdZw0}WTVWa zJ4}-gv@_=n-RZXO+}7>xnMjsqtZ3XFcQ+rp=d`B;d)7^+=j{Ie&wJGca5B?xeG@;t zdfw-LfBv`M5pvd%BMEg^(@$oM5Fp?rRKrb&99}HMyA|IPdBi>8ITR#Wjvs8tNocq~ zM!~)!XVaMQPKZUvNcG;El*1pA^Clc0$>M#9cC^QRB-qrg1RO&`O=TcI;hHUB!`7I(9ReT4mG2mD8sB?)u_I`V+IIsN=&1-$ zGNGEbKVqrAYiWsCTB4TLyOteL`-)n2P4^19^Yhz5^bkAHr zUpRj`YHpuC__5yf#=)xxzuG(9b#L45*b{@%ZO4CMeR82=cI2(G`J=J&4N=>Mu=UC5 zJ~&GbMYnQL`3oD^LbG4K@ygsQp%=d$csKB^mlu3LFWM2d4NmvNxz|cMm3zOt^B4Jh z=H0WJw{)QwV|Et+UBdZ$rVqdY+v2U$@0_`LCVcGaU(}wueKhpST?l^HZC`95Tllv+ ze_nUsPVK2^`7=@bvw#IfOB6W&uSQjxeP3^YFR>`)}?K4MerB4?4pe zJ8rc`8}~=G`0&t#oF6P$?Hn(x?vRueU~!EZ%T^Ch)Msq0kUD{h{D zX;Bv~-aB*fJ~%$Pck-(~vXtEeh^zxTtLMVUo{5$|I}6xe#rn2z+o6c7_pa+m#C0Tm zbTItniKy#jxUe=}vmv~3Po$>fZcSIDrt8-5?UrcG;n}`;b@Tiyk?JSzR(D3KJ8yN~ zHb<)u%^r@gZ~ybw#i`rw=#C?i^+&>mb@!Ytq#Do_ZrByKH^iFuKG0-U?o-b8pjXN_ z{63*$IqTl}^35-Y8#{X|N1l$Bo|^5t=WJa#cuRB37wdILou_Ae z;$_XhZH_ltj&*m;m3&)T7jteoZX?4JnKPX0zF$$BR>6DDDgtkf)$dxo9CaR`qSYZJ zUzs!k|ES{$Xd9O{%pJOFyykoA7msGv83~ zBuR$R4MC{I_Gsxoim`5vMl(92Ia1Vew`g;uX!FAGVoS7W*Q`2jDGFBpe!Hw#lTqhB zzE~eSHv)5veyp3N4l8N6>u8QRn&&Sr3`ZT?XAXUAuM9Ou?Jdg+qroz#o>k7a#M?GQ zue0X|#yf4@ZygV9n?L&Q&RE&T+nuwy_fdqQg`<%g(i91A@4NkCWc$&G{phUbeq}XO z#ogaq7u#~^wmY(=AFAHbO#_kUfk@q8q;hc97%#34y$tP>S>Q9+ZW!ha&^ZX2?-dpY zYeMGh{oG5<&8B&Cw6Fmx&%rIs3mWF``Pz5SVeVA5MI3Dl?r32<7q}&K>0M}1G(%6L za1#JBDhhtB$T3@%@==u~rviQV{LSa*YZ0@&ZQ;d8dHc-%+3u_TV8Q};;^J{q555@zJpL%rEq3mxfeq z@_*t{DKzG<^zn#o;ryogvH4SrdAGW6wcp+rK0Xv4{8IQ!BVog6OnpXNqN_tsg?hq< z=9qfJ{XFaJl~7|iuQ{x4{&aatqbS+->2j6A_LTC|Wrw2Zxbo9P{+>@0wWmH^&cdSh zNRL(P6Yyd;{pR68wZa6<3k|Ip9N2K!v2nq&pjkM!s9f-c?K@)noj5?xy00FZ>Ar7W z2Oh4WQ#q^TJ0R!_b%b7=ube-=&<*jq7sKTpKR4|KS~3FtW=Tise&!>R*Ee!cs?|R> z*iP!yKX%ofG^&5%w4Jo5e`;EH(ysq+SsPE5sc)+bPFAV@ro6ZO^cx+11LV4<&(H)P(ljAc@~MtyPv8j!E@ z0_tS))wp$ycp`Ih#ojf@S}H4(N-Rn1@ue(YBR69p8>P%oV$ZMH0>gkmPHulvTu3JT zUCHL}@+SXJa_M~92{R_CU>nli7}PFMx%iVZ0J7)`Kmznvm|&=P1lRoG5f3Le)=v;9 zZdgo%eZ^Et8sZTR3I4pv{7az8@#1P10CPOT8-;_00EE+q-QzGa+nmpt+6dbc!@f|?s?8Z01>Z}DtBL6A(V_{#;9ph-kHlP3(rVw zYlRo?LQINou;*~sz+lJWBk83*a;yjPS0|l^x=uRlxn6@ItTbApWJizsPWt5^;0H`k z-@dLBDFbHAb9v~(=vhXLWL6(n8bI}8PtY)T`JZ`=L{Y}aNH{}H{3kk?fkbYIuRN$b z_U(f$zTV@9`%Fqh(Gw5s)WA^!5ATPur2Fsdk~P4cv$LtPvP+1Lj}zY9c*;uj!B zes=#q?9*a0l=P~UXkjr(ib1hf8C5bW^V$?@N{gP9Stv_`n{XK@)pFW3n?33CK;(AN zglUADmDtD9V5i*qPOSkk?ZPPGg0sI#<2TuduNVT0fzqk(lrkO!N~@hbqEJEj$<_mo z8c&gBo-0%nf#fTL%(mS4rDm?uOi_)@5+?N|8^URm1(_y=lI*_LC;3z5)-uYrshzXQ zCM!i+8ktSBQR?&vrrNLr4#PsIhop`7Ve9JY>m20fAUG~vX^9GUWohjpiWB~GXkx`)B~v8W+27aQ z+k-|}WiGm$`}%2TC0lJ`j-Tl|xFd0ig6WXR^KNWkVO(hL!X))Rjvtu;_oYhtAm z%E)p1xRm_Q?(6F9!Y-b9LuM@PW=gwCam-px2Dhw&958Pg;i$wG#nqqh=t^Q9s9 z0H>aR!#BV{^R1Q0zQ9W1qw5hrMmVO5nGYhJqP#!G9pQfweewieK%SISyn2-ydvM)- zPGEOQVYKQWDvUbAQaK3$lMHX+SWJ5^&I|MwlJP4(aJXn4b~tV+eAKC=5-cg03l-U-ir|ZL&oiSBLb?{K zeyLWG@0fY+VVy-+wzQs49*JvN8Lt@txJ>BG!ma#D$R5fKuRnbI`P#Gc>>iE<|7aNMOr)OE@HZin+jVx()Ni)r+boP zlp|T*)DF*>{y##!myxC)N5;)97cMysU< z>fj^r#T8Cam1YImx}l|w&#QeK)vn2Xmb!vU#+@T!plG~_*A#S7^{5Dq$O-iyaYJQ^ zEynDoh+_+#0EulZ)E;d~S4Q73Zl|nhsNM^FD~x$rHcqbc&4ArRsp)7W+CyS6o|BWv zg?bjB(UzzKK+>QWHOAGIyrq7<;GR6&?CfTCx7juUUMIKvPL;c-^t0?e;vnN$$dPQFb^$8F4QXZ_YjXG>cn5JMWBTkx-aTk=)rw>c2^ zdH*X8Y5q1RhZ~T*DRS#rebR#M0V*cQcQ_3?y$7*r1N)qglit5XGVCnr{UP3%S{}73 zcStN0H#`oVfGzdjpCaH^P^5i6B!F{bsf;sf(>O)W=#rukN3wcFzIEp0Lsy~BE<2Jv zYC=e${%Ld^$5h~7UkTcSx#22U^?f${2 zO~;1hqZ1uS7$4wCxUVT+)4i7aTHb5=wGUCR;}n~bwk=9mK`JYA2pLCUg%EO%fMPXj z{)&0XI0ndrj`Jyxa6I#9NI-@78!){H0b6gy8pkzJc$N(Kcyj1J=_q(5$D@5c*Hi1! zeMO0bV(#lO68aUz>$(};OzuqHjDAKr3ZqR>Zh=wMDUHVf>bM4iwrYyv?8Q+jgU4L5^6IZyQlaA$NT)TH)H-KENrQ# z%u~8js!+Jd;Y(W~nQP)}@x}Vn`95Y5J*>wwaZ9w|maM|}S@RfP$#^7HZ^LjCZ-g`3 z1BC<}&8y#5LBzd=#I&VSr1yO4Vf{)De^{@(s1uGdo!lW#JdaCfS6nh01TI_$uR3S)v6+ON$sfX2!{-IN8wk|BS=c?q)P_iY z?L1(nm?Zzt1d+R@P3>q4$gWZMxagwC5*xkb#xQOHv57_6AWMK< zC8@^1AVwT^jXF&iV$T^gD_VoeM>a8hY8H(pSl`|5YIZVyA4KY5vI=48I^#omP{d(S z&oWWFP$IM+yQT3gL-&|)b*Ga9J*zP9#H595-xp=taqA0bE`k_ft8nAQ9^lPgtPz~M zg)@O?BJ8Ep=sDOP-g0XShi_9x4j&gotoviddBHs*XX|8Q1aiYi#$C)5==2yyz#3t^ zFTj$>8e#bRdNLzVY<0n^2|S*v&%m?H$lsIoA%Iwtjgkn{lN(XeFaHra+e~ayaX78P zF3Zql{sUhyMVgXHjplFWud|)~$NL8B*8>FYSpVUb_cKntVhd>1LOnz}vHKDPaA{AW zPtvZ+bpn&d?Q?$$O*4LV0@I;q@BmE4j6Y8XOQ28#n=Z+`53Xopc&=TE*K<~2ADvAm z<`e%&<|Q(a&hiWz&Ld-jDVM2~l(=;@eO_^9T?j9YcwcD=*C!Q+QC^yOQ!soS9&U>2 z&Ndjbz?_~=XU9M$<_qK7;MVrox$Ny4e5$Lf&j|z{+*Y?Z2M^!}N3KgNJGC}gCSN4^ ztt|PLRzCgqCcw9)-blI5{DK*@w6PpEZ4+v+xv6#I?=}m7W>Fh$7P4qAW)Terj^UCNPvCyZrUx63E6^%r>!A-K z=#%XDv(gN8u1_Db$NHZd!b-EZ>lmn{{ZJW5ncKo-KGJb)uxGFbl!d(~Ge}QJ@K-!8 zVVY?pAt8pU=;g?!f*)&JX@$wqAmXHRrpyg7Y{r%t8o>c&#pdvx=hi3uOX`m_vjid!W`*7!_oB0a5+y#rq; zIrnMb5KB_*3hGwYrLrOEJD>DfVxzBg=K^|KQVdEIqmM^dqk9+YJvRH(kk+`*Vh2)Z zB_+nqGNj-gr=zdg2Rq^P`HHYCg?9emj^K&ITfCGTGHlujeE zqHu$M#B=e&ibMHdZ)P`g8y?Rb5+!tgm(x78s%tj=c~3KI6RuDwp+}mm{IrADzv8q@ z%ka0Gc3&j9Ri>S6yqr&#t@s^GJCv0~H)zw%H0|V7Sz+2CNXoRMxxz`9;9JU4ycW5f z%`SPa$rEgq#V}>Qi8FVYr|B^w8p%8}J>GqI0j)R73m_RX-u-qKC|y~QTUNmblG36N ztZN1opoM9exx8$WB=&&DNABA?m{HaC`Lnr&!P|^9{W_7);I9_TK<%-@vD)nd{h1a5 zTx-%-aq)hD0J7_5rhpy2T?3t6^#tAOZk8`ZDImPur!`X$S6kKrDQO5F(^KhqazAA+ zY$9xj!UZ)wARVr`lWv9*J9j%bCN&;4AU*8p%tw&&*uWq&Q)=yG=R*>^b@FFHcBM-% z30Gvc=dt3_Jvcqoz|mua^%V3-|5G?!xtqCt*SX2%;?6+08K=iuy_N#3TJce1ORzR%4dq1a^$Xsao{u(d54u7JLfa$u z=37}aJxf_xcJqC6@oesr!cbBGZAVM-EF_nd4g;WNW)9)aE_+*AS5RnLQWO|W%_94h50EKH7O+3PZX?MjkJO@GLx)fB(C(q4dVDDlzZiCO(X9o7J%QsX$j!y zksy83{BjZ{DMOO7-iM&_qFM~Bei@gq#Cf1qnH ztyyL5h5e}OL}%~ueO>#SW!~v^PH#nd)by__rI6g_>1iZ?PfvyDh_W(ET_*QkVGU)p zLs?|0kL8gw_qnXg<$-11Y$V9PVPK?}Uru-2#gP}q{zX+^xk8Jzn1=Aij{ zdsx3^p&Cv)lo44=XXlg4$mJUv*cmZpvM_QrKcGzE%E6#7!cx^z9)aHrGd-{@Kw4nW z;1o0kyW$d{CCpXoJuod9(7n1opntVqW)EG|gz_kv#+Ad9CHpFZ`N;gl7-Tk&j7m)z zR;Y#fYC!-zVB(lwg$~sFJg1ZW5;f7-FVVOZ`z7Yee&r^`6AgT^Z(>>4FHvV}R4;IN zNx#dZVjSA6YjZGLm4!2%x{ncDnHe5tl?SGeWrDY$nzG^GvH+WzaW5HnfuTPql;hrS zpa&8;v;%=$;r$-{(#xlD!pQv+CS*c|x#lHXzzH=?oP?StN+M4-1Av2Z$?c6$5G}EM zhK9j_BSR^iAp#Q4U;?AGNJPIq77@6GWJ<+fzw)R}4w*|M`0tPtEThCZ zeb1ma)Oynx)7L?1eWrs)P`Lgw02d5Jv#R^;J3dh<&0A)Rf(@a-!gCOZogE1dhT7&! z;x@-D5FG4bOZ9w7*tO@!$8Yb5c0Cn-`qZ7SQ*lSxdz#S5xAY%eTsRfo&=GNVL>wKn zov^iXBX2G*crae*2p*a*T4?yF&^8;07FNG|dcHhb+x}-&ivvHX`>Rd2Mt-!7K22Uf zaPxGi@XfQ|$(rBse*T{oEO!2&2pGt&y~>9QmAO+11$Hx3ap$t^wJ=)TAm&tIOFaMC@{pPv%VmF4~KC>mEjy_^l~Hy|nE33J4Pgn|7K3hCD4 z{3Y!S9E*EJDhpGFp;7{G=i>NqeI12|X?U@2^o9CK`!bF27WI*suYBYj3DH;$OfYFB8HuE6e=e5a&b zdNW;|lj8ARpAtvDR$##L_4E7djmqM1CHGhdRET0)vW$<4iLvPrcP z=Dstv{XSn3X|2tdED2D9O-OD96U!i{Ky&L1PQ@bWnXM$`A!e zxn`3vPBY&bc@h^>DPp{Ql3Ptmi_+t#BnXPE^kR}&Uen3{T#|C|4Ct~ksQ`3Ox}sEC zX2`Gqm{T4f>T^WzNeU^og};qJg-H2fx}iN&O^IJGE7+WSsc zU!PnJ%ga=*{(|-?$?O=@z%oc8vk8NMN5+h7c|WA!zoKoIKLm2;u@R5dQ#`>Um+N~X zP`T=2cwks+4=0anNqm7@HyN z=GlWCck+Nc#$!($j_Lbg9^}yumg|y&`*TFOvrs>yeXuvnsxvJ$=iBr%xyw~b-PT9h zMx7OkWN@4cN6CzFDOc?<(;tPd=w|bL@ke^Hb@cLL4nVjbsFWppl`vGI+p9znEf>Ur zAYq$petUQi0bfv{Ct08;5$L|whEz({4pPK|G|l_tr8~p>PdtRdlBbmqR9PjbkXDxN z6lDn0e^jP&D$_*hm0J}t{ZYzpfMqv8L5w9-wWW3zGVI#6xIOCVj%d57B)s;S*rH2{ z9O{W2^n^)KVCA*SgkT>-iB5Y}8@AMh#sTlU@LbruFTQ!lqWe#uo;@A31+`)4#z{WGFsD4=0*@izqt%d2}qwa3o)d>V%@UWsGC-T z;*zensI5-sFB))>`H5Fyx;`U(?%QH0i1QLXfjzi9i{@8=tsx>{oD%`Qgpd*n!4!b7 zB-Gx35@@?v8t7qHbsdC>$pmf6Bh(lMaR@7xY{I5I9z9SoXoRpws_kT@;CHS{c$5Bm z3<2esC^+vivatX|D5E0k1up2h=W3)F!kbVckBNz~=XuQZpU|U-rO1Q-j;?v{WKZlS#O&%n~Lgy;*D1HScuTw#!5gn*PHYh{D1m6554zF`2;LxWHb z>90^qAU!CbUFNNn=TDg=d))z($4d1Ua-TBGkb+Gf8+{WI8u$`U6$DHf5iN?uR4npT zGKRGaVbfACM48y!nX<~zh)n^pC#P(an}s?Ctgw@AgNVGE^O-6ld=b?4GRQ^tfHq*K z(Rgl)5MKH_RUELqs)3d4VhRDKoq5U;aJ-rwaKKJD`qZnuR*l~!0mo#!5bJpd{8gUf zsgiLs{I{q0JK(PgSm9=o8z}LVTvB+y6tH=p3)nEKN*#)4uA--1PYk}yO6(~e2mQPhlKAtN008t>I z$}|Q7_H&(5`Gq%O%z7kjifY))ul9qaiL|RYEAHOxz&g)5rf#CV_RsiUK=quefiH(p zk>#mD-nCP;K*i}2(w6=P@&e|7U9#sdYX{M4zo$-UiHzZDg!YoG+XqTu7k|Hyi#$y* ze`{%+$@^!G^s87LB$hc&KsPE9EMb{ldF{(;a(@jt#no3Nmuz(v$(mT5*2{O!fZls? z?G@b9FjY5sK&bDzgTkBi_Z*#LQ=ULBPzm&&#z{K&vgd;)nPKYR?fToiLTxw5Ydc8? zVfJ7r%oBPYYdgQi=HNX%UfZy;%cf5_#m!Is&PVK*ve%r`dVj2UP-! zCn(s%iZJv^m_ciwlWFqE^;UdjXqdOWkJQ~4px|?vA!`Hg5(tcx62otHzQh0n&$>}i z7`-4<&(K^zkvSc6R!WqSBeNhDRLaxzcLe1jp`t#~+y|vYw=B08_MxHi&si(ut2lWP za*Bc`kpMa{h?^kOKIe;f8UU2gqL_5nkBv8R972$e2;UIo$Pwqn#mT0LQT~G{*v*v| z2K-4_{1Q{pf8ioD=1-5n#5O7BFj#S?05(ny3tf3!ztL)>IwNaZPTSnC-^&AGlJ38M z6W499q9Aq@>)VyO0XDteYGI7fIN?bEEF*Av(f32cAfuiE8Uj@L-M|uuMFYVH@~uE+ zPbPFB4PrWvO?)1XWv8nuG`bHO-*mmnDY%VH6z7#SJ;87VCFl2{+8+L942@bxF5 z-b5y$_}oR!^0c;v;J?g9cq3<-9aMjx561o_;tCZ^&YkD9J7g%Ua)y0!v}0W`*U?7= zUzzEkgn_09;j==w=!(y8I!uu47y{g!0p+Hg8>uS|Ovb_M^IJQivqbY=NRDYX`z>hh zF$Us+Ppp-;J3mvpIAh~)Qx8Jl5@$(rL>v0x0*db(GW9xaE8gED=L|XjiGtH?@P^1S zkn;{X|DL`Mk@G73l+5er3d<_&{}f4O6qXxCoN_;nI4h2T6<4UF7}PzY$Oogy3t2GL zEg8>)7uXO#uW0*A(%SnX2pm)(deSU_CD}Dg9w^E999(=RMf`=$MNu05`o|O}ae1#( z=lu-b=&$0)hS+~rY=8fINj<;|jY#d9N)m-QiF}TNj?2#>Zm#0D1X-LuOFh7xW~TRj zBWO}sh1mrGnWO@6cji-^eHKc=jt zNW@n3r6L@Bm%`-nN1!3Bt=#fKzy}i8LBb%8A4&vbZde-;cp{;t;|xVh!btPE;4e(V zHUDySX`7geaJr!7$L&?)>N7@y27R~<0@K*0D5-eIb<_1`Ls(rTn0yEX4+ozJuYV?N zcs8bf?qRmVQXBF_3mT>m;oV#w^eic|YqSp(*=F5z&%*+Rsf@=wh})f+a1B6aHQHv+ z248|<{rPPXTU*4i`L1Dm#IQYT*g4%T+Rd(bPZjF^R$g!l5_wIIdycZ;**BX$ST{fV zenZ5uIb6JDVQfjIGw94`+nD>% zkf+OAvMWl;-)X$r_-6CGIpS!T$$wOWgkg6h#d=0KzdAG&uHK9aZK;Y^Z46hoESRYd zZB&KLpqrZtrVo5vQ5||VTG0yA+_ueeM+uM+ifZ1wvfz%@wMT93h-}-8YS16lrKs%2 z#N0&GR`)>Bsnq0W!Y9atN)We|-FSZP`Ox~Pt!XCjp2Z$4dZ*-ONw{W5w0P&8;*ML& zsAVrKvx(3KA6wVYZ(Zn#T6fRnQoq-Pj>E|P)3+Q^Q}2xCep$l<#a3PZ%;Ed?vXCKS zZ@OF58Yya>IdH#qcewRnFfVlWj=ed2WDq|2MdAE)^V)@$Ki1zj*-;0R9Ug$YcVn%tm+w_ng<3*qzxe`;z*n@*>f?^uc}v96IGc07qJF;h<}2T;`hM-7)Gn@HEV=bk zWLsZ!Q-7qqKYZ+|NcmIYQ^VoWvB;^hNcp+hynFVNVCQ?*(3hi?txz;_zJrS)V_==+aGY8^&^X!R;zHFw8Kp&K^`-7wN*>}pD<0kukTS;(a?uD8C@zSbr z?bbV`TjvHA2Vm5l!$OLiDngdf3+UFbj|97}pAU6JO|{&w{ny7s?x?9gZgK>>`2eA# z^R{=Nj+AbU+FBM`qNdF&`NvJ=OZnO|u4GV zE5h|pgbjOQ>W)vc6^K-Kvs;~aW%@@uJM;ru)lc(m z16K8iWeo!kRb+F;K&>jO)eIEmL>-!ebve;m%>az#Ah`IDyrT;L9!!58j*mh@KNn~% zQ%aeDphz<(1m#IA``mL*E4)d6L5IDydWcL$>Lug+%)AZ5nUrAAR!{GJY&!Dv0DXVscIdSlA1Q<5lRWJ6!H6j zLYew_M8Q$iRI?gqi^s`UrvmID=M3B~A!pR2Qpi){1x;m7(#?=PGT$5-lB_rffKXp2 zg_XD^NSO{;>JZS|2;yK^mSR-^YylR6- z6ifzChRB~zhkaagR6h3nT4N>}0tSx}^yVaRiZ$Yx0^ps#Ei-nuMtJk*ZL!aldO<+{ zxx!llxt}Y%Y?Vc?tg)7^TXVgANs*Z*Kbv)}TSm`kfGvi=i!nL>Kl!@&8a7K5=ho5X_&sJ#=v#kyT2o6RO+wyZc)Wcyj_`aP~s zZI^Ks*D7nxq49j6CCl4QMR>~NYN3ZDDV0ooUD!$H+n8cu7c;$VLXQ-1ZxY7wf7q5z9a(myy6Zue2gkbjAzd;6?d}0icuzA@ z;z2cefKNtE_FKiy66#1~JntuHP46-szn)>WGFAh+7{%{|`Qi~0fyDa}svo_ZoX%rp zbdHB?g^x4}c%xyXus1Z~+3p;4DHGa}%frw|7@bIAQBcu^=dt=u2>cj*$mPF62MXyh zKVxI`6|5keV(^-{Rl zVB}8%kWFYYwpulsIXp1T|Y(5cb<#@VZ9-x$9-9(*QhTo3R| z7ib^fDf(XP_k7>K{3n;cyX%f?&o$rITW_?_wFf&xl{X2(G?rg;>-3CTxY*lcT8Hrd zM5rxPJFf|wn`7Dyf+q!7%2Mfy@9C?cz*HF1H{a)oao2j|S|fi)pE(gz+XQlgMy1L*nuTK>>oq4FrG8j=sTk(x&=AOC!T&N>rt)1z9 zq(eB8bmal1(HuqV=c@sd3R;cq4ZXdd=$6g1D_} zNrk`ncXZ#~(Hq&(8{N@&x4QpN+7?~tNY`7`tR;J9@%p|y6368 zt`mQzUQ{l&fX?cEA7F~#aYbDxmUEEM1BDuBi@A!rjo+U5=ET=kvpruoF1CP{GQ}F!{+vwcAL=oJHyVF zaK)yuxizM36Fj#s^nAbnPy62*4!PbMzikee9SWO!W7@;w_u%;;1aTW<+9uvBY*`mN z7}^;&6a1?Xj18>yRIJ>qxnI-&^U{~I6c`-9#{F9FXOC>?^-mOV{^h|lN=3~P<)^=N zop@A?s(hk=^P7k53cYn%Vb&DhOX(a+Uw2g(E^M4XI^P*C*tpm>ql;om%*OsMh22hbB$GBj-aa^C{CEt5btliCA=akCbYT*61RYMfBdTM3~4n-7$^kPF5D)@2HjZuCdd5 zS@y98#hrB;q;{uXMehv-$9Aa_+97~QkBtuvF)d;5-(%^6R$VrhHCJCEX9zp@Fuu`k z(3=ZSZpo%iBaw4{;tXtXp79cSjpLPp^??-|Sl}0~Fs>FbSs(}F_IdYF5MpN~@-O(V z3{PA*v#A;ShhAukTI3!h<350I`w~Xq@Ysbb%@Y^Exr4P%FHx=%C0P3>f%*n86DwCI0G3H&)Z-zR6096HV=@(*_qBF9k2{;r{Y9VZ85JiHwgk@4nUr{I4< z&RgUhq7OcbO<>()!<`f3qhn_h>Yl#tei&>Rne0UaVtZrzD%*k?Pm=awFBxF)zCw<; zmnL+;{PClOAaxX@FKlmR987|S=a`I6dUKI;fSl*Zd6}GPau~q;ReF1!9JU+(5xuc3 znQhZQrJw&u4sG&mf1@!n4u}bigB_GFk>S`L_?dPmdgJ2xk=bSBV$QA*J$@ zEWJ|wpj4sE{iOo`=du5dzfv?u6wUuzQSeKJ<(G>5Un*?BR5<=tQT;2$9<0hr-Cczx zqOjamR7Vump<~~E8r1fdXJd-z;tK8Ke>(B*iO7bNF~!sAvBXbUev3=O3M?n~u%aMt zEh4wr5mwmZWfftCBVG!(I9^aVozFRD%d|0GR2)_mf%H`TD<@V*=dS&7AGfJ1zviuympP)NwrBan;Jy57Ds_Fb?{SlSY zx_myn1tHHW%az5;dzBlM^~)Y*nX+PekFo(@Pb-gSDNW0l)Q6SIhUL+$oywidFDtW@ z4FL2t7sibRaf6B60&6_K;1ffRC2Kk#+8V|J)Y4#@&Rx!~Qd*XPRBuN;3-9+FjyG<6 zpz2mBH!L5^KBiRKm(OL@A^$;Tml|b%a7249OIfn~v}Qu7v@QYYSFNA%-%%BRq}Kjk MH&}3L2OC2FAApj46aWAK literal 0 HcmV?d00001 diff --git a/inventory-server/scripts/forecast/forecast_engine.py b/inventory-server/scripts/forecast/forecast_engine.py index 22abcb0..3e10097 100644 --- a/inventory-server/scripts/forecast/forecast_engine.py +++ b/inventory-server/scripts/forecast/forecast_engine.py @@ -634,6 +634,52 @@ def forecast_from_curve(curve_params, scale_factor, age_days, horizon_days): return np.array(forecasts) +def forecast_preorder(curve_params, scale_factor, days_until_arrival, + preorder_daily_rate, horizon_days): + """ + Piecewise pre-order forecast: a flat observed pre-order trickle until the + product is expected to arrive, then the scaled launch curve from age 0. + + The launch curve was fit on POST-receipt order history, so running it from + today (while the product is still weeks from arriving) front-loads full + first-week launch volume that hasn't happened yet — the main driver of the + ~2.15x preorder over-forecast. Instead we forecast the slow pre-order rate + up to the arrival date, then start the curve's day 0 on that date. + See FORECAST_FIX_PLAN F4. + + Args: + curve_params: (amplitude, decay_rate, baseline, ...) weekly curve + scale_factor: per-product multiplier for the post-arrival curve envelope + days_until_arrival: calendar days from today until expected arrival + preorder_daily_rate: observed pre-order units/day (trickle) + horizon_days: forecast horizon length + + Returns: + array of daily forecast values of length horizon_days + """ + amplitude, decay_rate, baseline = curve_params[:3] + forecasts = np.zeros(horizon_days) + + # Clamp the arrival offset into the horizon + dua = int(max(0, min(days_until_arrival, horizon_days))) + + # Pre-arrival segment: flat pre-order trickle, capped at the curve's scaled + # week-0 daily value (a pre-order day shouldn't out-sell the launch peak). + if dua > 0: + week0_daily = (amplitude / 7.0) * scale_factor + (baseline / 7.0) + pre_rate = preorder_daily_rate + if week0_daily > 0: + pre_rate = min(pre_rate, week0_daily) + forecasts[:dua] = max(0.0, pre_rate) + + # Post-arrival segment: scaled launch curve, curve day 0 = arrival date. + if dua < horizon_days: + curve_part = forecast_from_curve(curve_params, scale_factor, 0, horizon_days - dua) + forecasts[dua:] = curve_part + + return forecasts + + # --------------------------------------------------------------------------- # Batch data loading (eliminates N+1 per-product queries) # --------------------------------------------------------------------------- @@ -651,9 +697,11 @@ def batch_load_product_data(conn, products): data = { 'preorder_sales': {}, 'preorder_days': {}, + 'preorder_arrival_days': {}, 'launch_sales': {}, 'decay_velocity': {}, 'mature_history': {}, + 'dormant_rate': {}, } # Pre-order sales: orders placed BEFORE first received date @@ -677,6 +725,39 @@ def batch_load_product_data(conn, products): data['preorder_days'][int(row['pid'])] = float(row['preorder_days']) log.info(f"Batch loaded pre-order sales for {len(data['preorder_sales'])}/{len(preorder_pids)} preorder products") + # Expected arrival per pre-order product, to time the launch curve. + # Prefer the soonest FUTURE expected_date on an open PO; if the only open + # PO has a past expected_date assume 7 days; if there's no open PO at all + # assume 14 days. See FORECAST_FIX_PLAN F4. + arrival_sql = """ + SELECT pid, + MIN(expected_date) FILTER ( + WHERE expected_date IS NOT NULL AND expected_date >= CURRENT_DATE + ) AS future_arrival + FROM purchase_orders + WHERE pid = ANY(%s) + AND status IN ('created', 'ordered', 'electronically_sent', 'receiving_started') + GROUP BY pid + """ + adf = execute_query(conn, arrival_sql, [preorder_pids]) + today = date.today() + for _, row in adf.iterrows(): + pid = int(row['pid']) + fa = row['future_arrival'] + if pd.notna(fa): + fa_date = pd.Timestamp(fa).date() + data['preorder_arrival_days'][pid] = max(0, (fa_date - today).days) + else: + data['preorder_arrival_days'][pid] = 7 # open PO, expected_date already past + no_po = 0 + for pid in preorder_pids: + if int(pid) not in data['preorder_arrival_days']: + data['preorder_arrival_days'][int(pid)] = 14 # no open PO at all + no_po += 1 + log.info(f"Batch loaded preorder arrival for " + f"{len(data['preorder_arrival_days']) - no_po}/{len(preorder_pids)} via open POs, " + f"{no_po} defaulted to 14d") + # Launch sales: first 14 days after first received launch_pids = products[products['phase'] == 'launch']['pid'].tolist() if launch_pids: @@ -694,15 +775,23 @@ def batch_load_product_data(conn, products): data['launch_sales'][int(row['pid'])] = float(row['total_sold']) log.info(f"Batch loaded launch sales for {len(data['launch_sales'])}/{len(launch_pids)} launch products") - # Decay recent velocity: average daily sales over last 30 days + # Decay recent velocity: TRUE calendar-daily average over the last 30 days. + # We divide the summed units by calendar days (clipped to the product's age), + # NOT by the number of snapshot rows. Snapshots are sparse and mostly land on + # sold-days, so AVG(units_sold) averages over sold-days only and inflated the + # decay rate ~4x (measured 1.353 vs true 0.332 units/day). See FORECAST_FIX_PLAN F1. decay_pids = products[products['phase'] == 'decay']['pid'].tolist() if decay_pids: sql = """ - SELECT dps.pid, AVG(COALESCE(dps.units_sold, 0)) AS avg_daily + SELECT dps.pid, + SUM(COALESCE(dps.units_sold, 0))::float + / GREATEST(LEAST(30, (CURRENT_DATE - pm.date_first_received::date)), 1) AS avg_daily FROM daily_product_snapshots dps + JOIN product_metrics pm ON pm.pid = dps.pid WHERE dps.pid = ANY(%s) AND dps.snapshot_date >= CURRENT_DATE - INTERVAL '30 days' - GROUP BY dps.pid + AND dps.snapshot_date >= pm.date_first_received::date + GROUP BY dps.pid, pm.date_first_received """ df = execute_query(conn, sql, [decay_pids]) for _, row in df.iterrows(): @@ -724,6 +813,25 @@ def batch_load_product_data(conn, products): data['mature_history'][int(pid)] = group.copy() log.info(f"Batch loaded history for {len(data['mature_history'])}/{len(mature_pids)} mature products") + # Dormant trailing order rate: dormant products forecast 0 by default, but + # ~11K of them still sell (restocks, promos, long-tail) — ~11% of all demand + # currently forecast as a hard zero. Load a trailing-180-day daily order rate + # so the dormant branch can carry a small positive rate. See FORECAST_FIX_PLAN F5. + dormant_pids = products[products['phase'] == 'dormant']['pid'].tolist() + if dormant_pids: + sql = """ + SELECT o.pid, SUM(o.quantity) / 180.0 AS rate + FROM orders o + WHERE o.pid = ANY(%s) + AND o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - INTERVAL '180 days' + GROUP BY o.pid + """ + df = execute_query(conn, sql, [dormant_pids]) + for _, row in df.iterrows(): + data['dormant_rate'][int(row['pid'])] = float(row['rate']) + log.info(f"Batch loaded dormant order rate for {len(data['dormant_rate'])}/{len(dormant_pids)} dormant products") + return data @@ -829,11 +937,20 @@ def forecast_mature(product, history_df): # Not enough data — flat velocity return np.full(FORECAST_HORIZON_DAYS, velocity) - # Fill date gaps with 0 sales (days where product had no snapshot = no sales) + # Reindex over the FULL calendar window ending yesterday, not just the span + # between the first and last snapshot. resample() only covers first→last + # snapshot, so leading/trailing quiet periods are absent and the Holt level + # is fitted only on the product's busy span (can run ~4x too high). An + # explicit reindex fills every quiet calendar day with 0. (pid, snapshot_date) + # is unique so there is no duplicate-index risk; do NOT use combine_first + # (it keeps zeros over real data). See FORECAST_FIX_PLAN F2. hist = history_df.copy() hist['snapshot_date'] = pd.to_datetime(hist['snapshot_date']) - hist = hist.set_index('snapshot_date').resample('D').sum().fillna(0) - series = hist['units_sold'].values.astype(float) + hist = hist.set_index('snapshot_date')['units_sold'] + full_index = pd.date_range( + end=pd.Timestamp(date.today() - timedelta(days=1)), + periods=EXP_SMOOTHING_WINDOW, freq='D') + series = hist.reindex(full_index, fill_value=0.0).values.astype(float) # Need at least 2 non-zero values for smoothing if np.count_nonzero(series) < 2: @@ -956,9 +1073,24 @@ def generate_all_forecasts(conn, curves_df, dow_indices, monthly_indices=None, today = date.today() forecast_dates = [today + timedelta(days=i) for i in range(FORECAST_HORIZON_DAYS)] - # Pre-compute DOW and seasonal multipliers for each forecast date + # Pre-compute DOW and seasonal multipliers for each forecast date. + # DOW multipliers stay ABSOLUTE — every calibration is a multi-week average + # and therefore DOW-neutral, so reshaping by absolute DOW indices is correct. + # Seasonal indices must be applied RELATIVE to the calibration period: + # each per-product calibration (decay velocity, mature Holt level, launch / + # preorder scale) is fitted on raw recent actuals that already embed the + # current month's seasonal level. Multiplying by the absolute target-month + # index double-counts seasonality (~25% over-forecast at the May→June sale + # transition, worse near November). Divide by the trailing-30-day average + # index so only the seasonal *change* from calibration to target applies. + # See FORECAST_FIX_PLAN F3. dow_multipliers = [dow_indices.get(d.isoweekday(), 1.0) for d in forecast_dates] - seasonal_multipliers = [monthly_indices.get(d.month, 1.0) for d in forecast_dates] + trailing = [today - timedelta(days=i) for i in range(1, 31)] + calibration_index = float(np.mean([monthly_indices.get(d.month, 1.0) for d in trailing])) + seasonal_multipliers = [ + monthly_indices.get(d.month, 1.0) / max(calibration_index, 0.1) + for d in forecast_dates + ] # TRUNCATE before streaming writes with conn.cursor() as cur: @@ -1002,9 +1134,33 @@ def generate_all_forecasts(conn, curves_df, dow_indices, monthly_indices=None, try: curve_info = get_curve_for_product(product, curves_df) - if phase in ('preorder', 'launch'): + if phase == 'preorder': if curve_info: - scale = compute_scale_factor(phase, product, curve_info, batch_data) + scale = compute_scale_factor('preorder', product, curve_info, batch_data) + # Time the launch curve to expected arrival instead of + # running it from today (F4). Pre-arrival days carry the + # observed pre-order trickle rate. + days_until_arrival = batch_data['preorder_arrival_days'].get(pid, 14) + preorder_units = batch_data['preorder_sales'].get(pid, 0) + preorder_days = batch_data['preorder_days'].get(pid, 1) + preorder_daily_rate = preorder_units / max(preorder_days, 1) + forecasts = forecast_preorder( + curve_info, scale, days_until_arrival, + preorder_daily_rate, FORECAST_HORIZON_DAYS) + method = 'lifecycle_curve' + else: + # No reliable curve — fall back to velocity if available + velocity = product.get('sales_velocity_daily') or 0 + if velocity > 0: + forecasts = np.full(FORECAST_HORIZON_DAYS, velocity) + method = 'velocity' + else: + forecasts = forecast_dormant() + method = 'zero' + + elif phase == 'launch': + if curve_info: + scale = compute_scale_factor('launch', product, curve_info, batch_data) forecasts = forecast_from_curve(curve_info, scale, age, FORECAST_HORIZON_DAYS) method = 'lifecycle_curve' else: @@ -1038,8 +1194,16 @@ def generate_all_forecasts(conn, curves_df, dow_indices, monthly_indices=None, method = 'velocity' else: # dormant - forecasts = forecast_dormant() - method = 'zero' + # Carry a small positive rate for dormant products that still + # trickle sales (restocks/promos/long-tail); only truly dead + # products stay at zero. See FORECAST_FIX_PLAN F5. + rate = batch_data['dormant_rate'].get(pid, 0) + if rate > 0: + forecasts = np.full(FORECAST_HORIZON_DAYS, rate) + method = 'velocity' + else: + forecasts = forecast_dormant() + method = 'zero' # Confidence interval: use accuracy-calibrated margins per phase base_margin = accuracy_margins.get(phase, 0.5) @@ -1108,6 +1272,8 @@ def archive_forecasts(conn, run_id): """) cur.execute("CREATE INDEX IF NOT EXISTS idx_pfh_date ON product_forecasts_history(forecast_date)") cur.execute("CREATE INDEX IF NOT EXISTS idx_pfh_pid_date ON product_forecasts_history(pid, forecast_date)") + # Naive-baseline column for forecast value-added (FVA). See FORECAST_FIX_PLAN F8. + cur.execute("ALTER TABLE product_forecasts_history ADD COLUMN IF NOT EXISTS naive_units NUMERIC(10,2)") # Find the previous completed run (whose forecasts are still in product_forecasts) cur.execute(""" @@ -1124,15 +1290,27 @@ def archive_forecasts(conn, run_id): prev_run_id = prev_run[0] - # Archive only past-date forecasts (where actuals now exist) + # Archive only past-date forecasts (where actuals now exist). Attach the + # naive baseline (flat trailing-28-day daily average) at the same time so + # forecast value-added can be measured. See FORECAST_FIX_PLAN F8. cur.execute(""" INSERT INTO product_forecasts_history (run_id, pid, forecast_date, forecast_units, forecast_revenue, - lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at) - SELECT %s, pid, forecast_date, forecast_units, forecast_revenue, - lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at - FROM product_forecasts - WHERE forecast_date < CURRENT_DATE + lifecycle_phase, forecast_method, confidence_lower, confidence_upper, + generated_at, naive_units) + SELECT %s, pf.pid, pf.forecast_date, pf.forecast_units, pf.forecast_revenue, + pf.lifecycle_phase, pf.forecast_method, pf.confidence_lower, pf.confidence_upper, + pf.generated_at, COALESCE(nv.naive_daily, 0) + FROM product_forecasts pf + LEFT JOIN ( + SELECT o.pid, SUM(o.quantity) / 28.0 AS naive_daily + FROM orders o + WHERE o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - INTERVAL '28 days' + AND o.date < CURRENT_DATE + GROUP BY o.pid + ) nv ON nv.pid = pf.pid + WHERE pf.forecast_date < CURRENT_DATE ON CONFLICT (run_id, pid, forecast_date) DO NOTHING """, (prev_run_id,)) @@ -1154,6 +1332,48 @@ def archive_forecasts(conn, run_id): return archived +def archive_future_leads(conn, run_id): + """ + Archive a sampled set of FUTURE-lead forecasts from the just-generated + product_forecasts, attributed to the current run. + + The past-date archive in archive_forecasts() only ever captures the 1-day + slice that just elapsed, so every accuracy sample lands in the '1-7d' lead + bucket and the 15/30/60/90-day forecasts that purchasing actually rides on + are never validated. Here we snapshot the 7/14/30/60/89-day-ahead leads + (non-dormant) so that, once each date passes, compute_accuracy() can score + them in their lead bucket. The naive baseline is attached the same way as in + the past-date path. Future-dated rows survive the 90-day prune until their + own date passes. See FORECAST_FIX_PLAN F7. + """ + with conn.cursor() as cur: + cur.execute(""" + INSERT INTO product_forecasts_history + (run_id, pid, forecast_date, forecast_units, forecast_revenue, + lifecycle_phase, forecast_method, confidence_lower, confidence_upper, + generated_at, naive_units) + SELECT %s, pf.pid, pf.forecast_date, pf.forecast_units, pf.forecast_revenue, + pf.lifecycle_phase, pf.forecast_method, pf.confidence_lower, pf.confidence_upper, + pf.generated_at, COALESCE(nv.naive_daily, 0) + FROM product_forecasts pf + LEFT JOIN ( + SELECT o.pid, SUM(o.quantity) / 28.0 AS naive_daily + FROM orders o + WHERE o.canceled IS DISTINCT FROM TRUE + AND o.date >= CURRENT_DATE - INTERVAL '28 days' + AND o.date < CURRENT_DATE + GROUP BY o.pid + ) nv ON nv.pid = pf.pid + WHERE pf.lifecycle_phase != 'dormant' + AND pf.forecast_date - CURRENT_DATE IN (7, 14, 30, 60, 89) + ON CONFLICT (run_id, pid, forecast_date) DO NOTHING + """, (run_id,)) + archived = cur.rowcount + conn.commit() + log.info(f"Archived {archived} future-lead forecast rows (7/14/30/60/89d) for run {run_id}") + return archived + + def compute_accuracy(conn, run_id): """ Compute forecast accuracy metrics from archived history vs. actual sales. @@ -1162,11 +1382,18 @@ def compute_accuracy(conn, run_id): (pid, forecast_date = snapshot_date) to compare forecasted vs. actual units. Stores results in forecast_accuracy table, broken down by: - - overall: single aggregate row + - overall: two rows — 'all' (non-dormant) and 'all_incl_dormant' (F5) + - overall_weekly: per-product weekly-grain WMAPE — the informative headline + for intermittent demand (daily grain has a ~190% floor) (F9) - by_phase: per lifecycle phase - - by_lead_time: bucketed by how far ahead the forecast was + - by_lead_time: bucketed by how far ahead the forecast was — long-lead + buckets populate as the future-lead archives mature (F7) - by_method: per forecast method - daily: per forecast_date (for trend charts) + + Every dimension also stores naive_wmape (flat trailing-28d baseline) and + fva = 1 - wmape/naive_wmape, so the engine can be judged as value-over-naive + (F8). Only realized dates (forecast_date < CURRENT_DATE) are scored. """ with conn.cursor() as cur: # Ensure accuracy table exists @@ -1186,6 +1413,10 @@ def compute_accuracy(conn, run_id): PRIMARY KEY (run_id, metric_type, dimension_value) ) """) + # Naive-baseline WMAPE and forecast value-added (FVA = 1 - wmape/naive_wmape). + # See FORECAST_FIX_PLAN F8. + cur.execute("ALTER TABLE forecast_accuracy ADD COLUMN IF NOT EXISTS naive_wmape NUMERIC(10,4)") + cur.execute("ALTER TABLE forecast_accuracy ADD COLUMN IF NOT EXISTS fva NUMERIC(10,4)") conn.commit() # Check if we have any history to analyze @@ -1195,124 +1426,199 @@ def compute_accuracy(conn, run_id): log.info("No forecast history available for accuracy computation") return - # For each (pid, forecast_date) pair, keep only the most recent run's - # forecast row. This prevents double-counting when multiple runs have - # archived forecasts for the same product×date combination. - accuracy_cte = """ - WITH ranked_history AS ( + # Base CTEs (FORECAST_FIX_PLAN F7): + # - Only score realized dates (forecast_date < CURRENT_DATE); future-lead + # archives are excluded until their date passes. + # - short_lead*: lead 0-6 deduped per (pid, forecast_date) — preserves the + # meaning of the existing headline metrics. short_lead_eval keeps the + # raw snapshot grid (incl. zero-zero days) for complete-week detection; + # `accuracy` drops zero-zero days for daily-grain metrics. + # - lead_dedup/lead_accuracy: deduped per (pid, forecast_date, lead_bucket) + # so each long-lead bucket gets its own sample (the by_lead_time table). + base_cte = """ + WITH ranked_all AS ( SELECT - pfh.*, + pfh.pid, pfh.forecast_date, pfh.forecast_units, pfh.naive_units, + pfh.lifecycle_phase, pfh.forecast_method, fr.started_at, - ROW_NUMBER() OVER ( - PARTITION BY pfh.pid, pfh.forecast_date - ORDER BY fr.started_at DESC - ) AS rn + (pfh.forecast_date - fr.started_at::date) AS lead_days, + CASE + WHEN (pfh.forecast_date - fr.started_at::date) BETWEEN 0 AND 6 THEN '1-7d' + WHEN (pfh.forecast_date - fr.started_at::date) BETWEEN 7 AND 13 THEN '8-14d' + WHEN (pfh.forecast_date - fr.started_at::date) BETWEEN 14 AND 29 THEN '15-30d' + WHEN (pfh.forecast_date - fr.started_at::date) BETWEEN 30 AND 59 THEN '31-60d' + ELSE '61-90d' + END AS lead_bucket FROM product_forecasts_history pfh JOIN forecast_runs fr ON fr.id = pfh.run_id + WHERE pfh.forecast_date < CURRENT_DATE + ), + short_lead AS ( + SELECT *, + ROW_NUMBER() OVER ( + PARTITION BY pid, forecast_date ORDER BY started_at DESC + ) AS rn + FROM ranked_all + WHERE lead_days BETWEEN 0 AND 6 + ), + short_lead_eval AS ( + SELECT sl.pid, sl.lifecycle_phase, sl.forecast_method, sl.forecast_date, + sl.forecast_units, sl.naive_units, + COALESCE(dps.units_sold, 0) AS actual_units, + (sl.forecast_units - COALESCE(dps.units_sold, 0)) AS error, + ABS(sl.forecast_units - COALESCE(dps.units_sold, 0)) AS abs_error + FROM short_lead sl + LEFT JOIN daily_product_snapshots dps + ON dps.pid = sl.pid AND dps.snapshot_date = sl.forecast_date + WHERE sl.rn = 1 ), accuracy AS ( - SELECT - rh.lifecycle_phase, - rh.forecast_method, - rh.forecast_date, - (rh.forecast_date - rh.started_at::date) AS lead_days, - rh.forecast_units, + SELECT * FROM short_lead_eval + WHERE NOT (forecast_units = 0 AND actual_units = 0) + ), + lead_dedup AS ( + SELECT *, + ROW_NUMBER() OVER ( + PARTITION BY pid, forecast_date, lead_bucket ORDER BY started_at DESC + ) AS rn + FROM ranked_all + ), + lead_accuracy AS ( + SELECT ld.lead_bucket, ld.forecast_units, ld.naive_units, COALESCE(dps.units_sold, 0) AS actual_units, - (rh.forecast_units - COALESCE(dps.units_sold, 0)) AS error, - ABS(rh.forecast_units - COALESCE(dps.units_sold, 0)) AS abs_error - FROM ranked_history rh + (ld.forecast_units - COALESCE(dps.units_sold, 0)) AS error, + ABS(ld.forecast_units - COALESCE(dps.units_sold, 0)) AS abs_error + FROM lead_dedup ld LEFT JOIN daily_product_snapshots dps - ON dps.pid = rh.pid AND dps.snapshot_date = rh.forecast_date - WHERE rh.rn = 1 - AND NOT (rh.forecast_units = 0 AND COALESCE(dps.units_sold, 0) = 0) + ON dps.pid = ld.pid AND dps.snapshot_date = ld.forecast_date + WHERE ld.rn = 1 + AND ld.lifecycle_phase != 'dormant' + AND NOT (ld.forecast_units = 0 AND COALESCE(dps.units_sold, 0) = 0) ) """ - # Compute and insert metrics for each dimension - dimensions = { - 'overall': "SELECT 'all' AS dim", - 'by_phase': "SELECT DISTINCT lifecycle_phase AS dim FROM accuracy", - 'by_lead_time': """ - SELECT DISTINCT - CASE - WHEN lead_days BETWEEN 0 AND 6 THEN '1-7d' - WHEN lead_days BETWEEN 7 AND 13 THEN '8-14d' - WHEN lead_days BETWEEN 14 AND 29 THEN '15-30d' - WHEN lead_days BETWEEN 30 AND 59 THEN '31-60d' - ELSE '61-90d' - END AS dim - FROM accuracy - """, - 'by_method': "SELECT DISTINCT forecast_method AS dim FROM accuracy", - 'daily': "SELECT DISTINCT forecast_date::text AS dim FROM accuracy", - } - - filter_clauses = { - 'overall': "lifecycle_phase != 'dormant'", - 'by_phase': "lifecycle_phase = dims.dim", - 'by_lead_time': """ - CASE - WHEN lead_days BETWEEN 0 AND 6 THEN '1-7d' - WHEN lead_days BETWEEN 7 AND 13 THEN '8-14d' - WHEN lead_days BETWEEN 14 AND 29 THEN '15-30d' - WHEN lead_days BETWEEN 30 AND 59 THEN '31-60d' - ELSE '61-90d' - END = dims.dim - """, - 'by_method': "forecast_method = dims.dim", - 'daily': "forecast_date::text = dims.dim", - } - - total_inserted = 0 - - for metric_type, dim_query in dimensions.items(): - filter_clause = filter_clauses[metric_type] - - sql = f""" - {accuracy_cte}, - dims AS ({dim_query}) + # Daily-grain aggregate over a source CTE aliased `a`, computing the + # engine WMAPE plus the naive-baseline WMAPE (NULL-safe: rows archived + # before F8 have naive_units NULL and are excluded from the naive sums). + def daily_agg(dim_expr, source, where=None, group_by=None): + where_sql = f"WHERE {where}" if where else "" + group_sql = f"GROUP BY {group_by}" if group_by else "" + return f""" SELECT - dims.dim, + {dim_expr} AS dim, COUNT(*) AS sample_size, COALESCE(SUM(a.actual_units), 0) AS total_actual, COALESCE(SUM(a.forecast_units), 0) AS total_forecast, AVG(a.abs_error) AS mae, CASE WHEN SUM(a.actual_units) > 0 - THEN SUM(a.abs_error) / SUM(a.actual_units) - ELSE NULL END AS wmape, + THEN SUM(a.abs_error) / SUM(a.actual_units) ELSE NULL END AS wmape, AVG(a.error) AS bias, - SQRT(AVG(POWER(a.error, 2))) AS rmse - FROM dims - CROSS JOIN accuracy a - WHERE {filter_clause} - GROUP BY dims.dim + SQRT(AVG(POWER(a.error, 2))) AS rmse, + CASE WHEN SUM(a.actual_units) FILTER (WHERE a.naive_units IS NOT NULL) > 0 + THEN SUM(ABS(a.naive_units - a.actual_units)) FILTER (WHERE a.naive_units IS NOT NULL) + / SUM(a.actual_units) FILTER (WHERE a.naive_units IS NOT NULL) + ELSE NULL END AS naive_wmape + FROM {source} a + {where_sql} + {group_sql} """ - cur.execute(sql) - rows = cur.fetchall() + insert_sql = """ + INSERT INTO forecast_accuracy + (run_id, metric_type, dimension_value, sample_size, + total_actual_units, total_forecast_units, mae, wmape, bias, rmse, + naive_wmape, fva) + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) + ON CONFLICT (run_id, metric_type, dimension_value) + DO UPDATE SET + sample_size = EXCLUDED.sample_size, + total_actual_units = EXCLUDED.total_actual_units, + total_forecast_units = EXCLUDED.total_forecast_units, + mae = EXCLUDED.mae, wmape = EXCLUDED.wmape, + bias = EXCLUDED.bias, rmse = EXCLUDED.rmse, + naive_wmape = EXCLUDED.naive_wmape, fva = EXCLUDED.fva, + computed_at = NOW() + """ - for row in rows: - dim_val, sample_size, total_actual, total_forecast, mae, wmape, bias, rmse = row - cur.execute(""" - INSERT INTO forecast_accuracy - (run_id, metric_type, dimension_value, sample_size, - total_actual_units, total_forecast_units, mae, wmape, bias, rmse) - VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s) - ON CONFLICT (run_id, metric_type, dimension_value) - DO UPDATE SET - sample_size = EXCLUDED.sample_size, - total_actual_units = EXCLUDED.total_actual_units, - total_forecast_units = EXCLUDED.total_forecast_units, - mae = EXCLUDED.mae, wmape = EXCLUDED.wmape, - bias = EXCLUDED.bias, rmse = EXCLUDED.rmse, - computed_at = NOW() - """, (run_id, metric_type, dim_val, sample_size, - float(total_actual), float(total_forecast), - float(mae) if mae is not None else None, - float(wmape) if wmape is not None else None, - float(bias) if bias is not None else None, - float(rmse) if rmse is not None else None)) - total_inserted += 1 + def _f(x): + return float(x) if x is not None else None + + def run_and_insert(metric_type, sql): + cur.execute(base_cte + sql) + n = 0 + for row in cur.fetchall(): + (dim_val, sample_size, total_actual, total_forecast, + mae, wmape, bias, rmse, naive_wmape) = row + fva = None + if wmape is not None and naive_wmape is not None and float(naive_wmape) > 0: + fva = 1.0 - float(wmape) / float(naive_wmape) + cur.execute(insert_sql, ( + run_id, metric_type, dim_val, sample_size, + _f(total_actual), _f(total_forecast), _f(mae), _f(wmape), + _f(bias), _f(rmse), _f(naive_wmape), _f(fva))) + n += 1 + return n + + total_inserted = 0 + + # overall: two rows — 'all' (non-dormant, the headline) and + # 'all_incl_dormant' (everything, so the ~11% dormant demand stops being + # invisible). Both are short-lead (lead 0-6). F5. + overall_source = """( + SELECT a.*, 'all'::text AS dim FROM accuracy a WHERE a.lifecycle_phase != 'dormant' + UNION ALL + SELECT a.*, 'all_incl_dormant'::text AS dim FROM accuracy a + )""" + total_inserted += run_and_insert('overall', + daily_agg('a.dim', overall_source, group_by='a.dim')) + + # by_phase / by_method / daily — short-lead daily-grain over `accuracy`. + total_inserted += run_and_insert('by_phase', + daily_agg('a.lifecycle_phase', 'accuracy', group_by='a.lifecycle_phase')) + total_inserted += run_and_insert('by_method', + daily_agg('a.forecast_method', 'accuracy', group_by='a.forecast_method')) + total_inserted += run_and_insert('daily', + daily_agg('a.forecast_date::text', 'accuracy', + where="a.lifecycle_phase != 'dormant'", group_by='a.forecast_date')) + + # by_lead_time — one sample per (pid, date, lead bucket) over `lead_accuracy`. + # Buckets beyond '1-7d' populate as the future-lead archives (F7) mature. + total_inserted += run_and_insert('by_lead_time', + daily_agg('a.lead_bucket', 'lead_accuracy', group_by='a.lead_bucket')) + + # overall_weekly — the informative headline for intermittent retail demand. + # Aggregate the short-lead rows to (pid, complete week), then WMAPE over + # pid-weeks. Daily-grain WMAPE has a ~190% floor on this catalog; weekly + # grain is ~109% and responds to real improvement. F9. + weekly_sql = """, + weekly AS ( + SELECT pid, date_trunc('week', forecast_date) AS wk, + SUM(forecast_units) AS fc_week, + SUM(actual_units) AS act_week, + SUM(naive_units) AS naive_week, + bool_and(naive_units IS NOT NULL) AS naive_complete + FROM short_lead_eval + WHERE lifecycle_phase != 'dormant' + GROUP BY pid, date_trunc('week', forecast_date) + HAVING COUNT(*) = 7 + ) + SELECT 'all'::text AS dim, + COUNT(*) AS sample_size, + COALESCE(SUM(act_week), 0) AS total_actual, + COALESCE(SUM(fc_week), 0) AS total_forecast, + AVG(ABS(fc_week - act_week)) AS mae, + CASE WHEN SUM(act_week) > 0 + THEN SUM(ABS(fc_week - act_week)) / SUM(act_week) ELSE NULL END AS wmape, + AVG(fc_week - act_week) AS bias, + SQRT(AVG(POWER(fc_week - act_week, 2))) AS rmse, + CASE WHEN SUM(act_week) FILTER (WHERE naive_complete) > 0 + THEN SUM(ABS(naive_week - act_week)) FILTER (WHERE naive_complete) + / SUM(act_week) FILTER (WHERE naive_complete) + ELSE NULL END AS naive_wmape + FROM weekly + WHERE NOT (fc_week = 0 AND act_week = 0) + """ + total_inserted += run_and_insert('overall_weekly', weekly_sql) conn.commit() @@ -1562,6 +1868,10 @@ def main(): conn, curves_df, dow_indices, monthly_indices, accuracy_margins ) + # Phase 4b: Snapshot sampled future-lead forecasts (7/14/30/60/89d) from + # the fresh run so long-lead accuracy populates once those dates pass (F7). + archive_future_leads(conn, run_id) + duration = time.time() - start_time # Record run completion (include DOW indices in metadata) diff --git a/inventory-server/src/routes/dashboard.js b/inventory-server/src/routes/dashboard.js index 7f677d9..25563a9 100644 --- a/inventory-server/src/routes/dashboard.js +++ b/inventory-server/src/routes/dashboard.js @@ -357,6 +357,9 @@ router.get('/forecast/metrics', async (req, res) => { const active = parseInt(totals.active_products) || 1; const curveProducts = parseInt(totals.curve_products) || 0; + // NOTE: despite the name, this is "share of active products forecast via + // lifecycle curves" (curve coverage), NOT a statistical confidence. It only + // feeds a per-day tooltip field. See FORECAST_FIX_PLAN F9 (point 4). const confidenceLevel = parseFloat((curveProducts / active).toFixed(2)); // Daily series from actual forecast @@ -687,14 +690,29 @@ router.get('/forecast/accuracy', async (req, res) => { const { rows: metrics } = await executeQuery(` SELECT metric_type, dimension_value, sample_size, total_actual_units, total_forecast_units, - mae, wmape, bias, rmse + mae, wmape, bias, rmse, naive_wmape, fva FROM forecast_accuracy WHERE run_id = $1 ORDER BY metric_type, dimension_value `, [latestRunId]); + // Shared shaping for an "overall"-style aggregate row (daily or weekly grain). + const shapeOverall = (m) => m ? { + sampleSize: parseInt(m.sample_size), + totalActual: parseFloat(m.total_actual_units) || 0, + totalForecast: parseFloat(m.total_forecast_units) || 0, + mae: m.mae != null ? parseFloat(parseFloat(m.mae).toFixed(4)) : null, + wmape: m.wmape != null ? parseFloat((parseFloat(m.wmape) * 100).toFixed(1)) : null, + bias: m.bias != null ? parseFloat(parseFloat(m.bias).toFixed(4)) : null, + rmse: m.rmse != null ? parseFloat(parseFloat(m.rmse).toFixed(4)) : null, + naiveWmape: m.naive_wmape != null ? parseFloat((parseFloat(m.naive_wmape) * 100).toFixed(1)) : null, + fva: m.fva != null ? parseFloat(parseFloat(m.fva).toFixed(3)) : null, + } : null; + // Organize into response structure - const overall = metrics.find(m => m.metric_type === 'overall'); + const overall = metrics.find(m => m.metric_type === 'overall' && m.dimension_value === 'all') + const overallInclDormant = metrics.find(m => m.metric_type === 'overall' && m.dimension_value === 'all_incl_dormant') + const overallWeekly = metrics.find(m => m.metric_type === 'overall_weekly'); const byPhase = metrics .filter(m => m.metric_type === 'by_phase') .map(m => ({ @@ -706,6 +724,8 @@ router.get('/forecast/accuracy', async (req, res) => { wmape: m.wmape != null ? parseFloat((parseFloat(m.wmape) * 100).toFixed(1)) : null, bias: m.bias != null ? parseFloat(parseFloat(m.bias).toFixed(4)) : null, rmse: m.rmse != null ? parseFloat(parseFloat(m.rmse).toFixed(4)) : null, + naiveWmape: m.naive_wmape != null ? parseFloat((parseFloat(m.naive_wmape) * 100).toFixed(1)) : null, + fva: m.fva != null ? parseFloat(parseFloat(m.fva).toFixed(3)) : null, })) .sort((a, b) => (b.totalActual || 0) - (a.totalActual || 0)); @@ -763,6 +783,26 @@ router.get('/forecast/accuracy', async (req, res) => { sampleSize: parseInt(r.sample_size), })); + // Weekly-grain trend across runs (starts empty for old runs that predate + // the overall_weekly metric — that's expected, no backfill). F9. + const { rows: weeklyTrendRows } = await executeQuery(` + SELECT fr.finished_at::date AS run_date, + fa.wmape, fa.naive_wmape, fa.fva, fa.sample_size + FROM forecast_accuracy fa + JOIN forecast_runs fr ON fr.id = fa.run_id + WHERE fa.metric_type = 'overall_weekly' + AND fa.dimension_value = 'all' + ORDER BY fr.finished_at + `); + + const accuracyTrendWeekly = weeklyTrendRows.map(r => ({ + date: r.run_date instanceof Date ? r.run_date.toISOString().split('T')[0] : r.run_date, + wmape: r.wmape != null ? parseFloat((parseFloat(r.wmape) * 100).toFixed(1)) : null, + naiveWmape: r.naive_wmape != null ? parseFloat((parseFloat(r.naive_wmape) * 100).toFixed(1)) : null, + fva: r.fva != null ? parseFloat(parseFloat(r.fva).toFixed(3)) : null, + sampleSize: parseInt(r.sample_size), + })); + res.json({ hasData: true, computedAt, @@ -775,20 +815,15 @@ router.get('/forecast/accuracy', async (req, res) => { ? historyInfo.latest_date.toISOString().split('T')[0] : historyInfo.latest_date, }, - overall: overall ? { - sampleSize: parseInt(overall.sample_size), - totalActual: parseFloat(overall.total_actual_units) || 0, - totalForecast: parseFloat(overall.total_forecast_units) || 0, - mae: overall.mae != null ? parseFloat(parseFloat(overall.mae).toFixed(4)) : null, - wmape: overall.wmape != null ? parseFloat((parseFloat(overall.wmape) * 100).toFixed(1)) : null, - bias: overall.bias != null ? parseFloat(parseFloat(overall.bias).toFixed(4)) : null, - rmse: overall.rmse != null ? parseFloat(parseFloat(overall.rmse).toFixed(4)) : null, - } : null, + overall: shapeOverall(overall), + overallInclDormant: shapeOverall(overallInclDormant), + overallWeekly: shapeOverall(overallWeekly), byPhase, byLeadTime, byMethod, dailyTrend, accuracyTrend, + accuracyTrendWeekly, }); } catch (err) { console.error('Error fetching forecast accuracy:', err); diff --git a/inventory/src/components/overview/ForecastAccuracy.tsx b/inventory/src/components/overview/ForecastAccuracy.tsx index c0c3d9a..77e81cf 100644 --- a/inventory/src/components/overview/ForecastAccuracy.tsx +++ b/inventory/src/components/overview/ForecastAccuracy.tsx @@ -2,7 +2,7 @@ import { useQuery } from "@tanstack/react-query" import { apiFetch } from '@/utils/api'; import { BarChart, Bar, ResponsiveContainer, XAxis, YAxis, Tooltip as RechartsTooltip, Cell, LineChart, Line } from "recharts" import config from "@/config" -import { Target, TrendingDown, ArrowUpDown } from "lucide-react" +import { Target, TrendingDown, ArrowUpDown, Swords } from "lucide-react" import { Tooltip as UITooltip, TooltipContent, TooltipProvider, TooltipTrigger } from "@/components/ui/tooltip" import { PHASE_CONFIG } from "@/utils/lifecyclePhases" @@ -14,6 +14,8 @@ interface OverallMetrics { wmape: number | null bias: number | null rmse: number | null + naiveWmape?: number | null + fva?: number | null } interface PhaseAccuracy { @@ -25,6 +27,8 @@ interface PhaseAccuracy { wmape: number | null bias: number | null rmse: number | null + naiveWmape?: number | null + fva?: number | null } interface LeadTimeAccuracy { @@ -51,11 +55,14 @@ interface AccuracyData { daysOfHistory?: number historyRange?: { from: string; to: string } overall?: OverallMetrics + overallInclDormant?: OverallMetrics + overallWeekly?: OverallMetrics byPhase?: PhaseAccuracy[] byLeadTime?: LeadTimeAccuracy[] byMethod?: { method: string; sampleSize: number; mae: number | null; wmape: number | null; bias: number | null }[] dailyTrend?: { date: string; mae: number | null; wmape: number | null; bias: number | null }[] accuracyTrend?: AccuracyTrendPoint[] + accuracyTrendWeekly?: { date: string; wmape: number | null; naiveWmape: number | null; fva: number | null; sampleSize: number }[] } function MetricSkeleton() { @@ -74,12 +81,30 @@ function formatBias(bias: number | null): string { } function getAccuracyColor(wmape: number | null): string { + // Daily-grain thresholds (used for the by-phase / lead-time bars). if (wmape === null) return "text-muted-foreground" if (wmape <= 30) return "text-green-600" if (wmape <= 50) return "text-yellow-600" return "text-red-600" } +function getWeeklyAccuracyColor(wmape: number | null): string { + // Weekly per-product grain has a much lower achievable floor than daily grain + // on this intermittent-demand catalog, so the headline uses its own thresholds. + if (wmape === null) return "text-muted-foreground" + if (wmape <= 60) return "text-green-600" + if (wmape <= 90) return "text-yellow-600" + return "text-red-600" +} + +function formatSignedPct(ratio: number | null, digits = 0): string { + // ratio is a fraction (0.7 => +70%); null-safe. + if (ratio === null || ratio === undefined) return "N/A" + const pct = ratio * 100 + const sign = pct > 0 ? "+" : "" + return `${sign}${pct.toFixed(digits)}%` +} + export function ForecastAccuracy() { const { data, error, isLoading } = useQuery({ queryKey: ["forecast-accuracy"], @@ -133,6 +158,24 @@ export function ForecastAccuracy() { sampleSize: lt.sampleSize, })) + // Headline prefers the weekly-grain WMAPE (informative); falls back to the + // daily-grain number until enough complete weeks of history exist. + const weeklyWmape = data?.overallWeekly?.wmape ?? null + const usingWeekly = weeklyWmape !== null + const headlineWmape = usingWeekly ? weeklyWmape : (data?.overall?.wmape ?? null) + const headlineColor = usingWeekly + ? getWeeklyAccuracyColor(headlineWmape) + : getAccuracyColor(headlineWmape) + // Net forecast-vs-actual ratio (e.g. +70% = over-forecasting), from the + // daily 'all' totals — far more legible than bias in raw units. + const totalFc = data?.overall?.totalForecast ?? 0 + const totalAct = data?.overall?.totalActual ?? 0 + const fcVsAct = totalAct > 0 ? (totalFc / totalAct - 1) : null + // Value over the naive baseline; prefer weekly grain to match the headline. + const naiveSource = data?.overallWeekly ?? data?.overall + const naiveWmape = naiveSource?.naiveWmape ?? null + const fva = naiveSource?.fva ?? null + return (

Forecast Accuracy

@@ -148,10 +191,24 @@ export function ForecastAccuracy() {
-

WMAPE

+

+ WMAPE ({usingWeekly ? "weekly" : "daily"}) +

-

- {formatWmape(data?.overall?.wmape ?? null)} +

+ {formatWmape(headlineWmape)} +

+
+
+
+ +

Forecast vs actual

+
+

+ {formatSignedPct(fcVsAct)} + + {(fcVsAct ?? 0) > 0 ? "over" : (fcVsAct ?? 0) < 0 ? "under" : ""} +

@@ -160,20 +217,24 @@ export function ForecastAccuracy() {

MAE

- {data?.overall?.mae !== null ? data?.overall?.mae?.toFixed(2) : "N/A"} + {data?.overall?.mae != null ? data?.overall?.mae?.toFixed(2) : "N/A"} units

- -

Bias

+ +

vs naive

- {formatBias(data?.overall?.bias ?? null)} - - {(data?.overall?.bias ?? 0) > 0 ? "over" : (data?.overall?.bias ?? 0) < 0 ? "under" : ""} + 0 ? "text-green-600" : "text-red-600") : "text-muted-foreground"}> + {fva != null ? `${formatSignedPct(fva)} FVA` : "N/A"} + {naiveWmape != null && ( + + naive {formatWmape(naiveWmape)} + + )}