27 KiB
Import & Metrics Pipeline Fix Plan
Fixes for issues found in a full review (2026-06-10) of the full-update.js pipeline:
inventory-server/scripts/full-update.js → import-from-prod.js (6 importers in scripts/import/)
→ calculate-metrics-new.js (7 SQL modules in scripts/metrics-new/).
Every issue below was verified against the code, and where marked [verified-live], against the
live MySQL source (sg on 192.168.1.5 via the acot-db tooling / ssh workpi) and live PostgreSQL
(inventory_db — ssh netcup, then psql -U inventory_readonly, password in /Users/matt/Dev/inventory/CLAUDE.md).
Write credentials for migrations: see /var/www/inventory/.env on netcup (inventory_user).
Operational context (read first)
- Local
inventory-server/is NFS-mounted to/var/www/inventory/on the netcup server — edits appear on the server with no copy step. Run heavy validation/grep/find on the server viassh netcup, not locally (NFS hangs + AppleDouble._*noise). - The PG server timezone is Europe/Berlin. The business operates in America/Chicago. This matters for Fix 2.
- MySQL server is America/Chicago; the mysql2 driver is configured
timezone: '-05:00'and corrected at runtime byadjustDateForMySQL()inscripts/import/utils.js(seememory/TIMEZONE_ISSUE.md). Don't "fix" that part — it already works. - Orders/PO/products imports are incremental by default (
INCREMENTAL_UPDATE !== 'false'); a full orders sync = run withINCREMENTAL_UPDATE=false(5-year window). - Existing rebuild tooling:
scripts/metrics-new/backfill/rebuild_daily_snapshots.sql(rebuildsdaily_product_snapshotsfromorders/receivings). The full-pipeline order after data fixes: re-import → rebuild snapshots →node scripts/calculate-metrics-new.js. - Precedent:
scripts/metrics-new/migrations/002_fix_discount_double_counting.sqldocuments the procedure used last time a discount formula changed. Follow the same pattern (migration doc + code fix + full re-import + rebuild).
P0 — Data correctness (do both, then ONE re-import + rebuild)
Fix 1: Item-level promo discounts dropped (~$26K / 30 days ≈ 10% of product revenue) [verified-live]
File: scripts/import/orders.js — order_totals CTE (~lines 604-623) and the discount fetch in
processDiscountsBatch (~lines 379-383).
Problem. The discount applied to each PG orders row is:
prorated summary_discount_subtotal + item-level promo discounts. The item-level part is gated:
SUM(CASE WHEN COALESCE(md.discount_amount_subtotal, 0) > 0 THEN id.amount ELSE 0 END)
In the PHP source (/Users/matt/Dev/acot/website/website/lib/neworder.class.php):
order_items.prod_priceis the pre-promo price;summary_subtotal = Σ prod_price·qty(line ~3087).- Item-level promo discounts live in
order_discount_itemswithwhich = 2; they are applied to the order total viasummary_discount += amount + products_disc_sum(line ~6567) — i.e. they are not part ofdiscount_amount_subtotaland not baked intoprod_price. - Live data (90 days): of 10,010 type-10 promo discounts, 8,070 have item rows but only 8 have
discount_amount_subtotal > 0— the gate zeroes essentially all item-level promo discounts. - Live impact (30 days): $25,989 dropped across 2,021 orders, vs only $13,574 captured via the prorated subtotal component. Order discount components, 30d: total $54,957 = $13,574 subtotal + $15,395 shipping + ~$25,989 item-level. (Shipping discounts correctly excluded from product revenue.)
Consequence. orders.discount understated → net_revenue, profit_30d, margin_30d overstated
by ~10% of revenue; discounts_30d / discount_rate_30d ~3x understated. Flows into daily snapshots,
product/brand/vendor/category metrics, and dashboards.
Fix.
- In
processDiscountsBatch, fetch only real item discounts:SELECT order_id, pid, discount_id, amount FROM order_discount_items WHERE order_id IN (?) AND which = 2. (which=1rows store prices of free promo-added items;which=3are usage records — neither is a discount amount.) - In the
order_totalsCTE, remove the gate — sumid.amountunconditionally:SUM(COALESCE(id.amount, 0)) AS promo_discount_sum(drop the join/CASE ontemp_main_discounts;temp_main_discountsbecomes unused and can be removed entirely along with its insert loop). - Sanity guard (optional, recommended): clamp final per-row discount to
price * quantity.
Verification. After a FULL orders re-import, for a recent 30-day window PG should satisfy:
SUM(discount) ≈ MySQL Σ summary_discount_subtotal + Σ order_discount_items.amount (which=2)
over the same orders (± rounding from proration). Spot-check an order with a type-10 promo:
discount on the affected pid ≈ the which=2 amount. Re-run migration 002's verification query too
(pids 624756, 614513) to confirm no regression of the prior fix.
Fix 2: Daily snapshots bucket sales by Europe/Berlin days, not business days [verified-live]
Files: scripts/metrics-new/update_daily_snapshots.sql (SalesData join o.date::date = _target_date
~line 138; gap-fill and stale-detection aggregates at lines ~47-83);
scripts/metrics-new/backfill/rebuild_daily_snapshots.sql (same pattern — check & fix);
scripts/metrics-new/update_product_metrics.sql (HistoricalDates MIN(o.date)::date etc., lines ~131-147).
Problem. orders.date is timestamptz; ::date casts in the server TZ (Europe/Berlin,
verified via SHOW timezone). Berlin is 7-8h ahead of Central, so every order placed after
~5 PM Central lands on the next snapshot day. This shifts a large evening slice of daily sales
forward one day; skews yesterday_sales, day-of-week patterns (the forecast engine's DOW
multipliers, daily-grain forecast accuracy — see FORECAST_FIX_PLAN.md), and is inconsistent with
stock_snapshots, whose dates come from a Central-time MySQL cron.
Fix. Bucket all order/receiving dates in business time. Replace every o.date::date /
received_date::date used for day bucketing in the two snapshot SQL files with:
(o.date AT TIME ZONE 'America/Chicago')::date
Apply consistently in: SalesData, ReceivingData, the gap-fill date lists, the stale-detection
aggregates (they must match SalesData or every day looks permanently stale), and the rebuild script.
HistoricalDates in update_product_metrics (first/last sold dates) should match too.
Add an index to keep the per-day loop fast, e.g.
CREATE INDEX ON orders ( ((date AT TIME ZONE 'America/Chicago')::date) ); and equivalent on
receivings(received_date); check EXPLAIN on the SalesData query afterward.
Note: receivings.received_date came from MySQL DATETIME (Central literal) inserted as timestamptz —
it was interpreted in the session TZ at insert. Before converting, spot-check a few receivings
against MySQL to confirm which TZ the stored instants actually represent; the conversion expression
must yield the Central calendar day MySQL shows. Same check for orders.date (it originates from
_order.date_placed, a TIMESTAMP column, so it should be a correct instant — AT TIME ZONE 'America/Chicago' is right for it).
Verification. Pick 2-3 recent days; compare per-day units_sold totals in
daily_product_snapshots against MySQL
SELECT date_placed_onlydate, SUM(qty_ordered) ... WHERE order_status >= 20 GROUP BY 1
(MySQL stores Central days). They should now match closely (small diffs from canceled-status timing).
P0 execution order (single pass)
- Land Fix 1 (orders.js) and Fix 2 (both snapshot SQL files + product-metrics date CTE).
- Full orders re-import:
INCREMENTAL_UPDATE=false node scripts/import-from-prod.js(or at minimum the orders step) — run on the server, it's long. - Rebuild snapshots:
psql -f scripts/metrics-new/backfill/rebuild_daily_snapshots.sql(after confirming it contains the TZ fix). The hourly job's 90-day self-heal will NOT fix history beyond 90 days by itself; the explicit rebuild is required. node scripts/calculate-metrics-new.js.- Expect dashboards to show: margins down ~8-10 points (real), daily sales curves shifted, DOW profile changed. Tell the user before/after numbers.
P1 — Wrong or drifting numbers, fix soon
Fix 3: Vendor avg lead time computed over a near-cartesian join
File: scripts/metrics-new/calculate_vendor_metrics.sql, VendorPOAggregates (lines ~62-83).
Problem. Joins each done-PO line to every receiving of the same (pid, supplier) after the PO
date — a product received 10 times contributes 10 ever-growing lead times → overstated, busy-product-
weighted vendor lead time. The per-product version in update_periodic_metrics.sql (lines 27-48)
is correct (MIN receiving per PO within 180 days, then average).
Fix. Reuse the periodic shape, aggregated to vendor:
WITH po_first_receiving AS (
SELECT po.vendor, po.po_id, po.pid, po.date::date AS po_date,
MIN(r.received_date::date) AS first_receive_date
FROM purchase_orders po
JOIN receivings r ON r.pid = po.pid AND r.supplier_id = po.supplier_id
AND r.received_date >= po.date
AND r.received_date <= po.date + INTERVAL '180 days'
WHERE po.status = 'done' AND po.date >= CURRENT_DATE - INTERVAL '1 year'
AND po.vendor IS NOT NULL AND po.vendor <> ''
GROUP BY po.vendor, po.po_id, po.pid, po.date
)
SELECT vendor, COUNT(DISTINCT po_id) AS po_count_365d,
ROUND(AVG(GREATEST(1, first_receive_date - po_date)))::int AS avg_lead_time_days_hist
FROM po_first_receiving GROUP BY vendor
Verification. For a few vendors compare old vs new values; new should be materially lower and
roughly match AVG(product_metrics.avg_lead_time_days) for that vendor's products.
Fix 4: Deleted order items & combined orders never reconciled in PG [verified-live]
File: scripts/import/orders.js.
Problem. The orders import upserts but never deletes:
- Items removed from an order in MySQL (
DELETE FROM order_items ...happens, e.g. neworder.class.php ~line 6500 for unpicked promo items, plus staff edits) leave stale rows in PG forever. May 2026 check: PG has 49,841 item rows vs MySQL 49,377 (+0.9%) — and PG should be ≤ MySQL. - Combining orders (
combine_orders, neworder.class.php ~11946) sets the source orders to status 16 AND zeroesdate_placed, then copies all items to a NEW order. Because the import query filterso.date_placed >= …, a combined source order can never be re-fetched, so its stale 'placed' rows would double-count with the new merged order. Currently latent (last combine 2024-07, predating current PG data — verified no stale rows exist today), but it will silently corrupt the day combining is used again.
Fix. Two parts, both inside the orders import after the upsert phase:
- Item-set reconciliation for re-imported orders: the import already knows the set of changed
orderIdsand inserted their current items intotemp_order_items. Mirror the PO import's pattern (purchase-orders.jslines ~683-694):DELETE FROM orders o WHERE o.order_number = ANY($1) -- orders fetched this run AND NOT EXISTS (SELECT 1 FROM temp_order_items t WHERE t.order_id = o.order_number AND t.pid = o.pid); - Combined/cancelled sweep that does NOT depend on
date_placed: each run, fetch from MySQLSELECT order_id, order_status FROM _order WHERE order_status IN (15,16) AND stamp > ?(no date_placed filter) and update matching PG rows'status/canceled('combined' rows are then excluded from metrics — see Fix 5). Cheap (small result set).
Verification. Re-run the May-2026 row-count comparison (MySQL vs PG for one month) after one full run; counts should converge (PG ≤ MySQL, diff explained by TZ window edges only).
Fix 5: 'combined' orders are counted as sales
Files: scripts/metrics-new/update_daily_snapshots.sql (status filters, lines ~77, 120-134),
update_product_metrics.sql (HistoricalDates line ~145, LifetimeRevenue line ~249),
backfill/rebuild_daily_snapshots.sql.
Problem. Sales filters exclude only ('canceled', 'returned'). Status 16 'combined' = "merged
into another order" — the new order carries the same items, so counting both double-counts. 826
combined orders exist in MySQL; today none are in PG (see Fix 4), but once Fix 4's sweep starts
marking rows 'combined', the metrics filters must exclude them.
Fix. Change every NOT IN ('canceled', 'returned') in the metrics SQL to
NOT IN ('canceled', 'returned', 'combined'). Grep for the pattern in scripts/metrics-new/ and
src/routes/ (dashboard endpoints replicate these filters — see CLAUDE.md analytics-filters note).
Fix 6: Incremental sync watermark race (silent permanent misses)
Files: scripts/import/orders.js (~772), products.js (~934), purchase-orders.js (~833).
Problem. sync_status.last_sync_timestamp is set to NOW() after the import finishes. Any
MySQL row modified between the source query and that write is below the new watermark but was never
fetched → permanently skipped (until a full sync or the row changes again). Long imports widen the
window; PG/MySQL clock skew adds to it.
Fix. Capture the watermark before the source query and write that value:
const [[{ now: sourceNow }]] = await prodConnection.query('SELECT NOW() as now');
// ... do the import ...
await localConnection.query(
`INSERT INTO sync_status ... VALUES ('orders', $1) ON CONFLICT ... SET last_sync_timestamp = $1`,
[sourceNow]);
Using MySQL's own clock also eliminates cross-server skew. Note sourceNow comes back through the
mysql2 driver TZ conversion — verify round-tripping with adjustDateForMySQL produces a correct
comparison value, or store UTC_TIMESTAMP() and compare against CONVERT_TZ-normalized stamps.
Overlap (re-importing rows changed during the run) is harmless — everything is upserted.
Fix 7: Stockout days / service level / fill rate / avg stock built on activity-only snapshots
Files: scripts/metrics-new/update_product_metrics.sql — SnapshotAggregates
(stockout_days_30d, avg_stock_*_30d, lines ~177-189), ServiceLevels (lines ~304-323),
plus calculate_sales_velocity usage.
Problem. daily_product_snapshots only has rows on days with sales/receivings. So:
- A product that is out of stock (and therefore sells nothing) gets no row →
stockout_days_30d≈ 0 exactly when stockouts matter →calculate_sales_velocity(sales, stockout_days)'s adjustment is inert → velocity and replenishment understated for constrained products. service_level_30ddivides stockout days by COUNT(activity days), not 30.avg_stock_units_30d/avg_stock_cost_30daverage only activity days (biased toward in-stock days) → GMROI / stockturn / sell-through denominators biased.fill_rate_30d'sunits_sold * 0.2lost-sales heuristic is arbitrary — fine to keep, but document.
Fix. Derive stock-presence metrics from stock_snapshots (full daily coverage from MySQL
snap_product_value, imported by stock-snapshots.js) instead of daily_product_snapshots:
StockCoverage AS (
SELECT pid,
COUNT(*) FILTER (WHERE stock_quantity <= 0) AS stockout_days_30d,
AVG(stock_quantity) AS avg_stock_units_30d,
AVG(stock_value) AS avg_stock_cost_30d
FROM stock_snapshots
WHERE snapshot_date >= _current_date - INTERVAL '29 days'
GROUP BY pid
)
Treat products absent from stock_snapshots for a day as unknown (NULL), not in-stock. Keep
daily_product_snapshots for sales/revenue aggregates. service_level_30d denominator becomes the
count of covered days. Note stock_snapshots has no eod_stock_retail; keep retail/gross averages
on the old source or compute as stock_quantity * current price explicitly.
Verification. Pick products that had a known stockout period; stockout_days_30d should now be
0 and
sales_velocity_dailyshould rise accordingly.
P2 — Definition / robustness improvements
Fix 8: Returns don't reduce COGS; LifetimeRevenue ignores returns
update_daily_snapshots.sql SalesData: COGS accrues only on quantity > 0 rows; return rows
(negative qty — 15,875 rows live) subtract revenue but never COGS → margin understated in
return-heavy periods. Add a returns-COGS term mirroring the sales-COGS COALESCE chain
(SUM(... WHEN quantity < 0 THEN cost * ABS(quantity))) and subtract it in cogs (or store
returns_cogs separately and use cogs - returns_cogs in profit). Also LifetimeRevenue in
update_product_metrics.sql (line ~242) filters quantity > 0 — include negative-qty rows so
lifetime revenue nets out returns (drop the quantity filter; price*quantity is already signed,
but check the - discount term sign for return rows).
Fix 9: return_rate_30d definition
update_product_metrics.sql line ~468: returns / (sales + returns) → industry standard is
returns / sales. Change denominator to NULLIF(sa.sales_30d, 0).
Fix 10: GMROI not annualized
Line ~466: profit_30d / avg_stock_cost_30d is a monthly GMROI (~1/12 of the conventional annual
figure, benchmark ≥ 2-3). Either annualize (* 12.17) or rename the column/label "monthly".
Decision for Matt; annualizing is recommended for comparability. Frontend displays must be checked
either way.
Fix 11: get_weighted_avg_cost is a lifetime WAC
db/functions.sql (~line 81, deployed identically): averages ALL receivings ≤ date — decade-old
costs weigh equally. Recommended: window to recent receivings, e.g. last 365 days falling back to
lifetime when none. Used as fallback COGS when o.costeach is NULL, so impact is modest but real
for long-lived SKUs. Apply with CREATE OR REPLACE FUNCTION in db/functions.sql AND on the live DB.
Fix 12: exclude_from_forecast removes products from product_metrics entirely
update_product_metrics.sql line ~627 (WHERE s.exclude_forecast IS FALSE OR ... IS NULL): the
flag's name implies forecast-only, but excluded products get NO metrics row → vanish from brand/
vendor/category rollups and dashboards. Fix: always emit the row; instead NULL the
forecast/replenishment columns when excluded (wrap those expressions in
CASE WHEN s.exclude_forecast THEN NULL ELSE ... END).
Fix 13: Incremental products import misses category-only changes
products.js incremental WHERE (~lines 433-440) keys on p.stamp, ci.stamp, price/b2b dates —
product_category_index changes don't bump any of those → PG product_categories goes stale. Also
the needs_update comparison (~lines 604-625) doesn't compare categories, so even refetched rows
skip the category rewrite. Fix both: add t.categories IS NOT DISTINCT FROM p.categories to the
needs_update comparison (note: products.categories is the GROUP_CONCAT string — confirm PG column
holds the same representation), and add a cheap full-sweep (e.g. weekly, or compare
COUNT(*) GROUP BY pid hashes) OR include EXISTS (SELECT 1 FROM product_category_index pci WHERE pci.pid = p.pid AND pci.stamp > ?) in the incremental WHERE if that table has a stamp column —
verify schema first (DESCRIBE product_category_index).
Fix 14: PO/receivings OFFSET pagination over a moving filter
purchase-orders.js (~lines 275-298, 447-470): LIMIT/OFFSET with a date_updated > ? predicate;
concurrent updates shift rows between pages → silent skips. Fix: keyset pagination —
WHERE ... AND p.po_id > ? ORDER BY p.po_id LIMIT 500, carrying the last seen po_id (drop OFFSET).
Same for receivings on receiving_id.
Fix 15: Status map gaps and unsafe defaults
orders.jsorderStatusMap lacks 45 (payment_pending) and 67 (remote_send) → imported as numeric strings. Add both (mirror inmigrations/001_map_order_statuses.sqlas a follow-up update for existing rows).purchase-orders.jspoStatusMap[po.status] || 'created'(line ~335): an unknown cancel-like code would be treated as an open PO and inflate on-order FIFO. Default to a sentinel like'unknown_<code>'instead, and make the FIFO/on-order CTEs inupdate_product_metrics.sqltreat only the known-open statuses as open (they already whitelist open statuses — so the sentinel is safe there; just ensure nothing treats unknown as 'created'). Same for receivingStatusMap.
Fix 16: Transactions issued through the pool wrapper land on arbitrary connections
categories.js (lines ~17-152) and daily-deals.js (~27-130) call query('BEGIN') /
query('COMMIT') on the wrapper, which checks out a client per call — BEGIN/work/COMMIT are not
guaranteed to share a connection (works only by pool-LIFO accident). The categories
DISABLE TRIGGER rides on this too. Fix: use the wrapper's beginTransaction()/commit()/rollback()
(see utils.js lines 121-148) exactly as orders.js does. In categories.js also move the
post-COMMIT ENABLE TRIGGER inside the transaction (DISABLE/ENABLE both inside), or drop the
trigger toggling entirely if the trigger isn't actually problematic anymore.
Fix 17: stock-snapshots import swallows batch errors → permanent holes
stock-snapshots.js (~lines 153-155): a failed batch is logged and skipped, but the next
incremental starts at MAX(snapshot_date) — the hole is never revisited. Fix: rethrow (fail the
step) or collect failed date ranges and retry once, then fail if still failing. Also line ~168:
calculateRate(processedRows, startTime) — arguments reversed (signature is
calculateRate(startTime, current), see metrics-new/utils/progress.js:70).
Fix 18: Metrics cancellation targets an application_name that's never set
calculate-metrics-new.js line ~180 cancels backends WHERE application_name = 'node-metrics-calculator', but the Pool config never sets it → cancellation no-ops (the 30-min
statement_timeout is the only real guard). Fix: add application_name: 'node-metrics-calculator'
to both dbConfig branches.
Fix 19: Aggregate-table change-detection lists miss cost-only changes
calculate_brand_metrics.sql / calculate_vendor_metrics.sql / calculate_category_metrics.sql
ON CONFLICT WHERE lists don't include profit_30d/cogs_30d — a cost revision with unchanged
sales/revenue leaves stale rows (product_metrics has a 1-day staleness net; rollups don't). Add
... OR x.profit_30d IS DISTINCT FROM EXCLUDED.profit_30d OR x.cogs_30d IS DISTINCT FROM EXCLUDED.cogs_30d to each, or add a last_calculated < NOW() - INTERVAL '1 day' net like
product_metrics line ~707.
Fix 20: Snapshot stale-detection only compares unit counts
update_daily_snapshots.sql lines ~57-85: detects mismatches in units_sold/units_received only;
price/discount/costeach corrections older than the 2-day recheck are never repaired. Add a
revenue comparison to the stale check: compare SUM(net_revenue) per day against the equivalent
recomputed from orders (ROUND both to 2dp to avoid float-noise churn).
Fix 21: Category metrics positive-only revenue asymmetry
calculate_category_metrics.sql (lines ~27-36, 64-73): revenue summed only when > 0 while
cogs/profit use COALESCE-all → margin numerator/denominator from different populations, and
inconsistent with brand/vendor (plain COALESCE). Change the revenue/sales CASEs to
COALESCE(pm.revenue_7d, 0) etc., matching brand_metrics.
Fix 22 (decision needed): Demand-pattern & seasonality definitions
classify_demand_pattern(db/functions.sql): CV thresholds 0.2/0.5 + avg<1/day. Industry standard is Syntetos-Boylan: ADI ≥ 1.32 and CV² ≥ 0.49 quadrants (smooth/erratic/intermittent/lumpy). Today everything classifies sporadic/lumpy. If adopting SB: ADI = 30 / COUNT(days with sales), CV² computed on nonzero-demand sizes. Changes the vocabulary consumed by the forecast engine (scripts/forecast/forecast_engine.pyreadsdemand_pattern) — coordinate before changing.- SeasonalityAnalysis (
update_product_metrics.sql~360):month_avg = AVG(units_sold)over rows with sales only → intensity, not volume. Use monthly totals (SUM, with zero months counted) / overall monthly average for the index. - Safety stock: currently static config units;
sales_std_dev_30dexists but is unused. Optional upgrade:safety = z * σ_d * sqrt(lead_time)with z from a service-level setting.
These change user-facing semantics — confirm with Matt before implementing.
Verified non-issues (no action, or cleanup only)
costeachfallbackprice * 0.5(orders.js line ~615): fires on 2.1% of item rows (729/34,833, last 30d, live-verified). Accepted by Matt — 50% margin is a fair estimate for these products. Optional: nothing.- Missing-product order skips: zero occurrences — MySQL has no orphan order_items (1-year check),
PG products is a superset of MySQL products (687,579 vs 687,576), last 7 import runs all logged
totalSkipped: 0. Cleanup only: remove the unusedimportMissingProductsimport line atorders.js:2(the function itself stays in products.js — harmless utility). - Status 30 'cancelled_old' in
total_sold >= 20filter: zero rows live in_order— safe. - Duplicate (order_id, pid) order items: none exist in MySQL — the upsert PK is safe.
- base_discount in orders.js: computed/stored in temp table but unused since migration 002 — remove the column from temp table + queries for clarity (no behavior change).
full-update.jsrunScript: try/catch aroundconsole.logis dead code; per-stepstatus:'complete'messages could confuse a UI parser. Cosmetic only — tidy if touching the file.
Suggested implementation order
| Step | Fixes | Re-import/rebuild needed |
|---|---|---|
| 1 | Fix 1 + Fix 2 (+ Fix 5 filters, Fix 8/9 while editing the same SQL) | FULL orders re-import → snapshot rebuild → metrics (once) |
| 2 | Fix 4 + Fix 6 (orders.js reconciliation + watermarks; POs/products watermarks too) | no |
| 3 | Fix 3, Fix 7 (metrics SQL only) | metrics run |
| 4 | Fix 13-21 (robustness batch) | no |
| 5 | Fix 10-12, Fix 22 after Matt's sign-off (definition changes) | metrics run |
After step 1, expect: margin_30d down ~8-10 points, discounts_30d ~3x up, daily curves shifted to correct business days. Communicate before/after so the change isn't mistaken for a data incident.
Reference: verification snippets used in the review
-- MySQL: item-level discounts dropped by the gate (30d)
SELECT COUNT(DISTINCT o.order_id), ROUND(SUM(odi.amount),2)
FROM order_discount_items odi
JOIN order_discounts od ON od.order_id=odi.order_id AND od.discount_id=odi.discount_id
JOIN _order o ON o.order_id=odi.order_id
WHERE odi.which=2 AND o.date_placed >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
AND o.order_status >= 20 AND COALESCE(od.discount_amount_subtotal,0)=0;
-- → 2,021 orders / $25,989 (2026-06-10)
-- MySQL: costeach fallback frequency (30d)
SELECT COUNT(*),
SUM(CASE WHEN NOT EXISTS (SELECT 1 FROM order_costs oc WHERE oc.orderid=oi.order_id
AND oc.pid=oi.prod_pid AND oc.pending=0)
AND NOT EXISTS (SELECT 1 FROM product_inventory pi WHERE pi.pid=oi.prod_pid)
THEN 1 ELSE 0 END)
FROM order_items oi JOIN _order o ON o.order_id=oi.order_id
WHERE o.order_status >= 20 AND o.date_placed >= DATE_SUB(CURDATE(), INTERVAL 30 DAY);
-- → 729 / 34,833 = 2.1% (2026-06-10)
-- PG: timezone check
SHOW timezone; -- Europe/Berlin (2026-06-10)
-- Row drift, May 2026: MySQL 49,377 items / PG 49,841 (+0.9%)