Compare commits
13 Commits
c0f4f1de0d
..
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 069a44bd54 | |||
| 3b2f51e6b8 | |||
| 9ff744399f | |||
| 3e38d0e5ce | |||
| 8c707e28ea | |||
| 421b3d5922 | |||
| cfe3b29c98 | |||
| e83d975bd6 | |||
| cf71cc4dec | |||
| 4be0f877fa | |||
| 82e568d455 | |||
| 1ab14ba45f | |||
| 36f23b527e |
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,343 @@
|
|||||||
|
# Forecast Accuracy Fix Plan
|
||||||
|
|
||||||
|
**Written:** 2026-06-10, from a code + live-data review of the forecasting pipeline.
|
||||||
|
**Goal:** eliminate the systematic ~1.7–2x over-forecast bias, recover demand the model currently ignores, and fix the accuracy measurement so improvements are visible and long-lead forecasts are validated.
|
||||||
|
|
||||||
|
Read this whole document before starting. Fixes are grouped into phases; each phase is independently deployable and has its own validation step. Line numbers are as of 2026-06-10 — re-locate by function name if the file has drifted.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Diagnosis summary (measured 2026-06-10)
|
||||||
|
|
||||||
|
The dashboard headline is **202% WMAPE**. Decomposition of that number, all measured against `forecast_accuracy` run 129 and ad-hoc queries:
|
||||||
|
|
||||||
|
| Finding | Evidence |
|
||||||
|
|---|---|
|
||||||
|
| Daily-grain WMAPE has a ~190% *floor* for this catalog | Avg demand ≈ 0.11 units/product/day. A perfect rate forecast of intermittent demand scores ≈ 2e^−λ ≈ 190%. A trivial trailing-30d-average naive forecast scores **204%** on the same products/days; the engine scores 221% (slightly *worse than naive*). |
|
||||||
|
| Same forecasts at 21-day-per-product grain: **109%**; bias-corrected: **75%** | Half the headline is metric grain, most of the rest is bias. |
|
||||||
|
| Aggregate over-forecast **+70%** (227,690 forecast vs 133,861 actual units) | Portfolio daily ratio is 1.5–2.5x on most days. |
|
||||||
|
| Decay phase 2.47x over (fc 51,675 / act 20,915) | Root cause F1: velocity inflated **4.07x** (measured: 1.353 vs true 0.332 units/day) by averaging over sparse snapshot rows. |
|
||||||
|
| Preorder phase 2.15x over (fc 67,212 / act 31,189) | Root cause F4: launch curve applied at age=0 starting *today*, ignoring that the product hasn't arrived. |
|
||||||
|
| Mature phase 1.69x over (fc 57,857 / act 34,313) | Root causes F2 (history edge truncation) + F3 (seasonal double-count). |
|
||||||
|
| Dormant products sold **16,180 units** (~11% of demand) against zero forecasts | Root cause F5; also excluded from the headline metric, so invisible. |
|
||||||
|
| All 879,800 accuracy samples are in the **1–7d lead bucket** | Root cause F7: archiving design only ever saves yesterday's slice. 30–90d forecasts (what purchasing uses) are never validated. |
|
||||||
|
| Launch phase is healthy: WMAPE 100%, bias −6%, beats naive | The lifecycle-curve concept works; its calibration inputs are broken. Don't redesign it. |
|
||||||
|
|
||||||
|
**Key data fact** underlying several fixes: `daily_product_snapshots` is **activity-based and sparse** — only ~500–1,800 of ~38K products have a row on a given day. Verified: every pid-day with an order DOES have a snapshot row and units match (5,234/5,234 pid-days, 8,980 vs 8,984 units over 7 days). So *missing row = zero sales*, and any query that aggregates over only the rows that exist is averaging over sold-days.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Environment & operational notes
|
||||||
|
|
||||||
|
- **Files:** engine is `inventory-server/scripts/forecast/forecast_engine.py`; orchestrator `run_forecast.js` in the same dir; consumer endpoints in `inventory-server/src/routes/dashboard.js` (`/forecast/metrics` ~line 308, `/forecast/accuracy` ~line 647); overview UI in `inventory/src/components/overview/ForecastMetrics.tsx` and `ForecastAccuracy.tsx`.
|
||||||
|
- **Local `inventory-server/` is NFS-mounted to `/var/www/inventory/` on the netcup server.** Edits made locally appear on the server immediately — no copy step. Do NOT run bulk `grep`/`find`/`node --check` over `inventory-server/` locally (the mount hangs); `ssh netcup` and run them there.
|
||||||
|
- **Avoid the glob tool** for search in this repo; use bash (`grep`/`rg` via ssh for server-side trees).
|
||||||
|
- **Scheduling:** the engine runs daily at **09:30:01 server time** (runs table is conclusive), but the cron entry is NOT in matt's crontab, `/etc/cron.d`, or pm2. Likely root's crontab (`sudo crontab -l` to confirm). You do not need to touch the schedule for these fixes; just know a run fires at 09:30 daily and occasionally skips days (e.g. 2026-06-07/08).
|
||||||
|
- **Manual test runs:** `ssh netcup`, then `cd /var/www/inventory/scripts/forecast && node run_forecast.js`. Takes ~3.5–4 min. Safe to run any time: the engine TRUNCATEs and rebuilds `product_forecasts`, archives prior past-dated rows, and records a new `forecast_runs` row. Python deps live in the server venv (`venv/`); `run_forecast.js` handles env + venv automatically.
|
||||||
|
- **DB access for validation:** `ssh netcup`, then `PGPASSWORD=6D3GUkxuFgi2UghwgnUd psql -h localhost -U inventory_readonly -d inventory_db`. The engine itself connects with the write user via env vars loaded from `/var/www/inventory/.env` — schema changes should be made idempotently *inside the engine code* (the file already uses `CREATE TABLE IF NOT EXISTS` / `CREATE INDEX IF NOT EXISTS`; use `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` the same way) so no manual migration is needed.
|
||||||
|
- **Python gotchas already handled in this file (don't regress):** numpy types must go through the registered psycopg2 adapters; `pd.Series.combine_first()` keeps zeros over real data — use `reindex(..., fill_value=0.0)`.
|
||||||
|
- Engine runtime budget: currently ~212–227s. Phases 1–2 shouldn't move it meaningfully; Phase 3's extra archiving adds one INSERT…SELECT. If runtime balloons past ~6 min, investigate before shipping.
|
||||||
|
- `--backfill` mode (`backfill_accuracy_data`) is an in-sample backtest using the *old* formulas. **Do not run it anymore**; there is enough real out-of-sample history. Updating it to match the new logic is optional/low priority (F11).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1 — Bias bugs in the engine (no schema changes)
|
||||||
|
|
||||||
|
### F1. Decay velocity: stop averaging over sparse snapshot rows
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py`, `batch_load_product_data()`, the decay query (~lines 697–710).
|
||||||
|
|
||||||
|
**Problem:** `AVG(COALESCE(dps.units_sold, 0))` runs over only the snapshot rows that exist — mostly sold-days. Measured inflation on the current 975 decay products: **4.07x** (1.353 vs 0.332 true units/day). This feeds `compute_scale_factor()` for the decay phase and is the single largest bias source.
|
||||||
|
|
||||||
|
**Fix:** divide the sum by calendar days in the window, clipped to the product's age (decay products are 14–60 days old, so a 20-day-old product's window is 20 days, not 30):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT dps.pid,
|
||||||
|
SUM(COALESCE(dps.units_sold, 0))::float
|
||||||
|
/ GREATEST(LEAST(30, (CURRENT_DATE - pm.date_first_received::date)), 1) AS avg_daily
|
||||||
|
FROM daily_product_snapshots dps
|
||||||
|
JOIN product_metrics pm ON pm.pid = dps.pid
|
||||||
|
WHERE dps.pid = ANY(%s)
|
||||||
|
AND dps.snapshot_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||||
|
AND dps.snapshot_date >= pm.date_first_received::date
|
||||||
|
GROUP BY dps.pid, pm.date_first_received
|
||||||
|
```
|
||||||
|
|
||||||
|
No Python-side changes needed; `data['decay_velocity']` keeps the same shape. Products with zero snapshot rows in the window still get no entry → existing `scale = 1.0` fallback applies (acceptable: decay classification requires `sales_velocity_daily > 0`, so truly dead products don't reach this path).
|
||||||
|
|
||||||
|
### F2. Mature history: reindex over the full calendar window
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py`, `forecast_mature()` (~lines 833–836).
|
||||||
|
|
||||||
|
**Problem:** `hist.set_index('snapshot_date').resample('D').sum()` only spans first-snapshot → last-snapshot. Interior gaps correctly become zeros, but **leading and trailing quiet periods are absent**, so the Holt level is fitted on the product's busy span. A marginal mature product whose activity clusters in 2 of the last 8 weeks gets a level ~4x too high.
|
||||||
|
|
||||||
|
**Fix:** replace the resample with an explicit reindex over the full `EXP_SMOOTHING_WINDOW` ending yesterday:
|
||||||
|
|
||||||
|
```python
|
||||||
|
hist = history_df.copy()
|
||||||
|
hist['snapshot_date'] = pd.to_datetime(hist['snapshot_date'])
|
||||||
|
hist = hist.set_index('snapshot_date')['units_sold']
|
||||||
|
full_index = pd.date_range(
|
||||||
|
end=pd.Timestamp(date.today() - timedelta(days=1)),
|
||||||
|
periods=EXP_SMOOTHING_WINDOW, freq='D')
|
||||||
|
series = hist.reindex(full_index, fill_value=0.0).values.astype(float)
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes: (pid, snapshot_date) is unique in `daily_product_snapshots`, so no duplicate-index risk. `observed_mean` and the `cap` recompute over the full window automatically (intended — the cap gets correspondingly tighter). Mature products are by definition >60 days old, so the 60-day window never predates first receipt. Do NOT use `combine_first` (see gotchas above).
|
||||||
|
|
||||||
|
### F3. Stop double-applying the monthly seasonal index
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py`, `generate_all_forecasts()` — the `seasonal_multipliers` pre-compute (~lines 959–961) and application (~line 1050).
|
||||||
|
|
||||||
|
**Problem:** every per-product calibration (decay velocity, mature Holt level, launch first-week scale, preorder rate, slow-mover velocity) is fitted on *raw recent actuals*, which already embed the current month's seasonal level. The forecast then multiplies by the **absolute** monthly index of the target date. Example from the live indices (`forecast_runs.phase_counts` for run 129): May = 1.224 (sale month), June = 0.982. Early-June forecasts were calibrated on May-sale-inflated velocities and barely discounted — a structural ~25% over-forecast at that transition, and it'll be worse around November (1.316).
|
||||||
|
|
||||||
|
**Fix:** apply the seasonal index *relative to the calibration period*. Compute a calibration index as the average monthly index over the trailing 30 calendar days (robust at month boundaries), then divide:
|
||||||
|
|
||||||
|
```python
|
||||||
|
today = date.today()
|
||||||
|
trailing = [today - timedelta(days=i) for i in range(1, 31)]
|
||||||
|
calibration_index = float(np.mean([monthly_indices.get(d.month, 1.0) for d in trailing]))
|
||||||
|
seasonal_multipliers = [
|
||||||
|
monthly_indices.get(d.month, 1.0) / max(calibration_index, 0.1)
|
||||||
|
for d in forecast_dates
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
Leave the DOW multipliers absolute — every calibration is a multi-week average and therefore DOW-neutral, so reshaping by absolute DOW indices is correct.
|
||||||
|
|
||||||
|
**Optional sub-fix (same area, low priority):** the monthly indices are computed from a single trailing 365-day window, so each month appears once and YoY growth contaminates "seasonality". A cheap improvement is widening `SEASONAL_LOOKBACK_DAYS` to 730 and averaging the two observations of each month. Do this only after the main fixes are validated.
|
||||||
|
|
||||||
|
### Phase 1 validation
|
||||||
|
|
||||||
|
Deploy (edit locally; NFS propagates), run the engine manually once, wait for 3–5 daily cycles, then:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Portfolio ratio per day (target: drifts from ~2.0 toward 0.8–1.3)
|
||||||
|
WITH ranked AS (
|
||||||
|
SELECT pfh.pid, pfh.forecast_date, pfh.forecast_units, pfh.lifecycle_phase,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY pfh.pid, pfh.forecast_date ORDER BY fr.started_at DESC) rn
|
||||||
|
FROM product_forecasts_history pfh
|
||||||
|
JOIN forecast_runs fr ON fr.id = pfh.run_id
|
||||||
|
WHERE pfh.forecast_date >= CURRENT_DATE - 7)
|
||||||
|
SELECT r.forecast_date, round(SUM(r.forecast_units),0) AS fc,
|
||||||
|
SUM(COALESCE(dps.units_sold,0)) AS act,
|
||||||
|
round(SUM(r.forecast_units)/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS ratio
|
||||||
|
FROM ranked r
|
||||||
|
LEFT JOIN daily_product_snapshots dps ON dps.pid = r.pid AND dps.snapshot_date = r.forecast_date
|
||||||
|
WHERE r.rn = 1 AND r.lifecycle_phase != 'dormant'
|
||||||
|
GROUP BY 1 ORDER BY 1;
|
||||||
|
```
|
||||||
|
|
||||||
|
Also check `forecast_accuracy` `by_phase` rows for the newest run: decay bias should fall from +0.35 toward ~0, mature from +0.17 toward ~0. (Accuracy lags ~1 day behind each fix since it evaluates yesterday's forecasts.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2 — Demand the model currently ignores or mistimes
|
||||||
|
|
||||||
|
### F4. Preorder: forecast the preorder rate until arrival, launch curve after
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py` — `batch_load_product_data()` (add arrival dates), `generate_all_forecasts()` preorder branch (~lines 1005–1009), and `forecast_from_curve()` (or a small wrapper).
|
||||||
|
|
||||||
|
**Problem:** preorder products run the launch curve from `age=0` starting **today**, i.e. full first-week launch sales while the product is still weeks from arriving. Actual preorder-period sales are a much slower trickle.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
1. Batch-load each preorder product's expected arrival from `purchase_orders` (line-item grain: it has `pid` and `expected_date` directly). Open statuses verified against live data: `created`, `ordered`, `electronically_sent`, `receiving_started` (~705 open line items currently have a future `expected_date`):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT pid, MIN(expected_date) AS expected_arrival
|
||||||
|
FROM purchase_orders
|
||||||
|
WHERE pid = ANY(%s)
|
||||||
|
AND status IN ('created', 'ordered', 'electronically_sent', 'receiving_started')
|
||||||
|
AND expected_date IS NOT NULL
|
||||||
|
AND expected_date >= CURRENT_DATE
|
||||||
|
GROUP BY pid
|
||||||
|
```
|
||||||
|
|
||||||
|
Fallbacks, in order: (a) an open PO with a *past* `expected_date` → assume arrival in 7 days; (b) no PO at all → arrival in 14 days (and log a counter of how many hit this default).
|
||||||
|
|
||||||
|
2. In the preorder branch, build the daily array piecewise. Let `days_until_arrival = (expected_arrival - today).days`:
|
||||||
|
- Days `0 .. days_until_arrival-1`: flat observed preorder daily rate = `preorder_sales[pid] / max(preorder_days[pid], 1)` (both already batch-loaded), clamped to ≤ the curve's scaled week-0 daily value.
|
||||||
|
- Days `days_until_arrival .. horizon`: `forecast_from_curve(curve_info, scale, age_days=0, ...)` shifted so the curve's day 0 lands on the arrival date (i.e. pass `horizon_days - days_until_arrival` and offset into the output array).
|
||||||
|
- Keep the existing `compute_scale_factor('preorder', ...)` for the post-arrival curve; the pre-arrival segment doesn't use it.
|
||||||
|
|
||||||
|
This is consistent with how the reference curves were built: historical preorder units were recorded on their **order dates** (pre-arrival), so week-0 of the fitted curves reflects post-receipt orders, not the backlog.
|
||||||
|
|
||||||
|
### F5. Dormant products: small positive rate instead of hard zero, and count them
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py` — `generate_all_forecasts()` dormant branch (~lines 1040–1042), `batch_load_product_data()`, and `compute_accuracy()`.
|
||||||
|
|
||||||
|
**Problem:** all ~28K dormant products are forecast at exactly 0, yet they sold 16,180 units in the eval window (~11% of all demand) — restocks, promos, long-tail. Worse, dormant is *excluded* from the headline accuracy filter, so this miss is invisible.
|
||||||
|
|
||||||
|
**Fix (cheap version, do this now):**
|
||||||
|
|
||||||
|
1. Batch-load a trailing-180-day order rate for dormant products (11,362 of them have ≥1 sale in 180d — verified):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT o.pid, SUM(o.quantity) / 180.0 AS rate
|
||||||
|
FROM orders o
|
||||||
|
WHERE o.pid = ANY(%s)
|
||||||
|
AND o.canceled IS DISTINCT FROM TRUE
|
||||||
|
AND o.date >= CURRENT_DATE - INTERVAL '180 days'
|
||||||
|
GROUP BY o.pid
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Dormant branch: if the product has a rate > 0, forecast it flat with `method = 'velocity'`; else keep zeros with `method = 'zero'`. Apply the same DOW/seasonal multipliers as everything else (automatic — they're applied after the branch).
|
||||||
|
3. In `compute_accuracy()`, add a second overall row: `metric_type='overall', dimension_value='all_incl_dormant'` with no dormant filter (keep the existing `'all'` row unchanged for trend continuity). One extra entry in the `dimensions`/`filter_clauses` dicts.
|
||||||
|
|
||||||
|
**Upgrade path (optional, Phase 4):** replace flat rates for `slow_mover` + dormant-with-sales with TSB (Teunter–Syntetos–Babai), the standard intermittent-demand method with obsolescence handling. Per product over a daily series `d_t` (build it from snapshots the F2 way — full calendar reindex):
|
||||||
|
|
||||||
|
```
|
||||||
|
if d_t > 0: p_t = p_{t-1} + β·(1 − p_{t-1}); z_t = z_{t-1} + α·(d_t − z_{t-1})
|
||||||
|
else: p_t = p_{t-1}·(1 − β); z_t = z_{t-1}
|
||||||
|
forecast = p_T · z_T (flat across horizon)
|
||||||
|
```
|
||||||
|
|
||||||
|
Start with α=0.1, β=0.05, initialize p = (nonzero days / total days), z = mean of nonzero demands. Scope: slow_mover (~6K) + dormant with 180d sales (~11K); series from up to 180 days of snapshots (sparse rows → ~manageable volume). Only do this after Phase 3 measurement exists to prove it beats the flat rates.
|
||||||
|
|
||||||
|
### Phase 2 validation
|
||||||
|
|
||||||
|
After 3–5 cycles: preorder `by_phase` bias should drop from +0.85 toward < +0.3; the new `all_incl_dormant` row should appear and its `total_actual_units` minus `'all'`'s should be largely *covered* rather than all-miss (dormant `bias` rising from −1.36 toward ~−0.3 or better).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3 — Fix the measurement (schema + engine + API + UI)
|
||||||
|
|
||||||
|
> Without this phase you cannot see whether Phases 1–2 worked except by ad-hoc SQL, the lead-time chart stays a single bucket forever, and the dashboard keeps displaying a number with a 190% floor in red.
|
||||||
|
|
||||||
|
### F7. Archive long-lead forecasts so 15/30/60/90d accuracy exists
|
||||||
|
|
||||||
|
**Where:** `forecast_engine.py` — `archive_forecasts()` (~lines 1086–1154), `compute_accuracy()` CTE (~lines 1201–1228).
|
||||||
|
|
||||||
|
**Problem:** the current design archives only *past-dated* rows of the previous run before truncation. With daily runs, that's only ever the 1-day-ahead slice — all 879,800 accuracy samples sit in the '1-7d' bucket and the longer buckets in the UI chart can never populate. Purchasing decisions ride on 30–60d forecasts that are never validated.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
1. Keep the existing past-date archiving exactly as is (it provides dense short-lead coverage).
|
||||||
|
2. After `generate_all_forecasts()` completes, additionally archive a **sampled set of future leads** from the new run, non-dormant only, attributed to the *current* run id (correct attribution, unlike the past-date path which attributes to the previous run):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
INSERT INTO product_forecasts_history
|
||||||
|
(run_id, pid, forecast_date, forecast_units, forecast_revenue,
|
||||||
|
lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at)
|
||||||
|
SELECT %(run_id)s, pid, forecast_date, forecast_units, forecast_revenue,
|
||||||
|
lifecycle_phase, forecast_method, confidence_lower, confidence_upper, generated_at
|
||||||
|
FROM product_forecasts
|
||||||
|
WHERE lifecycle_phase != 'dormant'
|
||||||
|
AND forecast_date - CURRENT_DATE IN (7, 14, 30, 60, 89)
|
||||||
|
ON CONFLICT (run_id, pid, forecast_date) DO NOTHING
|
||||||
|
```
|
||||||
|
|
||||||
|
Volume: ~10K non-dormant products × 5 leads ≈ 50K rows/day; the existing 90-day prune (`forecast_date < CURRENT_DATE - 90`) bounds steady state at a few million rows. Note future-dated rows survive until their date passes + 90 days — that's intended.
|
||||||
|
|
||||||
|
3. **CRITICAL companion change** in `compute_accuracy()`: the accuracy CTE must now exclude not-yet-realized rows, or future-dated archives get scored against actual=0:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
FROM product_forecasts_history pfh
|
||||||
|
JOIN forecast_runs fr ON fr.id = pfh.run_id
|
||||||
|
WHERE pfh.forecast_date < CURRENT_DATE -- ADD THIS
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Dedup semantics change.** Today's `ROW_NUMBER() OVER (PARTITION BY pid, forecast_date ORDER BY started_at DESC)` keeps only the latest (= shortest-lead) row per pid/date, which would silently discard all the new long-lead rows. Restructure:
|
||||||
|
- Compute `lead_days = forecast_date - started_at::date` and the lead bucket *inside* `ranked_history`.
|
||||||
|
- For `by_lead_time`: dedup `PARTITION BY pid, forecast_date, lead_bucket` (one sample per pid/date/bucket, latest run wins within a bucket).
|
||||||
|
- For everything else (`overall`, `by_phase`, `by_method`, `daily`, and the new weekly metric below): restrict to `lead_days BETWEEN 0 AND 6` and keep the existing per-(pid, date) dedup. This preserves the current meaning of the headline metrics (short-lead) while the lead-time table becomes real.
|
||||||
|
|
||||||
|
### F8. Track a naive baseline (forecast value-added)
|
||||||
|
|
||||||
|
**Where:** `archive_forecasts()` (both INSERT paths), `compute_accuracy()`, `forecast_accuracy` schema, `/forecast/accuracy` endpoint.
|
||||||
|
|
||||||
|
**Problem:** the engine currently *loses* to a trailing-average naive forecast (221% vs 204% daily WMAPE) and nothing on the dashboard would ever reveal that. Every accuracy improvement should be judged as value-over-naive.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
1. Schema (idempotent, in the ensure blocks): `ALTER TABLE product_forecasts_history ADD COLUMN IF NOT EXISTS naive_units NUMERIC(10,2);` and `ALTER TABLE forecast_accuracy ADD COLUMN IF NOT EXISTS naive_wmape NUMERIC(10,4), ADD COLUMN IF NOT EXISTS fva NUMERIC(10,4);`
|
||||||
|
2. Populate `naive_units` during both archive INSERTs via a join — naive = flat trailing-28-day average daily units as of archive time (28 days = DOW-balanced; information available at generation; same value at every lead, which is exactly what a naive baseline means):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
LEFT JOIN (
|
||||||
|
SELECT o.pid, SUM(o.quantity) / 28.0 AS naive_daily
|
||||||
|
FROM orders o
|
||||||
|
WHERE o.canceled IS DISTINCT FROM TRUE
|
||||||
|
AND o.date >= CURRENT_DATE - INTERVAL '28 days' AND o.date < CURRENT_DATE
|
||||||
|
GROUP BY o.pid
|
||||||
|
) nv ON nv.pid = pf.pid
|
||||||
|
-- select COALESCE(nv.naive_daily, 0) AS naive_units
|
||||||
|
```
|
||||||
|
|
||||||
|
3. In `compute_accuracy()`, add to each dimension's aggregate: `SUM(ABS(naive_units - actual_units)) / NULLIF(SUM(actual_units),0) AS naive_wmape` and store `fva = 1 - wmape / naive_wmape` (NULL-safe). Rows archived before this change have `naive_units` NULL — treat NULL as excluded (`FILTER (WHERE naive_units IS NOT NULL)` on the naive sums) rather than as zero.
|
||||||
|
4. Endpoint: include `naiveWmape` and `fva` in the `overall` (and per-phase) payload of `/dashboard/forecast/accuracy` in `dashboard.js`.
|
||||||
|
|
||||||
|
### F9. Weekly-grain headline metric + bias as a percentage
|
||||||
|
|
||||||
|
**Where:** `compute_accuracy()`, `/forecast/accuracy` endpoint, `ForecastAccuracy.tsx`.
|
||||||
|
|
||||||
|
**Problem:** daily-grain WMAPE on this catalog has a ~190% floor — as a headline it's noise. The informative numbers are (a) weekly-per-product WMAPE (currently ~109%, target ~70–85% post-fix) and (b) aggregate bias, which the UI currently renders as `+0.108 units` — indistinguishable from zero while the reality is +70%.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
1. New metric in `compute_accuracy()`: `metric_type='overall_weekly', dimension_value='all'`. Definition: using the short-lead deduped rows (lead ≤ 6, non-dormant), aggregate per `(pid, date_trunc('week', forecast_date))` keeping only complete weeks (`COUNT(*) = 7`), then `WMAPE = SUM(ABS(fc_week − act_week)) / SUM(act_week)`, excluding pid-weeks where both are 0. Store sample_size = number of pid-weeks. Compute `naive_wmape`/`fva` the same way from `naive_units`.
|
||||||
|
2. Endpoint: expose as `overallWeekly`; also add a weekly variant to the `accuracyTrend` query (`metric_type='overall_weekly'`). The trend will start empty (old runs lack the row) — that's fine; don't backfill.
|
||||||
|
3. `ForecastAccuracy.tsx`:
|
||||||
|
- Headline WMAPE → `overallWeekly.wmape`, labeled "WMAPE (weekly)". Keep daily WMAPE available in a tooltip if desired.
|
||||||
|
- Color thresholds for weekly grain: green ≤ 60, yellow ≤ 90, red above (tunable; document that they're calibrated for intermittent retail demand).
|
||||||
|
- Replace the bias row: show `(totalForecast / totalActual − 1)` as a signed percentage labeled "Forecast vs actual" (both totals already arrive in `overall`). Keep MAE.
|
||||||
|
- Add a "vs naive" line: naive weekly WMAPE and FVA. FVA > 0 = engine adds value.
|
||||||
|
- The lead-time chart needs no code change — buckets will populate as F7 rows mature (7d lead evaluable after 7 days, 30d after 30, etc.).
|
||||||
|
4. `confidenceLevel` in `/forecast/metrics` ([dashboard.js ~line 360]) is "share of products forecast via lifecycle curves", not confidence. It only feeds a per-day tooltip field — rename the JSON field to `curveCoverage` and update the one consumer in `ForecastMetrics.tsx`, or leave it and add a comment; low priority.
|
||||||
|
|
||||||
|
### Phase 3 validation
|
||||||
|
|
||||||
|
- Next run after deploy: `forecast_accuracy` contains `overall_weekly` and `fva` values; `/dashboard/forecast/accuracy` returns them; the overview popover renders weekly WMAPE, bias %, and the naive comparison.
|
||||||
|
- After 7/14/30 days: `by_lead_time` rows appear for '8-14d', '15-30d', '31-60d' buckets respectively (61-90d after ~60 days).
|
||||||
|
- Confirm engine runtime still < ~5 min and `product_forecasts_history` growth ≈ 50–70K rows/day.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4 — Optional / after the above is proven
|
||||||
|
|
||||||
|
- **F6. TSB for slow movers + dormant** (spec in F5). Gate on Phase 3 measurement: ship only if weekly FVA improves on those phases.
|
||||||
|
- **F10. Confidence-margin source:** `load_accuracy_margins()` feeds daily-grain per-phase WMAPE (clamped to 1.0) into the intervals, so every interval is ±100% — uninformative. Once `overall_weekly` exists, add per-phase weekly rows (`by_phase_weekly`) and source margins from those instead.
|
||||||
|
- **F11.** Update or delete `backfill_accuracy_data()` (it encodes the old formulas). Until then, just don't run `--backfill`.
|
||||||
|
- **F12.** `compute_dow_indices()` weights by revenue but the multipliers are applied to units — switch `SUM(o.price * o.quantity)` to `SUM(o.quantity)`. Tiny effect.
|
||||||
|
- **F13.** Longer term: for reorder decisions the right target is P(lead-time demand > stock), not a point forecast. Evaluate quantile (pinball) loss at lead-time horizons using the existing confidence-interval columns. Design separately.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Success criteria
|
||||||
|
|
||||||
|
1. Rolling-14-day portfolio forecast/actual ratio within **0.8–1.25** (currently 1.5–2.5).
|
||||||
|
2. Weekly-grain WMAPE ≤ **90%** and **FVA > 0** (engine beats naive) sustained for 2+ weeks.
|
||||||
|
3. Decay/preorder/mature per-phase bias within ±0.1 units/day (currently +0.35 / +0.85 / +0.17).
|
||||||
|
4. `all_incl_dormant` actuals covered: dormant bias better than −0.4 (currently −1.36, i.e. 100% miss).
|
||||||
|
5. Lead-time buckets through 31–60d populated with ≥10K samples each within ~6 weeks.
|
||||||
|
6. Launch phase stays healthy (bias within ±0.15, WMAPE not degraded) — regression guard for F3/F4 changes.
|
||||||
|
|
||||||
|
## 5. Re-measurement appendix
|
||||||
|
|
||||||
|
The naive-vs-engine comparison used in the diagnosis (rerun any time; adjust dates):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
WITH ranked AS (
|
||||||
|
SELECT pfh.pid, pfh.forecast_date, pfh.forecast_units, pfh.lifecycle_phase,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY pfh.pid, pfh.forecast_date ORDER BY fr.started_at DESC) rn
|
||||||
|
FROM product_forecasts_history pfh
|
||||||
|
JOIN forecast_runs fr ON fr.id = pfh.run_id
|
||||||
|
WHERE pfh.forecast_date BETWEEN CURRENT_DATE - 9 AND CURRENT_DATE - 1),
|
||||||
|
eng AS (SELECT * FROM ranked WHERE rn = 1 AND lifecycle_phase != 'dormant'),
|
||||||
|
naive AS (
|
||||||
|
SELECT o.pid, SUM(o.quantity)/30.0 AS naive_daily FROM orders o
|
||||||
|
WHERE o.canceled IS DISTINCT FROM TRUE
|
||||||
|
AND o.date >= CURRENT_DATE - 39 AND o.date < CURRENT_DATE - 9
|
||||||
|
GROUP BY o.pid)
|
||||||
|
SELECT e.lifecycle_phase, COUNT(*) AS n, SUM(COALESCE(dps.units_sold,0)) AS actual,
|
||||||
|
round(SUM(e.forecast_units),0) AS engine_fc, round(SUM(COALESCE(nv.naive_daily,0)),0) AS naive_fc,
|
||||||
|
round(SUM(ABS(e.forecast_units - COALESCE(dps.units_sold,0)))/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS engine_wmape,
|
||||||
|
round(SUM(ABS(COALESCE(nv.naive_daily,0) - COALESCE(dps.units_sold,0)))/NULLIF(SUM(COALESCE(dps.units_sold,0)),0),2) AS naive_wmape
|
||||||
|
FROM eng e
|
||||||
|
LEFT JOIN naive nv ON nv.pid = e.pid
|
||||||
|
LEFT JOIN daily_product_snapshots dps ON dps.pid = e.pid AND dps.snapshot_date = e.forecast_date
|
||||||
|
GROUP BY ROLLUP(e.lifecycle_phase) ORDER BY 1;
|
||||||
|
```
|
||||||
|
|
||||||
|
Baseline numbers to beat (June 1–9, 2026): engine 221% / naive 204% daily WMAPE; engine_fc/actual = 1.82; per-phase table in §1.
|
||||||
@@ -0,0 +1,449 @@
|
|||||||
|
# Import & Metrics Pipeline Fix Plan
|
||||||
|
|
||||||
|
Fixes for issues found in a full review (2026-06-10) of the `full-update.js` pipeline:
|
||||||
|
`inventory-server/scripts/full-update.js` → `import-from-prod.js` (6 importers in `scripts/import/`)
|
||||||
|
→ `calculate-metrics-new.js` (7 SQL modules in `scripts/metrics-new/`).
|
||||||
|
|
||||||
|
Every issue below was verified against the code, and where marked **[verified-live]**, against the
|
||||||
|
live MySQL source (`sg` on 192.168.1.5 via the acot-db tooling / `ssh workpi`) and live PostgreSQL
|
||||||
|
(`inventory_db` — `ssh netcup`, then `psql -U inventory_readonly`, password in `/Users/matt/Dev/inventory/CLAUDE.md`).
|
||||||
|
Write credentials for migrations: see `/var/www/inventory/.env` on netcup (`inventory_user`).
|
||||||
|
|
||||||
|
## Operational context (read first)
|
||||||
|
|
||||||
|
- Local `inventory-server/` is **NFS-mounted** to `/var/www/inventory/` on the netcup server — edits
|
||||||
|
appear on the server with no copy step. Run heavy validation/grep/find **on the server via
|
||||||
|
`ssh netcup`**, not locally (NFS hangs + AppleDouble `._*` noise).
|
||||||
|
- The PG server timezone is **Europe/Berlin**. The business operates in **America/Chicago**. This
|
||||||
|
matters for Fix 2.
|
||||||
|
- MySQL server is America/Chicago; the mysql2 driver is configured `timezone: '-05:00'` and
|
||||||
|
corrected at runtime by `adjustDateForMySQL()` in `scripts/import/utils.js` (see
|
||||||
|
`memory/TIMEZONE_ISSUE.md`). Don't "fix" that part — it already works.
|
||||||
|
- Orders/PO/products imports are incremental by default (`INCREMENTAL_UPDATE !== 'false'`); a full
|
||||||
|
orders sync = run with `INCREMENTAL_UPDATE=false` (5-year window).
|
||||||
|
- Existing rebuild tooling: `scripts/metrics-new/backfill/rebuild_daily_snapshots.sql` (rebuilds
|
||||||
|
`daily_product_snapshots` from `orders`/`receivings`). The full-pipeline order after data fixes:
|
||||||
|
re-import → rebuild snapshots → `node scripts/calculate-metrics-new.js`.
|
||||||
|
- Precedent: `scripts/metrics-new/migrations/002_fix_discount_double_counting.sql` documents the
|
||||||
|
procedure used last time a discount formula changed. Follow the same pattern (migration doc +
|
||||||
|
code fix + full re-import + rebuild).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P0 — Data correctness (do both, then ONE re-import + rebuild)
|
||||||
|
|
||||||
|
### Fix 1: Item-level promo discounts dropped (~$26K / 30 days ≈ 10% of product revenue) [verified-live]
|
||||||
|
|
||||||
|
**File:** `scripts/import/orders.js` — `order_totals` CTE (~lines 604-623) and the discount fetch in
|
||||||
|
`processDiscountsBatch` (~lines 379-383).
|
||||||
|
|
||||||
|
**Problem.** The discount applied to each PG `orders` row is:
|
||||||
|
prorated `summary_discount_subtotal` + item-level promo discounts. The item-level part is gated:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SUM(CASE WHEN COALESCE(md.discount_amount_subtotal, 0) > 0 THEN id.amount ELSE 0 END)
|
||||||
|
```
|
||||||
|
|
||||||
|
In the PHP source (`/Users/matt/Dev/acot/website/website/lib/neworder.class.php`):
|
||||||
|
- `order_items.prod_price` is the **pre-promo** price; `summary_subtotal = Σ prod_price·qty` (line ~3087).
|
||||||
|
- Item-level promo discounts live in `order_discount_items` with `which = 2`; they are applied to the
|
||||||
|
order total via `summary_discount += amount + products_disc_sum` (line ~6567) — i.e. they are **not**
|
||||||
|
part of `discount_amount_subtotal` and **not** baked into `prod_price`.
|
||||||
|
- Live data (90 days): of 10,010 type-10 promo discounts, **8,070 have item rows but only 8 have
|
||||||
|
`discount_amount_subtotal > 0`** — the gate zeroes essentially all item-level promo discounts.
|
||||||
|
- Live impact (30 days): **$25,989 dropped** across 2,021 orders, vs only $13,574 captured via the
|
||||||
|
prorated subtotal component. Order discount components, 30d: total $54,957 = $13,574 subtotal +
|
||||||
|
$15,395 shipping + ~$25,989 item-level. (Shipping discounts correctly excluded from product revenue.)
|
||||||
|
|
||||||
|
**Consequence.** `orders.discount` understated → `net_revenue`, `profit_30d`, `margin_30d` overstated
|
||||||
|
by ~10% of revenue; `discounts_30d` / `discount_rate_30d` ~3x understated. Flows into daily snapshots,
|
||||||
|
product/brand/vendor/category metrics, and dashboards.
|
||||||
|
|
||||||
|
**Fix.**
|
||||||
|
1. In `processDiscountsBatch`, fetch only real item discounts:
|
||||||
|
`SELECT order_id, pid, discount_id, amount FROM order_discount_items WHERE order_id IN (?) AND which = 2`.
|
||||||
|
(`which=1` rows store prices of free promo-added items; `which=3` are usage records — neither is a
|
||||||
|
discount amount.)
|
||||||
|
2. In the `order_totals` CTE, remove the gate — sum `id.amount` unconditionally:
|
||||||
|
`SUM(COALESCE(id.amount, 0)) AS promo_discount_sum` (drop the join/CASE on `temp_main_discounts`;
|
||||||
|
`temp_main_discounts` becomes unused and can be removed entirely along with its insert loop).
|
||||||
|
3. Sanity guard (optional, recommended): clamp final per-row discount to `price * quantity`.
|
||||||
|
|
||||||
|
**Verification.** After a FULL orders re-import, for a recent 30-day window PG should satisfy:
|
||||||
|
`SUM(discount)` ≈ MySQL `Σ summary_discount_subtotal` + `Σ order_discount_items.amount (which=2)`
|
||||||
|
over the same orders (± rounding from proration). Spot-check an order with a type-10 promo:
|
||||||
|
discount on the affected pid ≈ the `which=2` amount. Re-run migration 002's verification query too
|
||||||
|
(pids 624756, 614513) to confirm no regression of the prior fix.
|
||||||
|
|
||||||
|
### Fix 2: Daily snapshots bucket sales by Europe/Berlin days, not business days [verified-live]
|
||||||
|
|
||||||
|
**Files:** `scripts/metrics-new/update_daily_snapshots.sql` (SalesData join `o.date::date = _target_date`
|
||||||
|
~line 138; gap-fill and stale-detection aggregates at lines ~47-83);
|
||||||
|
`scripts/metrics-new/backfill/rebuild_daily_snapshots.sql` (same pattern — check & fix);
|
||||||
|
`scripts/metrics-new/update_product_metrics.sql` (`HistoricalDates` `MIN(o.date)::date` etc., lines ~131-147).
|
||||||
|
|
||||||
|
**Problem.** `orders.date` is `timestamptz`; `::date` casts in the server TZ (**Europe/Berlin**,
|
||||||
|
verified via `SHOW timezone`). Berlin is 7-8h ahead of Central, so every order placed after
|
||||||
|
~5 PM Central lands on the **next** snapshot day. This shifts a large evening slice of daily sales
|
||||||
|
forward one day; skews `yesterday_sales`, day-of-week patterns (the forecast engine's DOW
|
||||||
|
multipliers, daily-grain forecast accuracy — see `FORECAST_FIX_PLAN.md`), and is inconsistent with
|
||||||
|
`stock_snapshots`, whose dates come from a Central-time MySQL cron.
|
||||||
|
|
||||||
|
**Fix.** Bucket all order/receiving dates in business time. Replace every `o.date::date` /
|
||||||
|
`received_date::date` used for *day bucketing* in the two snapshot SQL files with:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
(o.date AT TIME ZONE 'America/Chicago')::date
|
||||||
|
```
|
||||||
|
|
||||||
|
Apply consistently in: SalesData, ReceivingData, the gap-fill date lists, the stale-detection
|
||||||
|
aggregates (they must match SalesData or every day looks permanently stale), and the rebuild script.
|
||||||
|
`HistoricalDates` in update_product_metrics (first/last sold dates) should match too.
|
||||||
|
Add an index to keep the per-day loop fast, e.g.
|
||||||
|
`CREATE INDEX ON orders ( ((date AT TIME ZONE 'America/Chicago')::date) );` and equivalent on
|
||||||
|
`receivings(received_date)`; check `EXPLAIN` on the SalesData query afterward.
|
||||||
|
|
||||||
|
Note: `receivings.received_date` came from MySQL DATETIME (Central literal) inserted as timestamptz —
|
||||||
|
it was interpreted in the *session* TZ at insert. Before converting, spot-check a few receivings
|
||||||
|
against MySQL to confirm which TZ the stored instants actually represent; the conversion expression
|
||||||
|
must yield the Central calendar day MySQL shows. Same check for `orders.date` (it originates from
|
||||||
|
`_order.date_placed`, a TIMESTAMP column, so it should be a correct instant — `AT TIME ZONE
|
||||||
|
'America/Chicago'` is right for it).
|
||||||
|
|
||||||
|
**Verification.** Pick 2-3 recent days; compare per-day `units_sold` totals in
|
||||||
|
`daily_product_snapshots` against MySQL
|
||||||
|
`SELECT date_placed_onlydate, SUM(qty_ordered) ... WHERE order_status >= 20 GROUP BY 1`
|
||||||
|
(MySQL stores Central days). They should now match closely (small diffs from canceled-status timing).
|
||||||
|
|
||||||
|
### P0 execution order (single pass)
|
||||||
|
|
||||||
|
1. Land Fix 1 (orders.js) and Fix 2 (both snapshot SQL files + product-metrics date CTE).
|
||||||
|
2. Full orders re-import: `INCREMENTAL_UPDATE=false node scripts/import-from-prod.js` (or at minimum
|
||||||
|
the orders step) — run on the server, it's long.
|
||||||
|
3. Rebuild snapshots: `psql -f scripts/metrics-new/backfill/rebuild_daily_snapshots.sql` (after
|
||||||
|
confirming it contains the TZ fix). The hourly job's 90-day self-heal will NOT fix history beyond
|
||||||
|
90 days by itself; the explicit rebuild is required.
|
||||||
|
4. `node scripts/calculate-metrics-new.js`.
|
||||||
|
5. Expect dashboards to show: margins down ~8-10 points (real), daily sales curves shifted, DOW
|
||||||
|
profile changed. Tell the user before/after numbers.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P1 — Wrong or drifting numbers, fix soon
|
||||||
|
|
||||||
|
### Fix 3: Vendor avg lead time computed over a near-cartesian join
|
||||||
|
|
||||||
|
**File:** `scripts/metrics-new/calculate_vendor_metrics.sql`, `VendorPOAggregates` (lines ~62-83).
|
||||||
|
|
||||||
|
**Problem.** Joins each done-PO line to **every** receiving of the same (pid, supplier) after the PO
|
||||||
|
date — a product received 10 times contributes 10 ever-growing lead times → overstated, busy-product-
|
||||||
|
weighted vendor lead time. The per-product version in `update_periodic_metrics.sql` (lines 27-48)
|
||||||
|
is correct (MIN receiving per PO within 180 days, then average).
|
||||||
|
|
||||||
|
**Fix.** Reuse the periodic shape, aggregated to vendor:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
WITH po_first_receiving AS (
|
||||||
|
SELECT po.vendor, po.po_id, po.pid, po.date::date AS po_date,
|
||||||
|
MIN(r.received_date::date) AS first_receive_date
|
||||||
|
FROM purchase_orders po
|
||||||
|
JOIN receivings r ON r.pid = po.pid AND r.supplier_id = po.supplier_id
|
||||||
|
AND r.received_date >= po.date
|
||||||
|
AND r.received_date <= po.date + INTERVAL '180 days'
|
||||||
|
WHERE po.status = 'done' AND po.date >= CURRENT_DATE - INTERVAL '1 year'
|
||||||
|
AND po.vendor IS NOT NULL AND po.vendor <> ''
|
||||||
|
GROUP BY po.vendor, po.po_id, po.pid, po.date
|
||||||
|
)
|
||||||
|
SELECT vendor, COUNT(DISTINCT po_id) AS po_count_365d,
|
||||||
|
ROUND(AVG(GREATEST(1, first_receive_date - po_date)))::int AS avg_lead_time_days_hist
|
||||||
|
FROM po_first_receiving GROUP BY vendor
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verification.** For a few vendors compare old vs new values; new should be materially lower and
|
||||||
|
roughly match `AVG(product_metrics.avg_lead_time_days)` for that vendor's products.
|
||||||
|
|
||||||
|
### Fix 4: Deleted order items & combined orders never reconciled in PG [verified-live]
|
||||||
|
|
||||||
|
**File:** `scripts/import/orders.js`.
|
||||||
|
|
||||||
|
**Problem.** The orders import upserts but never deletes:
|
||||||
|
- Items removed from an order in MySQL (`DELETE FROM order_items ...` happens, e.g.
|
||||||
|
neworder.class.php ~line 6500 for unpicked promo items, plus staff edits) leave stale rows in PG
|
||||||
|
forever. May 2026 check: PG has 49,841 item rows vs MySQL 49,377 (+0.9%) — and PG should be ≤
|
||||||
|
MySQL.
|
||||||
|
- Combining orders (`combine_orders`, neworder.class.php ~11946) sets the source orders to status 16
|
||||||
|
AND **zeroes `date_placed`**, then copies all items to a NEW order. Because the import query
|
||||||
|
filters `o.date_placed >= …`, a combined source order can never be re-fetched, so its stale
|
||||||
|
'placed' rows would double-count with the new merged order. Currently latent (last combine
|
||||||
|
2024-07, predating current PG data — verified no stale rows exist today), but it will silently
|
||||||
|
corrupt the day combining is used again.
|
||||||
|
|
||||||
|
**Fix.** Two parts, both inside the orders import after the upsert phase:
|
||||||
|
1. **Item-set reconciliation** for re-imported orders: the import already knows the set of changed
|
||||||
|
`orderIds` and inserted their current items into `temp_order_items`. Mirror the PO import's
|
||||||
|
pattern (`purchase-orders.js` lines ~683-694):
|
||||||
|
```sql
|
||||||
|
DELETE FROM orders o
|
||||||
|
WHERE o.order_number = ANY($1) -- orders fetched this run
|
||||||
|
AND NOT EXISTS (SELECT 1 FROM temp_order_items t
|
||||||
|
WHERE t.order_id = o.order_number AND t.pid = o.pid);
|
||||||
|
```
|
||||||
|
2. **Combined/cancelled sweep** that does NOT depend on `date_placed`: each run, fetch from MySQL
|
||||||
|
`SELECT order_id, order_status FROM _order WHERE order_status IN (15,16) AND stamp > ?`
|
||||||
|
(no date_placed filter) and update matching PG rows' `status`/`canceled`
|
||||||
|
('combined' rows are then excluded from metrics — see Fix 5). Cheap (small result set).
|
||||||
|
|
||||||
|
**Verification.** Re-run the May-2026 row-count comparison (MySQL vs PG for one month) after one full
|
||||||
|
run; counts should converge (PG ≤ MySQL, diff explained by TZ window edges only).
|
||||||
|
|
||||||
|
### Fix 5: 'combined' orders are counted as sales
|
||||||
|
|
||||||
|
**Files:** `scripts/metrics-new/update_daily_snapshots.sql` (status filters, lines ~77, 120-134),
|
||||||
|
`update_product_metrics.sql` (`HistoricalDates` line ~145, `LifetimeRevenue` line ~249),
|
||||||
|
`backfill/rebuild_daily_snapshots.sql`.
|
||||||
|
|
||||||
|
**Problem.** Sales filters exclude only `('canceled', 'returned')`. Status 16 'combined' = "merged
|
||||||
|
into another order" — the new order carries the same items, so counting both double-counts. 826
|
||||||
|
combined orders exist in MySQL; today none are in PG (see Fix 4), but once Fix 4's sweep starts
|
||||||
|
marking rows 'combined', the metrics filters must exclude them.
|
||||||
|
|
||||||
|
**Fix.** Change every `NOT IN ('canceled', 'returned')` in the metrics SQL to
|
||||||
|
`NOT IN ('canceled', 'returned', 'combined')`. Grep for the pattern in `scripts/metrics-new/` and
|
||||||
|
`src/routes/` (dashboard endpoints replicate these filters — see CLAUDE.md analytics-filters note).
|
||||||
|
|
||||||
|
### Fix 6: Incremental sync watermark race (silent permanent misses)
|
||||||
|
|
||||||
|
**Files:** `scripts/import/orders.js` (~772), `products.js` (~934), `purchase-orders.js` (~833).
|
||||||
|
|
||||||
|
**Problem.** `sync_status.last_sync_timestamp` is set to `NOW()` *after* the import finishes. Any
|
||||||
|
MySQL row modified between the source query and that write is below the new watermark but was never
|
||||||
|
fetched → permanently skipped (until a full sync or the row changes again). Long imports widen the
|
||||||
|
window; PG/MySQL clock skew adds to it.
|
||||||
|
|
||||||
|
**Fix.** Capture the watermark **before** the source query and write that value:
|
||||||
|
```js
|
||||||
|
const [[{ now: sourceNow }]] = await prodConnection.query('SELECT NOW() as now');
|
||||||
|
// ... do the import ...
|
||||||
|
await localConnection.query(
|
||||||
|
`INSERT INTO sync_status ... VALUES ('orders', $1) ON CONFLICT ... SET last_sync_timestamp = $1`,
|
||||||
|
[sourceNow]);
|
||||||
|
```
|
||||||
|
Using MySQL's own clock also eliminates cross-server skew. Note `sourceNow` comes back through the
|
||||||
|
mysql2 driver TZ conversion — verify round-tripping with `adjustDateForMySQL` produces a correct
|
||||||
|
comparison value, or store `UTC_TIMESTAMP()` and compare against `CONVERT_TZ`-normalized stamps.
|
||||||
|
Overlap (re-importing rows changed during the run) is harmless — everything is upserted.
|
||||||
|
|
||||||
|
### Fix 7: Stockout days / service level / fill rate / avg stock built on activity-only snapshots
|
||||||
|
|
||||||
|
**Files:** `scripts/metrics-new/update_product_metrics.sql` — `SnapshotAggregates`
|
||||||
|
(`stockout_days_30d`, `avg_stock_*_30d`, lines ~177-189), `ServiceLevels` (lines ~304-323),
|
||||||
|
plus `calculate_sales_velocity` usage.
|
||||||
|
|
||||||
|
**Problem.** `daily_product_snapshots` only has rows on days with sales/receivings. So:
|
||||||
|
- A product that is out of stock (and therefore sells nothing) gets **no row** → `stockout_days_30d`
|
||||||
|
≈ 0 exactly when stockouts matter → `calculate_sales_velocity(sales, stockout_days)`'s adjustment
|
||||||
|
is inert → velocity and replenishment understated for constrained products.
|
||||||
|
- `service_level_30d` divides stockout days by COUNT(activity days), not 30.
|
||||||
|
- `avg_stock_units_30d` / `avg_stock_cost_30d` average only activity days (biased toward in-stock
|
||||||
|
days) → GMROI / stockturn / sell-through denominators biased.
|
||||||
|
- `fill_rate_30d`'s `units_sold * 0.2` lost-sales heuristic is arbitrary — fine to keep, but document.
|
||||||
|
|
||||||
|
**Fix.** Derive stock-presence metrics from `stock_snapshots` (full daily coverage from MySQL
|
||||||
|
`snap_product_value`, imported by `stock-snapshots.js`) instead of `daily_product_snapshots`:
|
||||||
|
```sql
|
||||||
|
StockCoverage AS (
|
||||||
|
SELECT pid,
|
||||||
|
COUNT(*) FILTER (WHERE stock_quantity <= 0) AS stockout_days_30d,
|
||||||
|
AVG(stock_quantity) AS avg_stock_units_30d,
|
||||||
|
AVG(stock_value) AS avg_stock_cost_30d
|
||||||
|
FROM stock_snapshots
|
||||||
|
WHERE snapshot_date >= _current_date - INTERVAL '29 days'
|
||||||
|
GROUP BY pid
|
||||||
|
)
|
||||||
|
```
|
||||||
|
Treat products absent from `stock_snapshots` for a day as unknown (NULL), not in-stock. Keep
|
||||||
|
`daily_product_snapshots` for sales/revenue aggregates. `service_level_30d` denominator becomes the
|
||||||
|
count of covered days. Note `stock_snapshots` has no `eod_stock_retail`; keep retail/gross averages
|
||||||
|
on the old source or compute as `stock_quantity * current price` explicitly.
|
||||||
|
|
||||||
|
**Verification.** Pick products that had a known stockout period; `stockout_days_30d` should now be
|
||||||
|
> 0 and `sales_velocity_daily` should rise accordingly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P2 — Definition / robustness improvements
|
||||||
|
|
||||||
|
### Fix 8: Returns don't reduce COGS; LifetimeRevenue ignores returns
|
||||||
|
`update_daily_snapshots.sql` SalesData: COGS accrues only on `quantity > 0` rows; return rows
|
||||||
|
(negative qty — 15,875 rows live) subtract revenue but never COGS → margin understated in
|
||||||
|
return-heavy periods. Add a returns-COGS term mirroring the sales-COGS COALESCE chain
|
||||||
|
(`SUM(... WHEN quantity < 0 THEN cost * ABS(quantity))`) and subtract it in `cogs` (or store
|
||||||
|
`returns_cogs` separately and use `cogs - returns_cogs` in profit). Also `LifetimeRevenue` in
|
||||||
|
`update_product_metrics.sql` (line ~242) filters `quantity > 0` — include negative-qty rows so
|
||||||
|
lifetime revenue nets out returns (drop the quantity filter; `price*quantity` is already signed,
|
||||||
|
but check the `- discount` term sign for return rows).
|
||||||
|
|
||||||
|
### Fix 9: return_rate_30d definition
|
||||||
|
`update_product_metrics.sql` line ~468: `returns / (sales + returns)` → industry standard is
|
||||||
|
`returns / sales`. Change denominator to `NULLIF(sa.sales_30d, 0)`.
|
||||||
|
|
||||||
|
### Fix 10: GMROI not annualized
|
||||||
|
Line ~466: `profit_30d / avg_stock_cost_30d` is a monthly GMROI (~1/12 of the conventional annual
|
||||||
|
figure, benchmark ≥ 2-3). Either annualize (`* 12.17`) or rename the column/label "monthly".
|
||||||
|
Decision for Matt; annualizing is recommended for comparability. Frontend displays must be checked
|
||||||
|
either way.
|
||||||
|
|
||||||
|
### Fix 11: get_weighted_avg_cost is a lifetime WAC
|
||||||
|
`db/functions.sql` (~line 81, deployed identically): averages ALL receivings ≤ date — decade-old
|
||||||
|
costs weigh equally. Recommended: window to recent receivings, e.g. last 365 days falling back to
|
||||||
|
lifetime when none. Used as fallback COGS when `o.costeach` is NULL, so impact is modest but real
|
||||||
|
for long-lived SKUs. Apply with `CREATE OR REPLACE FUNCTION` in `db/functions.sql` AND on the live DB.
|
||||||
|
|
||||||
|
### Fix 12: exclude_from_forecast removes products from product_metrics entirely
|
||||||
|
`update_product_metrics.sql` line ~627 (`WHERE s.exclude_forecast IS FALSE OR ... IS NULL`): the
|
||||||
|
flag's name implies forecast-only, but excluded products get NO metrics row → vanish from brand/
|
||||||
|
vendor/category rollups and dashboards. Fix: always emit the row; instead NULL the
|
||||||
|
forecast/replenishment columns when excluded (wrap those expressions in
|
||||||
|
`CASE WHEN s.exclude_forecast THEN NULL ELSE ... END`).
|
||||||
|
|
||||||
|
### Fix 13: Incremental products import misses category-only changes
|
||||||
|
`products.js` incremental WHERE (~lines 433-440) keys on `p.stamp`, `ci.stamp`, price/b2b dates —
|
||||||
|
`product_category_index` changes don't bump any of those → PG `product_categories` goes stale. Also
|
||||||
|
the `needs_update` comparison (~lines 604-625) doesn't compare `categories`, so even refetched rows
|
||||||
|
skip the category rewrite. Fix both: add `t.categories IS NOT DISTINCT FROM p.categories` to the
|
||||||
|
needs_update comparison (note: `products.categories` is the GROUP_CONCAT string — confirm PG column
|
||||||
|
holds the same representation), and add a cheap full-sweep (e.g. weekly, or compare
|
||||||
|
`COUNT(*) GROUP BY pid` hashes) OR include `EXISTS (SELECT 1 FROM product_category_index pci WHERE
|
||||||
|
pci.pid = p.pid AND pci.stamp > ?)` in the incremental WHERE if that table has a stamp column —
|
||||||
|
verify schema first (`DESCRIBE product_category_index`).
|
||||||
|
|
||||||
|
### Fix 14: PO/receivings OFFSET pagination over a moving filter
|
||||||
|
`purchase-orders.js` (~lines 275-298, 447-470): `LIMIT/OFFSET` with a `date_updated > ?` predicate;
|
||||||
|
concurrent updates shift rows between pages → silent skips. Fix: keyset pagination —
|
||||||
|
`WHERE ... AND p.po_id > ? ORDER BY p.po_id LIMIT 500`, carrying the last seen po_id (drop OFFSET).
|
||||||
|
Same for receivings on `receiving_id`.
|
||||||
|
|
||||||
|
### Fix 15: Status map gaps and unsafe defaults
|
||||||
|
- `orders.js` orderStatusMap lacks 45 (`payment_pending`) and 67 (`remote_send`) → imported as
|
||||||
|
numeric strings. Add both (mirror in `migrations/001_map_order_statuses.sql` as a follow-up update
|
||||||
|
for existing rows).
|
||||||
|
- `purchase-orders.js` `poStatusMap[po.status] || 'created'` (line ~335): an unknown *cancel-like*
|
||||||
|
code would be treated as an open PO and inflate on-order FIFO. Default to a sentinel like
|
||||||
|
`'unknown_<code>'` instead, and make the FIFO/on-order CTEs in `update_product_metrics.sql` treat
|
||||||
|
only the known-open statuses as open (they already whitelist open statuses — so the sentinel is
|
||||||
|
safe there; just ensure nothing treats unknown as 'created'). Same for receivingStatusMap.
|
||||||
|
|
||||||
|
### Fix 16: Transactions issued through the pool wrapper land on arbitrary connections
|
||||||
|
`categories.js` (lines ~17-152) and `daily-deals.js` (~27-130) call `query('BEGIN')` /
|
||||||
|
`query('COMMIT')` on the wrapper, which checks out a client per call — BEGIN/work/COMMIT are not
|
||||||
|
guaranteed to share a connection (works only by pool-LIFO accident). The categories
|
||||||
|
`DISABLE TRIGGER` rides on this too. Fix: use the wrapper's `beginTransaction()/commit()/rollback()`
|
||||||
|
(see `utils.js` lines 121-148) exactly as orders.js does. In categories.js also move the
|
||||||
|
post-COMMIT `ENABLE TRIGGER` inside the transaction (DISABLE/ENABLE both inside), or drop the
|
||||||
|
trigger toggling entirely if the trigger isn't actually problematic anymore.
|
||||||
|
|
||||||
|
### Fix 17: stock-snapshots import swallows batch errors → permanent holes
|
||||||
|
`stock-snapshots.js` (~lines 153-155): a failed batch is logged and skipped, but the next
|
||||||
|
incremental starts at `MAX(snapshot_date)` — the hole is never revisited. Fix: rethrow (fail the
|
||||||
|
step) or collect failed date ranges and retry once, then fail if still failing. Also line ~168:
|
||||||
|
`calculateRate(processedRows, startTime)` — arguments reversed (signature is
|
||||||
|
`calculateRate(startTime, current)`, see `metrics-new/utils/progress.js:70`).
|
||||||
|
|
||||||
|
### Fix 18: Metrics cancellation targets an application_name that's never set
|
||||||
|
`calculate-metrics-new.js` line ~180 cancels backends `WHERE application_name =
|
||||||
|
'node-metrics-calculator'`, but the Pool config never sets it → cancellation no-ops (the 30-min
|
||||||
|
`statement_timeout` is the only real guard). Fix: add `application_name: 'node-metrics-calculator'`
|
||||||
|
to both dbConfig branches.
|
||||||
|
|
||||||
|
### Fix 19: Aggregate-table change-detection lists miss cost-only changes
|
||||||
|
`calculate_brand_metrics.sql` / `calculate_vendor_metrics.sql` / `calculate_category_metrics.sql`
|
||||||
|
ON CONFLICT WHERE lists don't include `profit_30d`/`cogs_30d` — a cost revision with unchanged
|
||||||
|
sales/revenue leaves stale rows (product_metrics has a 1-day staleness net; rollups don't). Add
|
||||||
|
`... OR x.profit_30d IS DISTINCT FROM EXCLUDED.profit_30d OR x.cogs_30d IS DISTINCT FROM
|
||||||
|
EXCLUDED.cogs_30d` to each, or add a `last_calculated < NOW() - INTERVAL '1 day'` net like
|
||||||
|
product_metrics line ~707.
|
||||||
|
|
||||||
|
### Fix 20: Snapshot stale-detection only compares unit counts
|
||||||
|
`update_daily_snapshots.sql` lines ~57-85: detects mismatches in `units_sold`/`units_received` only;
|
||||||
|
price/discount/costeach corrections older than the 2-day recheck are never repaired. Add a
|
||||||
|
revenue comparison to the stale check: compare `SUM(net_revenue)` per day against the equivalent
|
||||||
|
recomputed from `orders` (ROUND both to 2dp to avoid float-noise churn).
|
||||||
|
|
||||||
|
### Fix 21: Category metrics positive-only revenue asymmetry
|
||||||
|
`calculate_category_metrics.sql` (lines ~27-36, 64-73): revenue summed only when `> 0` while
|
||||||
|
cogs/profit use COALESCE-all → margin numerator/denominator from different populations, and
|
||||||
|
inconsistent with brand/vendor (plain COALESCE). Change the revenue/sales CASEs to
|
||||||
|
`COALESCE(pm.revenue_7d, 0)` etc., matching brand_metrics.
|
||||||
|
|
||||||
|
### Fix 22 (decision needed): Demand-pattern & seasonality definitions
|
||||||
|
- `classify_demand_pattern` (db/functions.sql): CV thresholds 0.2/0.5 + avg<1/day. Industry standard
|
||||||
|
is Syntetos-Boylan: ADI ≥ 1.32 and CV² ≥ 0.49 quadrants (smooth/erratic/intermittent/lumpy).
|
||||||
|
Today everything classifies sporadic/lumpy. If adopting SB: ADI = 30 / COUNT(days with sales),
|
||||||
|
CV² computed on nonzero-demand sizes. Changes the vocabulary consumed by the forecast engine
|
||||||
|
(`scripts/forecast/forecast_engine.py` reads `demand_pattern`) — coordinate before changing.
|
||||||
|
- SeasonalityAnalysis (`update_product_metrics.sql` ~360): `month_avg = AVG(units_sold)` over rows
|
||||||
|
with sales only → intensity, not volume. Use monthly totals (SUM, with zero months counted) /
|
||||||
|
overall monthly average for the index.
|
||||||
|
- Safety stock: currently static config units; `sales_std_dev_30d` exists but is unused. Optional
|
||||||
|
upgrade: `safety = z * σ_d * sqrt(lead_time)` with z from a service-level setting.
|
||||||
|
|
||||||
|
These change user-facing semantics — confirm with Matt before implementing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verified non-issues (no action, or cleanup only)
|
||||||
|
|
||||||
|
- **`costeach` fallback `price * 0.5`** (orders.js line ~615): fires on **2.1%** of item rows
|
||||||
|
(729/34,833, last 30d, live-verified). Accepted by Matt — 50% margin is a fair estimate for these
|
||||||
|
products. Optional: nothing.
|
||||||
|
- **Missing-product order skips**: zero occurrences — MySQL has no orphan order_items (1-year check),
|
||||||
|
PG products is a superset of MySQL products (687,579 vs 687,576), last 7 import runs all logged
|
||||||
|
`totalSkipped: 0`. Cleanup only: remove the unused `importMissingProducts` import line at
|
||||||
|
`orders.js:2` (the function itself stays in products.js — harmless utility).
|
||||||
|
- **Status 30 'cancelled_old'** in `total_sold >= 20` filter: zero rows live in `_order` — safe.
|
||||||
|
- **Duplicate (order_id, pid) order items**: none exist in MySQL — the upsert PK is safe.
|
||||||
|
- **base_discount** in orders.js: computed/stored in temp table but unused since migration 002 —
|
||||||
|
remove the column from temp table + queries for clarity (no behavior change).
|
||||||
|
- **`full-update.js` `runScript`**: try/catch around `console.log` is dead code; per-step
|
||||||
|
`status:'complete'` messages could confuse a UI parser. Cosmetic only — tidy if touching the file.
|
||||||
|
|
||||||
|
## Suggested implementation order
|
||||||
|
|
||||||
|
| Step | Fixes | Re-import/rebuild needed |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | Fix 1 + Fix 2 (+ Fix 5 filters, Fix 8/9 while editing the same SQL) | FULL orders re-import → snapshot rebuild → metrics (once) |
|
||||||
|
| 2 | Fix 4 + Fix 6 (orders.js reconciliation + watermarks; POs/products watermarks too) | no |
|
||||||
|
| 3 | Fix 3, Fix 7 (metrics SQL only) | metrics run |
|
||||||
|
| 4 | Fix 13-21 (robustness batch) | no |
|
||||||
|
| 5 | Fix 10-12, Fix 22 after Matt's sign-off (definition changes) | metrics run |
|
||||||
|
|
||||||
|
After step 1, expect: margin_30d down ~8-10 points, discounts_30d ~3x up, daily curves shifted to
|
||||||
|
correct business days. Communicate before/after so the change isn't mistaken for a data incident.
|
||||||
|
|
||||||
|
## Reference: verification snippets used in the review
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- MySQL: item-level discounts dropped by the gate (30d)
|
||||||
|
SELECT COUNT(DISTINCT o.order_id), ROUND(SUM(odi.amount),2)
|
||||||
|
FROM order_discount_items odi
|
||||||
|
JOIN order_discounts od ON od.order_id=odi.order_id AND od.discount_id=odi.discount_id
|
||||||
|
JOIN _order o ON o.order_id=odi.order_id
|
||||||
|
WHERE odi.which=2 AND o.date_placed >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
|
||||||
|
AND o.order_status >= 20 AND COALESCE(od.discount_amount_subtotal,0)=0;
|
||||||
|
-- → 2,021 orders / $25,989 (2026-06-10)
|
||||||
|
|
||||||
|
-- MySQL: costeach fallback frequency (30d)
|
||||||
|
SELECT COUNT(*),
|
||||||
|
SUM(CASE WHEN NOT EXISTS (SELECT 1 FROM order_costs oc WHERE oc.orderid=oi.order_id
|
||||||
|
AND oc.pid=oi.prod_pid AND oc.pending=0)
|
||||||
|
AND NOT EXISTS (SELECT 1 FROM product_inventory pi WHERE pi.pid=oi.prod_pid)
|
||||||
|
THEN 1 ELSE 0 END)
|
||||||
|
FROM order_items oi JOIN _order o ON o.order_id=oi.order_id
|
||||||
|
WHERE o.order_status >= 20 AND o.date_placed >= DATE_SUB(CURDATE(), INTERVAL 30 DAY);
|
||||||
|
-- → 729 / 34,833 = 2.1% (2026-06-10)
|
||||||
|
|
||||||
|
-- PG: timezone check
|
||||||
|
SHOW timezone; -- Europe/Berlin (2026-06-10)
|
||||||
|
|
||||||
|
-- Row drift, May 2026: MySQL 49,377 items / PG 49,841 (+0.9%)
|
||||||
|
```
|
||||||
@@ -1,100 +1,72 @@
|
|||||||
require('dotenv').config({ path: '../.env' });
|
import bcrypt from 'bcrypt';
|
||||||
const bcrypt = require('bcrypt');
|
import pg from 'pg';
|
||||||
const { Pool } = require('pg');
|
import inquirer from 'inquirer';
|
||||||
const inquirer = require('inquirer');
|
|
||||||
|
|
||||||
// Log connection details for debugging (remove in production)
|
const { Pool } = pg;
|
||||||
console.log('Attempting to connect with:', {
|
import { config as loadEnv } from 'dotenv';
|
||||||
host: process.env.DB_HOST,
|
import { fileURLToPath } from 'node:url';
|
||||||
user: process.env.DB_USER,
|
import { dirname, resolve as resolvePath } from 'node:path';
|
||||||
database: process.env.DB_NAME,
|
|
||||||
port: process.env.DB_PORT
|
const __filename = fileURLToPath(import.meta.url);
|
||||||
});
|
const __dirname = dirname(__filename);
|
||||||
|
|
||||||
|
loadEnv({ path: resolvePath(__dirname, '../.env') });
|
||||||
|
|
||||||
const pool = new Pool({
|
const pool = new Pool({
|
||||||
host: process.env.DB_HOST,
|
host: process.env.DB_HOST,
|
||||||
user: process.env.DB_USER,
|
user: process.env.DB_USER,
|
||||||
password: process.env.DB_PASSWORD,
|
password: process.env.DB_PASSWORD,
|
||||||
database: process.env.DB_NAME,
|
database: process.env.DB_NAME,
|
||||||
port: process.env.DB_PORT,
|
port: Number(process.env.DB_PORT) || 5432,
|
||||||
});
|
});
|
||||||
|
|
||||||
async function promptUser() {
|
async function promptUser() {
|
||||||
const questions = [
|
return inquirer.prompt([
|
||||||
{
|
{
|
||||||
type: 'input',
|
type: 'input',
|
||||||
name: 'username',
|
name: 'username',
|
||||||
message: 'Enter username:',
|
message: 'Enter username:',
|
||||||
validate: (input) => {
|
validate: (input) => input.length >= 3 || 'Username must be at least 3 characters long',
|
||||||
if (input.length < 3) {
|
|
||||||
return 'Username must be at least 3 characters long';
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
type: 'password',
|
type: 'password',
|
||||||
name: 'password',
|
name: 'password',
|
||||||
message: 'Enter password:',
|
message: 'Enter password:',
|
||||||
mask: '*',
|
mask: '*',
|
||||||
validate: (input) => {
|
validate: (input) => input.length >= 8 || 'Password must be at least 8 characters long',
|
||||||
if (input.length < 8) {
|
|
||||||
return 'Password must be at least 8 characters long';
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
type: 'password',
|
type: 'password',
|
||||||
name: 'confirmPassword',
|
name: 'confirmPassword',
|
||||||
message: 'Confirm password:',
|
message: 'Confirm password:',
|
||||||
mask: '*',
|
mask: '*',
|
||||||
validate: (input, answers) => {
|
validate: (input, answers) => input === answers.password || 'Passwords do not match',
|
||||||
if (input !== answers.password) {
|
},
|
||||||
return 'Passwords do not match';
|
]);
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
];
|
|
||||||
|
|
||||||
return inquirer.prompt(questions);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
async function addUser() {
|
async function addUser() {
|
||||||
try {
|
try {
|
||||||
// Get user input
|
const { username, password } = await promptUser();
|
||||||
const answers = await promptUser();
|
const hashedPassword = await bcrypt.hash(password, 10);
|
||||||
const { username, password } = answers;
|
|
||||||
|
|
||||||
// Hash password
|
|
||||||
const saltRounds = 10;
|
|
||||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
|
||||||
|
|
||||||
// Check if user already exists
|
|
||||||
const checkResult = await pool.query(
|
const checkResult = await pool.query(
|
||||||
'SELECT id FROM users WHERE username = $1',
|
'SELECT id FROM users WHERE username = $1',
|
||||||
[username]
|
[username]
|
||||||
);
|
);
|
||||||
|
|
||||||
if (checkResult.rows.length > 0) {
|
if (checkResult.rows.length > 0) {
|
||||||
console.error('Error: Username already exists');
|
console.error('Error: Username already exists');
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Insert new user
|
|
||||||
const result = await pool.query(
|
const result = await pool.query(
|
||||||
'INSERT INTO users (username, password) VALUES ($1, $2) RETURNING id',
|
'INSERT INTO users (username, password) VALUES ($1, $2) RETURNING id',
|
||||||
[username, hashedPassword]
|
[username, hashedPassword]
|
||||||
);
|
);
|
||||||
|
|
||||||
console.log(`User ${username} created successfully with id ${result.rows[0].id}`);
|
console.log(`User ${username} created successfully with id ${result.rows[0].id}`);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error creating user:', error);
|
console.error('Error creating user:', error);
|
||||||
console.error('Error details:', error.message);
|
if (error.code) console.error('Error code:', error.code);
|
||||||
if (error.code) {
|
|
||||||
console.error('Error code:', error.code);
|
|
||||||
}
|
|
||||||
} finally {
|
} finally {
|
||||||
await pool.end();
|
await pool.end();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,18 +2,22 @@
|
|||||||
"name": "inventory-auth-server",
|
"name": "inventory-auth-server",
|
||||||
"version": "1.0.0",
|
"version": "1.0.0",
|
||||||
"description": "Authentication server for inventory management system",
|
"description": "Authentication server for inventory management system",
|
||||||
|
"type": "module",
|
||||||
"main": "server.js",
|
"main": "server.js",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"start": "node server.js"
|
"start": "node server.js",
|
||||||
|
"add-user": "node add-user.js"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"bcrypt": "^5.1.1",
|
"bcrypt": "^5.1.1",
|
||||||
"cors": "^2.8.5",
|
"cors": "^2.8.5",
|
||||||
"dotenv": "^16.4.7",
|
"dotenv": "^16.4.7",
|
||||||
"express": "^4.18.2",
|
"express": "^4.18.2",
|
||||||
|
"express-rate-limit": "^7.4.0",
|
||||||
"inquirer": "^8.2.6",
|
"inquirer": "^8.2.6",
|
||||||
"jsonwebtoken": "^9.0.2",
|
"jsonwebtoken": "^9.0.2",
|
||||||
"morgan": "^1.10.0",
|
"pg": "^8.11.3",
|
||||||
"pg": "^8.11.3"
|
"pino": "^9.5.0",
|
||||||
|
"pino-http": "^10.3.0"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,40 +1,12 @@
|
|||||||
// Get pool from global or create a new one if not available
|
export function createPermissionHelpers({ pool }) {
|
||||||
let pool;
|
async function checkPermission(userId, permissionCode) {
|
||||||
if (typeof global.pool !== 'undefined') {
|
|
||||||
pool = global.pool;
|
|
||||||
} else {
|
|
||||||
// If global pool is not available, create a new connection
|
|
||||||
const { Pool } = require('pg');
|
|
||||||
pool = new Pool({
|
|
||||||
host: process.env.DB_HOST,
|
|
||||||
user: process.env.DB_USER,
|
|
||||||
password: process.env.DB_PASSWORD,
|
|
||||||
database: process.env.DB_NAME,
|
|
||||||
port: process.env.DB_PORT,
|
|
||||||
});
|
|
||||||
console.log('Created new database pool in permissions.js');
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if a user has a specific permission
|
|
||||||
* @param {number} userId - The user ID to check
|
|
||||||
* @param {string} permissionCode - The permission code to check
|
|
||||||
* @returns {Promise<boolean>} - Whether the user has the permission
|
|
||||||
*/
|
|
||||||
async function checkPermission(userId, permissionCode) {
|
|
||||||
try {
|
try {
|
||||||
// First check if the user is an admin
|
|
||||||
const adminResult = await pool.query(
|
const adminResult = await pool.query(
|
||||||
'SELECT is_admin FROM users WHERE id = $1',
|
'SELECT is_admin FROM users WHERE id = $1',
|
||||||
[userId]
|
[userId]
|
||||||
);
|
);
|
||||||
|
if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) return true;
|
||||||
|
|
||||||
// If user is admin, automatically grant permission
|
|
||||||
if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Otherwise check for specific permission
|
|
||||||
const result = await pool.query(
|
const result = await pool.query(
|
||||||
`SELECT COUNT(*) AS has_permission
|
`SELECT COUNT(*) AS has_permission
|
||||||
FROM user_permissions up
|
FROM user_permissions up
|
||||||
@@ -42,69 +14,47 @@ async function checkPermission(userId, permissionCode) {
|
|||||||
WHERE up.user_id = $1 AND p.code = $2`,
|
WHERE up.user_id = $1 AND p.code = $2`,
|
||||||
[userId, permissionCode]
|
[userId, permissionCode]
|
||||||
);
|
);
|
||||||
|
return Number(result.rows[0].has_permission) > 0;
|
||||||
return result.rows[0].has_permission > 0;
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error checking permission:', error);
|
console.error('Error checking permission:', error);
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Middleware to require a specific permission
|
|
||||||
* @param {string} permissionCode - The permission code required
|
|
||||||
* @returns {Function} - Express middleware function
|
|
||||||
*/
|
|
||||||
function requirePermission(permissionCode) {
|
|
||||||
return async (req, res, next) => {
|
|
||||||
try {
|
|
||||||
// Check if user is authenticated
|
|
||||||
if (!req.user || !req.user.id) {
|
|
||||||
return res.status(401).json({ error: 'Authentication required' });
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function requirePermission(permissionCode) {
|
||||||
|
return async (req, res, next) => {
|
||||||
|
try {
|
||||||
|
if (!req.user?.id) {
|
||||||
|
return res.status(401).json({ error: 'Authentication required' });
|
||||||
|
}
|
||||||
const hasPermission = await checkPermission(req.user.id, permissionCode);
|
const hasPermission = await checkPermission(req.user.id, permissionCode);
|
||||||
|
|
||||||
if (!hasPermission) {
|
if (!hasPermission) {
|
||||||
return res.status(403).json({
|
return res.status(403).json({
|
||||||
error: 'Insufficient permissions',
|
error: 'Insufficient permissions',
|
||||||
requiredPermission: permissionCode
|
requiredPermission: permissionCode,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
next();
|
next();
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Permission middleware error:', error);
|
console.error('Permission middleware error:', error);
|
||||||
res.status(500).json({ error: 'Server error checking permissions' });
|
res.status(500).json({ error: 'Server error checking permissions' });
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
async function getUserPermissions(userId) {
|
||||||
* Get all permissions for a user
|
|
||||||
* @param {number} userId - The user ID
|
|
||||||
* @returns {Promise<string[]>} - Array of permission codes
|
|
||||||
*/
|
|
||||||
async function getUserPermissions(userId) {
|
|
||||||
try {
|
try {
|
||||||
// Check if user is admin
|
|
||||||
const adminResult = await pool.query(
|
const adminResult = await pool.query(
|
||||||
'SELECT is_admin FROM users WHERE id = $1',
|
'SELECT is_admin FROM users WHERE id = $1',
|
||||||
[userId]
|
[userId]
|
||||||
);
|
);
|
||||||
|
if (adminResult.rows.length === 0) return [];
|
||||||
|
|
||||||
if (adminResult.rows.length === 0) {
|
if (adminResult.rows[0].is_admin) {
|
||||||
return [];
|
const allPermissions = await pool.query('SELECT code FROM permissions');
|
||||||
|
return allPermissions.rows.map((p) => p.code);
|
||||||
}
|
}
|
||||||
|
|
||||||
const isAdmin = adminResult.rows[0].is_admin;
|
|
||||||
|
|
||||||
if (isAdmin) {
|
|
||||||
// Admin gets all permissions
|
|
||||||
const allPermissions = await pool.query('SELECT code FROM permissions');
|
|
||||||
return allPermissions.rows.map(p => p.code);
|
|
||||||
} else {
|
|
||||||
// Get assigned permissions
|
|
||||||
const permissions = await pool.query(
|
const permissions = await pool.query(
|
||||||
`SELECT p.code
|
`SELECT p.code
|
||||||
FROM permissions p
|
FROM permissions p
|
||||||
@@ -112,17 +62,12 @@ async function getUserPermissions(userId) {
|
|||||||
WHERE up.user_id = $1`,
|
WHERE up.user_id = $1`,
|
||||||
[userId]
|
[userId]
|
||||||
);
|
);
|
||||||
|
return permissions.rows.map((p) => p.code);
|
||||||
return permissions.rows.map(p => p.code);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting user permissions:', error);
|
console.error('Error getting user permissions:', error);
|
||||||
return [];
|
return [];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = {
|
return { checkPermission, requirePermission, getUserPermissions };
|
||||||
checkPermission,
|
}
|
||||||
requirePermission,
|
|
||||||
getUserPermissions
|
|
||||||
};
|
|
||||||
|
|||||||
+82
-298
@@ -1,102 +1,66 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const router = express.Router();
|
import bcrypt from 'bcrypt';
|
||||||
const bcrypt = require('bcrypt');
|
import jwt from 'jsonwebtoken';
|
||||||
const jwt = require('jsonwebtoken');
|
import { createPermissionHelpers } from './permissions.js';
|
||||||
const { requirePermission, getUserPermissions } = require('./permissions');
|
|
||||||
|
|
||||||
// Get pool from global or create a new one if not available
|
export function createAuthRoutes({ pool }) {
|
||||||
let pool;
|
const router = express.Router();
|
||||||
if (typeof global.pool !== 'undefined') {
|
const { requirePermission, getUserPermissions } = createPermissionHelpers({ pool });
|
||||||
pool = global.pool;
|
|
||||||
} else {
|
|
||||||
// If global pool is not available, create a new connection
|
|
||||||
const { Pool } = require('pg');
|
|
||||||
pool = new Pool({
|
|
||||||
host: process.env.DB_HOST,
|
|
||||||
user: process.env.DB_USER,
|
|
||||||
password: process.env.DB_PASSWORD,
|
|
||||||
database: process.env.DB_NAME,
|
|
||||||
port: process.env.DB_PORT,
|
|
||||||
});
|
|
||||||
console.log('Created new database pool in routes.js');
|
|
||||||
}
|
|
||||||
|
|
||||||
// Authentication middleware
|
// Local authenticate(): used by user-management endpoints that need req.user populated
|
||||||
const authenticate = async (req, res, next) => {
|
// with id/username/email/is_admin. NOT the per-service authenticate() — that lives in
|
||||||
|
// shared/auth/middleware.js and is used by downstream services. Auth-server's surface is
|
||||||
|
// small enough that a local copy is fine; the security boundary is the JWT verify step.
|
||||||
|
async function authenticate(req, res, next) {
|
||||||
try {
|
try {
|
||||||
const authHeader = req.headers.authorization;
|
const authHeader = req.headers.authorization;
|
||||||
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
||||||
return res.status(401).json({ error: 'Authentication required' });
|
return res.status(401).json({ error: 'Authentication required' });
|
||||||
}
|
}
|
||||||
|
|
||||||
const token = authHeader.split(' ')[1];
|
const token = authHeader.split(' ')[1];
|
||||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||||
|
|
||||||
// Get user from database
|
|
||||||
const result = await pool.query(
|
const result = await pool.query(
|
||||||
'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1',
|
'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1',
|
||||||
[decoded.userId]
|
[decoded.userId]
|
||||||
);
|
);
|
||||||
|
|
||||||
console.log('Database query result for user', decoded.userId, ':', result.rows[0]);
|
|
||||||
|
|
||||||
if (result.rows.length === 0) {
|
if (result.rows.length === 0) {
|
||||||
return res.status(401).json({ error: 'User not found' });
|
return res.status(401).json({ error: 'User not found' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Attach user to request
|
|
||||||
req.user = result.rows[0];
|
req.user = result.rows[0];
|
||||||
next();
|
next();
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Authentication error:', error);
|
|
||||||
res.status(401).json({ error: 'Invalid token' });
|
res.status(401).json({ error: 'Invalid token' });
|
||||||
}
|
}
|
||||||
};
|
}
|
||||||
|
|
||||||
// Login route
|
router.post('/login', async (req, res) => {
|
||||||
router.post('/login', async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const { username, password } = req.body;
|
const { username, password } = req.body;
|
||||||
|
|
||||||
// Get user from database
|
|
||||||
const result = await pool.query(
|
const result = await pool.query(
|
||||||
'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1',
|
'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1',
|
||||||
[username]
|
[username]
|
||||||
);
|
);
|
||||||
|
|
||||||
if (result.rows.length === 0) {
|
if (result.rows.length === 0) {
|
||||||
return res.status(401).json({ error: 'Invalid username or password' });
|
return res.status(401).json({ error: 'Invalid username or password' });
|
||||||
}
|
}
|
||||||
|
|
||||||
const user = result.rows[0];
|
const user = result.rows[0];
|
||||||
|
|
||||||
// Check if user is active
|
|
||||||
if (!user.is_active) {
|
if (!user.is_active) {
|
||||||
return res.status(403).json({ error: 'Account is inactive' });
|
return res.status(403).json({ error: 'Account is inactive' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Verify password
|
|
||||||
const validPassword = await bcrypt.compare(password, user.password);
|
const validPassword = await bcrypt.compare(password, user.password);
|
||||||
if (!validPassword) {
|
if (!validPassword) {
|
||||||
return res.status(401).json({ error: 'Invalid username or password' });
|
return res.status(401).json({ error: 'Invalid username or password' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update last login
|
|
||||||
await pool.query(
|
await pool.query(
|
||||||
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
||||||
[user.id]
|
[user.id]
|
||||||
);
|
);
|
||||||
|
|
||||||
// Generate JWT
|
|
||||||
const token = jwt.sign(
|
const token = jwt.sign(
|
||||||
{ userId: user.id, username: user.username },
|
{ userId: user.id, username: user.username },
|
||||||
process.env.JWT_SECRET,
|
process.env.JWT_SECRET,
|
||||||
{ expiresIn: '8h' }
|
{ expiresIn: '8h' }
|
||||||
);
|
);
|
||||||
|
|
||||||
// Get user permissions
|
|
||||||
const permissions = await getUserPermissions(user.id);
|
const permissions = await getUserPermissions(user.id);
|
||||||
|
|
||||||
res.json({
|
res.json({
|
||||||
token,
|
token,
|
||||||
user: {
|
user: {
|
||||||
@@ -104,21 +68,18 @@ router.post('/login', async (req, res) => {
|
|||||||
username: user.username,
|
username: user.username,
|
||||||
is_admin: user.is_admin,
|
is_admin: user.is_admin,
|
||||||
rocket_chat_user_id: user.rocket_chat_user_id,
|
rocket_chat_user_id: user.rocket_chat_user_id,
|
||||||
permissions
|
permissions,
|
||||||
}
|
},
|
||||||
});
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Login error:', error);
|
console.error('Login error:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Get current user
|
router.get('/me', authenticate, async (req, res) => {
|
||||||
router.get('/me', authenticate, async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
// Get user permissions
|
|
||||||
const permissions = await getUserPermissions(req.user.id);
|
const permissions = await getUserPermissions(req.user.id);
|
||||||
|
|
||||||
res.json({
|
res.json({
|
||||||
id: req.user.id,
|
id: req.user.id,
|
||||||
username: req.user.username,
|
username: req.user.username,
|
||||||
@@ -126,49 +87,38 @@ router.get('/me', authenticate, async (req, res) => {
|
|||||||
is_admin: req.user.is_admin,
|
is_admin: req.user.is_admin,
|
||||||
rocket_chat_user_id: req.user.rocket_chat_user_id,
|
rocket_chat_user_id: req.user.rocket_chat_user_id,
|
||||||
permissions,
|
permissions,
|
||||||
// Debug info
|
|
||||||
_debug_raw_user: req.user,
|
|
||||||
_server_identifier: "INVENTORY_AUTH_SERVER_MODIFIED"
|
|
||||||
});
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting current user:', error);
|
console.error('Error getting current user:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Get all users
|
router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||||
router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const result = await pool.query(`
|
const result = await pool.query(`
|
||||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||||
FROM users
|
FROM users
|
||||||
ORDER BY username
|
ORDER BY username
|
||||||
`);
|
`);
|
||||||
|
|
||||||
res.json(result.rows);
|
res.json(result.rows);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting users:', error);
|
console.error('Error getting users:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Get user with permissions
|
router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||||
router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const userId = req.params.id;
|
const userId = req.params.id;
|
||||||
|
|
||||||
// Get user details
|
|
||||||
const userResult = await pool.query(`
|
const userResult = await pool.query(`
|
||||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||||
FROM users
|
FROM users
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
`, [userId]);
|
`, [userId]);
|
||||||
|
|
||||||
if (userResult.rows.length === 0) {
|
if (userResult.rows.length === 0) {
|
||||||
return res.status(404).json({ error: 'User not found' });
|
return res.status(404).json({ error: 'User not found' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get user permissions
|
|
||||||
const permissionsResult = await pool.query(`
|
const permissionsResult = await pool.query(`
|
||||||
SELECT p.id, p.name, p.code, p.category, p.description
|
SELECT p.id, p.name, p.code, p.category, p.description
|
||||||
FROM permissions p
|
FROM permissions p
|
||||||
@@ -176,126 +126,54 @@ router.get('/users/:id', authenticate, requirePermission('view:users'), async (r
|
|||||||
WHERE up.user_id = $1
|
WHERE up.user_id = $1
|
||||||
ORDER BY p.category, p.name
|
ORDER BY p.category, p.name
|
||||||
`, [userId]);
|
`, [userId]);
|
||||||
|
res.json({
|
||||||
// Combine user and permissions
|
|
||||||
const user = {
|
|
||||||
...userResult.rows[0],
|
...userResult.rows[0],
|
||||||
permissions: permissionsResult.rows
|
permissions: permissionsResult.rows,
|
||||||
};
|
});
|
||||||
|
|
||||||
res.json(user);
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting user:', error);
|
console.error('Error getting user:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
|
||||||
|
|
||||||
// Create new user
|
|
||||||
router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => {
|
|
||||||
const client = await pool.connect();
|
|
||||||
|
|
||||||
try {
|
|
||||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
|
||||||
|
|
||||||
console.log("Create user request:", {
|
|
||||||
username,
|
|
||||||
email,
|
|
||||||
is_admin,
|
|
||||||
is_active,
|
|
||||||
rocket_chat_user_id,
|
|
||||||
permissions: permissions || []
|
|
||||||
});
|
});
|
||||||
|
|
||||||
// Validate required fields
|
router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => {
|
||||||
|
const client = await pool.connect();
|
||||||
|
try {
|
||||||
|
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||||
if (!username || !password) {
|
if (!username || !password) {
|
||||||
return res.status(400).json({ error: 'Username and password are required' });
|
return res.status(400).json({ error: 'Username and password are required' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if username is taken
|
|
||||||
const existingUser = await client.query(
|
const existingUser = await client.query(
|
||||||
'SELECT id FROM users WHERE username = $1',
|
'SELECT id FROM users WHERE username = $1',
|
||||||
[username]
|
[username]
|
||||||
);
|
);
|
||||||
|
|
||||||
if (existingUser.rows.length > 0) {
|
if (existingUser.rows.length > 0) {
|
||||||
return res.status(400).json({ error: 'Username already exists' });
|
return res.status(400).json({ error: 'Username already exists' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Start transaction
|
|
||||||
await client.query('BEGIN');
|
await client.query('BEGIN');
|
||||||
|
const hashedPassword = await bcrypt.hash(password, 10);
|
||||||
// Hash password
|
|
||||||
const saltRounds = 10;
|
|
||||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
|
||||||
|
|
||||||
// Insert new user
|
|
||||||
// Convert rocket_chat_user_id to integer if provided
|
|
||||||
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
||||||
|
|
||||||
const userResult = await client.query(`
|
const userResult = await client.query(`
|
||||||
INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at)
|
INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at)
|
||||||
VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP)
|
VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP)
|
||||||
RETURNING id
|
RETURNING id
|
||||||
`, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]);
|
`, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]);
|
||||||
|
|
||||||
const userId = userResult.rows[0].id;
|
const userId = userResult.rows[0].id;
|
||||||
|
|
||||||
// Assign permissions if provided and not admin
|
|
||||||
if (!is_admin && Array.isArray(permissions) && permissions.length > 0) {
|
if (!is_admin && Array.isArray(permissions) && permissions.length > 0) {
|
||||||
console.log("Adding permissions for new user:", userId);
|
const permissionIds = normalizePermissionIds(permissions);
|
||||||
console.log("Permissions received:", permissions);
|
|
||||||
|
|
||||||
// Check permission format
|
|
||||||
const permissionIds = permissions.map(p => {
|
|
||||||
if (typeof p === 'object' && p.id) {
|
|
||||||
console.log("Permission is an object with ID:", p.id);
|
|
||||||
return parseInt(p.id, 10);
|
|
||||||
} else if (typeof p === 'number') {
|
|
||||||
console.log("Permission is a number:", p);
|
|
||||||
return p;
|
|
||||||
} else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
|
|
||||||
console.log("Permission is a string that can be parsed as a number:", p);
|
|
||||||
return parseInt(p, 10);
|
|
||||||
} else {
|
|
||||||
console.log("Unknown permission format:", typeof p, p);
|
|
||||||
// If it's a permission code, we need to look up the ID
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}).filter(id => id !== null);
|
|
||||||
|
|
||||||
console.log("Filtered permission IDs:", permissionIds);
|
|
||||||
|
|
||||||
if (permissionIds.length > 0) {
|
if (permissionIds.length > 0) {
|
||||||
const permissionValues = permissionIds
|
await client.query(
|
||||||
.map(permId => `(${userId}, ${permId})`)
|
`INSERT INTO user_permissions (user_id, permission_id)
|
||||||
.join(',');
|
SELECT $1, unnest($2::int[])
|
||||||
|
ON CONFLICT DO NOTHING`,
|
||||||
console.log("Inserting permission values:", permissionValues);
|
[userId, permissionIds]
|
||||||
|
);
|
||||||
try {
|
|
||||||
await client.query(`
|
|
||||||
INSERT INTO user_permissions (user_id, permission_id)
|
|
||||||
VALUES ${permissionValues}
|
|
||||||
ON CONFLICT DO NOTHING
|
|
||||||
`);
|
|
||||||
console.log("Successfully inserted permissions for new user:", userId);
|
|
||||||
} catch (err) {
|
|
||||||
console.error("Error inserting permissions for new user:", err);
|
|
||||||
throw err;
|
|
||||||
}
|
}
|
||||||
} else {
|
|
||||||
console.log("No valid permission IDs found to insert for new user");
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
console.log("Not adding permissions: is_admin =", is_admin, "permissions array:", Array.isArray(permissions), "length:", permissions ? permissions.length : 0);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
await client.query('COMMIT');
|
await client.query('COMMIT');
|
||||||
|
res.status(201).json({ id: userId, message: 'User created successfully' });
|
||||||
res.status(201).json({
|
|
||||||
id: userId,
|
|
||||||
message: 'User created successfully'
|
|
||||||
});
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await client.query('ROLLBACK');
|
await client.query('ROLLBACK');
|
||||||
console.error('Error creating user:', error);
|
console.error('Error creating user:', error);
|
||||||
@@ -303,156 +181,66 @@ router.post('/users', authenticate, requirePermission('create:users'), async (re
|
|||||||
} finally {
|
} finally {
|
||||||
client.release();
|
client.release();
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Update user
|
router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => {
|
||||||
router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => {
|
|
||||||
const client = await pool.connect();
|
const client = await pool.connect();
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const userId = req.params.id;
|
const userId = req.params.id;
|
||||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||||
|
|
||||||
console.log("Update user request:", {
|
const userExists = await client.query('SELECT id FROM users WHERE id = $1', [userId]);
|
||||||
userId,
|
|
||||||
username,
|
|
||||||
email,
|
|
||||||
is_admin,
|
|
||||||
is_active,
|
|
||||||
rocket_chat_user_id,
|
|
||||||
permissions: permissions || []
|
|
||||||
});
|
|
||||||
|
|
||||||
// Check if user exists
|
|
||||||
const userExists = await client.query(
|
|
||||||
'SELECT id FROM users WHERE id = $1',
|
|
||||||
[userId]
|
|
||||||
);
|
|
||||||
|
|
||||||
if (userExists.rows.length === 0) {
|
if (userExists.rows.length === 0) {
|
||||||
return res.status(404).json({ error: 'User not found' });
|
return res.status(404).json({ error: 'User not found' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Start transaction
|
|
||||||
await client.query('BEGIN');
|
await client.query('BEGIN');
|
||||||
|
|
||||||
// Build update fields
|
|
||||||
const updateFields = [];
|
const updateFields = [];
|
||||||
const updateValues = [userId]; // First parameter is the user ID
|
const updateValues = [userId];
|
||||||
let paramIndex = 2;
|
let paramIndex = 2;
|
||||||
|
|
||||||
if (username !== undefined) {
|
if (username !== undefined) { updateFields.push(`username = $${paramIndex++}`); updateValues.push(username); }
|
||||||
updateFields.push(`username = $${paramIndex++}`);
|
if (email !== undefined) { updateFields.push(`email = $${paramIndex++}`); updateValues.push(email || null); }
|
||||||
updateValues.push(username);
|
if (is_admin !== undefined) { updateFields.push(`is_admin = $${paramIndex++}`); updateValues.push(!!is_admin); }
|
||||||
}
|
if (is_active !== undefined) { updateFields.push(`is_active = $${paramIndex++}`); updateValues.push(!!is_active); }
|
||||||
|
|
||||||
if (email !== undefined) {
|
|
||||||
updateFields.push(`email = $${paramIndex++}`);
|
|
||||||
updateValues.push(email || null);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (is_admin !== undefined) {
|
|
||||||
updateFields.push(`is_admin = $${paramIndex++}`);
|
|
||||||
updateValues.push(!!is_admin);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (is_active !== undefined) {
|
|
||||||
updateFields.push(`is_active = $${paramIndex++}`);
|
|
||||||
updateValues.push(!!is_active);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (rocket_chat_user_id !== undefined) {
|
if (rocket_chat_user_id !== undefined) {
|
||||||
updateFields.push(`rocket_chat_user_id = $${paramIndex++}`);
|
updateFields.push(`rocket_chat_user_id = $${paramIndex++}`);
|
||||||
// Convert to integer if not null/undefined, otherwise null
|
updateValues.push(rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null);
|
||||||
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
|
||||||
updateValues.push(rcUserId);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update password if provided
|
|
||||||
if (password) {
|
if (password) {
|
||||||
const saltRounds = 10;
|
const hashedPassword = await bcrypt.hash(password, 10);
|
||||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
|
||||||
updateFields.push(`password = $${paramIndex++}`);
|
updateFields.push(`password = $${paramIndex++}`);
|
||||||
updateValues.push(hashedPassword);
|
updateValues.push(hashedPassword);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update user if there are fields to update
|
|
||||||
if (updateFields.length > 0) {
|
if (updateFields.length > 0) {
|
||||||
updateFields.push(`updated_at = CURRENT_TIMESTAMP`);
|
updateFields.push(`updated_at = CURRENT_TIMESTAMP`);
|
||||||
|
|
||||||
await client.query(`
|
await client.query(`
|
||||||
UPDATE users
|
UPDATE users SET ${updateFields.join(', ')} WHERE id = $1
|
||||||
SET ${updateFields.join(', ')}
|
|
||||||
WHERE id = $1
|
|
||||||
`, updateValues);
|
`, updateValues);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update permissions if provided
|
|
||||||
if (Array.isArray(permissions)) {
|
if (Array.isArray(permissions)) {
|
||||||
console.log("Updating permissions for user:", userId);
|
await client.query('DELETE FROM user_permissions WHERE user_id = $1', [userId]);
|
||||||
console.log("Permissions received:", permissions);
|
const newIsAdmin = is_admin !== undefined
|
||||||
|
? is_admin
|
||||||
// First remove existing permissions
|
: (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin;
|
||||||
await client.query(
|
|
||||||
'DELETE FROM user_permissions WHERE user_id = $1',
|
|
||||||
[userId]
|
|
||||||
);
|
|
||||||
console.log("Deleted existing permissions for user:", userId);
|
|
||||||
|
|
||||||
// Add new permissions if any and not admin
|
|
||||||
const newIsAdmin = is_admin !== undefined ? is_admin : (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin;
|
|
||||||
|
|
||||||
console.log("User is admin:", newIsAdmin);
|
|
||||||
|
|
||||||
if (!newIsAdmin && permissions.length > 0) {
|
if (!newIsAdmin && permissions.length > 0) {
|
||||||
console.log("Adding permissions:", permissions);
|
const permissionIds = normalizePermissionIds(permissions);
|
||||||
|
|
||||||
// Check permission format
|
|
||||||
const permissionIds = permissions.map(p => {
|
|
||||||
if (typeof p === 'object' && p.id) {
|
|
||||||
console.log("Permission is an object with ID:", p.id);
|
|
||||||
return parseInt(p.id, 10);
|
|
||||||
} else if (typeof p === 'number') {
|
|
||||||
console.log("Permission is a number:", p);
|
|
||||||
return p;
|
|
||||||
} else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
|
|
||||||
console.log("Permission is a string that can be parsed as a number:", p);
|
|
||||||
return parseInt(p, 10);
|
|
||||||
} else {
|
|
||||||
console.log("Unknown permission format:", typeof p, p);
|
|
||||||
// If it's a permission code, we need to look up the ID
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}).filter(id => id !== null);
|
|
||||||
|
|
||||||
console.log("Filtered permission IDs:", permissionIds);
|
|
||||||
|
|
||||||
if (permissionIds.length > 0) {
|
if (permissionIds.length > 0) {
|
||||||
const permissionValues = permissionIds
|
await client.query(
|
||||||
.map(permId => `(${userId}, ${permId})`)
|
`INSERT INTO user_permissions (user_id, permission_id)
|
||||||
.join(',');
|
SELECT $1, unnest($2::int[])
|
||||||
|
ON CONFLICT DO NOTHING`,
|
||||||
console.log("Inserting permission values:", permissionValues);
|
[userId, permissionIds]
|
||||||
|
);
|
||||||
try {
|
|
||||||
await client.query(`
|
|
||||||
INSERT INTO user_permissions (user_id, permission_id)
|
|
||||||
VALUES ${permissionValues}
|
|
||||||
ON CONFLICT DO NOTHING
|
|
||||||
`);
|
|
||||||
console.log("Successfully inserted permissions for user:", userId);
|
|
||||||
} catch (err) {
|
|
||||||
console.error("Error inserting permissions:", err);
|
|
||||||
throw err;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
console.log("No valid permission IDs found to insert");
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
await client.query('COMMIT');
|
await client.query('COMMIT');
|
||||||
|
|
||||||
res.json({ message: 'User updated successfully' });
|
res.json({ message: 'User updated successfully' });
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await client.query('ROLLBACK');
|
await client.query('ROLLBACK');
|
||||||
@@ -461,73 +249,69 @@ router.put('/users/:id', authenticate, requirePermission('edit:users'), async (r
|
|||||||
} finally {
|
} finally {
|
||||||
client.release();
|
client.release();
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Delete user
|
router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => {
|
||||||
router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const userId = req.params.id;
|
const userId = req.params.id;
|
||||||
|
|
||||||
// Check that user is not deleting themselves
|
|
||||||
if (req.user.id === parseInt(userId, 10)) {
|
if (req.user.id === parseInt(userId, 10)) {
|
||||||
return res.status(400).json({ error: 'Cannot delete your own account' });
|
return res.status(400).json({ error: 'Cannot delete your own account' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Delete user (this will cascade to user_permissions due to FK constraints)
|
|
||||||
const result = await pool.query(
|
const result = await pool.query(
|
||||||
'DELETE FROM users WHERE id = $1 RETURNING id',
|
'DELETE FROM users WHERE id = $1 RETURNING id',
|
||||||
[userId]
|
[userId]
|
||||||
);
|
);
|
||||||
|
|
||||||
if (result.rows.length === 0) {
|
if (result.rows.length === 0) {
|
||||||
return res.status(404).json({ error: 'User not found' });
|
return res.status(404).json({ error: 'User not found' });
|
||||||
}
|
}
|
||||||
|
|
||||||
res.json({ message: 'User deleted successfully' });
|
res.json({ message: 'User deleted successfully' });
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error deleting user:', error);
|
console.error('Error deleting user:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Get all permissions grouped by category
|
router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||||
router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const result = await pool.query(`
|
const result = await pool.query(`
|
||||||
SELECT category, json_agg(
|
SELECT category, json_agg(
|
||||||
json_build_object(
|
json_build_object(
|
||||||
'id', id,
|
'id', id, 'name', name, 'code', code, 'description', description
|
||||||
'name', name,
|
|
||||||
'code', code,
|
|
||||||
'description', description
|
|
||||||
) ORDER BY name
|
) ORDER BY name
|
||||||
) as permissions
|
) as permissions
|
||||||
FROM permissions
|
FROM permissions
|
||||||
GROUP BY category
|
GROUP BY category
|
||||||
ORDER BY category
|
ORDER BY category
|
||||||
`);
|
`);
|
||||||
|
|
||||||
res.json(result.rows);
|
res.json(result.rows);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting permissions:', error);
|
console.error('Error getting permissions:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Get all permissions
|
router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||||
router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const result = await pool.query(`
|
const result = await pool.query(`
|
||||||
SELECT *
|
SELECT * FROM permissions ORDER BY category, name
|
||||||
FROM permissions
|
|
||||||
ORDER BY category, name
|
|
||||||
`);
|
`);
|
||||||
|
|
||||||
res.json(result.rows);
|
res.json(result.rows);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error getting permissions:', error);
|
console.error('Error getting permissions:', error);
|
||||||
res.status(500).json({ error: 'Server error' });
|
res.status(500).json({ error: 'Server error' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
return router;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizePermissionIds(permissions) {
|
||||||
|
return permissions
|
||||||
|
.map((p) => {
|
||||||
|
if (typeof p === 'object' && p?.id) return parseInt(p.id, 10);
|
||||||
|
if (typeof p === 'number') return p;
|
||||||
|
if (typeof p === 'string' && !Number.isNaN(parseInt(p, 10))) return parseInt(p, 10);
|
||||||
|
return null;
|
||||||
|
})
|
||||||
|
.filter((id) => id !== null && !Number.isNaN(id));
|
||||||
|
}
|
||||||
|
|||||||
+57
-149
@@ -1,176 +1,84 @@
|
|||||||
require('dotenv').config({ path: '../.env' });
|
import 'dotenv/config';
|
||||||
const express = require('express');
|
import express from 'express';
|
||||||
const cors = require('cors');
|
import cors from 'cors';
|
||||||
const bcrypt = require('bcrypt');
|
import pg from 'pg';
|
||||||
const jwt = require('jsonwebtoken');
|
import { fileURLToPath } from 'node:url';
|
||||||
const { Pool } = require('pg');
|
|
||||||
const morgan = require('morgan');
|
|
||||||
const authRoutes = require('./routes');
|
|
||||||
|
|
||||||
// Log startup configuration
|
const { Pool } = pg;
|
||||||
console.log('Starting auth server with config:', {
|
import { dirname, resolve as resolvePath } from 'node:path';
|
||||||
|
import { config as loadEnv } from 'dotenv';
|
||||||
|
|
||||||
|
import { corsOptions } from '../shared/cors/policy.js';
|
||||||
|
import { requestLog } from '../shared/logging/request-log.js';
|
||||||
|
import { logger } from '../shared/logging/logger.js';
|
||||||
|
import { errorHandler } from '../shared/errors/handler.js';
|
||||||
|
import { loginLimiter, verifyLimiter } from '../shared/rate-limit/login.js';
|
||||||
|
import { extractBearerToken, verifyToken, TokenError } from '../shared/auth/verify.js';
|
||||||
|
|
||||||
|
import { createAuthRoutes } from './routes.js';
|
||||||
|
|
||||||
|
const __filename = fileURLToPath(import.meta.url);
|
||||||
|
const __dirname = dirname(__filename);
|
||||||
|
|
||||||
|
// auth/ lives at inventory-server/auth/, so .env one level up
|
||||||
|
loadEnv({ path: resolvePath(__dirname, '../.env') });
|
||||||
|
|
||||||
|
if (!process.env.JWT_SECRET) {
|
||||||
|
logger.error('JWT_SECRET is not set; refusing to start');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.info({
|
||||||
host: process.env.DB_HOST,
|
host: process.env.DB_HOST,
|
||||||
user: process.env.DB_USER,
|
user: process.env.DB_USER,
|
||||||
database: process.env.DB_NAME,
|
database: process.env.DB_NAME,
|
||||||
port: process.env.DB_PORT,
|
port: process.env.DB_PORT,
|
||||||
auth_port: process.env.AUTH_PORT
|
auth_port: process.env.AUTH_PORT,
|
||||||
});
|
}, 'starting auth server');
|
||||||
|
|
||||||
const app = express();
|
const app = express();
|
||||||
const port = process.env.AUTH_PORT || 3011;
|
const port = Number(process.env.AUTH_PORT) || 3011;
|
||||||
|
|
||||||
// Database configuration
|
|
||||||
const pool = new Pool({
|
const pool = new Pool({
|
||||||
host: process.env.DB_HOST,
|
host: process.env.DB_HOST,
|
||||||
user: process.env.DB_USER,
|
user: process.env.DB_USER,
|
||||||
password: process.env.DB_PASSWORD,
|
password: process.env.DB_PASSWORD,
|
||||||
database: process.env.DB_NAME,
|
database: process.env.DB_NAME,
|
||||||
port: process.env.DB_PORT,
|
port: Number(process.env.DB_PORT) || 5432,
|
||||||
});
|
});
|
||||||
|
|
||||||
// Make pool available globally
|
app.use(requestLog());
|
||||||
global.pool = pool;
|
app.use(express.json({ limit: '1mb' }));
|
||||||
|
app.use(cors(corsOptions));
|
||||||
// Middleware
|
|
||||||
app.use(express.json());
|
|
||||||
app.use(morgan('combined'));
|
|
||||||
app.use(cors({
|
|
||||||
origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://acot.site', 'https://tools.acherryontop.com'],
|
|
||||||
credentials: true
|
|
||||||
}));
|
|
||||||
|
|
||||||
// Login endpoint
|
|
||||||
app.post('/login', async (req, res) => {
|
|
||||||
const { username, password } = req.body;
|
|
||||||
|
|
||||||
|
// Caddy forward_auth target: JWT signature check only, no DB hit.
|
||||||
|
// Returns 200 with X-User-Id / X-User-Username on success; 401 otherwise.
|
||||||
|
// Per-service middleware re-verifies independently; these headers are informational.
|
||||||
|
app.all('/verify', verifyLimiter, (req, res) => {
|
||||||
try {
|
try {
|
||||||
// Get user from database
|
const token = extractBearerToken(req.headers.authorization);
|
||||||
const result = await pool.query(
|
const decoded = verifyToken(token, process.env.JWT_SECRET);
|
||||||
'SELECT id, username, password, is_admin, is_active FROM users WHERE username = $1',
|
res.set('X-User-Id', String(decoded.userId));
|
||||||
[username]
|
if (decoded.username) res.set('X-User-Username', decoded.username);
|
||||||
);
|
res.status(200).end();
|
||||||
|
} catch (err) {
|
||||||
const user = result.rows[0];
|
if (err instanceof TokenError) {
|
||||||
|
return res.status(401).json({ error: err.message });
|
||||||
// Check if user exists and password is correct
|
|
||||||
if (!user || !(await bcrypt.compare(password, user.password))) {
|
|
||||||
return res.status(401).json({ error: 'Invalid username or password' });
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if user is active
|
|
||||||
if (!user.is_active) {
|
|
||||||
return res.status(403).json({ error: 'Account is inactive' });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Update last login timestamp
|
|
||||||
await pool.query(
|
|
||||||
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
|
||||||
[user.id]
|
|
||||||
);
|
|
||||||
|
|
||||||
// Generate JWT token
|
|
||||||
const token = jwt.sign(
|
|
||||||
{ userId: user.id, username: user.username },
|
|
||||||
process.env.JWT_SECRET,
|
|
||||||
{ expiresIn: '24h' }
|
|
||||||
);
|
|
||||||
|
|
||||||
// Get user permissions for the response
|
|
||||||
const permissionsResult = await pool.query(`
|
|
||||||
SELECT code
|
|
||||||
FROM permissions p
|
|
||||||
JOIN user_permissions up ON p.id = up.permission_id
|
|
||||||
WHERE up.user_id = $1
|
|
||||||
`, [user.id]);
|
|
||||||
|
|
||||||
const permissions = permissionsResult.rows.map(row => row.code);
|
|
||||||
|
|
||||||
res.json({
|
|
||||||
token,
|
|
||||||
user: {
|
|
||||||
id: user.id,
|
|
||||||
username: user.username,
|
|
||||||
is_admin: user.is_admin,
|
|
||||||
permissions: user.is_admin ? [] : permissions
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Login error:', error);
|
|
||||||
res.status(500).json({ error: 'Internal server error' });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// User info endpoint
|
|
||||||
app.get('/me', async (req, res) => {
|
|
||||||
const authHeader = req.headers.authorization;
|
|
||||||
|
|
||||||
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
|
||||||
return res.status(401).json({ error: 'No token provided' });
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const token = authHeader.split(' ')[1];
|
|
||||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
|
||||||
|
|
||||||
// Get user details from database
|
|
||||||
const userResult = await pool.query(
|
|
||||||
'SELECT id, username, email, is_admin, rocket_chat_user_id, is_active FROM users WHERE id = $1',
|
|
||||||
[decoded.userId]
|
|
||||||
);
|
|
||||||
|
|
||||||
if (userResult.rows.length === 0) {
|
|
||||||
return res.status(404).json({ error: 'User not found' });
|
|
||||||
}
|
|
||||||
|
|
||||||
const user = userResult.rows[0];
|
|
||||||
|
|
||||||
// Check if user is active
|
|
||||||
if (!user.is_active) {
|
|
||||||
return res.status(403).json({ error: 'Account is inactive' });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get user permissions
|
|
||||||
let permissions = [];
|
|
||||||
if (!user.is_admin) {
|
|
||||||
const permissionsResult = await pool.query(`
|
|
||||||
SELECT code
|
|
||||||
FROM permissions p
|
|
||||||
JOIN user_permissions up ON p.id = up.permission_id
|
|
||||||
WHERE up.user_id = $1
|
|
||||||
`, [user.id]);
|
|
||||||
|
|
||||||
permissions = permissionsResult.rows.map(row => row.code);
|
|
||||||
}
|
|
||||||
|
|
||||||
res.json({
|
|
||||||
id: user.id,
|
|
||||||
username: user.username,
|
|
||||||
email: user.email,
|
|
||||||
rocket_chat_user_id: user.rocket_chat_user_id,
|
|
||||||
is_admin: user.is_admin,
|
|
||||||
permissions: permissions
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Token verification error:', error);
|
|
||||||
res.status(401).json({ error: 'Invalid token' });
|
res.status(401).json({ error: 'Invalid token' });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Mount all routes from routes.js
|
// Login route gets its own rate limiter to slow credential stuffing.
|
||||||
app.use('/', authRoutes);
|
app.use('/login', loginLimiter);
|
||||||
|
|
||||||
// Health check endpoint
|
// Mount user-management + /login + /me from routes.js
|
||||||
app.get('/health', (req, res) => {
|
app.use('/', createAuthRoutes({ pool }));
|
||||||
res.json({ status: 'healthy' });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Error handling middleware
|
app.get('/health', (req, res) => res.json({ status: 'healthy' }));
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error(err.stack);
|
app.use(errorHandler);
|
||||||
res.status(500).json({ error: 'Something broke!' });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
app.listen(port, () => {
|
||||||
console.log(`Auth server running on port ${port}`);
|
logger.info({ port }, 'auth server listening');
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
"name": "chat-server",
|
"name": "chat-server",
|
||||||
"version": "1.0.0",
|
"version": "1.0.0",
|
||||||
"description": "Chat archive server for Rocket.Chat data",
|
"description": "Chat archive server for Rocket.Chat data",
|
||||||
|
"type": "module",
|
||||||
"main": "server.js",
|
"main": "server.js",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"start": "node server.js",
|
"start": "node server.js",
|
||||||
@@ -12,7 +13,10 @@
|
|||||||
"cors": "^2.8.5",
|
"cors": "^2.8.5",
|
||||||
"pg": "^8.11.0",
|
"pg": "^8.11.0",
|
||||||
"dotenv": "^16.0.3",
|
"dotenv": "^16.0.3",
|
||||||
"morgan": "^1.10.0"
|
"morgan": "^1.10.0",
|
||||||
|
"jsonwebtoken": "^9.0.2",
|
||||||
|
"pino": "^9.5.0",
|
||||||
|
"pino-http": "^10.3.0"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"nodemon": "^2.0.22"
|
"nodemon": "^2.0.22"
|
||||||
|
|||||||
@@ -1,5 +1,12 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const path = require('path');
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
// ESM polyfill — Phase 9 §9.1. Handlers below use __dirname to resolve the
|
||||||
|
// db-convert/db/files/{uploads,avatars} static asset paths.
|
||||||
|
const __filename = fileURLToPath(import.meta.url);
|
||||||
|
const __dirname = path.dirname(__filename);
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
|
|
||||||
// Serve uploaded files with proper mapping from database paths to actual file locations
|
// Serve uploaded files with proper mapping from database paths to actual file locations
|
||||||
@@ -646,4 +653,4 @@ router.get('/users/:userId/search', async (req, res) => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,23 +1,62 @@
|
|||||||
require('dotenv').config({ path: '../.env' });
|
// chat-server — Phase 9 §9.1 of CONSOLIDATION_PLAN.md.
|
||||||
const express = require('express');
|
//
|
||||||
const cors = require('cors');
|
// ESM conversion + in-process authenticate() defense-in-depth. Previously this
|
||||||
const { Pool } = require('pg');
|
// service relied on the Caddy `forward_auth` gate alone — `localhost:3014`
|
||||||
const morgan = require('morgan');
|
// was reachable unauthenticated. Now:
|
||||||
const chatRoutes = require('./routes');
|
// 1. Bound to 127.0.0.1 (was 0.0.0.0) so direct-port access is impossible.
|
||||||
|
// 2. authenticate() runs against an in-process `inventory_db` pool before
|
||||||
|
// any route handler sees the request.
|
||||||
|
//
|
||||||
|
// Two pools intentionally:
|
||||||
|
// - `inventoryPool`: used by authenticate() for users/permissions lookups
|
||||||
|
// against the main inventory_db (matches DB_* env vars).
|
||||||
|
// - `pool` (set as global.pool for routes.js): the existing
|
||||||
|
// `rocketchat_converted` pool driven by CHAT_DB_* env vars. routes.js
|
||||||
|
// reads global.pool throughout — no handler-body changes needed.
|
||||||
|
|
||||||
|
import { config as loadEnv } from 'dotenv';
|
||||||
|
import express from 'express';
|
||||||
|
import cors from 'cors';
|
||||||
|
import morgan from 'morgan';
|
||||||
|
import pg from 'pg';
|
||||||
|
import path from 'node:path';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
import { authenticate } from '../shared/auth/middleware.js';
|
||||||
|
import { corsOptions } from '../shared/cors/policy.js';
|
||||||
|
import { errorHandler } from '../shared/errors/handler.js';
|
||||||
|
import { requestLog } from '../shared/logging/request-log.js';
|
||||||
|
|
||||||
|
import chatRoutes from './routes.js';
|
||||||
|
|
||||||
|
const { Pool } = pg;
|
||||||
|
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||||
|
|
||||||
|
// Env layering matches dashboard-server (Deviation #18): shared .env wins on
|
||||||
|
// collisions for security-critical vars, local .env supplies CHAT_DB_*.
|
||||||
|
const sharedEnvPath = '/var/www/inventory/.env';
|
||||||
|
const localEnvPath = path.resolve(__dirname, '.env');
|
||||||
|
if (fs.existsSync(sharedEnvPath)) loadEnv({ path: sharedEnvPath });
|
||||||
|
if (fs.existsSync(localEnvPath)) loadEnv({ path: localEnvPath });
|
||||||
|
|
||||||
|
if (!process.env.JWT_SECRET) {
|
||||||
|
console.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const app = express();
|
||||||
|
const port = Number(process.env.CHAT_PORT) || 3014;
|
||||||
|
|
||||||
// Log startup configuration
|
|
||||||
console.log('Starting chat server with config:', {
|
console.log('Starting chat server with config:', {
|
||||||
host: process.env.CHAT_DB_HOST,
|
host: process.env.CHAT_DB_HOST,
|
||||||
user: process.env.CHAT_DB_USER,
|
user: process.env.CHAT_DB_USER,
|
||||||
database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
|
database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
|
||||||
port: process.env.CHAT_DB_PORT,
|
port: process.env.CHAT_DB_PORT,
|
||||||
chat_port: process.env.CHAT_PORT || 3014
|
chat_port: port,
|
||||||
});
|
});
|
||||||
|
|
||||||
const app = express();
|
// Rocket.Chat archive pool — routes.js reads it via global.pool.
|
||||||
const port = process.env.CHAT_PORT || 3014;
|
|
||||||
|
|
||||||
// Database configuration for rocketchat_converted database
|
|
||||||
const pool = new Pool({
|
const pool = new Pool({
|
||||||
host: process.env.CHAT_DB_HOST,
|
host: process.env.CHAT_DB_HOST,
|
||||||
user: process.env.CHAT_DB_USER,
|
user: process.env.CHAT_DB_USER,
|
||||||
@@ -25,59 +64,69 @@ const pool = new Pool({
|
|||||||
database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
|
database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
|
||||||
port: process.env.CHAT_DB_PORT,
|
port: process.env.CHAT_DB_PORT,
|
||||||
});
|
});
|
||||||
|
|
||||||
// Make pool available globally
|
|
||||||
global.pool = pool;
|
global.pool = pool;
|
||||||
|
|
||||||
// Middleware
|
// inventory_db pool — used by authenticate() for user/permission lookups.
|
||||||
|
const inventoryPool = new Pool({
|
||||||
|
host: process.env.DB_HOST,
|
||||||
|
user: process.env.DB_USER,
|
||||||
|
password: process.env.DB_PASSWORD,
|
||||||
|
database: process.env.DB_NAME,
|
||||||
|
port: Number(process.env.DB_PORT) || 5432,
|
||||||
|
ssl: process.env.DB_SSL === 'true' ? { rejectUnauthorized: false } : false,
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use(requestLog());
|
||||||
app.use(express.json());
|
app.use(express.json());
|
||||||
app.use(morgan('combined'));
|
app.use(morgan('combined'));
|
||||||
app.use(cors({
|
app.use(cors(corsOptions));
|
||||||
origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://acot.site', 'https://tools.acherryontop.com'],
|
|
||||||
credentials: true
|
|
||||||
}));
|
|
||||||
|
|
||||||
// Test database connection endpoint
|
// /health stays unauthenticated for out-of-band probes — mounted BEFORE
|
||||||
app.get('/test-db', async (req, res) => {
|
// authenticate() so monitoring tools on the host can poll without a JWT.
|
||||||
|
// Only reachable via localhost:3014 directly (Caddy routes /health to
|
||||||
|
// inventory-server:3010, not here).
|
||||||
|
app.get('/health', (req, res) => res.json({ status: 'healthy' }));
|
||||||
|
|
||||||
|
// Phase 9 §9.1 — per-server auth re-verification. Every chat route must pass
|
||||||
|
// authenticate() in addition to the Caddy forward_auth gate.
|
||||||
|
app.use(authenticate({ pool: inventoryPool, secret: process.env.JWT_SECRET }));
|
||||||
|
|
||||||
|
app.get('/test-db', async (req, res, next) => {
|
||||||
try {
|
try {
|
||||||
const result = await pool.query('SELECT COUNT(*) as user_count FROM users WHERE active = true');
|
const result = await pool.query('SELECT COUNT(*) as user_count FROM users WHERE active = true');
|
||||||
const messageResult = await pool.query('SELECT COUNT(*) as message_count FROM message');
|
const messageResult = await pool.query('SELECT COUNT(*) as message_count FROM message');
|
||||||
const roomResult = await pool.query('SELECT COUNT(*) as room_count FROM room');
|
const roomResult = await pool.query('SELECT COUNT(*) as room_count FROM room');
|
||||||
|
|
||||||
res.json({
|
res.json({
|
||||||
status: 'success',
|
status: 'success',
|
||||||
database: 'rocketchat_converted',
|
database: 'rocketchat_converted',
|
||||||
stats: {
|
stats: {
|
||||||
active_users: parseInt(result.rows[0].user_count),
|
active_users: parseInt(result.rows[0].user_count, 10),
|
||||||
total_messages: parseInt(messageResult.rows[0].message_count),
|
total_messages: parseInt(messageResult.rows[0].message_count, 10),
|
||||||
total_rooms: parseInt(roomResult.rows[0].room_count)
|
total_rooms: parseInt(roomResult.rows[0].room_count, 10),
|
||||||
}
|
},
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Database test error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
status: 'error',
|
|
||||||
error: 'Database connection failed',
|
|
||||||
details: error.message
|
|
||||||
});
|
});
|
||||||
|
} catch (err) {
|
||||||
|
next(err);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Mount all routes from routes.js
|
|
||||||
app.use('/', chatRoutes);
|
app.use('/', chatRoutes);
|
||||||
|
|
||||||
// Health check endpoint
|
app.use(errorHandler);
|
||||||
app.get('/health', (req, res) => {
|
|
||||||
res.json({ status: 'healthy' });
|
// Phase 9 §9.1 — bind to 127.0.0.1. Caddy reverse_proxy targets localhost:3014
|
||||||
|
// already; this closes the gap where unauthenticated direct-port access from
|
||||||
|
// any host on the network was possible.
|
||||||
|
const server = app.listen(port, '127.0.0.1', () => {
|
||||||
|
console.log(`Chat server running on 127.0.0.1:${port}`);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Error handling middleware
|
const shutdown = async (signal) => {
|
||||||
app.use((err, req, res, next) => {
|
console.log(`chat-server shutting down (${signal})`);
|
||||||
console.error(err.stack);
|
server.close();
|
||||||
res.status(500).json({ error: 'Something broke!' });
|
try { await pool.end(); } catch { /* ignore */ }
|
||||||
});
|
try { await inventoryPool.end(); } catch { /* ignore */ }
|
||||||
|
process.exit(0);
|
||||||
// Start server
|
};
|
||||||
app.listen(port, () => {
|
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||||
console.log(`Chat server running on port ${port}`);
|
process.on('SIGINT', () => shutdown('SIGINT'));
|
||||||
});
|
|
||||||
|
|||||||
@@ -1,20 +0,0 @@
|
|||||||
# Caching Server Configuration
|
|
||||||
PORT=3010
|
|
||||||
NODE_ENV=production
|
|
||||||
|
|
||||||
# Database Configuration
|
|
||||||
MONGODB_URI=mongodb://dashboard_user:WDRFWiGXEeaC6aAyUKuT@localhost:27017/dashboard?authSource=dashboard
|
|
||||||
REDIS_URL=redis://:Wgj32YXxxVLtPZoVzUnP@localhost:6379
|
|
||||||
|
|
||||||
# Gorgias
|
|
||||||
GORGIAS_API_USERNAME=matt@acherryontop.com
|
|
||||||
GORGIAS_API_PASSWORD=d2ed0d23d2a7bf11a633a12fb260769f4e4a970d440693e7d64b8d2223fa6503
|
|
||||||
|
|
||||||
# GA4 credentials
|
|
||||||
GA_PROPERTY_ID=281045851
|
|
||||||
GOOGLE_APPLICATION_CREDENTIALS_JSON={"type": "service_account","project_id": "acot-stats","private_key_id": "259d1fd9864efbfa38b8ba02fdd74dc008ace3c5","private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC5Y6foai8WF98k\nIA0yLn94Y3lmDYlyvI9xL2YqSZSyvgK35wdWRTIaEvHKdiUWuYi3ZPdkYmz1OYiV\njVfR2g+mFpA7MI/JMwyGWwjnV4WW2q6INfgi/PvHlbP3LyyQo0B8CvAY0CHqrpDs\nlJQhAkqmteU24dqcdZoV3vM8JMsDiXm44DqwXsEfWibKv4i0mWNkwiEQr0yImHwb\nbjgclwVLLi5kdM2+49PXr47LCODdL+xmX0uSdgSG6XYqEIVsEOXIUJKzqUe036b/\nEFQ0BxWdJBWs/MYOapn/NNv+Mts+am2ipUuIcgPbOut4xa2Fkky93WnJf0tB+VJP\njFnyZJhdAgMBAAECggEAC980Cp/4zvSNZMNWr6l8ST8u2thavnRmcoGYtx7ffQjK\nT3Dl2TefgJLzqpr2lLt3OVint7p5LsUAmE8lBLpu+RxbH9HkIKbPvQTfD5gyZQQx\nBruqCGzkn2st9fzZNj6gwQYe9P/TGYkUnR8wqI0nLwDZTQful3QNKixiWC4lAAoK\nqdd6H++pqjVUiTqgFwFD3zBAhO0Lp8m/c5vTRT5kxi0wCTK66FaaGLr2OwZHcohp\nE8rEcTZ5kaJzBwqEz522R6ufQqN1Swoq4K6Ul3aAc59539VdrLNs++/eRH38MMVq\n5UTwBrH+zIkXIYv4mtGpR1NWGO2bZ652GzGXNEXcQQKBgQD9WsMmioIeWR9P9I0r\nIY+yyxz1EyscutUtnOtROT36OxokrzQaAKDz/OC3jVnhZSkzG6RcmmK/AJrcU+2m\n1L4mZGfF3DdeTqtK/KkNzGs9yRPDkbb/MF0wgtcvfE8tJH/suiDJKQNsjeaQIQW3\n4NvDxs0w60m9r9tk1CQau94ovQKBgQC7UzeA0mDSxIB5agGbvnzaJJTvAFvnCvhz\nu3ZakTlNecAHu4eOMc0+OCHFPLJlLL4b0oraOxZIszX9BTlgcstBmTUk03TibNsS\nsDiImHFC4hE5x6EPdifnkVFUXPMZ/eF0mHUPBEn41ipw1hoLfl6W+aYW9QUxBMWA\nzdMH4rg4IQKBgQCFcMaUiCNchKhfXnj0HKspCp3n3v64FReu/JVcpH+mSnbMl5Mj\nlu0vVSOuyb5rXvLCPm7lb1NPMqxeG75yPl8grYWSyxhGjbzetBD+eYqKclv8h8UQ\nx5JtuJxKIHk7V5whPS+DhByPknW7uAjg/ogBp7XvbB3c0MEHbEzP3991KQKBgC+a\n610Kmd6WX4v7e6Mn2rTZXRwL/E8QA6nttxs3Etf0m++bIczqLR2lyDdGwJNjtoB9\nlhn1sCkTmiHOBRHUuoDWPaI5NtggD+CE9ikIjKgRqY0EhZLXVTbNQFzvLjypv3UR\nFZaWYXIigzCfyIipOcKmeSYWaJZXfxXHuNylKmnhAoGAFa84AuOOGUr+pEvtUzIr\nvBKu1mnQbbsLEhgf3Tw88K3sO5OlguAwBEvD4eitj/aU5u2vJJhFa67cuERLsZru\n0sjtQwP6CJbWF4uaH0Hso4KQvnwl4BfdKwUncqoKtHrQiuGMvr5P5G941+Ax8brE\nJlC2e/RPUQKxScpK3nNK9mc=\n-----END PRIVATE KEY-----\n","client_email": "matt-dashboard@acot-stats.iam.gserviceaccount.com","client_id": "106112731322970982546","auth_uri": "https://accounts.google.com/o/oauth2/auth","token_uri": "https://oauth2.googleapis.com/token","auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs","client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/matt-dashboard%40acot-stats.iam.gserviceaccount.com","universe_domain": "googleapis.com"}
|
|
||||||
|
|
||||||
# Logging
|
|
||||||
LOG_LEVEL=info
|
|
||||||
LOG_MAX_SIZE=10m
|
|
||||||
LOG_MAX_FILES=5
|
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# dashboard-server .env template (Phase 4)
|
||||||
|
#
|
||||||
|
# The merged dashboard-server reads /var/www/inventory/.env FIRST (provides
|
||||||
|
# JWT_SECRET, DB_*, REDIS_*) and then layers this .env on top for vendor keys.
|
||||||
|
# Shared/security-critical vars stay in /var/www/inventory/.env so they aren't
|
||||||
|
# duplicated; vendor keys live here.
|
||||||
|
#
|
||||||
|
# Copy to .env and populate. Do NOT commit the populated file.
|
||||||
|
|
||||||
|
# Port the merged service listens on
|
||||||
|
DASHBOARD_PORT=3015
|
||||||
|
|
||||||
|
# Klaviyo (replaces klaviyo-server/.env)
|
||||||
|
KLAVIYO_API_KEY=
|
||||||
|
KLAVIYO_API_REVISION=2024-02-15
|
||||||
|
KLAVIYO_API_URL=https://a.klaviyo.com/api
|
||||||
|
|
||||||
|
# Meta / Facebook Ads (replaces meta-server/.env)
|
||||||
|
META_ACCESS_TOKEN=
|
||||||
|
META_AD_ACCOUNT_ID=
|
||||||
|
META_API_VERSION=v21.0
|
||||||
|
|
||||||
|
# Google Analytics (replaces google-server/.env)
|
||||||
|
GA_PROPERTY_ID=
|
||||||
|
GOOGLE_APPLICATION_CREDENTIALS_JSON=
|
||||||
|
|
||||||
|
# Typeform (replaces typeform-server/.env)
|
||||||
|
TYPEFORM_ACCESS_TOKEN=
|
||||||
|
|
||||||
|
# Vendors share the inventory REDIS_URL or REDIS_HOST/PORT/USERNAME/PASSWORD
|
||||||
|
# from the parent .env. Do NOT redeclare here unless you need a vendor-only
|
||||||
|
# override (rare; would need to fork shared/db/redis.js too).
|
||||||
@@ -1,6 +1,11 @@
|
|||||||
const { Client } = require('ssh2');
|
// Per Deviation #13 in CONSOLIDATION_PLAN.md: `ssh2` is CJS and its named export
|
||||||
const mysql = require('mysql2/promise');
|
// (`Client`) isn't reliably detected by Node's CJS→ESM interop static analysis.
|
||||||
const fs = require('fs');
|
// Default-import + destructure is the bulletproof pattern.
|
||||||
|
import ssh2 from 'ssh2';
|
||||||
|
import mysql from 'mysql2/promise';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
|
||||||
|
const { Client } = ssh2;
|
||||||
|
|
||||||
// Connection pool configuration
|
// Connection pool configuration
|
||||||
const connectionPool = {
|
const connectionPool = {
|
||||||
@@ -288,10 +293,10 @@ function getPoolStatus() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = {
|
export {
|
||||||
getDbConnection,
|
getDbConnection,
|
||||||
getCachedQuery,
|
getCachedQuery,
|
||||||
clearQueryCache,
|
clearQueryCache,
|
||||||
closeAllConnections,
|
closeAllConnections,
|
||||||
getPoolStatus
|
getPoolStatus,
|
||||||
};
|
};
|
||||||
@@ -15,6 +15,7 @@
|
|||||||
"luxon": "^3.5.0",
|
"luxon": "^3.5.0",
|
||||||
"morgan": "^1.10.0",
|
"morgan": "^1.10.0",
|
||||||
"mysql2": "^3.6.5",
|
"mysql2": "^3.6.5",
|
||||||
|
"pg": "^8.21.0",
|
||||||
"ssh2": "^1.14.0"
|
"ssh2": "^1.14.0"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
@@ -1142,6 +1143,95 @@
|
|||||||
"integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==",
|
"integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==",
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/pg": {
|
||||||
|
"version": "8.21.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg/-/pg-8.21.0.tgz",
|
||||||
|
"integrity": "sha512-AUP1EYJuHraQGsVoCQVIcM7TEJVGtDzxWtGFZd8rds9d+CCXlU5Js1rYgfLNvxy9iJrpHjGrRjoi/3BT9fRyiA==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"pg-connection-string": "^2.13.0",
|
||||||
|
"pg-pool": "^3.14.0",
|
||||||
|
"pg-protocol": "^1.14.0",
|
||||||
|
"pg-types": "2.2.0",
|
||||||
|
"pgpass": "1.0.5"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">= 16.0.0"
|
||||||
|
},
|
||||||
|
"optionalDependencies": {
|
||||||
|
"pg-cloudflare": "^1.4.0"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"pg-native": ">=3.0.1"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"pg-native": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/pg-cloudflare": {
|
||||||
|
"version": "1.4.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
|
||||||
|
"integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
|
||||||
|
"license": "MIT",
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"node_modules/pg-connection-string": {
|
||||||
|
"version": "2.13.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.13.0.tgz",
|
||||||
|
"integrity": "sha512-EMnU9E2fSULdsbErBbMaXJvFeD9B4+nPcM3f+4lsiCR0BHLPrLVjv3DbyM2hgQQviKJaTWIRRTjKjWlHg3p2ig==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
|
"node_modules/pg-int8": {
|
||||||
|
"version": "1.0.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
|
||||||
|
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
|
||||||
|
"license": "ISC",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=4.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/pg-pool": {
|
||||||
|
"version": "3.14.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
|
||||||
|
"integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
|
||||||
|
"license": "MIT",
|
||||||
|
"peerDependencies": {
|
||||||
|
"pg": ">=8.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/pg-protocol": {
|
||||||
|
"version": "1.14.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.14.0.tgz",
|
||||||
|
"integrity": "sha512-n5taZ1kO3s9ngDTVxsEznOqCyToTgz0FLuPq0B33COy5pPpuWJpY3/2oRBVETuOgzdqRXfWpM9HIhp2LBBT1BA==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
|
"node_modules/pg-types": {
|
||||||
|
"version": "2.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
|
||||||
|
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"pg-int8": "1.0.1",
|
||||||
|
"postgres-array": "~2.0.0",
|
||||||
|
"postgres-bytea": "~1.0.0",
|
||||||
|
"postgres-date": "~1.0.4",
|
||||||
|
"postgres-interval": "^1.1.0"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=4"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/pgpass": {
|
||||||
|
"version": "1.0.5",
|
||||||
|
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
|
||||||
|
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"split2": "^4.1.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/picomatch": {
|
"node_modules/picomatch": {
|
||||||
"version": "2.3.1",
|
"version": "2.3.1",
|
||||||
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
|
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
|
||||||
@@ -1155,6 +1245,45 @@
|
|||||||
"url": "https://github.com/sponsors/jonschlinkert"
|
"url": "https://github.com/sponsors/jonschlinkert"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/postgres-array": {
|
||||||
|
"version": "2.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
|
||||||
|
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=4"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/postgres-bytea": {
|
||||||
|
"version": "1.0.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
|
||||||
|
"integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=0.10.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/postgres-date": {
|
||||||
|
"version": "1.0.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
|
||||||
|
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=0.10.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/postgres-interval": {
|
||||||
|
"version": "1.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
|
||||||
|
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"xtend": "^4.0.0"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=0.10.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/proxy-addr": {
|
"node_modules/proxy-addr": {
|
||||||
"version": "2.0.7",
|
"version": "2.0.7",
|
||||||
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
|
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
|
||||||
@@ -1416,6 +1545,15 @@
|
|||||||
"node": ">=10"
|
"node": ">=10"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/split2": {
|
||||||
|
"version": "4.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
|
||||||
|
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
|
||||||
|
"license": "ISC",
|
||||||
|
"engines": {
|
||||||
|
"node": ">= 10.x"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/sqlstring": {
|
"node_modules/sqlstring": {
|
||||||
"version": "2.3.3",
|
"version": "2.3.3",
|
||||||
"resolved": "https://registry.npmjs.org/sqlstring/-/sqlstring-2.3.3.tgz",
|
"resolved": "https://registry.npmjs.org/sqlstring/-/sqlstring-2.3.3.tgz",
|
||||||
@@ -1548,6 +1686,15 @@
|
|||||||
"engines": {
|
"engines": {
|
||||||
"node": ">= 0.8"
|
"node": ">= 0.8"
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"node_modules/xtend": {
|
||||||
|
"version": "4.0.2",
|
||||||
|
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
|
||||||
|
"integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=0.4"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,20 +2,22 @@
|
|||||||
"name": "acot-server",
|
"name": "acot-server",
|
||||||
"version": "1.0.0",
|
"version": "1.0.0",
|
||||||
"description": "A Cherry On Top production database server",
|
"description": "A Cherry On Top production database server",
|
||||||
|
"type": "module",
|
||||||
"main": "server.js",
|
"main": "server.js",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"start": "node server.js",
|
"start": "node server.js",
|
||||||
"dev": "nodemon server.js"
|
"dev": "nodemon server.js"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"express": "^4.18.2",
|
"compression": "^1.7.4",
|
||||||
"cors": "^2.8.5",
|
"cors": "^2.8.5",
|
||||||
"dotenv": "^16.3.1",
|
"dotenv": "^16.3.1",
|
||||||
|
"express": "^4.18.2",
|
||||||
|
"luxon": "^3.5.0",
|
||||||
"morgan": "^1.10.0",
|
"morgan": "^1.10.0",
|
||||||
"ssh2": "^1.14.0",
|
|
||||||
"mysql2": "^3.6.5",
|
"mysql2": "^3.6.5",
|
||||||
"compression": "^1.7.4",
|
"pg": "^8.21.0",
|
||||||
"luxon": "^3.5.0"
|
"ssh2": "^1.14.0"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"nodemon": "^3.0.1"
|
"nodemon": "^3.0.1"
|
||||||
|
|||||||
@@ -8,10 +8,11 @@
|
|||||||
// NOTE: `users.phone` is not yet indexed in production. Admin will add
|
// NOTE: `users.phone` is not yet indexed in production. Admin will add
|
||||||
// `idx_phone (phone)` — queries here assume that exists for acceptable latency.
|
// `idx_phone (phone)` — queries here assume that exists for acceptable latency.
|
||||||
|
|
||||||
const express = require('express');
|
import express from 'express';
|
||||||
|
import { getDbConnection, getCachedQuery } from '../db/connection.js';
|
||||||
|
import { requirePhoneApiKey } from '../utils/phoneAuth.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const { getDbConnection, getCachedQuery } = require('../db/connection');
|
|
||||||
const { requirePhoneApiKey } = require('../utils/phoneAuth');
|
|
||||||
|
|
||||||
// Order status labels mirror ACOTCustomerDataServiceProvider.php.
|
// Order status labels mirror ACOTCustomerDataServiceProvider.php.
|
||||||
const ORDER_STATUS_LABEL = {
|
const ORDER_STATUS_LABEL = {
|
||||||
@@ -319,4 +320,4 @@ router.get('/:cid/orders', async (req, res) => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
const { getDbConnection } = require('../db/connection');
|
import { getDbConnection } from '../db/connection.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
|
|
||||||
@@ -573,4 +573,4 @@ router.post('/simulate', async (req, res) => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,12 +1,9 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
|
import { getDbConnection, getPoolStatus } from '../db/connection.js';
|
||||||
|
import { getTimeRangeConditions, _internal as timeHelpers } from '../utils/timeUtils.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const { getDbConnection, getPoolStatus } = require('../db/connection');
|
|
||||||
const {
|
|
||||||
getTimeRangeConditions,
|
|
||||||
_internal: timeHelpers
|
|
||||||
} = require('../utils/timeUtils');
|
|
||||||
|
|
||||||
const TIMEZONE = 'America/New_York';
|
const TIMEZONE = 'America/New_York';
|
||||||
|
|
||||||
@@ -680,4 +677,4 @@ function getPreviousTimeRange(timeRange) {
|
|||||||
return map[timeRange] || timeRange;
|
return map[timeRange] || timeRange;
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
|
import { getDbConnection, getPoolStatus } from '../db/connection.js';
|
||||||
const router = express.Router();
|
import {
|
||||||
const { getDbConnection, getPoolStatus } = require('../db/connection');
|
|
||||||
const {
|
|
||||||
getTimeRangeConditions,
|
getTimeRangeConditions,
|
||||||
formatBusinessDate,
|
formatBusinessDate,
|
||||||
getBusinessDayBounds,
|
getBusinessDayBounds,
|
||||||
_internal: timeHelpers
|
_internal as timeHelpers,
|
||||||
} = require('../utils/timeUtils');
|
} from '../utils/timeUtils.js';
|
||||||
|
|
||||||
|
const router = express.Router();
|
||||||
|
|
||||||
const TIMEZONE = 'America/New_York';
|
const TIMEZONE = 'America/New_York';
|
||||||
const BUSINESS_DAY_START_HOUR = timeHelpers?.BUSINESS_DAY_START_HOUR ?? 1;
|
const BUSINESS_DAY_START_HOUR = timeHelpers?.BUSINESS_DAY_START_HOUR ?? 1;
|
||||||
@@ -51,6 +51,7 @@ router.get('/stats', async (req, res) => {
|
|||||||
console.log(`[STATS] Getting DB connection... (excludeCherryBox: ${excludeCB})`);
|
console.log(`[STATS] Getting DB connection... (excludeCherryBox: ${excludeCB})`);
|
||||||
const { connection, release } = await getDbConnection();
|
const { connection, release } = await getDbConnection();
|
||||||
console.log(`[STATS] DB connection obtained in ${Date.now() - startTime}ms`);
|
console.log(`[STATS] DB connection obtained in ${Date.now() - startTime}ms`);
|
||||||
|
try {
|
||||||
|
|
||||||
const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
|
const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
|
||||||
|
|
||||||
@@ -374,33 +375,27 @@ router.get('/stats', async (req, res) => {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
return { response, release };
|
return response;
|
||||||
|
} finally {
|
||||||
|
// Always release the connection regardless of whether the outer Promise.race
|
||||||
|
// used our result. If the timeout wins, this IIFE keeps running in the
|
||||||
|
// background until MySQL responds, then this finally releases. Without it,
|
||||||
|
// every timed-out request permanently leaks one pool slot.
|
||||||
|
release();
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
// Race between the main operation and timeout
|
const response = await Promise.race([mainOperation(), timeoutPromise]);
|
||||||
let result;
|
|
||||||
try {
|
|
||||||
result = await Promise.race([mainOperation(), timeoutPromise]);
|
|
||||||
} catch (error) {
|
|
||||||
// If it's a timeout, we don't have a release function to call
|
|
||||||
if (error.message.includes('timeout')) {
|
|
||||||
console.log(`[STATS] Request timed out in ${Date.now() - startTime}ms`);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
// For other errors, re-throw
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
|
|
||||||
const { response, release } = result;
|
|
||||||
|
|
||||||
// Release connection back to pool
|
|
||||||
if (release) release();
|
|
||||||
|
|
||||||
console.log(`[STATS] Request completed in ${Date.now() - startTime}ms`);
|
console.log(`[STATS] Request completed in ${Date.now() - startTime}ms`);
|
||||||
res.json(response);
|
res.json(response);
|
||||||
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
if (error.message.includes('timeout')) {
|
||||||
|
console.log(`[STATS] Request timed out in ${Date.now() - startTime}ms`);
|
||||||
|
} else {
|
||||||
console.error('Error in /stats:', error);
|
console.error('Error in /stats:', error);
|
||||||
|
}
|
||||||
console.log(`[STATS] Request failed in ${Date.now() - startTime}ms`);
|
console.log(`[STATS] Request failed in ${Date.now() - startTime}ms`);
|
||||||
res.status(500).json({ error: error.message });
|
res.status(500).json({ error: error.message });
|
||||||
}
|
}
|
||||||
@@ -1794,4 +1789,5 @@ router.get('/debug/pool', (req, res) => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,9 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
|
import { getDbConnection, getPoolStatus } from '../db/connection.js';
|
||||||
|
import { getTimeRangeConditions } from '../utils/timeUtils.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const { getDbConnection, getPoolStatus } = require('../db/connection');
|
|
||||||
const {
|
|
||||||
getTimeRangeConditions,
|
|
||||||
} = require('../utils/timeUtils');
|
|
||||||
|
|
||||||
const TIMEZONE = 'America/New_York';
|
const TIMEZONE = 'America/New_York';
|
||||||
|
|
||||||
@@ -24,6 +22,7 @@ router.get('/', async (req, res) => {
|
|||||||
console.log(`[OPERATIONS-METRICS] Getting DB connection...`);
|
console.log(`[OPERATIONS-METRICS] Getting DB connection...`);
|
||||||
const { connection, release } = await getDbConnection();
|
const { connection, release } = await getDbConnection();
|
||||||
console.log(`[OPERATIONS-METRICS] DB connection obtained in ${Date.now() - startTime}ms`);
|
console.log(`[OPERATIONS-METRICS] DB connection obtained in ${Date.now() - startTime}ms`);
|
||||||
|
try {
|
||||||
|
|
||||||
const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
|
const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
|
||||||
|
|
||||||
@@ -372,29 +371,26 @@ router.get('/', async (req, res) => {
|
|||||||
trend,
|
trend,
|
||||||
};
|
};
|
||||||
|
|
||||||
return { response, release };
|
return response;
|
||||||
|
} finally {
|
||||||
|
// Always release the connection regardless of who wins Promise.race.
|
||||||
|
// If the timeout wins, this IIFE keeps running until MySQL responds; this
|
||||||
|
// finally ensures the connection still returns to the pool.
|
||||||
|
release();
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
let result;
|
const response = await Promise.race([mainOperation(), timeoutPromise]);
|
||||||
try {
|
|
||||||
result = await Promise.race([mainOperation(), timeoutPromise]);
|
|
||||||
} catch (error) {
|
|
||||||
if (error.message.includes('timeout')) {
|
|
||||||
console.log(`[OPERATIONS-METRICS] Request timed out in ${Date.now() - startTime}ms`);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
|
|
||||||
const { response, release } = result;
|
|
||||||
|
|
||||||
if (release) release();
|
|
||||||
|
|
||||||
console.log(`[OPERATIONS-METRICS] Request completed in ${Date.now() - startTime}ms`);
|
console.log(`[OPERATIONS-METRICS] Request completed in ${Date.now() - startTime}ms`);
|
||||||
res.json(response);
|
res.json(response);
|
||||||
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
if (error.message.includes('timeout')) {
|
||||||
|
console.log(`[OPERATIONS-METRICS] Request timed out in ${Date.now() - startTime}ms`);
|
||||||
|
} else {
|
||||||
console.error('Error in /operations-metrics:', error);
|
console.error('Error in /operations-metrics:', error);
|
||||||
|
}
|
||||||
console.log(`[OPERATIONS-METRICS] Request failed in ${Date.now() - startTime}ms`);
|
console.log(`[OPERATIONS-METRICS] Request failed in ${Date.now() - startTime}ms`);
|
||||||
res.status(500).json({ error: error.message });
|
res.status(500).json({ error: error.message });
|
||||||
}
|
}
|
||||||
@@ -481,4 +477,4 @@ function getPreviousTimeRange(timeRange) {
|
|||||||
return map[timeRange] || timeRange;
|
return map[timeRange] || timeRange;
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
|
import { getDbConnection, getPoolStatus } from '../db/connection.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const { getDbConnection, getPoolStatus } = require('../db/connection');
|
|
||||||
|
|
||||||
const TIMEZONE = 'America/New_York';
|
const TIMEZONE = 'America/New_York';
|
||||||
|
|
||||||
@@ -281,6 +281,7 @@ router.get('/', async (req, res) => {
|
|||||||
console.log(`[PAYROLL-METRICS] Getting DB connection...`);
|
console.log(`[PAYROLL-METRICS] Getting DB connection...`);
|
||||||
const { connection, release } = await getDbConnection();
|
const { connection, release } = await getDbConnection();
|
||||||
console.log(`[PAYROLL-METRICS] DB connection obtained in ${Date.now() - startTime}ms`);
|
console.log(`[PAYROLL-METRICS] DB connection obtained in ${Date.now() - startTime}ms`);
|
||||||
|
try {
|
||||||
|
|
||||||
// Build query for the pay period
|
// Build query for the pay period
|
||||||
const periodStart = payPeriod.start.toJSDate();
|
const periodStart = payPeriod.start.toJSDate();
|
||||||
@@ -373,29 +374,26 @@ router.get('/', async (req, res) => {
|
|||||||
byWeek: hoursData.byWeek,
|
byWeek: hoursData.byWeek,
|
||||||
};
|
};
|
||||||
|
|
||||||
return { response, release };
|
return response;
|
||||||
|
} finally {
|
||||||
|
// Always release the connection regardless of who wins Promise.race.
|
||||||
|
// If the timeout wins, this IIFE keeps running until MySQL responds; this
|
||||||
|
// finally ensures the connection still returns to the pool.
|
||||||
|
release();
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
let result;
|
const response = await Promise.race([mainOperation(), timeoutPromise]);
|
||||||
try {
|
|
||||||
result = await Promise.race([mainOperation(), timeoutPromise]);
|
|
||||||
} catch (error) {
|
|
||||||
if (error.message.includes('timeout')) {
|
|
||||||
console.log(`[PAYROLL-METRICS] Request timed out in ${Date.now() - startTime}ms`);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
|
|
||||||
const { response, release } = result;
|
|
||||||
|
|
||||||
if (release) release();
|
|
||||||
|
|
||||||
console.log(`[PAYROLL-METRICS] Request completed in ${Date.now() - startTime}ms`);
|
console.log(`[PAYROLL-METRICS] Request completed in ${Date.now() - startTime}ms`);
|
||||||
res.json(response);
|
res.json(response);
|
||||||
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
if (error.message.includes('timeout')) {
|
||||||
|
console.log(`[PAYROLL-METRICS] Request timed out in ${Date.now() - startTime}ms`);
|
||||||
|
} else {
|
||||||
console.error('Error in /payroll-metrics:', error);
|
console.error('Error in /payroll-metrics:', error);
|
||||||
|
}
|
||||||
console.log(`[PAYROLL-METRICS] Request failed in ${Date.now() - startTime}ms`);
|
console.log(`[PAYROLL-METRICS] Request failed in ${Date.now() - startTime}ms`);
|
||||||
res.status(500).json({ error: error.message });
|
res.status(500).json({ error: error.message });
|
||||||
}
|
}
|
||||||
@@ -502,4 +500,4 @@ function isCurrentPayPeriod(payPeriod) {
|
|||||||
return now >= payPeriod.start && now <= payPeriod.end;
|
return now >= payPeriod.start && now <= payPeriod.end;
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
const express = require('express');
|
import express from 'express';
|
||||||
|
import { getDbConnection, getCachedQuery } from '../db/connection.js';
|
||||||
|
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const { getDbConnection, getCachedQuery } = require('../db/connection');
|
|
||||||
|
|
||||||
// Test endpoint to count orders
|
// Test endpoint to count orders
|
||||||
router.get('/order-count', async (req, res) => {
|
router.get('/order-count', async (req, res) => {
|
||||||
@@ -54,4 +55,4 @@ router.get('/test-connection', async (req, res) => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
module.exports = router;
|
export default router;
|
||||||
@@ -1,103 +1,163 @@
|
|||||||
require('dotenv').config();
|
// acot-server — Phase 5 of CONSOLIDATION_PLAN.md.
|
||||||
const express = require('express');
|
// Standalone service on ACOT_PORT (default 3012) exposing /api/acot/* against
|
||||||
const cors = require('cors');
|
// the production MySQL `sg` database via an ssh2 tunnel (see db/connection.js).
|
||||||
const morgan = require('morgan');
|
//
|
||||||
const compression = require('compression');
|
// Auth model (two flavors, deliberate):
|
||||||
const fs = require('fs');
|
// - /api/acot/customers/* : x-acot-api-key shared secret (used by acot-phone-server).
|
||||||
const path = require('path');
|
// Mounted BEFORE authenticate() so its requirePhoneApiKey
|
||||||
const { closeAllConnections } = require('./db/connection');
|
// path is the only gate.
|
||||||
|
// - everything else : JWT Bearer via shared/auth/middleware.js authenticate().
|
||||||
|
// Defense-in-depth on top of Caddy forward_auth.
|
||||||
|
//
|
||||||
|
// Shared infrastructure (Phase 2 + Phase 6):
|
||||||
|
// - shared/auth/middleware.js authenticate() for SPA-served routes
|
||||||
|
// - shared/cors/policy.js explicit allowed-origins list (Phase 6.6)
|
||||||
|
// - shared/logging/request-log.js pino-http, Authorization/Cookie redacted (Phase 6.5/6.9)
|
||||||
|
// - shared/errors/handler.js consistent error envelope, no leak in prod
|
||||||
|
//
|
||||||
|
// Env layering: /var/www/inventory/.env loaded FIRST (JWT_SECRET, DB_* for the
|
||||||
|
// shared PG pool used by authenticate to look up user permissions). Local .env
|
||||||
|
// loaded SECOND for ACOT-specific keys (PROD_DB_*, PROD_SSH_*, ACOT_PHONE_API_KEY).
|
||||||
|
// dotenv defaults to override:false, so the first file wins on collisions.
|
||||||
|
|
||||||
|
import { config as loadEnv } from 'dotenv';
|
||||||
|
import express from 'express';
|
||||||
|
import cors from 'cors';
|
||||||
|
import compression from 'compression';
|
||||||
|
import morgan from 'morgan';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import pg from 'pg';
|
||||||
|
|
||||||
|
import { authenticate } from '../../shared/auth/middleware.js';
|
||||||
|
import { corsOptions } from '../../shared/cors/policy.js';
|
||||||
|
import { errorHandler } from '../../shared/errors/handler.js';
|
||||||
|
import { logger } from '../../shared/logging/logger.js';
|
||||||
|
import { requestLog } from '../../shared/logging/request-log.js';
|
||||||
|
|
||||||
|
import { closeAllConnections } from './db/connection.js';
|
||||||
|
|
||||||
|
import testRouter from './routes/test.js';
|
||||||
|
import eventsRouter from './routes/events.js';
|
||||||
|
import discountsRouter from './routes/discounts.js';
|
||||||
|
import employeeMetricsRouter from './routes/employee-metrics.js';
|
||||||
|
import payrollMetricsRouter from './routes/payroll-metrics.js';
|
||||||
|
import operationsMetricsRouter from './routes/operations-metrics.js';
|
||||||
|
import customersRouter from './routes/customers.js';
|
||||||
|
|
||||||
|
const { Pool } = pg;
|
||||||
|
|
||||||
|
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||||
|
|
||||||
|
// Layer envs: shared inventory .env first (JWT_SECRET, DB_*) then acot .env.
|
||||||
|
const sharedEnvPath = '/var/www/inventory/.env';
|
||||||
|
const localEnvPath = path.resolve(__dirname, '.env');
|
||||||
|
if (fs.existsSync(sharedEnvPath)) loadEnv({ path: sharedEnvPath });
|
||||||
|
if (fs.existsSync(localEnvPath)) loadEnv({ path: localEnvPath });
|
||||||
|
|
||||||
|
// Phase 6.4 — refuse to start without JWT_SECRET. authenticate() would reject
|
||||||
|
// every request anyway; failing fast surfaces the misconfiguration immediately.
|
||||||
|
if (!process.env.JWT_SECRET) {
|
||||||
|
logger.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
const app = express();
|
const app = express();
|
||||||
const PORT = process.env.ACOT_PORT || 3012;
|
const PORT = Number(process.env.ACOT_PORT) || 3012;
|
||||||
|
|
||||||
// Create logs directory if it doesn't exist
|
// Trust X-Forwarded-* only when the immediate hop is loopback (Caddy on the same
|
||||||
|
// host). Required for the KIOSK_IPS bypass in shared/auth/middleware.js to see
|
||||||
|
// real client IPs instead of 127.0.0.1.
|
||||||
|
app.set('trust proxy', 'loopback');
|
||||||
|
|
||||||
|
// Postgres pool for authenticate() (user/permission lookups against inventory_db).
|
||||||
|
// All MySQL access goes through db/connection.js (separate, ssh-tunneled).
|
||||||
|
const pool = new Pool({
|
||||||
|
host: process.env.DB_HOST,
|
||||||
|
user: process.env.DB_USER,
|
||||||
|
password: process.env.DB_PASSWORD,
|
||||||
|
database: process.env.DB_NAME,
|
||||||
|
port: Number(process.env.DB_PORT) || 5432,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Per-app access log on disk (kept from pre-conversion behavior; pino request-log
|
||||||
|
// is mounted below for structured/redacted server-side logging).
|
||||||
const logDir = path.join(__dirname, 'logs/app');
|
const logDir = path.join(__dirname, 'logs/app');
|
||||||
if (!fs.existsSync(logDir)) {
|
if (!fs.existsSync(logDir)) {
|
||||||
fs.mkdirSync(logDir, { recursive: true });
|
fs.mkdirSync(logDir, { recursive: true });
|
||||||
}
|
}
|
||||||
|
const accessLogStream = fs.createWriteStream(path.join(logDir, 'access.log'), { flags: 'a' });
|
||||||
|
|
||||||
// Create a write stream for access logs
|
app.use(requestLog());
|
||||||
const accessLogStream = fs.createWriteStream(
|
|
||||||
path.join(logDir, 'access.log'),
|
|
||||||
{ flags: 'a' }
|
|
||||||
);
|
|
||||||
|
|
||||||
// Middleware
|
|
||||||
app.use(compression());
|
app.use(compression());
|
||||||
app.use(cors());
|
app.use(cors(corsOptions));
|
||||||
app.use(express.json());
|
app.use(express.json());
|
||||||
app.use(express.urlencoded({ extended: true }));
|
app.use(express.urlencoded({ extended: true }));
|
||||||
|
|
||||||
// Logging middleware
|
|
||||||
if (process.env.NODE_ENV === 'production') {
|
if (process.env.NODE_ENV === 'production') {
|
||||||
app.use(morgan('combined', { stream: accessLogStream }));
|
app.use(morgan('combined', { stream: accessLogStream }));
|
||||||
} else {
|
} else {
|
||||||
app.use(morgan('dev'));
|
app.use(morgan('dev'));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Health check endpoint
|
|
||||||
app.get('/health', (req, res) => {
|
app.get('/health', (req, res) => {
|
||||||
res.json({
|
res.json({
|
||||||
status: 'healthy',
|
status: 'healthy',
|
||||||
service: 'acot-server',
|
service: 'acot-server',
|
||||||
timestamp: new Date().toISOString(),
|
timestamp: new Date().toISOString(),
|
||||||
uptime: process.uptime()
|
uptime: process.uptime(),
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// Routes
|
// Customers route uses x-acot-api-key (shared secret with acot-phone-server),
|
||||||
app.use('/api/acot/test', require('./routes/test'));
|
// NOT JWT — mount BEFORE authenticate() so requirePhoneApiKey is the only gate.
|
||||||
app.use('/api/acot/events', require('./routes/events'));
|
app.use('/api/acot/customers', customersRouter);
|
||||||
app.use('/api/acot/discounts', require('./routes/discounts'));
|
|
||||||
app.use('/api/acot/employee-metrics', require('./routes/employee-metrics'));
|
|
||||||
app.use('/api/acot/payroll-metrics', require('./routes/payroll-metrics'));
|
|
||||||
app.use('/api/acot/operations-metrics', require('./routes/operations-metrics'));
|
|
||||||
app.use('/api/acot/customers', require('./routes/customers'));
|
|
||||||
|
|
||||||
// Error handling middleware
|
// All remaining /api/acot/* routes require a valid JWT.
|
||||||
app.use((err, req, res, next) => {
|
app.use('/api/acot', authenticate({ pool, secret: process.env.JWT_SECRET }));
|
||||||
console.error('Unhandled error:', err);
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
error: process.env.NODE_ENV === 'production'
|
|
||||||
? 'Internal server error'
|
|
||||||
: err.message
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// 404 handler
|
app.use('/api/acot/test', testRouter);
|
||||||
|
app.use('/api/acot/events', eventsRouter);
|
||||||
|
app.use('/api/acot/discounts', discountsRouter);
|
||||||
|
app.use('/api/acot/employee-metrics', employeeMetricsRouter);
|
||||||
|
app.use('/api/acot/payroll-metrics', payrollMetricsRouter);
|
||||||
|
app.use('/api/acot/operations-metrics', operationsMetricsRouter);
|
||||||
|
|
||||||
|
// 404 for unmatched /api routes (keeps prior behavior).
|
||||||
app.use((req, res) => {
|
app.use((req, res) => {
|
||||||
res.status(404).json({
|
res.status(404).json({ success: false, error: 'Route not found' });
|
||||||
success: false,
|
|
||||||
error: 'Route not found'
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
// Start server
|
app.use(errorHandler);
|
||||||
const server = app.listen(PORT, () => {
|
|
||||||
console.log(`ACOT Server running on port ${PORT}`);
|
const server = app.listen(PORT, '0.0.0.0', () => {
|
||||||
console.log(`Environment: ${process.env.NODE_ENV}`);
|
logger.info({ port: PORT, mode: process.env.NODE_ENV || 'development' }, 'acot-server listening');
|
||||||
});
|
});
|
||||||
|
|
||||||
// Graceful shutdown
|
const gracefulShutdown = async (signal) => {
|
||||||
const gracefulShutdown = async () => {
|
logger.info({ signal }, 'acot-server shutting down');
|
||||||
console.log('SIGTERM signal received: closing HTTP server');
|
|
||||||
server.close(async () => {
|
server.close(async () => {
|
||||||
console.log('HTTP server closed');
|
|
||||||
|
|
||||||
// Close database connections
|
|
||||||
try {
|
try {
|
||||||
await closeAllConnections();
|
await closeAllConnections();
|
||||||
console.log('Database connections closed');
|
} catch (err) {
|
||||||
} catch (error) {
|
logger.error({ err: { message: err.message } }, 'error closing MySQL pool');
|
||||||
console.error('Error closing database connections:', error);
|
|
||||||
}
|
}
|
||||||
|
try {
|
||||||
|
await pool.end();
|
||||||
|
} catch { /* ignore */ }
|
||||||
process.exit(0);
|
process.exit(0);
|
||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
process.on('SIGTERM', gracefulShutdown);
|
process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
|
||||||
process.on('SIGINT', gracefulShutdown);
|
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
|
||||||
|
|
||||||
module.exports = app;
|
process.on('uncaughtException', (err) => {
|
||||||
|
logger.error({ err: { message: err.message, stack: err.stack } }, 'uncaughtException');
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
process.on('unhandledRejection', (reason) => {
|
||||||
|
logger.error({ reason }, 'unhandledRejection');
|
||||||
|
});
|
||||||
|
|
||||||
|
export default app;
|
||||||
|
|||||||
@@ -2,9 +2,9 @@
|
|||||||
// The acot-phone-server sends `x-acot-api-key` on every request; we compare
|
// The acot-phone-server sends `x-acot-api-key` on every request; we compare
|
||||||
// against ACOT_PHONE_API_KEY from the environment using timing-safe comparison.
|
// against ACOT_PHONE_API_KEY from the environment using timing-safe comparison.
|
||||||
|
|
||||||
const crypto = require('crypto');
|
import crypto from 'node:crypto';
|
||||||
|
|
||||||
function requirePhoneApiKey(req, res, next) {
|
export function requirePhoneApiKey(req, res, next) {
|
||||||
const expected = process.env.ACOT_PHONE_API_KEY;
|
const expected = process.env.ACOT_PHONE_API_KEY;
|
||||||
if (!expected) {
|
if (!expected) {
|
||||||
console.error('ACOT_PHONE_API_KEY not configured; rejecting all requests');
|
console.error('ACOT_PHONE_API_KEY not configured; rejecting all requests');
|
||||||
@@ -24,5 +24,3 @@ function requirePhoneApiKey(req, res, next) {
|
|||||||
|
|
||||||
next();
|
next();
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = { requirePhoneApiKey };
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
const { DateTime } = require('luxon');
|
import { DateTime } from 'luxon';
|
||||||
|
|
||||||
const TIMEZONE = 'America/New_York';
|
const TIMEZONE = 'America/New_York';
|
||||||
const DB_TIMEZONE = 'UTC-05:00';
|
const DB_TIMEZONE = 'UTC-05:00';
|
||||||
@@ -294,19 +294,24 @@ const formatMySQLDate = (input) => {
|
|||||||
return dt.setZone(DB_TIMEZONE).toFormat(DB_DATETIME_FORMAT);
|
return dt.setZone(DB_TIMEZONE).toFormat(DB_DATETIME_FORMAT);
|
||||||
};
|
};
|
||||||
|
|
||||||
module.exports = {
|
// Expose helpers for tests or advanced consumers.
|
||||||
|
// Kept as a named `_internal` export so existing destructuring sites
|
||||||
|
// (`const { _internal: timeHelpers } = require(...)` → ESM equivalent works)
|
||||||
|
// don't need to change beyond the import-statement rewrite.
|
||||||
|
const _internal = {
|
||||||
|
getDayStart,
|
||||||
|
getDayEnd,
|
||||||
|
getWeekStart,
|
||||||
|
getRangeForTimeRange,
|
||||||
|
BUSINESS_DAY_START_HOUR,
|
||||||
|
};
|
||||||
|
|
||||||
|
export {
|
||||||
getBusinessDayBounds,
|
getBusinessDayBounds,
|
||||||
getTimeRangeConditions,
|
getTimeRangeConditions,
|
||||||
formatBusinessDate,
|
formatBusinessDate,
|
||||||
getTimeRangeLabel,
|
getTimeRangeLabel,
|
||||||
parseBusinessDate,
|
parseBusinessDate,
|
||||||
formatMySQLDate,
|
formatMySQLDate,
|
||||||
// Expose helpers for tests or advanced consumers
|
_internal,
|
||||||
_internal: {
|
|
||||||
getDayStart,
|
|
||||||
getDayEnd,
|
|
||||||
getWeekStart,
|
|
||||||
getRangeForTimeRange,
|
|
||||||
BUSINESS_DAY_START_HOUR
|
|
||||||
}
|
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,21 +0,0 @@
|
|||||||
# Server Configuration
|
|
||||||
NODE_ENV=development
|
|
||||||
AIRCALL_PORT=3002
|
|
||||||
LOG_LEVEL=info
|
|
||||||
|
|
||||||
# Aircall API Credentials
|
|
||||||
AIRCALL_API_ID=your_aircall_api_id
|
|
||||||
AIRCALL_API_TOKEN=your_aircall_api_token
|
|
||||||
|
|
||||||
# Database Configuration
|
|
||||||
MONGODB_URI=mongodb://localhost:27017/dashboard
|
|
||||||
MONGODB_DB=dashboard
|
|
||||||
REDIS_URL=redis://localhost:6379
|
|
||||||
|
|
||||||
# Service Configuration
|
|
||||||
TIMEZONE=America/New_York
|
|
||||||
DAY_STARTS_AT=1 # Business day starts at 1 AM ET
|
|
||||||
|
|
||||||
# Optional Settings
|
|
||||||
REDIS_TTL=300 # Cache TTL in seconds (5 minutes)
|
|
||||||
COLLECTION_NAME=aircall_daily_data
|
|
||||||
@@ -1,55 +0,0 @@
|
|||||||
# Aircall Server
|
|
||||||
|
|
||||||
A standalone server for handling Aircall metrics and data processing.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
1. Install dependencies:
|
|
||||||
```bash
|
|
||||||
npm install
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Set up environment variables:
|
|
||||||
```bash
|
|
||||||
cp .env.example .env
|
|
||||||
```
|
|
||||||
Then edit `.env` with your configuration.
|
|
||||||
|
|
||||||
Required environment variables:
|
|
||||||
- `AIRCALL_API_ID`: Your Aircall API ID
|
|
||||||
- `AIRCALL_API_TOKEN`: Your Aircall API Token
|
|
||||||
- `MONGODB_URI`: MongoDB connection string
|
|
||||||
- `REDIS_URL`: Redis connection string
|
|
||||||
- `AIRCALL_PORT`: Server port (default: 3002)
|
|
||||||
|
|
||||||
## Running the Server
|
|
||||||
|
|
||||||
### Development
|
|
||||||
```bash
|
|
||||||
npm run dev
|
|
||||||
```
|
|
||||||
|
|
||||||
### Production
|
|
||||||
Using PM2:
|
|
||||||
```bash
|
|
||||||
pm2 start ecosystem.config.js --env production
|
|
||||||
```
|
|
||||||
|
|
||||||
## API Endpoints
|
|
||||||
|
|
||||||
### GET /api/aircall/metrics/:timeRange
|
|
||||||
Get Aircall metrics for a specific time range.
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
- `timeRange`: One of ['today', 'yesterday', 'last7days', 'last30days', 'last90days']
|
|
||||||
|
|
||||||
### GET /api/aircall/health
|
|
||||||
Get server health status.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
The server uses:
|
|
||||||
- Express.js for the API
|
|
||||||
- MongoDB for data storage
|
|
||||||
- Redis for caching
|
|
||||||
- Winston for logging
|
|
||||||
-1914
File diff suppressed because it is too large
Load Diff
@@ -1,23 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "aircall-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"description": "Aircall metrics server",
|
|
||||||
"type": "module",
|
|
||||||
"main": "server.js",
|
|
||||||
"scripts": {
|
|
||||||
"start": "node server.js",
|
|
||||||
"dev": "nodemon server.js"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"axios": "^1.6.2",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.3.1",
|
|
||||||
"express": "^4.18.2",
|
|
||||||
"mongodb": "^6.3.0",
|
|
||||||
"redis": "^4.6.11",
|
|
||||||
"winston": "^3.11.0"
|
|
||||||
},
|
|
||||||
"devDependencies": {
|
|
||||||
"nodemon": "^3.0.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
import express from 'express';
|
|
||||||
import cors from 'cors';
|
|
||||||
import dotenv from 'dotenv';
|
|
||||||
import path from 'path';
|
|
||||||
import { fileURLToPath } from 'url';
|
|
||||||
import { createRoutes } from './src/routes/index.js';
|
|
||||||
import { aircallConfig } from './src/config/aircall.config.js';
|
|
||||||
import { connectMongoDB } from './src/utils/db.js';
|
|
||||||
import { createRedisClient } from './src/utils/redis.js';
|
|
||||||
import { createLogger } from './src/utils/logger.js';
|
|
||||||
|
|
||||||
// Get directory name in ES modules
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
|
||||||
const __dirname = path.dirname(__filename);
|
|
||||||
|
|
||||||
// Load environment variables from the correct path
|
|
||||||
dotenv.config({ path: path.resolve(__dirname, '.env') });
|
|
||||||
|
|
||||||
// Validate required environment variables
|
|
||||||
const requiredEnvVars = ['AIRCALL_API_ID', 'AIRCALL_API_TOKEN', 'MONGODB_URI', 'REDIS_URL'];
|
|
||||||
const missingEnvVars = requiredEnvVars.filter(envVar => !process.env[envVar]);
|
|
||||||
|
|
||||||
if (missingEnvVars.length > 0) {
|
|
||||||
console.error('Missing required environment variables:', missingEnvVars);
|
|
||||||
process.exit(1);
|
|
||||||
}
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.AIRCALL_PORT || 3002;
|
|
||||||
const logger = createLogger('aircall-server');
|
|
||||||
|
|
||||||
// Middleware
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Connect to databases
|
|
||||||
let mongodb;
|
|
||||||
let redis;
|
|
||||||
|
|
||||||
async function initializeServer() {
|
|
||||||
try {
|
|
||||||
// Connect to MongoDB
|
|
||||||
mongodb = await connectMongoDB();
|
|
||||||
logger.info('Connected to MongoDB');
|
|
||||||
|
|
||||||
// Connect to Redis
|
|
||||||
redis = await createRedisClient();
|
|
||||||
logger.info('Connected to Redis');
|
|
||||||
|
|
||||||
// Initialize configs with database connections
|
|
||||||
const configs = {
|
|
||||||
aircall: {
|
|
||||||
...aircallConfig,
|
|
||||||
mongodb,
|
|
||||||
redis,
|
|
||||||
logger
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Initialize routes
|
|
||||||
const routes = createRoutes(configs, logger);
|
|
||||||
app.use('/api', routes);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
logger.error('Server error:', err);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Internal server error',
|
|
||||||
message: err.message
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
|
||||||
logger.info(`Aircall server listening on port ${port}`);
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Failed to initialize server:', error);
|
|
||||||
process.exit(1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
initializeServer();
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
export const aircallConfig = {
|
|
||||||
serviceName: 'aircall',
|
|
||||||
apiId: process.env.AIRCALL_API_ID,
|
|
||||||
apiToken: process.env.AIRCALL_API_TOKEN,
|
|
||||||
timezone: 'America/New_York',
|
|
||||||
dayStartsAt: 1,
|
|
||||||
storeHistory: true,
|
|
||||||
collection: 'aircall_daily_data',
|
|
||||||
redisTTL: 300, // 5 minutes cache for current day
|
|
||||||
endpoints: {
|
|
||||||
metrics: {
|
|
||||||
ttl: 300
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
import express from 'express';
|
|
||||||
import { AircallService } from '../services/aircall/AircallService.js';
|
|
||||||
|
|
||||||
export const createAircallRoutes = (config, logger) => {
|
|
||||||
const router = express.Router();
|
|
||||||
const aircallService = new AircallService(config);
|
|
||||||
|
|
||||||
router.get('/metrics/:timeRange?', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { timeRange = 'today' } = req.params;
|
|
||||||
const allowedRanges = ['today', 'yesterday', 'last7days', 'last30days', 'last90days'];
|
|
||||||
|
|
||||||
if (!allowedRanges.includes(timeRange)) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Invalid time range',
|
|
||||||
allowedRanges
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const metrics = await aircallService.getMetrics(timeRange);
|
|
||||||
|
|
||||||
res.json({
|
|
||||||
...metrics,
|
|
||||||
_meta: {
|
|
||||||
timeRange,
|
|
||||||
generatedAt: new Date().toISOString(),
|
|
||||||
dataPoints: metrics.daily_data?.length || 0
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Error fetching Aircall metrics:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch Aircall metrics',
|
|
||||||
message: error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Health check endpoint
|
|
||||||
router.get('/health', (req, res) => {
|
|
||||||
const mongoConnected = !!aircallService.mongodb?.db;
|
|
||||||
const redisConnected = !!aircallService.redis?.isOpen;
|
|
||||||
|
|
||||||
const health = {
|
|
||||||
status: mongoConnected && redisConnected ? 'ok' : 'degraded',
|
|
||||||
service: 'aircall',
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
connections: {
|
|
||||||
mongodb: mongoConnected,
|
|
||||||
redis: redisConnected
|
|
||||||
}
|
|
||||||
};
|
|
||||||
res.json(health);
|
|
||||||
});
|
|
||||||
|
|
||||||
return router;
|
|
||||||
};
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
import express from 'express';
|
|
||||||
import { createAircallRoutes } from './aircall.routes.js';
|
|
||||||
|
|
||||||
export const createRoutes = (configs, logger) => {
|
|
||||||
const router = express.Router();
|
|
||||||
|
|
||||||
// Mount Aircall routes
|
|
||||||
router.use('/aircall', createAircallRoutes(configs.aircall, logger));
|
|
||||||
|
|
||||||
// Health check endpoint
|
|
||||||
router.get('/health', (req, res) => {
|
|
||||||
const services = req.services || {};
|
|
||||||
res.status(200).json({
|
|
||||||
status: 'ok',
|
|
||||||
timestamp: new Date(),
|
|
||||||
services: {
|
|
||||||
redis: services.redis?.isReady || false,
|
|
||||||
mongodb: services.mongo?.readyState === 1 || false
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Catch-all 404 handler
|
|
||||||
router.use('*', (req, res) => {
|
|
||||||
res.status(404).json({
|
|
||||||
error: 'Not Found',
|
|
||||||
message: `Route ${req.originalUrl} not found`
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
return router;
|
|
||||||
};
|
|
||||||
@@ -1,298 +0,0 @@
|
|||||||
import { DataManager } from "../base/DataManager.js";
|
|
||||||
|
|
||||||
export class AircallDataManager extends DataManager {
|
|
||||||
constructor(mongodb, redis, timeManager) {
|
|
||||||
const options = {
|
|
||||||
collection: "aircall_daily_data",
|
|
||||||
redisTTL: 300 // 5 minutes cache
|
|
||||||
};
|
|
||||||
super(mongodb, redis, timeManager, options);
|
|
||||||
this.options = options;
|
|
||||||
}
|
|
||||||
|
|
||||||
ensureDate(d) {
|
|
||||||
if (d instanceof Date) return d;
|
|
||||||
if (typeof d === 'string') return new Date(d);
|
|
||||||
if (typeof d === 'number') return new Date(d);
|
|
||||||
console.error('Invalid date value:', d);
|
|
||||||
return new Date(); // fallback to current date
|
|
||||||
}
|
|
||||||
|
|
||||||
async storeHistoricalPeriod(start, end, calls) {
|
|
||||||
if (!this.mongodb) return;
|
|
||||||
|
|
||||||
try {
|
|
||||||
if (!Array.isArray(calls)) {
|
|
||||||
console.error("Invalid calls data:", calls);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Group calls by true day boundaries using TimeManager
|
|
||||||
const dailyCallsMap = new Map();
|
|
||||||
|
|
||||||
calls.forEach((call) => {
|
|
||||||
try {
|
|
||||||
const timestamp = call.started_at * 1000; // Convert to milliseconds
|
|
||||||
const callDate = this.ensureDate(timestamp);
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(callDate);
|
|
||||||
const dayKey = dayBounds.start.toISOString();
|
|
||||||
|
|
||||||
if (!dailyCallsMap.has(dayKey)) {
|
|
||||||
dailyCallsMap.set(dayKey, {
|
|
||||||
date: dayBounds.start,
|
|
||||||
calls: [],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
dailyCallsMap.get(dayKey).calls.push(call);
|
|
||||||
} catch (err) {
|
|
||||||
console.error('Error processing call:', err, call);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Iterate over each day in the period using day boundaries
|
|
||||||
const dates = [];
|
|
||||||
let currentDate = this.ensureDate(start);
|
|
||||||
const endDate = this.ensureDate(end);
|
|
||||||
|
|
||||||
while (currentDate < endDate) {
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(currentDate);
|
|
||||||
dates.push(dayBounds.start);
|
|
||||||
currentDate.setUTCDate(currentDate.getUTCDate() + 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
for (const date of dates) {
|
|
||||||
try {
|
|
||||||
const dateKey = date.toISOString();
|
|
||||||
const dayData = dailyCallsMap.get(dateKey);
|
|
||||||
const dayCalls = dayData ? dayData.calls : [];
|
|
||||||
|
|
||||||
// Process calls for this day using the same processing logic
|
|
||||||
const metrics = this.processCallData(dayCalls);
|
|
||||||
|
|
||||||
// Insert a daily_data record for this day
|
|
||||||
metrics.daily_data = [
|
|
||||||
{
|
|
||||||
date: date.toISOString().split("T")[0],
|
|
||||||
inbound: metrics.by_direction.inbound,
|
|
||||||
outbound: metrics.by_direction.outbound,
|
|
||||||
},
|
|
||||||
];
|
|
||||||
|
|
||||||
// Store this day's processed data as historical
|
|
||||||
await this.storeHistoricalDay(date, metrics);
|
|
||||||
} catch (err) {
|
|
||||||
console.error('Error processing date:', err, date);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error("Error storing historical period:", error, error.stack);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
processCallData(calls) {
|
|
||||||
// If calls is already processed (has total, by_direction, etc.), return it
|
|
||||||
if (calls && calls.total !== undefined) {
|
|
||||||
console.log('Data already processed:', {
|
|
||||||
total: calls.total,
|
|
||||||
by_direction: calls.by_direction
|
|
||||||
});
|
|
||||||
// Return a clean copy of the processed data
|
|
||||||
return {
|
|
||||||
total: calls.total,
|
|
||||||
by_direction: calls.by_direction,
|
|
||||||
by_status: calls.by_status,
|
|
||||||
by_missed_reason: calls.by_missed_reason,
|
|
||||||
by_hour: calls.by_hour,
|
|
||||||
by_users: calls.by_users,
|
|
||||||
daily_data: calls.daily_data,
|
|
||||||
duration_distribution: calls.duration_distribution,
|
|
||||||
average_duration: calls.average_duration
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Processing raw calls:', {
|
|
||||||
count: calls.length,
|
|
||||||
sample: calls.length > 0 ? {
|
|
||||||
id: calls[0].id,
|
|
||||||
direction: calls[0].direction,
|
|
||||||
status: calls[0].status
|
|
||||||
} : null
|
|
||||||
});
|
|
||||||
|
|
||||||
// Process raw calls
|
|
||||||
const metrics = {
|
|
||||||
total: calls.length,
|
|
||||||
by_direction: { inbound: 0, outbound: 0 },
|
|
||||||
by_status: { answered: 0, missed: 0 },
|
|
||||||
by_missed_reason: {},
|
|
||||||
by_hour: Array(24).fill(0),
|
|
||||||
by_users: {},
|
|
||||||
daily_data: [],
|
|
||||||
duration_distribution: [
|
|
||||||
{ range: "0-1m", count: 0 },
|
|
||||||
{ range: "1-5m", count: 0 },
|
|
||||||
{ range: "5-15m", count: 0 },
|
|
||||||
{ range: "15-30m", count: 0 },
|
|
||||||
{ range: "30m+", count: 0 },
|
|
||||||
],
|
|
||||||
average_duration: 0,
|
|
||||||
total_duration: 0,
|
|
||||||
};
|
|
||||||
|
|
||||||
// Group calls by date for daily data
|
|
||||||
const dailyCallsMap = new Map();
|
|
||||||
|
|
||||||
calls.forEach((call) => {
|
|
||||||
try {
|
|
||||||
// Direction metrics
|
|
||||||
metrics.by_direction[call.direction]++;
|
|
||||||
|
|
||||||
// Get call date and hour using TimeManager
|
|
||||||
const timestamp = call.started_at * 1000; // Convert to milliseconds
|
|
||||||
const callDate = this.ensureDate(timestamp);
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(callDate);
|
|
||||||
const dayKey = dayBounds.start.toISOString().split("T")[0];
|
|
||||||
const hour = callDate.getHours();
|
|
||||||
metrics.by_hour[hour]++;
|
|
||||||
|
|
||||||
// Status and duration metrics
|
|
||||||
if (call.answered_at) {
|
|
||||||
metrics.by_status.answered++;
|
|
||||||
const duration = call.ended_at - call.answered_at;
|
|
||||||
metrics.total_duration += duration;
|
|
||||||
|
|
||||||
// Duration distribution
|
|
||||||
if (duration <= 60) {
|
|
||||||
metrics.duration_distribution[0].count++;
|
|
||||||
} else if (duration <= 300) {
|
|
||||||
metrics.duration_distribution[1].count++;
|
|
||||||
} else if (duration <= 900) {
|
|
||||||
metrics.duration_distribution[2].count++;
|
|
||||||
} else if (duration <= 1800) {
|
|
||||||
metrics.duration_distribution[3].count++;
|
|
||||||
} else {
|
|
||||||
metrics.duration_distribution[4].count++;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Track user performance
|
|
||||||
if (call.user) {
|
|
||||||
const userId = call.user.id;
|
|
||||||
if (!metrics.by_users[userId]) {
|
|
||||||
metrics.by_users[userId] = {
|
|
||||||
id: userId,
|
|
||||||
name: call.user.name,
|
|
||||||
total: 0,
|
|
||||||
answered: 0,
|
|
||||||
missed: 0,
|
|
||||||
total_duration: 0,
|
|
||||||
average_duration: 0,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
metrics.by_users[userId].total++;
|
|
||||||
metrics.by_users[userId].answered++;
|
|
||||||
metrics.by_users[userId].total_duration += duration;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
metrics.by_status.missed++;
|
|
||||||
if (call.missed_call_reason) {
|
|
||||||
metrics.by_missed_reason[call.missed_call_reason] =
|
|
||||||
(metrics.by_missed_reason[call.missed_call_reason] || 0) + 1;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Track missed calls by user
|
|
||||||
if (call.user) {
|
|
||||||
const userId = call.user.id;
|
|
||||||
if (!metrics.by_users[userId]) {
|
|
||||||
metrics.by_users[userId] = {
|
|
||||||
id: userId,
|
|
||||||
name: call.user.name,
|
|
||||||
total: 0,
|
|
||||||
answered: 0,
|
|
||||||
missed: 0,
|
|
||||||
total_duration: 0,
|
|
||||||
average_duration: 0,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
metrics.by_users[userId].total++;
|
|
||||||
metrics.by_users[userId].missed++;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Group by date for daily data
|
|
||||||
if (!dailyCallsMap.has(dayKey)) {
|
|
||||||
dailyCallsMap.set(dayKey, { date: dayKey, inbound: 0, outbound: 0 });
|
|
||||||
}
|
|
||||||
dailyCallsMap.get(dayKey)[call.direction]++;
|
|
||||||
} catch (err) {
|
|
||||||
console.error('Error processing call:', err, call);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Calculate average durations for users
|
|
||||||
Object.values(metrics.by_users).forEach((user) => {
|
|
||||||
if (user.answered > 0) {
|
|
||||||
user.average_duration = Math.round(user.total_duration / user.answered);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Calculate global average duration
|
|
||||||
if (metrics.by_status.answered > 0) {
|
|
||||||
metrics.average_duration = Math.round(
|
|
||||||
metrics.total_duration / metrics.by_status.answered
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert daily data map to sorted array
|
|
||||||
metrics.daily_data = Array.from(dailyCallsMap.values()).sort((a, b) =>
|
|
||||||
a.date.localeCompare(b.date)
|
|
||||||
);
|
|
||||||
|
|
||||||
delete metrics.total_duration;
|
|
||||||
|
|
||||||
console.log('Processed metrics:', {
|
|
||||||
total: metrics.total,
|
|
||||||
by_direction: metrics.by_direction,
|
|
||||||
by_status: metrics.by_status,
|
|
||||||
daily_data_count: metrics.daily_data.length
|
|
||||||
});
|
|
||||||
|
|
||||||
return metrics;
|
|
||||||
}
|
|
||||||
|
|
||||||
async storeHistoricalDay(date, data) {
|
|
||||||
if (!this.mongodb) return;
|
|
||||||
|
|
||||||
try {
|
|
||||||
const collection = this.mongodb.collection(this.options.collection);
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(this.ensureDate(date));
|
|
||||||
|
|
||||||
// Ensure consistent data structure with metrics nested in data field
|
|
||||||
const document = {
|
|
||||||
date: dayBounds.start,
|
|
||||||
data: {
|
|
||||||
total: data.total,
|
|
||||||
by_direction: data.by_direction,
|
|
||||||
by_status: data.by_status,
|
|
||||||
by_missed_reason: data.by_missed_reason,
|
|
||||||
by_hour: data.by_hour,
|
|
||||||
by_users: data.by_users,
|
|
||||||
daily_data: data.daily_data,
|
|
||||||
duration_distribution: data.duration_distribution,
|
|
||||||
average_duration: data.average_duration
|
|
||||||
},
|
|
||||||
updatedAt: new Date()
|
|
||||||
};
|
|
||||||
|
|
||||||
await collection.updateOne(
|
|
||||||
{ date: dayBounds.start },
|
|
||||||
{ $set: document },
|
|
||||||
{ upsert: true }
|
|
||||||
);
|
|
||||||
} catch (error) {
|
|
||||||
console.error("Error storing historical day:", error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
import axios from "axios";
|
|
||||||
import { Buffer } from "buffer";
|
|
||||||
import { BaseService } from "../base/BaseService.js";
|
|
||||||
import { AircallDataManager } from "./AircallDataManager.js";
|
|
||||||
|
|
||||||
export class AircallService extends BaseService {
|
|
||||||
constructor(config) {
|
|
||||||
super(config);
|
|
||||||
this.baseUrl = "https://api.aircall.io/v1";
|
|
||||||
console.log('Initializing Aircall service with credentials:', {
|
|
||||||
apiId: config.apiId ? 'present' : 'missing',
|
|
||||||
apiToken: config.apiToken ? 'present' : 'missing'
|
|
||||||
});
|
|
||||||
this.auth = Buffer.from(`${config.apiId}:${config.apiToken}`).toString(
|
|
||||||
"base64"
|
|
||||||
);
|
|
||||||
this.dataManager = new AircallDataManager(
|
|
||||||
this.mongodb,
|
|
||||||
this.redis,
|
|
||||||
this.timeManager
|
|
||||||
);
|
|
||||||
|
|
||||||
if (!config.apiId || !config.apiToken) {
|
|
||||||
throw new Error("Aircall API credentials are required");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getMetrics(timeRange) {
|
|
||||||
const dateRange = await this.timeManager.getDateRange(timeRange);
|
|
||||||
console.log('Fetching metrics for date range:', {
|
|
||||||
start: dateRange.start.toISOString(),
|
|
||||||
end: dateRange.end.toISOString()
|
|
||||||
});
|
|
||||||
|
|
||||||
return this.dataManager.getData(dateRange, async (range) => {
|
|
||||||
const calls = await this.fetchAllCalls(range.start, range.end);
|
|
||||||
console.log('Fetched calls:', {
|
|
||||||
count: calls.length,
|
|
||||||
sample: calls.length > 0 ? calls[0] : null
|
|
||||||
});
|
|
||||||
return calls;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async fetchAllCalls(start, end) {
|
|
||||||
try {
|
|
||||||
let allCalls = [];
|
|
||||||
let currentPage = 1;
|
|
||||||
let hasMore = true;
|
|
||||||
let totalPages = null;
|
|
||||||
|
|
||||||
while (hasMore) {
|
|
||||||
const response = await this.makeRequest("/calls", {
|
|
||||||
from: Math.floor(start.getTime() / 1000),
|
|
||||||
to: Math.floor(end.getTime() / 1000),
|
|
||||||
order: "asc",
|
|
||||||
page: currentPage,
|
|
||||||
per_page: 50,
|
|
||||||
});
|
|
||||||
|
|
||||||
console.log('API Response:', {
|
|
||||||
page: currentPage,
|
|
||||||
totalPages: response.meta.total_pages,
|
|
||||||
callsCount: response.calls?.length,
|
|
||||||
params: {
|
|
||||||
from: Math.floor(start.getTime() / 1000),
|
|
||||||
to: Math.floor(end.getTime() / 1000)
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
if (!response.calls) {
|
|
||||||
throw new Error("Invalid API response format");
|
|
||||||
}
|
|
||||||
|
|
||||||
allCalls = [...allCalls, ...response.calls];
|
|
||||||
hasMore = response.meta.next_page_link !== null;
|
|
||||||
totalPages = response.meta.total_pages;
|
|
||||||
currentPage++;
|
|
||||||
|
|
||||||
if (hasMore) {
|
|
||||||
// Rate limiting pause
|
|
||||||
await new Promise((resolve) => setTimeout(resolve, 1));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return allCalls;
|
|
||||||
} catch (error) {
|
|
||||||
console.error("Error fetching all calls:", error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async makeRequest(endpoint, params = {}) {
|
|
||||||
try {
|
|
||||||
console.log('Making API request:', {
|
|
||||||
endpoint,
|
|
||||||
params
|
|
||||||
});
|
|
||||||
const response = await axios.get(`${this.baseUrl}${endpoint}`, {
|
|
||||||
headers: {
|
|
||||||
Authorization: `Basic ${this.auth}`,
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
},
|
|
||||||
params,
|
|
||||||
});
|
|
||||||
return response.data;
|
|
||||||
} catch (error) {
|
|
||||||
if (error.response?.status === 429) {
|
|
||||||
console.log("Rate limit reached, waiting before retry...");
|
|
||||||
await new Promise((resolve) => setTimeout(resolve, 5000));
|
|
||||||
return this.makeRequest(endpoint, params);
|
|
||||||
}
|
|
||||||
|
|
||||||
this.handleApiError(error, `Error making request to ${endpoint}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
validateApiResponse(response, context = "") {
|
|
||||||
if (!response || typeof response !== "object") {
|
|
||||||
throw new Error(`${context}: Invalid API response format`);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (response.error) {
|
|
||||||
throw new Error(`${context}: ${response.error}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
getPaginationInfo(meta) {
|
|
||||||
return {
|
|
||||||
currentPage: meta.current_page,
|
|
||||||
totalPages: meta.total_pages,
|
|
||||||
hasNextPage: meta.next_page_link !== null,
|
|
||||||
totalRecords: meta.total,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
import { createTimeManager } from '../../utils/timeUtils.js';
|
|
||||||
|
|
||||||
export class BaseService {
|
|
||||||
constructor(config) {
|
|
||||||
this.config = config;
|
|
||||||
this.mongodb = config.mongodb;
|
|
||||||
this.redis = config.redis;
|
|
||||||
this.logger = config.logger;
|
|
||||||
this.timeManager = createTimeManager(config.timezone, config.dayStartsAt);
|
|
||||||
}
|
|
||||||
|
|
||||||
handleApiError(error, context = '') {
|
|
||||||
this.logger.error(`API Error ${context}:`, {
|
|
||||||
message: error.message,
|
|
||||||
status: error.response?.status,
|
|
||||||
data: error.response?.data,
|
|
||||||
});
|
|
||||||
|
|
||||||
if (error.response) {
|
|
||||||
const status = error.response.status;
|
|
||||||
const message = error.response.data?.message || error.response.statusText;
|
|
||||||
|
|
||||||
if (status === 429) {
|
|
||||||
throw new Error('API rate limit exceeded. Please try again later.');
|
|
||||||
}
|
|
||||||
|
|
||||||
throw new Error(`API error (${status}): ${message}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,320 +0,0 @@
|
|||||||
export class DataManager {
|
|
||||||
constructor(mongodb, redis, timeManager, options) {
|
|
||||||
this.mongodb = mongodb;
|
|
||||||
this.redis = redis;
|
|
||||||
this.timeManager = timeManager;
|
|
||||||
this.options = options || {};
|
|
||||||
}
|
|
||||||
|
|
||||||
ensureDate(d) {
|
|
||||||
if (d instanceof Date) return d;
|
|
||||||
if (typeof d === 'string') return new Date(d);
|
|
||||||
if (typeof d === 'number') return new Date(d);
|
|
||||||
if (d && d.date) return new Date(d.date); // Handle MongoDB records
|
|
||||||
console.error('Invalid date value:', d);
|
|
||||||
return new Date(); // fallback to current date
|
|
||||||
}
|
|
||||||
|
|
||||||
async getData(dateRange, fetchFn) {
|
|
||||||
try {
|
|
||||||
// Get historical data from MongoDB
|
|
||||||
const historicalData = await this.getHistoricalDays(dateRange.start, dateRange.end);
|
|
||||||
|
|
||||||
// Find any missing date ranges
|
|
||||||
const missingRanges = this.findMissingDateRanges(dateRange.start, dateRange.end, historicalData);
|
|
||||||
|
|
||||||
// Fetch missing data
|
|
||||||
for (const range of missingRanges) {
|
|
||||||
const data = await fetchFn(range);
|
|
||||||
await this.storeHistoricalPeriod(range.start, range.end, data);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get updated historical data
|
|
||||||
const updatedData = await this.getHistoricalDays(dateRange.start, dateRange.end);
|
|
||||||
|
|
||||||
// Handle both nested and flat data structures
|
|
||||||
if (updatedData && updatedData.length > 0) {
|
|
||||||
// Process each record and combine them
|
|
||||||
const processedData = updatedData.map(record => {
|
|
||||||
if (record.data) {
|
|
||||||
return record.data;
|
|
||||||
}
|
|
||||||
if (record.total !== undefined) {
|
|
||||||
return {
|
|
||||||
total: record.total,
|
|
||||||
by_direction: record.by_direction,
|
|
||||||
by_status: record.by_status,
|
|
||||||
by_missed_reason: record.by_missed_reason,
|
|
||||||
by_hour: record.by_hour,
|
|
||||||
by_users: record.by_users,
|
|
||||||
daily_data: record.daily_data,
|
|
||||||
duration_distribution: record.duration_distribution,
|
|
||||||
average_duration: record.average_duration
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}).filter(Boolean);
|
|
||||||
|
|
||||||
// Combine the data
|
|
||||||
if (processedData.length > 0) {
|
|
||||||
return this.combineMetrics(processedData);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Otherwise process as raw call data
|
|
||||||
return this.processCallData(updatedData);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getData:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
findMissingDateRanges(start, end, existingDates) {
|
|
||||||
const missingRanges = [];
|
|
||||||
const existingDatesSet = new Set(
|
|
||||||
existingDates.map((d) => {
|
|
||||||
// Handle both nested and flat data structures
|
|
||||||
const date = d.date ? d.date : d;
|
|
||||||
return this.ensureDate(date).toISOString().split("T")[0];
|
|
||||||
})
|
|
||||||
);
|
|
||||||
|
|
||||||
let current = new Date(start);
|
|
||||||
const endDate = new Date(end);
|
|
||||||
|
|
||||||
while (current < endDate) {
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(current);
|
|
||||||
const dayKey = dayBounds.start.toISOString().split("T")[0];
|
|
||||||
|
|
||||||
if (!existingDatesSet.has(dayKey)) {
|
|
||||||
// Found a missing day
|
|
||||||
const missingStart = new Date(dayBounds.start);
|
|
||||||
const missingEnd = new Date(dayBounds.end);
|
|
||||||
|
|
||||||
missingRanges.push({
|
|
||||||
start: missingStart,
|
|
||||||
end: missingEnd,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Move to the next day using timeManager to ensure proper business day boundaries
|
|
||||||
current = new Date(dayBounds.end.getTime() + 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
return missingRanges;
|
|
||||||
}
|
|
||||||
|
|
||||||
async getCurrentDay(fetchFn) {
|
|
||||||
const now = new Date();
|
|
||||||
const todayBounds = this.timeManager.getDayBounds(now);
|
|
||||||
const todayKey = this.timeManager.formatDate(todayBounds.start);
|
|
||||||
const cacheKey = `${this.options.collection}:current_day:${todayKey}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Check cache first
|
|
||||||
if (this.redis?.isOpen) {
|
|
||||||
const cached = await this.redis.get(cacheKey);
|
|
||||||
if (cached) {
|
|
||||||
const parsedCache = JSON.parse(cached);
|
|
||||||
if (parsedCache.total !== undefined) {
|
|
||||||
// Use timeManager to check if the cached data is for today
|
|
||||||
const cachedDate = new Date(parsedCache.daily_data[0].date);
|
|
||||||
const isToday = this.timeManager.isToday(cachedDate);
|
|
||||||
|
|
||||||
if (isToday) {
|
|
||||||
return parsedCache;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get safe end time that's never in the future
|
|
||||||
const safeEnd = this.timeManager.getCurrentBusinessDayEnd();
|
|
||||||
|
|
||||||
// Fetch and process current day data with safe end time
|
|
||||||
const data = await fetchFn({
|
|
||||||
start: todayBounds.start,
|
|
||||||
end: safeEnd
|
|
||||||
});
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cache the data with a shorter TTL for today's data
|
|
||||||
if (this.redis?.isOpen) {
|
|
||||||
const ttl = Math.min(
|
|
||||||
this.options.redisTTL,
|
|
||||||
60 * 5 // 5 minutes max for today's data
|
|
||||||
);
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(data), {
|
|
||||||
EX: ttl,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getCurrentDay:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
getDayCount(start, end) {
|
|
||||||
// Calculate full days between dates using timeManager
|
|
||||||
const startDay = this.timeManager.getDayBounds(start);
|
|
||||||
const endDay = this.timeManager.getDayBounds(end);
|
|
||||||
return Math.ceil((endDay.end - startDay.start) / (24 * 60 * 60 * 1000));
|
|
||||||
}
|
|
||||||
|
|
||||||
async fetchMissingDays(start, end, existingData, fetchFn) {
|
|
||||||
const existingDates = new Set(
|
|
||||||
existingData.map((d) => this.timeManager.formatDate(d.date))
|
|
||||||
);
|
|
||||||
const missingData = [];
|
|
||||||
|
|
||||||
let currentDate = new Date(start);
|
|
||||||
while (currentDate < end) {
|
|
||||||
const dayBounds = this.timeManager.getDayBounds(currentDate);
|
|
||||||
const dateString = this.timeManager.formatDate(dayBounds.start);
|
|
||||||
|
|
||||||
if (!existingDates.has(dateString)) {
|
|
||||||
const data = await fetchFn({
|
|
||||||
start: dayBounds.start,
|
|
||||||
end: dayBounds.end,
|
|
||||||
});
|
|
||||||
|
|
||||||
await this.storeHistoricalDay(dayBounds.start, data);
|
|
||||||
missingData.push(data);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Move to next day using timeManager to ensure proper business day boundaries
|
|
||||||
currentDate = new Date(dayBounds.end.getTime() + 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
return missingData;
|
|
||||||
}
|
|
||||||
|
|
||||||
async getHistoricalDays(start, end) {
|
|
||||||
try {
|
|
||||||
if (!this.mongodb) return [];
|
|
||||||
|
|
||||||
const collection = this.mongodb.collection(this.options.collection);
|
|
||||||
const startDay = this.timeManager.getDayBounds(start);
|
|
||||||
const endDay = this.timeManager.getDayBounds(end);
|
|
||||||
|
|
||||||
const records = await collection
|
|
||||||
.find({
|
|
||||||
date: {
|
|
||||||
$gte: startDay.start,
|
|
||||||
$lt: endDay.start,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
.sort({ date: 1 })
|
|
||||||
.toArray();
|
|
||||||
|
|
||||||
return records;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error getting historical days:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
combineMetrics(metricsArray) {
|
|
||||||
if (!metricsArray || metricsArray.length === 0) return null;
|
|
||||||
if (metricsArray.length === 1) return metricsArray[0];
|
|
||||||
|
|
||||||
const combined = {
|
|
||||||
total: 0,
|
|
||||||
by_direction: { inbound: 0, outbound: 0 },
|
|
||||||
by_status: { answered: 0, missed: 0 },
|
|
||||||
by_missed_reason: {},
|
|
||||||
by_hour: Array(24).fill(0),
|
|
||||||
by_users: {},
|
|
||||||
daily_data: [],
|
|
||||||
duration_distribution: [
|
|
||||||
{ range: '0-1m', count: 0 },
|
|
||||||
{ range: '1-5m', count: 0 },
|
|
||||||
{ range: '5-15m', count: 0 },
|
|
||||||
{ range: '15-30m', count: 0 },
|
|
||||||
{ range: '30m+', count: 0 }
|
|
||||||
],
|
|
||||||
average_duration: 0
|
|
||||||
};
|
|
||||||
|
|
||||||
let totalAnswered = 0;
|
|
||||||
let totalDuration = 0;
|
|
||||||
|
|
||||||
metricsArray.forEach(metrics => {
|
|
||||||
// Sum basic metrics
|
|
||||||
combined.total += metrics.total;
|
|
||||||
combined.by_direction.inbound += metrics.by_direction.inbound;
|
|
||||||
combined.by_direction.outbound += metrics.by_direction.outbound;
|
|
||||||
combined.by_status.answered += metrics.by_status.answered;
|
|
||||||
combined.by_status.missed += metrics.by_status.missed;
|
|
||||||
|
|
||||||
// Combine missed reasons
|
|
||||||
Object.entries(metrics.by_missed_reason).forEach(([reason, count]) => {
|
|
||||||
combined.by_missed_reason[reason] = (combined.by_missed_reason[reason] || 0) + count;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Sum hourly data
|
|
||||||
metrics.by_hour.forEach((count, hour) => {
|
|
||||||
combined.by_hour[hour] += count;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Combine user data
|
|
||||||
Object.entries(metrics.by_users).forEach(([userId, userData]) => {
|
|
||||||
if (!combined.by_users[userId]) {
|
|
||||||
combined.by_users[userId] = {
|
|
||||||
id: userData.id,
|
|
||||||
name: userData.name,
|
|
||||||
total: 0,
|
|
||||||
answered: 0,
|
|
||||||
missed: 0,
|
|
||||||
total_duration: 0,
|
|
||||||
average_duration: 0
|
|
||||||
};
|
|
||||||
}
|
|
||||||
combined.by_users[userId].total += userData.total;
|
|
||||||
combined.by_users[userId].answered += userData.answered;
|
|
||||||
combined.by_users[userId].missed += userData.missed;
|
|
||||||
combined.by_users[userId].total_duration += userData.total_duration || 0;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Combine duration distribution
|
|
||||||
metrics.duration_distribution.forEach((dist, index) => {
|
|
||||||
combined.duration_distribution[index].count += dist.count;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Accumulate for average duration calculation
|
|
||||||
if (metrics.average_duration && metrics.by_status.answered) {
|
|
||||||
totalDuration += metrics.average_duration * metrics.by_status.answered;
|
|
||||||
totalAnswered += metrics.by_status.answered;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Merge daily data
|
|
||||||
if (metrics.daily_data) {
|
|
||||||
combined.daily_data.push(...metrics.daily_data);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Calculate final average duration
|
|
||||||
if (totalAnswered > 0) {
|
|
||||||
combined.average_duration = Math.round(totalDuration / totalAnswered);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Calculate user averages
|
|
||||||
Object.values(combined.by_users).forEach(user => {
|
|
||||||
if (user.answered > 0) {
|
|
||||||
user.average_duration = Math.round(user.total_duration / user.answered);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Sort and deduplicate daily data
|
|
||||||
combined.daily_data = Array.from(
|
|
||||||
new Map(combined.daily_data.map(item => [item.date, item])).values()
|
|
||||||
).sort((a, b) => a.date.localeCompare(b.date));
|
|
||||||
|
|
||||||
return combined;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
import { MongoClient } from 'mongodb';
|
|
||||||
|
|
||||||
const MONGODB_URI = process.env.MONGODB_URI || 'mongodb://localhost:27017/dashboard';
|
|
||||||
const DB_NAME = process.env.MONGODB_DB || 'dashboard';
|
|
||||||
|
|
||||||
export async function connectMongoDB() {
|
|
||||||
try {
|
|
||||||
const client = await MongoClient.connect(MONGODB_URI);
|
|
||||||
console.log('Connected to MongoDB');
|
|
||||||
return client.db(DB_NAME);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('MongoDB connection error:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
import winston from 'winston';
|
|
||||||
import path from 'path';
|
|
||||||
import { fileURLToPath } from 'url';
|
|
||||||
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
|
||||||
const __dirname = path.dirname(__filename);
|
|
||||||
|
|
||||||
export function createLogger(service) {
|
|
||||||
// Create logs directory relative to the project root (two levels up from utils)
|
|
||||||
const logsDir = path.join(__dirname, '../../logs');
|
|
||||||
|
|
||||||
return winston.createLogger({
|
|
||||||
level: process.env.LOG_LEVEL || 'info',
|
|
||||||
format: winston.format.combine(
|
|
||||||
winston.format.timestamp(),
|
|
||||||
winston.format.json()
|
|
||||||
),
|
|
||||||
defaultMeta: { service },
|
|
||||||
transports: [
|
|
||||||
// Write all logs to console
|
|
||||||
new winston.transports.Console({
|
|
||||||
format: winston.format.combine(
|
|
||||||
winston.format.colorize(),
|
|
||||||
winston.format.simple()
|
|
||||||
)
|
|
||||||
}),
|
|
||||||
// Write all logs to service-specific files
|
|
||||||
new winston.transports.File({
|
|
||||||
filename: path.join(logsDir, `${service}-error.log`),
|
|
||||||
level: 'error'
|
|
||||||
}),
|
|
||||||
new winston.transports.File({
|
|
||||||
filename: path.join(logsDir, `${service}-combined.log`)
|
|
||||||
})
|
|
||||||
]
|
|
||||||
});
|
|
||||||
}
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
import { createClient } from 'redis';
|
|
||||||
|
|
||||||
const REDIS_URL = process.env.REDIS_URL || 'redis://localhost:6379';
|
|
||||||
|
|
||||||
export async function createRedisClient() {
|
|
||||||
try {
|
|
||||||
const client = createClient({
|
|
||||||
url: REDIS_URL
|
|
||||||
});
|
|
||||||
|
|
||||||
await client.connect();
|
|
||||||
console.log('Connected to Redis');
|
|
||||||
|
|
||||||
client.on('error', (err) => {
|
|
||||||
console.error('Redis error:', err);
|
|
||||||
});
|
|
||||||
|
|
||||||
return client;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Redis connection error:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,262 +0,0 @@
|
|||||||
class TimeManager {
|
|
||||||
static ALLOWED_RANGES = ['today', 'yesterday', 'last2days', 'last7days', 'last30days', 'last90days',
|
|
||||||
'previous7days', 'previous30days', 'previous90days'];
|
|
||||||
|
|
||||||
constructor(timezone = 'America/New_York', dayStartsAt = 1) {
|
|
||||||
this.timezone = timezone;
|
|
||||||
this.dayStartsAt = dayStartsAt;
|
|
||||||
}
|
|
||||||
|
|
||||||
getDayBounds(date) {
|
|
||||||
try {
|
|
||||||
const now = new Date();
|
|
||||||
const targetDate = new Date(date);
|
|
||||||
|
|
||||||
// For today
|
|
||||||
if (
|
|
||||||
targetDate.getUTCFullYear() === now.getUTCFullYear() &&
|
|
||||||
targetDate.getUTCMonth() === now.getUTCMonth() &&
|
|
||||||
targetDate.getUTCDate() === now.getUTCDate()
|
|
||||||
) {
|
|
||||||
// If current time is before day start (1 AM ET / 6 AM UTC),
|
|
||||||
// use previous day's start until now
|
|
||||||
const todayStart = new Date(Date.UTC(
|
|
||||||
now.getUTCFullYear(),
|
|
||||||
now.getUTCMonth(),
|
|
||||||
now.getUTCDate(),
|
|
||||||
this.dayStartsAt + 5,
|
|
||||||
0,
|
|
||||||
0,
|
|
||||||
0
|
|
||||||
));
|
|
||||||
|
|
||||||
if (now < todayStart) {
|
|
||||||
const yesterdayStart = new Date(todayStart);
|
|
||||||
yesterdayStart.setUTCDate(yesterdayStart.getUTCDate() - 1);
|
|
||||||
return { start: yesterdayStart, end: now };
|
|
||||||
}
|
|
||||||
|
|
||||||
return { start: todayStart, end: now };
|
|
||||||
}
|
|
||||||
|
|
||||||
// For past days, use full 24-hour period
|
|
||||||
const normalizedDate = new Date(Date.UTC(
|
|
||||||
targetDate.getUTCFullYear(),
|
|
||||||
targetDate.getUTCMonth(),
|
|
||||||
targetDate.getUTCDate()
|
|
||||||
));
|
|
||||||
|
|
||||||
const dayStart = new Date(normalizedDate);
|
|
||||||
dayStart.setUTCHours(this.dayStartsAt + 5, 0, 0, 0);
|
|
||||||
|
|
||||||
const dayEnd = new Date(dayStart);
|
|
||||||
dayEnd.setUTCDate(dayEnd.getUTCDate() + 1);
|
|
||||||
|
|
||||||
return { start: dayStart, end: dayEnd };
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getDayBounds:', error);
|
|
||||||
throw new Error(`Failed to calculate day bounds: ${error.message}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
getDateRange(period) {
|
|
||||||
try {
|
|
||||||
const now = new Date();
|
|
||||||
const todayBounds = this.getDayBounds(now);
|
|
||||||
const end = new Date();
|
|
||||||
|
|
||||||
switch (period) {
|
|
||||||
case 'today':
|
|
||||||
return {
|
|
||||||
start: todayBounds.start,
|
|
||||||
end
|
|
||||||
};
|
|
||||||
case 'yesterday': {
|
|
||||||
const yesterday = new Date(now);
|
|
||||||
yesterday.setDate(yesterday.getDate() - 1);
|
|
||||||
return this.getDayBounds(yesterday);
|
|
||||||
}
|
|
||||||
case 'last2days': {
|
|
||||||
const twoDaysAgo = new Date(now);
|
|
||||||
twoDaysAgo.setDate(twoDaysAgo.getDate() - 2);
|
|
||||||
return this.getDayBounds(twoDaysAgo);
|
|
||||||
}
|
|
||||||
case 'last7days': {
|
|
||||||
const start = new Date(now);
|
|
||||||
start.setDate(start.getDate() - 6);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'previous7days': {
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 7);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 6);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'last30days': {
|
|
||||||
const start = new Date(now);
|
|
||||||
start.setDate(start.getDate() - 29);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'previous30days': {
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 30);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 29);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'last90days': {
|
|
||||||
const start = new Date(now);
|
|
||||||
start.setDate(start.getDate() - 89);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'previous90days': {
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 90);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 89);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
throw new Error(`Unsupported time period: ${period}`);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getDateRange:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
getPreviousPeriod(period) {
|
|
||||||
try {
|
|
||||||
const now = new Date();
|
|
||||||
|
|
||||||
switch (period) {
|
|
||||||
case 'today':
|
|
||||||
return 'yesterday';
|
|
||||||
case 'yesterday': {
|
|
||||||
// Return bounds for 2 days ago
|
|
||||||
const twoDaysAgo = new Date(now);
|
|
||||||
twoDaysAgo.setDate(twoDaysAgo.getDate() - 2);
|
|
||||||
return this.getDayBounds(twoDaysAgo);
|
|
||||||
}
|
|
||||||
case 'last7days': {
|
|
||||||
// Return bounds for previous 7 days
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 7);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 7);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'last30days': {
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 30);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 30);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
case 'last90days': {
|
|
||||||
const end = new Date(now);
|
|
||||||
end.setDate(end.getDate() - 90);
|
|
||||||
const start = new Date(end);
|
|
||||||
start.setDate(start.getDate() - 90);
|
|
||||||
return {
|
|
||||||
start: this.getDayBounds(start).start,
|
|
||||||
end: this.getDayBounds(end).end
|
|
||||||
};
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
throw new Error(`Unsupported time period: ${period}`);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getPreviousPeriod:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
getCurrentBusinessDayEnd() {
|
|
||||||
try {
|
|
||||||
const now = new Date();
|
|
||||||
const todayBounds = this.getDayBounds(now);
|
|
||||||
|
|
||||||
// If current time is before day start (1 AM ET / 6 AM UTC),
|
|
||||||
// then we're still in yesterday's business day
|
|
||||||
const todayStart = new Date(Date.UTC(
|
|
||||||
now.getUTCFullYear(),
|
|
||||||
now.getUTCMonth(),
|
|
||||||
now.getUTCDate(),
|
|
||||||
this.dayStartsAt + 5,
|
|
||||||
0,
|
|
||||||
0,
|
|
||||||
0
|
|
||||||
));
|
|
||||||
|
|
||||||
if (now < todayStart) {
|
|
||||||
const yesterdayBounds = this.getDayBounds(new Date(now.getTime() - 24 * 60 * 60 * 1000));
|
|
||||||
return yesterdayBounds.end;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Return the earlier of current time or today's end
|
|
||||||
return now < todayBounds.end ? now : todayBounds.end;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getCurrentBusinessDayEnd:', error);
|
|
||||||
return new Date();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
isValidTimeRange(timeRange) {
|
|
||||||
return TimeManager.ALLOWED_RANGES.includes(timeRange);
|
|
||||||
}
|
|
||||||
|
|
||||||
isToday(date) {
|
|
||||||
const now = new Date();
|
|
||||||
const targetDate = new Date(date);
|
|
||||||
return (
|
|
||||||
targetDate.getUTCFullYear() === now.getUTCFullYear() &&
|
|
||||||
targetDate.getUTCMonth() === now.getUTCMonth() &&
|
|
||||||
targetDate.getUTCDate() === now.getUTCDate()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
formatDate(date) {
|
|
||||||
try {
|
|
||||||
return date.toLocaleString('en-US', {
|
|
||||||
timeZone: this.timezone,
|
|
||||||
year: 'numeric',
|
|
||||||
month: '2-digit',
|
|
||||||
day: '2-digit',
|
|
||||||
hour: '2-digit',
|
|
||||||
minute: '2-digit',
|
|
||||||
second: '2-digit'
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error formatting date:', error);
|
|
||||||
return date.toISOString();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export const createTimeManager = (timezone, dayStartsAt) => new TimeManager(timezone, dayStartsAt);
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
# Server Configuration
|
|
||||||
NODE_ENV=development
|
|
||||||
PORT=3003
|
|
||||||
|
|
||||||
# Authentication
|
|
||||||
JWT_SECRET=your-secret-key-here
|
|
||||||
DASHBOARD_PASSWORD=your-dashboard-password-here
|
|
||||||
|
|
||||||
# Cookie Settings
|
|
||||||
COOKIE_DOMAIN=localhost # In production: .kent.pw
|
|
||||||
@@ -1,203 +0,0 @@
|
|||||||
// auth-server/index.js
|
|
||||||
const path = require('path');
|
|
||||||
require('dotenv').config({ path: path.join(__dirname, '.env') });
|
|
||||||
const express = require('express');
|
|
||||||
const cors = require('cors');
|
|
||||||
const cookieParser = require('cookie-parser');
|
|
||||||
const jwt = require('jsonwebtoken');
|
|
||||||
|
|
||||||
// Debug environment variables
|
|
||||||
console.log('Environment variables loaded from:', path.join(__dirname, '.env'));
|
|
||||||
console.log('Current directory:', __dirname);
|
|
||||||
console.log('Available env vars:', Object.keys(process.env));
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const PORT = process.env.PORT || 3003;
|
|
||||||
const JWT_SECRET = process.env.JWT_SECRET;
|
|
||||||
const DASHBOARD_PASSWORD = process.env.DASHBOARD_PASSWORD;
|
|
||||||
|
|
||||||
// Validate required environment variables
|
|
||||||
if (!JWT_SECRET || !DASHBOARD_PASSWORD) {
|
|
||||||
console.error('Missing required environment variables:');
|
|
||||||
if (!JWT_SECRET) console.error('- JWT_SECRET');
|
|
||||||
if (!DASHBOARD_PASSWORD) console.error('- DASHBOARD_PASSWORD');
|
|
||||||
process.exit(1);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Middleware
|
|
||||||
app.use(express.json());
|
|
||||||
app.use(cookieParser());
|
|
||||||
|
|
||||||
// Configure CORS
|
|
||||||
const corsOptions = {
|
|
||||||
origin: function(origin, callback) {
|
|
||||||
const allowedOrigins = [
|
|
||||||
'http://localhost:3000',
|
|
||||||
'https://tools.acherryontop.com'
|
|
||||||
];
|
|
||||||
|
|
||||||
console.log('CORS check for origin:', origin);
|
|
||||||
|
|
||||||
// Allow local network IPs (192.168.1.xxx)
|
|
||||||
if (origin && origin.match(/^http:\/\/192\.168\.1\.\d{1,3}(:\d+)?$/)) {
|
|
||||||
callback(null, true);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check if origin is in allowed list
|
|
||||||
if (!origin || allowedOrigins.indexOf(origin) !== -1) {
|
|
||||||
callback(null, true);
|
|
||||||
} else {
|
|
||||||
callback(new Error('Not allowed by CORS'));
|
|
||||||
}
|
|
||||||
},
|
|
||||||
credentials: true,
|
|
||||||
methods: ['GET', 'POST', 'OPTIONS'],
|
|
||||||
allowedHeaders: ['Content-Type', 'Authorization', 'Cookie', 'Accept'],
|
|
||||||
exposedHeaders: ['Set-Cookie']
|
|
||||||
};
|
|
||||||
|
|
||||||
app.use(cors(corsOptions));
|
|
||||||
app.options('*', cors(corsOptions));
|
|
||||||
|
|
||||||
// Debug logging
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
console.log(`${new Date().toISOString()} ${req.method} ${req.url}`);
|
|
||||||
console.log('Headers:', req.headers);
|
|
||||||
console.log('Cookies:', req.cookies);
|
|
||||||
next();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Health check endpoint
|
|
||||||
app.get('/health', (req, res) => {
|
|
||||||
res.json({
|
|
||||||
status: 'ok',
|
|
||||||
timestamp: new Date().toISOString()
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Auth endpoints
|
|
||||||
app.post('/login', (req, res) => {
|
|
||||||
console.log('Login attempt received');
|
|
||||||
console.log('Request body:', req.body);
|
|
||||||
console.log('Origin:', req.headers.origin);
|
|
||||||
|
|
||||||
const { password } = req.body;
|
|
||||||
|
|
||||||
if (!password) {
|
|
||||||
console.log('No password provided');
|
|
||||||
return res.status(400).json({
|
|
||||||
success: false,
|
|
||||||
message: 'Password is required'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Comparing passwords...');
|
|
||||||
console.log('Provided password length:', password.length);
|
|
||||||
console.log('Expected password length:', DASHBOARD_PASSWORD.length);
|
|
||||||
|
|
||||||
if (password === DASHBOARD_PASSWORD) {
|
|
||||||
console.log('Password matched');
|
|
||||||
const token = jwt.sign({ authorized: true }, JWT_SECRET, {
|
|
||||||
expiresIn: '24h'
|
|
||||||
});
|
|
||||||
|
|
||||||
// Determine if request is from local network
|
|
||||||
const isLocalNetwork = req.headers.origin?.includes('192.168.1.') || req.headers.origin?.includes('localhost');
|
|
||||||
|
|
||||||
const cookieOptions = {
|
|
||||||
httpOnly: true,
|
|
||||||
secure: !isLocalNetwork, // Only use secure for non-local requests
|
|
||||||
sameSite: isLocalNetwork ? 'lax' : 'none',
|
|
||||||
path: '/',
|
|
||||||
maxAge: 24 * 60 * 60 * 1000 // 24 hours
|
|
||||||
};
|
|
||||||
|
|
||||||
// Only set domain for production
|
|
||||||
if (!isLocalNetwork) {
|
|
||||||
cookieOptions.domain = '.kent.pw';
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Setting cookie with options:', cookieOptions);
|
|
||||||
res.cookie('token', token, cookieOptions);
|
|
||||||
|
|
||||||
console.log('Response headers:', res.getHeaders());
|
|
||||||
res.json({
|
|
||||||
success: true,
|
|
||||||
debug: {
|
|
||||||
origin: req.headers.origin,
|
|
||||||
cookieOptions
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
console.log('Password mismatch');
|
|
||||||
res.status(401).json({
|
|
||||||
success: false,
|
|
||||||
message: 'Invalid password'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Modify the check endpoint to log more info
|
|
||||||
app.get('/check', (req, res) => {
|
|
||||||
console.log('Auth check received');
|
|
||||||
console.log('All cookies:', req.cookies);
|
|
||||||
console.log('Headers:', req.headers);
|
|
||||||
|
|
||||||
const token = req.cookies.token;
|
|
||||||
|
|
||||||
if (!token) {
|
|
||||||
console.log('No token found in cookies');
|
|
||||||
return res.status(401).json({
|
|
||||||
authenticated: false,
|
|
||||||
error: 'no_token'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const decoded = jwt.verify(token, JWT_SECRET);
|
|
||||||
console.log('Token verified successfully:', decoded);
|
|
||||||
res.json({ authenticated: true });
|
|
||||||
} catch (err) {
|
|
||||||
console.log('Token verification failed:', err.message);
|
|
||||||
res.status(401).json({
|
|
||||||
authenticated: false,
|
|
||||||
error: 'invalid_token',
|
|
||||||
message: err.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
app.post('/logout', (req, res) => {
|
|
||||||
const isLocalNetwork = req.headers.origin?.includes('192.168.1.') || req.headers.origin?.includes('localhost');
|
|
||||||
const cookieOptions = {
|
|
||||||
httpOnly: true,
|
|
||||||
secure: !isLocalNetwork,
|
|
||||||
sameSite: isLocalNetwork ? 'lax' : 'none',
|
|
||||||
path: '/',
|
|
||||||
domain: isLocalNetwork ? undefined : '.kent.pw'
|
|
||||||
};
|
|
||||||
|
|
||||||
console.log('Clearing cookie with options:', cookieOptions);
|
|
||||||
res.clearCookie('token', cookieOptions);
|
|
||||||
res.json({ success: true });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error('Server error:', err);
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
message: 'Internal server error',
|
|
||||||
error: err.message
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(PORT, () => {
|
|
||||||
console.log(`Auth server running on port ${PORT}`);
|
|
||||||
console.log('Environment:', process.env.NODE_ENV);
|
|
||||||
console.log('CORS origins:', corsOptions.origin);
|
|
||||||
console.log('JWT_SECRET length:', JWT_SECRET?.length);
|
|
||||||
console.log('DASHBOARD_PASSWORD length:', DASHBOARD_PASSWORD?.length);
|
|
||||||
});
|
|
||||||
-1044
File diff suppressed because it is too large
Load Diff
@@ -1,22 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "auth-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"main": "index.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"description": "",
|
|
||||||
"dependencies": {
|
|
||||||
"cookie-parser": "^1.4.7",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"date-fns": "^4.1.0",
|
|
||||||
"date-fns-tz": "^3.2.0",
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"express": "^4.21.1",
|
|
||||||
"express-session": "^1.18.1",
|
|
||||||
"jsonwebtoken": "^9.0.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
/etc/nginx/sites-enabled/dashboard.conf
|
|
||||||
-2538
File diff suppressed because it is too large
Load Diff
@@ -1,21 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "google-analytics-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"description": "Google Analytics server for dashboard",
|
|
||||||
"main": "server.js",
|
|
||||||
"scripts": {
|
|
||||||
"start": "node server.js",
|
|
||||||
"dev": "nodemon server.js"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"@google-analytics/data": "^4.0.0",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.3.1",
|
|
||||||
"express": "^4.18.2",
|
|
||||||
"redis": "^4.6.11",
|
|
||||||
"winston": "^3.11.0"
|
|
||||||
},
|
|
||||||
"devDependencies": {
|
|
||||||
"nodemon": "^3.0.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,254 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const { BetaAnalyticsDataClient } = require('@google-analytics/data');
|
|
||||||
const router = express.Router();
|
|
||||||
const logger = require('../utils/logger');
|
|
||||||
|
|
||||||
// Initialize GA4 client
|
|
||||||
const analyticsClient = new BetaAnalyticsDataClient({
|
|
||||||
credentials: JSON.parse(process.env.GOOGLE_APPLICATION_CREDENTIALS_JSON)
|
|
||||||
});
|
|
||||||
|
|
||||||
const propertyId = process.env.GA_PROPERTY_ID;
|
|
||||||
|
|
||||||
// Cache durations
|
|
||||||
const CACHE_DURATIONS = {
|
|
||||||
REALTIME_BASIC: 60, // 1 minute
|
|
||||||
REALTIME_DETAILED: 300, // 5 minutes
|
|
||||||
BASIC_METRICS: 3600, // 1 hour
|
|
||||||
USER_BEHAVIOR: 3600 // 1 hour
|
|
||||||
};
|
|
||||||
|
|
||||||
// Basic metrics endpoint
|
|
||||||
router.get('/metrics', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { startDate = '7daysAgo' } = req.query;
|
|
||||||
const cacheKey = `analytics:basic_metrics:${startDate}`;
|
|
||||||
|
|
||||||
// Check Redis cache
|
|
||||||
const cachedData = await req.redisClient.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
logger.info('Returning cached basic metrics data');
|
|
||||||
return res.json({ success: true, data: JSON.parse(cachedData) });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch from GA4
|
|
||||||
const [response] = await analyticsClient.runReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dateRanges: [{ startDate, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'date' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'activeUsers' },
|
|
||||||
{ name: 'newUsers' },
|
|
||||||
{ name: 'averageSessionDuration' },
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'bounceRate' },
|
|
||||||
{ name: 'conversions' }
|
|
||||||
],
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await req.redisClient.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: CACHE_DURATIONS.BASIC_METRICS
|
|
||||||
});
|
|
||||||
|
|
||||||
res.json({ success: true, data: response });
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Error fetching basic metrics:', error);
|
|
||||||
res.status(500).json({ success: false, error: error.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Realtime basic data endpoint
|
|
||||||
router.get('/realtime/basic', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const cacheKey = 'analytics:realtime:basic';
|
|
||||||
|
|
||||||
// Check Redis cache
|
|
||||||
const cachedData = await req.redisClient.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
logger.info('Returning cached realtime basic data');
|
|
||||||
return res.json({ success: true, data: JSON.parse(cachedData) });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch active users
|
|
||||||
const [userResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch last 5 minutes
|
|
||||||
const [fiveMinResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
minuteRanges: [{ startMinutesAgo: 5, endMinutesAgo: 0 }]
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch time series data
|
|
||||||
const [timeSeriesResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dimensions: [{ name: 'minutesAgo' }],
|
|
||||||
metrics: [{ name: 'activeUsers' }]
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
userResponse,
|
|
||||||
fiveMinResponse,
|
|
||||||
timeSeriesResponse,
|
|
||||||
quotaInfo: {
|
|
||||||
projectHourly: userResponse.propertyQuota.tokensPerProjectPerHour,
|
|
||||||
daily: userResponse.propertyQuota.tokensPerDay,
|
|
||||||
serverErrors: userResponse.propertyQuota.serverErrorsPerProjectPerHour,
|
|
||||||
thresholdedRequests: userResponse.propertyQuota.potentiallyThresholdedRequestsPerHour
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await req.redisClient.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: CACHE_DURATIONS.REALTIME_BASIC
|
|
||||||
});
|
|
||||||
|
|
||||||
res.json({ success: true, data: response });
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Error fetching realtime basic data:', error);
|
|
||||||
res.status(500).json({ success: false, error: error.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Realtime detailed data endpoint
|
|
||||||
router.get('/realtime/detailed', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const cacheKey = 'analytics:realtime:detailed';
|
|
||||||
|
|
||||||
// Check Redis cache
|
|
||||||
const cachedData = await req.redisClient.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
logger.info('Returning cached realtime detailed data');
|
|
||||||
return res.json({ success: true, data: JSON.parse(cachedData) });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch current pages
|
|
||||||
const [pageResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dimensions: [{ name: 'unifiedScreenName' }],
|
|
||||||
metrics: [{ name: 'screenPageViews' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'screenPageViews' }, desc: true }],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch events
|
|
||||||
const [eventResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dimensions: [{ name: 'eventName' }],
|
|
||||||
metrics: [{ name: 'eventCount' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'eventCount' }, desc: true }],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch device categories
|
|
||||||
const [deviceResponse] = await analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dimensions: [{ name: 'deviceCategory' }],
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'activeUsers' }, desc: true }],
|
|
||||||
limit: 10,
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
pageResponse,
|
|
||||||
eventResponse,
|
|
||||||
sourceResponse: deviceResponse
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await req.redisClient.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: CACHE_DURATIONS.REALTIME_DETAILED
|
|
||||||
});
|
|
||||||
|
|
||||||
res.json({ success: true, data: response });
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Error fetching realtime detailed data:', error);
|
|
||||||
res.status(500).json({ success: false, error: error.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// User behavior endpoint
|
|
||||||
router.get('/user-behavior', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { timeRange = '30' } = req.query;
|
|
||||||
const cacheKey = `analytics:user_behavior:${timeRange}`;
|
|
||||||
|
|
||||||
// Check Redis cache
|
|
||||||
const cachedData = await req.redisClient.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
logger.info('Returning cached user behavior data');
|
|
||||||
return res.json({ success: true, data: JSON.parse(cachedData) });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch page data
|
|
||||||
const [pageResponse] = await analyticsClient.runReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'pagePath' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'averageSessionDuration' },
|
|
||||||
{ name: 'bounceRate' },
|
|
||||||
{ name: 'sessions' }
|
|
||||||
],
|
|
||||||
orderBy: [{
|
|
||||||
metric: { metricName: 'screenPageViews' },
|
|
||||||
desc: true
|
|
||||||
}],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch device data
|
|
||||||
const [deviceResponse] = await analyticsClient.runReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'deviceCategory' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'sessions' }
|
|
||||||
]
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch source data
|
|
||||||
const [sourceResponse] = await analyticsClient.runReport({
|
|
||||||
property: `properties/${propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'sessionSource' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'sessions' },
|
|
||||||
{ name: 'conversions' }
|
|
||||||
],
|
|
||||||
orderBy: [{
|
|
||||||
metric: { metricName: 'sessions' },
|
|
||||||
desc: true
|
|
||||||
}],
|
|
||||||
limit: 25,
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
pageResponse,
|
|
||||||
deviceResponse,
|
|
||||||
sourceResponse
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await req.redisClient.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: CACHE_DURATIONS.USER_BEHAVIOR
|
|
||||||
});
|
|
||||||
|
|
||||||
res.json({ success: true, data: response });
|
|
||||||
} catch (error) {
|
|
||||||
logger.error('Error fetching user behavior data:', error);
|
|
||||||
res.status(500).json({ success: false, error: error.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = router;
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const router = express.Router();
|
|
||||||
const analyticsService = require('../services/analytics.service');
|
|
||||||
|
|
||||||
// Basic metrics endpoint
|
|
||||||
router.get('/metrics', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { startDate = '7daysAgo' } = req.query;
|
|
||||||
console.log(`Fetching metrics with startDate: ${startDate}`);
|
|
||||||
|
|
||||||
const data = await analyticsService.getBasicMetrics(startDate);
|
|
||||||
res.json({ success: true, data });
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Metrics error:', {
|
|
||||||
startDate: req.query.startDate,
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
error: 'Failed to fetch metrics',
|
|
||||||
details: error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Realtime basic data endpoint
|
|
||||||
router.get('/realtime/basic', async (req, res) => {
|
|
||||||
try {
|
|
||||||
console.log('Fetching realtime basic data');
|
|
||||||
const data = await analyticsService.getRealTimeBasicData();
|
|
||||||
res.json({ success: true, data });
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Realtime basic error:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
error: 'Failed to fetch realtime basic data',
|
|
||||||
details: error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Realtime detailed data endpoint
|
|
||||||
router.get('/realtime/detailed', async (req, res) => {
|
|
||||||
try {
|
|
||||||
console.log('Fetching realtime detailed data');
|
|
||||||
const data = await analyticsService.getRealTimeDetailedData();
|
|
||||||
res.json({ success: true, data });
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Realtime detailed error:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
error: 'Failed to fetch realtime detailed data',
|
|
||||||
details: error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// User behavior endpoint
|
|
||||||
router.get('/user-behavior', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { timeRange = '30' } = req.query;
|
|
||||||
console.log(`Fetching user behavior with timeRange: ${timeRange}`);
|
|
||||||
|
|
||||||
const data = await analyticsService.getUserBehavior(timeRange);
|
|
||||||
res.json({ success: true, data });
|
|
||||||
} catch (error) {
|
|
||||||
console.error('User behavior error:', {
|
|
||||||
timeRange: req.query.timeRange,
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
success: false,
|
|
||||||
error: 'Failed to fetch user behavior data',
|
|
||||||
details: error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = router;
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const cors = require('cors');
|
|
||||||
const { createClient } = require('redis');
|
|
||||||
const analyticsRoutes = require('./routes/analytics.routes');
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.GOOGLE_ANALYTICS_PORT || 3007;
|
|
||||||
|
|
||||||
// Redis client setup
|
|
||||||
const redisClient = createClient({
|
|
||||||
url: process.env.REDIS_URL || 'redis://localhost:6379'
|
|
||||||
});
|
|
||||||
|
|
||||||
redisClient.on('error', (err) => console.error('Redis Client Error:', err));
|
|
||||||
redisClient.on('connect', () => console.log('Redis Client Connected'));
|
|
||||||
|
|
||||||
// Connect to Redis
|
|
||||||
(async () => {
|
|
||||||
try {
|
|
||||||
await redisClient.connect();
|
|
||||||
} catch (err) {
|
|
||||||
console.error('Redis connection error:', err);
|
|
||||||
}
|
|
||||||
})();
|
|
||||||
|
|
||||||
// Middleware
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Make Redis client available in requests
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
req.redisClient = redisClient;
|
|
||||||
next();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Routes
|
|
||||||
app.use('/api/analytics', analyticsRoutes);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error('Server error:', err);
|
|
||||||
res.status(err.status || 500).json({
|
|
||||||
success: false,
|
|
||||||
message: err.message || 'Internal server error',
|
|
||||||
error: process.env.NODE_ENV === 'production' ? err : {}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
|
||||||
console.log(`Google Analytics server running on port ${port}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle graceful shutdown
|
|
||||||
process.on('SIGTERM', async () => {
|
|
||||||
console.log('SIGTERM received. Shutting down gracefully...');
|
|
||||||
await redisClient.quit();
|
|
||||||
process.exit(0);
|
|
||||||
});
|
|
||||||
|
|
||||||
process.on('SIGINT', async () => {
|
|
||||||
console.log('SIGINT received. Shutting down gracefully...');
|
|
||||||
await redisClient.quit();
|
|
||||||
process.exit(0);
|
|
||||||
});
|
|
||||||
@@ -1,283 +0,0 @@
|
|||||||
const { BetaAnalyticsDataClient } = require('@google-analytics/data');
|
|
||||||
const { createClient } = require('redis');
|
|
||||||
|
|
||||||
class AnalyticsService {
|
|
||||||
constructor() {
|
|
||||||
// Initialize Redis client
|
|
||||||
this.redis = createClient({
|
|
||||||
url: process.env.REDIS_URL || 'redis://localhost:6379'
|
|
||||||
});
|
|
||||||
|
|
||||||
this.redis.on('error', err => console.error('Redis Client Error:', err));
|
|
||||||
this.redis.connect().catch(err => console.error('Redis connection error:', err));
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Initialize GA4 client
|
|
||||||
const credentials = process.env.GOOGLE_APPLICATION_CREDENTIALS_JSON;
|
|
||||||
this.analyticsClient = new BetaAnalyticsDataClient({
|
|
||||||
credentials: typeof credentials === 'string' ? JSON.parse(credentials) : credentials
|
|
||||||
});
|
|
||||||
|
|
||||||
this.propertyId = process.env.GA_PROPERTY_ID;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Failed to initialize GA4 client:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cache durations
|
|
||||||
CACHE_DURATIONS = {
|
|
||||||
REALTIME_BASIC: 60, // 1 minute
|
|
||||||
REALTIME_DETAILED: 300, // 5 minutes
|
|
||||||
BASIC_METRICS: 3600, // 1 hour
|
|
||||||
USER_BEHAVIOR: 3600 // 1 hour
|
|
||||||
};
|
|
||||||
|
|
||||||
async getBasicMetrics(startDate = '7daysAgo') {
|
|
||||||
const cacheKey = `analytics:basic_metrics:${startDate}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log('Analytics metrics found in Redis cache');
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch from GA4
|
|
||||||
console.log('Fetching fresh metrics data from GA4');
|
|
||||||
const [response] = await this.analyticsClient.runReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dateRanges: [{ startDate, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'date' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'activeUsers' },
|
|
||||||
{ name: 'newUsers' },
|
|
||||||
{ name: 'averageSessionDuration' },
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'bounceRate' },
|
|
||||||
{ name: 'conversions' }
|
|
||||||
],
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: this.CACHE_DURATIONS.BASIC_METRICS
|
|
||||||
});
|
|
||||||
|
|
||||||
return response;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error fetching analytics metrics:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getRealTimeBasicData() {
|
|
||||||
const cacheKey = 'analytics:realtime:basic';
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log('Realtime basic data found in Redis cache');
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Fetching fresh realtime data from GA4');
|
|
||||||
|
|
||||||
// Fetch active users
|
|
||||||
const [userResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch last 5 minutes
|
|
||||||
const [fiveMinResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
minuteRanges: [{ startMinutesAgo: 5, endMinutesAgo: 0 }]
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch time series data
|
|
||||||
const [timeSeriesResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dimensions: [{ name: 'minutesAgo' }],
|
|
||||||
metrics: [{ name: 'activeUsers' }]
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
userResponse,
|
|
||||||
fiveMinResponse,
|
|
||||||
timeSeriesResponse,
|
|
||||||
quotaInfo: {
|
|
||||||
projectHourly: userResponse.propertyQuota.tokensPerProjectPerHour,
|
|
||||||
daily: userResponse.propertyQuota.tokensPerDay,
|
|
||||||
serverErrors: userResponse.propertyQuota.serverErrorsPerProjectPerHour,
|
|
||||||
thresholdedRequests: userResponse.propertyQuota.potentiallyThresholdedRequestsPerHour
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: this.CACHE_DURATIONS.REALTIME_BASIC
|
|
||||||
});
|
|
||||||
|
|
||||||
return response;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error fetching realtime basic data:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getRealTimeDetailedData() {
|
|
||||||
const cacheKey = 'analytics:realtime:detailed';
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log('Realtime detailed data found in Redis cache');
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Fetching fresh realtime detailed data from GA4');
|
|
||||||
|
|
||||||
// Fetch current pages
|
|
||||||
const [pageResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dimensions: [{ name: 'unifiedScreenName' }],
|
|
||||||
metrics: [{ name: 'screenPageViews' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'screenPageViews' }, desc: true }],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch events
|
|
||||||
const [eventResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dimensions: [{ name: 'eventName' }],
|
|
||||||
metrics: [{ name: 'eventCount' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'eventCount' }, desc: true }],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch device categories
|
|
||||||
const [deviceResponse] = await this.analyticsClient.runRealtimeReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dimensions: [{ name: 'deviceCategory' }],
|
|
||||||
metrics: [{ name: 'activeUsers' }],
|
|
||||||
orderBy: [{ metric: { metricName: 'activeUsers' }, desc: true }],
|
|
||||||
limit: 10,
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
pageResponse,
|
|
||||||
eventResponse,
|
|
||||||
sourceResponse: deviceResponse
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: this.CACHE_DURATIONS.REALTIME_DETAILED
|
|
||||||
});
|
|
||||||
|
|
||||||
return response;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error fetching realtime detailed data:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getUserBehavior(timeRange = '30') {
|
|
||||||
const cacheKey = `analytics:user_behavior:${timeRange}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log('User behavior data found in Redis cache');
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Fetching fresh user behavior data from GA4');
|
|
||||||
|
|
||||||
// Fetch page data
|
|
||||||
const [pageResponse] = await this.analyticsClient.runReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'pagePath' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'averageSessionDuration' },
|
|
||||||
{ name: 'bounceRate' },
|
|
||||||
{ name: 'sessions' }
|
|
||||||
],
|
|
||||||
orderBy: [{
|
|
||||||
metric: { metricName: 'screenPageViews' },
|
|
||||||
desc: true
|
|
||||||
}],
|
|
||||||
limit: 25
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch device data
|
|
||||||
const [deviceResponse] = await this.analyticsClient.runReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'deviceCategory' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'screenPageViews' },
|
|
||||||
{ name: 'sessions' }
|
|
||||||
]
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch source data
|
|
||||||
const [sourceResponse] = await this.analyticsClient.runReport({
|
|
||||||
property: `properties/${this.propertyId}`,
|
|
||||||
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
|
||||||
dimensions: [{ name: 'sessionSource' }],
|
|
||||||
metrics: [
|
|
||||||
{ name: 'sessions' },
|
|
||||||
{ name: 'conversions' }
|
|
||||||
],
|
|
||||||
orderBy: [{
|
|
||||||
metric: { metricName: 'sessions' },
|
|
||||||
desc: true
|
|
||||||
}],
|
|
||||||
limit: 25,
|
|
||||||
returnPropertyQuota: true
|
|
||||||
});
|
|
||||||
|
|
||||||
const response = {
|
|
||||||
pageResponse,
|
|
||||||
deviceResponse,
|
|
||||||
sourceResponse
|
|
||||||
};
|
|
||||||
|
|
||||||
// Cache the response
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(response), {
|
|
||||||
EX: this.CACHE_DURATIONS.USER_BEHAVIOR
|
|
||||||
});
|
|
||||||
|
|
||||||
return response;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error fetching user behavior data:', {
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = new AnalyticsService();
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
const winston = require('winston');
|
|
||||||
const path = require('path');
|
|
||||||
|
|
||||||
const logger = winston.createLogger({
|
|
||||||
level: process.env.LOG_LEVEL || 'info',
|
|
||||||
format: winston.format.combine(
|
|
||||||
winston.format.timestamp(),
|
|
||||||
winston.format.json()
|
|
||||||
),
|
|
||||||
transports: [
|
|
||||||
new winston.transports.File({
|
|
||||||
filename: path.join(__dirname, '../logs/pm2/error.log'),
|
|
||||||
level: 'error',
|
|
||||||
maxsize: 10485760, // 10MB
|
|
||||||
maxFiles: 5
|
|
||||||
}),
|
|
||||||
new winston.transports.File({
|
|
||||||
filename: path.join(__dirname, '../logs/pm2/combined.log'),
|
|
||||||
maxsize: 10485760, // 10MB
|
|
||||||
maxFiles: 5
|
|
||||||
})
|
|
||||||
]
|
|
||||||
});
|
|
||||||
|
|
||||||
// Add console transport in development
|
|
||||||
if (process.env.NODE_ENV !== 'production') {
|
|
||||||
logger.add(new winston.transports.Console({
|
|
||||||
format: winston.format.combine(
|
|
||||||
winston.format.colorize(),
|
|
||||||
winston.format.simple()
|
|
||||||
)
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = logger;
|
|
||||||
-1068
File diff suppressed because it is too large
Load Diff
@@ -1,19 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "gorgias-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"main": "index.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"description": "",
|
|
||||||
"dependencies": {
|
|
||||||
"axios": "^1.7.9",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"express": "^4.21.2",
|
|
||||||
"redis": "^4.7.0"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,119 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const router = express.Router();
|
|
||||||
const gorgiasService = require('../services/gorgias.service');
|
|
||||||
|
|
||||||
// Get statistics
|
|
||||||
router.post('/stats/:name', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { name } = req.params;
|
|
||||||
const filters = req.body;
|
|
||||||
|
|
||||||
console.log(`Fetching ${name} statistics with filters:`, filters);
|
|
||||||
|
|
||||||
if (!name) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Missing statistic name',
|
|
||||||
details: 'The name parameter is required'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const data = await gorgiasService.getStatistics(name, filters);
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'No data found',
|
|
||||||
details: `No statistics found for ${name}`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.json({ data });
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Statistics error:', {
|
|
||||||
name: req.params.name,
|
|
||||||
filters: req.body,
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle specific error cases
|
|
||||||
if (error.response?.status === 401) {
|
|
||||||
return res.status(401).json({
|
|
||||||
error: 'Authentication failed',
|
|
||||||
details: 'Invalid Gorgias API credentials'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 404) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'Not found',
|
|
||||||
details: `Statistics type '${req.params.name}' not found`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 400) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Invalid request',
|
|
||||||
details: error.response?.data?.message || 'The request was invalid',
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch statistics',
|
|
||||||
details: error.response?.data?.message || error.message,
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Get tickets
|
|
||||||
router.get('/tickets', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const data = await gorgiasService.getTickets(req.query);
|
|
||||||
res.json(data);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Tickets error:', {
|
|
||||||
params: req.query,
|
|
||||||
error: error.message,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
|
|
||||||
if (error.response?.status === 401) {
|
|
||||||
return res.status(401).json({
|
|
||||||
error: 'Authentication failed',
|
|
||||||
details: 'Invalid Gorgias API credentials'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 400) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Invalid request',
|
|
||||||
details: error.response?.data?.message || 'The request was invalid',
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch tickets',
|
|
||||||
details: error.response?.data?.message || error.message,
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Get customer satisfaction
|
|
||||||
router.get('/satisfaction', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const data = await gorgiasService.getCustomerSatisfaction(req.query);
|
|
||||||
res.json(data);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Satisfaction error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch customer satisfaction',
|
|
||||||
details: error.response?.data || error.message
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = router;
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const cors = require('cors');
|
|
||||||
const path = require('path');
|
|
||||||
require('dotenv').config({
|
|
||||||
path: path.resolve(__dirname, '.env')
|
|
||||||
});
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.PORT || 3006;
|
|
||||||
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Import routes
|
|
||||||
const gorgiasRoutes = require('./routes/gorgias.routes');
|
|
||||||
|
|
||||||
// Use routes
|
|
||||||
app.use('/api/gorgias', gorgiasRoutes);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error(err.stack);
|
|
||||||
res.status(500).json({ error: 'Something went wrong!' });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
|
||||||
console.log(`Gorgias API server running on port ${port}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = app;
|
|
||||||
@@ -1,119 +0,0 @@
|
|||||||
const axios = require('axios');
|
|
||||||
const { createClient } = require('redis');
|
|
||||||
|
|
||||||
class GorgiasService {
|
|
||||||
constructor() {
|
|
||||||
this.redis = createClient({
|
|
||||||
url: process.env.REDIS_URL
|
|
||||||
});
|
|
||||||
|
|
||||||
this.redis.on('error', err => console.error('Redis Client Error:', err));
|
|
||||||
this.redis.connect().catch(err => console.error('Redis connection error:', err));
|
|
||||||
|
|
||||||
// Create base64 encoded auth string
|
|
||||||
const auth = Buffer.from(`${process.env.GORGIAS_API_USERNAME}:${process.env.GORGIAS_API_KEY}`).toString('base64');
|
|
||||||
|
|
||||||
this.apiClient = axios.create({
|
|
||||||
baseURL: `https://${process.env.GORGIAS_DOMAIN}.gorgias.com/api`,
|
|
||||||
headers: {
|
|
||||||
'Authorization': `Basic ${auth}`,
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async getStatistics(name, filters = {}) {
|
|
||||||
const cacheKey = `gorgias:stats:${name}:${JSON.stringify(filters)}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log(`Statistics ${name} found in Redis cache`);
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log(`Fetching ${name} statistics with filters:`, filters);
|
|
||||||
|
|
||||||
// Convert dates to UTC midnight if not already set
|
|
||||||
if (!filters.start_datetime || !filters.end_datetime) {
|
|
||||||
const start = new Date(filters.start_datetime || filters.start_date);
|
|
||||||
start.setUTCHours(0, 0, 0, 0);
|
|
||||||
const end = new Date(filters.end_datetime || filters.end_date);
|
|
||||||
end.setUTCHours(23, 59, 59, 999);
|
|
||||||
|
|
||||||
filters = {
|
|
||||||
...filters,
|
|
||||||
start_datetime: start.toISOString(),
|
|
||||||
end_datetime: end.toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch from API
|
|
||||||
const response = await this.apiClient.post(`/stats/${name}`, filters);
|
|
||||||
const data = response.data;
|
|
||||||
|
|
||||||
// Save to Redis with 5 minute expiry
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(data), {
|
|
||||||
EX: 300 // 5 minutes
|
|
||||||
});
|
|
||||||
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error(`Error in getStatistics for ${name}:`, {
|
|
||||||
error: error.message,
|
|
||||||
filters,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getTickets(params = {}) {
|
|
||||||
const cacheKey = `gorgias:tickets:${JSON.stringify(params)}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log('Tickets found in Redis cache');
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert dates to UTC midnight
|
|
||||||
const formattedParams = { ...params };
|
|
||||||
if (params.start_date) {
|
|
||||||
const start = new Date(params.start_date);
|
|
||||||
start.setUTCHours(0, 0, 0, 0);
|
|
||||||
formattedParams.start_datetime = start.toISOString();
|
|
||||||
delete formattedParams.start_date;
|
|
||||||
}
|
|
||||||
if (params.end_date) {
|
|
||||||
const end = new Date(params.end_date);
|
|
||||||
end.setUTCHours(23, 59, 59, 999);
|
|
||||||
formattedParams.end_datetime = end.toISOString();
|
|
||||||
delete formattedParams.end_date;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch from API
|
|
||||||
const response = await this.apiClient.get('/tickets', { params: formattedParams });
|
|
||||||
const data = response.data;
|
|
||||||
|
|
||||||
// Save to Redis with 5 minute expiry
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(data), {
|
|
||||||
EX: 300 // 5 minutes
|
|
||||||
});
|
|
||||||
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error fetching tickets:', {
|
|
||||||
error: error.message,
|
|
||||||
params,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = new GorgiasService();
|
|
||||||
-2104
File diff suppressed because it is too large
Load Diff
@@ -1,26 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "klaviyo-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"description": "Klaviyo API integration server",
|
|
||||||
"main": "server.js",
|
|
||||||
"type": "module",
|
|
||||||
"scripts": {
|
|
||||||
"start": "node server.js",
|
|
||||||
"dev": "nodemon server.js"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"esm": "^3.2.25",
|
|
||||||
"express": "^4.18.2",
|
|
||||||
"express-rate-limit": "^7.5.0",
|
|
||||||
"ioredis": "^5.4.1",
|
|
||||||
"luxon": "^3.5.0",
|
|
||||||
"node-fetch": "^3.3.2",
|
|
||||||
"pg": "^8.18.0",
|
|
||||||
"recharts": "^2.15.0"
|
|
||||||
},
|
|
||||||
"devDependencies": {
|
|
||||||
"nodemon": "^3.0.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
import express from 'express';
|
|
||||||
import { createEventsRouter } from './events.routes.js';
|
|
||||||
import { createMetricsRoutes } from './metrics.routes.js';
|
|
||||||
import { createCampaignsRouter } from './campaigns.routes.js';
|
|
||||||
import { createReportingRouter } from './reporting.routes.js';
|
|
||||||
|
|
||||||
export function createApiRouter(apiKey, apiRevision) {
|
|
||||||
const router = express.Router();
|
|
||||||
|
|
||||||
// Mount routers
|
|
||||||
router.use('/events', createEventsRouter(apiKey, apiRevision));
|
|
||||||
router.use('/metrics', createMetricsRoutes(apiKey, apiRevision));
|
|
||||||
router.use('/campaigns', createCampaignsRouter(apiKey, apiRevision));
|
|
||||||
router.use('/reporting', createReportingRouter(apiKey, apiRevision));
|
|
||||||
|
|
||||||
return router;
|
|
||||||
}
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
import express from 'express';
|
|
||||||
import cors from 'cors';
|
|
||||||
import dotenv from 'dotenv';
|
|
||||||
import rateLimit from 'express-rate-limit';
|
|
||||||
import { createApiRouter } from './routes/index.js';
|
|
||||||
import path from 'path';
|
|
||||||
import { fileURLToPath } from 'url';
|
|
||||||
|
|
||||||
// Get directory name in ES modules
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
|
||||||
const __dirname = path.dirname(__filename);
|
|
||||||
|
|
||||||
// Load environment variables
|
|
||||||
const envPath = path.resolve(__dirname, '.env');
|
|
||||||
console.log('[Server] Loading .env file from:', envPath);
|
|
||||||
dotenv.config({ path: envPath });
|
|
||||||
|
|
||||||
// Debug environment variables (without exposing sensitive data)
|
|
||||||
console.log('[Server] Environment variables loaded:', {
|
|
||||||
REDIS_HOST: process.env.REDIS_HOST || '(not set)',
|
|
||||||
REDIS_PORT: process.env.REDIS_PORT || '(not set)',
|
|
||||||
REDIS_USERNAME: process.env.REDIS_USERNAME || '(not set)',
|
|
||||||
REDIS_PASSWORD: process.env.REDIS_PASSWORD ? '(set)' : '(not set)',
|
|
||||||
NODE_ENV: process.env.NODE_ENV || '(not set)',
|
|
||||||
});
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.KLAVIYO_PORT || 3004;
|
|
||||||
|
|
||||||
// Rate limiting for reporting endpoints
|
|
||||||
const reportingLimiter = rateLimit({
|
|
||||||
windowMs: 10 * 60 * 1000, // 10 minutes
|
|
||||||
max: 10, // limit each IP to 10 requests per windowMs
|
|
||||||
message: 'Too many requests to reporting endpoint, please try again later',
|
|
||||||
keyGenerator: (req) => {
|
|
||||||
// Use a combination of IP and endpoint for more granular control
|
|
||||||
return `${req.ip}-reporting`;
|
|
||||||
},
|
|
||||||
skip: (req) => {
|
|
||||||
// Only apply to campaign-values-reports endpoint
|
|
||||||
return !req.path.includes('campaign-values-reports');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Middleware
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Debug middleware to log all requests
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
console.log(`[${new Date().toISOString()}] ${req.method} ${req.url}`);
|
|
||||||
next();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Apply rate limiting to reporting endpoints
|
|
||||||
app.use('/api/klaviyo/reporting', reportingLimiter);
|
|
||||||
|
|
||||||
// Create and mount API routes
|
|
||||||
const apiRouter = createApiRouter(
|
|
||||||
process.env.KLAVIYO_API_KEY,
|
|
||||||
process.env.KLAVIYO_API_REVISION || '2024-02-15'
|
|
||||||
);
|
|
||||||
app.use('/api/klaviyo', apiRouter);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error('Unhandled error:', err);
|
|
||||||
res.status(500).json({
|
|
||||||
status: 'error',
|
|
||||||
message: 'Internal server error',
|
|
||||||
details: process.env.NODE_ENV === 'development' ? err.message : undefined
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, '0.0.0.0', () => {
|
|
||||||
console.log(`Klaviyo server listening at http://0.0.0.0:${port}`);
|
|
||||||
});
|
|
||||||
@@ -1,262 +0,0 @@
|
|||||||
import Redis from 'ioredis';
|
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
|
||||||
import dotenv from 'dotenv';
|
|
||||||
import path from 'path';
|
|
||||||
import { fileURLToPath } from 'url';
|
|
||||||
|
|
||||||
// Get directory name in ES modules
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
|
||||||
const __dirname = path.dirname(__filename);
|
|
||||||
|
|
||||||
// Load environment variables again (redundant but safe)
|
|
||||||
const envPath = path.resolve(__dirname, '../.env');
|
|
||||||
console.log('[RedisService] Loading .env file from:', envPath);
|
|
||||||
dotenv.config({ path: envPath });
|
|
||||||
|
|
||||||
export class RedisService {
|
|
||||||
constructor() {
|
|
||||||
this.timeManager = new TimeManager();
|
|
||||||
this.DEFAULT_TTL = 5 * 60; // 5 minutes default TTL
|
|
||||||
this.isConnected = false;
|
|
||||||
this._initializeRedis();
|
|
||||||
}
|
|
||||||
|
|
||||||
_initializeRedis() {
|
|
||||||
try {
|
|
||||||
// Debug: Print all environment variables we're looking for
|
|
||||||
console.log('[RedisService] Environment variables state:', {
|
|
||||||
REDIS_HOST: process.env.REDIS_HOST ? '(set)' : '(not set)',
|
|
||||||
REDIS_PORT: process.env.REDIS_PORT ? '(set)' : '(not set)',
|
|
||||||
REDIS_USERNAME: process.env.REDIS_USERNAME ? '(set)' : '(not set)',
|
|
||||||
REDIS_PASSWORD: process.env.REDIS_PASSWORD ? '(set)' : '(not set)',
|
|
||||||
});
|
|
||||||
|
|
||||||
// Log Redis configuration (without password)
|
|
||||||
const host = process.env.REDIS_HOST || 'localhost';
|
|
||||||
const port = parseInt(process.env.REDIS_PORT) || 6379;
|
|
||||||
const username = process.env.REDIS_USERNAME || 'default';
|
|
||||||
const password = process.env.REDIS_PASSWORD;
|
|
||||||
|
|
||||||
console.log('[RedisService] Initializing Redis with config:', {
|
|
||||||
host,
|
|
||||||
port,
|
|
||||||
username,
|
|
||||||
hasPassword: !!password
|
|
||||||
});
|
|
||||||
|
|
||||||
const config = {
|
|
||||||
host,
|
|
||||||
port,
|
|
||||||
username,
|
|
||||||
retryStrategy: (times) => {
|
|
||||||
const delay = Math.min(times * 50, 2000);
|
|
||||||
return delay;
|
|
||||||
},
|
|
||||||
maxRetriesPerRequest: 3,
|
|
||||||
enableReadyCheck: true,
|
|
||||||
connectTimeout: 10000,
|
|
||||||
showFriendlyErrorStack: true,
|
|
||||||
retryUnfulfilled: true,
|
|
||||||
maxRetryAttempts: 5
|
|
||||||
};
|
|
||||||
|
|
||||||
// Only add password if it exists
|
|
||||||
if (password) {
|
|
||||||
console.log('[RedisService] Adding password to config');
|
|
||||||
config.password = password;
|
|
||||||
} else {
|
|
||||||
console.warn('[RedisService] No Redis password found in environment variables!');
|
|
||||||
}
|
|
||||||
|
|
||||||
this.client = new Redis(config);
|
|
||||||
|
|
||||||
// Handle connection events
|
|
||||||
this.client.on('connect', () => {
|
|
||||||
console.log('[RedisService] Connected to Redis');
|
|
||||||
this.isConnected = true;
|
|
||||||
});
|
|
||||||
|
|
||||||
this.client.on('ready', () => {
|
|
||||||
console.log('[RedisService] Redis is ready');
|
|
||||||
this.isConnected = true;
|
|
||||||
});
|
|
||||||
|
|
||||||
this.client.on('error', (err) => {
|
|
||||||
console.error('[RedisService] Redis error:', err);
|
|
||||||
this.isConnected = false;
|
|
||||||
// Log more details about the error
|
|
||||||
if (err.code === 'WRONGPASS') {
|
|
||||||
console.error('[RedisService] Authentication failed. Please check your Redis password.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
this.client.on('close', () => {
|
|
||||||
console.log('[RedisService] Redis connection closed');
|
|
||||||
this.isConnected = false;
|
|
||||||
});
|
|
||||||
|
|
||||||
this.client.on('reconnecting', (params) => {
|
|
||||||
console.log('[RedisService] Reconnecting to Redis:', params);
|
|
||||||
});
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error initializing Redis:', error);
|
|
||||||
this.isConnected = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async get(key) {
|
|
||||||
if (!this.isConnected) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const data = await this.client.get(key);
|
|
||||||
return data ? JSON.parse(data) : null;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error getting data:', error);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async set(key, data, ttl = this.DEFAULT_TTL) {
|
|
||||||
if (!this.isConnected) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
await this.client.setex(key, ttl, JSON.stringify(data));
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error setting data:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Helper to generate cache keys
|
|
||||||
_getCacheKey(type, params = {}) {
|
|
||||||
const {
|
|
||||||
timeRange,
|
|
||||||
startDate,
|
|
||||||
endDate,
|
|
||||||
metricId,
|
|
||||||
metric,
|
|
||||||
daily,
|
|
||||||
cacheKey,
|
|
||||||
isPreviousPeriod,
|
|
||||||
customFilters
|
|
||||||
} = params;
|
|
||||||
|
|
||||||
let key = `klaviyo:${type}`;
|
|
||||||
|
|
||||||
// Handle "stats:details" for daily or metric-based keys
|
|
||||||
if (type === 'stats:details') {
|
|
||||||
// Add metric to key
|
|
||||||
key += `:${metric || 'all'}`;
|
|
||||||
|
|
||||||
// Add daily flag if present
|
|
||||||
if (daily) {
|
|
||||||
key += ':daily';
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add custom filters hash if present
|
|
||||||
if (customFilters?.length) {
|
|
||||||
const filterHash = customFilters.join('').replace(/[^a-zA-Z0-9]/g, '');
|
|
||||||
key += `:${filterHash}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// If a specific cache key is provided, use it (highest priority)
|
|
||||||
if (cacheKey) {
|
|
||||||
key += `:${cacheKey}`;
|
|
||||||
}
|
|
||||||
// Otherwise, build a default cache key
|
|
||||||
else if (timeRange) {
|
|
||||||
key += `:${timeRange}`;
|
|
||||||
if (metricId) {
|
|
||||||
key += `:${metricId}`;
|
|
||||||
}
|
|
||||||
if (isPreviousPeriod) {
|
|
||||||
key += ':prev';
|
|
||||||
}
|
|
||||||
} else if (startDate && endDate) {
|
|
||||||
// For custom date ranges, include both dates in the key
|
|
||||||
key += `:custom:${startDate}:${endDate}`;
|
|
||||||
if (metricId) {
|
|
||||||
key += `:${metricId}`;
|
|
||||||
}
|
|
||||||
if (isPreviousPeriod) {
|
|
||||||
key += ':prev';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add order type to key if present
|
|
||||||
if (['pre_orders', 'local_pickup', 'on_hold'].includes(metric)) {
|
|
||||||
key += `:${metric}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
return key;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
// Get TTL based on time range
|
|
||||||
_getTTL(timeRange) {
|
|
||||||
const TTL_MAP = {
|
|
||||||
'today': 2 * 60, // 2 minutes
|
|
||||||
'yesterday': 30 * 60, // 30 minutes
|
|
||||||
'thisWeek': 5 * 60, // 5 minutes
|
|
||||||
'lastWeek': 60 * 60, // 1 hour
|
|
||||||
'thisMonth': 10 * 60, // 10 minutes
|
|
||||||
'lastMonth': 2 * 60 * 60, // 2 hours
|
|
||||||
'last7days': 5 * 60, // 5 minutes
|
|
||||||
'last30days': 15 * 60, // 15 minutes
|
|
||||||
'custom': 15 * 60 // 15 minutes
|
|
||||||
};
|
|
||||||
return TTL_MAP[timeRange] || this.DEFAULT_TTL;
|
|
||||||
}
|
|
||||||
|
|
||||||
async getEventData(type, params) {
|
|
||||||
if (!this.isConnected) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const baseKey = this._getCacheKey('events', params);
|
|
||||||
const data = await this.get(`${baseKey}:${type}`);
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error getting event data:', error);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async cacheEventData(type, params, data) {
|
|
||||||
if (!this.isConnected) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const ttl = this._getTTL(params.timeRange);
|
|
||||||
const baseKey = this._getCacheKey('events', params);
|
|
||||||
|
|
||||||
// Cache raw event data
|
|
||||||
await this.set(`${baseKey}:${type}`, data, ttl);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error caching event data:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async clearCache(params = {}) {
|
|
||||||
if (!this.isConnected) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const pattern = this._getCacheKey('events', params) + '*';
|
|
||||||
const keys = await this.client.keys(pattern);
|
|
||||||
if (keys.length > 0) {
|
|
||||||
await this.client.del(...keys);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[RedisService] Error clearing cache:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,967 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "meta-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"lockfileVersion": 3,
|
|
||||||
"requires": true,
|
|
||||||
"packages": {
|
|
||||||
"": {
|
|
||||||
"name": "meta-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"license": "ISC",
|
|
||||||
"dependencies": {
|
|
||||||
"axios": "^1.7.9",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"express": "^4.21.2"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/accepts": {
|
|
||||||
"version": "1.3.8",
|
|
||||||
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
|
|
||||||
"integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"mime-types": "~2.1.34",
|
|
||||||
"negotiator": "0.6.3"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/array-flatten": {
|
|
||||||
"version": "1.1.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
|
|
||||||
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/asynckit": {
|
|
||||||
"version": "0.4.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
|
|
||||||
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/axios": {
|
|
||||||
"version": "1.12.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/axios/-/axios-1.12.2.tgz",
|
|
||||||
"integrity": "sha512-vMJzPewAlRyOgxV2dU0Cuz2O8zzzx9VYtbJOaBgXFeLc4IV/Eg50n4LowmehOOR61S8ZMpc2K5Sa7g6A4jfkUw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"follow-redirects": "^1.15.6",
|
|
||||||
"form-data": "^4.0.4",
|
|
||||||
"proxy-from-env": "^1.1.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/body-parser": {
|
|
||||||
"version": "1.20.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.3.tgz",
|
|
||||||
"integrity": "sha512-7rAxByjUMqQ3/bHJy7D6OGXvx/MMc4IqBn/X0fcM1QUcAItpZrBEYhWGem+tzXH90c+G01ypMcYJBO9Y30203g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"bytes": "3.1.2",
|
|
||||||
"content-type": "~1.0.5",
|
|
||||||
"debug": "2.6.9",
|
|
||||||
"depd": "2.0.0",
|
|
||||||
"destroy": "1.2.0",
|
|
||||||
"http-errors": "2.0.0",
|
|
||||||
"iconv-lite": "0.4.24",
|
|
||||||
"on-finished": "2.4.1",
|
|
||||||
"qs": "6.13.0",
|
|
||||||
"raw-body": "2.5.2",
|
|
||||||
"type-is": "~1.6.18",
|
|
||||||
"unpipe": "1.0.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8",
|
|
||||||
"npm": "1.2.8000 || >= 1.4.16"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/bytes": {
|
|
||||||
"version": "3.1.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz",
|
|
||||||
"integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/call-bind-apply-helpers": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-BhYE+WDaywFg2TBWYNXAE+8B1ATnThNBqXHP5nQu0jWJdVvY2hvkpyB3qOmtmDePiS5/BDQ8wASEWGMWRG148g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"function-bind": "^1.1.2"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/call-bound": {
|
|
||||||
"version": "1.0.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.3.tgz",
|
|
||||||
"integrity": "sha512-YTd+6wGlNlPxSuri7Y6X8tY2dmm12UMH66RpKMhiX6rsk5wXXnYgbUcOt8kiS31/AjfoTOvCsE+w8nZQLQnzHA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"call-bind-apply-helpers": "^1.0.1",
|
|
||||||
"get-intrinsic": "^1.2.6"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/combined-stream": {
|
|
||||||
"version": "1.0.8",
|
|
||||||
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
|
|
||||||
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"delayed-stream": "~1.0.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/content-disposition": {
|
|
||||||
"version": "0.5.4",
|
|
||||||
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
|
|
||||||
"integrity": "sha512-FveZTNuGw04cxlAiWbzi6zTAL/lhehaWbTtgluJh4/E95DqMwTmha3KZN1aAWA8cFIhHzMZUvLevkw5Rqk+tSQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"safe-buffer": "5.2.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/content-type": {
|
|
||||||
"version": "1.0.5",
|
|
||||||
"resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz",
|
|
||||||
"integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/cookie": {
|
|
||||||
"version": "0.7.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.1.tgz",
|
|
||||||
"integrity": "sha512-6DnInpx7SJ2AK3+CTUE/ZM0vWTUboZCegxhC2xiIydHR9jNuTAASBrfEpHhiGOZw/nX51bHt6YQl8jsGo4y/0w==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/cookie-signature": {
|
|
||||||
"version": "1.0.6",
|
|
||||||
"resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
|
|
||||||
"integrity": "sha512-QADzlaHc8icV8I7vbaJXJwod9HWYp8uCqf1xa4OfNu1T7JVxQIrUgOWtHdNDtPiywmFbiS12VjotIXLrKM3orQ==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/cors": {
|
|
||||||
"version": "2.8.5",
|
|
||||||
"resolved": "https://registry.npmjs.org/cors/-/cors-2.8.5.tgz",
|
|
||||||
"integrity": "sha512-KIHbLJqu73RGr/hnbrO9uBeixNGuvSQjul/jdFvS/KFSIH1hWVd1ng7zOHx+YrEfInLG7q4n6GHQ9cDtxv/P6g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"object-assign": "^4",
|
|
||||||
"vary": "^1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.10"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/debug": {
|
|
||||||
"version": "2.6.9",
|
|
||||||
"resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
|
|
||||||
"integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"ms": "2.0.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/delayed-stream": {
|
|
||||||
"version": "1.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
|
|
||||||
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=0.4.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/depd": {
|
|
||||||
"version": "2.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
|
|
||||||
"integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/destroy": {
|
|
||||||
"version": "1.2.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/destroy/-/destroy-1.2.0.tgz",
|
|
||||||
"integrity": "sha512-2sJGJTaXIIaR1w4iJSNoN0hnMY7Gpc/n8D4qSCJw8QqFWXf7cuAgnEHxBpweaVcPevC2l3KpjYCx3NypQQgaJg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8",
|
|
||||||
"npm": "1.2.8000 || >= 1.4.16"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/dotenv": {
|
|
||||||
"version": "16.4.7",
|
|
||||||
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.4.7.tgz",
|
|
||||||
"integrity": "sha512-47qPchRCykZC03FhkYAhrvwU4xDBFIj1QPqaarj6mdM/hgUzfPHcpkHJOn3mJAufFeeAxAzeGsr5X0M4k6fLZQ==",
|
|
||||||
"license": "BSD-2-Clause",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=12"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://dotenvx.com"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/dunder-proto": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"call-bind-apply-helpers": "^1.0.1",
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"gopd": "^1.2.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/ee-first": {
|
|
||||||
"version": "1.1.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
|
|
||||||
"integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/encodeurl": {
|
|
||||||
"version": "2.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz",
|
|
||||||
"integrity": "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/es-define-property": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/es-errors": {
|
|
||||||
"version": "1.3.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
|
|
||||||
"integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/es-object-atoms": {
|
|
||||||
"version": "1.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.0.0.tgz",
|
|
||||||
"integrity": "sha512-MZ4iQ6JwHOBQjahnjwaC1ZtIBH+2ohjamzAO3oaHcXYup7qxjF2fixyH+Q71voWHeOkI2q/TnJao/KfXYIZWbw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"es-errors": "^1.3.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/es-set-tostringtag": {
|
|
||||||
"version": "2.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
|
|
||||||
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"get-intrinsic": "^1.2.6",
|
|
||||||
"has-tostringtag": "^1.0.2",
|
|
||||||
"hasown": "^2.0.2"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/escape-html": {
|
|
||||||
"version": "1.0.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
|
|
||||||
"integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/etag": {
|
|
||||||
"version": "1.8.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
|
|
||||||
"integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/express": {
|
|
||||||
"version": "4.21.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/express/-/express-4.21.2.tgz",
|
|
||||||
"integrity": "sha512-28HqgMZAmih1Czt9ny7qr6ek2qddF4FclbMzwhCREB6OFfH+rXAnuNCwo1/wFvrtbgsQDb4kSbX9de9lFbrXnA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"accepts": "~1.3.8",
|
|
||||||
"array-flatten": "1.1.1",
|
|
||||||
"body-parser": "1.20.3",
|
|
||||||
"content-disposition": "0.5.4",
|
|
||||||
"content-type": "~1.0.4",
|
|
||||||
"cookie": "0.7.1",
|
|
||||||
"cookie-signature": "1.0.6",
|
|
||||||
"debug": "2.6.9",
|
|
||||||
"depd": "2.0.0",
|
|
||||||
"encodeurl": "~2.0.0",
|
|
||||||
"escape-html": "~1.0.3",
|
|
||||||
"etag": "~1.8.1",
|
|
||||||
"finalhandler": "1.3.1",
|
|
||||||
"fresh": "0.5.2",
|
|
||||||
"http-errors": "2.0.0",
|
|
||||||
"merge-descriptors": "1.0.3",
|
|
||||||
"methods": "~1.1.2",
|
|
||||||
"on-finished": "2.4.1",
|
|
||||||
"parseurl": "~1.3.3",
|
|
||||||
"path-to-regexp": "0.1.12",
|
|
||||||
"proxy-addr": "~2.0.7",
|
|
||||||
"qs": "6.13.0",
|
|
||||||
"range-parser": "~1.2.1",
|
|
||||||
"safe-buffer": "5.2.1",
|
|
||||||
"send": "0.19.0",
|
|
||||||
"serve-static": "1.16.2",
|
|
||||||
"setprototypeof": "1.2.0",
|
|
||||||
"statuses": "2.0.1",
|
|
||||||
"type-is": "~1.6.18",
|
|
||||||
"utils-merge": "1.0.1",
|
|
||||||
"vary": "~1.1.2"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.10.0"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"type": "opencollective",
|
|
||||||
"url": "https://opencollective.com/express"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/finalhandler": {
|
|
||||||
"version": "1.3.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.1.tgz",
|
|
||||||
"integrity": "sha512-6BN9trH7bp3qvnrRyzsBz+g3lZxTNZTbVO2EV1CS0WIcDbawYVdYvGflME/9QP0h0pYlCDBCTjYa9nZzMDpyxQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"debug": "2.6.9",
|
|
||||||
"encodeurl": "~2.0.0",
|
|
||||||
"escape-html": "~1.0.3",
|
|
||||||
"on-finished": "2.4.1",
|
|
||||||
"parseurl": "~1.3.3",
|
|
||||||
"statuses": "2.0.1",
|
|
||||||
"unpipe": "~1.0.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/follow-redirects": {
|
|
||||||
"version": "1.15.9",
|
|
||||||
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.9.tgz",
|
|
||||||
"integrity": "sha512-gew4GsXizNgdoRyqmyfMHyAmXsZDk6mHkSxZFCzW9gwlbtOW44CDtYavM+y+72qD/Vq2l550kMF52DT8fOLJqQ==",
|
|
||||||
"funding": [
|
|
||||||
{
|
|
||||||
"type": "individual",
|
|
||||||
"url": "https://github.com/sponsors/RubenVerborgh"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=4.0"
|
|
||||||
},
|
|
||||||
"peerDependenciesMeta": {
|
|
||||||
"debug": {
|
|
||||||
"optional": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/form-data": {
|
|
||||||
"version": "4.0.4",
|
|
||||||
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.4.tgz",
|
|
||||||
"integrity": "sha512-KrGhL9Q4zjj0kiUt5OO4Mr/A/jlI2jDYs5eHBpYHPcBEVSiipAvn2Ko2HnPe20rmcuuvMHNdZFp+4IlGTMF0Ow==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"asynckit": "^0.4.0",
|
|
||||||
"combined-stream": "^1.0.8",
|
|
||||||
"es-set-tostringtag": "^2.1.0",
|
|
||||||
"hasown": "^2.0.2",
|
|
||||||
"mime-types": "^2.1.12"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/forwarded": {
|
|
||||||
"version": "0.2.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
|
|
||||||
"integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/fresh": {
|
|
||||||
"version": "0.5.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
|
|
||||||
"integrity": "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/function-bind": {
|
|
||||||
"version": "1.1.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
|
|
||||||
"integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/get-intrinsic": {
|
|
||||||
"version": "1.2.6",
|
|
||||||
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.6.tgz",
|
|
||||||
"integrity": "sha512-qxsEs+9A+u85HhllWJJFicJfPDhRmjzoYdl64aMWW9yRIJmSyxdn8IEkuIM530/7T+lv0TIHd8L6Q/ra0tEoeA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"call-bind-apply-helpers": "^1.0.1",
|
|
||||||
"dunder-proto": "^1.0.0",
|
|
||||||
"es-define-property": "^1.0.1",
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"es-object-atoms": "^1.0.0",
|
|
||||||
"function-bind": "^1.1.2",
|
|
||||||
"gopd": "^1.2.0",
|
|
||||||
"has-symbols": "^1.1.0",
|
|
||||||
"hasown": "^2.0.2",
|
|
||||||
"math-intrinsics": "^1.0.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/gopd": {
|
|
||||||
"version": "1.2.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
|
|
||||||
"integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/has-symbols": {
|
|
||||||
"version": "1.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
|
|
||||||
"integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/has-tostringtag": {
|
|
||||||
"version": "1.0.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
|
|
||||||
"integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"has-symbols": "^1.0.3"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/hasown": {
|
|
||||||
"version": "2.0.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
|
|
||||||
"integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"function-bind": "^1.1.2"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/http-errors": {
|
|
||||||
"version": "2.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.0.tgz",
|
|
||||||
"integrity": "sha512-FtwrG/euBzaEjYeRqOgly7G0qviiXoJWnvEH2Z1plBdXgbyjv34pHTSb9zoeHMyDy33+DWy5Wt9Wo+TURtOYSQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"depd": "2.0.0",
|
|
||||||
"inherits": "2.0.4",
|
|
||||||
"setprototypeof": "1.2.0",
|
|
||||||
"statuses": "2.0.1",
|
|
||||||
"toidentifier": "1.0.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/iconv-lite": {
|
|
||||||
"version": "0.4.24",
|
|
||||||
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz",
|
|
||||||
"integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"safer-buffer": ">= 2.1.2 < 3"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">=0.10.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/inherits": {
|
|
||||||
"version": "2.0.4",
|
|
||||||
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
|
|
||||||
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
|
|
||||||
"license": "ISC"
|
|
||||||
},
|
|
||||||
"node_modules/ipaddr.js": {
|
|
||||||
"version": "1.9.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
|
|
||||||
"integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.10"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/math-intrinsics": {
|
|
||||||
"version": "1.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
|
|
||||||
"integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/media-typer": {
|
|
||||||
"version": "0.3.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
|
|
||||||
"integrity": "sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/merge-descriptors": {
|
|
||||||
"version": "1.0.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.3.tgz",
|
|
||||||
"integrity": "sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/sindresorhus"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/methods": {
|
|
||||||
"version": "1.1.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
|
|
||||||
"integrity": "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/mime": {
|
|
||||||
"version": "1.6.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz",
|
|
||||||
"integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"bin": {
|
|
||||||
"mime": "cli.js"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">=4"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/mime-db": {
|
|
||||||
"version": "1.52.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
|
|
||||||
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/mime-types": {
|
|
||||||
"version": "2.1.35",
|
|
||||||
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
|
|
||||||
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"mime-db": "1.52.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/ms": {
|
|
||||||
"version": "2.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
|
|
||||||
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/negotiator": {
|
|
||||||
"version": "0.6.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
|
|
||||||
"integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/object-assign": {
|
|
||||||
"version": "4.1.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
|
|
||||||
"integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=0.10.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/object-inspect": {
|
|
||||||
"version": "1.13.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.3.tgz",
|
|
||||||
"integrity": "sha512-kDCGIbxkDSXE3euJZZXzc6to7fCrKHNI/hSRQnRuQ+BWjFNzZwiFF8fj/6o2t2G9/jTj8PSIYTfCLelLZEeRpA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/on-finished": {
|
|
||||||
"version": "2.4.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
|
|
||||||
"integrity": "sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"ee-first": "1.1.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/parseurl": {
|
|
||||||
"version": "1.3.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
|
|
||||||
"integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/path-to-regexp": {
|
|
||||||
"version": "0.1.12",
|
|
||||||
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.12.tgz",
|
|
||||||
"integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/proxy-addr": {
|
|
||||||
"version": "2.0.7",
|
|
||||||
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
|
|
||||||
"integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"forwarded": "0.2.0",
|
|
||||||
"ipaddr.js": "1.9.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.10"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/proxy-from-env": {
|
|
||||||
"version": "1.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
|
|
||||||
"integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/qs": {
|
|
||||||
"version": "6.13.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/qs/-/qs-6.13.0.tgz",
|
|
||||||
"integrity": "sha512-+38qI9SOr8tfZ4QmJNplMUxqjbe7LKvvZgWdExBOmd+egZTtjLB67Gu0HRX3u/XOq7UU2Nx6nsjvS16Z9uwfpg==",
|
|
||||||
"license": "BSD-3-Clause",
|
|
||||||
"dependencies": {
|
|
||||||
"side-channel": "^1.0.6"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">=0.6"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/range-parser": {
|
|
||||||
"version": "1.2.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz",
|
|
||||||
"integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/raw-body": {
|
|
||||||
"version": "2.5.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.2.tgz",
|
|
||||||
"integrity": "sha512-8zGqypfENjCIqGhgXToC8aB2r7YrBX+AQAfIPs/Mlk+BtPTztOvTS01NRW/3Eh60J+a48lt8qsCzirQ6loCVfA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"bytes": "3.1.2",
|
|
||||||
"http-errors": "2.0.0",
|
|
||||||
"iconv-lite": "0.4.24",
|
|
||||||
"unpipe": "1.0.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/safe-buffer": {
|
|
||||||
"version": "5.2.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
|
|
||||||
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
|
|
||||||
"funding": [
|
|
||||||
{
|
|
||||||
"type": "github",
|
|
||||||
"url": "https://github.com/sponsors/feross"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "patreon",
|
|
||||||
"url": "https://www.patreon.com/feross"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "consulting",
|
|
||||||
"url": "https://feross.org/support"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/safer-buffer": {
|
|
||||||
"version": "2.1.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
|
|
||||||
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/send": {
|
|
||||||
"version": "0.19.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/send/-/send-0.19.0.tgz",
|
|
||||||
"integrity": "sha512-dW41u5VfLXu8SJh5bwRmyYUbAoSB3c9uQh6L8h/KtsFREPWpbX1lrljJo186Jc4nmci/sGUZ9a0a0J2zgfq2hw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"debug": "2.6.9",
|
|
||||||
"depd": "2.0.0",
|
|
||||||
"destroy": "1.2.0",
|
|
||||||
"encodeurl": "~1.0.2",
|
|
||||||
"escape-html": "~1.0.3",
|
|
||||||
"etag": "~1.8.1",
|
|
||||||
"fresh": "0.5.2",
|
|
||||||
"http-errors": "2.0.0",
|
|
||||||
"mime": "1.6.0",
|
|
||||||
"ms": "2.1.3",
|
|
||||||
"on-finished": "2.4.1",
|
|
||||||
"range-parser": "~1.2.1",
|
|
||||||
"statuses": "2.0.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/send/node_modules/encodeurl": {
|
|
||||||
"version": "1.0.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.2.tgz",
|
|
||||||
"integrity": "sha512-TPJXq8JqFaVYm2CWmPvnP2Iyo4ZSM7/QKcSmuMLDObfpH5fi7RUGmd/rTDf+rut/saiDiQEeVTNgAmJEdAOx0w==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/send/node_modules/ms": {
|
|
||||||
"version": "2.1.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
|
|
||||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/serve-static": {
|
|
||||||
"version": "1.16.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.16.2.tgz",
|
|
||||||
"integrity": "sha512-VqpjJZKadQB/PEbEwvFdO43Ax5dFBZ2UECszz8bQ7pi7wt//PWe1P6MN7eCnjsatYtBT6EuiClbjSWP2WrIoTw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"encodeurl": "~2.0.0",
|
|
||||||
"escape-html": "~1.0.3",
|
|
||||||
"parseurl": "~1.3.3",
|
|
||||||
"send": "0.19.0"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/setprototypeof": {
|
|
||||||
"version": "1.2.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz",
|
|
||||||
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
|
|
||||||
"license": "ISC"
|
|
||||||
},
|
|
||||||
"node_modules/side-channel": {
|
|
||||||
"version": "1.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz",
|
|
||||||
"integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"object-inspect": "^1.13.3",
|
|
||||||
"side-channel-list": "^1.0.0",
|
|
||||||
"side-channel-map": "^1.0.1",
|
|
||||||
"side-channel-weakmap": "^1.0.2"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/side-channel-list": {
|
|
||||||
"version": "1.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.0.tgz",
|
|
||||||
"integrity": "sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"object-inspect": "^1.13.3"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/side-channel-map": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"call-bound": "^1.0.2",
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"get-intrinsic": "^1.2.5",
|
|
||||||
"object-inspect": "^1.13.3"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/side-channel-weakmap": {
|
|
||||||
"version": "1.0.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz",
|
|
||||||
"integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"call-bound": "^1.0.2",
|
|
||||||
"es-errors": "^1.3.0",
|
|
||||||
"get-intrinsic": "^1.2.5",
|
|
||||||
"object-inspect": "^1.13.3",
|
|
||||||
"side-channel-map": "^1.0.1"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4"
|
|
||||||
},
|
|
||||||
"funding": {
|
|
||||||
"url": "https://github.com/sponsors/ljharb"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/statuses": {
|
|
||||||
"version": "2.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz",
|
|
||||||
"integrity": "sha512-RwNA9Z/7PrK06rYLIzFMlaF+l73iwpzsqRIFgbMLbTcLD6cOao82TaWefPXQvB2fOC4AjuYSEndS7N/mTCbkdQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/toidentifier": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/type-is": {
|
|
||||||
"version": "1.6.18",
|
|
||||||
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
|
|
||||||
"integrity": "sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"media-typer": "0.3.0",
|
|
||||||
"mime-types": "~2.1.24"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.6"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/unpipe": {
|
|
||||||
"version": "1.0.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
|
|
||||||
"integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/utils-merge": {
|
|
||||||
"version": "1.0.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
|
|
||||||
"integrity": "sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.4.0"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/vary": {
|
|
||||||
"version": "1.1.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
|
|
||||||
"integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 0.8"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "meta-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"main": "index.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1",
|
|
||||||
"start": "node server.js",
|
|
||||||
"dev": "nodemon server.js"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"description": "",
|
|
||||||
"dependencies": {
|
|
||||||
"axios": "^1.7.9",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"express": "^4.21.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const router = express.Router();
|
|
||||||
const {
|
|
||||||
fetchCampaigns,
|
|
||||||
fetchAccountInsights,
|
|
||||||
updateCampaignBudget,
|
|
||||||
updateCampaignStatus,
|
|
||||||
} = require('../services/meta.service');
|
|
||||||
|
|
||||||
// Get all campaigns with insights
|
|
||||||
router.get('/campaigns', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { since, until } = req.query;
|
|
||||||
|
|
||||||
if (!since || !until) {
|
|
||||||
return res.status(400).json({ error: 'Date range is required (since, until)' });
|
|
||||||
}
|
|
||||||
|
|
||||||
const campaigns = await fetchCampaigns(since, until);
|
|
||||||
res.json(campaigns);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Campaign fetch error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch campaigns',
|
|
||||||
details: error.response?.data?.error?.message || error.message,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Get account insights
|
|
||||||
router.get('/account-insights', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { since, until } = req.query;
|
|
||||||
|
|
||||||
if (!since || !until) {
|
|
||||||
return res.status(400).json({ error: 'Date range is required (since, until)' });
|
|
||||||
}
|
|
||||||
|
|
||||||
const insights = await fetchAccountInsights(since, until);
|
|
||||||
res.json(insights);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Account insights fetch error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch account insights',
|
|
||||||
details: error.response?.data?.error?.message || error.message,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Update campaign budget
|
|
||||||
router.patch('/campaigns/:campaignId/budget', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { campaignId } = req.params;
|
|
||||||
const { budget } = req.body;
|
|
||||||
|
|
||||||
if (!budget) {
|
|
||||||
return res.status(400).json({ error: 'Budget is required' });
|
|
||||||
}
|
|
||||||
|
|
||||||
const result = await updateCampaignBudget(campaignId, budget);
|
|
||||||
res.json(result);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Budget update error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to update campaign budget',
|
|
||||||
details: error.response?.data?.error?.message || error.message,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Update campaign status (pause/unpause)
|
|
||||||
router.post('/campaigns/:campaignId/:action', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { campaignId, action } = req.params;
|
|
||||||
|
|
||||||
if (!['pause', 'unpause'].includes(action)) {
|
|
||||||
return res.status(400).json({ error: 'Invalid action. Use "pause" or "unpause"' });
|
|
||||||
}
|
|
||||||
|
|
||||||
const result = await updateCampaignStatus(campaignId, action);
|
|
||||||
res.json(result);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Status update error:', error);
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to update campaign status',
|
|
||||||
details: error.response?.data?.error?.message || error.message,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = router;
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const cors = require('cors');
|
|
||||||
const path = require('path');
|
|
||||||
require('dotenv').config({
|
|
||||||
path: path.resolve(__dirname, '.env')
|
|
||||||
});
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.PORT || 3005;
|
|
||||||
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Import routes
|
|
||||||
const campaignRoutes = require('./routes/campaigns.routes');
|
|
||||||
|
|
||||||
// Use routes
|
|
||||||
app.use('/api/meta', campaignRoutes);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error(err.stack);
|
|
||||||
res.status(500).json({ error: 'Something went wrong!' });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
|
||||||
console.log(`Meta API server running on port ${port}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = app;
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
const { default: axios } = require('axios');
|
|
||||||
|
|
||||||
const META_API_VERSION = process.env.META_API_VERSION || 'v21.0';
|
|
||||||
const META_API_BASE_URL = `https://graph.facebook.com/${META_API_VERSION}`;
|
|
||||||
const META_ACCESS_TOKEN = process.env.META_ACCESS_TOKEN;
|
|
||||||
const AD_ACCOUNT_ID = process.env.META_AD_ACCOUNT_ID;
|
|
||||||
|
|
||||||
const metaApiRequest = async (endpoint, params = {}) => {
|
|
||||||
try {
|
|
||||||
const response = await axios.get(`${META_API_BASE_URL}/${endpoint}`, {
|
|
||||||
params: {
|
|
||||||
access_token: META_ACCESS_TOKEN,
|
|
||||||
time_zone: 'America/New_York',
|
|
||||||
...params,
|
|
||||||
},
|
|
||||||
});
|
|
||||||
return response.data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Meta API Error:', {
|
|
||||||
message: error.message,
|
|
||||||
response: error.response?.data,
|
|
||||||
endpoint,
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const fetchCampaigns = async (since, until) => {
|
|
||||||
const campaigns = await metaApiRequest(`act_${AD_ACCOUNT_ID}/campaigns`, {
|
|
||||||
fields: [
|
|
||||||
'id',
|
|
||||||
'name',
|
|
||||||
'status',
|
|
||||||
'objective',
|
|
||||||
'daily_budget',
|
|
||||||
'lifetime_budget',
|
|
||||||
'adsets{daily_budget,lifetime_budget}',
|
|
||||||
`insights.time_range({'since':'${since}','until':'${until}'}).level(campaign){
|
|
||||||
spend,
|
|
||||||
impressions,
|
|
||||||
clicks,
|
|
||||||
ctr,
|
|
||||||
reach,
|
|
||||||
frequency,
|
|
||||||
cpm,
|
|
||||||
cpc,
|
|
||||||
actions,
|
|
||||||
action_values,
|
|
||||||
cost_per_action_type
|
|
||||||
}`,
|
|
||||||
].join(','),
|
|
||||||
limit: 100,
|
|
||||||
});
|
|
||||||
|
|
||||||
return campaigns.data.filter(c => c.insights?.data?.[0]?.spend > 0);
|
|
||||||
};
|
|
||||||
|
|
||||||
const fetchAccountInsights = async (since, until) => {
|
|
||||||
const accountInsights = await metaApiRequest(`act_${AD_ACCOUNT_ID}/insights`, {
|
|
||||||
fields: 'reach,spend,impressions,clicks,ctr,cpm,actions,action_values',
|
|
||||||
time_range: JSON.stringify({ since, until }),
|
|
||||||
});
|
|
||||||
|
|
||||||
return accountInsights.data[0] || null;
|
|
||||||
};
|
|
||||||
|
|
||||||
const updateCampaignBudget = async (campaignId, budget) => {
|
|
||||||
try {
|
|
||||||
const response = await axios.post(`${META_API_BASE_URL}/${campaignId}`, {
|
|
||||||
access_token: META_ACCESS_TOKEN,
|
|
||||||
daily_budget: budget * 100, // Convert to cents
|
|
||||||
});
|
|
||||||
return response.data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Update campaign budget error:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const updateCampaignStatus = async (campaignId, action) => {
|
|
||||||
try {
|
|
||||||
const status = action === 'pause' ? 'PAUSED' : 'ACTIVE';
|
|
||||||
const response = await axios.post(`${META_API_BASE_URL}/${campaignId}`, {
|
|
||||||
access_token: META_ACCESS_TOKEN,
|
|
||||||
status,
|
|
||||||
});
|
|
||||||
return response.data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Update campaign status error:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
module.exports = {
|
|
||||||
fetchCampaigns,
|
|
||||||
fetchAccountInsights,
|
|
||||||
updateCampaignBudget,
|
|
||||||
updateCampaignStatus,
|
|
||||||
};
|
|
||||||
+2917
-2
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,29 @@
|
|||||||
{
|
{
|
||||||
|
"name": "dashboard-server",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"description": "Merged ESM dashboard server (klaviyo + meta + google-analytics + typeform). Phase 4 of CONSOLIDATION_PLAN.md.",
|
||||||
|
"main": "server.js",
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"start": "node server.js",
|
||||||
|
"dev": "nodemon server.js"
|
||||||
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"dotenv": "^16.4.7"
|
"@google-analytics/data": "^4.0.0",
|
||||||
|
"axios": "^1.7.9",
|
||||||
|
"cors": "^2.8.5",
|
||||||
|
"dotenv": "^16.4.7",
|
||||||
|
"express": "^4.21.2",
|
||||||
|
"ioredis": "^5.4.1",
|
||||||
|
"jsonwebtoken": "^9.0.2",
|
||||||
|
"lodash": "^4.17.21",
|
||||||
|
"luxon": "^3.5.0",
|
||||||
|
"node-fetch": "^3.3.2",
|
||||||
|
"pg": "^8.18.0",
|
||||||
|
"pino": "^9.5.0",
|
||||||
|
"pino-http": "^10.3.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"nodemon": "^3.0.2"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
// Google Analytics router — ESM conversion of google-server/routes/analytics.routes.js.
|
||||||
|
// All routes are read-only — authenticated-only is sufficient; no extra permission.
|
||||||
|
// google_write is reserved for future write endpoints (per migration 005).
|
||||||
|
|
||||||
|
import express from 'express';
|
||||||
|
import { AnalyticsService } from '../../services/google/analytics.service.js';
|
||||||
|
|
||||||
|
export function createGoogleRouter({ redis }) {
|
||||||
|
const router = express.Router();
|
||||||
|
const service = new AnalyticsService(redis);
|
||||||
|
|
||||||
|
router.get('/metrics', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { startDate = '7daysAgo' } = req.query;
|
||||||
|
const data = await service.getBasicMetrics(startDate);
|
||||||
|
res.json({ success: true, data });
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Metrics error:', { startDate: req.query.startDate, error: error.message });
|
||||||
|
res.status(500).json({ success: false, error: 'Failed to fetch metrics', details: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.get('/realtime/basic', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const data = await service.getRealTimeBasicData();
|
||||||
|
res.json({ success: true, data });
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Realtime basic error:', { error: error.message });
|
||||||
|
res.status(500).json({ success: false, error: 'Failed to fetch realtime basic data', details: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.get('/realtime/detailed', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const data = await service.getRealTimeDetailedData();
|
||||||
|
res.json({ success: true, data });
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Realtime detailed error:', { error: error.message });
|
||||||
|
res.status(500).json({ success: false, error: 'Failed to fetch realtime detailed data', details: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.get('/user-behavior', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { timeRange = '30' } = req.query;
|
||||||
|
const data = await service.getUserBehavior(timeRange);
|
||||||
|
res.json({ success: true, data });
|
||||||
|
} catch (error) {
|
||||||
|
console.error('User behavior error:', { timeRange: req.query.timeRange, error: error.message });
|
||||||
|
res.status(500).json({ success: false, error: 'Failed to fetch user behavior data', details: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return router;
|
||||||
|
}
|
||||||
+4
-4
@@ -1,11 +1,11 @@
|
|||||||
import express from 'express';
|
import express from 'express';
|
||||||
import { CampaignsService } from '../services/campaigns.service.js';
|
import { CampaignsService } from '../../services/klaviyo/campaigns.service.js';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
|
|
||||||
export function createCampaignsRouter(apiKey, apiRevision) {
|
export function createCampaignsRouter(apiKey, apiRevision, redis) {
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const timeManager = new TimeManager();
|
const timeManager = new TimeManager();
|
||||||
const campaignsService = new CampaignsService(apiKey, apiRevision);
|
const campaignsService = new CampaignsService(apiKey, apiRevision, redis);
|
||||||
|
|
||||||
// Get campaigns with optional filtering
|
// Get campaigns with optional filtering
|
||||||
router.get('/', async (req, res) => {
|
router.get('/', async (req, res) => {
|
||||||
+11
-6
@@ -1,7 +1,8 @@
|
|||||||
import express from 'express';
|
import express from 'express';
|
||||||
import { EventsService } from '../services/events.service.js';
|
import { EventsService } from '../../services/klaviyo/events.service.js';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
import { RedisService } from '../services/redis.service.js';
|
import { RedisService } from '../../services/klaviyo/redis.service.js';
|
||||||
|
import { requirePermission } from '../../../shared/auth/middleware.js';
|
||||||
|
|
||||||
// Import METRIC_IDS from events service
|
// Import METRIC_IDS from events service
|
||||||
const METRIC_IDS = {
|
const METRIC_IDS = {
|
||||||
@@ -13,11 +14,15 @@ const METRIC_IDS = {
|
|||||||
PAYMENT_REFUNDED: 'R7XUYh'
|
PAYMENT_REFUNDED: 'R7XUYh'
|
||||||
};
|
};
|
||||||
|
|
||||||
export function createEventsRouter(apiKey, apiRevision) {
|
export function createEventsRouter(apiKey, apiRevision, redis) {
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const timeManager = new TimeManager();
|
const timeManager = new TimeManager();
|
||||||
const eventsService = new EventsService(apiKey, apiRevision);
|
const eventsService = new EventsService(apiKey, apiRevision, redis);
|
||||||
const redisService = new RedisService();
|
const redisService = new RedisService(redis);
|
||||||
|
|
||||||
|
// Phase 6.2: clearCache is operational maintenance — requires klaviyo_admin.
|
||||||
|
// Mounted as path-level middleware so the existing POST handler below stays untouched.
|
||||||
|
router.use('/clearCache', requirePermission('klaviyo_admin'));
|
||||||
|
|
||||||
// Get events with optional filtering
|
// Get events with optional filtering
|
||||||
router.get('/', async (req, res) => {
|
router.get('/', async (req, res) => {
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
// Klaviyo router factory. Phase 4 merge: takes the injected redis client and
|
||||||
|
// the env-resolved API key/revision, returns the mounted /api/klaviyo router
|
||||||
|
// (matches Caddy proxy path; no rewrite needed).
|
||||||
|
|
||||||
|
import express from 'express';
|
||||||
|
import rateLimit from 'express-rate-limit';
|
||||||
|
import { requirePermission } from '../../../shared/auth/middleware.js';
|
||||||
|
|
||||||
|
import { createEventsRouter } from './events.routes.js';
|
||||||
|
import { createMetricsRouter } from './metrics.routes.js';
|
||||||
|
import { createCampaignsRouter } from './campaigns.routes.js';
|
||||||
|
import { createReportingRouter } from './reporting.routes.js';
|
||||||
|
|
||||||
|
export function createKlaviyoRouter({ redis }) {
|
||||||
|
const apiKey = process.env.KLAVIYO_API_KEY;
|
||||||
|
const apiRevision = process.env.KLAVIYO_API_REVISION || '2024-02-15';
|
||||||
|
|
||||||
|
if (!apiKey) {
|
||||||
|
// Loud at startup; the routes themselves will 500 on every call without it.
|
||||||
|
console.warn('[klaviyo] KLAVIYO_API_KEY not set — Klaviyo endpoints will fail');
|
||||||
|
}
|
||||||
|
|
||||||
|
const router = express.Router();
|
||||||
|
|
||||||
|
// Phase 4 carryover from klaviyo-server: throttle the heavy /reporting/campaign-values-reports
|
||||||
|
// endpoint. authenticate() already runs upstream so we don't add a per-user limiter here.
|
||||||
|
const reportingLimiter = rateLimit({
|
||||||
|
windowMs: 10 * 60 * 1000,
|
||||||
|
max: 10,
|
||||||
|
message: 'Too many requests to reporting endpoint, please try again later',
|
||||||
|
keyGenerator: (req) => `${req.ip}-klaviyo-reporting`,
|
||||||
|
skip: (req) => !req.path.includes('campaign-values-reports'),
|
||||||
|
standardHeaders: true,
|
||||||
|
legacyHeaders: false,
|
||||||
|
});
|
||||||
|
router.use('/reporting', reportingLimiter);
|
||||||
|
|
||||||
|
router.use('/events', createEventsRouter(apiKey, apiRevision, redis));
|
||||||
|
router.use('/metrics', createMetricsRouter(apiKey, apiRevision));
|
||||||
|
router.use('/campaigns', createCampaignsRouter(apiKey, apiRevision, redis));
|
||||||
|
router.use('/reporting', createReportingRouter(apiKey, apiRevision, redis));
|
||||||
|
|
||||||
|
return router;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-exported so the dashboard server / future tests can attach the
|
||||||
|
// klaviyo_admin gate without reaching into the events router file.
|
||||||
|
export { requirePermission };
|
||||||
+3
-4
@@ -1,9 +1,8 @@
|
|||||||
import express from 'express';
|
import express from 'express';
|
||||||
import { MetricsService } from '../services/metrics.service.js';
|
import { MetricsService } from '../../services/klaviyo/metrics.service.js';
|
||||||
|
|
||||||
const router = express.Router();
|
export function createMetricsRouter(apiKey, apiRevision) {
|
||||||
|
const router = express.Router();
|
||||||
export function createMetricsRoutes(apiKey, apiRevision) {
|
|
||||||
const metricsService = new MetricsService(apiKey, apiRevision);
|
const metricsService = new MetricsService(apiKey, apiRevision);
|
||||||
|
|
||||||
// Get all metrics
|
// Get all metrics
|
||||||
+4
-4
@@ -1,10 +1,10 @@
|
|||||||
import express from 'express';
|
import express from 'express';
|
||||||
import { ReportingService } from '../services/reporting.service.js';
|
import { ReportingService } from '../../services/klaviyo/reporting.service.js';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
|
|
||||||
export function createReportingRouter(apiKey, apiRevision) {
|
export function createReportingRouter(apiKey, apiRevision, redis) {
|
||||||
const router = express.Router();
|
const router = express.Router();
|
||||||
const reportingService = new ReportingService(apiKey, apiRevision);
|
const reportingService = new ReportingService(apiKey, apiRevision, redis);
|
||||||
const timeManager = new TimeManager();
|
const timeManager = new TimeManager();
|
||||||
|
|
||||||
// Get campaign reports by time range
|
// Get campaign reports by time range
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
// Meta router factory — ESM conversion of meta-server/routes/campaigns.routes.js.
|
||||||
|
// Phase 6.2: mutations (PATCH /campaigns/:id/budget, POST /campaigns/:id/:action)
|
||||||
|
// require the `meta_write` permission. Reads (GET) stay authenticated-only.
|
||||||
|
|
||||||
|
import express from 'express';
|
||||||
|
import { requirePermission } from '../../../shared/auth/middleware.js';
|
||||||
|
import {
|
||||||
|
fetchCampaigns,
|
||||||
|
fetchAccountInsights,
|
||||||
|
updateCampaignBudget,
|
||||||
|
updateCampaignStatus,
|
||||||
|
} from '../../services/meta/meta.service.js';
|
||||||
|
|
||||||
|
export function createMetaRouter() {
|
||||||
|
const router = express.Router();
|
||||||
|
|
||||||
|
// Reads — authenticated-only
|
||||||
|
router.get('/campaigns', async (req, res, next) => {
|
||||||
|
try {
|
||||||
|
const { since, until } = req.query;
|
||||||
|
if (!since || !until) {
|
||||||
|
return res.status(400).json({ error: 'Date range is required (since, until)' });
|
||||||
|
}
|
||||||
|
const campaigns = await fetchCampaigns(since, until);
|
||||||
|
res.json(campaigns);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Campaign fetch error:', error);
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to fetch campaigns',
|
||||||
|
details: error.response?.data?.error?.message || error.message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.get('/account-insights', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { since, until } = req.query;
|
||||||
|
if (!since || !until) {
|
||||||
|
return res.status(400).json({ error: 'Date range is required (since, until)' });
|
||||||
|
}
|
||||||
|
const insights = await fetchAccountInsights(since, until);
|
||||||
|
res.json(insights);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Account insights fetch error:', error);
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to fetch account insights',
|
||||||
|
details: error.response?.data?.error?.message || error.message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Writes — meta_write
|
||||||
|
router.patch('/campaigns/:campaignId/budget', requirePermission('meta_write'), async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { campaignId } = req.params;
|
||||||
|
const { budget } = req.body;
|
||||||
|
if (!budget) {
|
||||||
|
return res.status(400).json({ error: 'Budget is required' });
|
||||||
|
}
|
||||||
|
const result = await updateCampaignBudget(campaignId, budget);
|
||||||
|
res.json(result);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Budget update error:', error);
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to update campaign budget',
|
||||||
|
details: error.response?.data?.error?.message || error.message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.post('/campaigns/:campaignId/:action', requirePermission('meta_write'), async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { campaignId, action } = req.params;
|
||||||
|
if (!['pause', 'unpause'].includes(action)) {
|
||||||
|
return res.status(400).json({ error: 'Invalid action. Use "pause" or "unpause"' });
|
||||||
|
}
|
||||||
|
const result = await updateCampaignStatus(campaignId, action);
|
||||||
|
res.json(result);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Status update error:', error);
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to update campaign status',
|
||||||
|
details: error.response?.data?.error?.message || error.message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return router;
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
// Typeform router — ESM conversion of typeform-server/routes/typeform.routes.js.
|
||||||
|
// All routes read-only — authenticated-only is sufficient; typeform_write reserved
|
||||||
|
// for future write endpoints (per migration 005).
|
||||||
|
|
||||||
|
import express from 'express';
|
||||||
|
import { TypeformService } from '../../services/typeform/typeform.service.js';
|
||||||
|
|
||||||
|
export function createTypeformRouter({ redis }) {
|
||||||
|
const router = express.Router();
|
||||||
|
const typeform = new TypeformService(redis);
|
||||||
|
|
||||||
|
router.get('/forms/:formId/responses', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { formId } = req.params;
|
||||||
|
const filters = req.query;
|
||||||
|
if (!formId) {
|
||||||
|
return res.status(400).json({ error: 'Missing form ID', details: 'The form ID parameter is required' });
|
||||||
|
}
|
||||||
|
const data = await typeform.getFormResponsesWithFilters(formId, filters);
|
||||||
|
if (!data) {
|
||||||
|
return res.status(404).json({ error: 'No data found', details: `No responses found for form ${formId}` });
|
||||||
|
}
|
||||||
|
res.json(data);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Form responses error:', {
|
||||||
|
formId: req.params.formId,
|
||||||
|
filters: req.query,
|
||||||
|
error: error.message,
|
||||||
|
response: error.response?.data,
|
||||||
|
});
|
||||||
|
if (error.response?.status === 401) {
|
||||||
|
return res.status(401).json({ error: 'Authentication failed', details: 'Invalid Typeform API credentials' });
|
||||||
|
}
|
||||||
|
if (error.response?.status === 404) {
|
||||||
|
return res.status(404).json({ error: 'Not found', details: `Form '${req.params.formId}' not found` });
|
||||||
|
}
|
||||||
|
if (error.response?.status === 400) {
|
||||||
|
return res.status(400).json({
|
||||||
|
error: 'Invalid request',
|
||||||
|
details: error.response?.data?.message || 'The request was invalid',
|
||||||
|
data: error.response?.data,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to fetch form responses',
|
||||||
|
details: error.response?.data?.message || error.message,
|
||||||
|
data: error.response?.data,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
router.get('/forms/:formId/insights', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { formId } = req.params;
|
||||||
|
if (!formId) {
|
||||||
|
return res.status(400).json({ error: 'Missing form ID', details: 'The form ID parameter is required' });
|
||||||
|
}
|
||||||
|
const data = await typeform.getFormInsights(formId);
|
||||||
|
if (!data) {
|
||||||
|
return res.status(404).json({ error: 'No data found', details: `No insights found for form ${formId}` });
|
||||||
|
}
|
||||||
|
res.json(data);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Form insights error:', {
|
||||||
|
formId: req.params.formId,
|
||||||
|
error: error.message,
|
||||||
|
response: error.response?.data,
|
||||||
|
});
|
||||||
|
if (error.response?.status === 401) {
|
||||||
|
return res.status(401).json({ error: 'Authentication failed', details: 'Invalid Typeform API credentials' });
|
||||||
|
}
|
||||||
|
if (error.response?.status === 404) {
|
||||||
|
return res.status(404).json({ error: 'Not found', details: `Form '${req.params.formId}' not found` });
|
||||||
|
}
|
||||||
|
res.status(500).json({
|
||||||
|
error: 'Failed to fetch form insights',
|
||||||
|
details: error.response?.data?.message || error.message,
|
||||||
|
data: error.response?.data,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return router;
|
||||||
|
}
|
||||||
@@ -0,0 +1,132 @@
|
|||||||
|
// dashboard-server — Phase 4 of CONSOLIDATION_PLAN.md.
|
||||||
|
// Merges the four per-vendor PM2 apps (klaviyo, meta, google-analytics, typeform)
|
||||||
|
// into a single ESM service on DASHBOARD_PORT (default 3015).
|
||||||
|
//
|
||||||
|
// Mount points (matches Caddy proxy paths):
|
||||||
|
// /api/klaviyo/* → routes/klaviyo (was klaviyo-server :3004)
|
||||||
|
// /api/meta/* → routes/meta (was meta-server :3005)
|
||||||
|
// /api/dashboard-analytics/* → routes/google (was google-server :3007 via Caddy /api/analytics rewrite)
|
||||||
|
// /api/typeform/* → routes/typeform (was typeform-server :3008)
|
||||||
|
//
|
||||||
|
// Shared infrastructure (Phase 2 + Phase 6):
|
||||||
|
// - shared/auth/middleware.js authenticate() guards /api/* (Phase 6.1/6.2 — second line of defense)
|
||||||
|
// - shared/cors/policy.js explicit allowed-origins list (Phase 6.6)
|
||||||
|
// - shared/logging/request-log.js pino-http, Authorization/Cookie redacted (Phase 6.5/6.9)
|
||||||
|
// - shared/errors/handler.js consistent error envelope, no leak in prod
|
||||||
|
// - shared/db/pg.js / shared/db/redis.js one Pool + one ioredis client for all vendors
|
||||||
|
//
|
||||||
|
// Per-route permission gates (Phase 6.2):
|
||||||
|
// - meta_write PATCH/POST mutations to Meta campaigns
|
||||||
|
// - klaviyo_admin POST /api/klaviyo/events/clearCache (operational maintenance)
|
||||||
|
// Read-only Google + Typeform endpoints stay authenticated-only.
|
||||||
|
|
||||||
|
import { config as loadEnv } from 'dotenv';
|
||||||
|
import express from 'express';
|
||||||
|
import cors from 'cors';
|
||||||
|
import path from 'node:path';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
import { authenticate } from '../shared/auth/middleware.js';
|
||||||
|
import { corsOptions } from '../shared/cors/policy.js';
|
||||||
|
import { createPool } from '../shared/db/pg.js';
|
||||||
|
import { createRedis } from '../shared/db/redis.js';
|
||||||
|
import { errorHandler } from '../shared/errors/handler.js';
|
||||||
|
import { logger } from '../shared/logging/logger.js';
|
||||||
|
import { requestLog } from '../shared/logging/request-log.js';
|
||||||
|
|
||||||
|
import { createKlaviyoRouter } from './routes/klaviyo/index.js';
|
||||||
|
import { createMetaRouter } from './routes/meta/index.js';
|
||||||
|
import { createGoogleRouter } from './routes/google/index.js';
|
||||||
|
import { createTypeformRouter } from './routes/typeform/index.js';
|
||||||
|
|
||||||
|
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||||
|
|
||||||
|
// Layer envs: shared inventory .env wins on collisions (security-critical vars come
|
||||||
|
// from one place); vendor-specific keys come from the per-service .env.
|
||||||
|
//
|
||||||
|
// dotenv defaults to override:false so the first file loaded wins. Order matters.
|
||||||
|
const sharedEnvPath = '/var/www/inventory/.env';
|
||||||
|
const dashboardEnvPath = path.resolve(__dirname, '.env');
|
||||||
|
|
||||||
|
if (fs.existsSync(sharedEnvPath)) loadEnv({ path: sharedEnvPath });
|
||||||
|
if (fs.existsSync(dashboardEnvPath)) loadEnv({ path: dashboardEnvPath });
|
||||||
|
|
||||||
|
// Phase 6.4 — refuse to start without JWT_SECRET. Without it authenticate() falls
|
||||||
|
// back to res.status(401) on every request and the service is useless anyway.
|
||||||
|
if (!process.env.JWT_SECRET) {
|
||||||
|
logger.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const app = express();
|
||||||
|
const PORT = Number(process.env.DASHBOARD_PORT) || 3015;
|
||||||
|
|
||||||
|
// Trust X-Forwarded-* only when the immediate hop is loopback (Caddy on the same
|
||||||
|
// host). Required for the KIOSK_IPS bypass in shared/auth/middleware.js to see
|
||||||
|
// real client IPs instead of 127.0.0.1.
|
||||||
|
app.set('trust proxy', 'loopback');
|
||||||
|
|
||||||
|
// Single Postgres pool — used by authenticate() to load user permissions.
|
||||||
|
// All four vendors share this pool (auth lookups are the only DB hits at runtime).
|
||||||
|
const pool = createPool('DB');
|
||||||
|
|
||||||
|
// Single ioredis client shared across all vendors. lazyConnect:true means the
|
||||||
|
// first .get/.set triggers the actual connect — keeps startup non-blocking even
|
||||||
|
// if Redis is temporarily unavailable, and aligns with shared/db/redis.js defaults.
|
||||||
|
const redis = createRedis();
|
||||||
|
|
||||||
|
app.use(requestLog());
|
||||||
|
app.use(cors(corsOptions));
|
||||||
|
app.use(express.json({ limit: '10mb' }));
|
||||||
|
|
||||||
|
// Phase 6.1/6.2: every /api request requires a valid JWT. authenticate() also
|
||||||
|
// loads user permissions, which the per-route requirePermission() checks rely on.
|
||||||
|
app.use('/api', authenticate({ pool, secret: process.env.JWT_SECRET }));
|
||||||
|
|
||||||
|
app.use('/api/klaviyo', createKlaviyoRouter({ redis }));
|
||||||
|
app.use('/api/meta', createMetaRouter());
|
||||||
|
// Note: frontend calls /api/dashboard-analytics (Caddy used to rewrite it to
|
||||||
|
// /api/analytics for the standalone google-server). Mount at the public path so
|
||||||
|
// Caddy can drop the rewrite — see Caddyfile.proposed.
|
||||||
|
app.use('/api/dashboard-analytics', createGoogleRouter({ redis }));
|
||||||
|
app.use('/api/typeform', createTypeformRouter({ redis }));
|
||||||
|
|
||||||
|
app.get('/health', (req, res) => {
|
||||||
|
res.json({
|
||||||
|
status: 'ok',
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
service: 'dashboard-server',
|
||||||
|
redis: redis.status,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use(errorHandler);
|
||||||
|
|
||||||
|
// Connect Redis up front so the first request doesn't pay the connect cost.
|
||||||
|
// Failures here are non-fatal — vendors degrade to cache-miss → upstream fetch.
|
||||||
|
redis.connect().catch((err) => {
|
||||||
|
logger.error({ err: { message: err.message, code: err.code } }, 'redis lazy-connect failed');
|
||||||
|
});
|
||||||
|
|
||||||
|
const server = app.listen(PORT, '0.0.0.0', () => {
|
||||||
|
logger.info({ port: PORT, mode: process.env.NODE_ENV || 'development' }, 'dashboard-server listening');
|
||||||
|
});
|
||||||
|
|
||||||
|
const shutdown = async (signal) => {
|
||||||
|
logger.info({ signal }, 'dashboard-server shutting down');
|
||||||
|
server.close();
|
||||||
|
try { await redis.quit(); } catch { /* ignore */ }
|
||||||
|
try { await pool.end(); } catch { /* ignore */ }
|
||||||
|
process.exit(0);
|
||||||
|
};
|
||||||
|
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||||
|
process.on('SIGINT', () => shutdown('SIGINT'));
|
||||||
|
|
||||||
|
process.on('uncaughtException', (err) => {
|
||||||
|
logger.error({ err: { message: err.message, stack: err.stack } }, 'uncaughtException');
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
process.on('unhandledRejection', (reason) => {
|
||||||
|
logger.error({ reason }, 'unhandledRejection');
|
||||||
|
});
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
// Google Analytics (GA4) service — ESM conversion of google-server/services/analytics.service.js.
|
||||||
|
// Phase 4: accepts injected ioredis client (was self-constructing node-redis v4 before).
|
||||||
|
// node-redis v4 set syntax `{ EX: 300 }` is translated to ioredis `setex(key, 300, val)`.
|
||||||
|
|
||||||
|
import { BetaAnalyticsDataClient } from '@google-analytics/data';
|
||||||
|
|
||||||
|
const CACHE_DURATIONS = {
|
||||||
|
REALTIME_BASIC: 60,
|
||||||
|
REALTIME_DETAILED: 300,
|
||||||
|
BASIC_METRICS: 3600,
|
||||||
|
USER_BEHAVIOR: 3600,
|
||||||
|
};
|
||||||
|
|
||||||
|
export class AnalyticsService {
|
||||||
|
constructor(redis) {
|
||||||
|
if (!redis) {
|
||||||
|
throw new Error('AnalyticsService requires an ioredis client (Phase 4: injected)');
|
||||||
|
}
|
||||||
|
this.redis = redis;
|
||||||
|
|
||||||
|
const credentials = process.env.GOOGLE_APPLICATION_CREDENTIALS_JSON;
|
||||||
|
this.analyticsClient = new BetaAnalyticsDataClient({
|
||||||
|
credentials: typeof credentials === 'string' ? JSON.parse(credentials) : credentials,
|
||||||
|
});
|
||||||
|
this.propertyId = process.env.GA_PROPERTY_ID;
|
||||||
|
}
|
||||||
|
|
||||||
|
get _redisReady() {
|
||||||
|
return this.redis.status === 'ready' || this.redis.status === 'connect';
|
||||||
|
}
|
||||||
|
|
||||||
|
async _cacheGet(key) {
|
||||||
|
if (!this._redisReady) return null;
|
||||||
|
try {
|
||||||
|
const raw = await this.redis.get(key);
|
||||||
|
return raw ? JSON.parse(raw) : null;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn('[AnalyticsService] cache get failed:', err.message);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async _cacheSet(key, value, ttlSec) {
|
||||||
|
if (!this._redisReady) return;
|
||||||
|
try {
|
||||||
|
await this.redis.setex(key, ttlSec, JSON.stringify(value));
|
||||||
|
} catch (err) {
|
||||||
|
console.warn('[AnalyticsService] cache set failed:', err.message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async getBasicMetrics(startDate = '7daysAgo') {
|
||||||
|
const cacheKey = `analytics:basic_metrics:${startDate}`;
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const [response] = await this.analyticsClient.runReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dateRanges: [{ startDate, endDate: 'today' }],
|
||||||
|
dimensions: [{ name: 'date' }],
|
||||||
|
metrics: [
|
||||||
|
{ name: 'activeUsers' },
|
||||||
|
{ name: 'newUsers' },
|
||||||
|
{ name: 'averageSessionDuration' },
|
||||||
|
{ name: 'screenPageViews' },
|
||||||
|
{ name: 'bounceRate' },
|
||||||
|
{ name: 'conversions' },
|
||||||
|
],
|
||||||
|
returnPropertyQuota: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
await this._cacheSet(cacheKey, response, CACHE_DURATIONS.BASIC_METRICS);
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getRealTimeBasicData() {
|
||||||
|
const cacheKey = 'analytics:realtime:basic';
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const [userResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
metrics: [{ name: 'activeUsers' }],
|
||||||
|
returnPropertyQuota: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const [fiveMinResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
metrics: [{ name: 'activeUsers' }],
|
||||||
|
minuteRanges: [{ startMinutesAgo: 5, endMinutesAgo: 0 }],
|
||||||
|
});
|
||||||
|
|
||||||
|
const [timeSeriesResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dimensions: [{ name: 'minutesAgo' }],
|
||||||
|
metrics: [{ name: 'activeUsers' }],
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = {
|
||||||
|
userResponse,
|
||||||
|
fiveMinResponse,
|
||||||
|
timeSeriesResponse,
|
||||||
|
quotaInfo: {
|
||||||
|
projectHourly: userResponse.propertyQuota.tokensPerProjectPerHour,
|
||||||
|
daily: userResponse.propertyQuota.tokensPerDay,
|
||||||
|
serverErrors: userResponse.propertyQuota.serverErrorsPerProjectPerHour,
|
||||||
|
thresholdedRequests: userResponse.propertyQuota.potentiallyThresholdedRequestsPerHour,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await this._cacheSet(cacheKey, response, CACHE_DURATIONS.REALTIME_BASIC);
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getRealTimeDetailedData() {
|
||||||
|
const cacheKey = 'analytics:realtime:detailed';
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const [pageResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dimensions: [{ name: 'unifiedScreenName' }],
|
||||||
|
metrics: [{ name: 'screenPageViews' }],
|
||||||
|
orderBy: [{ metric: { metricName: 'screenPageViews' }, desc: true }],
|
||||||
|
limit: 25,
|
||||||
|
});
|
||||||
|
|
||||||
|
const [eventResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dimensions: [{ name: 'eventName' }],
|
||||||
|
metrics: [{ name: 'eventCount' }],
|
||||||
|
orderBy: [{ metric: { metricName: 'eventCount' }, desc: true }],
|
||||||
|
limit: 25,
|
||||||
|
});
|
||||||
|
|
||||||
|
const [deviceResponse] = await this.analyticsClient.runRealtimeReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dimensions: [{ name: 'deviceCategory' }],
|
||||||
|
metrics: [{ name: 'activeUsers' }],
|
||||||
|
orderBy: [{ metric: { metricName: 'activeUsers' }, desc: true }],
|
||||||
|
limit: 10,
|
||||||
|
returnPropertyQuota: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = {
|
||||||
|
pageResponse,
|
||||||
|
eventResponse,
|
||||||
|
sourceResponse: deviceResponse,
|
||||||
|
};
|
||||||
|
|
||||||
|
await this._cacheSet(cacheKey, response, CACHE_DURATIONS.REALTIME_DETAILED);
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getUserBehavior(timeRange = '30') {
|
||||||
|
const cacheKey = `analytics:user_behavior:${timeRange}`;
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const [pageResponse] = await this.analyticsClient.runReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
||||||
|
dimensions: [{ name: 'pagePath' }],
|
||||||
|
metrics: [
|
||||||
|
{ name: 'screenPageViews' },
|
||||||
|
{ name: 'averageSessionDuration' },
|
||||||
|
{ name: 'bounceRate' },
|
||||||
|
{ name: 'sessions' },
|
||||||
|
],
|
||||||
|
orderBy: [{ metric: { metricName: 'screenPageViews' }, desc: true }],
|
||||||
|
limit: 25,
|
||||||
|
});
|
||||||
|
|
||||||
|
const [deviceResponse] = await this.analyticsClient.runReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
||||||
|
dimensions: [{ name: 'deviceCategory' }],
|
||||||
|
metrics: [{ name: 'screenPageViews' }, { name: 'sessions' }],
|
||||||
|
});
|
||||||
|
|
||||||
|
const [sourceResponse] = await this.analyticsClient.runReport({
|
||||||
|
property: `properties/${this.propertyId}`,
|
||||||
|
dateRanges: [{ startDate: `${timeRange}daysAgo`, endDate: 'today' }],
|
||||||
|
dimensions: [{ name: 'sessionSource' }],
|
||||||
|
metrics: [{ name: 'sessions' }, { name: 'conversions' }],
|
||||||
|
orderBy: [{ metric: { metricName: 'sessions' }, desc: true }],
|
||||||
|
limit: 25,
|
||||||
|
returnPropertyQuota: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = { pageResponse, deviceResponse, sourceResponse };
|
||||||
|
await this._cacheSet(cacheKey, response, CACHE_DURATIONS.USER_BEHAVIOR);
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
}
|
||||||
+3
-3
@@ -1,14 +1,14 @@
|
|||||||
import fetch from 'node-fetch';
|
import fetch from 'node-fetch';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
import { RedisService } from './redis.service.js';
|
import { RedisService } from './redis.service.js';
|
||||||
|
|
||||||
export class CampaignsService {
|
export class CampaignsService {
|
||||||
constructor(apiKey, apiRevision) {
|
constructor(apiKey, apiRevision, redis) {
|
||||||
this.apiKey = apiKey;
|
this.apiKey = apiKey;
|
||||||
this.apiRevision = apiRevision;
|
this.apiRevision = apiRevision;
|
||||||
this.baseUrl = 'https://a.klaviyo.com/api';
|
this.baseUrl = 'https://a.klaviyo.com/api';
|
||||||
this.timeManager = new TimeManager();
|
this.timeManager = new TimeManager();
|
||||||
this.redisService = new RedisService();
|
this.redisService = new RedisService(redis);
|
||||||
}
|
}
|
||||||
|
|
||||||
async getCampaigns(params = {}) {
|
async getCampaigns(params = {}) {
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
import fetch from 'node-fetch';
|
import fetch from 'node-fetch';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
import { RedisService } from './redis.service.js';
|
import { RedisService } from './redis.service.js';
|
||||||
import _ from 'lodash';
|
import _ from 'lodash';
|
||||||
|
|
||||||
@@ -13,12 +13,12 @@ const METRIC_IDS = {
|
|||||||
};
|
};
|
||||||
|
|
||||||
export class EventsService {
|
export class EventsService {
|
||||||
constructor(apiKey, apiRevision) {
|
constructor(apiKey, apiRevision, redis) {
|
||||||
this.apiKey = apiKey;
|
this.apiKey = apiKey;
|
||||||
this.apiRevision = apiRevision;
|
this.apiRevision = apiRevision;
|
||||||
this.baseUrl = 'https://a.klaviyo.com/api';
|
this.baseUrl = 'https://a.klaviyo.com/api';
|
||||||
this.timeManager = new TimeManager();
|
this.timeManager = new TimeManager();
|
||||||
this.redisService = new RedisService();
|
this.redisService = new RedisService(redis);
|
||||||
}
|
}
|
||||||
|
|
||||||
async getEvents(params = {}) {
|
async getEvents(params = {}) {
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
// Klaviyo cache wrapper. Was a self-instantiating ioredis client per service in
|
||||||
|
// the standalone klaviyo-server; now accepts an injected client so the merged
|
||||||
|
// dashboard-server shares one connection across all vendors (Phase 4).
|
||||||
|
//
|
||||||
|
// Public surface kept identical to the original so the ~3K LOC of klaviyo
|
||||||
|
// service code (events/campaigns/reporting) needs no other changes:
|
||||||
|
// - get(key)
|
||||||
|
// - set(key, data, ttl)
|
||||||
|
// - _getCacheKey(type, params)
|
||||||
|
// - _getTTL(timeRange)
|
||||||
|
// - getEventData(type, params) / cacheEventData(type, params, data)
|
||||||
|
// - clearCache(params)
|
||||||
|
//
|
||||||
|
// Reads short-circuit to null when the client isn't ready; writes are no-ops.
|
||||||
|
// Same "Redis hiccup → fall through to upstream" behavior as before.
|
||||||
|
|
||||||
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
|
|
||||||
|
export class RedisService {
|
||||||
|
constructor(redis) {
|
||||||
|
if (!redis) {
|
||||||
|
throw new Error('RedisService requires an ioredis client (Phase 4: injected, no longer self-constructed)');
|
||||||
|
}
|
||||||
|
this.client = redis;
|
||||||
|
this.timeManager = new TimeManager();
|
||||||
|
this.DEFAULT_TTL = 5 * 60;
|
||||||
|
}
|
||||||
|
|
||||||
|
get isConnected() {
|
||||||
|
// ioredis: 'wait' | 'reconnecting' | 'connecting' | 'connect' | 'ready' | 'close' | 'end'
|
||||||
|
return this.client.status === 'ready' || this.client.status === 'connect';
|
||||||
|
}
|
||||||
|
|
||||||
|
async get(key) {
|
||||||
|
if (!this.isConnected) return null;
|
||||||
|
try {
|
||||||
|
const data = await this.client.get(key);
|
||||||
|
return data ? JSON.parse(data) : null;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[RedisService] Error getting data:', error);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async set(key, data, ttl = this.DEFAULT_TTL) {
|
||||||
|
if (!this.isConnected) return;
|
||||||
|
try {
|
||||||
|
await this.client.setex(key, ttl, JSON.stringify(data));
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[RedisService] Error setting data:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_getCacheKey(type, params = {}) {
|
||||||
|
const {
|
||||||
|
timeRange,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
metricId,
|
||||||
|
metric,
|
||||||
|
daily,
|
||||||
|
cacheKey,
|
||||||
|
isPreviousPeriod,
|
||||||
|
customFilters,
|
||||||
|
} = params;
|
||||||
|
|
||||||
|
let key = `klaviyo:${type}`;
|
||||||
|
|
||||||
|
if (type === 'stats:details') {
|
||||||
|
key += `:${metric || 'all'}`;
|
||||||
|
if (daily) key += ':daily';
|
||||||
|
if (customFilters?.length) {
|
||||||
|
const filterHash = customFilters.join('').replace(/[^a-zA-Z0-9]/g, '');
|
||||||
|
key += `:${filterHash}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (cacheKey) {
|
||||||
|
key += `:${cacheKey}`;
|
||||||
|
} else if (timeRange) {
|
||||||
|
key += `:${timeRange}`;
|
||||||
|
if (metricId) key += `:${metricId}`;
|
||||||
|
if (isPreviousPeriod) key += ':prev';
|
||||||
|
} else if (startDate && endDate) {
|
||||||
|
key += `:custom:${startDate}:${endDate}`;
|
||||||
|
if (metricId) key += `:${metricId}`;
|
||||||
|
if (isPreviousPeriod) key += ':prev';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (['pre_orders', 'local_pickup', 'on_hold'].includes(metric)) {
|
||||||
|
key += `:${metric}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
return key;
|
||||||
|
}
|
||||||
|
|
||||||
|
_getTTL(timeRange) {
|
||||||
|
const TTL_MAP = {
|
||||||
|
today: 2 * 60,
|
||||||
|
yesterday: 30 * 60,
|
||||||
|
thisWeek: 5 * 60,
|
||||||
|
lastWeek: 60 * 60,
|
||||||
|
thisMonth: 10 * 60,
|
||||||
|
lastMonth: 2 * 60 * 60,
|
||||||
|
last7days: 5 * 60,
|
||||||
|
last30days: 15 * 60,
|
||||||
|
custom: 15 * 60,
|
||||||
|
};
|
||||||
|
return TTL_MAP[timeRange] || this.DEFAULT_TTL;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getEventData(type, params) {
|
||||||
|
if (!this.isConnected) return null;
|
||||||
|
try {
|
||||||
|
const baseKey = this._getCacheKey('events', params);
|
||||||
|
return await this.get(`${baseKey}:${type}`);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[RedisService] Error getting event data:', error);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async cacheEventData(type, params, data) {
|
||||||
|
if (!this.isConnected) return;
|
||||||
|
try {
|
||||||
|
const ttl = this._getTTL(params.timeRange);
|
||||||
|
const baseKey = this._getCacheKey('events', params);
|
||||||
|
await this.set(`${baseKey}:${type}`, data, ttl);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[RedisService] Error caching event data:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async clearCache(params = {}) {
|
||||||
|
if (!this.isConnected) return;
|
||||||
|
try {
|
||||||
|
const pattern = this._getCacheKey('events', params) + '*';
|
||||||
|
const keys = await this.client.keys(pattern);
|
||||||
|
if (keys.length > 0) {
|
||||||
|
await this.client.del(...keys);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[RedisService] Error clearing cache:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
import fetch from 'node-fetch';
|
import fetch from 'node-fetch';
|
||||||
import { TimeManager } from '../utils/time.utils.js';
|
import { TimeManager } from '../../utils/time.utils.js';
|
||||||
import { RedisService } from './redis.service.js';
|
import { RedisService } from './redis.service.js';
|
||||||
|
|
||||||
const METRIC_IDS = {
|
const METRIC_IDS = {
|
||||||
@@ -7,12 +7,12 @@ const METRIC_IDS = {
|
|||||||
};
|
};
|
||||||
|
|
||||||
export class ReportingService {
|
export class ReportingService {
|
||||||
constructor(apiKey, apiRevision) {
|
constructor(apiKey, apiRevision, redis) {
|
||||||
this.apiKey = apiKey;
|
this.apiKey = apiKey;
|
||||||
this.apiRevision = apiRevision;
|
this.apiRevision = apiRevision;
|
||||||
this.baseUrl = 'https://a.klaviyo.com/api';
|
this.baseUrl = 'https://a.klaviyo.com/api';
|
||||||
this.timeManager = new TimeManager();
|
this.timeManager = new TimeManager();
|
||||||
this.redisService = new RedisService();
|
this.redisService = new RedisService(redis);
|
||||||
this._pendingReportRequest = null;
|
this._pendingReportRequest = null;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
// Meta (Facebook Ads) service — ESM conversion of meta-server/services/meta.service.js.
|
||||||
|
// No Redis caching (matches the original — Meta calls are cheap-enough; reach/spend
|
||||||
|
// rolls over once per request). Uses axios.
|
||||||
|
|
||||||
|
import axios from 'axios';
|
||||||
|
|
||||||
|
function getConfig() {
|
||||||
|
const version = process.env.META_API_VERSION || 'v21.0';
|
||||||
|
return {
|
||||||
|
baseUrl: `https://graph.facebook.com/${version}`,
|
||||||
|
accessToken: process.env.META_ACCESS_TOKEN,
|
||||||
|
adAccountId: process.env.META_AD_ACCOUNT_ID,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function metaApiRequest(endpoint, params = {}) {
|
||||||
|
const { baseUrl, accessToken } = getConfig();
|
||||||
|
try {
|
||||||
|
const response = await axios.get(`${baseUrl}/${endpoint}`, {
|
||||||
|
params: {
|
||||||
|
access_token: accessToken,
|
||||||
|
time_zone: 'America/New_York',
|
||||||
|
...params,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Meta API Error:', {
|
||||||
|
message: error.message,
|
||||||
|
response: error.response?.data,
|
||||||
|
endpoint,
|
||||||
|
});
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function fetchCampaigns(since, until) {
|
||||||
|
const { adAccountId } = getConfig();
|
||||||
|
const campaigns = await metaApiRequest(`act_${adAccountId}/campaigns`, {
|
||||||
|
fields: [
|
||||||
|
'id',
|
||||||
|
'name',
|
||||||
|
'status',
|
||||||
|
'objective',
|
||||||
|
'daily_budget',
|
||||||
|
'lifetime_budget',
|
||||||
|
'adsets{daily_budget,lifetime_budget}',
|
||||||
|
`insights.time_range({'since':'${since}','until':'${until}'}).level(campaign){
|
||||||
|
spend,
|
||||||
|
impressions,
|
||||||
|
clicks,
|
||||||
|
ctr,
|
||||||
|
reach,
|
||||||
|
frequency,
|
||||||
|
cpm,
|
||||||
|
cpc,
|
||||||
|
actions,
|
||||||
|
action_values,
|
||||||
|
cost_per_action_type
|
||||||
|
}`,
|
||||||
|
].join(','),
|
||||||
|
limit: 100,
|
||||||
|
});
|
||||||
|
|
||||||
|
return campaigns.data.filter((c) => c.insights?.data?.[0]?.spend > 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function fetchAccountInsights(since, until) {
|
||||||
|
const { adAccountId } = getConfig();
|
||||||
|
const accountInsights = await metaApiRequest(`act_${adAccountId}/insights`, {
|
||||||
|
fields: 'reach,spend,impressions,clicks,ctr,cpm,actions,action_values',
|
||||||
|
time_range: JSON.stringify({ since, until }),
|
||||||
|
});
|
||||||
|
return accountInsights.data[0] || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function updateCampaignBudget(campaignId, budget) {
|
||||||
|
const { baseUrl, accessToken } = getConfig();
|
||||||
|
try {
|
||||||
|
const response = await axios.post(`${baseUrl}/${campaignId}`, {
|
||||||
|
access_token: accessToken,
|
||||||
|
daily_budget: budget * 100, // dollars → cents
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Update campaign budget error:', error);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function updateCampaignStatus(campaignId, action) {
|
||||||
|
const { baseUrl, accessToken } = getConfig();
|
||||||
|
try {
|
||||||
|
const status = action === 'pause' ? 'PAUSED' : 'ACTIVE';
|
||||||
|
const response = await axios.post(`${baseUrl}/${campaignId}`, {
|
||||||
|
access_token: accessToken,
|
||||||
|
status,
|
||||||
|
});
|
||||||
|
return response.data;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Update campaign status error:', error);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
// Typeform service — ESM conversion of typeform-server/services/typeform.service.js.
|
||||||
|
// Phase 4: accepts injected ioredis client. node-redis v4 set syntax `{ EX: 300 }`
|
||||||
|
// translated to ioredis `setex(key, 300, val)`.
|
||||||
|
|
||||||
|
import axios from 'axios';
|
||||||
|
|
||||||
|
export class TypeformService {
|
||||||
|
constructor(redis) {
|
||||||
|
if (!redis) {
|
||||||
|
throw new Error('TypeformService requires an ioredis client (Phase 4: injected)');
|
||||||
|
}
|
||||||
|
this.redis = redis;
|
||||||
|
|
||||||
|
const token = process.env.TYPEFORM_ACCESS_TOKEN;
|
||||||
|
if (!token) {
|
||||||
|
console.warn('[Typeform] TYPEFORM_ACCESS_TOKEN not set — all calls will 401');
|
||||||
|
}
|
||||||
|
|
||||||
|
this.apiClient = axios.create({
|
||||||
|
baseURL: 'https://api.typeform.com',
|
||||||
|
headers: {
|
||||||
|
Authorization: `Bearer ${token}`,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
get _redisReady() {
|
||||||
|
return this.redis.status === 'ready' || this.redis.status === 'connect';
|
||||||
|
}
|
||||||
|
|
||||||
|
async _cacheGet(key) {
|
||||||
|
if (!this._redisReady) return null;
|
||||||
|
try {
|
||||||
|
const raw = await this.redis.get(key);
|
||||||
|
return raw ? JSON.parse(raw) : null;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn('[Typeform] cache get failed:', err.message);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async _cacheSet(key, value, ttlSec) {
|
||||||
|
if (!this._redisReady) return;
|
||||||
|
try {
|
||||||
|
await this.redis.setex(key, ttlSec, JSON.stringify(value));
|
||||||
|
} catch (err) {
|
||||||
|
console.warn('[Typeform] cache set failed:', err.message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async getFormResponses(formId, params = {}) {
|
||||||
|
const cacheKey = `typeform:responses:${formId}:${JSON.stringify(params)}`;
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const response = await this.apiClient.get(`/forms/${formId}/responses`, { params });
|
||||||
|
const data = response.data;
|
||||||
|
await this._cacheSet(cacheKey, data, 300);
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getFormInsights(formId) {
|
||||||
|
const cacheKey = `typeform:insights:${formId}`;
|
||||||
|
const cached = await this._cacheGet(cacheKey);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
const response = await this.apiClient.get(`/insights/${formId}/summary`);
|
||||||
|
const data = response.data;
|
||||||
|
await this._cacheSet(cacheKey, data, 300);
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async getFormResponsesWithFilters(formId, { since, until, pageSize = 25, ...otherParams } = {}) {
|
||||||
|
const params = { page_size: pageSize, ...otherParams };
|
||||||
|
if (since) params.since = new Date(since).toISOString();
|
||||||
|
if (until) params.until = new Date(until).toISOString();
|
||||||
|
return this.getFormResponses(formId, params);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
# Server Configuration
|
|
||||||
NODE_ENV=development
|
|
||||||
TYPEFORM_PORT=3008
|
|
||||||
|
|
||||||
# Redis Configuration
|
|
||||||
REDIS_URL=redis://localhost:6379
|
|
||||||
|
|
||||||
# Typeform API Configuration
|
|
||||||
TYPEFORM_ACCESS_TOKEN=your_typeform_access_token_here
|
|
||||||
|
|
||||||
# Optional: Form IDs (if you want to store them in env)
|
|
||||||
TYPEFORM_FORM_ID_1=your_first_form_id
|
|
||||||
TYPEFORM_FORM_ID_2=your_second_form_id
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,20 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "typeform-server",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"description": "Typeform API integration server",
|
|
||||||
"main": "server.js",
|
|
||||||
"scripts": {
|
|
||||||
"start": "node server.js",
|
|
||||||
"dev": "nodemon server.js"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"axios": "^1.6.2",
|
|
||||||
"cors": "^2.8.5",
|
|
||||||
"dotenv": "^16.3.1",
|
|
||||||
"express": "^4.18.2",
|
|
||||||
"redis": "^4.6.11"
|
|
||||||
},
|
|
||||||
"devDependencies": {
|
|
||||||
"nodemon": "^3.0.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,121 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const router = express.Router();
|
|
||||||
const typeformService = require('../services/typeform.service');
|
|
||||||
|
|
||||||
// Get form responses
|
|
||||||
router.get('/forms/:formId/responses', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { formId } = req.params;
|
|
||||||
const filters = req.query;
|
|
||||||
|
|
||||||
console.log(`Fetching responses for form ${formId} with filters:`, filters);
|
|
||||||
|
|
||||||
if (!formId) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Missing form ID',
|
|
||||||
details: 'The form ID parameter is required'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const data = await typeformService.getFormResponsesWithFilters(formId, filters);
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'No data found',
|
|
||||||
details: `No responses found for form ${formId}`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.json(data);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Form responses error:', {
|
|
||||||
formId: req.params.formId,
|
|
||||||
filters: req.query,
|
|
||||||
error: error.message,
|
|
||||||
stack: error.stack,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle specific error cases
|
|
||||||
if (error.response?.status === 401) {
|
|
||||||
return res.status(401).json({
|
|
||||||
error: 'Authentication failed',
|
|
||||||
details: 'Invalid Typeform API credentials'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 404) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'Not found',
|
|
||||||
details: `Form '${req.params.formId}' not found`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 400) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Invalid request',
|
|
||||||
details: error.response?.data?.message || 'The request was invalid',
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch form responses',
|
|
||||||
details: error.response?.data?.message || error.message,
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Get form insights
|
|
||||||
router.get('/forms/:formId/insights', async (req, res) => {
|
|
||||||
try {
|
|
||||||
const { formId } = req.params;
|
|
||||||
|
|
||||||
if (!formId) {
|
|
||||||
return res.status(400).json({
|
|
||||||
error: 'Missing form ID',
|
|
||||||
details: 'The form ID parameter is required'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const data = await typeformService.getFormInsights(formId);
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'No data found',
|
|
||||||
details: `No insights found for form ${formId}`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.json(data);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Form insights error:', {
|
|
||||||
formId: req.params.formId,
|
|
||||||
error: error.message,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
|
|
||||||
if (error.response?.status === 401) {
|
|
||||||
return res.status(401).json({
|
|
||||||
error: 'Authentication failed',
|
|
||||||
details: 'Invalid Typeform API credentials'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if (error.response?.status === 404) {
|
|
||||||
return res.status(404).json({
|
|
||||||
error: 'Not found',
|
|
||||||
details: `Form '${req.params.formId}' not found`
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
res.status(500).json({
|
|
||||||
error: 'Failed to fetch form insights',
|
|
||||||
details: error.response?.data?.message || error.message,
|
|
||||||
data: error.response?.data
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = router;
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
const express = require('express');
|
|
||||||
const cors = require('cors');
|
|
||||||
const path = require('path');
|
|
||||||
require('dotenv').config({
|
|
||||||
path: path.resolve(__dirname, '.env')
|
|
||||||
});
|
|
||||||
|
|
||||||
const app = express();
|
|
||||||
const port = process.env.TYPEFORM_PORT || 3008;
|
|
||||||
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
|
|
||||||
// Import routes
|
|
||||||
const typeformRoutes = require('./routes/typeform.routes');
|
|
||||||
|
|
||||||
// Use routes
|
|
||||||
app.use('/api/typeform', typeformRoutes);
|
|
||||||
|
|
||||||
// Error handling middleware
|
|
||||||
app.use((err, req, res, next) => {
|
|
||||||
console.error(err.stack);
|
|
||||||
res.status(500).json({ error: 'Something went wrong!' });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Start server
|
|
||||||
app.listen(port, () => {
|
|
||||||
console.log(`Typeform API server running on port ${port}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
module.exports = app;
|
|
||||||
@@ -1,142 +0,0 @@
|
|||||||
const axios = require('axios');
|
|
||||||
const { createClient } = require('redis');
|
|
||||||
|
|
||||||
class TypeformService {
|
|
||||||
constructor() {
|
|
||||||
this.redis = createClient({
|
|
||||||
url: process.env.REDIS_URL
|
|
||||||
});
|
|
||||||
|
|
||||||
this.redis.on('error', err => console.error('Redis Client Error:', err));
|
|
||||||
this.redis.connect().catch(err => console.error('Redis connection error:', err));
|
|
||||||
|
|
||||||
const token = process.env.TYPEFORM_ACCESS_TOKEN;
|
|
||||||
console.log('Initializing Typeform client with token:', token ? `${token.slice(0, 10)}...` : 'missing');
|
|
||||||
|
|
||||||
this.apiClient = axios.create({
|
|
||||||
baseURL: 'https://api.typeform.com',
|
|
||||||
headers: {
|
|
||||||
'Authorization': `Bearer ${token}`,
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Test the token
|
|
||||||
this.testConnection();
|
|
||||||
}
|
|
||||||
|
|
||||||
async testConnection() {
|
|
||||||
try {
|
|
||||||
const response = await this.apiClient.get('/forms');
|
|
||||||
console.log('Typeform connection test successful:', {
|
|
||||||
status: response.status,
|
|
||||||
headers: response.headers,
|
|
||||||
});
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Typeform connection test failed:', {
|
|
||||||
error: error.message,
|
|
||||||
response: error.response?.data,
|
|
||||||
status: error.response?.status,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getFormResponses(formId, params = {}) {
|
|
||||||
const cacheKey = `typeform:responses:${formId}:${JSON.stringify(params)}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log(`Form responses for ${formId} found in Redis cache`);
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fetch from API
|
|
||||||
const response = await this.apiClient.get(`/forms/${formId}/responses`, { params });
|
|
||||||
const data = response.data;
|
|
||||||
|
|
||||||
// Save to Redis with 5 minute expiry
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(data), {
|
|
||||||
EX: 300 // 5 minutes
|
|
||||||
});
|
|
||||||
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error(`Error fetching form responses for ${formId}:`, {
|
|
||||||
error: error.message,
|
|
||||||
params,
|
|
||||||
response: error.response?.data
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getFormInsights(formId) {
|
|
||||||
const cacheKey = `typeform:insights:${formId}`;
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Try Redis first
|
|
||||||
const cachedData = await this.redis.get(cacheKey);
|
|
||||||
if (cachedData) {
|
|
||||||
console.log(`Form insights for ${formId} found in Redis cache`);
|
|
||||||
return JSON.parse(cachedData);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Log the request details
|
|
||||||
console.log(`Fetching insights for form ${formId}...`, {
|
|
||||||
url: `/insights/${formId}/summary`,
|
|
||||||
headers: this.apiClient.defaults.headers
|
|
||||||
});
|
|
||||||
|
|
||||||
// Fetch from API
|
|
||||||
const response = await this.apiClient.get(`/insights/${formId}/summary`);
|
|
||||||
console.log('Typeform insights response:', {
|
|
||||||
status: response.status,
|
|
||||||
headers: response.headers,
|
|
||||||
data: response.data
|
|
||||||
});
|
|
||||||
const data = response.data;
|
|
||||||
|
|
||||||
// Save to Redis with 5 minute expiry
|
|
||||||
await this.redis.set(cacheKey, JSON.stringify(data), {
|
|
||||||
EX: 300 // 5 minutes
|
|
||||||
});
|
|
||||||
|
|
||||||
return data;
|
|
||||||
} catch (error) {
|
|
||||||
console.error(`Error fetching form insights for ${formId}:`, {
|
|
||||||
error: error.message,
|
|
||||||
response: error.response?.data,
|
|
||||||
status: error.response?.status,
|
|
||||||
headers: error.response?.headers,
|
|
||||||
requestUrl: `/insights/${formId}/summary`,
|
|
||||||
requestHeaders: this.apiClient.defaults.headers
|
|
||||||
});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async getFormResponsesWithFilters(formId, { since, until, pageSize = 25, ...otherParams } = {}) {
|
|
||||||
try {
|
|
||||||
const params = {
|
|
||||||
page_size: pageSize,
|
|
||||||
...otherParams
|
|
||||||
};
|
|
||||||
|
|
||||||
if (since) {
|
|
||||||
params.since = new Date(since).toISOString();
|
|
||||||
}
|
|
||||||
if (until) {
|
|
||||||
params.until = new Date(until).toISOString();
|
|
||||||
}
|
|
||||||
|
|
||||||
return await this.getFormResponses(formId, params);
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error in getFormResponsesWithFilters:', error);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = new TypeformService();
|
|
||||||
@@ -76,7 +76,9 @@ $function$;
|
|||||||
|
|
||||||
-- =============================================================================
|
-- =============================================================================
|
||||||
-- get_weighted_avg_cost: Weighted average cost from receivings up to a given date.
|
-- get_weighted_avg_cost: Weighted average cost from receivings up to a given date.
|
||||||
-- Uses all non-canceled receivings (no row limit) weighted by quantity.
|
-- Prefers receivings from the 365 days before p_date so decade-old costs don't
|
||||||
|
-- weigh equally with recent ones; falls back to the lifetime average when the
|
||||||
|
-- product had no receivings in that window.
|
||||||
-- =============================================================================
|
-- =============================================================================
|
||||||
CREATE OR REPLACE FUNCTION public.get_weighted_avg_cost(
|
CREATE OR REPLACE FUNCTION public.get_weighted_avg_cost(
|
||||||
p_pid bigint,
|
p_pid bigint,
|
||||||
@@ -97,8 +99,21 @@ BEGIN
|
|||||||
FROM receivings
|
FROM receivings
|
||||||
WHERE pid = p_pid
|
WHERE pid = p_pid
|
||||||
AND received_date <= p_date
|
AND received_date <= p_date
|
||||||
|
AND received_date > p_date - INTERVAL '365 days'
|
||||||
AND status != 'canceled';
|
AND status != 'canceled';
|
||||||
|
|
||||||
|
IF weighted_cost IS NULL THEN
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN SUM(qty_each) > 0 THEN SUM(cost_each * qty_each) / SUM(qty_each)
|
||||||
|
ELSE NULL
|
||||||
|
END INTO weighted_cost
|
||||||
|
FROM receivings
|
||||||
|
WHERE pid = p_pid
|
||||||
|
AND received_date <= p_date
|
||||||
|
AND status != 'canceled';
|
||||||
|
END IF;
|
||||||
|
|
||||||
RETURN weighted_cost;
|
RETURN weighted_cost;
|
||||||
END;
|
END;
|
||||||
$function$;
|
$function$;
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user