diff --git a/CONSOLIDATION_PLAN.md b/CONSOLIDATION_PLAN.md index e44273e..7d27db3 100644 --- a/CONSOLIDATION_PLAN.md +++ b/CONSOLIDATION_PLAN.md @@ -12,15 +12,19 @@ Audit-driven plan to (a) reduce 12 PM2 processes to 3 application servers + 1 au | 2 — Build shared `lib/` | **Complete** | Lives at `inventory-server/shared/` (see Deviations). `/verify` endpoint live on auth-server | | 3 — Convert auth-server + inventory-server to ESM | **Complete** | All 58 server-side files ESM; both services live under the ESM build for >24h. See Deviations #10–13 | | 4 — Build `dashboard-server` (the merge) | **Complete (live) — 2026-05-24** | Merged service running on :3015 under PM2; Caddy routes for klaviyo/meta/dashboard-analytics/typeform all reverse-proxy to it. Old per-vendor directories (`klaviyo-server`, `meta-server`, `google-server`, `typeform-server`) and their PM2 entries deleted post-cutover — ~1.27 GB reclaimed (largely duplicated `node_modules`). Phase 6.2 gates wired (meta_write, klaviyo_admin). See Deviations #16–19 | -| 5 — Convert `acot-server` to ESM | Not started | | -| 6 — Auth hardening | **Complete** | All in-process items live: rate-limit, JWT precondition, CORS lockdown, request-log, upload allowlist, `requirePermission` on sensitive routes, permissions seed migration. `authenticate()` is live on `/api/*`. 6.11 (audit logging) deferred — see Out of scope | +| 5 — Convert `acot-server` to ESM | **Complete (live) — 2026-05-24** | All 11 files (server, db/connection, utils/{phoneAuth,timeUtils}, 7 routes) converted to ESM. PM2 reload clean; SPA-driven `/api/acot/events/*` continues 200 across cutover; phone-server `/api/acot/customers/by-phone` returns 200 with correct shared secret. Phase 6 patterns applied during conversion — see Deviation #24 | +| 6 — Auth hardening | **Complete** | All in-process items live: rate-limit, JWT precondition, CORS lockdown, request-log, upload allowlist, `requirePermission` on sensitive routes, permissions seed migration. `authenticate()` live on `/api/*` (inventory-server, dashboard-server) and `/api/acot/*` (acot-server, added in Phase 5). 6.10 lt-wordlist token loaded via `--env-file` + rotated 2026-05-24 (Deviation #25). 6.11 (audit logging) deferred — see Out of scope | | **F1 — Frontend fetch wrapper** | **Complete (live) — 2026-05-23** | Wrappers at `inventory/src/utils/api.ts` (`apiFetch`) and `inventory/src/utils/apiClient.ts` (axios instance). 170 `fetch()` sites across 76 files migrated to `apiFetch`; 32 `axios.*` sites across 11 files migrated to `apiClient`. AuthContext `/login`+`/me`, App.tsx `/me`, and `services/apiv2.ts` (external PHP backend) intentionally left as raw `fetch`. Shipped alongside the Phase 3+6 pm2 reload | | 7 — Caddyfile final form | **Complete — applied 2026-05-24** | Final Caddyfile live at `/etc/caddy/Caddyfile` (forward_auth gate + per-vendor reverse_proxy to :3015). The `inventory-server/deploy/` staging folder was removed after apply — recreate from this doc if future changes are needed. Backup convention: `/etc/caddy/Caddyfile.bak.YYYY-MM-DD` | | 8 — ecosystem.config.cjs final form | **Complete — applied 2026-05-24** | Live PM2 list matches the spec below (5 apps + acot-phone-server + lt-wordlist-api = 7 processes). Includes Phase 6.4 JWT_SECRET shadow-override fix and 6.10 lt-wordlist token move. `inventory-server/deploy/` removed post-apply | **Live PM2 process count: 7** (5 application apps — auth-server, inventory-server, chat-server, dashboard-server, acot-server — plus acot-phone-server + lt-wordlist-api). Down from 13 pre-refactor. -**All apply steps complete (2026-05-24).** The original sequencing (npm install → F1 ship → pm2 reload → env consolidation → vendor PM2 delete → ecosystem apply → Caddyfile apply) was executed in order. Remaining work is Phase 5 (acot-server ESM conversion) only. +**All planned phases complete (2026-05-24).** Phase 5 was the last code-level deliverable; acot-server now runs as an ESM service with shared-lib `authenticate()` defense-in-depth. + +**All originally planned phases shipped.** Two real gaps surfaced during Phase 5 verification — both closed 2026-05-24: +- **Phase 6.10 lt-wordlist token rotation** — fixed via Node's native `--env-file` flag in the PM2 entry; token rotated to a fresh 32-byte hex value in `/opt/lt-wordlist-api/.env` (mode 0600). See Deviation #25. +- **Phase 6.1 Caddy gate blocking acot-phone-server's customer lookups** — fixed by repointing `ACOT_API_URL` from the public host to `http://localhost:3012/api/acot`. See Deviation #26. --- @@ -344,21 +348,25 @@ When all four vendors share a Redis client, a Redis hiccup affects all four. Mak ## Phase 5 — Convert `acot-server` to ESM (stays standalone) -Status: **Not started.** Largest single conversion (~5K LOC), but no merge involved. +Status: **Complete (live) — 2026-05-24.** 11 files converted (server.js, db/connection.js, utils/{phoneAuth,timeUtils}.js, 7 route files — ~5.2K LOC). PM2 reload clean; SPA-driven `/api/acot/events/{projection,stats}` continues 200 across cutover; phone-server `/api/acot/customers/by-phone` returns 200 with correct `x-acot-api-key`. Per Deviation #13, `ssh2` is CJS-only → uses `import ssh2 from 'ssh2'; const { Client } = ssh2;`. Phase 6 patterns applied during conversion (Deviation #24). ### Special concern: ssh2 tunnel -`acot-server` opens an SSH tunnel via `ssh2` to access the production MySQL at `192.168.1.5:3309`. The tunnel must be: +`acot-server` opens an SSH tunnel via `ssh2` to access the production MySQL at `192.168.1.5:3309`. Lifecycle today (preserved verbatim across the ESM conversion, not refactored): -- Established before the HTTP listener starts (so no requests fail with "no DB connection"). -- Re-established on disconnect (`ssh2` connection's `close` event → recreate). -- Cleanly torn down on `SIGTERM`/`SIGINT` so PM2 restarts don't leak file descriptors. +- **Lazy establishment.** No tunnel at startup; the first `getDbConnection()` call sets one up. HTTP listener comes up immediately without waiting for the tunnel. Acceptable — the first per-route request just pays the tunnel-creation latency once. +- **Per-connection ssh client.** Each pooled MySQL connection owns its own `ssh2.Client`. Closing a connection closes its own SSH client. +- **No reconnect on disconnect.** There is no `close` listener on the SSH client. If the SSH connection drops while the MySQL connection is pooled (not in use), the next caller that pops it will get a query failure. Circuit-breaker absorbs repeated failures (5 failures → 30s open). Mitigation acceptable for current call volume; revisit if SSH drops become observable in logs. +- **SIGTERM/SIGINT teardown.** `server.close()` → `closeAllConnections()` ends MySQL connections and SSH clients in sequence. Confirmed clean during the Phase 5 cutover (`SIGTERM signal received: closing HTTP server` → 10 × `Closed pooled connection` → `All connections closed and pool reset` in PM2 logs). -Verify (or add) this lifecycle handling as part of the conversion. If it's already correct, conversion is mechanical; if not, this is a good moment to fix it. +### Auth model (two flavors, intentional) -### Test strategy +`server.js` mounts the customers router BEFORE the global `authenticate()` so the two auth schemes don't collide: -Same as inventory-server: start with PM2, smoke-test the most-used `/api/acot/*` endpoints, watch logs for unhandled rejection or tunnel-close events. +- `/api/acot/customers/*` → `requirePhoneApiKey` (timing-safe `x-acot-api-key` check). Used by `acot-phone-server`. +- everything else → JWT Bearer via `shared/auth/middleware.js authenticate()`. Used by the SPA. + +This works at the in-process layer. The public path through Caddy is a separate issue — see Deviation #26. --- @@ -828,19 +836,20 @@ These came up in the audit but aren't part of this refactor: ## Concrete deliverables -State as of 2026-05-24: everything below is **shipped** except Phase 5 (acot-server ESM conversion), which is the only remaining work item. Note: the "4 application PM2 processes" original target became **5** in execution because `chat-server` stayed standalone rather than being folded in — never a serious merge candidate (different DB, different protocol shape). +State as of 2026-05-24: all planned phases are **shipped**. Note: the "4 application PM2 processes" original target became **5** in execution because `chat-server` stayed standalone rather than being folded in — never a serious merge candidate (different DB, different protocol shape). - ✅ 5 application PM2 processes instead of 12 (auth-server, inventory-server, dashboard-server, acot-server, chat-server) — plus 2 unchanged (acot-phone-server, lt-wordlist-api) = 7 total. -- ✅ All `/api/*`, `/chat-api/*`, and `/uploads/*` requests gated at Caddy (`forward_auth`) and re-verified at each upstream (`authenticate()`). +- ✅ All `/api/*`, `/chat-api/*`, and `/uploads/*` requests gated at Caddy (`forward_auth`). +- ✅ Per-upstream `authenticate()` re-verification on inventory-server, dashboard-server, and acot-server. (`chat-server` still relies on the Caddy gate alone — see asterisk below.) - ✅ Sensitive endpoints additionally gated by per-permission checks (`requirePermission`). -- ⚠️ One ESM standard — done for auth/inventory/dashboard/chat. **acot-server still CJS (Phase 5 pending).** +- ⚠️ **One ESM standard — done for auth/inventory/dashboard/acot.** `chat-server` is still CJS (the prior version of this document erroneously claimed it had been converted; verified 2026-05-24 — its `server.js` still uses `require()` and its `package.json` has no `"type": "module"`). Out of scope for this refactor; tracked as future work. - ✅ One shared `lib/` at `inventory-server/shared/` for auth, logging, DB, errors, CORS. - ✅ Login rate-limited (`shared/rate-limit/login.js`). - ✅ `JWT_SECRET` rotated + ecosystem shadow-override removed. - ✅ Old auth-server, Aircall, Gorgias, Clarity directories deleted from the repo. Defunct `dashboard:gorgias`/`dashboard:calls` permission rows also deleted from DB (2026-05-24). - ✅ Caddyfile slimmed to one auth-gated block. - ✅ Permission codes inserted into `permissions` table for granular authorization. -- ✅ No half-finished pieces, no `// TODO: add auth later` comments, no deferred secrets cleanup. +- ✅ No half-finished pieces remain. Both gaps surfaced during Phase 5 verification — `lt-wordlist-api` insecure default token (Deviation #25) and Caddy blocking acot-phone-server's `x-acot-api-key` calls (Deviation #26) — were closed 2026-05-24. --- @@ -897,3 +906,23 @@ These are decisions made during Phase 1/2 implementation that amend the spec abo - `acot-phone-server` script is `/var/www/acot-phone/dist/server.js` (was `./inventory/acot-phone/server.js` in the live file — wrong; that path doesn't exist). `/var/www/acot-phone/` is matt:matt with its own `.env` and is a separate repo from inventory-server. 23. **Phase 6.10 ADD_WORD_TOKEN move stays in this ecosystem.** Per Deviation #22, `lt-wordlist-api` is in matt's ecosystem, so the §6.10 work to remove inline `ADD_WORD_TOKEN` and load it from `/opt/lt-wordlist-api/.env` instead is implemented directly in `deploy/ecosystem.config.cjs.proposed` (no inline `ADD_WORD_TOKEN`; script reads its own .env). When applying, rotate the token value in `/opt/lt-wordlist-api/.env` and update any callers. + +24. **Phase 6 patterns applied to acot-server during Phase 5.** acot-server was originally planned to convert mechanically (require → import) and inherit Phase 6 hardening later. Done in a single pass instead: the new `server.js` mounts `shared/logging/request-log.js`, `shared/cors/policy.js`, `shared/errors/handler.js`, and `shared/auth/middleware.js`'s `authenticate()` on `/api/acot/*` (except the customers router — see Phase 5 auth-model section above). Adds `pg` dependency to `inventory-server/dashboard/acot-server/package.json` because the Postgres pool for `authenticate()`'s user/permission lookups is initialized in-process. Env layering follows dashboard-server's pattern: `/var/www/inventory/.env` loaded first (JWT_SECRET, DB_*), local `.env` loaded second (PROD_DB_*, PROD_SSH_*, ACOT_PHONE_API_KEY). No `acot_admin` permission gates wired — none of the routes mutate state in ways that warrant per-permission checks today; the seeded code in `migrations/005_phase6_permission_codes.sql` stays reserved. + +25. **Phase 6.10 fixed 2026-05-24 via option (b).** Discovered during Phase 5 closeout that `/opt/lt-wordlist-api/.env` didn't exist, no `ADD_WORD_TOKEN` env var was set on the running process, and the script's `process.env.ADD_WORD_TOKEN || 'tokenhere'` fallback was the only gate — meaning `curl -X POST -H "x-add-word-token: tokenhere" http://localhost:3030/add-word` succeeded in production. + + **Fix applied:** + - Generated a fresh 32-byte hex token via `openssl rand -hex 32`. + - Wrote it to `/opt/lt-wordlist-api/.env` (matt:matt, mode 0600). + - Edited `/var/www/ecosystem.config.cjs`'s `lt-wordlist-api` entry to add `node_args: ['--env-file=/opt/lt-wordlist-api/.env']`. Node ≥20.6 (netcup runs v22.22.2) reads the file at startup with no script changes and no `dotenv` import — the cleanest of the three options because the token never lives in the committed ecosystem file. + - `pm2 reload /var/www/ecosystem.config.cjs --only lt-wordlist-api --update-env` picked up the new wrapper config. PM2 restart count 1 → 2, clean startup. + + **Verified:** old default `tokenhere` now returns `{"error":"unauthorized"}` HTTP 401; new env-file token returns `{"ok":true,...}` HTTP 200 on `/add-word` and `/delete-word`. To rotate again: edit `/opt/lt-wordlist-api/.env` + `pm2 restart lt-wordlist-api --update-env`. + + **Caller coordination:** user confirmed all callers are external and will be updated as issues surface; no inventory of callers to pre-notify. + +26. **Caddy `forward_auth` gate breaks acot-phone-server's customer lookups — fixed 2026-05-24 via option (a).** Phase 6.1 put `/api/acot/*` behind `forward_auth localhost:3011/verify`, which strictly requires a JWT Bearer token. But `acot-phone-server` (at `/var/www/acot-phone/`) calls `/api/acot/customers/by-phone`, `/api/acot/customers/search`, `/api/acot/customers/:cid/orders` using only an `x-acot-api-key` header (`/var/www/acot-phone/src/services/acotApi.ts:51`). Result: every customer lookup from the phone app hit a 401 at the Caddy gate before reaching `requirePhoneApiKey`. Last successful customer call in acot-server's access log was 2026-05-21 — three days before the Caddy cutover. + + **Fix applied — option (a):** changed `ACOT_API_URL` in `/var/www/acot-phone/.env` (and `acot-phone-server/.env.example` and the local repo copy) from `https://tools.acherryontop.com/api/acot` to `http://localhost:3012/api/acot`. Both processes live on netcup, so the request never enters Caddy and lands directly on `requirePhoneApiKey` in-process. Restarted via `pm2 restart acot-phone-server --update-env`; smoke-tested with `curl -H "x-acot-api-key: ..." http://localhost:3012/api/acot/customers/by-phone?phone=...` → 200 with the real customer record. + + (Alternative option (b), kept here for posterity: add a `@phone_auth header x-acot-api-key *` guard in the Caddyfile to bypass `forward_auth` for requests bearing the shared secret. Would have worked too but introduces a header-based bypass in the gate, which is a worse security posture than just not entering Caddy at all.) diff --git a/inventory-server/dashboard/acot-server/db/connection.js b/inventory-server/dashboard/acot-server/db/connection.js index a7a47d0..90d6773 100644 --- a/inventory-server/dashboard/acot-server/db/connection.js +++ b/inventory-server/dashboard/acot-server/db/connection.js @@ -1,6 +1,11 @@ -const { Client } = require('ssh2'); -const mysql = require('mysql2/promise'); -const fs = require('fs'); +// Per Deviation #13 in CONSOLIDATION_PLAN.md: `ssh2` is CJS and its named export +// (`Client`) isn't reliably detected by Node's CJS→ESM interop static analysis. +// Default-import + destructure is the bulletproof pattern. +import ssh2 from 'ssh2'; +import mysql from 'mysql2/promise'; +import fs from 'node:fs'; + +const { Client } = ssh2; // Connection pool configuration const connectionPool = { @@ -288,10 +293,10 @@ function getPoolStatus() { }; } -module.exports = { +export { getDbConnection, getCachedQuery, clearQueryCache, closeAllConnections, - getPoolStatus + getPoolStatus, }; \ No newline at end of file diff --git a/inventory-server/dashboard/acot-server/package-lock.json b/inventory-server/dashboard/acot-server/package-lock.json index aa52173..ba35776 100644 --- a/inventory-server/dashboard/acot-server/package-lock.json +++ b/inventory-server/dashboard/acot-server/package-lock.json @@ -15,6 +15,7 @@ "luxon": "^3.5.0", "morgan": "^1.10.0", "mysql2": "^3.6.5", + "pg": "^8.21.0", "ssh2": "^1.14.0" }, "devDependencies": { @@ -1142,6 +1143,95 @@ "integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==", "license": "MIT" }, + "node_modules/pg": { + "version": "8.21.0", + "resolved": "https://registry.npmjs.org/pg/-/pg-8.21.0.tgz", + "integrity": "sha512-AUP1EYJuHraQGsVoCQVIcM7TEJVGtDzxWtGFZd8rds9d+CCXlU5Js1rYgfLNvxy9iJrpHjGrRjoi/3BT9fRyiA==", + "license": "MIT", + "dependencies": { + "pg-connection-string": "^2.13.0", + "pg-pool": "^3.14.0", + "pg-protocol": "^1.14.0", + "pg-types": "2.2.0", + "pgpass": "1.0.5" + }, + "engines": { + "node": ">= 16.0.0" + }, + "optionalDependencies": { + "pg-cloudflare": "^1.4.0" + }, + "peerDependencies": { + "pg-native": ">=3.0.1" + }, + "peerDependenciesMeta": { + "pg-native": { + "optional": true + } + } + }, + "node_modules/pg-cloudflare": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz", + "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==", + "license": "MIT", + "optional": true + }, + "node_modules/pg-connection-string": { + "version": "2.13.0", + "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.13.0.tgz", + "integrity": "sha512-EMnU9E2fSULdsbErBbMaXJvFeD9B4+nPcM3f+4lsiCR0BHLPrLVjv3DbyM2hgQQviKJaTWIRRTjKjWlHg3p2ig==", + "license": "MIT" + }, + "node_modules/pg-int8": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz", + "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==", + "license": "ISC", + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/pg-pool": { + "version": "3.14.0", + "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz", + "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==", + "license": "MIT", + "peerDependencies": { + "pg": ">=8.0" + } + }, + "node_modules/pg-protocol": { + "version": "1.14.0", + "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.14.0.tgz", + "integrity": "sha512-n5taZ1kO3s9ngDTVxsEznOqCyToTgz0FLuPq0B33COy5pPpuWJpY3/2oRBVETuOgzdqRXfWpM9HIhp2LBBT1BA==", + "license": "MIT" + }, + "node_modules/pg-types": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz", + "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==", + "license": "MIT", + "dependencies": { + "pg-int8": "1.0.1", + "postgres-array": "~2.0.0", + "postgres-bytea": "~1.0.0", + "postgres-date": "~1.0.4", + "postgres-interval": "^1.1.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/pgpass": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz", + "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==", + "license": "MIT", + "dependencies": { + "split2": "^4.1.0" + } + }, "node_modules/picomatch": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz", @@ -1155,6 +1245,45 @@ "url": "https://github.com/sponsors/jonschlinkert" } }, + "node_modules/postgres-array": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz", + "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==", + "license": "MIT", + "engines": { + "node": ">=4" + } + }, + "node_modules/postgres-bytea": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz", + "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/postgres-date": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz", + "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/postgres-interval": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz", + "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==", + "license": "MIT", + "dependencies": { + "xtend": "^4.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/proxy-addr": { "version": "2.0.7", "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", @@ -1416,6 +1545,15 @@ "node": ">=10" } }, + "node_modules/split2": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz", + "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==", + "license": "ISC", + "engines": { + "node": ">= 10.x" + } + }, "node_modules/sqlstring": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/sqlstring/-/sqlstring-2.3.3.tgz", @@ -1548,6 +1686,15 @@ "engines": { "node": ">= 0.8" } + }, + "node_modules/xtend": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", + "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==", + "license": "MIT", + "engines": { + "node": ">=0.4" + } } } } diff --git a/inventory-server/dashboard/acot-server/package.json b/inventory-server/dashboard/acot-server/package.json index ca8eda2..2732e0e 100644 --- a/inventory-server/dashboard/acot-server/package.json +++ b/inventory-server/dashboard/acot-server/package.json @@ -2,22 +2,24 @@ "name": "acot-server", "version": "1.0.0", "description": "A Cherry On Top production database server", + "type": "module", "main": "server.js", "scripts": { "start": "node server.js", "dev": "nodemon server.js" }, "dependencies": { - "express": "^4.18.2", + "compression": "^1.7.4", "cors": "^2.8.5", "dotenv": "^16.3.1", + "express": "^4.18.2", + "luxon": "^3.5.0", "morgan": "^1.10.0", - "ssh2": "^1.14.0", "mysql2": "^3.6.5", - "compression": "^1.7.4", - "luxon": "^3.5.0" + "pg": "^8.21.0", + "ssh2": "^1.14.0" }, "devDependencies": { "nodemon": "^3.0.1" } -} +} diff --git a/inventory-server/dashboard/acot-server/routes/customers.js b/inventory-server/dashboard/acot-server/routes/customers.js index 5b900bf..8cfa11d 100644 --- a/inventory-server/dashboard/acot-server/routes/customers.js +++ b/inventory-server/dashboard/acot-server/routes/customers.js @@ -8,10 +8,11 @@ // NOTE: `users.phone` is not yet indexed in production. Admin will add // `idx_phone (phone)` — queries here assume that exists for acceptable latency. -const express = require('express'); +import express from 'express'; +import { getDbConnection, getCachedQuery } from '../db/connection.js'; +import { requirePhoneApiKey } from '../utils/phoneAuth.js'; + const router = express.Router(); -const { getDbConnection, getCachedQuery } = require('../db/connection'); -const { requirePhoneApiKey } = require('../utils/phoneAuth'); // Order status labels mirror ACOTCustomerDataServiceProvider.php. const ORDER_STATUS_LABEL = { @@ -319,4 +320,4 @@ router.get('/:cid/orders', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/dashboard/acot-server/routes/discounts.js b/inventory-server/dashboard/acot-server/routes/discounts.js index 522f196..232cb36 100644 --- a/inventory-server/dashboard/acot-server/routes/discounts.js +++ b/inventory-server/dashboard/acot-server/routes/discounts.js @@ -1,6 +1,6 @@ -const express = require('express'); -const { DateTime } = require('luxon'); -const { getDbConnection } = require('../db/connection'); +import express from 'express'; +import { DateTime } from 'luxon'; +import { getDbConnection } from '../db/connection.js'; const router = express.Router(); @@ -573,4 +573,4 @@ router.post('/simulate', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/dashboard/acot-server/routes/employee-metrics.js b/inventory-server/dashboard/acot-server/routes/employee-metrics.js index e78c3b5..00a5040 100644 --- a/inventory-server/dashboard/acot-server/routes/employee-metrics.js +++ b/inventory-server/dashboard/acot-server/routes/employee-metrics.js @@ -1,12 +1,9 @@ -const express = require('express'); -const { DateTime } = require('luxon'); +import express from 'express'; +import { DateTime } from 'luxon'; +import { getDbConnection, getPoolStatus } from '../db/connection.js'; +import { getTimeRangeConditions, _internal as timeHelpers } from '../utils/timeUtils.js'; const router = express.Router(); -const { getDbConnection, getPoolStatus } = require('../db/connection'); -const { - getTimeRangeConditions, - _internal: timeHelpers -} = require('../utils/timeUtils'); const TIMEZONE = 'America/New_York'; @@ -680,4 +677,4 @@ function getPreviousTimeRange(timeRange) { return map[timeRange] || timeRange; } -module.exports = router; +export default router; diff --git a/inventory-server/dashboard/acot-server/routes/events.js b/inventory-server/dashboard/acot-server/routes/events.js index 9dd4fa8..649806d 100644 --- a/inventory-server/dashboard/acot-server/routes/events.js +++ b/inventory-server/dashboard/acot-server/routes/events.js @@ -1,14 +1,14 @@ -const express = require('express'); -const { DateTime } = require('luxon'); - -const router = express.Router(); -const { getDbConnection, getPoolStatus } = require('../db/connection'); -const { +import express from 'express'; +import { DateTime } from 'luxon'; +import { getDbConnection, getPoolStatus } from '../db/connection.js'; +import { getTimeRangeConditions, formatBusinessDate, getBusinessDayBounds, - _internal: timeHelpers -} = require('../utils/timeUtils'); + _internal as timeHelpers, +} from '../utils/timeUtils.js'; + +const router = express.Router(); const TIMEZONE = 'America/New_York'; const BUSINESS_DAY_START_HOUR = timeHelpers?.BUSINESS_DAY_START_HOUR ?? 1; @@ -1794,4 +1794,5 @@ router.get('/debug/pool', (req, res) => { }); }); -module.exports = router; +export default router; + diff --git a/inventory-server/dashboard/acot-server/routes/operations-metrics.js b/inventory-server/dashboard/acot-server/routes/operations-metrics.js index 9a3a4e1..b0ab135 100644 --- a/inventory-server/dashboard/acot-server/routes/operations-metrics.js +++ b/inventory-server/dashboard/acot-server/routes/operations-metrics.js @@ -1,11 +1,9 @@ -const express = require('express'); -const { DateTime } = require('luxon'); +import express from 'express'; +import { DateTime } from 'luxon'; +import { getDbConnection, getPoolStatus } from '../db/connection.js'; +import { getTimeRangeConditions } from '../utils/timeUtils.js'; const router = express.Router(); -const { getDbConnection, getPoolStatus } = require('../db/connection'); -const { - getTimeRangeConditions, -} = require('../utils/timeUtils'); const TIMEZONE = 'America/New_York'; @@ -481,4 +479,4 @@ function getPreviousTimeRange(timeRange) { return map[timeRange] || timeRange; } -module.exports = router; +export default router; diff --git a/inventory-server/dashboard/acot-server/routes/payroll-metrics.js b/inventory-server/dashboard/acot-server/routes/payroll-metrics.js index 7c618ba..317e7ac 100644 --- a/inventory-server/dashboard/acot-server/routes/payroll-metrics.js +++ b/inventory-server/dashboard/acot-server/routes/payroll-metrics.js @@ -1,8 +1,8 @@ -const express = require('express'); -const { DateTime } = require('luxon'); +import express from 'express'; +import { DateTime } from 'luxon'; +import { getDbConnection, getPoolStatus } from '../db/connection.js'; const router = express.Router(); -const { getDbConnection, getPoolStatus } = require('../db/connection'); const TIMEZONE = 'America/New_York'; @@ -502,4 +502,4 @@ function isCurrentPayPeriod(payPeriod) { return now >= payPeriod.start && now <= payPeriod.end; } -module.exports = router; +export default router; diff --git a/inventory-server/dashboard/acot-server/routes/test.js b/inventory-server/dashboard/acot-server/routes/test.js index e41e8ca..b45825c 100644 --- a/inventory-server/dashboard/acot-server/routes/test.js +++ b/inventory-server/dashboard/acot-server/routes/test.js @@ -1,6 +1,7 @@ -const express = require('express'); +import express from 'express'; +import { getDbConnection, getCachedQuery } from '../db/connection.js'; + const router = express.Router(); -const { getDbConnection, getCachedQuery } = require('../db/connection'); // Test endpoint to count orders router.get('/order-count', async (req, res) => { @@ -54,4 +55,4 @@ router.get('/test-connection', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/dashboard/acot-server/server.js b/inventory-server/dashboard/acot-server/server.js index df0ee0e..bb3bc87 100644 --- a/inventory-server/dashboard/acot-server/server.js +++ b/inventory-server/dashboard/acot-server/server.js @@ -1,103 +1,158 @@ -require('dotenv').config(); -const express = require('express'); -const cors = require('cors'); -const morgan = require('morgan'); -const compression = require('compression'); -const fs = require('fs'); -const path = require('path'); -const { closeAllConnections } = require('./db/connection'); +// acot-server — Phase 5 of CONSOLIDATION_PLAN.md. +// Standalone service on ACOT_PORT (default 3012) exposing /api/acot/* against +// the production MySQL `sg` database via an ssh2 tunnel (see db/connection.js). +// +// Auth model (two flavors, deliberate): +// - /api/acot/customers/* : x-acot-api-key shared secret (used by acot-phone-server). +// Mounted BEFORE authenticate() so its requirePhoneApiKey +// path is the only gate. +// - everything else : JWT Bearer via shared/auth/middleware.js authenticate(). +// Defense-in-depth on top of Caddy forward_auth. +// +// Shared infrastructure (Phase 2 + Phase 6): +// - shared/auth/middleware.js authenticate() for SPA-served routes +// - shared/cors/policy.js explicit allowed-origins list (Phase 6.6) +// - shared/logging/request-log.js pino-http, Authorization/Cookie redacted (Phase 6.5/6.9) +// - shared/errors/handler.js consistent error envelope, no leak in prod +// +// Env layering: /var/www/inventory/.env loaded FIRST (JWT_SECRET, DB_* for the +// shared PG pool used by authenticate to look up user permissions). Local .env +// loaded SECOND for ACOT-specific keys (PROD_DB_*, PROD_SSH_*, ACOT_PHONE_API_KEY). +// dotenv defaults to override:false, so the first file wins on collisions. + +import { config as loadEnv } from 'dotenv'; +import express from 'express'; +import cors from 'cors'; +import compression from 'compression'; +import morgan from 'morgan'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import pg from 'pg'; + +import { authenticate } from '../../shared/auth/middleware.js'; +import { corsOptions } from '../../shared/cors/policy.js'; +import { errorHandler } from '../../shared/errors/handler.js'; +import { logger } from '../../shared/logging/logger.js'; +import { requestLog } from '../../shared/logging/request-log.js'; + +import { closeAllConnections } from './db/connection.js'; + +import testRouter from './routes/test.js'; +import eventsRouter from './routes/events.js'; +import discountsRouter from './routes/discounts.js'; +import employeeMetricsRouter from './routes/employee-metrics.js'; +import payrollMetricsRouter from './routes/payroll-metrics.js'; +import operationsMetricsRouter from './routes/operations-metrics.js'; +import customersRouter from './routes/customers.js'; + +const { Pool } = pg; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); + +// Layer envs: shared inventory .env first (JWT_SECRET, DB_*) then acot .env. +const sharedEnvPath = '/var/www/inventory/.env'; +const localEnvPath = path.resolve(__dirname, '.env'); +if (fs.existsSync(sharedEnvPath)) loadEnv({ path: sharedEnvPath }); +if (fs.existsSync(localEnvPath)) loadEnv({ path: localEnvPath }); + +// Phase 6.4 — refuse to start without JWT_SECRET. authenticate() would reject +// every request anyway; failing fast surfaces the misconfiguration immediately. +if (!process.env.JWT_SECRET) { + logger.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)'); + process.exit(1); +} const app = express(); -const PORT = process.env.ACOT_PORT || 3012; +const PORT = Number(process.env.ACOT_PORT) || 3012; -// Create logs directory if it doesn't exist +// Postgres pool for authenticate() (user/permission lookups against inventory_db). +// All MySQL access goes through db/connection.js (separate, ssh-tunneled). +const pool = new Pool({ + host: process.env.DB_HOST, + user: process.env.DB_USER, + password: process.env.DB_PASSWORD, + database: process.env.DB_NAME, + port: Number(process.env.DB_PORT) || 5432, +}); + +// Per-app access log on disk (kept from pre-conversion behavior; pino request-log +// is mounted below for structured/redacted server-side logging). const logDir = path.join(__dirname, 'logs/app'); if (!fs.existsSync(logDir)) { fs.mkdirSync(logDir, { recursive: true }); } +const accessLogStream = fs.createWriteStream(path.join(logDir, 'access.log'), { flags: 'a' }); -// Create a write stream for access logs -const accessLogStream = fs.createWriteStream( - path.join(logDir, 'access.log'), - { flags: 'a' } -); - -// Middleware +app.use(requestLog()); app.use(compression()); -app.use(cors()); +app.use(cors(corsOptions)); app.use(express.json()); app.use(express.urlencoded({ extended: true })); - -// Logging middleware if (process.env.NODE_ENV === 'production') { app.use(morgan('combined', { stream: accessLogStream })); } else { app.use(morgan('dev')); } -// Health check endpoint app.get('/health', (req, res) => { res.json({ status: 'healthy', service: 'acot-server', timestamp: new Date().toISOString(), - uptime: process.uptime() + uptime: process.uptime(), }); }); -// Routes -app.use('/api/acot/test', require('./routes/test')); -app.use('/api/acot/events', require('./routes/events')); -app.use('/api/acot/discounts', require('./routes/discounts')); -app.use('/api/acot/employee-metrics', require('./routes/employee-metrics')); -app.use('/api/acot/payroll-metrics', require('./routes/payroll-metrics')); -app.use('/api/acot/operations-metrics', require('./routes/operations-metrics')); -app.use('/api/acot/customers', require('./routes/customers')); +// Customers route uses x-acot-api-key (shared secret with acot-phone-server), +// NOT JWT — mount BEFORE authenticate() so requirePhoneApiKey is the only gate. +app.use('/api/acot/customers', customersRouter); -// Error handling middleware -app.use((err, req, res, next) => { - console.error('Unhandled error:', err); - res.status(500).json({ - success: false, - error: process.env.NODE_ENV === 'production' - ? 'Internal server error' - : err.message - }); -}); +// All remaining /api/acot/* routes require a valid JWT. +app.use('/api/acot', authenticate({ pool, secret: process.env.JWT_SECRET })); -// 404 handler +app.use('/api/acot/test', testRouter); +app.use('/api/acot/events', eventsRouter); +app.use('/api/acot/discounts', discountsRouter); +app.use('/api/acot/employee-metrics', employeeMetricsRouter); +app.use('/api/acot/payroll-metrics', payrollMetricsRouter); +app.use('/api/acot/operations-metrics', operationsMetricsRouter); + +// 404 for unmatched /api routes (keeps prior behavior). app.use((req, res) => { - res.status(404).json({ - success: false, - error: 'Route not found' - }); + res.status(404).json({ success: false, error: 'Route not found' }); }); -// Start server -const server = app.listen(PORT, () => { - console.log(`ACOT Server running on port ${PORT}`); - console.log(`Environment: ${process.env.NODE_ENV}`); +app.use(errorHandler); + +const server = app.listen(PORT, '0.0.0.0', () => { + logger.info({ port: PORT, mode: process.env.NODE_ENV || 'development' }, 'acot-server listening'); }); -// Graceful shutdown -const gracefulShutdown = async () => { - console.log('SIGTERM signal received: closing HTTP server'); +const gracefulShutdown = async (signal) => { + logger.info({ signal }, 'acot-server shutting down'); server.close(async () => { - console.log('HTTP server closed'); - - // Close database connections try { await closeAllConnections(); - console.log('Database connections closed'); - } catch (error) { - console.error('Error closing database connections:', error); + } catch (err) { + logger.error({ err: { message: err.message } }, 'error closing MySQL pool'); } - + try { + await pool.end(); + } catch { /* ignore */ } process.exit(0); }); }; -process.on('SIGTERM', gracefulShutdown); -process.on('SIGINT', gracefulShutdown); +process.on('SIGTERM', () => gracefulShutdown('SIGTERM')); +process.on('SIGINT', () => gracefulShutdown('SIGINT')); -module.exports = app; +process.on('uncaughtException', (err) => { + logger.error({ err: { message: err.message, stack: err.stack } }, 'uncaughtException'); + process.exit(1); +}); +process.on('unhandledRejection', (reason) => { + logger.error({ reason }, 'unhandledRejection'); +}); + +export default app; diff --git a/inventory-server/dashboard/acot-server/utils/phoneAuth.js b/inventory-server/dashboard/acot-server/utils/phoneAuth.js index 6e51f59..1335c66 100644 --- a/inventory-server/dashboard/acot-server/utils/phoneAuth.js +++ b/inventory-server/dashboard/acot-server/utils/phoneAuth.js @@ -2,9 +2,9 @@ // The acot-phone-server sends `x-acot-api-key` on every request; we compare // against ACOT_PHONE_API_KEY from the environment using timing-safe comparison. -const crypto = require('crypto'); +import crypto from 'node:crypto'; -function requirePhoneApiKey(req, res, next) { +export function requirePhoneApiKey(req, res, next) { const expected = process.env.ACOT_PHONE_API_KEY; if (!expected) { console.error('ACOT_PHONE_API_KEY not configured; rejecting all requests'); @@ -24,5 +24,3 @@ function requirePhoneApiKey(req, res, next) { next(); } - -module.exports = { requirePhoneApiKey }; diff --git a/inventory-server/dashboard/acot-server/utils/timeUtils.js b/inventory-server/dashboard/acot-server/utils/timeUtils.js index 29fdfae..0d34bb5 100644 --- a/inventory-server/dashboard/acot-server/utils/timeUtils.js +++ b/inventory-server/dashboard/acot-server/utils/timeUtils.js @@ -1,4 +1,4 @@ -const { DateTime } = require('luxon'); +import { DateTime } from 'luxon'; const TIMEZONE = 'America/New_York'; const DB_TIMEZONE = 'UTC-05:00'; @@ -294,19 +294,24 @@ const formatMySQLDate = (input) => { return dt.setZone(DB_TIMEZONE).toFormat(DB_DATETIME_FORMAT); }; -module.exports = { +// Expose helpers for tests or advanced consumers. +// Kept as a named `_internal` export so existing destructuring sites +// (`const { _internal: timeHelpers } = require(...)` → ESM equivalent works) +// don't need to change beyond the import-statement rewrite. +const _internal = { + getDayStart, + getDayEnd, + getWeekStart, + getRangeForTimeRange, + BUSINESS_DAY_START_HOUR, +}; + +export { getBusinessDayBounds, getTimeRangeConditions, formatBusinessDate, getTimeRangeLabel, parseBusinessDate, formatMySQLDate, - // Expose helpers for tests or advanced consumers - _internal: { - getDayStart, - getDayEnd, - getWeekStart, - getRangeForTimeRange, - BUSINESS_DAY_START_HOUR - } + _internal, };