diff --git a/CONSOLIDATION_PLAN.md b/CONSOLIDATION_PLAN.md index b8bb753..7dac8e5 100644 --- a/CONSOLIDATION_PLAN.md +++ b/CONSOLIDATION_PLAN.md @@ -10,15 +10,18 @@ Audit-driven plan to (a) reduce 12 PM2 processes to 3 application servers + 1 au |---|---|---| | 1 — Decommission dead services | **Complete** | aircall/gorgias/clarity/legacy-auth-server deleted from repo + PM2 + Caddyfile + ecosystem.cjs | | 2 — Build shared `lib/` | **Complete** | Lives at `inventory-server/shared/` (see Deviations). `/verify` endpoint live on auth-server | -| 3 — Convert auth-server + inventory-server to ESM | Not started | Next up. Auth-server is still CJS but has the new `/verify` route added in CJS form | +| 3 — Convert auth-server + inventory-server to ESM | **Complete (code)** | All 58 server-side files ESM; verified 0 import failures on netcup. Pending: `npm install` on server + pm2 reload to actually run the new code. See Deviations #10–13 | | 4 — Build `dashboard-server` (the merge) | Not started | klaviyo/meta/google/typeform still run as 4 separate PM2 apps | | 5 — Convert `acot-server` to ESM | Not started | | -| 6 — Auth hardening | Not started | Shared modules exist (`shared/rate-limit`, `shared/cors`, `shared/logging`) but no service consumes them yet. JWT_SECRET footgun discovered — see 6.4 | -| 7 — Caddyfile final form | Partial | Dead routes removed; `forward_auth` gate + `/uploads/*` gating + per-vendor cleanup deferred to after Phase 4 | -| 8 — ecosystem.config.cjs final form | Partial | Dead apps removed; final shape depends on Phase 4 merge | +| 6 — Auth hardening | **Complete (code) — gated on Phase F1** | All in-process items wired (rate-limit, JWT precondition, CORS lockdown, request-log, upload allowlist, `requirePermission` on sensitive routes, permissions seed migration). `authenticate()` is live on `/api/*`. Server-side artefacts (Caddyfile, ecosystem.cjs) written to `inventory-server/deploy/` for review. 6.11 (audit logging) deferred. **Frontend cannot use the app until Phase F1 ships** — see below | +| **F1 — Frontend fetch wrapper (NEW)** | **Not started — CRITICAL** | Frontend uses raw `fetch()` in ~220 sites; only 7 send `Authorization: Bearer`. With Phase 6's `authenticate()` middleware live, every refresh 401s until the frontend uniformly attaches the token. See "Phase F1" below | +| 7 — Caddyfile final form | Partial | Proposed file at `inventory-server/deploy/Caddyfile.proposed`. Apply blocked on F1 (forward_auth would 401 every page load until then) | +| 8 — ecosystem.config.cjs final form | Partial | Proposed at `inventory-server/deploy/ecosystem.config.cjs.proposed`. Includes Phase 6.4 JWT_SECRET footgun fix and 6.10 lt-wordlist token move | **Live PM2 count: 10** (down from 13). Target after Phase 4: 5 application apps + acot-phone-server + lt-wordlist-api. +**Apply order from current state:** (a) `npm install` on netcup to install the new shared-module deps (`pino`, `pino-http`, `ioredis`, `express-rate-limit`, `jsonwebtoken`), (b) ship Phase F1 frontend fetch wrapper, (c) `pm2 reload inventory-server new-auth-server` (Phase 3+6 code goes live, requests carry tokens, app keeps working), (d) apply `deploy/ecosystem.config.cjs.proposed` (Phase 6.4 + 6.10), (e) apply `deploy/Caddyfile.proposed` (Phase 6.1 — edge gate). + --- ## Goals @@ -199,7 +202,11 @@ Caddy's `forward_auth` only needs "is this token valid? give me a user-id." Toda ## Phase 3 — Convert `auth-server` and `inventory-server` to ESM -Status: **Not started.** Lift the easy ones first. These two stay standalone (don't merge into anything), so they're isolated changes. The auth-server's new `/verify` route (added in Phase 2) is currently CJS — refactor it during this phase to import from `../shared/auth/verify.js`. +Status: **Complete (code) — 2026-05-23.** Both servers + all sub-trees converted to ESM. 58 importable .js files load cleanly on netcup (verified via dynamic-import sweep). Two latent bugs surfaced and fixed: `??`/`||` precedence in `shared/db/{pg,redis}.js`, and CJS named-import of `Pool` from `pg` in both auth files (now uses `import pg from 'pg'; const { Pool } = pg`). + +Scripts under `inventory-server/scripts/` (one-shot maintenance / orchestrators) kept CommonJS via a sibling `scripts/package.json` declaring `"type": "commonjs"` — Node's package-type resolution walks up directory by directory, so this overrides the parent's `"type": "module"` without renaming any file or touching any `spawn()` callsite. Convert individual scripts to ESM if/when touched. + +Pending to actually go live: `npm install` on netcup (new deps: `pino`, `pino-http`, `ioredis`, `express-rate-limit`, `jsonwebtoken`) + `pm2 reload`. See "Phase F1" — the frontend fetch wrapper should ship in the same deploy or this immediately breaks the app. ### Mechanical conversion @@ -357,7 +364,23 @@ Same as inventory-server: start with PM2, smoke-test the most-used `/api/acot/*` ## Phase 6 — Auth hardening -Status: **Not started.** This is the security work that justifies the whole refactor. Runs in parallel with phases 3–5 where possible. Shared building blocks already exist (`shared/rate-limit/login.js`, `shared/cors/policy.js`, `shared/logging/request-log.js`, `shared/errors/handler.js`) — Phase 6 is about *applying* them per-service. +Status: **Complete (code) — 2026-05-23. Application gated on Phase F1.** All in-process hardening shipped alongside the Phase 3 ESM conversion. The `authenticate()` middleware is wired live on `/api/*` in inventory-server — **the moment that code reaches production, the frontend stops working until Phase F1 lands**, because today's frontend doesn't include `Authorization: Bearer` on the vast majority of fetch calls (see Phase F1 below for the diagnosis). + +Per-item status: + +| # | Item | Status | Where | +|---|---|---|---| +| 6.1 | Caddy `forward_auth` gate | **Proposed** — apply *after* F1 | `inventory-server/deploy/Caddyfile.proposed` | +| 6.2 | `requirePermission` on sensitive routes + permissions migration | **Done** | inline in `config.js`, `data-management.js`, `import.js`, `ai-prompts.js`, `ai-validation.js`, `templates.js`, `reusable-images.js`; codes seeded by `migrations/005_phase6_permission_codes.sql` | +| 6.3 | Login rate-limit + `/verify` rate-limit | **Done** | `auth/server.js` uses `shared/rate-limit/login.js` (`loginLimiter`, `verifyLimiter`) | +| 6.4 | JWT_SECRET as startup precondition + ecosystem footgun fix | **Done in code; proposed for ecosystem.cjs** | Both auth-server and inventory-server `process.exit(1)` if `JWT_SECRET` is unset. `inventory-server/deploy/ecosystem.config.cjs.proposed` removes the `JWT_SECRET: process.env.JWT_SECRET` override that was shadowing `.env` | +| 6.5 | Structured request logging w/ redaction | **Done** | `shared/logging/request-log.js` (pino-http, redacts Authorization/Cookie); mounted in both `auth/server.js` and `src/server.js` | +| 6.6 | CORS lockdown | **Done** | `src/middleware/cors.js` now re-exports `shared/cors/policy.js`. LAN wildcards (`192.168.*`, `10.*`) and `*` defaults gone | +| 6.7 | Upload hardening | **Done** | Exact-match MIME+extension allowlist on `routes/import.js` and `routes/reusable-images.js`; dead `multer({ dest })` removed from `routes/products.js` (no upload route was using it — strongest hardening was deletion) | +| 6.8 | Frontend token storage stays localStorage + XSS audit | **Audited** | Confirmed `dangerouslySetInnerHTML` is sanitized in `ProductEditor.tsx`. **Flagged: `ChatRoom.tsx:277,392` renders user-controlled chat content as raw HTML — real XSS vector, separate fix needed** | +| 6.9 | Remove debug middleware | **Done** | The header-dumping `app.use((req,res,next)=>{ console.log(... req.headers ...) })` block removed from `src/server.js`. Replaced with `shared/logging/request-log.js` (which redacts). | +| 6.10 | `lt-wordlist-api` token move | **Proposed for ecosystem.cjs** | `inventory-server/deploy/ecosystem.config.cjs.proposed` shows the entry without inline token; apply alongside rotating the secret value into `/opt/lt-wordlist-api/.env` | +| 6.11 | Audit logging for sensitive ops | **Deferred** | Out of scope for this pass per user direction. Existing `import_audit_log` and `product_editor_audit_log` tables stay as-is; generic `system_audit_log` table + middleware is its own project | ### 6.1 Caddy `forward_auth` gate @@ -494,8 +517,99 @@ Already have `import-audit-log` and `product-editor-audit-log` tables. Extend th --- +## Phase F1 — Frontend fetch wrapper (NEW — 2026-05-23) + +Status: **Not started. CRITICAL. Blocks the Phase 3+6 deploy from being usable.** + +### The discovery + +While wiring `authenticate()` on `/api/*` in Phase 6.1/6.2, we audited the frontend's fetch usage and found: + +- **7** call sites send `Authorization: Bearer ${token}` explicitly (all in `AuthContext.tsx` for `/me` + `/login`, plus a couple of `settings/*` pages). +- **~220** other `fetch(...)` / `axios.*(...)` call sites across `inventory/src/services/`, `inventory/src/pages/`, `inventory/src/components/` send **no** Authorization header at all. +- There is no global fetch wrapper, axios interceptor, or service-worker shim that injects the token. + +Today this works because nothing on the server checks. Caddy currently has no `forward_auth` gate (Phase 6.1 is a Caddyfile change that hasn't shipped yet) and the previous inventory-server had no `authenticate()` middleware. The frontend's auth model was "you log in once to get the token; the token is checked only by `/me`; everything else is implicitly trusted at the network layer." + +With Phase 6 code in production, **every page refresh 401s** on the first API call after the next pm2 reload. The user explicitly accepted this when authorising the Phase 6 work — but the fix is its own deliverable, and shipping Phase 3+6 to PM2 without F1 in the same window means an outage window measured in *however long F1 takes* (not minutes). + +### Recommended approach + +Add a single fetch wrapper at `inventory/src/utils/api.ts` (or similar) and migrate the ~220 call sites to use it. The wrapper: + +1. Reads `localStorage.getItem('token')` on every call (cheap; localStorage is sync). +2. Merges `Authorization: Bearer ${token}` into the request headers if a token exists. +3. Intercepts 401 responses → fires `window.dispatchEvent(new Event('auth:logout'))` (a listener already exists in `AuthContext.tsx:117`) so the user gets bounced to `/login` cleanly instead of seeing broken pages. +4. Preserves the existing call shape — `apiFetch(url, init)` should be a drop-in for `fetch(url, init)` so the migration is mechanical. + +```ts +// inventory/src/utils/api.ts (sketch) +export async function apiFetch(input: RequestInfo | URL, init: RequestInit = {}): Promise { + const token = localStorage.getItem('token'); + const headers = new Headers(init.headers); + if (token && !headers.has('Authorization')) { + headers.set('Authorization', `Bearer ${token}`); + } + const res = await fetch(input, { ...init, headers }); + if (res.status === 401 && token) { + // Token expired or revoked — bounce to /login. AuthContext already listens. + window.dispatchEvent(new Event('auth:logout')); + } + return res; +} +``` + +Same shape for axios: + +```ts +// inventory/src/utils/apiClient.ts (sketch) +import axios from 'axios'; +export const apiClient = axios.create(); +apiClient.interceptors.request.use((config) => { + const token = localStorage.getItem('token'); + if (token) config.headers.Authorization = `Bearer ${token}`; + return config; +}); +apiClient.interceptors.response.use( + (r) => r, + (err) => { + if (err?.response?.status === 401) window.dispatchEvent(new Event('auth:logout')); + return Promise.reject(err); + }, +); +``` + +### Migration plan + +1. Land the two wrapper modules above. ~50 LOC total. +2. Codemod or sed-loop: in `inventory/src/`, replace `fetch(` → `apiFetch(` (with the right import) and `axios.get/post/...` → `apiClient.get/post/...`. ~220 call sites — a half-day of careful find-and-replace plus per-page verification. Spot-check the ones with custom `Content-Type` (multipart uploads especially) so the wrapper doesn't clobber multipart boundaries. +3. Leave the `AuthContext.tsx` `/login` and `/me` calls alone — they already work and migrating them adds no value. +4. Run the SPA: log in, exercise Overview / Products / Analytics / Dashboard / etc. with browser devtools open watching for `Authorization` header on every `/api/*` request. + +### Sequencing with Phase 3+6 deploy + +**Two options:** + +A) **Ship F1 first** (recommended). Frontend goes out with the wrapper; nothing changes server-side. Then `pm2 reload` Phase 3+6. Zero-downtime, zero broken-page window. + +B) **Ship together.** F1 and Phase 3+6 land in the same deploy. Brief window (seconds) where the frontend has the wrapper but the server hasn't reloaded yet — wrapper just sends extra headers the old server ignores. Safe. + +Do **not** ship Phase 3+6 first and F1 second. That gives a broken app for as long as F1 takes. + +### Out of scope (kept on `localStorage`) + +Per Phase 6.8, we're not migrating to httpOnly cookie auth. F1 is the minimum work to make the per-service `authenticate()` (Phase 6) actually usable. A future Phase F2 could move to cookies + CSRF double-submit, but that's a much larger change touching the AuthContext, the login flow, and every backend that reads tokens. Not justified for an internal tool with no public sign-up. + +### Note on `/uploads/*` gating (Phase 6.7's Caddyfile change) + +The proposed Caddyfile moves `/uploads/*` behind `forward_auth`. Most product images today are referenced from `` in the SPA — those requests are made by the browser, which **does not include `Authorization` headers on image requests**. Fixing this is part of F1's scope too: either (a) keep `/uploads/*` public (revert that part of 6.7) and accept that uploaded images leak to anyone who guesses a URL, or (b) issue per-image signed URLs from the API and gate those at Caddy. Decide before applying the Caddyfile. + +--- + ## Phase 7 — Caddyfile final form +Status: **Proposed (2026-05-23). Apply blocked on Phase F1.** The full proposed file lives at `inventory-server/deploy/Caddyfile.proposed` and matches the spec below except that vendor handle blocks still point to per-vendor PM2 apps (Phase 4 hasn't merged them yet). See `inventory-server/deploy/README.md` for the apply commands (admin-API + sudo cp pattern from Phase 2 deviation #8). + After all phases, the `tools.acherryontop.com` block looks like: ```caddyfile @@ -571,6 +685,8 @@ Removed: `/dashboard-auth/*`, `/api/aircall/*`, `/api/gorgias/*`, `/api/clarity/ ## Phase 8 — ecosystem.config.cjs final form +Status: **Proposed (2026-05-23).** Full proposed file at `inventory-server/deploy/ecosystem.config.cjs.proposed`. Includes the Phase 6.4 `JWT_SECRET` shadow-override fix and the Phase 6.10 `lt-wordlist-api` token move. Still lists per-vendor PM2 apps until Phase 4 merge ships — that's the only thing keeping app count at 10 instead of the target 5. + ```js module.exports = { apps: [ @@ -646,10 +762,11 @@ Phase 1 unblocks everything (fewer services to convert). Phase 2 is the foundation; nothing else can start until shared `lib/` exists. Phases 3–5 can run in parallel; they touch independent services. Phase 6's sub-items can be developed alongside 3–5 but **enabled** only after them (no point adding `requirePermission` to a route that doesn't yet have `authenticate`). -Phase 7 is the cutover: Caddyfile flip happens when all backend changes are deployed. +**Phase F1 must precede the Phase 3+6 pm2 reload** — without the fetch wrapper, the moment the new code goes live the SPA breaks. Discovered during Phase 3+6 implementation; see Phase F1. +Phase 7 is the cutover: Caddyfile flip happens after F1 ships AND after the `/uploads/*` gating decision in F1 is made. Phase 8 is cleanup: remove dead PM2 entries. -Estimated effort, end-to-end: **~3 weeks of focused work** by one engineer. Phase 1 ≈ 1 day, Phase 2 ≈ 2 days, Phase 3 ≈ 3 days (both services), Phase 4 ≈ 5–7 days (the merge), Phase 5 ≈ 2–3 days, Phase 6 ≈ 3–4 days, Phase 7+8 ≈ 1 day. +Estimated effort, end-to-end: **~3 weeks of focused work** by one engineer. Phase 1 ≈ 1 day, Phase 2 ≈ 2 days, Phase 3 ≈ 3 days (both services), Phase 4 ≈ 5–7 days (the merge), Phase 5 ≈ 2–3 days, Phase 6 ≈ 3–4 days, Phase F1 ≈ 0.5–1 day, Phase 7+8 ≈ 1 day. --- @@ -699,12 +816,13 @@ Each phase produces an independently deployable state. Rollback per phase: These came up in the audit but aren't part of this refactor: -- `httpOnly` cookie auth (deferred — current `localStorage` acceptable for internal tool). +- `httpOnly` cookie auth ("Phase F2" — deferred). Phase F1 keeps `localStorage` + Bearer header because that's the minimum to unblock the Phase 6 `authenticate()` rollout. A future move to cookie auth would touch `AuthContext`, every backend that reads tokens, and introduce CSRF concerns — much larger project. - Replacing PM2 with systemd or Docker. - Test coverage beyond the auth-critical surface. - `apiv2`/`apiv2-test` proxies to `backend.acherryontop.com` — separate system, not touched. - `acot-phone-server` and `lt-wordlist-api` — staying as-is. - Centralized observability stack (Prometheus, Grafana). The logger work in Phase 6.5 sets up the data, but shipping it somewhere is future work. +- ChatRoom XSS remediation (flagged during Phase 6.8 audit — `inventory/src/components/chat/ChatRoom.tsx:277,392` renders user-controlled chat content via `dangerouslySetInnerHTML` without sanitization). Real vulnerability for an internal-but-multi-user tool; separate fix. --- @@ -749,3 +867,15 @@ These are decisions made during Phase 1/2 implementation that amend the spec abo 8. **Caddyfile changes via admin API on `:2020`.** The Caddyfile is owned by root and matt has no passwordless sudo. Cutover used `curl -X POST .../load` on the Caddy admin port (which matt can hit), then a separate `sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile` step to persist the on-disk file. Future Caddyfile changes can follow the same pattern. Backup convention: `/etc/caddy/Caddyfile.bak.YYYY-MM-DD`. 9. **Path-naming.** Plan uses `inventory/` as the top-level (server-side path convention). Locally the equivalent is `inventory-server/`. Whenever the plan says `inventory/dashboard/foo/`, read that as `/var/www/inventory/dashboard/foo/` on the server or `inventory-server/dashboard/foo/` locally. + +10. **Scripts directory kept CJS via package.json shim.** Original plan called for converting "any spawned script" to ESM alongside its caller. Implemented: added `inventory-server/scripts/package.json` with `"type": "commonjs"`. Node's package-type resolution walks up directory by directory, so this overrides the parent's `"type": "module"` for the entire `scripts/` tree (≈15 files including `import/*.js`, `metrics-new/utils/*`, the orchestrator scripts) without renaming any file or touching any `spawn()` callsite. Convert individual scripts to ESM when touched; don't bulk-migrate. + +11. **`src/routes/products.js` had dead multer setup.** Phase 6.7 spec called for hardening the upload route in products.js. There was no upload route — the `multer({ dest })` instance and `importProductsFromCSV` import were dead code left over from a long-ago migration. Strongest 6.7 hardening was deletion: no upload handler = no attack surface. The two real upload paths (`/api/import/upload-image` and `/api/reusable-images/upload`) got tightened MIME+extension allowlists instead. + +12. **Two pre-existing syntax errors in shared/db/ surfaced.** `shared/db/pg.js:13` and `shared/db/redis.js:22` both had `?? Number(...) || N` — mixing `??` and `||` without parentheses is a TC39 syntax error. They passed Phase 2 because nothing imported them yet; Phase 3 smoke-test exposed it. Fixed with parens. + +13. **`import { Pool } from 'pg'` doesn't work in ESM.** The `pg` package is CJS using `module.exports = { Pool, ... }`. Node's ESM-from-CJS interop fails to detect `Pool` as a named export via static analysis. The bulletproof pattern, now used everywhere: `import pg from 'pg'; const { Pool } = pg;`. Same idea for any future CJS-only deps. `src/utils/db.js` already had it; the two auth files needed the fix during execution. + +14. **Frontend Bearer-header gap discovered (drives new Phase F1).** Phase 6 was specified assuming the frontend already sends `Authorization: Bearer` on every API call. It does not — only 7 of ~220 call sites do. Phase 6's `authenticate()` middleware is shipped and ready to enable, but until F1 lands the SPA will 401 on every page. The plan now has Phase F1 to address this explicitly; until then, the Phase 3+6 pm2 reload should not ship unless F1 ships in the same window. + +15. **macOS NFS workflow note.** The `inventory-server/` directory locally is an NFS mount of `/var/www/inventory/` on netcup. Bulk operations (`find`/`grep -r`/mass `node --check`/`npm install`) hang or take minutes locally and pollute file listings with macOS AppleDouble `._*` sidecar files. Default to `ssh netcup` for any sweep across the tree — individual file edits via the editor are fine. diff --git a/inventory-server/auth/add-user.js b/inventory-server/auth/add-user.js index d6857bc..85542b1 100644 --- a/inventory-server/auth/add-user.js +++ b/inventory-server/auth/add-user.js @@ -1,103 +1,75 @@ -require('dotenv').config({ path: '../.env' }); -const bcrypt = require('bcrypt'); -const { Pool } = require('pg'); -const inquirer = require('inquirer'); +import bcrypt from 'bcrypt'; +import pg from 'pg'; +import inquirer from 'inquirer'; -// Log connection details for debugging (remove in production) -console.log('Attempting to connect with:', { - host: process.env.DB_HOST, - user: process.env.DB_USER, - database: process.env.DB_NAME, - port: process.env.DB_PORT -}); +const { Pool } = pg; +import { config as loadEnv } from 'dotenv'; +import { fileURLToPath } from 'node:url'; +import { dirname, resolve as resolvePath } from 'node:path'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +loadEnv({ path: resolvePath(__dirname, '../.env') }); const pool = new Pool({ host: process.env.DB_HOST, user: process.env.DB_USER, password: process.env.DB_PASSWORD, database: process.env.DB_NAME, - port: process.env.DB_PORT, + port: Number(process.env.DB_PORT) || 5432, }); async function promptUser() { - const questions = [ + return inquirer.prompt([ { type: 'input', name: 'username', message: 'Enter username:', - validate: (input) => { - if (input.length < 3) { - return 'Username must be at least 3 characters long'; - } - return true; - } + validate: (input) => input.length >= 3 || 'Username must be at least 3 characters long', }, { type: 'password', name: 'password', message: 'Enter password:', mask: '*', - validate: (input) => { - if (input.length < 8) { - return 'Password must be at least 8 characters long'; - } - return true; - } + validate: (input) => input.length >= 8 || 'Password must be at least 8 characters long', }, { type: 'password', name: 'confirmPassword', message: 'Confirm password:', mask: '*', - validate: (input, answers) => { - if (input !== answers.password) { - return 'Passwords do not match'; - } - return true; - } - } - ]; - - return inquirer.prompt(questions); + validate: (input, answers) => input === answers.password || 'Passwords do not match', + }, + ]); } async function addUser() { try { - // Get user input - const answers = await promptUser(); - const { username, password } = answers; + const { username, password } = await promptUser(); + const hashedPassword = await bcrypt.hash(password, 10); - // Hash password - const saltRounds = 10; - const hashedPassword = await bcrypt.hash(password, saltRounds); - - // Check if user already exists const checkResult = await pool.query( 'SELECT id FROM users WHERE username = $1', [username] ); - if (checkResult.rows.length > 0) { console.error('Error: Username already exists'); process.exit(1); } - // Insert new user const result = await pool.query( 'INSERT INTO users (username, password) VALUES ($1, $2) RETURNING id', [username, hashedPassword] ); - console.log(`User ${username} created successfully with id ${result.rows[0].id}`); } catch (error) { console.error('Error creating user:', error); - console.error('Error details:', error.message); - if (error.code) { - console.error('Error code:', error.code); - } + if (error.code) console.error('Error code:', error.code); } finally { await pool.end(); } } -addUser(); \ No newline at end of file +addUser(); diff --git a/inventory-server/auth/package.json b/inventory-server/auth/package.json index bbbc607..253d981 100644 --- a/inventory-server/auth/package.json +++ b/inventory-server/auth/package.json @@ -2,18 +2,22 @@ "name": "inventory-auth-server", "version": "1.0.0", "description": "Authentication server for inventory management system", + "type": "module", "main": "server.js", "scripts": { - "start": "node server.js" + "start": "node server.js", + "add-user": "node add-user.js" }, "dependencies": { "bcrypt": "^5.1.1", "cors": "^2.8.5", "dotenv": "^16.4.7", "express": "^4.18.2", + "express-rate-limit": "^7.4.0", "inquirer": "^8.2.6", "jsonwebtoken": "^9.0.2", - "morgan": "^1.10.0", - "pg": "^8.11.3" + "pg": "^8.11.3", + "pino": "^9.5.0", + "pino-http": "^10.3.0" } } diff --git a/inventory-server/auth/permissions.js b/inventory-server/auth/permissions.js index cfdd688..717a0b3 100644 --- a/inventory-server/auth/permissions.js +++ b/inventory-server/auth/permissions.js @@ -1,128 +1,73 @@ -// Get pool from global or create a new one if not available -let pool; -if (typeof global.pool !== 'undefined') { - pool = global.pool; -} else { - // If global pool is not available, create a new connection - const { Pool } = require('pg'); - pool = new Pool({ - host: process.env.DB_HOST, - user: process.env.DB_USER, - password: process.env.DB_PASSWORD, - database: process.env.DB_NAME, - port: process.env.DB_PORT, - }); - console.log('Created new database pool in permissions.js'); -} - -/** - * Check if a user has a specific permission - * @param {number} userId - The user ID to check - * @param {string} permissionCode - The permission code to check - * @returns {Promise} - Whether the user has the permission - */ -async function checkPermission(userId, permissionCode) { - try { - // First check if the user is an admin - const adminResult = await pool.query( - 'SELECT is_admin FROM users WHERE id = $1', - [userId] - ); - - // If user is admin, automatically grant permission - if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) { - return true; - } - - // Otherwise check for specific permission - const result = await pool.query( - `SELECT COUNT(*) AS has_permission - FROM user_permissions up - JOIN permissions p ON up.permission_id = p.id - WHERE up.user_id = $1 AND p.code = $2`, - [userId, permissionCode] - ); - - return result.rows[0].has_permission > 0; - } catch (error) { - console.error('Error checking permission:', error); - return false; - } -} - -/** - * Middleware to require a specific permission - * @param {string} permissionCode - The permission code required - * @returns {Function} - Express middleware function - */ -function requirePermission(permissionCode) { - return async (req, res, next) => { +export function createPermissionHelpers({ pool }) { + async function checkPermission(userId, permissionCode) { try { - // Check if user is authenticated - if (!req.user || !req.user.id) { - return res.status(401).json({ error: 'Authentication required' }); - } - - const hasPermission = await checkPermission(req.user.id, permissionCode); - - if (!hasPermission) { - return res.status(403).json({ - error: 'Insufficient permissions', - requiredPermission: permissionCode - }); - } - - next(); - } catch (error) { - console.error('Permission middleware error:', error); - res.status(500).json({ error: 'Server error checking permissions' }); - } - }; -} - -/** - * Get all permissions for a user - * @param {number} userId - The user ID - * @returns {Promise} - Array of permission codes - */ -async function getUserPermissions(userId) { - try { - // Check if user is admin - const adminResult = await pool.query( - 'SELECT is_admin FROM users WHERE id = $1', - [userId] - ); - - if (adminResult.rows.length === 0) { - return []; - } - - const isAdmin = adminResult.rows[0].is_admin; - - if (isAdmin) { - // Admin gets all permissions - const allPermissions = await pool.query('SELECT code FROM permissions'); - return allPermissions.rows.map(p => p.code); - } else { - // Get assigned permissions - const permissions = await pool.query( - `SELECT p.code - FROM permissions p - JOIN user_permissions up ON p.id = up.permission_id - WHERE up.user_id = $1`, + const adminResult = await pool.query( + 'SELECT is_admin FROM users WHERE id = $1', [userId] ); - - return permissions.rows.map(p => p.code); - } - } catch (error) { - console.error('Error getting user permissions:', error); - return []; - } -} + if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) return true; -module.exports = { - checkPermission, - requirePermission, - getUserPermissions -}; \ No newline at end of file + const result = await pool.query( + `SELECT COUNT(*) AS has_permission + FROM user_permissions up + JOIN permissions p ON up.permission_id = p.id + WHERE up.user_id = $1 AND p.code = $2`, + [userId, permissionCode] + ); + return Number(result.rows[0].has_permission) > 0; + } catch (error) { + console.error('Error checking permission:', error); + return false; + } + } + + function requirePermission(permissionCode) { + return async (req, res, next) => { + try { + if (!req.user?.id) { + return res.status(401).json({ error: 'Authentication required' }); + } + const hasPermission = await checkPermission(req.user.id, permissionCode); + if (!hasPermission) { + return res.status(403).json({ + error: 'Insufficient permissions', + requiredPermission: permissionCode, + }); + } + next(); + } catch (error) { + console.error('Permission middleware error:', error); + res.status(500).json({ error: 'Server error checking permissions' }); + } + }; + } + + async function getUserPermissions(userId) { + try { + const adminResult = await pool.query( + 'SELECT is_admin FROM users WHERE id = $1', + [userId] + ); + if (adminResult.rows.length === 0) return []; + + if (adminResult.rows[0].is_admin) { + const allPermissions = await pool.query('SELECT code FROM permissions'); + return allPermissions.rows.map((p) => p.code); + } + + const permissions = await pool.query( + `SELECT p.code + FROM permissions p + JOIN user_permissions up ON p.id = up.permission_id + WHERE up.user_id = $1`, + [userId] + ); + return permissions.rows.map((p) => p.code); + } catch (error) { + console.error('Error getting user permissions:', error); + return []; + } + } + + return { checkPermission, requirePermission, getUserPermissions }; +} diff --git a/inventory-server/auth/routes.js b/inventory-server/auth/routes.js index ef6636d..1783e52 100644 --- a/inventory-server/auth/routes.js +++ b/inventory-server/auth/routes.js @@ -1,533 +1,317 @@ -const express = require('express'); -const router = express.Router(); -const bcrypt = require('bcrypt'); -const jwt = require('jsonwebtoken'); -const { requirePermission, getUserPermissions } = require('./permissions'); +import express from 'express'; +import bcrypt from 'bcrypt'; +import jwt from 'jsonwebtoken'; +import { createPermissionHelpers } from './permissions.js'; -// Get pool from global or create a new one if not available -let pool; -if (typeof global.pool !== 'undefined') { - pool = global.pool; -} else { - // If global pool is not available, create a new connection - const { Pool } = require('pg'); - pool = new Pool({ - host: process.env.DB_HOST, - user: process.env.DB_USER, - password: process.env.DB_PASSWORD, - database: process.env.DB_NAME, - port: process.env.DB_PORT, +export function createAuthRoutes({ pool }) { + const router = express.Router(); + const { requirePermission, getUserPermissions } = createPermissionHelpers({ pool }); + + // Local authenticate(): used by user-management endpoints that need req.user populated + // with id/username/email/is_admin. NOT the per-service authenticate() — that lives in + // shared/auth/middleware.js and is used by downstream services. Auth-server's surface is + // small enough that a local copy is fine; the security boundary is the JWT verify step. + async function authenticate(req, res, next) { + try { + const authHeader = req.headers.authorization; + if (!authHeader || !authHeader.startsWith('Bearer ')) { + return res.status(401).json({ error: 'Authentication required' }); + } + const token = authHeader.split(' ')[1]; + const decoded = jwt.verify(token, process.env.JWT_SECRET); + const result = await pool.query( + 'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1', + [decoded.userId] + ); + if (result.rows.length === 0) { + return res.status(401).json({ error: 'User not found' }); + } + req.user = result.rows[0]; + next(); + } catch (error) { + res.status(401).json({ error: 'Invalid token' }); + } + } + + router.post('/login', async (req, res) => { + try { + const { username, password } = req.body; + const result = await pool.query( + 'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1', + [username] + ); + if (result.rows.length === 0) { + return res.status(401).json({ error: 'Invalid username or password' }); + } + const user = result.rows[0]; + if (!user.is_active) { + return res.status(403).json({ error: 'Account is inactive' }); + } + const validPassword = await bcrypt.compare(password, user.password); + if (!validPassword) { + return res.status(401).json({ error: 'Invalid username or password' }); + } + await pool.query( + 'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1', + [user.id] + ); + const token = jwt.sign( + { userId: user.id, username: user.username }, + process.env.JWT_SECRET, + { expiresIn: '8h' } + ); + const permissions = await getUserPermissions(user.id); + res.json({ + token, + user: { + id: user.id, + username: user.username, + is_admin: user.is_admin, + rocket_chat_user_id: user.rocket_chat_user_id, + permissions, + }, + }); + } catch (error) { + console.error('Login error:', error); + res.status(500).json({ error: 'Server error' }); + } }); - console.log('Created new database pool in routes.js'); -} -// Authentication middleware -const authenticate = async (req, res, next) => { - try { - const authHeader = req.headers.authorization; - if (!authHeader || !authHeader.startsWith('Bearer ')) { - return res.status(401).json({ error: 'Authentication required' }); + router.get('/me', authenticate, async (req, res) => { + try { + const permissions = await getUserPermissions(req.user.id); + res.json({ + id: req.user.id, + username: req.user.username, + email: req.user.email, + is_admin: req.user.is_admin, + rocket_chat_user_id: req.user.rocket_chat_user_id, + permissions, + }); + } catch (error) { + console.error('Error getting current user:', error); + res.status(500).json({ error: 'Server error' }); } + }); - const token = authHeader.split(' ')[1]; - const decoded = jwt.verify(token, process.env.JWT_SECRET); - - // Get user from database - const result = await pool.query( - 'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1', - [decoded.userId] - ); - - console.log('Database query result for user', decoded.userId, ':', result.rows[0]); - - if (result.rows.length === 0) { - return res.status(401).json({ error: 'User not found' }); + router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => { + try { + const result = await pool.query(` + SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login + FROM users + ORDER BY username + `); + res.json(result.rows); + } catch (error) { + console.error('Error getting users:', error); + res.status(500).json({ error: 'Server error' }); } - - // Attach user to request - req.user = result.rows[0]; - next(); - } catch (error) { - console.error('Authentication error:', error); - res.status(401).json({ error: 'Invalid token' }); - } -}; + }); -// Login route -router.post('/login', async (req, res) => { - try { - const { username, password } = req.body; - - // Get user from database - const result = await pool.query( - 'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1', - [username] - ); - - if (result.rows.length === 0) { - return res.status(401).json({ error: 'Invalid username or password' }); - } - - const user = result.rows[0]; - - // Check if user is active - if (!user.is_active) { - return res.status(403).json({ error: 'Account is inactive' }); - } - - // Verify password - const validPassword = await bcrypt.compare(password, user.password); - if (!validPassword) { - return res.status(401).json({ error: 'Invalid username or password' }); - } - - // Update last login - await pool.query( - 'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1', - [user.id] - ); - - // Generate JWT - const token = jwt.sign( - { userId: user.id, username: user.username }, - process.env.JWT_SECRET, - { expiresIn: '8h' } - ); - - // Get user permissions - const permissions = await getUserPermissions(user.id); - - res.json({ - token, - user: { - id: user.id, - username: user.username, - is_admin: user.is_admin, - rocket_chat_user_id: user.rocket_chat_user_id, - permissions - } - }); - } catch (error) { - console.error('Login error:', error); - res.status(500).json({ error: 'Server error' }); - } -}); - -// Get current user -router.get('/me', authenticate, async (req, res) => { - try { - // Get user permissions - const permissions = await getUserPermissions(req.user.id); - - res.json({ - id: req.user.id, - username: req.user.username, - email: req.user.email, - is_admin: req.user.is_admin, - rocket_chat_user_id: req.user.rocket_chat_user_id, - permissions, - // Debug info - _debug_raw_user: req.user, - _server_identifier: "INVENTORY_AUTH_SERVER_MODIFIED" - }); - } catch (error) { - console.error('Error getting current user:', error); - res.status(500).json({ error: 'Server error' }); - } -}); - -// Get all users -router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => { - try { - const result = await pool.query(` - SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login - FROM users - ORDER BY username - `); - - res.json(result.rows); - } catch (error) { - console.error('Error getting users:', error); - res.status(500).json({ error: 'Server error' }); - } -}); - -// Get user with permissions -router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => { - try { - const userId = req.params.id; - - // Get user details - const userResult = await pool.query(` - SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login - FROM users - WHERE id = $1 - `, [userId]); - - if (userResult.rows.length === 0) { - return res.status(404).json({ error: 'User not found' }); - } - - // Get user permissions - const permissionsResult = await pool.query(` - SELECT p.id, p.name, p.code, p.category, p.description - FROM permissions p - JOIN user_permissions up ON p.id = up.permission_id - WHERE up.user_id = $1 - ORDER BY p.category, p.name - `, [userId]); - - // Combine user and permissions - const user = { - ...userResult.rows[0], - permissions: permissionsResult.rows - }; - - res.json(user); - } catch (error) { - console.error('Error getting user:', error); - res.status(500).json({ error: 'Server error' }); - } -}); - -// Create new user -router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => { - const client = await pool.connect(); - - try { - const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body; - - console.log("Create user request:", { - username, - email, - is_admin, - is_active, - rocket_chat_user_id, - permissions: permissions || [] - }); - - // Validate required fields - if (!username || !password) { - return res.status(400).json({ error: 'Username and password are required' }); - } - - // Check if username is taken - const existingUser = await client.query( - 'SELECT id FROM users WHERE username = $1', - [username] - ); - - if (existingUser.rows.length > 0) { - return res.status(400).json({ error: 'Username already exists' }); - } - - // Start transaction - await client.query('BEGIN'); - - // Hash password - const saltRounds = 10; - const hashedPassword = await bcrypt.hash(password, saltRounds); - - // Insert new user - // Convert rocket_chat_user_id to integer if provided - const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null; - - const userResult = await client.query(` - INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at) - VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP) - RETURNING id - `, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]); - - const userId = userResult.rows[0].id; - - // Assign permissions if provided and not admin - if (!is_admin && Array.isArray(permissions) && permissions.length > 0) { - console.log("Adding permissions for new user:", userId); - console.log("Permissions received:", permissions); - - // Check permission format - const permissionIds = permissions.map(p => { - if (typeof p === 'object' && p.id) { - console.log("Permission is an object with ID:", p.id); - return parseInt(p.id, 10); - } else if (typeof p === 'number') { - console.log("Permission is a number:", p); - return p; - } else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) { - console.log("Permission is a string that can be parsed as a number:", p); - return parseInt(p, 10); - } else { - console.log("Unknown permission format:", typeof p, p); - // If it's a permission code, we need to look up the ID - return null; - } - }).filter(id => id !== null); - - console.log("Filtered permission IDs:", permissionIds); - - if (permissionIds.length > 0) { - const permissionValues = permissionIds - .map(permId => `(${userId}, ${permId})`) - .join(','); - - console.log("Inserting permission values:", permissionValues); - - try { - await client.query(` - INSERT INTO user_permissions (user_id, permission_id) - VALUES ${permissionValues} - ON CONFLICT DO NOTHING - `); - console.log("Successfully inserted permissions for new user:", userId); - } catch (err) { - console.error("Error inserting permissions for new user:", err); - throw err; - } - } else { - console.log("No valid permission IDs found to insert for new user"); - } - } else { - console.log("Not adding permissions: is_admin =", is_admin, "permissions array:", Array.isArray(permissions), "length:", permissions ? permissions.length : 0); - } - - await client.query('COMMIT'); - - res.status(201).json({ - id: userId, - message: 'User created successfully' - }); - } catch (error) { - await client.query('ROLLBACK'); - console.error('Error creating user:', error); - res.status(500).json({ error: 'Server error' }); - } finally { - client.release(); - } -}); - -// Update user -router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => { - const client = await pool.connect(); - - try { - const userId = req.params.id; - const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body; - - console.log("Update user request:", { - userId, - username, - email, - is_admin, - is_active, - rocket_chat_user_id, - permissions: permissions || [] - }); - - // Check if user exists - const userExists = await client.query( - 'SELECT id FROM users WHERE id = $1', - [userId] - ); - - if (userExists.rows.length === 0) { - return res.status(404).json({ error: 'User not found' }); - } - - // Start transaction - await client.query('BEGIN'); - - // Build update fields - const updateFields = []; - const updateValues = [userId]; // First parameter is the user ID - let paramIndex = 2; - - if (username !== undefined) { - updateFields.push(`username = $${paramIndex++}`); - updateValues.push(username); - } - - if (email !== undefined) { - updateFields.push(`email = $${paramIndex++}`); - updateValues.push(email || null); - } - - if (is_admin !== undefined) { - updateFields.push(`is_admin = $${paramIndex++}`); - updateValues.push(!!is_admin); - } - - if (is_active !== undefined) { - updateFields.push(`is_active = $${paramIndex++}`); - updateValues.push(!!is_active); - } - - if (rocket_chat_user_id !== undefined) { - updateFields.push(`rocket_chat_user_id = $${paramIndex++}`); - // Convert to integer if not null/undefined, otherwise null - const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null; - updateValues.push(rcUserId); - } - - // Update password if provided - if (password) { - const saltRounds = 10; - const hashedPassword = await bcrypt.hash(password, saltRounds); - updateFields.push(`password = $${paramIndex++}`); - updateValues.push(hashedPassword); - } - - // Update user if there are fields to update - if (updateFields.length > 0) { - updateFields.push(`updated_at = CURRENT_TIMESTAMP`); - - await client.query(` - UPDATE users - SET ${updateFields.join(', ')} + router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => { + try { + const userId = req.params.id; + const userResult = await pool.query(` + SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login + FROM users WHERE id = $1 - `, updateValues); + `, [userId]); + if (userResult.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + const permissionsResult = await pool.query(` + SELECT p.id, p.name, p.code, p.category, p.description + FROM permissions p + JOIN user_permissions up ON p.id = up.permission_id + WHERE up.user_id = $1 + ORDER BY p.category, p.name + `, [userId]); + res.json({ + ...userResult.rows[0], + permissions: permissionsResult.rows, + }); + } catch (error) { + console.error('Error getting user:', error); + res.status(500).json({ error: 'Server error' }); } - - // Update permissions if provided - if (Array.isArray(permissions)) { - console.log("Updating permissions for user:", userId); - console.log("Permissions received:", permissions); - - // First remove existing permissions - await client.query( - 'DELETE FROM user_permissions WHERE user_id = $1', + }); + + router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => { + const client = await pool.connect(); + try { + const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body; + if (!username || !password) { + return res.status(400).json({ error: 'Username and password are required' }); + } + const existingUser = await client.query( + 'SELECT id FROM users WHERE username = $1', + [username] + ); + if (existingUser.rows.length > 0) { + return res.status(400).json({ error: 'Username already exists' }); + } + await client.query('BEGIN'); + const hashedPassword = await bcrypt.hash(password, 10); + const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null; + const userResult = await client.query(` + INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at) + VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP) + RETURNING id + `, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]); + const userId = userResult.rows[0].id; + + if (!is_admin && Array.isArray(permissions) && permissions.length > 0) { + const permissionIds = normalizePermissionIds(permissions); + if (permissionIds.length > 0) { + await client.query( + `INSERT INTO user_permissions (user_id, permission_id) + SELECT $1, unnest($2::int[]) + ON CONFLICT DO NOTHING`, + [userId, permissionIds] + ); + } + } + + await client.query('COMMIT'); + res.status(201).json({ id: userId, message: 'User created successfully' }); + } catch (error) { + await client.query('ROLLBACK'); + console.error('Error creating user:', error); + res.status(500).json({ error: 'Server error' }); + } finally { + client.release(); + } + }); + + router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => { + const client = await pool.connect(); + try { + const userId = req.params.id; + const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body; + + const userExists = await client.query('SELECT id FROM users WHERE id = $1', [userId]); + if (userExists.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + + await client.query('BEGIN'); + + const updateFields = []; + const updateValues = [userId]; + let paramIndex = 2; + + if (username !== undefined) { updateFields.push(`username = $${paramIndex++}`); updateValues.push(username); } + if (email !== undefined) { updateFields.push(`email = $${paramIndex++}`); updateValues.push(email || null); } + if (is_admin !== undefined) { updateFields.push(`is_admin = $${paramIndex++}`); updateValues.push(!!is_admin); } + if (is_active !== undefined) { updateFields.push(`is_active = $${paramIndex++}`); updateValues.push(!!is_active); } + if (rocket_chat_user_id !== undefined) { + updateFields.push(`rocket_chat_user_id = $${paramIndex++}`); + updateValues.push(rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null); + } + if (password) { + const hashedPassword = await bcrypt.hash(password, 10); + updateFields.push(`password = $${paramIndex++}`); + updateValues.push(hashedPassword); + } + + if (updateFields.length > 0) { + updateFields.push(`updated_at = CURRENT_TIMESTAMP`); + await client.query(` + UPDATE users SET ${updateFields.join(', ')} WHERE id = $1 + `, updateValues); + } + + if (Array.isArray(permissions)) { + await client.query('DELETE FROM user_permissions WHERE user_id = $1', [userId]); + const newIsAdmin = is_admin !== undefined + ? is_admin + : (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin; + + if (!newIsAdmin && permissions.length > 0) { + const permissionIds = normalizePermissionIds(permissions); + if (permissionIds.length > 0) { + await client.query( + `INSERT INTO user_permissions (user_id, permission_id) + SELECT $1, unnest($2::int[]) + ON CONFLICT DO NOTHING`, + [userId, permissionIds] + ); + } + } + } + + await client.query('COMMIT'); + res.json({ message: 'User updated successfully' }); + } catch (error) { + await client.query('ROLLBACK'); + console.error('Error updating user:', error); + res.status(500).json({ error: 'Server error' }); + } finally { + client.release(); + } + }); + + router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => { + try { + const userId = req.params.id; + if (req.user.id === parseInt(userId, 10)) { + return res.status(400).json({ error: 'Cannot delete your own account' }); + } + const result = await pool.query( + 'DELETE FROM users WHERE id = $1 RETURNING id', [userId] ); - console.log("Deleted existing permissions for user:", userId); - - // Add new permissions if any and not admin - const newIsAdmin = is_admin !== undefined ? is_admin : (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin; - - console.log("User is admin:", newIsAdmin); - - if (!newIsAdmin && permissions.length > 0) { - console.log("Adding permissions:", permissions); - - // Check permission format - const permissionIds = permissions.map(p => { - if (typeof p === 'object' && p.id) { - console.log("Permission is an object with ID:", p.id); - return parseInt(p.id, 10); - } else if (typeof p === 'number') { - console.log("Permission is a number:", p); - return p; - } else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) { - console.log("Permission is a string that can be parsed as a number:", p); - return parseInt(p, 10); - } else { - console.log("Unknown permission format:", typeof p, p); - // If it's a permission code, we need to look up the ID - return null; - } - }).filter(id => id !== null); - - console.log("Filtered permission IDs:", permissionIds); - - if (permissionIds.length > 0) { - const permissionValues = permissionIds - .map(permId => `(${userId}, ${permId})`) - .join(','); - - console.log("Inserting permission values:", permissionValues); - - try { - await client.query(` - INSERT INTO user_permissions (user_id, permission_id) - VALUES ${permissionValues} - ON CONFLICT DO NOTHING - `); - console.log("Successfully inserted permissions for user:", userId); - } catch (err) { - console.error("Error inserting permissions:", err); - throw err; - } - } else { - console.log("No valid permission IDs found to insert"); - } + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); } + res.json({ message: 'User deleted successfully' }); + } catch (error) { + console.error('Error deleting user:', error); + res.status(500).json({ error: 'Server error' }); } - - await client.query('COMMIT'); - - res.json({ message: 'User updated successfully' }); - } catch (error) { - await client.query('ROLLBACK'); - console.error('Error updating user:', error); - res.status(500).json({ error: 'Server error' }); - } finally { - client.release(); - } -}); + }); -// Delete user -router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => { - try { - const userId = req.params.id; - - // Check that user is not deleting themselves - if (req.user.id === parseInt(userId, 10)) { - return res.status(400).json({ error: 'Cannot delete your own account' }); + router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => { + try { + const result = await pool.query(` + SELECT category, json_agg( + json_build_object( + 'id', id, 'name', name, 'code', code, 'description', description + ) ORDER BY name + ) as permissions + FROM permissions + GROUP BY category + ORDER BY category + `); + res.json(result.rows); + } catch (error) { + console.error('Error getting permissions:', error); + res.status(500).json({ error: 'Server error' }); } - - // Delete user (this will cascade to user_permissions due to FK constraints) - const result = await pool.query( - 'DELETE FROM users WHERE id = $1 RETURNING id', - [userId] - ); - - if (result.rows.length === 0) { - return res.status(404).json({ error: 'User not found' }); + }); + + router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => { + try { + const result = await pool.query(` + SELECT * FROM permissions ORDER BY category, name + `); + res.json(result.rows); + } catch (error) { + console.error('Error getting permissions:', error); + res.status(500).json({ error: 'Server error' }); } - - res.json({ message: 'User deleted successfully' }); - } catch (error) { - console.error('Error deleting user:', error); - res.status(500).json({ error: 'Server error' }); - } -}); + }); -// Get all permissions grouped by category -router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => { - try { - const result = await pool.query(` - SELECT category, json_agg( - json_build_object( - 'id', id, - 'name', name, - 'code', code, - 'description', description - ) ORDER BY name - ) as permissions - FROM permissions - GROUP BY category - ORDER BY category - `); - - res.json(result.rows); - } catch (error) { - console.error('Error getting permissions:', error); - res.status(500).json({ error: 'Server error' }); - } -}); + return router; +} -// Get all permissions -router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => { - try { - const result = await pool.query(` - SELECT * - FROM permissions - ORDER BY category, name - `); - - res.json(result.rows); - } catch (error) { - console.error('Error getting permissions:', error); - res.status(500).json({ error: 'Server error' }); - } -}); - -module.exports = router; \ No newline at end of file +function normalizePermissionIds(permissions) { + return permissions + .map((p) => { + if (typeof p === 'object' && p?.id) return parseInt(p.id, 10); + if (typeof p === 'number') return p; + if (typeof p === 'string' && !Number.isNaN(parseInt(p, 10))) return parseInt(p, 10); + return null; + }) + .filter((id) => id !== null && !Number.isNaN(id)); +} diff --git a/inventory-server/auth/server.js b/inventory-server/auth/server.js index fd6e32c..82b26e1 100644 --- a/inventory-server/auth/server.js +++ b/inventory-server/auth/server.js @@ -1,195 +1,84 @@ -require('dotenv').config({ path: '../.env' }); -const express = require('express'); -const cors = require('cors'); -const bcrypt = require('bcrypt'); -const jwt = require('jsonwebtoken'); -const { Pool } = require('pg'); -const morgan = require('morgan'); -const authRoutes = require('./routes'); +import 'dotenv/config'; +import express from 'express'; +import cors from 'cors'; +import pg from 'pg'; +import { fileURLToPath } from 'node:url'; -// Log startup configuration -console.log('Starting auth server with config:', { +const { Pool } = pg; +import { dirname, resolve as resolvePath } from 'node:path'; +import { config as loadEnv } from 'dotenv'; + +import { corsOptions } from '../shared/cors/policy.js'; +import { requestLog } from '../shared/logging/request-log.js'; +import { logger } from '../shared/logging/logger.js'; +import { errorHandler } from '../shared/errors/handler.js'; +import { loginLimiter, verifyLimiter } from '../shared/rate-limit/login.js'; +import { extractBearerToken, verifyToken, TokenError } from '../shared/auth/verify.js'; + +import { createAuthRoutes } from './routes.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +// auth/ lives at inventory-server/auth/, so .env one level up +loadEnv({ path: resolvePath(__dirname, '../.env') }); + +if (!process.env.JWT_SECRET) { + logger.error('JWT_SECRET is not set; refusing to start'); + process.exit(1); +} + +logger.info({ host: process.env.DB_HOST, user: process.env.DB_USER, database: process.env.DB_NAME, port: process.env.DB_PORT, - auth_port: process.env.AUTH_PORT -}); + auth_port: process.env.AUTH_PORT, +}, 'starting auth server'); const app = express(); -const port = process.env.AUTH_PORT || 3011; +const port = Number(process.env.AUTH_PORT) || 3011; -// Database configuration const pool = new Pool({ host: process.env.DB_HOST, user: process.env.DB_USER, password: process.env.DB_PASSWORD, database: process.env.DB_NAME, - port: process.env.DB_PORT, + port: Number(process.env.DB_PORT) || 5432, }); -// Make pool available globally -global.pool = pool; - -// Middleware -app.use(express.json()); -app.use(morgan('combined')); -app.use(cors({ - origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://acot.site', 'https://tools.acherryontop.com'], - credentials: true -})); - -// Login endpoint -app.post('/login', async (req, res) => { - const { username, password } = req.body; - - try { - // Get user from database - const result = await pool.query( - 'SELECT id, username, password, is_admin, is_active FROM users WHERE username = $1', - [username] - ); - - const user = result.rows[0]; - - // Check if user exists and password is correct - if (!user || !(await bcrypt.compare(password, user.password))) { - return res.status(401).json({ error: 'Invalid username or password' }); - } - - // Check if user is active - if (!user.is_active) { - return res.status(403).json({ error: 'Account is inactive' }); - } - - // Update last login timestamp - await pool.query( - 'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1', - [user.id] - ); - - // Generate JWT token - const token = jwt.sign( - { userId: user.id, username: user.username }, - process.env.JWT_SECRET, - { expiresIn: '24h' } - ); - - // Get user permissions for the response - const permissionsResult = await pool.query(` - SELECT code - FROM permissions p - JOIN user_permissions up ON p.id = up.permission_id - WHERE up.user_id = $1 - `, [user.id]); - - const permissions = permissionsResult.rows.map(row => row.code); - - res.json({ - token, - user: { - id: user.id, - username: user.username, - is_admin: user.is_admin, - permissions: user.is_admin ? [] : permissions - } - }); - } catch (error) { - console.error('Login error:', error); - res.status(500).json({ error: 'Internal server error' }); - } -}); - -// User info endpoint -app.get('/me', async (req, res) => { - const authHeader = req.headers.authorization; - - if (!authHeader || !authHeader.startsWith('Bearer ')) { - return res.status(401).json({ error: 'No token provided' }); - } - - try { - const token = authHeader.split(' ')[1]; - const decoded = jwt.verify(token, process.env.JWT_SECRET); - - // Get user details from database - const userResult = await pool.query( - 'SELECT id, username, email, is_admin, rocket_chat_user_id, is_active FROM users WHERE id = $1', - [decoded.userId] - ); - - if (userResult.rows.length === 0) { - return res.status(404).json({ error: 'User not found' }); - } - - const user = userResult.rows[0]; - - // Check if user is active - if (!user.is_active) { - return res.status(403).json({ error: 'Account is inactive' }); - } - - // Get user permissions - let permissions = []; - if (!user.is_admin) { - const permissionsResult = await pool.query(` - SELECT code - FROM permissions p - JOIN user_permissions up ON p.id = up.permission_id - WHERE up.user_id = $1 - `, [user.id]); - - permissions = permissionsResult.rows.map(row => row.code); - } - - res.json({ - id: user.id, - username: user.username, - email: user.email, - rocket_chat_user_id: user.rocket_chat_user_id, - is_admin: user.is_admin, - permissions: permissions - }); - } catch (error) { - console.error('Token verification error:', error); - res.status(401).json({ error: 'Invalid token' }); - } -}); +app.use(requestLog()); +app.use(express.json({ limit: '1mb' })); +app.use(cors(corsOptions)); // Caddy forward_auth target: JWT signature check only, no DB hit. -// Returns 200 with X-User-Id / X-User-Username on success, 401 otherwise. -// Per-service middleware re-verifies the token independently; these headers -// are informational and must not be trusted by upstreams. -app.all('/verify', (req, res) => { - const header = req.headers.authorization; - if (!header || !header.startsWith('Bearer ')) { - return res.status(401).json({ error: 'No token provided' }); - } +// Returns 200 with X-User-Id / X-User-Username on success; 401 otherwise. +// Per-service middleware re-verifies independently; these headers are informational. +app.all('/verify', verifyLimiter, (req, res) => { try { - const decoded = jwt.verify(header.slice(7), process.env.JWT_SECRET); + const token = extractBearerToken(req.headers.authorization); + const decoded = verifyToken(token, process.env.JWT_SECRET); res.set('X-User-Id', String(decoded.userId)); if (decoded.username) res.set('X-User-Username', decoded.username); res.status(200).end(); } catch (err) { - res.status(401).json({ error: err.name === 'TokenExpiredError' ? 'Token expired' : 'Invalid token' }); + if (err instanceof TokenError) { + return res.status(401).json({ error: err.message }); + } + res.status(401).json({ error: 'Invalid token' }); } }); -// Mount all routes from routes.js -app.use('/', authRoutes); +// Login route gets its own rate limiter to slow credential stuffing. +app.use('/login', loginLimiter); -// Health check endpoint -app.get('/health', (req, res) => { - res.json({ status: 'healthy' }); -}); +// Mount user-management + /login + /me from routes.js +app.use('/', createAuthRoutes({ pool })); -// Error handling middleware -app.use((err, req, res, next) => { - console.error(err.stack); - res.status(500).json({ error: 'Something broke!' }); -}); +app.get('/health', (req, res) => res.json({ status: 'healthy' })); + +app.use(errorHandler); -// Start server app.listen(port, () => { - console.log(`Auth server running on port ${port}`); + logger.info({ port }, 'auth server listening'); }); diff --git a/inventory-server/deploy/Caddyfile.proposed b/inventory-server/deploy/Caddyfile.proposed new file mode 100644 index 0000000..58c843a --- /dev/null +++ b/inventory-server/deploy/Caddyfile.proposed @@ -0,0 +1,87 @@ +# Phase 6.1 + 6.6 + 6.7: tools.acherryontop.com final form +# +# Apply on the server with: +# curl -X POST http://localhost:2020/load \ +# -H 'Content-Type: text/caddyfile' \ +# --data-binary @/home/matt/Caddyfile.new +# sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile +# sudo cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.bak.$(date +%F) +# +# Differences from current /etc/caddy/Caddyfile: +# 1. forward_auth gate added in front of /api/* and /chat-api/* (Phase 6.1). +# 2. /uploads/* moved behind the forward_auth gate (Phase 6.7 — was public). +# 3. LAN wildcards / Access-Control-Allow-Origin "*" defaults dropped from /api/* (Phase 6.6). +# 4. Removed dead /api/{aircall,gorgias,clarity}/* routes (Phase 1 — already cleaned up here). +# +# Phase 4 (dashboard-server merge) is NOT yet reflected — klaviyo/meta/google/typeform +# still route to their per-vendor PM2 apps in the live Caddyfile. Update those handle +# blocks to localhost:3015 when dashboard-server ships. + +tools.acherryontop.com { + import security_headers + + # Public: login endpoint + handle /auth-inv/* { + uri strip_prefix /auth-inv + reverse_proxy localhost:3011 + } + + # Public: static frontend assets (long-cache) + @static path *.js *.css *.png *.jpg *.jpeg *.gif *.ico *.svg *.woff *.woff2 + handle @static { + header Cache-Control "public, max-age=2592000" + root * /var/www/inventory/frontend/build + file_server + } + + # ----- Authenticated zone ----- + # Phase 6.1: forward_auth subrequest to auth-server:/verify. 2xx → proceeds. + # 401/403 → Caddy returns auth-server response to client; backend never sees it. + @gated path /api/* /chat-api/* /uploads/* + handle @gated { + forward_auth localhost:3011 { + uri /verify + copy_headers Authorization + } + + # Phase 6.7: /uploads/* now behind the gate (was a public file_server before) + handle /uploads/* { + root * /var/www/inventory + file_server + } + + # Vendor dashboard routes + # NOTE: pre-Phase-4 these are still on separate ports; updates here when merged. + handle /api/klaviyo/* { reverse_proxy localhost:3015 } + handle /api/meta/* { reverse_proxy localhost:3015 } + handle /api/google-analytics/* { reverse_proxy localhost:3015 } + handle /api/typeform/* { reverse_proxy localhost:3015 } + + # ACOT + handle /api/acot/* { reverse_proxy localhost:3012 } + + # Chat + handle /chat-api/* { + uri strip_prefix /chat-api + reverse_proxy localhost:3014 + } + + # Catch-all: inventory-server + handle /api/* { reverse_proxy localhost:3010 } + } + + # Out-of-band probes (unauthenticated) + handle /health { reverse_proxy localhost:3010 } + + # SPA fallback (public assets) + handle { + root * /var/www/inventory/frontend/build + try_files {path} /index.html + file_server + encode gzip + } + + handle_errors { + respond "{err.status_code} {err.status_text}" + } +} diff --git a/inventory-server/deploy/README.md b/inventory-server/deploy/README.md new file mode 100644 index 0000000..e735f2a --- /dev/null +++ b/inventory-server/deploy/README.md @@ -0,0 +1,61 @@ +# Server-side deployment artefacts for Phase 3 + 6 + +This directory contains proposed versions of files that live outside the +inventory-server tree on production. Each is a recommendation — apply +deliberately and only after the Node-side ESM + auth changes are deployed and +smoke-tested. + +| Source | Target | Phase | +| --------------------------------------- | ------------------------------------- | -------- | +| `Caddyfile.proposed` | `/etc/caddy/Caddyfile` | 6.1, 6.6, 6.7 | +| `ecosystem.config.cjs.proposed` | `/var/www/ecosystem.config.cjs` | 6.4, 6.10 | + +## Recommended apply order + +1. **Deploy the Node code first** (this repo). PM2 reload picks up the new + ESM-mode inventory-server and auth-server. At this point the frontend will + start hitting 401s on every API call because the new `authenticate()` + middleware is live and the frontend doesn't carry Bearer tokens on most + fetches. **This is expected per the discussion in CONSOLIDATION_PLAN.md + §6** — the frontend fetch-wrapper work is the next deliverable. + +2. **Apply the ecosystem.cjs change** (Phase 6.4) to fix the `JWT_SECRET` + shadow-override before the next pm2 restart silently re-introduces it. + +3. **Apply the Caddyfile change** (Phase 6.1) only after the frontend is + sending Bearer tokens. Until then, `forward_auth` will reject every page + refresh at the edge. + +## Caddyfile apply pattern + +Caddy admin API is on `:2020` (matt has access). On-disk file needs root. + +```bash +# Upload + load atomically into the running Caddy +curl -X POST http://localhost:2020/load \ + -H 'Content-Type: text/caddyfile' \ + --data-binary @/home/matt/Caddyfile.new + +# Persist to disk (separate sudo step) +sudo cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.bak.$(date +%F) +sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile +``` + +## ecosystem.cjs apply pattern + +```bash +sudo cp /var/www/ecosystem.config.cjs /var/www/ecosystem.config.cjs.bak.$(date +%F) +sudo cp /home/matt/ecosystem.config.cjs.new /var/www/ecosystem.config.cjs +pm2 reload ecosystem.config.cjs --update-env +pm2 env new-auth-server | grep -i jwt # JWT_SECRET from .env only +``` + +## Rollback + +Every applied file leaves a `.bak.YYYY-MM-DD` next to it. `sudo cp +` then `caddy reload` / `pm2 reload`. + +Phase 6 changes are *additive* — if `forward_auth` causes problems, comment +out the directive in the live Caddyfile and per-server middleware +(`authenticate()` in inventory-server, in particular) continues protecting +routes. diff --git a/inventory-server/deploy/ecosystem.config.cjs.proposed b/inventory-server/deploy/ecosystem.config.cjs.proposed new file mode 100644 index 0000000..90b81dc --- /dev/null +++ b/inventory-server/deploy/ecosystem.config.cjs.proposed @@ -0,0 +1,114 @@ +// Phase 6.4 + 6.10: proposed /var/www/ecosystem.config.cjs +// +// Diffs from the live file (CONSOLIDATION_PLAN.md §6.4 / §6.10): +// 1. Drop the `JWT_SECRET: process.env.JWT_SECRET` override in new-auth-server's +// env block. That override shadowed the .env value with whatever shell var was +// exported when pm2 was last started — causing the silent-divergence footgun +// called out in CLAUDE.md memory. With it removed, .env is the single source. +// 2. Move ADD_WORD_TOKEN out of the inline env into /opt/lt-wordlist-api/.env. +// The PM2 entry below no longer references it. +// 3. Rename the placeholder script paths if you have any half-finished ones; this +// version only lists the apps that actually run today (post-Phase-1 cleanup). +// +// To apply: +// sudo cp /var/www/ecosystem.config.cjs /var/www/ecosystem.config.cjs.bak.$(date +%F) +// sudo cp /home/matt/ecosystem.config.cjs.new /var/www/ecosystem.config.cjs +// pm2 reload ecosystem.config.cjs --update-env +// +// Verify after reload: +// pm2 env new-auth-server | grep -i jwt # should show JWT_SECRET from .env only +// pm2 env new-auth-server | grep ADD_WORD # should be empty + +const inventoryEnv = require('dotenv').config({ path: '/var/www/inventory/.env' }).parsed; + +module.exports = { + apps: [ + { + name: 'new-auth-server', // Phase 8 may rename to 'auth-server' — cosmetic + script: './inventory/auth/server.js', + cwd: '/var/www', + env: { + ...inventoryEnv, + NODE_ENV: 'production', + AUTH_PORT: 3011, + // PHASE 6.4 FIX: no JWT_SECRET override here. .env wins. + }, + max_memory_restart: '500M', + error_file: '/var/log/pm2/new-auth-server-error.log', + out_file: '/var/log/pm2/new-auth-server-out.log', + }, + { + name: 'inventory-server', + script: './inventory/src/server.js', + cwd: '/var/www', + env: { + ...inventoryEnv, + NODE_ENV: 'production', + PORT: 3010, + UPLOADS_DIR: '/var/www/inventory/uploads', + }, + max_memory_restart: '1G', + error_file: '/var/log/pm2/inventory-server-error.log', + out_file: '/var/log/pm2/inventory-server-out.log', + }, + { + name: 'chat-server', + script: './inventory/chat/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production', PORT: 3014 }, + max_memory_restart: '500M', + }, + { + name: 'acot-server', + script: './inventory/dashboard/acot-server/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production', ACOT_PORT: 3012 }, + max_memory_restart: '1G', + }, + // Per-vendor dashboard apps stay until Phase 4 merge ships. + { + name: 'klaviyo-server', + script: './inventory/dashboard/klaviyo-server/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production' }, + max_memory_restart: '500M', + }, + { + name: 'meta-server', + script: './inventory/dashboard/meta-server/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production' }, + max_memory_restart: '300M', + }, + { + name: 'google-server', + script: './inventory/dashboard/google-server/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production' }, + max_memory_restart: '300M', + }, + { + name: 'typeform-server', + script: './inventory/dashboard/typeform-server/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production' }, + max_memory_restart: '300M', + }, + // PHASE 6.10: lt-wordlist-api now loads ADD_WORD_TOKEN from /opt/lt-wordlist-api/.env + // (no longer hardcoded here). Rotate that token's value when applying this change. + { + name: 'lt-wordlist-api', + script: '/opt/lt-wordlist-api/server.js', + cwd: '/opt/lt-wordlist-api', + env: { NODE_ENV: 'production' }, + max_memory_restart: '200M', + }, + { + name: 'acot-phone-server', + script: './inventory/acot-phone/server.js', + cwd: '/var/www', + env: { ...inventoryEnv, NODE_ENV: 'production' }, + max_memory_restart: '300M', + }, + ], +}; diff --git a/inventory-server/migrations/005_phase6_permission_codes.sql b/inventory-server/migrations/005_phase6_permission_codes.sql new file mode 100644 index 0000000..b9030f0 --- /dev/null +++ b/inventory-server/migrations/005_phase6_permission_codes.sql @@ -0,0 +1,52 @@ +-- Phase 6.2: per-route permission codes +-- Seeds the permission codes referenced by Phase 6 hardening middleware. +-- Safe to run multiple times (ON CONFLICT DO NOTHING). +-- +-- Codes follow the plan's spec (CONSOLIDATION_PLAN.md §6.2): +-- product_import — POST/PUT/DELETE on /api/import +-- data_management — POST/PUT/DELETE on /api/csv (data-management.js) +-- ai_admin — POST/PUT/DELETE on /api/ai-prompts, /api/ai-validation +-- templates_write — POST/PUT/DELETE on /api/templates +-- image_admin — POST/DELETE on /api/reusable-images +-- audit_read — reserved for future read-gating on audit logs +-- acot_admin — reserved for acot-server (Phase 5 scope) +-- klaviyo_* / meta_* / google_* / typeform_* — reserved for dashboard-server (Phase 4 scope) +-- +-- Admin users (is_admin = true) automatically pass any requirePermission() check, +-- so this migration does not auto-grant codes to admins. New non-admin users get +-- write access only when explicitly granted via the user-management UI. + +INSERT INTO permissions (code, name, category, description) VALUES + ('product_import', 'Product Import (write)', 'Imports', + 'Allows POST/PUT/DELETE on /api/import — uploads, deletes, generate-upc, etc.'), + ('data_management', 'Data Management (write)', 'Data', + 'Allows POST/PUT/DELETE on /api/csv — CSV operations, full updates, full resets.'), + ('ai_admin', 'AI Settings Admin', 'AI', + 'Allows write access to AI prompts and AI validation endpoints.'), + ('templates_write', 'Template Editing', 'Templates', + 'Allows POST/PUT/DELETE on /api/templates.'), + ('image_admin', 'Image Management', 'Images', + 'Allows uploads and deletions on /api/reusable-images.'), + ('audit_read', 'Audit Log Access', 'Audit', + 'Reserved for future read-gating of import + product-editor audit logs.'), + ('klaviyo_write', 'Klaviyo Write', 'Dashboard', + 'Reserved for dashboard-server: mutates Klaviyo lists/segments.'), + ('klaviyo_admin', 'Klaviyo Admin', 'Dashboard', + 'Reserved for dashboard-server: triggers campaign syncs.'), + ('meta_write', 'Meta Write', 'Dashboard', + 'Reserved for dashboard-server: Meta API write operations.'), + ('google_write', 'Google Analytics Write', 'Dashboard', + 'Reserved for dashboard-server: GA write operations.'), + ('typeform_write', 'Typeform Write', 'Dashboard', + 'Reserved for dashboard-server: Typeform write operations.'), + ('acot_admin', 'ACOT Server Admin', 'ACOT', + 'Reserved for acot-server admin endpoints.') +ON CONFLICT (code) DO NOTHING; + +-- Phase 2 deviation #6 cleanup: drop defunct frontend permissions if present. +-- These corresponded to the removed Aircall/Gorgias dashboards. +DELETE FROM user_permissions + WHERE permission_id IN ( + SELECT id FROM permissions WHERE code IN ('dashboard:gorgias', 'dashboard:calls') + ); +DELETE FROM permissions WHERE code IN ('dashboard:gorgias', 'dashboard:calls'); diff --git a/inventory-server/package-lock.json b/inventory-server/package-lock.json index 20923ec..332b43f 100644 --- a/inventory-server/package-lock.json +++ b/inventory-server/package-lock.json @@ -18,10 +18,15 @@ "diff": "^7.0.0", "dotenv": "^16.4.7", "express": "^4.18.2", + "express-rate-limit": "^7.4.0", + "ioredis": "^5.10.1", + "jsonwebtoken": "^9.0.2", "multer": "^1.4.5-lts.1", "mysql2": "^3.12.0", "openai": "^6.0.0", "pg": "^8.14.1", + "pino": "^9.5.0", + "pino-http": "^10.3.0", "pm2": "^5.3.0", "sharp": "^0.33.5", "ssh2": "^1.16.0", @@ -409,6 +414,12 @@ "url": "https://opencollective.com/libvips" } }, + "node_modules/@ioredis/commands": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/@ioredis/commands/-/commands-1.5.1.tgz", + "integrity": "sha512-JH8ZL/ywcJyR9MmJ5BNqZllXNZQqQbnVZOqpPQqE1vHiFgAw4NHbvE0FOduNU8IX9babitBT46571OnPTT0Zcw==", + "license": "MIT" + }, "node_modules/@mapbox/node-pre-gyp": { "version": "1.0.11", "resolved": "https://registry.npmjs.org/@mapbox/node-pre-gyp/-/node-pre-gyp-1.0.11.tgz", @@ -477,6 +488,12 @@ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", "license": "MIT" }, + "node_modules/@pinojs/redact": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/@pinojs/redact/-/redact-0.4.0.tgz", + "integrity": "sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==", + "license": "MIT" + }, "node_modules/@pm2/agent": { "version": "2.0.4", "resolved": "https://registry.npmjs.org/@pm2/agent/-/agent-2.0.4.tgz", @@ -958,6 +975,15 @@ "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", "license": "MIT" }, + "node_modules/atomic-sleep": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/atomic-sleep/-/atomic-sleep-1.0.0.tgz", + "integrity": "sha512-kNOjDqAh7px0XWNI+4QbzoiR/nTkHAWNud2uvnJquD1/x5a7EQZMJT0AczqK0Qn67oY/TTQ1LbUKajZpp3I9tQ==", + "license": "MIT", + "engines": { + "node": ">=8.0.0" + } + }, "node_modules/aws-ssl-profiles": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/aws-ssl-profiles/-/aws-ssl-profiles-1.1.2.tgz", @@ -1092,6 +1118,12 @@ "node": ">=8" } }, + "node_modules/buffer-equal-constant-time": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz", + "integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==", + "license": "BSD-3-Clause" + }, "node_modules/buffer-from": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz", @@ -1219,6 +1251,15 @@ "node": ">=8.10.0" } }, + "node_modules/cluster-key-slot": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz", + "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==", + "license": "Apache-2.0", + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/color": { "version": "4.2.3", "resolved": "https://registry.npmjs.org/color/-/color-4.2.3.tgz", @@ -1529,6 +1570,15 @@ "node": ">= 0.4" } }, + "node_modules/ecdsa-sig-formatter": { + "version": "1.0.11", + "resolved": "https://registry.npmjs.org/ecdsa-sig-formatter/-/ecdsa-sig-formatter-1.0.11.tgz", + "integrity": "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==", + "license": "Apache-2.0", + "dependencies": { + "safe-buffer": "^5.0.1" + } + }, "node_modules/ee-first": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", @@ -1738,6 +1788,21 @@ "url": "https://opencollective.com/express" } }, + "node_modules/express-rate-limit": { + "version": "7.5.1", + "resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-7.5.1.tgz", + "integrity": "sha512-7iN8iPMDzOMHPUYllBEsQdWVB6fPDMPqwjBaFrgr4Jgr/+okjvzAy+UHlYYL/Vs0OsOrMkwS6PJDkFlJwoxUnw==", + "license": "MIT", + "engines": { + "node": ">= 16" + }, + "funding": { + "url": "https://github.com/sponsors/express-rate-limit" + }, + "peerDependencies": { + "express": ">= 4.11" + } + }, "node_modules/extrareqp2": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/extrareqp2/-/extrareqp2-1.0.0.tgz", @@ -1912,6 +1977,15 @@ "is-property": "^1.0.2" } }, + "node_modules/get-caller-file": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz", + "integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==", + "license": "ISC", + "engines": { + "node": "6.* || 8.* || >= 10.*" + } + }, "node_modules/get-intrinsic": { "version": "1.2.7", "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.7.tgz", @@ -2227,6 +2301,53 @@ "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==", "license": "ISC" }, + "node_modules/ioredis": { + "version": "5.10.1", + "resolved": "https://registry.npmjs.org/ioredis/-/ioredis-5.10.1.tgz", + "integrity": "sha512-HuEDBTI70aYdx1v6U97SbNx9F1+svQKBDo30o0b9fw055LMepzpOOd0Ccg9Q6tbqmBSJaMuY0fB7yw9/vjBYCA==", + "license": "MIT", + "dependencies": { + "@ioredis/commands": "1.5.1", + "cluster-key-slot": "^1.1.0", + "debug": "^4.3.4", + "denque": "^2.1.0", + "lodash.defaults": "^4.2.0", + "lodash.isarguments": "^3.1.0", + "redis-errors": "^1.2.0", + "redis-parser": "^3.0.0", + "standard-as-callback": "^2.1.0" + }, + "engines": { + "node": ">=12.22.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/ioredis" + } + }, + "node_modules/ioredis/node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/ioredis/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, "node_modules/ip-address": { "version": "9.0.5", "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-9.0.5.tgz", @@ -2376,6 +2497,55 @@ "license": "ISC", "optional": true }, + "node_modules/jsonwebtoken": { + "version": "9.0.3", + "resolved": "https://registry.npmjs.org/jsonwebtoken/-/jsonwebtoken-9.0.3.tgz", + "integrity": "sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g==", + "license": "MIT", + "dependencies": { + "jws": "^4.0.1", + "lodash.includes": "^4.3.0", + "lodash.isboolean": "^3.0.3", + "lodash.isinteger": "^4.0.4", + "lodash.isnumber": "^3.0.3", + "lodash.isplainobject": "^4.0.6", + "lodash.isstring": "^4.0.1", + "lodash.once": "^4.0.0", + "ms": "^2.1.1", + "semver": "^7.5.4" + }, + "engines": { + "node": ">=12", + "npm": ">=6" + } + }, + "node_modules/jsonwebtoken/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/jwa": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/jwa/-/jwa-2.0.1.tgz", + "integrity": "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==", + "license": "MIT", + "dependencies": { + "buffer-equal-constant-time": "^1.0.1", + "ecdsa-sig-formatter": "1.0.11", + "safe-buffer": "^5.0.1" + } + }, + "node_modules/jws": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/jws/-/jws-4.0.1.tgz", + "integrity": "sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA==", + "license": "MIT", + "dependencies": { + "jwa": "^2.0.1", + "safe-buffer": "^5.0.1" + } + }, "node_modules/lazy": { "version": "1.0.11", "resolved": "https://registry.npmjs.org/lazy/-/lazy-1.0.11.tgz", @@ -2391,6 +2561,60 @@ "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==", "license": "MIT" }, + "node_modules/lodash.defaults": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz", + "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==", + "license": "MIT" + }, + "node_modules/lodash.includes": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz", + "integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==", + "license": "MIT" + }, + "node_modules/lodash.isarguments": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz", + "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==", + "license": "MIT" + }, + "node_modules/lodash.isboolean": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz", + "integrity": "sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg==", + "license": "MIT" + }, + "node_modules/lodash.isinteger": { + "version": "4.0.4", + "resolved": "https://registry.npmjs.org/lodash.isinteger/-/lodash.isinteger-4.0.4.tgz", + "integrity": "sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA==", + "license": "MIT" + }, + "node_modules/lodash.isnumber": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/lodash.isnumber/-/lodash.isnumber-3.0.3.tgz", + "integrity": "sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw==", + "license": "MIT" + }, + "node_modules/lodash.isplainobject": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/lodash.isplainobject/-/lodash.isplainobject-4.0.6.tgz", + "integrity": "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA==", + "license": "MIT" + }, + "node_modules/lodash.isstring": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/lodash.isstring/-/lodash.isstring-4.0.1.tgz", + "integrity": "sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw==", + "license": "MIT" + }, + "node_modules/lodash.once": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/lodash.once/-/lodash.once-4.1.1.tgz", + "integrity": "sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg==", + "license": "MIT" + }, "node_modules/long": { "version": "5.2.3", "resolved": "https://registry.npmjs.org/long/-/long-5.2.3.tgz", @@ -2910,6 +3134,15 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/on-exit-leak-free": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/on-exit-leak-free/-/on-exit-leak-free-2.1.2.tgz", + "integrity": "sha512-0eJJY6hXLGf1udHwfNftBqH+g73EU4B504nZeKpz1sYRKafAghwxEJunB2O7rDZkL4PGfsMVnTXZ2EjibbqcsA==", + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, "node_modules/on-finished": { "version": "2.4.1", "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz", @@ -3156,6 +3389,55 @@ "node": ">=10" } }, + "node_modules/pino": { + "version": "9.14.0", + "resolved": "https://registry.npmjs.org/pino/-/pino-9.14.0.tgz", + "integrity": "sha512-8OEwKp5juEvb/MjpIc4hjqfgCNysrS94RIOMXYvpYCdm/jglrKEiAYmiumbmGhCvs+IcInsphYDFwqrjr7398w==", + "license": "MIT", + "dependencies": { + "@pinojs/redact": "^0.4.0", + "atomic-sleep": "^1.0.0", + "on-exit-leak-free": "^2.1.0", + "pino-abstract-transport": "^2.0.0", + "pino-std-serializers": "^7.0.0", + "process-warning": "^5.0.0", + "quick-format-unescaped": "^4.0.3", + "real-require": "^0.2.0", + "safe-stable-stringify": "^2.3.1", + "sonic-boom": "^4.0.1", + "thread-stream": "^3.0.0" + }, + "bin": { + "pino": "bin.js" + } + }, + "node_modules/pino-abstract-transport": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/pino-abstract-transport/-/pino-abstract-transport-2.0.0.tgz", + "integrity": "sha512-F63x5tizV6WCh4R6RHyi2Ml+M70DNRXt/+HANowMflpgGFMAym/VKm6G7ZOQRjqN7XbGxK1Lg9t6ZrtzOaivMw==", + "license": "MIT", + "dependencies": { + "split2": "^4.0.0" + } + }, + "node_modules/pino-http": { + "version": "10.5.0", + "resolved": "https://registry.npmjs.org/pino-http/-/pino-http-10.5.0.tgz", + "integrity": "sha512-hD91XjgaKkSsdn8P7LaebrNzhGTdB086W3pyPihX0EzGPjq5uBJBXo4N5guqNaK6mUjg9aubMF7wDViYek9dRA==", + "license": "MIT", + "dependencies": { + "get-caller-file": "^2.0.5", + "pino": "^9.0.0", + "pino-std-serializers": "^7.0.0", + "process-warning": "^5.0.0" + } + }, + "node_modules/pino-std-serializers": { + "version": "7.1.0", + "resolved": "https://registry.npmjs.org/pino-std-serializers/-/pino-std-serializers-7.1.0.tgz", + "integrity": "sha512-BndPH67/JxGExRgiX1dX0w1FvZck5Wa4aal9198SrRhZjH3GxKQUKIBnYJTdj2HDN3UQAS06HlfcSbQj2OHmaw==", + "license": "MIT" + }, "node_modules/pm2": { "version": "5.4.3", "resolved": "https://registry.npmjs.org/pm2/-/pm2-5.4.3.tgz", @@ -3426,6 +3708,22 @@ "integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==", "license": "MIT" }, + "node_modules/process-warning": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/process-warning/-/process-warning-5.0.0.tgz", + "integrity": "sha512-a39t9ApHNx2L4+HBnQKqxxHNs1r7KF+Intd8Q/g1bUh6q0WIp9voPXJ/x0j+ZL45KF1pJd9+q2jLIRMfvEshkA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "MIT" + }, "node_modules/promptly": { "version": "2.2.0", "resolved": "https://registry.npmjs.org/promptly/-/promptly-2.2.0.tgz", @@ -3518,6 +3816,12 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/quick-format-unescaped": { + "version": "4.0.4", + "resolved": "https://registry.npmjs.org/quick-format-unescaped/-/quick-format-unescaped-4.0.4.tgz", + "integrity": "sha512-tYC1Q1hgyRuHgloV/YXs2w15unPVh8qfu/qCTfhTYamaw7fyhumKa2yGpdSo87vY32rIclj+4fWYQXUMs9EHvg==", + "license": "MIT" + }, "node_modules/range-parser": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz", @@ -3587,6 +3891,36 @@ "node": ">=8.10.0" } }, + "node_modules/real-require": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/real-require/-/real-require-0.2.0.tgz", + "integrity": "sha512-57frrGM/OCTLqLOAh0mhVA9VBMHd+9U7Zb2THMGdBUoZVOtGbJzjxsYGDJ3A9AYYCP4hn6y1TVbaOfzWtm5GFg==", + "license": "MIT", + "engines": { + "node": ">= 12.13.0" + } + }, + "node_modules/redis-errors": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz", + "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==", + "license": "MIT", + "engines": { + "node": ">=4" + } + }, + "node_modules/redis-parser": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz", + "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==", + "license": "MIT", + "dependencies": { + "redis-errors": "^1.0.0" + }, + "engines": { + "node": ">=4" + } + }, "node_modules/require-in-the-middle": { "version": "5.2.0", "resolved": "https://registry.npmjs.org/require-in-the-middle/-/require-in-the-middle-5.2.0.tgz", @@ -3700,6 +4034,15 @@ ], "license": "MIT" }, + "node_modules/safe-stable-stringify": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/safe-stable-stringify/-/safe-stable-stringify-2.5.0.tgz", + "integrity": "sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA==", + "license": "MIT", + "engines": { + "node": ">=10" + } + }, "node_modules/safer-buffer": { "version": "2.1.2", "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", @@ -4001,6 +4344,15 @@ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", "license": "MIT" }, + "node_modules/sonic-boom": { + "version": "4.2.1", + "resolved": "https://registry.npmjs.org/sonic-boom/-/sonic-boom-4.2.1.tgz", + "integrity": "sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q==", + "license": "MIT", + "dependencies": { + "atomic-sleep": "^1.0.0" + } + }, "node_modules/source-map": { "version": "0.6.1", "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz", @@ -4061,6 +4413,12 @@ "nan": "^2.20.0" } }, + "node_modules/standard-as-callback": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz", + "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==", + "license": "MIT" + }, "node_modules/statuses": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz", @@ -4187,6 +4545,15 @@ "node": ">=10" } }, + "node_modules/thread-stream": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/thread-stream/-/thread-stream-3.1.0.tgz", + "integrity": "sha512-OqyPZ9u96VohAyMfJykzmivOrY2wfMSf3C5TtFJVgN+Hm6aj+voFhlK+kZEIv2FBh1X6Xp3DlnCOfEQ3B2J86A==", + "license": "MIT", + "dependencies": { + "real-require": "^0.2.0" + } + }, "node_modules/to-regex-range": { "version": "5.0.1", "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", diff --git a/inventory-server/package.json b/inventory-server/package.json index d86785e..efb66e8 100644 --- a/inventory-server/package.json +++ b/inventory-server/package.json @@ -2,6 +2,7 @@ "name": "inventory-server", "version": "1.0.0", "description": "Backend server for inventory management system", + "type": "module", "main": "src/server.js", "scripts": { "start": "node src/server.js", @@ -27,10 +28,15 @@ "diff": "^7.0.0", "dotenv": "^16.4.7", "express": "^4.18.2", + "express-rate-limit": "^7.4.0", + "ioredis": "^5.10.1", + "jsonwebtoken": "^9.0.2", "multer": "^1.4.5-lts.1", "mysql2": "^3.12.0", "openai": "^6.0.0", "pg": "^8.14.1", + "pino": "^9.5.0", + "pino-http": "^10.3.0", "pm2": "^5.3.0", "sharp": "^0.33.5", "ssh2": "^1.16.0", diff --git a/inventory-server/scripts/package.json b/inventory-server/scripts/package.json new file mode 100644 index 0000000..ae245a7 --- /dev/null +++ b/inventory-server/scripts/package.json @@ -0,0 +1,6 @@ +{ + "name": "inventory-server-scripts", + "private": true, + "type": "commonjs", + "description": "One-shot maintenance scripts (imports, metrics, forecasting orchestration). Kept CommonJS so the existing `require()` graph still works after the parent server moved to ESM. Convert individual scripts when touching them." +} diff --git a/inventory-server/shared/db/pg.js b/inventory-server/shared/db/pg.js index 50c2349..f3b9a05 100644 --- a/inventory-server/shared/db/pg.js +++ b/inventory-server/shared/db/pg.js @@ -10,7 +10,7 @@ export function createPool(envPrefix = 'DB', overrides = {}) { user: overrides.user ?? get('USER'), password: overrides.password ?? get('PASSWORD'), database: overrides.database ?? get('NAME'), - port: overrides.port ?? Number(get('PORT')) || 5432, + port: overrides.port ?? (Number(get('PORT')) || 5432), ssl: (overrides.ssl ?? get('SSL')) === 'true' ? { rejectUnauthorized: false } : false, max: overrides.max ?? 20, idleTimeoutMillis: overrides.idleTimeoutMillis ?? 30_000, diff --git a/inventory-server/shared/db/redis.js b/inventory-server/shared/db/redis.js index 0d26d8c..937c217 100644 --- a/inventory-server/shared/db/redis.js +++ b/inventory-server/shared/db/redis.js @@ -19,7 +19,7 @@ export function createRedis(overrides = {}) { return new Redis({ host: overrides.host ?? process.env.REDIS_HOST ?? 'localhost', - port: overrides.port ?? Number(process.env.REDIS_PORT) || 6379, + port: overrides.port ?? (Number(process.env.REDIS_PORT) || 6379), username: overrides.username ?? process.env.REDIS_USERNAME, password: overrides.password ?? process.env.REDIS_PASSWORD, ...options, diff --git a/inventory-server/src/middleware/cors.js b/inventory-server/src/middleware/cors.js index 2a3ce24..3114505 100644 --- a/inventory-server/src/middleware/cors.js +++ b/inventory-server/src/middleware/cors.js @@ -1,41 +1,14 @@ -const cors = require('cors'); +import cors from 'cors'; +import { corsOptions } from '../../shared/cors/policy.js'; -// Single CORS middleware for all endpoints -const corsMiddleware = cors({ - origin: [ - 'https://inventory.kent.pw', - 'http://localhost:5175', - 'https://acot.site', - 'https://tools.acherryontop.com', - /^http:\/\/192\.168\.\d+\.\d+(:\d+)?$/, - /^http:\/\/10\.\d+\.\d+\.\d+(:\d+)?$/ - ], - methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'], - allowedHeaders: ['Content-Type', 'Authorization'], - exposedHeaders: ['Content-Type'], - credentials: true -}); +export const corsMiddleware = cors(corsOptions); -// Error handler for CORS -const corsErrorHandler = (err, req, res, next) => { - if (err.message === 'CORS not allowed') { - console.error('CORS Error:', { - origin: req.get('Origin'), - method: req.method, - path: req.path, - headers: req.headers - }); - res.status(403).json({ +export function corsErrorHandler(err, req, res, next) { + if (err && err.message === 'CORS not allowed') { + return res.status(403).json({ error: 'CORS not allowed', origin: req.get('Origin'), - message: 'Origin not in allowed list: https://inventory.kent.pw, https://acot.site, https://tools.acherryontop.com, localhost:5175, 192.168.x.x, or 10.x.x.x' }); - } else { - next(err); } -}; - -module.exports = { - corsMiddleware, - corsErrorHandler -}; \ No newline at end of file + next(err); +} diff --git a/inventory-server/src/routes/ai-prompts.js b/inventory-server/src/routes/ai-prompts.js index 1ec6a1c..105ba1a 100644 --- a/inventory-server/src/routes/ai-prompts.js +++ b/inventory-server/src/routes/ai-prompts.js @@ -1,6 +1,14 @@ -const express = require('express'); +import express from 'express'; +import { requirePermission } from '../../shared/auth/middleware.js'; + const router = express.Router(); +// Phase 6.2: prompt edits require ai_admin. Reads remain authenticated-only. +router.use((req, res, next) => { + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('ai_admin')(req, res, next); +}); + // Get all AI prompts router.get('/', async (req, res) => { try { @@ -307,4 +315,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; \ No newline at end of file +export default router; diff --git a/inventory-server/src/routes/ai-validation.js b/inventory-server/src/routes/ai-validation.js index 54fd8cb..21c783f 100644 --- a/inventory-server/src/routes/ai-validation.js +++ b/inventory-server/src/routes/ai-validation.js @@ -1,12 +1,25 @@ -const express = require("express"); +import express from "express"; +import OpenAI from "openai"; +import { promises as fs } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import dotenv from "dotenv"; +import mysql from 'mysql2/promise'; +import { Client } from 'ssh2'; +import { getDbConnection, closeAllConnections } from '../utils/dbConnection.js'; // Import the optimized connection function +import { requirePermission } from '../../shared/auth/middleware.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); + const router = express.Router(); -const OpenAI = require("openai"); -const fs = require("fs").promises; -const path = require("path"); -const dotenv = require("dotenv"); -const mysql = require('mysql2/promise'); -const { Client } = require('ssh2'); -const { getDbConnection, closeAllConnections } = require('../utils/dbConnection'); // Import the optimized connection function + +// Phase 6.2: AI validation runs (which trigger OpenAI calls + DB writes) require ai_admin. +// Status/health reads stay authenticated-only. +router.use((req, res, next) => { + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('ai_admin')(req, res, next); +}); // Ensure environment variables are loaded dotenv.config({ path: path.join(__dirname, "../../.env") }); @@ -1401,7 +1414,7 @@ router.get("/test-taxonomy", async (req, res) => { } }); -module.exports = router; +export default router; function extractResponseText(response) { if (!response) return ""; diff --git a/inventory-server/src/routes/ai.js b/inventory-server/src/routes/ai.js index 04771d4..074e645 100644 --- a/inventory-server/src/routes/ai.js +++ b/inventory-server/src/routes/ai.js @@ -5,10 +5,11 @@ * Provides embedding generation and similarity-based suggestions. */ -const express = require('express'); +import express from 'express'; +import aiService from '../services/ai/index.js'; +import { getDbConnection, closeAllConnections } from '../utils/dbConnection.js'; + const router = express.Router(); -const aiService = require('../services/ai'); -const { getDbConnection, closeAllConnections } = require('../utils/dbConnection'); // Track initialization state let initializationPromise = null; @@ -440,11 +441,10 @@ router.post('/validate/sanity-check', async (req, res) => { * Call once from server startup so the taxonomy embeddings are ready before * the first user request hits a taxonomy dropdown. */ -function initInBackground() { +export function initInBackground() { ensureInitialized().catch(err => console.error('[AI Routes] Background initialization failed:', err) ); } -module.exports = router; -module.exports.initInBackground = initInBackground; +export default router; diff --git a/inventory-server/src/routes/analytics.js b/inventory-server/src/routes/analytics.js index f3e09d7..e814fba 100644 --- a/inventory-server/src/routes/analytics.js +++ b/inventory-server/src/routes/analytics.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Forecasting: summarize sales for products received in a period by brand @@ -980,4 +980,4 @@ router.get('/seasonal', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/brandsAggregate.js b/inventory-server/src/routes/brandsAggregate.js index 587e730..8a08d6c 100644 --- a/inventory-server/src/routes/brandsAggregate.js +++ b/inventory-server/src/routes/brandsAggregate.js @@ -1,6 +1,6 @@ -const express = require('express'); +import express from 'express'; +import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed const router = express.Router(); -const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed // --- Configuration & Helpers --- const DEFAULT_PAGE_LIMIT = 50; @@ -281,4 +281,4 @@ router.get('/', async (req, res) => { // GET /brands-aggregate/:name (Get single brand metric) // Implement if needed, remember to URL-decode the name parameter -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/categoriesAggregate.js b/inventory-server/src/routes/categoriesAggregate.js index 7a9f0da..dca6f76 100644 --- a/inventory-server/src/routes/categoriesAggregate.js +++ b/inventory-server/src/routes/categoriesAggregate.js @@ -1,6 +1,6 @@ -const express = require('express'); +import express from 'express'; +import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed const router = express.Router(); -const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed // --- Configuration & Helpers --- const DEFAULT_PAGE_LIMIT = 50; @@ -360,4 +360,4 @@ router.get('/', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/config.js b/inventory-server/src/routes/config.js index 5ccfba6..6933e14 100644 --- a/inventory-server/src/routes/config.js +++ b/inventory-server/src/routes/config.js @@ -1,10 +1,13 @@ -const express = require('express'); +import express from 'express'; +import { requireAdmin } from '../../shared/auth/middleware.js'; + const router = express.Router(); -// Debug middleware +// Phase 6.2: global settings are admin-only on write. Reads pass through to any +// authenticated user (the server-level authenticate() already gates that). router.use((req, res, next) => { - console.log(`[Config Route] ${req.method} ${req.path}`); - next(); + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requireAdmin(req, res, next); }); // ===== GLOBAL SETTINGS ===== @@ -322,4 +325,4 @@ router.post('/vendors/:vendor/reset', async (req, res) => { }); // Export the router -module.exports = router; \ No newline at end of file +export default router; diff --git a/inventory-server/src/routes/dashboard.js b/inventory-server/src/routes/dashboard.js index ac1e01d..5594a3b 100644 --- a/inventory-server/src/routes/dashboard.js +++ b/inventory-server/src/routes/dashboard.js @@ -1,6 +1,6 @@ -const express = require('express'); +import express from 'express'; +import db from '../utils/db.js'; const router = express.Router(); -const db = require('../utils/db'); // Helper function to execute queries using the connection pool async function executeQuery(sql, params = []) { @@ -1288,4 +1288,4 @@ router.get('/replenish/products', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/data-management.js b/inventory-server/src/routes/data-management.js index 984710d..d9021cf 100644 --- a/inventory-server/src/routes/data-management.js +++ b/inventory-server/src/routes/data-management.js @@ -1,13 +1,20 @@ -const express = require('express'); -const router = express.Router(); -const { spawn } = require('child_process'); -const path = require('path'); -const db = require('../utils/db'); +import express from 'express'; +import { spawn } from 'node:child_process'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import db from '../utils/db.js'; +import { requirePermission } from '../../shared/auth/middleware.js'; -// Debug middleware MUST be first +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); + +const router = express.Router(); + +// Phase 6.2: CSV / full-update / full-reset / metrics-recalc are destructive. +// Writes require data_management; reads (status polls, SSE streams) pass through. router.use((req, res, next) => { - console.log(`[CSV Route Debug] ${req.method} ${req.path}`); - next(); + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('data_management')(req, res, next); }); // Store active processes and their progress @@ -437,4 +444,4 @@ router.get('/status/table-counts', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/hts-lookup.js b/inventory-server/src/routes/hts-lookup.js index 9fa8f60..8ce08e6 100644 --- a/inventory-server/src/routes/hts-lookup.js +++ b/inventory-server/src/routes/hts-lookup.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // GET /api/hts-lookup?search=term @@ -167,4 +167,4 @@ router.get('/', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/import-audit-log.js b/inventory-server/src/routes/import-audit-log.js index bd91c4a..78ee4de 100644 --- a/inventory-server/src/routes/import-audit-log.js +++ b/inventory-server/src/routes/import-audit-log.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Create a new audit log entry @@ -190,4 +190,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/import-sessions.js b/inventory-server/src/routes/import-sessions.js index 913cf68..0c30a22 100644 --- a/inventory-server/src/routes/import-sessions.js +++ b/inventory-server/src/routes/import-sessions.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Get all import sessions for a user (named + unnamed) @@ -334,4 +334,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/import.js b/inventory-server/src/routes/import.js index 1a7fee2..cb4fb31 100644 --- a/inventory-server/src/routes/import.js +++ b/inventory-server/src/routes/import.js @@ -1,14 +1,23 @@ -const express = require('express'); +import express from 'express'; +import { Client } from 'ssh2'; +import mysql from 'mysql2/promise'; +import multer from 'multer'; +import path from 'node:path'; +import fs from 'node:fs'; +import sharp from 'sharp'; +import axios from 'axios'; +import net from 'node:net'; +import { requirePermission } from '../../shared/auth/middleware.js'; + const router = express.Router(); -const { Client } = require('ssh2'); -const mysql = require('mysql2/promise'); -const multer = require('multer'); -const path = require('path'); -const fs = require('fs'); const fsp = fs.promises; -const sharp = require('sharp'); -const axios = require('axios'); -const net = require('net'); + +// Phase 6.2: imports, uploads, generate-upc and deletions all require product_import. +// Reads (list-uploads, status checks) remain authenticated-only. +router.use((req, res, next) => { + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('product_import')(req, res, next); +}); // Create uploads directory if it doesn't exist const uploadsDir = path.join('/var/www/inventory/uploads/products'); @@ -515,21 +524,30 @@ const storage = multer.diskStorage({ const MAX_UPLOAD_BYTES = 25 * 1024 * 1024; +// Phase 6.7: exact-match MIME + extension allowlist. Substring-based regex +// matchers (the previous /jpeg|png|.../ approach) accepted MIMEs like +// `application/jpeg-payload` because of partial matches; this rejects them. +const ALLOWED_MIME_TYPES = new Set([ + 'image/jpeg', + 'image/png', + 'image/gif', + 'image/webp', + 'image/tiff', +]); +const ALLOWED_EXTENSIONS = new Set(['.jpg', '.jpeg', '.png', '.gif', '.webp', '.tif', '.tiff']); + const upload = multer({ storage: storage, limits: { fileSize: MAX_UPLOAD_BYTES, + files: 1, }, fileFilter: function (req, file, cb) { - // Accept only image files - const filetypes = /jpeg|jpg|png|gif|webp|tiff?/; - const mimetype = filetypes.test(file.mimetype); - const extname = filetypes.test(path.extname(file.originalname).toLowerCase()); - - if (mimetype && extname) { - return cb(null, true); + const ext = path.extname(file.originalname).toLowerCase(); + if (!ALLOWED_MIME_TYPES.has(file.mimetype) || !ALLOWED_EXTENSIONS.has(ext)) { + return cb(new Error('Only image files are allowed (jpg, png, gif, webp, tiff)')); } - cb(new Error('Only image files are allowed')); + cb(null, true); } }); @@ -647,7 +665,7 @@ async function setupSshTunnel() { port: process.env.PROD_SSH_PORT || 22, username: process.env.PROD_SSH_USER, privateKey: process.env.PROD_SSH_KEY_PATH - ? require('fs').readFileSync(process.env.PROD_SSH_KEY_PATH) + ? fs.readFileSync(process.env.PROD_SSH_KEY_PATH) : undefined, compress: true }; @@ -2872,4 +2890,4 @@ router.get('/query-products', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/linesAggregate.js b/inventory-server/src/routes/linesAggregate.js index 85f9a95..f9cf6b8 100644 --- a/inventory-server/src/routes/linesAggregate.js +++ b/inventory-server/src/routes/linesAggregate.js @@ -1,6 +1,6 @@ -const express = require('express'); +import express from 'express'; +import { parseValue } from '../utils/apiHelpers.js'; const router = express.Router(); -const { parseValue } = require('../utils/apiHelpers'); // --- Configuration & Helpers --- const DEFAULT_PAGE_LIMIT = 50; @@ -378,4 +378,4 @@ router.get('/:brand/:line/products', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/metrics.js b/inventory-server/src/routes/metrics.js index 6fe13ba..a5bb3f7 100644 --- a/inventory-server/src/routes/metrics.js +++ b/inventory-server/src/routes/metrics.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // --- Configuration & Helpers --- @@ -645,4 +645,4 @@ function parseValue(value, type) { } } -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/newsletter.js b/inventory-server/src/routes/newsletter.js index faea0b9..3fffcea 100644 --- a/inventory-server/src/routes/newsletter.js +++ b/inventory-server/src/routes/newsletter.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Shared CTE fragment for the reference date. @@ -721,4 +721,4 @@ router.get('/campaigns/links', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/orders.js b/inventory-server/src/routes/orders.js index b109111..1c7c564 100644 --- a/inventory-server/src/routes/orders.js +++ b/inventory-server/src/routes/orders.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Get all orders with pagination, filtering, and sorting @@ -258,4 +258,4 @@ router.get('/:orderNumber', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; diff --git a/inventory-server/src/routes/product-editor-audit-log.js b/inventory-server/src/routes/product-editor-audit-log.js index 3fd188f..ceee982 100644 --- a/inventory-server/src/routes/product-editor-audit-log.js +++ b/inventory-server/src/routes/product-editor-audit-log.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Create a new audit log entry @@ -192,4 +192,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/products.js b/inventory-server/src/routes/products.js index d9b1b67..a8cd1eb 100644 --- a/inventory-server/src/routes/products.js +++ b/inventory-server/src/routes/products.js @@ -1,27 +1,7 @@ -const express = require('express'); +import express from 'express'; +import { PurchaseOrderStatus, ReceivingStatus } from '../types/status-codes.js'; + const router = express.Router(); -const multer = require('multer'); -const path = require('path'); -const fs = require('fs'); -const { importProductsFromCSV } = require('../utils/csvImporter'); -const { PurchaseOrderStatus, ReceivingStatus } = require('../types/status-codes'); - -// Configure multer for file uploads without silent fallbacks -const configuredUploadsDir = process.env.UPLOADS_DIR; -const uploadsDir = configuredUploadsDir - ? (path.isAbsolute(configuredUploadsDir) - ? configuredUploadsDir - : path.resolve(__dirname, '../../', configuredUploadsDir)) - : path.resolve(__dirname, '../../uploads'); - -try { - fs.mkdirSync(uploadsDir, { recursive: true }); -} catch (error) { - console.error(`Failed to initialize uploads directory at ${uploadsDir}:`, error); - throw error; -} - -const upload = multer({ dest: uploadsDir }); // Get unique brands router.get('/brands', async (req, res) => { @@ -983,4 +963,4 @@ router.get('/:id/forecast', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/purchase-orders.js b/inventory-server/src/routes/purchase-orders.js index d51b30c..9d8548a 100644 --- a/inventory-server/src/routes/purchase-orders.js +++ b/inventory-server/src/routes/purchase-orders.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Status code constants @@ -1277,4 +1277,4 @@ router.get('/pipeline', async (req, res) => { } }); -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/routes/repeat-orders.js b/inventory-server/src/routes/repeat-orders.js index b06a2f0..a70cfae 100644 --- a/inventory-server/src/routes/repeat-orders.js +++ b/inventory-server/src/routes/repeat-orders.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); // Stale PO statuses to exclude from our counting — these are the "still active" @@ -388,4 +388,4 @@ router.get('/:pid/history', async (req, res) => { } }); -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/reusable-images.js b/inventory-server/src/routes/reusable-images.js index 3889d0a..e129cbf 100644 --- a/inventory-server/src/routes/reusable-images.js +++ b/inventory-server/src/routes/reusable-images.js @@ -1,8 +1,16 @@ -const express = require('express'); +import express from 'express'; +import multer from 'multer'; +import path from 'node:path'; +import fs from 'node:fs'; +import { requirePermission } from '../../shared/auth/middleware.js'; + const router = express.Router(); -const multer = require('multer'); -const path = require('path'); -const fs = require('fs'); + +// Phase 6.2: uploads + deletions of reusable images require image_admin. +router.use((req, res, next) => { + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('image_admin')(req, res, next); +}); // Create reusable uploads directory if it doesn't exist const uploadsDir = path.join('/var/www/inventory/uploads/reusable'); @@ -38,21 +46,24 @@ const storage = multer.diskStorage({ } }); -const upload = multer({ +// Phase 6.7: exact-match MIME + extension allowlist. +const ALLOWED_MIME_TYPES = new Set([ + 'image/jpeg', 'image/png', 'image/gif', 'image/webp', +]); +const ALLOWED_EXTENSIONS = new Set(['.jpg', '.jpeg', '.png', '.gif', '.webp']); + +const upload = multer({ storage: storage, limits: { - fileSize: 5 * 1024 * 1024, // 5MB max file size + fileSize: 5 * 1024 * 1024, + files: 1, }, fileFilter: function (req, file, cb) { - // Accept only image files - const filetypes = /jpeg|jpg|png|gif|webp/; - const mimetype = filetypes.test(file.mimetype); - const extname = filetypes.test(path.extname(file.originalname).toLowerCase()); - - if (mimetype && extname) { - return cb(null, true); + const ext = path.extname(file.originalname).toLowerCase(); + if (!ALLOWED_MIME_TYPES.has(file.mimetype) || !ALLOWED_EXTENSIONS.has(ext)) { + return cb(new Error('Only image files are allowed (jpg, png, gif, webp)')); } - cb(new Error('Only image files are allowed')); + cb(null, true); } }); @@ -393,4 +404,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; \ No newline at end of file +export default router; diff --git a/inventory-server/src/routes/spec-lookup.js b/inventory-server/src/routes/spec-lookup.js index b6caf95..65cd0ee 100644 --- a/inventory-server/src/routes/spec-lookup.js +++ b/inventory-server/src/routes/spec-lookup.js @@ -1,4 +1,4 @@ -const express = require('express'); +import express from 'express'; const router = express.Router(); const MAX_MATCHES = 500; @@ -267,4 +267,4 @@ function descriptionAggregate(products) { return { duplicates, samples }; } -module.exports = router; +export default router; diff --git a/inventory-server/src/routes/templates.js b/inventory-server/src/routes/templates.js index d38b9ae..7cfff86 100644 --- a/inventory-server/src/routes/templates.js +++ b/inventory-server/src/routes/templates.js @@ -1,12 +1,23 @@ -const express = require('express'); -const { getPool } = require('../utils/db'); -const dotenv = require('dotenv'); -const path = require('path'); +import express from 'express'; +import dotenv from 'dotenv'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { getPool } from '../utils/db.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); dotenv.config({ path: path.join(__dirname, "../../.env") }); const router = express.Router(); +// Phase 6.2: template edits require templates_write. Reads pass through. +import { requirePermission } from '../../shared/auth/middleware.js'; +router.use((req, res, next) => { + if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next(); + return requirePermission('templates_write')(req, res, next); +}); + // Get all templates router.get('/', async (req, res) => { try { @@ -278,4 +289,4 @@ router.use((err, req, res, next) => { }); }); -module.exports = router; \ No newline at end of file +export default router; diff --git a/inventory-server/src/routes/vendorsAggregate.js b/inventory-server/src/routes/vendorsAggregate.js index 84e0544..6eace11 100644 --- a/inventory-server/src/routes/vendorsAggregate.js +++ b/inventory-server/src/routes/vendorsAggregate.js @@ -1,6 +1,6 @@ -const express = require('express'); +import express from 'express'; +import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed const router = express.Router(); -const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed // --- Configuration & Helpers --- const DEFAULT_PAGE_LIMIT = 50; @@ -320,4 +320,4 @@ router.get('/', async (req, res) => { // GET /vendors-aggregate/:name (Get single vendor metric) // Implement if needed, remember to URL-decode the name parameter -module.exports = router; \ No newline at end of file +export default router; \ No newline at end of file diff --git a/inventory-server/src/server.js b/inventory-server/src/server.js index d023704..a7cf6f9 100644 --- a/inventory-server/src/server.js +++ b/inventory-server/src/server.js @@ -1,59 +1,64 @@ -const express = require('express'); -const cors = require('cors'); -const { spawn } = require('child_process'); -const path = require('path'); -const fs = require('fs'); -const { corsMiddleware, corsErrorHandler } = require('./middleware/cors'); -const { initPool } = require('./utils/db'); -const productsRouter = require('./routes/products'); -const dashboardRouter = require('./routes/dashboard'); -const ordersRouter = require('./routes/orders'); -const csvRouter = require('./routes/data-management'); -const analyticsRouter = require('./routes/analytics'); -const purchaseOrdersRouter = require('./routes/purchase-orders'); -const configRouter = require('./routes/config'); -const metricsRouter = require('./routes/metrics'); -const importRouter = require('./routes/import'); -const aiValidationRouter = require('./routes/ai-validation'); -const aiRouter = require('./routes/ai'); -const templatesRouter = require('./routes/templates'); -const aiPromptsRouter = require('./routes/ai-prompts'); -const reusableImagesRouter = require('./routes/reusable-images'); -const categoriesAggregateRouter = require('./routes/categoriesAggregate'); -const vendorsAggregateRouter = require('./routes/vendorsAggregate'); -const brandsAggregateRouter = require('./routes/brandsAggregate'); -const htsLookupRouter = require('./routes/hts-lookup'); -const specLookupRouter = require('./routes/spec-lookup'); -const importSessionsRouter = require('./routes/import-sessions'); -const importAuditLogRouter = require('./routes/import-audit-log'); -const productEditorAuditLogRouter = require('./routes/product-editor-audit-log'); -const newsletterRouter = require('./routes/newsletter'); -const linesAggregateRouter = require('./routes/linesAggregate'); -const repeatOrdersRouter = require('./routes/repeat-orders'); +import { config as loadEnv } from 'dotenv'; +import express from 'express'; +import path from 'node:path'; +import fs from 'node:fs'; +import { fileURLToPath } from 'node:url'; + +import { corsMiddleware, corsErrorHandler } from './middleware/cors.js'; +import { initPool } from './utils/db.js'; + +import { authenticate } from '../shared/auth/middleware.js'; +import { requestLog } from '../shared/logging/request-log.js'; +import { logger } from '../shared/logging/logger.js'; +import { errorHandler } from '../shared/errors/handler.js'; + +import productsRouter from './routes/products.js'; +import dashboardRouter from './routes/dashboard.js'; +import ordersRouter from './routes/orders.js'; +import csvRouter from './routes/data-management.js'; +import analyticsRouter from './routes/analytics.js'; +import purchaseOrdersRouter from './routes/purchase-orders.js'; +import configRouter from './routes/config.js'; +import metricsRouter from './routes/metrics.js'; +import importRouter from './routes/import.js'; +import aiValidationRouter from './routes/ai-validation.js'; +import aiRouter, { initInBackground as initAiInBackground } from './routes/ai.js'; +import templatesRouter from './routes/templates.js'; +import aiPromptsRouter from './routes/ai-prompts.js'; +import reusableImagesRouter from './routes/reusable-images.js'; +import categoriesAggregateRouter from './routes/categoriesAggregate.js'; +import vendorsAggregateRouter from './routes/vendorsAggregate.js'; +import brandsAggregateRouter from './routes/brandsAggregate.js'; +import htsLookupRouter from './routes/hts-lookup.js'; +import specLookupRouter from './routes/spec-lookup.js'; +import importSessionsRouter from './routes/import-sessions.js'; +import importAuditLogRouter from './routes/import-audit-log.js'; +import productEditorAuditLogRouter from './routes/product-editor-audit-log.js'; +import newsletterRouter from './routes/newsletter.js'; +import linesAggregateRouter from './routes/linesAggregate.js'; +import repeatOrdersRouter from './routes/repeat-orders.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); -// Get the absolute path to the .env file const envPath = '/var/www/inventory/.env'; -console.log('Looking for .env file at:', envPath); -console.log('.env file exists:', fs.existsSync(envPath)); +loadEnv({ path: envPath }); +logger.info({ + envPath, + envExists: fs.existsSync(envPath), + NODE_ENV: process.env.NODE_ENV || 'not set', + PORT: process.env.PORT || 'not set', + DB_HOST: process.env.DB_HOST || 'not set', + DB_NAME: process.env.DB_NAME || 'not set', + DB_PASSWORD: process.env.DB_PASSWORD ? '[set]' : 'not set', + DB_SSL: process.env.DB_SSL || 'not set', +}, 'inventory-server starting'); -try { - require('dotenv').config({ path: envPath }); - console.log('.env file loaded successfully'); - console.log('Environment check:', { - NODE_ENV: process.env.NODE_ENV || 'not set', - PORT: process.env.PORT || 'not set', - DB_HOST: process.env.DB_HOST || 'not set', - DB_USER: process.env.DB_USER || 'not set', - DB_NAME: process.env.DB_NAME || 'not set', - DB_PASSWORD: process.env.DB_PASSWORD ? '[password set]' : 'not set', - DB_PORT: process.env.DB_PORT || 'not set', - DB_SSL: process.env.DB_SSL || 'not set' - }); -} catch (error) { - console.error('Error loading .env file:', error); +if (!process.env.JWT_SECRET) { + logger.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)'); + process.exit(1); } -// Resolve important directories relative to the project root const serverRoot = path.resolve(__dirname, '..'); const configuredUploadsDir = process.env.UPLOADS_DIR; const uploadsDir = configuredUploadsDir @@ -62,12 +67,10 @@ const uploadsDir = configuredUploadsDir : path.resolve(serverRoot, configuredUploadsDir)) : path.resolve(serverRoot, 'uploads'); -// Persist the resolved uploads directory so downstream modules share the same path process.env.UPLOADS_DIR = uploadsDir; const requiredDirs = [path.resolve(serverRoot, 'logs'), uploadsDir]; - -requiredDirs.forEach(dir => { +requiredDirs.forEach((dir) => { if (!fs.existsSync(dir)) { fs.mkdirSync(dir, { recursive: true }); } @@ -75,28 +78,18 @@ requiredDirs.forEach(dir => { const app = express(); -// Debug middleware to log request details -app.use((req, res, next) => { - console.log('Request details:', { - method: req.method, - url: req.url, - origin: req.get('Origin'), - headers: req.headers - }); - next(); -}); +// Phase 6.5/6.9: structured access log (replaces the previous header-dumping debug +// middleware that wrote raw Authorization values to stdout). Pino redaction strips +// `authorization` and `cookie` automatically — see shared/logging/logger.js. +app.use(requestLog()); -// Apply CORS middleware first, before any other middleware app.use(corsMiddleware); -// Body parser middleware app.use(express.json({ limit: '10mb' })); app.use(express.urlencoded({ extended: true, limit: '10mb' })); -// Initialize database pool and start server async function startServer() { try { - // Initialize database pool const pool = await initPool({ host: process.env.DB_HOST, user: process.env.DB_USER, @@ -104,17 +97,18 @@ async function startServer() { database: process.env.DB_NAME, port: process.env.DB_PORT || 5432, max: process.env.NODE_ENV === 'production' ? 20 : 10, - idleTimeoutMillis: 30000, - connectionTimeoutMillis: 2000, - ssl: process.env.DB_SSL === 'true' ? { - rejectUnauthorized: false - } : false + idleTimeoutMillis: 30_000, + connectionTimeoutMillis: 2_000, + ssl: process.env.DB_SSL === 'true' ? { rejectUnauthorized: false } : false, }); - // Make pool available to routes app.locals.pool = pool; - // Set up routes after pool is initialized + // Phase 6.1/6.2: every /api request requires a valid JWT. Defense in depth — Caddy + // forward_auth (when enabled) is the first reject; this is the second. Frontend + // service files MUST include `Authorization: Bearer ` on every fetch. + app.use('/api', authenticate({ pool, secret: process.env.JWT_SECRET })); + app.use('/api/products', productsRouter); app.use('/api/dashboard', dashboardRouter); app.use('/api/orders', ordersRouter); @@ -123,10 +117,8 @@ async function startServer() { app.use('/api/purchase-orders', purchaseOrdersRouter); app.use('/api/config', configRouter); app.use('/api/metrics', metricsRouter); - // Use only the aggregate routes for vendors and categories - app.use('/api/vendors', vendorsAggregateRouter); + app.use('/api/vendors', vendorsAggregateRouter); app.use('/api/categories', categoriesAggregateRouter); - // Keep the aggregate-specific endpoints for backward compatibility app.use('/api/categories-aggregate', categoriesAggregateRouter); app.use('/api/vendors-aggregate', vendorsAggregateRouter); app.use('/api/brands-aggregate', brandsAggregateRouter); @@ -145,101 +137,35 @@ async function startServer() { app.use('/api/lines-aggregate', linesAggregateRouter); app.use('/api/repeat-orders', repeatOrdersRouter); - // Basic health check route app.get('/health', (req, res) => { - res.json({ + res.json({ status: 'ok', timestamp: new Date().toISOString(), - environment: process.env.NODE_ENV + environment: process.env.NODE_ENV, }); }); - // CORS error handler - must be before other error handlers app.use(corsErrorHandler); - - // Error handling middleware - MUST be after routes and CORS error handler - app.use((err, req, res, next) => { - console.error(`[${new Date().toISOString()}] Error:`, err); - - // Send detailed error in development, generic in production - const error = process.env.NODE_ENV === 'production' - ? 'An internal server error occurred' - : err.message || err; - - res.status(err.status || 500).json({ error }); - }); + app.use(errorHandler); const PORT = process.env.PORT || 3000; app.listen(PORT, () => { - console.log(`[Server] Running in ${process.env.NODE_ENV || 'development'} mode on port ${PORT}`); - // Pre-warm AI service so taxonomy embeddings are ready before first user request - aiRouter.initInBackground(); + logger.info({ port: PORT, mode: process.env.NODE_ENV || 'development' }, 'inventory-server listening'); + initAiInBackground(); }); } catch (error) { - console.error('Failed to start server:', error); + logger.error({ err: error }, 'Failed to start server'); process.exit(1); } } -// Handle uncaught exceptions process.on('uncaughtException', (err) => { - console.error(`[${new Date().toISOString()}] Uncaught Exception:`, err); + logger.error({ err }, 'Uncaught Exception'); process.exit(1); }); process.on('unhandledRejection', (reason, promise) => { - console.error(`[${new Date().toISOString()}] Unhandled Rejection at:`, promise, 'reason:', reason); + logger.error({ reason, promise }, 'Unhandled Rejection'); }); -// Initialize client sets for SSE -const importClients = new Set(); -const updateClients = new Set(); -const resetClients = new Set(); -const resetMetricsClients = new Set(); - -// Helper function to send progress to SSE clients -const sendProgressToClients = (clients, data) => { - clients.forEach(client => { - try { - client.write(`data: ${JSON.stringify(data)}\n\n`); - } catch (error) { - console.error('Error sending SSE update:', error); - } - }); -}; - -// Setup SSE connection -const setupSSE = (req, res) => { - const { type } = req.params; - - // Set headers for SSE - res.writeHead(200, { - 'Content-Type': 'text/event-stream', - 'Cache-Control': 'no-cache', - 'Connection': 'keep-alive', - 'Access-Control-Allow-Origin': req.headers.origin || '*', - 'Access-Control-Allow-Credentials': 'true' - }); - - // Send initial message - res.write('data: {"status":"connected"}\n\n'); - - // Add client to appropriate set - const clientSet = type === 'import' ? importClients : - type === 'update' ? updateClients : - type === 'reset' ? resetClients : - type === 'reset-metrics' ? resetMetricsClients : - null; - - if (clientSet) { - clientSet.add(res); - - // Remove client when connection closes - req.on('close', () => { - clientSet.delete(res); - }); - } -}; - -// Start the server -startServer(); +startServer(); diff --git a/inventory-server/src/services/ai/embeddings/similarity.js b/inventory-server/src/services/ai/embeddings/similarity.js index 8803b36..908da76 100644 --- a/inventory-server/src/services/ai/embeddings/similarity.js +++ b/inventory-server/src/services/ai/embeddings/similarity.js @@ -1,82 +1,36 @@ -/** - * Vector similarity utilities - */ - -/** - * Compute cosine similarity between two vectors - * @param {number[]} a - * @param {number[]} b - * @returns {number} Similarity score between -1 and 1 - */ -function cosineSimilarity(a, b) { - if (!a || !b || a.length !== b.length) { - return 0; - } +export function cosineSimilarity(a, b) { + if (!a || !b || a.length !== b.length) return 0; let dotProduct = 0; let normA = 0; let normB = 0; - for (let i = 0; i < a.length; i++) { dotProduct += a[i] * b[i]; normA += a[i] * a[i]; normB += b[i] * b[i]; } - const denominator = Math.sqrt(normA) * Math.sqrt(normB); - if (denominator === 0) return 0; - - return dotProduct / denominator; + return denominator === 0 ? 0 : dotProduct / denominator; } -/** - * Find top K most similar items from a collection - * @param {number[]} queryEmbedding - The embedding to search for - * @param {Array<{id: any, embedding: number[]}>} items - Items with embeddings - * @param {number} topK - Number of results to return - * @returns {Array<{id: any, similarity: number}>} - */ -function findTopMatches(queryEmbedding, items, topK = 10) { - if (!queryEmbedding || !items || items.length === 0) { - return []; - } - - const scored = items.map(item => ({ +export function findTopMatches(queryEmbedding, items, topK = 10) { + if (!queryEmbedding || !items || items.length === 0) return []; + const scored = items.map((item) => ({ id: item.id, - similarity: cosineSimilarity(queryEmbedding, item.embedding) + similarity: cosineSimilarity(queryEmbedding, item.embedding), })); - scored.sort((a, b) => b.similarity - a.similarity); - return scored.slice(0, topK); } -/** - * Find matches above a similarity threshold - * @param {number[]} queryEmbedding - * @param {Array<{id: any, embedding: number[]}>} items - * @param {number} threshold - Minimum similarity (0-1) - * @returns {Array<{id: any, similarity: number}>} - */ -function findMatchesAboveThreshold(queryEmbedding, items, threshold = 0.5) { - if (!queryEmbedding || !items || items.length === 0) { - return []; - } - +export function findMatchesAboveThreshold(queryEmbedding, items, threshold = 0.5) { + if (!queryEmbedding || !items || items.length === 0) return []; const scored = items - .map(item => ({ + .map((item) => ({ id: item.id, - similarity: cosineSimilarity(queryEmbedding, item.embedding) + similarity: cosineSimilarity(queryEmbedding, item.embedding), })) - .filter(item => item.similarity >= threshold); - + .filter((item) => item.similarity >= threshold); scored.sort((a, b) => b.similarity - a.similarity); - return scored; } - -module.exports = { - cosineSimilarity, - findTopMatches, - findMatchesAboveThreshold -}; diff --git a/inventory-server/src/services/ai/embeddings/taxonomyEmbeddings.js b/inventory-server/src/services/ai/embeddings/taxonomyEmbeddings.js index 5526f14..40d7070 100644 --- a/inventory-server/src/services/ai/embeddings/taxonomyEmbeddings.js +++ b/inventory-server/src/services/ai/embeddings/taxonomyEmbeddings.js @@ -1,74 +1,52 @@ /** * Taxonomy Embedding Service * - * Generates and caches embeddings for categories, themes, and colors. * Excludes "Black Friday", "Gifts", "Deals" categories and their children. - * - * Disk cache: embeddings are saved to data/taxonomy-embeddings.json and reused - * across server restarts. Cache is invalidated by content hash — if the taxonomy - * rows in MySQL change, the next check will detect it and regenerate automatically. - * - * Background check: after initialization, call startBackgroundCheck(getConnectionFn) - * to poll for taxonomy changes on a configurable interval (default 1h). + * Disk cache at data/taxonomy-embeddings.json; content-hash invalidated. */ -const fs = require('fs'); -const path = require('path'); -const crypto = require('crypto'); -const { findTopMatches } = require('./similarity'); +import fs from 'node:fs'; +import path from 'node:path'; +import crypto from 'node:crypto'; +import { fileURLToPath } from 'node:url'; +import { findTopMatches } from './similarity.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); -// Categories to exclude (and all their children) const EXCLUDED_CATEGORY_NAMES = ['black friday', 'gifts', 'deals']; - -// Disk cache config const CACHE_PATH = path.join(__dirname, '..', '..', '..', '..', 'data', 'taxonomy-embeddings.json'); -class TaxonomyEmbeddings { +export class TaxonomyEmbeddings { constructor({ provider, logger }) { this.provider = provider; this.logger = logger || console; - - // Cached taxonomy with embeddings this.categories = []; this.themes = []; this.colors = []; - - // Raw data without embeddings (for lookup) this.categoryMap = new Map(); this.themeMap = new Map(); this.colorMap = new Map(); - - // Content hash of the last successfully built taxonomy (from DB rows) this.contentHash = null; - this.initialized = false; this.initializing = false; this._checkInterval = null; this._regenerating = false; } - /** - * Initialize embeddings — fetches raw taxonomy rows to compute a content hash, - * then either loads the matching disk cache or generates fresh embeddings. - */ async initialize(connection) { if (this.initialized) { return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length }; } - if (this.initializing) { - // Wait for existing initialization while (this.initializing) { - await new Promise(resolve => setTimeout(resolve, 100)); + await new Promise((resolve) => setTimeout(resolve, 100)); } return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length }; } this.initializing = true; - try { - // Always fetch raw rows first — cheap (~10ms), no OpenAI calls. - // Used to compute a content hash for cache validation. const rawRows = await this._fetchRawRows(connection); const freshHash = this._computeContentHash(rawRows); @@ -77,9 +55,9 @@ class TaxonomyEmbeddings { this.categories = cached.categories; this.themes = cached.themes; this.colors = cached.colors; - this.categoryMap = new Map(this.categories.map(c => [c.id, c])); - this.themeMap = new Map(this.themes.map(t => [t.id, t])); - this.colorMap = new Map(this.colors.map(c => [c.id, c])); + this.categoryMap = new Map(this.categories.map((c) => [c.id, c])); + this.themeMap = new Map(this.themes.map((t) => [t.id, t])); + this.colorMap = new Map(this.colors.map((c) => [c.id, c])); this.contentHash = freshHash; this.initialized = true; this.logger.info(`[TaxonomyEmbeddings] Loaded from cache: ${this.categories.length} categories, ${this.themes.length} themes, ${this.colors.length} colors`); @@ -95,7 +73,6 @@ class TaxonomyEmbeddings { await this._buildAndEmbed(rawRows, freshHash); this.initialized = true; this.logger.info('[TaxonomyEmbeddings] Initialization complete'); - return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length }; } catch (error) { this.logger.error('[TaxonomyEmbeddings] Initialization failed:', error); @@ -105,26 +82,16 @@ class TaxonomyEmbeddings { } } - /** - * Start a background interval that checks for taxonomy changes and regenerates - * embeddings automatically if the content hash differs. - * - * @param {Function} getConnectionFn - async function returning { connection } - * @param {number} intervalMs - check interval, default 1 hour - */ startBackgroundCheck(getConnectionFn, intervalMs = 60 * 60 * 1000) { if (this._checkInterval) return; - this.logger.info(`[TaxonomyEmbeddings] Background taxonomy check started (every ${intervalMs / 60000} min)`); this._checkInterval = setInterval(async () => { if (this._regenerating) return; - try { const { connection } = await getConnectionFn(); const rawRows = await this._fetchRawRows(connection); const freshHash = this._computeContentHash(rawRows); - if (freshHash === this.contentHash) return; this.logger.info('[TaxonomyEmbeddings] Taxonomy changed, regenerating embeddings in background...'); @@ -146,123 +113,79 @@ class TaxonomyEmbeddings { } } - /** - * Find similar categories for a product embedding - */ findSimilarCategories(productEmbedding, topK = 10) { - if (!this.initialized || !productEmbedding) { - return []; - } - + if (!this.initialized || !productEmbedding) return []; const matches = findTopMatches(productEmbedding, this.categories, topK); - - return matches.map(match => { + return matches.map((match) => { const cat = this.categoryMap.get(match.id); return { id: match.id, name: cat?.name || '', fullPath: cat?.fullPath || '', - similarity: match.similarity + similarity: match.similarity, }; }); } - /** - * Find similar themes for a product embedding - */ findSimilarThemes(productEmbedding, topK = 5) { - if (!this.initialized || !productEmbedding) { - return []; - } - + if (!this.initialized || !productEmbedding) return []; const matches = findTopMatches(productEmbedding, this.themes, topK); - - return matches.map(match => { + return matches.map((match) => { const theme = this.themeMap.get(match.id); return { id: match.id, name: theme?.name || '', fullPath: theme?.fullPath || '', - similarity: match.similarity + similarity: match.similarity, }; }); } - /** - * Find similar colors for a product embedding - */ findSimilarColors(productEmbedding, topK = 5) { - if (!this.initialized || !productEmbedding) { - return []; - } - + if (!this.initialized || !productEmbedding) return []; const matches = findTopMatches(productEmbedding, this.colors, topK); - - return matches.map(match => { + return matches.map((match) => { const color = this.colorMap.get(match.id); return { id: match.id, name: color?.name || '', - similarity: match.similarity + similarity: match.similarity, }; }); } - /** - * Get all taxonomy data (without embeddings) for frontend - */ getTaxonomyData() { return { categories: this.categories.map(({ id, name, fullPath, parentId }) => ({ id, name, fullPath, parentId })), themes: this.themes.map(({ id, name, fullPath, parentId }) => ({ id, name, fullPath, parentId })), - colors: this.colors.map(({ id, name }) => ({ id, name })) + colors: this.colors.map(({ id, name }) => ({ id, name })), }; } - /** - * Check if service is ready - */ isReady() { return this.initialized; } - // ============================================================================ - // Private Methods - // ============================================================================ - - /** - * Fetch minimal raw rows from MySQL — used for content hash computation. - * This is the cheap path: no path-building, no embeddings, just the raw data. - */ async _fetchRawRows(connection) { const [[catRows], [themeRows], [colorRows]] = await Promise.all([ connection.query('SELECT cat_id, name, master_cat_id, type FROM product_categories WHERE type IN (10, 11, 12, 13) ORDER BY cat_id'), connection.query('SELECT cat_id, name, master_cat_id, type FROM product_categories WHERE type IN (20, 21) ORDER BY cat_id'), - connection.query('SELECT color, name, hex_color FROM product_color_list ORDER BY `order`') + connection.query('SELECT color, name, hex_color FROM product_color_list ORDER BY `order`'), ]); return { catRows, themeRows, colorRows }; } - /** - * Compute a stable SHA-256 hash of the taxonomy row content. - * Any change to IDs, names, or parent relationships will produce a different hash. - */ _computeContentHash({ catRows, themeRows, colorRows }) { const content = JSON.stringify({ - cats: catRows.map(r => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]), - themes: themeRows.map(r => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]), - colors: colorRows.map(r => [r.color, r.name]).sort() + cats: catRows.map((r) => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]), + themes: themeRows.map((r) => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]), + colors: colorRows.map((r) => [r.color, r.name]).sort(), }); return crypto.createHash('sha256').update(content).digest('hex').slice(0, 16); } - /** - * Build full taxonomy objects and generate embeddings, then atomically swap - * the in-memory state. Called on cache miss and on background change detection. - */ async _buildAndEmbed(rawRows, contentHash) { const { catRows, themeRows, colorRows } = rawRows; - const categories = this._buildCategories(catRows); const themes = this._buildThemes(themeRows); const colors = this._buildColors(colorRows); @@ -272,23 +195,22 @@ class TaxonomyEmbeddings { const [catEmbeddings, themeEmbeddings, colorEmbeddings] = await Promise.all([ this._generateEmbeddings(categories, 'categories'), this._generateEmbeddings(themes, 'themes'), - this._generateEmbeddings(colors, 'colors') + this._generateEmbeddings(colors, 'colors'), ]); - // Atomic in-memory swap (single-threaded JS — readers always see a consistent state) this.categories = catEmbeddings; this.themes = themeEmbeddings; this.colors = colorEmbeddings; - this.categoryMap = new Map(this.categories.map(c => [c.id, c])); - this.themeMap = new Map(this.themes.map(t => [t.id, t])); - this.colorMap = new Map(this.colors.map(c => [c.id, c])); + this.categoryMap = new Map(this.categories.map((c) => [c.id, c])); + this.themeMap = new Map(this.themes.map((t) => [t.id, t])); + this.colorMap = new Map(this.colors.map((c) => [c.id, c])); this.contentHash = contentHash; this._saveCache(); } _buildCategories(rows) { - const byId = new Map(rows.map(r => [r.cat_id, r])); + const byId = new Map(rows.map((r) => [r.cat_id, r])); const excludedIds = new Set(); for (const row of rows) { @@ -297,7 +219,6 @@ class TaxonomyEmbeddings { } } - // Multiple passes to find all descendants of excluded categories let foundNew = true; while (foundNew) { foundNew = false; @@ -314,98 +235,80 @@ class TaxonomyEmbeddings { const categories = []; for (const row of rows) { if (excludedIds.has(row.cat_id)) continue; - const pathParts = []; let current = row; while (current) { pathParts.unshift(current.name); current = current.master_cat_id ? byId.get(current.master_cat_id) : null; } - categories.push({ id: row.cat_id, name: row.name, parentId: row.master_cat_id, type: row.type, fullPath: pathParts.join(' > '), - embeddingText: pathParts.join(' ') + embeddingText: pathParts.join(' '), }); } - return categories; } _buildThemes(rows) { - const byId = new Map(rows.map(r => [r.cat_id, r])); - - return rows.map(row => { + const byId = new Map(rows.map((r) => [r.cat_id, r])); + return rows.map((row) => { const pathParts = []; let current = row; while (current) { pathParts.unshift(current.name); current = current.master_cat_id ? byId.get(current.master_cat_id) : null; } - return { id: row.cat_id, name: row.name, parentId: row.master_cat_id, type: row.type, fullPath: pathParts.join(' > '), - embeddingText: pathParts.join(' ') + embeddingText: pathParts.join(' '), }; }); } _buildColors(rows) { - return rows.map(row => ({ + return rows.map((row) => ({ id: row.color, name: row.name, hexColor: row.hex_color, - embeddingText: row.name + embeddingText: row.name, })); } async _generateEmbeddings(items, label) { - if (items.length === 0) { - return items; - } + if (items.length === 0) return items; const startTime = Date.now(); - const texts = items.map(item => item.embeddingText); + const texts = items.map((item) => item.embeddingText); const results = [...items]; - // Process in batches for await (const chunk of this.provider.embedBatchChunked(texts, { batchSize: 100 })) { for (let i = 0; i < chunk.embeddings.length; i++) { const globalIndex = chunk.startIndex + i; - results[globalIndex] = { - ...results[globalIndex], - embedding: chunk.embeddings[i] - }; + results[globalIndex] = { ...results[globalIndex], embedding: chunk.embeddings[i] }; } } const elapsed = Date.now() - startTime; this.logger.info(`[TaxonomyEmbeddings] Generated ${items.length} ${label} embeddings in ${elapsed}ms`); - return results; } - // ============================================================================ - // Disk Cache Methods - // ============================================================================ - _loadCache() { try { if (!fs.existsSync(CACHE_PATH)) return null; - const data = JSON.parse(fs.readFileSync(CACHE_PATH, 'utf8')); if (!data.contentHash || !data.categories?.length || !data.themes?.length || !data.colors?.length) { this.logger.warn('[TaxonomyEmbeddings] Disk cache malformed or missing content hash, will regenerate'); return null; } - return data; } catch (err) { this.logger.warn('[TaxonomyEmbeddings] Failed to load disk cache:', err.message); @@ -429,5 +332,3 @@ class TaxonomyEmbeddings { } } } - -module.exports = { TaxonomyEmbeddings }; diff --git a/inventory-server/src/services/ai/index.js b/inventory-server/src/services/ai/index.js index 9f4538e..bef3d5c 100644 --- a/inventory-server/src/services/ai/index.js +++ b/inventory-server/src/services/ai/index.js @@ -1,17 +1,16 @@ /** * AI Service * - * Main entry point for AI functionality including: - * - Embeddings for taxonomy suggestions (OpenAI) - * - Chat completions for validation tasks (Groq) - * - Task registry for AI operations + * Main entry point for AI functionality (embeddings + chat completions + task registry). */ -const { OpenAIProvider } = require('./providers/openaiProvider'); -const { GroqProvider, MODELS: GROQ_MODELS } = require('./providers/groqProvider'); -const { TaxonomyEmbeddings } = require('./embeddings/taxonomyEmbeddings'); -const { cosineSimilarity, findTopMatches } = require('./embeddings/similarity'); -const { getRegistry, TASK_IDS, registerAllTasks } = require('./tasks'); +import { OpenAIProvider } from './providers/openaiProvider.js'; +import { GroqProvider, MODELS as GROQ_MODELS } from './providers/groqProvider.js'; +import { TaxonomyEmbeddings } from './embeddings/taxonomyEmbeddings.js'; +import { cosineSimilarity, findTopMatches } from './embeddings/similarity.js'; +import { getRegistry, TASK_IDS, registerAllTasks } from './tasks/index.js'; + +export { TASK_IDS, GROQ_MODELS, cosineSimilarity, findTopMatches }; let initialized = false; let initializing = false; @@ -19,54 +18,28 @@ let openaiProvider = null; let groqProvider = null; let taxonomyEmbeddings = null; let logger = console; - -// Store pool reference for task access let appPool = null; -/** - * Initialize the AI service - * @param {Object} options - * @param {string} options.openaiApiKey - OpenAI API key (for embeddings) - * @param {string} [options.groqApiKey] - Groq API key (for chat completions) - * @param {Object} options.mysqlConnection - MySQL connection for taxonomy data - * @param {Object} [options.pool] - PostgreSQL pool for prompt loading - * @param {Object} [options.logger] - Logger instance - */ -async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, logger: customLogger }) { - if (initialized) { - return { success: true, message: 'Already initialized' }; - } +export async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, logger: customLogger }) { + if (initialized) return { success: true, message: 'Already initialized' }; if (initializing) { - // Wait for existing initialization while (initializing) { - await new Promise(resolve => setTimeout(resolve, 100)); + await new Promise((resolve) => setTimeout(resolve, 100)); } return { success: initialized, message: initialized ? 'Initialized' : 'Initialization failed' }; } initializing = true; - try { - if (customLogger) { - logger = customLogger; - } - - if (!openaiApiKey) { - throw new Error('OpenAI API key is required'); - } + if (customLogger) logger = customLogger; + if (!openaiApiKey) throw new Error('OpenAI API key is required'); logger.info('[AI] Initializing AI service...'); + if (pool) appPool = pool; - // Store pool reference for tasks - if (pool) { - appPool = pool; - } - - // Create OpenAI provider (for embeddings) openaiProvider = new OpenAIProvider({ apiKey: openaiApiKey }); - // Create Groq provider (for chat completions) if API key provided if (groqApiKey) { groqProvider = new GroqProvider({ apiKey: groqApiKey }); logger.info('[AI] Groq provider initialized for chat completions'); @@ -74,32 +47,19 @@ async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, log logger.warn('[AI] No Groq API key provided - chat completion tasks will not be available'); } - // Create and initialize taxonomy embeddings - taxonomyEmbeddings = new TaxonomyEmbeddings({ - provider: openaiProvider, - logger - }); - + taxonomyEmbeddings = new TaxonomyEmbeddings({ provider: openaiProvider, logger }); const stats = await taxonomyEmbeddings.initialize(mysqlConnection); - // Register validation tasks if Groq is available - if (groqProvider) { - registerValidationTasks(); - } + if (groqProvider) registerValidationTasks(); initialized = true; logger.info('[AI] AI service initialized', { ...stats, groqEnabled: !!groqProvider, - tasksRegistered: getRegistry().list() + tasksRegistered: getRegistry().list(), }); - return { - success: true, - message: 'Initialized', - stats, - groqEnabled: !!groqProvider - }; + return { success: true, message: 'Initialized', stats, groqEnabled: !!groqProvider }; } catch (error) { logger.error('[AI] Initialization failed:', error); return { success: false, message: error.message }; @@ -108,41 +68,20 @@ async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, log } } -/** - * Register validation tasks with the task registry - * Called during initialization if Groq is available - */ function registerValidationTasks() { registerAllTasks(logger); logger.info('[AI] Validation tasks registered'); } -/** - * Check if service is ready - */ -function isReady() { +export function isReady() { return initialized && taxonomyEmbeddings?.isReady(); } -/** - * Start background taxonomy change detection. - * Call once after initialization, passing a function that returns { connection }. - * @param {Function} getConnectionFn - * @param {number} [intervalMs] - default 1 hour - */ -function startBackgroundCheck(getConnectionFn, intervalMs) { +export function startBackgroundCheck(getConnectionFn, intervalMs) { if (!initialized || !taxonomyEmbeddings) return; taxonomyEmbeddings.startBackgroundCheck(getConnectionFn, intervalMs); } -/** - * Build weighted product text for embedding. - * Weights the product name heavily by repeating it, and truncates long descriptions - * to prevent verbose marketing copy from drowning out the product signal. - * - * @param {Object} product - Product with name, description, company, line - * @returns {string} - Combined text for embedding - */ function buildProductText(product) { const parts = []; const name = product.name?.trim(); @@ -150,20 +89,9 @@ function buildProductText(product) { const company = (product.company_name || product.company)?.trim(); const line = (product.line_name || product.line)?.trim(); - // Name is most important - repeat 3x to weight it heavily in the embedding - if (name) { - parts.push(name, name, name); - } - - // Company and line provide context - if (company) { - parts.push(company); - } - if (line) { - parts.push(line); - } - - // Truncate description to prevent it from overwhelming the signal + if (name) parts.push(name, name, name); + if (company) parts.push(company); + if (line) parts.push(line); if (description) { const truncated = description.length > 500 ? description.substring(0, 500) + '...' @@ -174,74 +102,35 @@ function buildProductText(product) { return parts.join(' ').trim(); } -/** - * Generate embedding for a product - * @param {Object} product - Product with name, description, company, line - * @returns {Promise<{embedding: number[], latencyMs: number}>} - */ -async function getProductEmbedding(product) { - if (!initialized || !openaiProvider) { - throw new Error('AI service not initialized'); - } +export async function getProductEmbedding(product) { + if (!initialized || !openaiProvider) throw new Error('AI service not initialized'); const text = buildProductText(product); - - if (!text) { - return { embedding: null, latencyMs: 0 }; - } + if (!text) return { embedding: null, latencyMs: 0 }; const result = await openaiProvider.embed(text); - - return { - embedding: result.embeddings[0], - latencyMs: result.latencyMs - }; + return { embedding: result.embeddings[0], latencyMs: result.latencyMs }; } -/** - * Generate embeddings for multiple products - * @param {Object[]} products - Array of products - * @returns {Promise<{embeddings: Array<{index: number, embedding: number[]}>, latencyMs: number}>} - */ -async function getProductEmbeddings(products) { - if (!initialized || !openaiProvider) { - throw new Error('AI service not initialized'); - } +export async function getProductEmbeddings(products) { + if (!initialized || !openaiProvider) throw new Error('AI service not initialized'); const texts = products.map(buildProductText); + const validIndices = texts.map((t, i) => t ? i : -1).filter((i) => i >= 0); + const validTexts = texts.filter((t) => t); - // Track which products have empty text - const validIndices = texts.map((t, i) => t ? i : -1).filter(i => i >= 0); - const validTexts = texts.filter(t => t); - - if (validTexts.length === 0) { - return { embeddings: [], latencyMs: 0 }; - } + if (validTexts.length === 0) return { embeddings: [], latencyMs: 0 }; const result = await openaiProvider.embed(validTexts); - - // Map embeddings back to original indices const embeddings = validIndices.map((originalIndex, resultIndex) => ({ index: originalIndex, - embedding: result.embeddings[resultIndex] + embedding: result.embeddings[resultIndex], })); - - return { - embeddings, - latencyMs: result.latencyMs - }; + return { embeddings, latencyMs: result.latencyMs }; } -/** - * Find similar taxonomy items for a product embedding - * @param {number[]} productEmbedding - * @param {Object} options - * @returns {{categories: Array, themes: Array, colors: Array}} - */ -function findSimilarTaxonomy(productEmbedding, options = {}) { - if (!initialized || !taxonomyEmbeddings) { - throw new Error('AI service not initialized'); - } +export function findSimilarTaxonomy(productEmbedding, options = {}) { + if (!initialized || !taxonomyEmbeddings) throw new Error('AI service not initialized'); const topCategories = options.topCategories ?? 10; const topThemes = options.topThemes ?? 5; @@ -250,25 +139,15 @@ function findSimilarTaxonomy(productEmbedding, options = {}) { return { categories: taxonomyEmbeddings.findSimilarCategories(productEmbedding, topCategories), themes: taxonomyEmbeddings.findSimilarThemes(productEmbedding, topThemes), - colors: taxonomyEmbeddings.findSimilarColors(productEmbedding, topColors) + colors: taxonomyEmbeddings.findSimilarColors(productEmbedding, topColors), }; } -/** - * Get product embedding and find similar taxonomy in one call - * @param {Object} product - * @param {Object} options - */ -async function getSuggestionsForProduct(product, options = {}) { +export async function getSuggestionsForProduct(product, options = {}) { const { embedding, latencyMs: embeddingLatency } = await getProductEmbedding(product); if (!embedding) { - return { - categories: [], - themes: [], - colors: [], - latencyMs: embeddingLatency - }; + return { categories: [], themes: [], colors: [], latencyMs: embeddingLatency }; } const startSearch = Date.now(); @@ -279,27 +158,17 @@ async function getSuggestionsForProduct(product, options = {}) { ...suggestions, latencyMs: embeddingLatency + searchLatency, embeddingLatencyMs: embeddingLatency, - searchLatencyMs: searchLatency + searchLatencyMs: searchLatency, }; } -/** - * Get all taxonomy data (without embeddings) for frontend - */ -function getTaxonomyData() { - if (!initialized || !taxonomyEmbeddings) { - throw new Error('AI service not initialized'); - } - +export function getTaxonomyData() { + if (!initialized || !taxonomyEmbeddings) throw new Error('AI service not initialized'); return taxonomyEmbeddings.getTaxonomyData(); } -/** - * Get service status - */ -function getStatus() { +export function getStatus() { const registry = getRegistry(); - return { initialized, ready: isReady(), @@ -309,90 +178,56 @@ function getStatus() { taxonomyStats: taxonomyEmbeddings ? { categories: taxonomyEmbeddings.categories?.length || 0, themes: taxonomyEmbeddings.themes?.length || 0, - colors: taxonomyEmbeddings.colors?.length || 0 + colors: taxonomyEmbeddings.colors?.length || 0, } : null, tasks: { registered: registry.list(), - count: registry.size() - } + count: registry.size(), + }, }; } -/** - * Run an AI task by ID - * @param {string} taskId - Task identifier from TASK_IDS - * @param {Object} payload - Task-specific input - * @returns {Promise} Task result - */ -async function runTask(taskId, payload = {}) { - if (!initialized) { - throw new Error('AI service not initialized'); - } - - if (!groqProvider) { - throw new Error('Groq provider not available - chat completion tasks require GROQ_API_KEY'); - } +export async function runTask(taskId, payload = {}) { + if (!initialized) throw new Error('AI service not initialized'); + if (!groqProvider) throw new Error('Groq provider not available - chat completion tasks require GROQ_API_KEY'); const registry = getRegistry(); return registry.runTask(taskId, { ...payload, - // Inject dependencies tasks may need provider: groqProvider, - // Use pool from payload if provided (from route), fall back to stored appPool pool: payload.pool || appPool, - logger + logger, }); } -/** - * Get the Groq provider instance (for direct use if needed) - * @returns {GroqProvider|null} - */ -function getGroqProvider() { +export function getGroqProvider() { return groqProvider; } -/** - * Get the PostgreSQL pool (for tasks that need DB access) - * @returns {Object|null} - */ -function getPool() { +export function getPool() { return appPool; } -/** - * Check if chat completion tasks are available - * @returns {boolean} - */ -function hasChatCompletion() { +export function hasChatCompletion() { return !!groqProvider; } -module.exports = { - // Initialization +export default { initialize, isReady, getStatus, startBackgroundCheck, - - // Embeddings (OpenAI) getProductEmbedding, getProductEmbeddings, findSimilarTaxonomy, getSuggestionsForProduct, getTaxonomyData, - - // Chat completions (Groq) runTask, hasChatCompletion, getGroqProvider, getPool, - - // Constants TASK_IDS, GROQ_MODELS, - - // Re-export utilities cosineSimilarity, - findTopMatches + findTopMatches, }; diff --git a/inventory-server/src/services/ai/prompts/descriptionPrompts.js b/inventory-server/src/services/ai/prompts/descriptionPrompts.js index a7665b6..67c9438 100644 --- a/inventory-server/src/services/ai/prompts/descriptionPrompts.js +++ b/inventory-server/src/services/ai/prompts/descriptionPrompts.js @@ -1,81 +1,41 @@ /** * Description Validation Prompts - * - * Functions for building and parsing description validation prompts. - * System and general prompts are loaded from the database. */ -/** - * Sanitize an issue string from AI response - * AI sometimes returns malformed strings with escape sequences - * - * @param {string} issue - Raw issue string - * @returns {string} Cleaned issue string - */ function sanitizeIssue(issue) { if (!issue || typeof issue !== 'string') return ''; - - let cleaned = issue - // Remove trailing backslashes (incomplete escapes) + return issue .replace(/\\+$/, '') - // Fix malformed escaped quotes at end of string .replace(/\\",?\)?$/, '') - // Clean up double-escaped quotes .replace(/\\\\"/g, '"') - // Clean up single escaped quotes that aren't needed .replace(/\\"/g, '"') - // Remove any remaining trailing punctuation artifacts .replace(/[,\s]+$/, '') - // Trim whitespace .trim(); - - return cleaned; } -/** - * Build the user prompt for description validation - * Combines database prompts with product data - * - * @param {Object} product - Product data - * @param {string} product.name - Product name - * @param {string} product.description - Current description - * @param {string} [product.company_name] - Company name - * @param {string} [product.categories] - Product categories - * @param {Object} prompts - Prompts loaded from database - * @param {string} prompts.general - General description guidelines - * @param {string} [prompts.companySpecific] - Company-specific rules - * @returns {string} Complete user prompt - */ -function buildDescriptionUserPrompt(product, prompts) { +export function buildDescriptionUserPrompt(product, prompts) { const parts = []; - // Add general prompt/guidelines if provided if (prompts.general) { parts.push(prompts.general); - parts.push(''); // Empty line for separation + parts.push(''); } - // Add company-specific rules if provided if (prompts.companySpecific) { parts.push(`COMPANY-SPECIFIC RULES FOR ${product.company_name || 'THIS COMPANY'}:`); parts.push(prompts.companySpecific); - parts.push(''); // Empty line for separation + parts.push(''); } - // Add product information parts.push('PRODUCT TO VALIDATE:'); parts.push(`NAME: "${product.name || ''}"`); parts.push(`COMPANY: ${product.company_name || 'Unknown'}`); - - if (product.categories) { - parts.push(`CATEGORIES: ${product.categories}`); - } + if (product.categories) parts.push(`CATEGORIES: ${product.categories}`); parts.push(''); parts.push('CURRENT DESCRIPTION:'); parts.push(`"${product.description || '(empty)'}"`); - // Add response format instructions parts.push(''); parts.push('CRITICAL RULES:'); parts.push('- If isValid is false, you MUST provide a suggestion with the improved description'); @@ -86,68 +46,40 @@ function buildDescriptionUserPrompt(product, prompts) { parts.push(JSON.stringify({ isValid: 'true if perfect, false if ANY changes needed', suggestion: 'REQUIRED when isValid is false - the complete improved description', - issues: ['list each problem found (empty array only if isValid is true)'] + issues: ['list each problem found (empty array only if isValid is true)'], }, null, 2)); return parts.join('\n'); } -/** - * Parse the AI response for description validation - * - * @param {Object|null} parsed - Parsed JSON from AI - * @param {string} content - Raw response content - * @returns {Object} - */ -function parseDescriptionResponse(parsed, content) { - // If we got valid parsed JSON, use it +export function parseDescriptionResponse(parsed, content) { if (parsed && typeof parsed.isValid === 'boolean') { - // Sanitize issues - AI sometimes returns malformed escape sequences const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : []; - const issues = rawIssues - .map(sanitizeIssue) - .filter(issue => issue.length > 0); - + const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0); const suggestion = parsed.suggestion || null; - - // IMPORTANT: LLMs sometimes return contradictory data (isValid: true with issues). - // If there are issues, treat as invalid regardless of what the AI said. - // Also if there's a suggestion, the AI thought something needed to change. const isValid = parsed.isValid && issues.length === 0 && !suggestion; - return { isValid, suggestion, issues }; } - // Handle case where isValid is a string "true"/"false" instead of boolean if (parsed && typeof parsed.isValid === 'string') { const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : []; - const issues = rawIssues - .map(sanitizeIssue) - .filter(issue => issue.length > 0); + const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0); const suggestion = parsed.suggestion || null; const rawIsValid = parsed.isValid.toLowerCase() !== 'false'; - - // Same defensive logic: if there are issues, it's not valid const isValid = rawIsValid && issues.length === 0 && !suggestion; - return { isValid, suggestion, issues }; } - // Try to extract from content if parsing failed try { - // Look for isValid pattern const isValidMatch = content.match(/"isValid"\s*:\s*(true|false)/i); const isValid = isValidMatch ? isValidMatch[1].toLowerCase() === 'true' : true; - // Look for suggestion (might be multiline) const suggestionMatch = content.match(/"suggestion"\s*:\s*"((?:[^"\\]|\\.)*)"/s); let suggestion = suggestionMatch ? suggestionMatch[1] : null; if (suggestion) { - // Unescape common escapes suggestion = suggestion.replace(/\\n/g, '\n').replace(/\\"/g, '"'); } - // Look for issues array const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/); let issues = []; if (issuesMatch) { @@ -155,22 +87,13 @@ function parseDescriptionResponse(parsed, content) { const issueStrings = issuesContent.match(/"([^"]+)"/g); if (issueStrings) { issues = issueStrings - .map(s => sanitizeIssue(s.replace(/"/g, ''))) - .filter(issue => issue.length > 0); + .map((s) => sanitizeIssue(s.replace(/"/g, ''))) + .filter((issue) => issue.length > 0); } } - - // Same logic: if there are issues, it's not valid const finalIsValid = isValid && issues.length === 0 && !suggestion; - return { isValid: finalIsValid, suggestion, issues }; } catch { - // Default to valid if we can't parse anything return { isValid: true, suggestion: null, issues: [] }; } } - -module.exports = { - buildDescriptionUserPrompt, - parseDescriptionResponse -}; diff --git a/inventory-server/src/services/ai/prompts/namePrompts.js b/inventory-server/src/services/ai/prompts/namePrompts.js index 187fbf2..cd2f2c1 100644 --- a/inventory-server/src/services/ai/prompts/namePrompts.js +++ b/inventory-server/src/services/ai/prompts/namePrompts.js @@ -1,164 +1,94 @@ /** * Name Validation Prompts - * - * Functions for building and parsing name validation prompts. - * System and general prompts are loaded from the database. */ -/** - * Sanitize an issue string from AI response - * AI sometimes returns malformed strings with escape sequences - * - * @param {string} issue - Raw issue string - * @returns {string} Cleaned issue string - */ function sanitizeIssue(issue) { if (!issue || typeof issue !== 'string') return ''; - - let cleaned = issue - // Remove trailing backslashes (incomplete escapes) + return issue .replace(/\\+$/, '') - // Fix malformed escaped quotes at end of string .replace(/\\",?\)?$/, '') - // Clean up double-escaped quotes .replace(/\\\\"/g, '"') - // Clean up single escaped quotes that aren't needed .replace(/\\"/g, '"') - // Remove any remaining trailing punctuation artifacts .replace(/[,\s]+$/, '') - // Trim whitespace .trim(); - - return cleaned; } -/** - * Build the user prompt for name validation - * Combines database prompts with product data - * - * @param {Object} product - Product data - * @param {string} product.name - Current product name - * @param {string} [product.company_name] - Company name - * @param {string} [product.line_name] - Product line name - * @param {string} [product.subline_name] - Product subline name - * @param {string[]} [product.siblingNames] - Names of other products in the same line - * @param {Object} prompts - Prompts loaded from database - * @param {string} prompts.general - General naming conventions - * @param {string} [prompts.companySpecific] - Company-specific rules - * @returns {string} Complete user prompt - */ -function buildNameUserPrompt(product, prompts) { +export function buildNameUserPrompt(product, prompts) { const parts = []; - // Add general prompt/conventions if provided if (prompts.general) { parts.push(prompts.general); - parts.push(''); // Empty line for separation + parts.push(''); } - // Add company-specific rules if provided if (prompts.companySpecific) { parts.push(`COMPANY-SPECIFIC RULES FOR ${product.company_name || 'THIS COMPANY'}:`); parts.push(prompts.companySpecific); - parts.push(''); // Empty line for separation + parts.push(''); } - // Add product information parts.push('PRODUCT TO VALIDATE:'); parts.push(`NAME: "${product.name || ''}"`); parts.push(`COMPANY: ${product.company_name || 'Unknown'}`); parts.push(`LINE: ${product.line_name || 'None'}`); - if (product.subline_name) { - parts.push(`SUBLINE: ${product.subline_name}`); - } + if (product.subline_name) parts.push(`SUBLINE: ${product.subline_name}`); - // Add sibling context for naming decisions if (product.siblingNames && product.siblingNames.length > 0) { parts.push(''); parts.push(`OTHER PRODUCTS IN THIS LINE (${product.siblingNames.length + 1} total including this one):`); - product.siblingNames.forEach(name => { - parts.push(`- ${name}`); - }); + product.siblingNames.forEach((name) => parts.push(`- ${name}`)); } - // Add response format instructions parts.push(''); parts.push('RESPOND WITH JSON:'); parts.push(JSON.stringify({ isValid: 'true/false', suggestion: 'corrected name if changes needed, or null if valid', - issues: ['issue 1', 'issue 2 (empty array if valid)'] + issues: ['issue 1', 'issue 2 (empty array if valid)'], }, null, 2)); return parts.join('\n'); } -/** - * Parse the AI response for name validation - * - * @param {Object|null} parsed - Parsed JSON from AI - * @param {string} content - Raw response content - * @returns {Object} - */ -function parseNameResponse(parsed, content) { - // Debug: Log what we're trying to parse +export function parseNameResponse(parsed, content) { console.log('[parseNameResponse] Input:', { hasParsed: !!parsed, parsedIsValid: parsed?.isValid, parsedType: typeof parsed?.isValid, - contentPreview: content?.substring(0, 3000) + contentPreview: content?.substring(0, 3000), }); - // If we got valid parsed JSON, use it if (parsed && typeof parsed.isValid === 'boolean') { - // Sanitize issues - AI sometimes returns malformed escape sequences const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : []; - const issues = rawIssues - .map(sanitizeIssue) - .filter(issue => issue.length > 0); + const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0); const suggestion = parsed.suggestion || null; - - // IMPORTANT: LLMs sometimes return contradictory data (isValid: true with issues). - // If there are issues, treat as invalid regardless of what the AI said. const isValid = parsed.isValid && issues.length === 0 && !suggestion; - return { isValid, suggestion, issues }; } - // Handle case where isValid is a string "true"/"false" instead of boolean if (parsed && typeof parsed.isValid === 'string') { const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : []; - const issues = rawIssues - .map(sanitizeIssue) - .filter(issue => issue.length > 0); + const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0); const suggestion = parsed.suggestion || null; const rawIsValid = parsed.isValid.toLowerCase() !== 'false'; - - // Same defensive logic: if there are issues, it's not valid const isValid = rawIsValid && issues.length === 0 && !suggestion; - console.log('[parseNameResponse] Parsed isValid as string:', parsed.isValid, '→', isValid); return { isValid, suggestion, issues }; } - // Try to extract from content if parsing failed try { - // Look for isValid pattern - handle both boolean and quoted string - // Matches: "isValid": true, "isValid": false, "isValid": "true", "isValid": "false" const isValidMatch = content.match(/"isValid"\s*:\s*"?(true|false)"?/i); const isValid = isValidMatch ? isValidMatch[1].toLowerCase() === 'true' : true; console.log('[parseNameResponse] Regex extraction:', { isValidMatch: isValidMatch?.[0], isValidValue: isValidMatch?.[1], - resultIsValid: isValid + resultIsValid: isValid, }); - // Look for suggestion - handle escaped quotes and null const suggestionMatch = content.match(/"suggestion"\s*:\s*(?:"([^"\\]*(?:\\.[^"\\]*)*)"|null)/); const suggestion = suggestionMatch ? (suggestionMatch[1] || null) : null; - // Look for issues array const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/); let issues = []; if (issuesMatch) { @@ -166,22 +96,13 @@ function parseNameResponse(parsed, content) { const issueStrings = issuesContent.match(/"([^"]+)"/g); if (issueStrings) { issues = issueStrings - .map(s => sanitizeIssue(s.replace(/"/g, ''))) - .filter(issue => issue.length > 0); + .map((s) => sanitizeIssue(s.replace(/"/g, ''))) + .filter((issue) => issue.length > 0); } } - - // Same defensive logic: if there are issues, it's not valid const finalIsValid = isValid && issues.length === 0 && !suggestion; - return { isValid: finalIsValid, suggestion, issues }; } catch { - // Default to valid if we can't parse anything return { isValid: true, suggestion: null, issues: [] }; } } - -module.exports = { - buildNameUserPrompt, - parseNameResponse -}; diff --git a/inventory-server/src/services/ai/prompts/promptLoader.js b/inventory-server/src/services/ai/prompts/promptLoader.js index 44282dc..79f3caa 100644 --- a/inventory-server/src/services/ai/prompts/promptLoader.js +++ b/inventory-server/src/services/ai/prompts/promptLoader.js @@ -1,35 +1,18 @@ /** - * Prompt Loader - * - * Utilities to load AI prompts from the ai_prompts PostgreSQL table. - * Supports loading prompts by base type (e.g., 'name_validation' loads - * name_validation_system, name_validation_general, and optionally - * name_validation_company_specific). + * Prompt Loader — loads AI prompts from the ai_prompts PostgreSQL table. */ -/** - * Load a single prompt by exact type - * @param {Object} pool - PostgreSQL pool - * @param {string} promptType - Exact prompt type (e.g., 'name_validation_system') - * @param {string} [company] - Company identifier (for company_specific types) - * @returns {Promise} Prompt text or null if not found - */ -async function loadPromptByType(pool, promptType, company = null) { +export async function loadPromptByType(pool, promptType, company = null) { try { - let result; - - if (company) { - result = await pool.query( - 'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company = $2', - [promptType, company] - ); - } else { - result = await pool.query( - 'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company IS NULL', - [promptType] - ); - } - + const result = company + ? await pool.query( + 'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company = $2', + [promptType, company] + ) + : await pool.query( + 'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company IS NULL', + [promptType] + ); return result.rows[0]?.prompt_text || null; } catch (error) { console.error(`[PromptLoader] Error loading ${promptType} prompt:`, error.message); @@ -37,93 +20,46 @@ async function loadPromptByType(pool, promptType, company = null) { } } -/** - * Load all prompts for a task type (system, general, and optionally company-specific) - * - * @param {Object} pool - PostgreSQL pool - * @param {string} baseType - Base type name (e.g., 'name_validation', 'description_validation') - * @param {string|null} [company] - Optional company ID for company-specific prompts - * @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>} - */ -async function loadPromptsByType(pool, baseType, company = null) { +export async function loadPromptsByType(pool, baseType, company = null) { const systemType = `${baseType}_system`; const generalType = `${baseType}_general`; const companyType = `${baseType}_company_specific`; - // Load system and general prompts in parallel const [system, general] = await Promise.all([ loadPromptByType(pool, systemType), - loadPromptByType(pool, generalType) + loadPromptByType(pool, generalType), ]); - // Load company-specific prompt if company is provided let companySpecific = null; if (company) { companySpecific = await loadPromptByType(pool, companyType, company); } - - return { - system, - general, - companySpecific - }; + return { system, general, companySpecific }; } -/** - * Load name validation prompts - * @param {Object} pool - PostgreSQL pool - * @param {string|null} [company] - Optional company ID - * @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>} - */ -async function loadNameValidationPrompts(pool, company = null) { +export function loadNameValidationPrompts(pool, company = null) { return loadPromptsByType(pool, 'name_validation', company); } -/** - * Load description validation prompts - * @param {Object} pool - PostgreSQL pool - * @param {string|null} [company] - Optional company ID - * @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>} - */ -async function loadDescriptionValidationPrompts(pool, company = null) { +export function loadDescriptionValidationPrompts(pool, company = null) { return loadPromptsByType(pool, 'description_validation', company); } -/** - * Load sanity check prompts (no company-specific variant) - * @param {Object} pool - PostgreSQL pool - * @returns {Promise<{system: string|null, general: string|null, companySpecific: null}>} - */ -async function loadSanityCheckPrompts(pool) { +export function loadSanityCheckPrompts(pool) { return loadPromptsByType(pool, 'sanity_check', null); } -/** - * Load bulk validation prompts (GPT-5 validation) - * @param {Object} pool - PostgreSQL pool - * @param {string|null} [company] - Optional company ID - * @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>} - */ -async function loadBulkValidationPrompts(pool, company = null) { +export function loadBulkValidationPrompts(pool, company = null) { return loadPromptsByType(pool, 'bulk_validation', company); } -/** - * Load bulk validation prompts for multiple companies at once - * @param {Object} pool - PostgreSQL pool - * @param {string[]} companyIds - Array of company IDs - * @returns {Promise<{system: string|null, general: string|null, companyPrompts: Map}>} - */ -async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) { - // Load system and general prompts +export async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) { const [system, general] = await Promise.all([ loadPromptByType(pool, 'bulk_validation_system'), - loadPromptByType(pool, 'bulk_validation_general') + loadPromptByType(pool, 'bulk_validation_general'), ]); - // Load company-specific prompts for all provided companies const companyPrompts = new Map(); - if (companyIds.length > 0) { try { const result = await pool.query( @@ -132,7 +68,6 @@ async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) { AND company = ANY($1)`, [companyIds] ); - for (const row of result.rows) { companyPrompts.set(row.company, row.prompt_text); } @@ -140,35 +75,14 @@ async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) { console.error('[PromptLoader] Error loading company-specific prompts:', error.message); } } - - return { - system, - general, - companyPrompts - }; + return { system, general, companyPrompts }; } -/** - * Validate that required prompts exist, throw error if missing - * @param {Object} prompts - Prompts object from loadPromptsByType - * @param {string} baseType - Base type for error messages - * @param {Object} options - Validation options - * @param {boolean} [options.requireSystem=true] - Require system prompt - * @param {boolean} [options.requireGeneral=true] - Require general prompt - * @throws {Error} If required prompts are missing - */ -function validateRequiredPrompts(prompts, baseType, options = {}) { +export function validateRequiredPrompts(prompts, baseType, options = {}) { const { requireSystem = true, requireGeneral = true } = options; const missing = []; - - if (requireSystem && !prompts.system) { - missing.push(`${baseType}_system`); - } - - if (requireGeneral && !prompts.general) { - missing.push(`${baseType}_general`); - } - + if (requireSystem && !prompts.system) missing.push(`${baseType}_system`); + if (requireGeneral && !prompts.general) missing.push(`${baseType}_general`); if (missing.length > 0) { throw new Error( `Missing required AI prompts: ${missing.join(', ')}. ` + @@ -176,19 +90,3 @@ function validateRequiredPrompts(prompts, baseType, options = {}) { ); } } - -module.exports = { - // Core loader - loadPromptByType, - loadPromptsByType, - - // Task-specific loaders - loadNameValidationPrompts, - loadDescriptionValidationPrompts, - loadSanityCheckPrompts, - loadBulkValidationPrompts, - loadBulkValidationPromptsForCompanies, - - // Validation - validateRequiredPrompts -}; diff --git a/inventory-server/src/services/ai/prompts/sanityCheckPrompts.js b/inventory-server/src/services/ai/prompts/sanityCheckPrompts.js index f628556..7a4a41d 100644 --- a/inventory-server/src/services/ai/prompts/sanityCheckPrompts.js +++ b/inventory-server/src/services/ai/prompts/sanityCheckPrompts.js @@ -1,21 +1,8 @@ /** * Sanity Check Prompts - * - * Functions for building and parsing batch product consistency validation prompts. - * System and general prompts are loaded from the database. */ -/** - * Build the user prompt for sanity check - * Combines database prompts with product data - * - * @param {Object[]} products - Array of product data (limited fields for context) - * @param {Object} prompts - Prompts loaded from database - * @param {string} prompts.general - General sanity check rules - * @returns {string} Complete user prompt - */ -function buildSanityCheckUserPrompt(products, prompts) { - // Build a simplified product list for the prompt +export function buildSanityCheckUserPrompt(products, prompts) { const productSummaries = products.map((p, index) => ({ index, name: p.name, @@ -33,22 +20,17 @@ function buildSanityCheckUserPrompt(products, prompts) { weight: p.weight, length: p.length, width: p.width, - height: p.height + height: p.height, })); const parts = []; - - // Add general prompt/rules if provided if (prompts.general) { parts.push(prompts.general); - parts.push(''); // Empty line for separation + parts.push(''); } - - // Add products to review parts.push(`PRODUCTS TO REVIEW (${products.length} items):`); parts.push(JSON.stringify(productSummaries, null, 2)); - // Add response format parts.push(''); parts.push('RESPOND WITH JSON:'); parts.push(JSON.stringify({ @@ -57,10 +39,10 @@ function buildSanityCheckUserPrompt(products, prompts) { productIndex: 0, field: 'msrp', issue: 'Description of the issue found', - suggestion: 'Suggested fix or verification (optional)' - } + suggestion: 'Suggested fix or verification (optional)', + }, ], - summary: '2-3 sentences summarizing the overall product quality' + summary: '2-3 sentences summarizing the overall product quality', }, null, 2)); parts.push(''); @@ -69,60 +51,40 @@ function buildSanityCheckUserPrompt(products, prompts) { return parts.join('\n'); } -/** - * Parse the AI response for sanity check - * - * @param {Object|null} parsed - Parsed JSON from AI - * @param {string} content - Raw response content - * @returns {Object} - */ -function parseSanityCheckResponse(parsed, content) { - // If we got valid parsed JSON, use it +export function parseSanityCheckResponse(parsed, content) { if (parsed && Array.isArray(parsed.issues)) { return { - issues: parsed.issues.map(issue => ({ + issues: parsed.issues.map((issue) => ({ productIndex: issue.productIndex ?? issue.index ?? 0, field: issue.field || 'unknown', issue: issue.issue || issue.message || '', - suggestion: issue.suggestion || null + suggestion: issue.suggestion || null, })), - summary: parsed.summary || 'Review complete' + summary: parsed.summary || 'Review complete', }; } - // Try to extract from content if parsing failed try { - // Try to find issues array const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/); let issues = []; - if (issuesMatch) { - // Try to parse the array content try { const arrayContent = `[${issuesMatch[1]}]`; const parsedIssues = JSON.parse(arrayContent); - issues = parsedIssues.map(issue => ({ + issues = parsedIssues.map((issue) => ({ productIndex: issue.productIndex ?? issue.index ?? 0, field: issue.field || 'unknown', issue: issue.issue || issue.message || '', - suggestion: issue.suggestion || null + suggestion: issue.suggestion || null, })); } catch { - // Couldn't parse the array + /* fall through */ } } - - // Try to find summary const summaryMatch = content.match(/"summary"\s*:\s*"([^"]+)"/); const summary = summaryMatch ? summaryMatch[1] : 'Review complete'; - return { issues, summary }; } catch { return { issues: [], summary: 'Could not parse review results' }; } } - -module.exports = { - buildSanityCheckUserPrompt, - parseSanityCheckResponse -}; diff --git a/inventory-server/src/services/ai/providers/groqProvider.js b/inventory-server/src/services/ai/providers/groqProvider.js index a4232c3..8420e5b 100644 --- a/inventory-server/src/services/ai/providers/groqProvider.js +++ b/inventory-server/src/services/ai/providers/groqProvider.js @@ -1,25 +1,15 @@ /** - * Groq Provider - Handles chat completions via Groq's OpenAI-compatible API - * - * Uses Groq's fast inference for real-time AI validation tasks. - * Supports models like openai/gpt-oss-120b (complex) and openai/gpt-oss-20b (simple). + * Groq Provider - chat completions via Groq's OpenAI-compatible API */ -const GROQ_BASE_URL = 'https://api.groq.com/openai/v1'; +export const GROQ_BASE_URL = 'https://api.groq.com/openai/v1'; -// Default models -const MODELS = { - LARGE: 'openai/gpt-oss-120b', // For complex tasks (descriptions, sanity checks) - SMALL: 'openai/gpt-oss-20b' // For simple tasks (name validation) +export const MODELS = { + LARGE: 'openai/gpt-oss-120b', + SMALL: 'openai/gpt-oss-20b', }; -class GroqProvider { - /** - * @param {Object} options - * @param {string} options.apiKey - Groq API key - * @param {string} [options.baseUrl] - Override base URL - * @param {number} [options.timeoutMs=30000] - Default timeout - */ +export class GroqProvider { constructor({ apiKey, baseUrl = GROQ_BASE_URL, timeoutMs = 30000 }) { if (!apiKey) { throw new Error('Groq API key is required'); @@ -29,41 +19,25 @@ class GroqProvider { this.timeoutMs = timeoutMs; } - /** - * Send a chat completion request - * - * @param {Object} params - * @param {Array<{role: string, content: string}>} params.messages - Conversation messages - * @param {string} [params.model] - Model to use (defaults to LARGE) - * @param {number} [params.temperature=0.3] - Response randomness (0-2) - * @param {number} [params.maxTokens=500] - Max tokens in response - * @param {Object} [params.responseFormat] - For JSON mode: { type: 'json_object' } - * @param {number} [params.timeoutMs] - Request timeout override - * @returns {Promise<{content: string, parsed: Object|null, usage: Object, latencyMs: number, model: string}>} - */ async chatCompletion({ messages, model = MODELS.LARGE, temperature = 0.3, maxTokens = 500, responseFormat = null, - timeoutMs = this.timeoutMs + timeoutMs = this.timeoutMs, }) { const started = Date.now(); - const body = { model, messages, temperature, - max_completion_tokens: maxTokens + max_completion_tokens: maxTokens, }; - - // Enable JSON mode if requested if (responseFormat?.type === 'json_object') { body.response_format = { type: 'json_object' }; } - // Debug: Log request being sent console.log('[Groq] Request:', { model: body.model, temperature: body.temperature, @@ -71,12 +45,11 @@ class GroqProvider { hasResponseFormat: !!body.response_format, messageCount: body.messages?.length, systemPromptLength: body.messages?.[0]?.content?.length, - userPromptLength: body.messages?.[1]?.content?.length + userPromptLength: body.messages?.[1]?.content?.length, }); const response = await this._makeRequest('chat/completions', body, timeoutMs); - // Debug: Log raw response structure console.log('[Groq] Raw response:', { hasChoices: !!response.choices, choicesLength: response.choices?.length, @@ -84,22 +57,20 @@ class GroqProvider { finishReason: response.choices[0].finish_reason, hasMessage: !!response.choices[0].message, contentLength: response.choices[0].message?.content?.length, - contentPreview: response.choices[0].message?.content?.substring(0, 200) + contentPreview: response.choices[0].message?.content?.substring(0, 200), } : null, usage: response.usage, - model: response.model + model: response.model, }); const content = response.choices?.[0]?.message?.content || ''; const usage = response.usage || {}; - // Attempt to parse JSON if response format was requested let parsed = null; if (responseFormat && content) { try { parsed = JSON.parse(content); } catch { - // Content isn't valid JSON - try to extract JSON from markdown parsed = this._extractJson(content); } } @@ -110,74 +81,50 @@ class GroqProvider { usage: { promptTokens: usage.prompt_tokens || 0, completionTokens: usage.completion_tokens || 0, - totalTokens: usage.total_tokens || 0 + totalTokens: usage.total_tokens || 0, }, latencyMs: Date.now() - started, - model: response.model || model + model: response.model || model, }; } - /** - * Extract JSON from content that might be wrapped in markdown code blocks - * @private - */ _extractJson(content) { - // Try to find JSON in code blocks const codeBlockMatch = content.match(/```(?:json)?\s*([\s\S]*?)```/); if (codeBlockMatch) { - try { - return JSON.parse(codeBlockMatch[1].trim()); - } catch { - // Fall through - } + try { return JSON.parse(codeBlockMatch[1].trim()); } catch { /* fall through */ } } - - // Try to find JSON object/array directly const jsonMatch = content.match(/(\{[\s\S]*\}|\[[\s\S]*\])/); if (jsonMatch) { - try { - return JSON.parse(jsonMatch[1]); - } catch { - // Fall through - } + try { return JSON.parse(jsonMatch[1]); } catch { /* fall through */ } } - return null; } - /** - * Make an HTTP request to Groq API - * @private - */ async _makeRequest(endpoint, body, timeoutMs) { const controller = new AbortController(); const timeout = setTimeout(() => controller.abort(), timeoutMs); - try { const response = await fetch(`${this.baseUrl}/${endpoint}`, { method: 'POST', headers: { 'Content-Type': 'application/json', - 'Authorization': `Bearer ${this.apiKey}` + 'Authorization': `Bearer ${this.apiKey}`, }, body: JSON.stringify(body), - signal: controller.signal + signal: controller.signal, }); - if (!response.ok) { const error = await response.json().catch(() => ({})); const message = error.error?.message || `Groq API error: ${response.status}`; const err = new Error(message); err.status = response.status; err.code = error.error?.code; - // Include failed_generation if available (for JSON mode failures) if (error.error?.failed_generation) { err.failedGeneration = error.error.failed_generation; console.error('[Groq] JSON validation failed. Model output:', error.error.failed_generation); } throw err; } - return response.json(); } catch (error) { if (error.name === 'AbortError') { @@ -191,13 +138,7 @@ class GroqProvider { } } - /** - * Check if the provider is properly configured - * @returns {boolean} - */ isConfigured() { return !!this.apiKey; } } - -module.exports = { GroqProvider, MODELS, GROQ_BASE_URL }; diff --git a/inventory-server/src/services/ai/providers/openaiProvider.js b/inventory-server/src/services/ai/providers/openaiProvider.js index c12902a..e232341 100644 --- a/inventory-server/src/services/ai/providers/openaiProvider.js +++ b/inventory-server/src/services/ai/providers/openaiProvider.js @@ -2,11 +2,11 @@ * OpenAI Provider - Handles embedding generation */ -const EMBEDDING_MODEL = 'text-embedding-3-small'; -const EMBEDDING_DIMENSIONS = 1536; +export const EMBEDDING_MODEL = 'text-embedding-3-small'; +export const EMBEDDING_DIMENSIONS = 1536; const MAX_BATCH_SIZE = 2048; -class OpenAIProvider { +export class OpenAIProvider { constructor({ apiKey, baseUrl = 'https://api.openai.com/v1', timeoutMs = 60000 }) { if (!apiKey) { throw new Error('OpenAI API key is required'); @@ -16,12 +16,6 @@ class OpenAIProvider { this.timeoutMs = timeoutMs; } - /** - * Generate embeddings for one or more texts - * @param {string|string[]} input - Text or array of texts - * @param {Object} options - * @returns {Promise<{embeddings: number[][], usage: Object, model: string, latencyMs: number}>} - */ async embed(input, options = {}) { const texts = Array.isArray(input) ? input : [input]; const model = options.model || EMBEDDING_MODEL; @@ -33,56 +27,39 @@ class OpenAIProvider { } const started = Date.now(); - - // Clean and truncate input texts - const cleanedTexts = texts.map(t => + const cleanedTexts = texts.map((t) => (t || '').replace(/\n+/g, ' ').trim().substring(0, 8000) ); - const body = { - input: cleanedTexts, - model, - encoding_format: 'float' - }; - - // Only embedding-3 models support dimensions parameter - if (model.includes('embedding-3')) { - body.dimensions = dimensions; - } + const body = { input: cleanedTexts, model, encoding_format: 'float' }; + if (model.includes('embedding-3')) body.dimensions = dimensions; const response = await this._makeRequest('embeddings', body, timeoutMs); - - // Sort by index to ensure order matches input const sortedData = response.data.sort((a, b) => a.index - b.index); return { - embeddings: sortedData.map(item => item.embedding), + embeddings: sortedData.map((item) => item.embedding), usage: { promptTokens: response.usage?.prompt_tokens || 0, - totalTokens: response.usage?.total_tokens || 0 + totalTokens: response.usage?.total_tokens || 0, }, model: response.model || model, - latencyMs: Date.now() - started + latencyMs: Date.now() - started, }; } - /** - * Generator for processing large batches in chunks - */ async *embedBatchChunked(texts, options = {}) { const batchSize = Math.min(options.batchSize || 100, MAX_BATCH_SIZE); - for (let i = 0; i < texts.length; i += batchSize) { const chunk = texts.slice(i, i + batchSize); const result = await this.embed(chunk, options); - yield { embeddings: result.embeddings, startIndex: i, endIndex: i + chunk.length, usage: result.usage, model: result.model, - latencyMs: result.latencyMs + latencyMs: result.latencyMs, }; } } @@ -90,28 +67,23 @@ class OpenAIProvider { async _makeRequest(endpoint, body, timeoutMs) { const controller = new AbortController(); const timeout = setTimeout(() => controller.abort(), timeoutMs); - try { const response = await fetch(`${this.baseUrl}/${endpoint}`, { method: 'POST', headers: { 'Content-Type': 'application/json', - 'Authorization': `Bearer ${this.apiKey}` + 'Authorization': `Bearer ${this.apiKey}`, }, body: JSON.stringify(body), - signal: controller.signal + signal: controller.signal, }); - if (!response.ok) { const error = await response.json().catch(() => ({})); throw new Error(error.error?.message || `OpenAI API error: ${response.status}`); } - return response.json(); } finally { clearTimeout(timeout); } } } - -module.exports = { OpenAIProvider, EMBEDDING_MODEL, EMBEDDING_DIMENSIONS }; diff --git a/inventory-server/src/services/ai/tasks/descriptionValidationTask.js b/inventory-server/src/services/ai/tasks/descriptionValidationTask.js index b56c1ec..26d61a3 100644 --- a/inventory-server/src/services/ai/tasks/descriptionValidationTask.js +++ b/inventory-server/src/services/ai/tasks/descriptionValidationTask.js @@ -1,132 +1,84 @@ /** * Description Validation Task - * - * Validates a product description for quality, accuracy, and guideline compliance. - * Uses Groq with the larger model for better reasoning about content quality. - * Loads all prompts from the database (no hardcoded prompts). */ -const { MODELS } = require('../providers/groqProvider'); -const { +import { MODELS } from '../providers/groqProvider.js'; +import { loadDescriptionValidationPrompts, - validateRequiredPrompts -} = require('../prompts/promptLoader'); -const { + validateRequiredPrompts, +} from '../prompts/promptLoader.js'; +import { buildDescriptionUserPrompt, - parseDescriptionResponse -} = require('../prompts/descriptionPrompts'); + parseDescriptionResponse, +} from '../prompts/descriptionPrompts.js'; -const TASK_ID = 'validate.description'; +export const TASK_ID = 'validate.description'; -/** - * Create the description validation task - * - * @returns {Object} Task definition - */ -function createDescriptionValidationTask() { +export function createDescriptionValidationTask() { return { id: TASK_ID, description: 'Validate product description for quality and guideline compliance', - /** - * Run the description validation - * - * @param {Object} payload - * @param {Object} payload.product - Product data - * @param {string} payload.product.name - Product name (for context) - * @param {string} payload.product.description - Description to validate - * @param {string} [payload.product.company_name] - Company name - * @param {string} [payload.product.company_id] - Company ID for loading specific rules - * @param {string} [payload.product.categories] - Product categories - * @param {Object} payload.provider - Groq provider instance - * @param {Object} payload.pool - PostgreSQL pool - * @param {Object} [payload.logger] - Logger instance - * @returns {Promise} - */ async run(payload) { const { product, provider, pool, logger } = payload; const log = logger || console; - // Validate required input if (!product?.name && !product?.description) { - return { - isValid: true, - suggestion: null, - issues: [], - skipped: true, - reason: 'No name or description provided' - }; - } - - if (!provider) { - throw new Error('Groq provider not available'); - } - - if (!pool) { - throw new Error('Database pool not available'); + return { isValid: true, suggestion: null, issues: [], skipped: true, reason: 'No name or description provided' }; } + if (!provider) throw new Error('Groq provider not available'); + if (!pool) throw new Error('Database pool not available'); try { - // Load prompts from database const companyKey = product.company_id || product.company_name || product.company; const prompts = await loadDescriptionValidationPrompts(pool, companyKey); - - // Validate required prompts exist validateRequiredPrompts(prompts, 'description_validation'); - // Build the user prompt with database-loaded prompts const userPrompt = buildDescriptionUserPrompt(product, prompts); let response; let result; - try { - // Try with JSON mode first response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], - model: MODELS.LARGE, // openai/gpt-oss-120b - better for content analysis - temperature: 0.3, // Slightly higher for creative suggestions - maxTokens: 2000, // Reasoning models need extra tokens for thinking - responseFormat: { type: 'json_object' } + model: MODELS.LARGE, + temperature: 0.3, + maxTokens: 2000, + responseFormat: { type: 'json_object' }, }); - // Log full raw response for debugging log.info('[DescriptionValidation] Raw AI response:', { parsed: response.parsed, content: response.content, - contentLength: response.content?.length + contentLength: response.content?.length, }); - // Parse the response result = parseDescriptionResponse(response.parsed, response.content); } catch (jsonError) { - // If JSON mode failed, check if we have failedGeneration to parse if (jsonError.failedGeneration) { log.warn('[DescriptionValidation] JSON mode failed, attempting to parse failed_generation:', { - failedGeneration: jsonError.failedGeneration + failedGeneration: jsonError.failedGeneration, }); result = parseDescriptionResponse(null, jsonError.failedGeneration); response = { latencyMs: 0, usage: {}, model: MODELS.LARGE }; } else { - // Retry without JSON mode log.warn('[DescriptionValidation] JSON mode failed, retrying without JSON mode'); response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], model: MODELS.LARGE, temperature: 0.3, - maxTokens: 2000 // Reasoning models need extra tokens for thinking - // No responseFormat - let the model respond freely + maxTokens: 2000, }); log.info('[DescriptionValidation] Raw AI response (no JSON mode):', { parsed: response.parsed, content: response.content, - contentLength: response.content?.length + contentLength: response.content?.length, }); result = parseDescriptionResponse(response.parsed, response.content); } @@ -135,24 +87,19 @@ function createDescriptionValidationTask() { log.info(`[DescriptionValidation] Validated description for "${product.name}" in ${response.latencyMs}ms`, { isValid: result.isValid, hasSuggestion: !!result.suggestion, - issueCount: result.issues.length + issueCount: result.issues.length, }); return { ...result, latencyMs: response.latencyMs, usage: response.usage, - model: response.model + model: response.model, }; } catch (error) { log.error('[DescriptionValidation] Error:', error.message); throw error; } - } + }, }; } - -module.exports = { - TASK_ID, - createDescriptionValidationTask -}; diff --git a/inventory-server/src/services/ai/tasks/index.js b/inventory-server/src/services/ai/tasks/index.js index 199d69b..d18f863 100644 --- a/inventory-server/src/services/ai/tasks/index.js +++ b/inventory-server/src/services/ai/tasks/index.js @@ -1,166 +1,87 @@ /** * AI Task Registry - * - * Simple registry pattern for AI tasks. Each task has: - * - id: Unique identifier - * - run: Async function that executes the task - * - * This allows adding new AI capabilities without modifying core code. */ -const { createNameValidationTask, TASK_ID: NAME_TASK_ID } = require('./nameValidationTask'); -const { createDescriptionValidationTask, TASK_ID: DESC_TASK_ID } = require('./descriptionValidationTask'); -const { createSanityCheckTask, TASK_ID: SANITY_TASK_ID } = require('./sanityCheckTask'); +import { createNameValidationTask, TASK_ID as NAME_TASK_ID } from './nameValidationTask.js'; +import { createDescriptionValidationTask, TASK_ID as DESC_TASK_ID } from './descriptionValidationTask.js'; +import { createSanityCheckTask, TASK_ID as SANITY_TASK_ID } from './sanityCheckTask.js'; -/** - * Task IDs - frozen constants for type safety - */ -const TASK_IDS = Object.freeze({ - // Inline validation (triggered on field blur) +export { createNameValidationTask, createDescriptionValidationTask, createSanityCheckTask }; + +export const TASK_IDS = Object.freeze({ VALIDATE_NAME: NAME_TASK_ID, VALIDATE_DESCRIPTION: DESC_TASK_ID, - - // Batch operations (triggered on user action) - SANITY_CHECK: SANITY_TASK_ID + SANITY_CHECK: SANITY_TASK_ID, }); -/** - * Task Registry - */ -class TaskRegistry { +export class TaskRegistry { constructor() { this.tasks = new Map(); } - /** - * Register a task - * @param {Object} task - * @param {string} task.id - Unique task identifier - * @param {Function} task.run - Async function: (payload) => result - * @param {string} [task.description] - Human-readable description - */ register(task) { - if (!task?.id) { - throw new Error('Task must have an id'); - } - if (typeof task.run !== 'function') { - throw new Error(`Task ${task.id} must have a run function`); - } - if (this.tasks.has(task.id)) { - throw new Error(`Task ${task.id} is already registered`); - } - + if (!task?.id) throw new Error('Task must have an id'); + if (typeof task.run !== 'function') throw new Error(`Task ${task.id} must have a run function`); + if (this.tasks.has(task.id)) throw new Error(`Task ${task.id} is already registered`); this.tasks.set(task.id, task); return this; } - /** - * Get a task by ID - * @param {string} taskId - * @returns {Object|null} - */ get(taskId) { return this.tasks.get(taskId) || null; } - /** - * Check if a task exists - * @param {string} taskId - * @returns {boolean} - */ has(taskId) { return this.tasks.has(taskId); } - /** - * Run a task by ID - * @param {string} taskId - * @param {Object} payload - Task-specific input - * @returns {Promise} Task result - */ async runTask(taskId, payload = {}) { const task = this.get(taskId); - if (!task) { - throw new Error(`Unknown task: ${taskId}`); - } - + if (!task) throw new Error(`Unknown task: ${taskId}`); try { const result = await task.run(payload); - return { - success: true, - taskId, - ...result - }; + return { success: true, taskId, ...result }; } catch (error) { return { success: false, taskId, error: error.message, - code: error.code + code: error.code, }; } } - /** - * List all registered task IDs - * @returns {string[]} - */ list() { return Array.from(this.tasks.keys()); } - /** - * Get count of registered tasks - * @returns {number} - */ size() { return this.tasks.size; } } -// Singleton instance let registry = null; -/** - * Get or create the task registry - * @returns {TaskRegistry} - */ -function getRegistry() { - if (!registry) { - registry = new TaskRegistry(); - } +export function getRegistry() { + if (!registry) registry = new TaskRegistry(); return registry; } -/** - * Reset the registry (mainly for testing) - */ -function resetRegistry() { +export function resetRegistry() { registry = null; } -/** - * Register all validation tasks with the registry - * Call this during initialization after the registry is created - * - * @param {Object} [logger] - Optional logger - */ -function registerAllTasks(logger = console) { +export function registerAllTasks(logger = console) { const reg = getRegistry(); - // Register name validation if (!reg.has(TASK_IDS.VALIDATE_NAME)) { reg.register(createNameValidationTask()); logger.info(`[Tasks] Registered: ${TASK_IDS.VALIDATE_NAME}`); } - - // Register description validation if (!reg.has(TASK_IDS.VALIDATE_DESCRIPTION)) { reg.register(createDescriptionValidationTask()); logger.info(`[Tasks] Registered: ${TASK_IDS.VALIDATE_DESCRIPTION}`); } - - // Register sanity check if (!reg.has(TASK_IDS.SANITY_CHECK)) { reg.register(createSanityCheckTask()); logger.info(`[Tasks] Registered: ${TASK_IDS.SANITY_CHECK}`); @@ -168,19 +89,3 @@ function registerAllTasks(logger = console) { return reg; } - -module.exports = { - // Constants - TASK_IDS, - - // Registry - TaskRegistry, - getRegistry, - resetRegistry, - registerAllTasks, - - // Task factories (for custom registration) - createNameValidationTask, - createDescriptionValidationTask, - createSanityCheckTask -}; diff --git a/inventory-server/src/services/ai/tasks/nameValidationTask.js b/inventory-server/src/services/ai/tasks/nameValidationTask.js index f949a3c..c36b182 100644 --- a/inventory-server/src/services/ai/tasks/nameValidationTask.js +++ b/inventory-server/src/services/ai/tasks/nameValidationTask.js @@ -1,77 +1,38 @@ /** * Name Validation Task - * - * Validates a product name for spelling, grammar, and naming conventions. - * Uses Groq with the smaller model for fast response times. - * Loads all prompts from the database (no hardcoded prompts). */ -const { MODELS } = require('../providers/groqProvider'); -const { +import { MODELS } from '../providers/groqProvider.js'; +import { loadNameValidationPrompts, - validateRequiredPrompts -} = require('../prompts/promptLoader'); -const { + validateRequiredPrompts, +} from '../prompts/promptLoader.js'; +import { buildNameUserPrompt, - parseNameResponse -} = require('../prompts/namePrompts'); + parseNameResponse, +} from '../prompts/namePrompts.js'; -const TASK_ID = 'validate.name'; +export const TASK_ID = 'validate.name'; -/** - * Create the name validation task - * - * @returns {Object} Task definition - */ -function createNameValidationTask() { +export function createNameValidationTask() { return { id: TASK_ID, description: 'Validate product name for spelling, grammar, and conventions', - /** - * Run the name validation - * - * @param {Object} payload - * @param {Object} payload.product - Product data - * @param {string} payload.product.name - Product name to validate - * @param {string} [payload.product.company_name] - Company name - * @param {string} [payload.product.company_id] - Company ID for loading specific rules - * @param {string} [payload.product.line_name] - Product line - * @param {string} [payload.product.description] - Description for context - * @param {Object} payload.provider - Groq provider instance - * @param {Object} payload.pool - PostgreSQL pool - * @param {Object} [payload.logger] - Logger instance - * @returns {Promise} - */ async run(payload) { const { product, provider, pool, logger } = payload; const log = logger || console; - // Validate required input if (!product?.name) { - return { - isValid: true, - suggestion: null, - issues: [], - skipped: true, - reason: 'No name provided' - }; - } - - if (!provider) { - throw new Error('Groq provider not available'); - } - - if (!pool) { - throw new Error('Database pool not available'); + return { isValid: true, suggestion: null, issues: [], skipped: true, reason: 'No name provided' }; } + if (!provider) throw new Error('Groq provider not available'); + if (!pool) throw new Error('Database pool not available'); try { - // Load prompts from database const companyKey = product.company_id || product.company_name || product.company; const prompts = await loadNameValidationPrompts(pool, companyKey); - // Debug: Log loaded prompts log.info('[NameValidation] Loaded prompts:', { hasSystem: !!prompts.system, systemLength: prompts.system?.length || 0, @@ -79,68 +40,57 @@ function createNameValidationTask() { generalLength: prompts.general?.length || 0, generalPreview: prompts.general?.substring(0, 100) || '(empty)', hasCompanySpecific: !!prompts.companySpecific, - companyKey + companyKey, }); - // Validate required prompts exist validateRequiredPrompts(prompts, 'name_validation'); - // Build the user prompt with database-loaded prompts const userPrompt = buildNameUserPrompt(product, prompts); - - // Debug: Log the full user prompt being sent log.info('[NameValidation] User prompt:', userPrompt.substring(0, 500)); let response; let result; - try { - // Try with JSON mode first response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], - model: MODELS.LARGE, // openai/gpt-oss-120b - reasoning model - temperature: 0.2, // Low temperature for consistent results - maxTokens: 3000, // Reasoning models need extra tokens for thinking - responseFormat: { type: 'json_object' } + model: MODELS.LARGE, + temperature: 0.2, + maxTokens: 3000, + responseFormat: { type: 'json_object' }, }); - // Log full raw response for debugging log.info('[NameValidation] Raw AI response:', { parsed: response.parsed, content: response.content, - contentLength: response.content?.length + contentLength: response.content?.length, }); - // Parse the response result = parseNameResponse(response.parsed, response.content); } catch (jsonError) { - // If JSON mode failed, check if we have failedGeneration to parse if (jsonError.failedGeneration) { log.warn('[NameValidation] JSON mode failed, attempting to parse failed_generation:', { - failedGeneration: jsonError.failedGeneration + failedGeneration: jsonError.failedGeneration, }); result = parseNameResponse(null, jsonError.failedGeneration); response = { latencyMs: 0, usage: {}, model: MODELS.SMALL }; } else { - // Retry without JSON mode log.warn('[NameValidation] JSON mode failed, retrying without JSON mode'); response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], model: MODELS.SMALL, temperature: 0.2, - maxTokens: 1500 // Reasoning models need extra tokens for thinking - // No responseFormat - let the model respond freely + maxTokens: 1500, }); log.info('[NameValidation] Raw AI response (no JSON mode):', { parsed: response.parsed, content: response.content, - contentLength: response.content?.length + contentLength: response.content?.length, }); result = parseNameResponse(response.parsed, response.content); } @@ -149,24 +99,19 @@ function createNameValidationTask() { log.info(`[NameValidation] Validated "${product.name}" in ${response.latencyMs}ms`, { isValid: result.isValid, hassuggestion: !!result.suggestion, - issueCount: result.issues.length + issueCount: result.issues.length, }); return { ...result, latencyMs: response.latencyMs, usage: response.usage, - model: response.model + model: response.model, }; } catch (error) { log.error('[NameValidation] Error:', error.message); throw error; } - } + }, }; } - -module.exports = { - TASK_ID, - createNameValidationTask -}; diff --git a/inventory-server/src/services/ai/tasks/sanityCheckTask.js b/inventory-server/src/services/ai/tasks/sanityCheckTask.js index c715976..9880386 100644 --- a/inventory-server/src/services/ai/tasks/sanityCheckTask.js +++ b/inventory-server/src/services/ai/tasks/sanityCheckTask.js @@ -1,96 +1,55 @@ /** * Sanity Check Task - * - * Reviews a batch of products for consistency and appropriateness. - * Uses Groq with the larger model for complex batch analysis. - * Loads all prompts from the database (no hardcoded prompts). */ -const { MODELS } = require('../providers/groqProvider'); -const { +import { MODELS } from '../providers/groqProvider.js'; +import { loadSanityCheckPrompts, - validateRequiredPrompts -} = require('../prompts/promptLoader'); -const { + validateRequiredPrompts, +} from '../prompts/promptLoader.js'; +import { buildSanityCheckUserPrompt, - parseSanityCheckResponse -} = require('../prompts/sanityCheckPrompts'); + parseSanityCheckResponse, +} from '../prompts/sanityCheckPrompts.js'; -const TASK_ID = 'sanity.check'; +export const TASK_ID = 'sanity.check'; +export const MAX_PRODUCTS_PER_REQUEST = 50; -// Maximum products to send in a single request (to avoid token limits) -const MAX_PRODUCTS_PER_REQUEST = 50; - -/** - * Create the sanity check task - * - * @returns {Object} Task definition - */ -function createSanityCheckTask() { +export function createSanityCheckTask() { return { id: TASK_ID, description: 'Review batch of products for consistency and appropriateness', - /** - * Run the sanity check - * - * @param {Object} payload - * @param {Object[]} payload.products - Array of products to check - * @param {Object} payload.provider - Groq provider instance - * @param {Object} payload.pool - PostgreSQL pool - * @param {Object} [payload.logger] - Logger instance - * @returns {Promise} - */ async run(payload) { const { products, provider, pool, logger } = payload; const log = logger || console; - // Validate required input if (!Array.isArray(products) || products.length === 0) { - return { - issues: [], - summary: 'No products to check', - skipped: true - }; - } - - if (!provider) { - throw new Error('Groq provider not available'); - } - - if (!pool) { - throw new Error('Database pool not available'); + return { issues: [], summary: 'No products to check', skipped: true }; } + if (!provider) throw new Error('Groq provider not available'); + if (!pool) throw new Error('Database pool not available'); try { - // Load prompts from database const prompts = await loadSanityCheckPrompts(pool); - - // Validate required prompts exist validateRequiredPrompts(prompts, 'sanity_check'); - // If batch is small enough, process in one request if (products.length <= MAX_PRODUCTS_PER_REQUEST) { return await checkBatch(products, prompts, provider, log); } - // Otherwise, process in chunks and combine results log.info(`[SanityCheck] Processing ${products.length} products in chunks`); const allIssues = []; const summaries = []; for (let i = 0; i < products.length; i += MAX_PRODUCTS_PER_REQUEST) { const chunk = products.slice(i, i + MAX_PRODUCTS_PER_REQUEST); - const chunkOffset = i; // To adjust product indices in results - + const chunkOffset = i; const result = await checkBatch(chunk, prompts, provider, log); - - // Adjust product indices to match original array - const adjustedIssues = result.issues.map(issue => ({ + const adjustedIssues = result.issues.map((issue) => ({ ...issue, - productIndex: issue.productIndex + chunkOffset + productIndex: issue.productIndex + chunkOffset, })); - allIssues.push(...adjustedIssues); summaries.push(result.summary); } @@ -101,82 +60,61 @@ function createSanityCheckTask() { ? `Reviewed ${products.length} products in ${summaries.length} batches. ${allIssues.length} issues found.` : summaries[0], totalProducts: products.length, - issueCount: allIssues.length + issueCount: allIssues.length, }; } catch (error) { log.error('[SanityCheck] Error:', error.message); throw error; } - } + }, }; } -/** - * Check a single batch of products - * - * @param {Object[]} products - Products to check - * @param {Object} prompts - Loaded prompts from database - * @param {Object} provider - Groq provider - * @param {Object} log - Logger - * @returns {Promise} - */ async function checkBatch(products, prompts, provider, log) { const userPrompt = buildSanityCheckUserPrompt(products, prompts); - let response; let result; try { - // Try with JSON mode first response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], - model: MODELS.LARGE, // openai/gpt-oss-120b - needed for complex batch analysis - temperature: 0.2, // Low temperature for consistent analysis - maxTokens: 2000, // More tokens for batch results - responseFormat: { type: 'json_object' } + model: MODELS.LARGE, + temperature: 0.2, + maxTokens: 2000, + responseFormat: { type: 'json_object' }, }); - result = parseSanityCheckResponse(response.parsed, response.content); } catch (jsonError) { - // If JSON mode failed, check if we have failedGeneration to parse if (jsonError.failedGeneration) { log.warn('[SanityCheck] JSON mode failed, attempting to parse failed_generation'); result = parseSanityCheckResponse(null, jsonError.failedGeneration); response = { latencyMs: 0, usage: {}, model: MODELS.LARGE }; } else { - // Retry without JSON mode log.warn('[SanityCheck] JSON mode failed, retrying without JSON mode'); response = await provider.chatCompletion({ messages: [ { role: 'system', content: prompts.system }, - { role: 'user', content: userPrompt } + { role: 'user', content: userPrompt }, ], model: MODELS.LARGE, temperature: 0.2, - maxTokens: 2000 - // No responseFormat - let the model respond freely + maxTokens: 2000, }); result = parseSanityCheckResponse(response.parsed, response.content); } } log.info(`[SanityCheck] Checked ${products.length} products in ${response.latencyMs}ms`, { - issueCount: result.issues.length + issueCount: result.issues.length, }); return { ...result, latencyMs: response.latencyMs, usage: response.usage, - model: response.model + model: response.model, }; } - -module.exports = { - TASK_ID, - createSanityCheckTask, - MAX_PRODUCTS_PER_REQUEST -}; diff --git a/inventory-server/src/types/status-codes.js b/inventory-server/src/types/status-codes.js index 8c94f85..aed5baa 100644 --- a/inventory-server/src/types/status-codes.js +++ b/inventory-server/src/types/status-codes.js @@ -1,79 +1,63 @@ // Purchase Order Status Codes -const PurchaseOrderStatus = { - Canceled: 0, - Created: 1, - ElectronicallyReadySend: 10, - Ordered: 11, - Preordered: 12, - ElectronicallySent: 13, - ReceivingStarted: 15, - Done: 50 +export const PurchaseOrderStatus = { + Canceled: 0, + Created: 1, + ElectronicallyReadySend: 10, + Ordered: 11, + Preordered: 12, + ElectronicallySent: 13, + ReceivingStarted: 15, + Done: 50, }; // Receiving Status Codes -const ReceivingStatus = { - Canceled: 0, - Created: 1, - PartialReceived: 30, - FullReceived: 40, - Paid: 50 +export const ReceivingStatus = { + Canceled: 0, + Created: 1, + PartialReceived: 30, + FullReceived: 40, + Paid: 50, }; -// Status Code Display Names -const PurchaseOrderStatusLabels = { - [PurchaseOrderStatus.Canceled]: 'Canceled', - [PurchaseOrderStatus.Created]: 'Created', - [PurchaseOrderStatus.ElectronicallyReadySend]: 'Ready to Send', - [PurchaseOrderStatus.Ordered]: 'Ordered', - [PurchaseOrderStatus.Preordered]: 'Preordered', - [PurchaseOrderStatus.ElectronicallySent]: 'Sent', - [PurchaseOrderStatus.ReceivingStarted]: 'Receiving Started', - [PurchaseOrderStatus.Done]: 'Done' +export const PurchaseOrderStatusLabels = { + [PurchaseOrderStatus.Canceled]: 'Canceled', + [PurchaseOrderStatus.Created]: 'Created', + [PurchaseOrderStatus.ElectronicallyReadySend]: 'Ready to Send', + [PurchaseOrderStatus.Ordered]: 'Ordered', + [PurchaseOrderStatus.Preordered]: 'Preordered', + [PurchaseOrderStatus.ElectronicallySent]: 'Sent', + [PurchaseOrderStatus.ReceivingStarted]: 'Receiving Started', + [PurchaseOrderStatus.Done]: 'Done', }; -const ReceivingStatusLabels = { - [ReceivingStatus.Canceled]: 'Canceled', - [ReceivingStatus.Created]: 'Created', - [ReceivingStatus.PartialReceived]: 'Partially Received', - [ReceivingStatus.FullReceived]: 'Fully Received', - [ReceivingStatus.Paid]: 'Paid' +export const ReceivingStatusLabels = { + [ReceivingStatus.Canceled]: 'Canceled', + [ReceivingStatus.Created]: 'Created', + [ReceivingStatus.PartialReceived]: 'Partially Received', + [ReceivingStatus.FullReceived]: 'Fully Received', + [ReceivingStatus.Paid]: 'Paid', }; -// Helper functions -function getPurchaseOrderStatusLabel(status) { - return PurchaseOrderStatusLabels[status] || 'Unknown'; +export function getPurchaseOrderStatusLabel(status) { + return PurchaseOrderStatusLabels[status] || 'Unknown'; } -function getReceivingStatusLabel(status) { - return ReceivingStatusLabels[status] || 'Unknown'; +export function getReceivingStatusLabel(status) { + return ReceivingStatusLabels[status] || 'Unknown'; } -// Status checks -function isReceivingComplete(status) { - return status >= ReceivingStatus.PartialReceived; +export function isReceivingComplete(status) { + return status >= ReceivingStatus.PartialReceived; } -function isPurchaseOrderComplete(status) { - return status === PurchaseOrderStatus.Done; +export function isPurchaseOrderComplete(status) { + return status === PurchaseOrderStatus.Done; } -function isPurchaseOrderCanceled(status) { - return status === PurchaseOrderStatus.Canceled; +export function isPurchaseOrderCanceled(status) { + return status === PurchaseOrderStatus.Canceled; } -function isReceivingCanceled(status) { - return status === ReceivingStatus.Canceled; +export function isReceivingCanceled(status) { + return status === ReceivingStatus.Canceled; } - -module.exports = { - PurchaseOrderStatus, - ReceivingStatus, - PurchaseOrderStatusLabels, - ReceivingStatusLabels, - getPurchaseOrderStatusLabel, - getReceivingStatusLabel, - isReceivingComplete, - isPurchaseOrderComplete, - isPurchaseOrderCanceled, - isReceivingCanceled -}; \ No newline at end of file diff --git a/inventory-server/src/utils/apiHelpers.js b/inventory-server/src/utils/apiHelpers.js index 559fa29..a60d3c8 100644 --- a/inventory-server/src/utils/apiHelpers.js +++ b/inventory-server/src/utils/apiHelpers.js @@ -1,45 +1,29 @@ /** * Parses a query parameter value based on its expected type. - * Throws error for invalid formats. Adjust date handling as needed. + * Throws on invalid formats. */ -function parseValue(value, type) { - if (value === null || value === undefined || value === '') return null; +export function parseValue(value, type) { + if (value === null || value === undefined || value === '') return null; - console.log(`Parsing value: "${value}" as type: "${type}"`); - - switch (type) { - case 'number': - const num = parseFloat(value); - if (isNaN(num)) { - console.error(`Invalid number format: "${value}"`); - throw new Error(`Invalid number format: "${value}"`); - } - return num; - case 'integer': // Specific type for integer IDs etc. - const int = parseInt(value, 10); - if (isNaN(int)) { - console.error(`Invalid integer format: "${value}"`); - throw new Error(`Invalid integer format: "${value}"`); - } - console.log(`Successfully parsed integer: ${int}`); - return int; - case 'boolean': - if (String(value).toLowerCase() === 'true') return true; - if (String(value).toLowerCase() === 'false') return false; - console.error(`Invalid boolean format: "${value}"`); - throw new Error(`Invalid boolean format: "${value}"`); - case 'date': - // Basic ISO date format validation (YYYY-MM-DD) - if (!String(value).match(/^\d{4}-\d{2}-\d{2}$/)) { - console.warn(`Potentially invalid date format passed: "${value}"`); - // Optionally throw an error or return null depending on strictness - // throw new Error(`Invalid date format (YYYY-MM-DD expected): "${value}"`); - } - return String(value); // Send as string, let DB handle casting/comparison - case 'string': - default: - return String(value); + switch (type) { + case 'number': { + const num = parseFloat(value); + if (Number.isNaN(num)) throw new Error(`Invalid number format: "${value}"`); + return num; } + case 'integer': { + const int = parseInt(value, 10); + if (Number.isNaN(int)) throw new Error(`Invalid integer format: "${value}"`); + return int; + } + case 'boolean': + if (String(value).toLowerCase() === 'true') return true; + if (String(value).toLowerCase() === 'false') return false; + throw new Error(`Invalid boolean format: "${value}"`); + case 'date': + return String(value); + case 'string': + default: + return String(value); + } } - -module.exports = { parseValue }; \ No newline at end of file diff --git a/inventory-server/src/utils/csvImporter.js b/inventory-server/src/utils/csvImporter.js index 6f69c71..9276d2c 100644 --- a/inventory-server/src/utils/csvImporter.js +++ b/inventory-server/src/utils/csvImporter.js @@ -1,45 +1,37 @@ -const fs = require('fs'); -const { parse } = require('csv-parse'); -const { v4: uuidv4 } = require('uuid'); +import fs from 'node:fs'; +import { parse } from 'csv-parse'; +import { v4 as uuidv4 } from 'uuid'; -async function importProductsFromCSV(filePath, pool) { +export async function importProductsFromCSV(filePath, pool) { return new Promise((resolve, reject) => { const products = []; - + fs.createReadStream(filePath) - .pipe(parse({ - columns: true, - skip_empty_lines: true - })) - .on('data', async (row) => { + .pipe(parse({ columns: true, skip_empty_lines: true })) + .on('data', (row) => { products.push({ id: uuidv4(), sku: row.sku, name: row.name, description: row.description || null, - category: row.category || null + category: row.category || null, }); }) .on('end', async () => { try { const connection = await pool.getConnection(); - try { await connection.beginTransaction(); - for (const product of products) { await connection.query( 'INSERT INTO products (id, sku, name, description, category) VALUES (?, ?, ?, ?, ?)', [product.id, product.sku, product.name, product.description, product.category] ); - - // Initialize inventory level for the product await connection.query( 'INSERT INTO inventory_levels (id, product_id, quantity) VALUES (?, ?, 0)', [uuidv4(), product.id] ); } - await connection.commit(); resolve({ imported: products.length }); } catch (error) { @@ -52,12 +44,6 @@ async function importProductsFromCSV(filePath, pool) { reject(error); } }) - .on('error', (error) => { - reject(error); - }); + .on('error', reject); }); } - -module.exports = { - importProductsFromCSV -}; \ No newline at end of file diff --git a/inventory-server/src/utils/db.js b/inventory-server/src/utils/db.js index 5ece8ba..cf95450 100644 --- a/inventory-server/src/utils/db.js +++ b/inventory-server/src/utils/db.js @@ -1,21 +1,23 @@ -const { Pool } = require('pg'); +import pg from 'pg'; + +const { Pool } = pg; let pool; -function initPool(config) { +export function initPool(config) { pool = new Pool(config); return pool; } -async function getConnection() { +export async function getConnection() { if (!pool) { throw new Error('Database pool not initialized'); } return pool.connect(); } -module.exports = { - initPool, - getConnection, - getPool: () => pool -}; \ No newline at end of file +export function getPool() { + return pool; +} + +export default { initPool, getConnection, getPool }; diff --git a/inventory-server/src/utils/dbConnection.js b/inventory-server/src/utils/dbConnection.js index 3763e1d..21a306e 100644 --- a/inventory-server/src/utils/dbConnection.js +++ b/inventory-server/src/utils/dbConnection.js @@ -1,158 +1,94 @@ -const { Client } = require('ssh2'); -const mysql = require('mysql2/promise'); -const fs = require('fs'); +import { Client } from 'ssh2'; +import mysql from 'mysql2/promise'; +import fs from 'node:fs'; -// Connection pooling and cache configuration const connectionCache = { ssh: null, dbConnection: null, lastUsed: 0, isConnecting: false, connectionPromise: null, - // Cache expiration time in milliseconds (5 minutes) expirationTime: 5 * 60 * 1000, - // Cache for query results (key: query string, value: {data, timestamp}) queryCache: new Map(), - // Cache duration for different query types in milliseconds cacheDuration: { - 'field-options': 30 * 60 * 1000, // 30 minutes for field options - 'product-lines': 10 * 60 * 1000, // 10 minutes for product lines - 'sublines': 10 * 60 * 1000, // 10 minutes for sublines - 'taxonomy': 30 * 60 * 1000, // 30 minutes for taxonomy data - 'default': 60 * 1000 // 1 minute default - } + 'field-options': 30 * 60 * 1000, + 'product-lines': 10 * 60 * 1000, + 'sublines': 10 * 60 * 1000, + 'taxonomy': 30 * 60 * 1000, + 'default': 60 * 1000, + }, }; -/** - * Get a database connection with connection pooling - * @returns {Promise<{ssh: object, connection: object}>} The SSH and database connection - */ -async function getDbConnection() { +export async function getDbConnection() { const now = Date.now(); - - // Check if we need to refresh the connection due to inactivity - const needsRefresh = !connectionCache.ssh || - !connectionCache.dbConnection || - (now - connectionCache.lastUsed > connectionCache.expirationTime); - - // If connection is still valid, update last used time and return existing connection + const needsRefresh = !connectionCache.ssh + || !connectionCache.dbConnection + || (now - connectionCache.lastUsed > connectionCache.expirationTime); + if (!needsRefresh) { connectionCache.lastUsed = now; - return { - ssh: connectionCache.ssh, - connection: connectionCache.dbConnection - }; + return { ssh: connectionCache.ssh, connection: connectionCache.dbConnection }; } - - // If another request is already establishing a connection, wait for that promise + if (connectionCache.isConnecting && connectionCache.connectionPromise) { try { await connectionCache.connectionPromise; - return { - ssh: connectionCache.ssh, - connection: connectionCache.dbConnection - }; + return { ssh: connectionCache.ssh, connection: connectionCache.dbConnection }; } catch (error) { - // If that connection attempt failed, we'll try again below console.error('Error waiting for existing connection:', error); } } - - // Close existing connections if they exist + if (connectionCache.dbConnection) { - try { - await connectionCache.dbConnection.end(); - } catch (error) { - console.error('Error closing existing database connection:', error); - } + try { await connectionCache.dbConnection.end(); } + catch (error) { console.error('Error closing existing database connection:', error); } } - if (connectionCache.ssh) { - try { - connectionCache.ssh.end(); - } catch (error) { - console.error('Error closing existing SSH connection:', error); - } + try { connectionCache.ssh.end(); } + catch (error) { console.error('Error closing existing SSH connection:', error); } } - - // Mark that we're establishing a new connection + connectionCache.isConnecting = true; - - // Create a new promise for this connection attempt - connectionCache.connectionPromise = setupSshTunnel().then(tunnel => { + connectionCache.connectionPromise = setupSshTunnel().then((tunnel) => { const { ssh, stream, dbConfig } = tunnel; - - return mysql.createConnection({ - ...dbConfig, - stream - }).then(connection => { - // Store the new connections + return mysql.createConnection({ ...dbConfig, stream }).then((connection) => { connectionCache.ssh = ssh; connectionCache.dbConnection = connection; connectionCache.lastUsed = Date.now(); connectionCache.isConnecting = false; - - return { - ssh, - connection - }; + return { ssh, connection }; }); - }).catch(error => { + }).catch((error) => { connectionCache.isConnecting = false; throw error; }); - - // Wait for the connection to be established + return connectionCache.connectionPromise; } -/** - * Get cached query results or execute query if not cached - * @param {string} cacheKey - Unique key to identify the query - * @param {string} queryType - Type of query (field-options, product-lines, etc.) - * @param {Function} queryFn - Function to execute if cache miss - * @returns {Promise} The query result - */ -async function getCachedQuery(cacheKey, queryType, queryFn) { - // Get cache duration based on query type +export async function getCachedQuery(cacheKey, queryType, queryFn) { const cacheDuration = connectionCache.cacheDuration[queryType] || connectionCache.cacheDuration.default; - - // Check if we have a valid cached result const cachedResult = connectionCache.queryCache.get(cacheKey); const now = Date.now(); - + if (cachedResult && (now - cachedResult.timestamp < cacheDuration)) { - console.log(`Cache hit for ${queryType} query: ${cacheKey}`); return cachedResult.data; } - - // No valid cache found, execute the query - console.log(`Cache miss for ${queryType} query: ${cacheKey}`); + const result = await queryFn(); - - // Cache the result - connectionCache.queryCache.set(cacheKey, { - data: result, - timestamp: now - }); - + connectionCache.queryCache.set(cacheKey, { data: result, timestamp: now }); return result; } -/** - * Setup SSH tunnel to production database - * @private - Should only be used by getDbConnection - * @returns {Promise<{ssh: object, stream: object, dbConfig: object}>} - */ async function setupSshTunnel() { const sshConfig = { host: process.env.PROD_SSH_HOST, - port: process.env.PROD_SSH_PORT || 22, + port: Number(process.env.PROD_SSH_PORT) || 22, username: process.env.PROD_SSH_USER, privateKey: process.env.PROD_SSH_KEY_PATH ? fs.readFileSync(process.env.PROD_SSH_KEY_PATH) : undefined, - compress: true + compress: true, }; const dbConfig = { @@ -160,80 +96,42 @@ async function setupSshTunnel() { user: process.env.PROD_DB_USER, password: process.env.PROD_DB_PASSWORD, database: process.env.PROD_DB_NAME, - port: process.env.PROD_DB_PORT || 3306, - timezone: 'Z' + port: Number(process.env.PROD_DB_PORT) || 3306, + timezone: 'Z', }; return new Promise((resolve, reject) => { const ssh = new Client(); - ssh.on('error', (err) => { console.error('SSH connection error:', err); reject(err); }); - ssh.on('ready', () => { - ssh.forwardOut( - '127.0.0.1', - 0, - dbConfig.host, - dbConfig.port, - (err, stream) => { - if (err) reject(err); - resolve({ ssh, stream, dbConfig }); - } - ); + ssh.forwardOut('127.0.0.1', 0, dbConfig.host, dbConfig.port, (err, stream) => { + if (err) reject(err); + else resolve({ ssh, stream, dbConfig }); + }); }).connect(sshConfig); }); } -/** - * Clear cached query results - * @param {string} [cacheKey] - Specific cache key to clear (clears all if not provided) - */ -function clearQueryCache(cacheKey) { - if (cacheKey) { - connectionCache.queryCache.delete(cacheKey); - console.log(`Cleared cache for key: ${cacheKey}`); - } else { - connectionCache.queryCache.clear(); - console.log('Cleared all query cache'); - } +export function clearQueryCache(cacheKey) { + if (cacheKey) connectionCache.queryCache.delete(cacheKey); + else connectionCache.queryCache.clear(); } -/** - * Force close all active connections - * Useful for server shutdown or manual connection reset - */ -async function closeAllConnections() { +export async function closeAllConnections() { if (connectionCache.dbConnection) { - try { - await connectionCache.dbConnection.end(); - console.log('Closed database connection'); - } catch (error) { - console.error('Error closing database connection:', error); - } + try { await connectionCache.dbConnection.end(); } + catch (error) { console.error('Error closing database connection:', error); } connectionCache.dbConnection = null; } - if (connectionCache.ssh) { - try { - connectionCache.ssh.end(); - console.log('Closed SSH connection'); - } catch (error) { - console.error('Error closing SSH connection:', error); - } + try { connectionCache.ssh.end(); } + catch (error) { console.error('Error closing SSH connection:', error); } connectionCache.ssh = null; } - connectionCache.lastUsed = 0; connectionCache.isConnecting = false; connectionCache.connectionPromise = null; } - -module.exports = { - getDbConnection, - getCachedQuery, - clearQueryCache, - closeAllConnections -}; \ No newline at end of file