Phase 3 + 6
This commit is contained in:
+139
-9
@@ -10,15 +10,18 @@ Audit-driven plan to (a) reduce 12 PM2 processes to 3 application servers + 1 au
|
||||
|---|---|---|
|
||||
| 1 — Decommission dead services | **Complete** | aircall/gorgias/clarity/legacy-auth-server deleted from repo + PM2 + Caddyfile + ecosystem.cjs |
|
||||
| 2 — Build shared `lib/` | **Complete** | Lives at `inventory-server/shared/` (see Deviations). `/verify` endpoint live on auth-server |
|
||||
| 3 — Convert auth-server + inventory-server to ESM | Not started | Next up. Auth-server is still CJS but has the new `/verify` route added in CJS form |
|
||||
| 3 — Convert auth-server + inventory-server to ESM | **Complete (code)** | All 58 server-side files ESM; verified 0 import failures on netcup. Pending: `npm install` on server + pm2 reload to actually run the new code. See Deviations #10–13 |
|
||||
| 4 — Build `dashboard-server` (the merge) | Not started | klaviyo/meta/google/typeform still run as 4 separate PM2 apps |
|
||||
| 5 — Convert `acot-server` to ESM | Not started | |
|
||||
| 6 — Auth hardening | Not started | Shared modules exist (`shared/rate-limit`, `shared/cors`, `shared/logging`) but no service consumes them yet. JWT_SECRET footgun discovered — see 6.4 |
|
||||
| 7 — Caddyfile final form | Partial | Dead routes removed; `forward_auth` gate + `/uploads/*` gating + per-vendor cleanup deferred to after Phase 4 |
|
||||
| 8 — ecosystem.config.cjs final form | Partial | Dead apps removed; final shape depends on Phase 4 merge |
|
||||
| 6 — Auth hardening | **Complete (code) — gated on Phase F1** | All in-process items wired (rate-limit, JWT precondition, CORS lockdown, request-log, upload allowlist, `requirePermission` on sensitive routes, permissions seed migration). `authenticate()` is live on `/api/*`. Server-side artefacts (Caddyfile, ecosystem.cjs) written to `inventory-server/deploy/` for review. 6.11 (audit logging) deferred. **Frontend cannot use the app until Phase F1 ships** — see below |
|
||||
| **F1 — Frontend fetch wrapper (NEW)** | **Not started — CRITICAL** | Frontend uses raw `fetch()` in ~220 sites; only 7 send `Authorization: Bearer`. With Phase 6's `authenticate()` middleware live, every refresh 401s until the frontend uniformly attaches the token. See "Phase F1" below |
|
||||
| 7 — Caddyfile final form | Partial | Proposed file at `inventory-server/deploy/Caddyfile.proposed`. Apply blocked on F1 (forward_auth would 401 every page load until then) |
|
||||
| 8 — ecosystem.config.cjs final form | Partial | Proposed at `inventory-server/deploy/ecosystem.config.cjs.proposed`. Includes Phase 6.4 JWT_SECRET footgun fix and 6.10 lt-wordlist token move |
|
||||
|
||||
**Live PM2 count: 10** (down from 13). Target after Phase 4: 5 application apps + acot-phone-server + lt-wordlist-api.
|
||||
|
||||
**Apply order from current state:** (a) `npm install` on netcup to install the new shared-module deps (`pino`, `pino-http`, `ioredis`, `express-rate-limit`, `jsonwebtoken`), (b) ship Phase F1 frontend fetch wrapper, (c) `pm2 reload inventory-server new-auth-server` (Phase 3+6 code goes live, requests carry tokens, app keeps working), (d) apply `deploy/ecosystem.config.cjs.proposed` (Phase 6.4 + 6.10), (e) apply `deploy/Caddyfile.proposed` (Phase 6.1 — edge gate).
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
@@ -199,7 +202,11 @@ Caddy's `forward_auth` only needs "is this token valid? give me a user-id." Toda
|
||||
|
||||
## Phase 3 — Convert `auth-server` and `inventory-server` to ESM
|
||||
|
||||
Status: **Not started.** Lift the easy ones first. These two stay standalone (don't merge into anything), so they're isolated changes. The auth-server's new `/verify` route (added in Phase 2) is currently CJS — refactor it during this phase to import from `../shared/auth/verify.js`.
|
||||
Status: **Complete (code) — 2026-05-23.** Both servers + all sub-trees converted to ESM. 58 importable .js files load cleanly on netcup (verified via dynamic-import sweep). Two latent bugs surfaced and fixed: `??`/`||` precedence in `shared/db/{pg,redis}.js`, and CJS named-import of `Pool` from `pg` in both auth files (now uses `import pg from 'pg'; const { Pool } = pg`).
|
||||
|
||||
Scripts under `inventory-server/scripts/` (one-shot maintenance / orchestrators) kept CommonJS via a sibling `scripts/package.json` declaring `"type": "commonjs"` — Node's package-type resolution walks up directory by directory, so this overrides the parent's `"type": "module"` without renaming any file or touching any `spawn()` callsite. Convert individual scripts to ESM if/when touched.
|
||||
|
||||
Pending to actually go live: `npm install` on netcup (new deps: `pino`, `pino-http`, `ioredis`, `express-rate-limit`, `jsonwebtoken`) + `pm2 reload`. See "Phase F1" — the frontend fetch wrapper should ship in the same deploy or this immediately breaks the app.
|
||||
|
||||
### Mechanical conversion
|
||||
|
||||
@@ -357,7 +364,23 @@ Same as inventory-server: start with PM2, smoke-test the most-used `/api/acot/*`
|
||||
|
||||
## Phase 6 — Auth hardening
|
||||
|
||||
Status: **Not started.** This is the security work that justifies the whole refactor. Runs in parallel with phases 3–5 where possible. Shared building blocks already exist (`shared/rate-limit/login.js`, `shared/cors/policy.js`, `shared/logging/request-log.js`, `shared/errors/handler.js`) — Phase 6 is about *applying* them per-service.
|
||||
Status: **Complete (code) — 2026-05-23. Application gated on Phase F1.** All in-process hardening shipped alongside the Phase 3 ESM conversion. The `authenticate()` middleware is wired live on `/api/*` in inventory-server — **the moment that code reaches production, the frontend stops working until Phase F1 lands**, because today's frontend doesn't include `Authorization: Bearer` on the vast majority of fetch calls (see Phase F1 below for the diagnosis).
|
||||
|
||||
Per-item status:
|
||||
|
||||
| # | Item | Status | Where |
|
||||
|---|---|---|---|
|
||||
| 6.1 | Caddy `forward_auth` gate | **Proposed** — apply *after* F1 | `inventory-server/deploy/Caddyfile.proposed` |
|
||||
| 6.2 | `requirePermission` on sensitive routes + permissions migration | **Done** | inline in `config.js`, `data-management.js`, `import.js`, `ai-prompts.js`, `ai-validation.js`, `templates.js`, `reusable-images.js`; codes seeded by `migrations/005_phase6_permission_codes.sql` |
|
||||
| 6.3 | Login rate-limit + `/verify` rate-limit | **Done** | `auth/server.js` uses `shared/rate-limit/login.js` (`loginLimiter`, `verifyLimiter`) |
|
||||
| 6.4 | JWT_SECRET as startup precondition + ecosystem footgun fix | **Done in code; proposed for ecosystem.cjs** | Both auth-server and inventory-server `process.exit(1)` if `JWT_SECRET` is unset. `inventory-server/deploy/ecosystem.config.cjs.proposed` removes the `JWT_SECRET: process.env.JWT_SECRET` override that was shadowing `.env` |
|
||||
| 6.5 | Structured request logging w/ redaction | **Done** | `shared/logging/request-log.js` (pino-http, redacts Authorization/Cookie); mounted in both `auth/server.js` and `src/server.js` |
|
||||
| 6.6 | CORS lockdown | **Done** | `src/middleware/cors.js` now re-exports `shared/cors/policy.js`. LAN wildcards (`192.168.*`, `10.*`) and `*` defaults gone |
|
||||
| 6.7 | Upload hardening | **Done** | Exact-match MIME+extension allowlist on `routes/import.js` and `routes/reusable-images.js`; dead `multer({ dest })` removed from `routes/products.js` (no upload route was using it — strongest hardening was deletion) |
|
||||
| 6.8 | Frontend token storage stays localStorage + XSS audit | **Audited** | Confirmed `dangerouslySetInnerHTML` is sanitized in `ProductEditor.tsx`. **Flagged: `ChatRoom.tsx:277,392` renders user-controlled chat content as raw HTML — real XSS vector, separate fix needed** |
|
||||
| 6.9 | Remove debug middleware | **Done** | The header-dumping `app.use((req,res,next)=>{ console.log(... req.headers ...) })` block removed from `src/server.js`. Replaced with `shared/logging/request-log.js` (which redacts). |
|
||||
| 6.10 | `lt-wordlist-api` token move | **Proposed for ecosystem.cjs** | `inventory-server/deploy/ecosystem.config.cjs.proposed` shows the entry without inline token; apply alongside rotating the secret value into `/opt/lt-wordlist-api/.env` |
|
||||
| 6.11 | Audit logging for sensitive ops | **Deferred** | Out of scope for this pass per user direction. Existing `import_audit_log` and `product_editor_audit_log` tables stay as-is; generic `system_audit_log` table + middleware is its own project |
|
||||
|
||||
### 6.1 Caddy `forward_auth` gate
|
||||
|
||||
@@ -494,8 +517,99 @@ Already have `import-audit-log` and `product-editor-audit-log` tables. Extend th
|
||||
|
||||
---
|
||||
|
||||
## Phase F1 — Frontend fetch wrapper (NEW — 2026-05-23)
|
||||
|
||||
Status: **Not started. CRITICAL. Blocks the Phase 3+6 deploy from being usable.**
|
||||
|
||||
### The discovery
|
||||
|
||||
While wiring `authenticate()` on `/api/*` in Phase 6.1/6.2, we audited the frontend's fetch usage and found:
|
||||
|
||||
- **7** call sites send `Authorization: Bearer ${token}` explicitly (all in `AuthContext.tsx` for `/me` + `/login`, plus a couple of `settings/*` pages).
|
||||
- **~220** other `fetch(...)` / `axios.*(...)` call sites across `inventory/src/services/`, `inventory/src/pages/`, `inventory/src/components/` send **no** Authorization header at all.
|
||||
- There is no global fetch wrapper, axios interceptor, or service-worker shim that injects the token.
|
||||
|
||||
Today this works because nothing on the server checks. Caddy currently has no `forward_auth` gate (Phase 6.1 is a Caddyfile change that hasn't shipped yet) and the previous inventory-server had no `authenticate()` middleware. The frontend's auth model was "you log in once to get the token; the token is checked only by `/me`; everything else is implicitly trusted at the network layer."
|
||||
|
||||
With Phase 6 code in production, **every page refresh 401s** on the first API call after the next pm2 reload. The user explicitly accepted this when authorising the Phase 6 work — but the fix is its own deliverable, and shipping Phase 3+6 to PM2 without F1 in the same window means an outage window measured in *however long F1 takes* (not minutes).
|
||||
|
||||
### Recommended approach
|
||||
|
||||
Add a single fetch wrapper at `inventory/src/utils/api.ts` (or similar) and migrate the ~220 call sites to use it. The wrapper:
|
||||
|
||||
1. Reads `localStorage.getItem('token')` on every call (cheap; localStorage is sync).
|
||||
2. Merges `Authorization: Bearer ${token}` into the request headers if a token exists.
|
||||
3. Intercepts 401 responses → fires `window.dispatchEvent(new Event('auth:logout'))` (a listener already exists in `AuthContext.tsx:117`) so the user gets bounced to `/login` cleanly instead of seeing broken pages.
|
||||
4. Preserves the existing call shape — `apiFetch(url, init)` should be a drop-in for `fetch(url, init)` so the migration is mechanical.
|
||||
|
||||
```ts
|
||||
// inventory/src/utils/api.ts (sketch)
|
||||
export async function apiFetch(input: RequestInfo | URL, init: RequestInit = {}): Promise<Response> {
|
||||
const token = localStorage.getItem('token');
|
||||
const headers = new Headers(init.headers);
|
||||
if (token && !headers.has('Authorization')) {
|
||||
headers.set('Authorization', `Bearer ${token}`);
|
||||
}
|
||||
const res = await fetch(input, { ...init, headers });
|
||||
if (res.status === 401 && token) {
|
||||
// Token expired or revoked — bounce to /login. AuthContext already listens.
|
||||
window.dispatchEvent(new Event('auth:logout'));
|
||||
}
|
||||
return res;
|
||||
}
|
||||
```
|
||||
|
||||
Same shape for axios:
|
||||
|
||||
```ts
|
||||
// inventory/src/utils/apiClient.ts (sketch)
|
||||
import axios from 'axios';
|
||||
export const apiClient = axios.create();
|
||||
apiClient.interceptors.request.use((config) => {
|
||||
const token = localStorage.getItem('token');
|
||||
if (token) config.headers.Authorization = `Bearer ${token}`;
|
||||
return config;
|
||||
});
|
||||
apiClient.interceptors.response.use(
|
||||
(r) => r,
|
||||
(err) => {
|
||||
if (err?.response?.status === 401) window.dispatchEvent(new Event('auth:logout'));
|
||||
return Promise.reject(err);
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
### Migration plan
|
||||
|
||||
1. Land the two wrapper modules above. ~50 LOC total.
|
||||
2. Codemod or sed-loop: in `inventory/src/`, replace `fetch(` → `apiFetch(` (with the right import) and `axios.get/post/...` → `apiClient.get/post/...`. ~220 call sites — a half-day of careful find-and-replace plus per-page verification. Spot-check the ones with custom `Content-Type` (multipart uploads especially) so the wrapper doesn't clobber multipart boundaries.
|
||||
3. Leave the `AuthContext.tsx` `/login` and `/me` calls alone — they already work and migrating them adds no value.
|
||||
4. Run the SPA: log in, exercise Overview / Products / Analytics / Dashboard / etc. with browser devtools open watching for `Authorization` header on every `/api/*` request.
|
||||
|
||||
### Sequencing with Phase 3+6 deploy
|
||||
|
||||
**Two options:**
|
||||
|
||||
A) **Ship F1 first** (recommended). Frontend goes out with the wrapper; nothing changes server-side. Then `pm2 reload` Phase 3+6. Zero-downtime, zero broken-page window.
|
||||
|
||||
B) **Ship together.** F1 and Phase 3+6 land in the same deploy. Brief window (seconds) where the frontend has the wrapper but the server hasn't reloaded yet — wrapper just sends extra headers the old server ignores. Safe.
|
||||
|
||||
Do **not** ship Phase 3+6 first and F1 second. That gives a broken app for as long as F1 takes.
|
||||
|
||||
### Out of scope (kept on `localStorage`)
|
||||
|
||||
Per Phase 6.8, we're not migrating to httpOnly cookie auth. F1 is the minimum work to make the per-service `authenticate()` (Phase 6) actually usable. A future Phase F2 could move to cookies + CSRF double-submit, but that's a much larger change touching the AuthContext, the login flow, and every backend that reads tokens. Not justified for an internal tool with no public sign-up.
|
||||
|
||||
### Note on `/uploads/*` gating (Phase 6.7's Caddyfile change)
|
||||
|
||||
The proposed Caddyfile moves `/uploads/*` behind `forward_auth`. Most product images today are referenced from `<img src="/uploads/...">` in the SPA — those requests are made by the browser, which **does not include `Authorization` headers on image requests**. Fixing this is part of F1's scope too: either (a) keep `/uploads/*` public (revert that part of 6.7) and accept that uploaded images leak to anyone who guesses a URL, or (b) issue per-image signed URLs from the API and gate those at Caddy. Decide before applying the Caddyfile.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Caddyfile final form
|
||||
|
||||
Status: **Proposed (2026-05-23). Apply blocked on Phase F1.** The full proposed file lives at `inventory-server/deploy/Caddyfile.proposed` and matches the spec below except that vendor handle blocks still point to per-vendor PM2 apps (Phase 4 hasn't merged them yet). See `inventory-server/deploy/README.md` for the apply commands (admin-API + sudo cp pattern from Phase 2 deviation #8).
|
||||
|
||||
After all phases, the `tools.acherryontop.com` block looks like:
|
||||
|
||||
```caddyfile
|
||||
@@ -571,6 +685,8 @@ Removed: `/dashboard-auth/*`, `/api/aircall/*`, `/api/gorgias/*`, `/api/clarity/
|
||||
|
||||
## Phase 8 — ecosystem.config.cjs final form
|
||||
|
||||
Status: **Proposed (2026-05-23).** Full proposed file at `inventory-server/deploy/ecosystem.config.cjs.proposed`. Includes the Phase 6.4 `JWT_SECRET` shadow-override fix and the Phase 6.10 `lt-wordlist-api` token move. Still lists per-vendor PM2 apps until Phase 4 merge ships — that's the only thing keeping app count at 10 instead of the target 5.
|
||||
|
||||
```js
|
||||
module.exports = {
|
||||
apps: [
|
||||
@@ -646,10 +762,11 @@ Phase 1 unblocks everything (fewer services to convert).
|
||||
Phase 2 is the foundation; nothing else can start until shared `lib/` exists.
|
||||
Phases 3–5 can run in parallel; they touch independent services.
|
||||
Phase 6's sub-items can be developed alongside 3–5 but **enabled** only after them (no point adding `requirePermission` to a route that doesn't yet have `authenticate`).
|
||||
Phase 7 is the cutover: Caddyfile flip happens when all backend changes are deployed.
|
||||
**Phase F1 must precede the Phase 3+6 pm2 reload** — without the fetch wrapper, the moment the new code goes live the SPA breaks. Discovered during Phase 3+6 implementation; see Phase F1.
|
||||
Phase 7 is the cutover: Caddyfile flip happens after F1 ships AND after the `/uploads/*` gating decision in F1 is made.
|
||||
Phase 8 is cleanup: remove dead PM2 entries.
|
||||
|
||||
Estimated effort, end-to-end: **~3 weeks of focused work** by one engineer. Phase 1 ≈ 1 day, Phase 2 ≈ 2 days, Phase 3 ≈ 3 days (both services), Phase 4 ≈ 5–7 days (the merge), Phase 5 ≈ 2–3 days, Phase 6 ≈ 3–4 days, Phase 7+8 ≈ 1 day.
|
||||
Estimated effort, end-to-end: **~3 weeks of focused work** by one engineer. Phase 1 ≈ 1 day, Phase 2 ≈ 2 days, Phase 3 ≈ 3 days (both services), Phase 4 ≈ 5–7 days (the merge), Phase 5 ≈ 2–3 days, Phase 6 ≈ 3–4 days, Phase F1 ≈ 0.5–1 day, Phase 7+8 ≈ 1 day.
|
||||
|
||||
---
|
||||
|
||||
@@ -699,12 +816,13 @@ Each phase produces an independently deployable state. Rollback per phase:
|
||||
|
||||
These came up in the audit but aren't part of this refactor:
|
||||
|
||||
- `httpOnly` cookie auth (deferred — current `localStorage` acceptable for internal tool).
|
||||
- `httpOnly` cookie auth ("Phase F2" — deferred). Phase F1 keeps `localStorage` + Bearer header because that's the minimum to unblock the Phase 6 `authenticate()` rollout. A future move to cookie auth would touch `AuthContext`, every backend that reads tokens, and introduce CSRF concerns — much larger project.
|
||||
- Replacing PM2 with systemd or Docker.
|
||||
- Test coverage beyond the auth-critical surface.
|
||||
- `apiv2`/`apiv2-test` proxies to `backend.acherryontop.com` — separate system, not touched.
|
||||
- `acot-phone-server` and `lt-wordlist-api` — staying as-is.
|
||||
- Centralized observability stack (Prometheus, Grafana). The logger work in Phase 6.5 sets up the data, but shipping it somewhere is future work.
|
||||
- ChatRoom XSS remediation (flagged during Phase 6.8 audit — `inventory/src/components/chat/ChatRoom.tsx:277,392` renders user-controlled chat content via `dangerouslySetInnerHTML` without sanitization). Real vulnerability for an internal-but-multi-user tool; separate fix.
|
||||
|
||||
---
|
||||
|
||||
@@ -749,3 +867,15 @@ These are decisions made during Phase 1/2 implementation that amend the spec abo
|
||||
8. **Caddyfile changes via admin API on `:2020`.** The Caddyfile is owned by root and matt has no passwordless sudo. Cutover used `curl -X POST .../load` on the Caddy admin port (which matt can hit), then a separate `sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile` step to persist the on-disk file. Future Caddyfile changes can follow the same pattern. Backup convention: `/etc/caddy/Caddyfile.bak.YYYY-MM-DD`.
|
||||
|
||||
9. **Path-naming.** Plan uses `inventory/` as the top-level (server-side path convention). Locally the equivalent is `inventory-server/`. Whenever the plan says `inventory/dashboard/foo/`, read that as `/var/www/inventory/dashboard/foo/` on the server or `inventory-server/dashboard/foo/` locally.
|
||||
|
||||
10. **Scripts directory kept CJS via package.json shim.** Original plan called for converting "any spawned script" to ESM alongside its caller. Implemented: added `inventory-server/scripts/package.json` with `"type": "commonjs"`. Node's package-type resolution walks up directory by directory, so this overrides the parent's `"type": "module"` for the entire `scripts/` tree (≈15 files including `import/*.js`, `metrics-new/utils/*`, the orchestrator scripts) without renaming any file or touching any `spawn()` callsite. Convert individual scripts to ESM when touched; don't bulk-migrate.
|
||||
|
||||
11. **`src/routes/products.js` had dead multer setup.** Phase 6.7 spec called for hardening the upload route in products.js. There was no upload route — the `multer({ dest })` instance and `importProductsFromCSV` import were dead code left over from a long-ago migration. Strongest 6.7 hardening was deletion: no upload handler = no attack surface. The two real upload paths (`/api/import/upload-image` and `/api/reusable-images/upload`) got tightened MIME+extension allowlists instead.
|
||||
|
||||
12. **Two pre-existing syntax errors in shared/db/ surfaced.** `shared/db/pg.js:13` and `shared/db/redis.js:22` both had `?? Number(...) || N` — mixing `??` and `||` without parentheses is a TC39 syntax error. They passed Phase 2 because nothing imported them yet; Phase 3 smoke-test exposed it. Fixed with parens.
|
||||
|
||||
13. **`import { Pool } from 'pg'` doesn't work in ESM.** The `pg` package is CJS using `module.exports = { Pool, ... }`. Node's ESM-from-CJS interop fails to detect `Pool` as a named export via static analysis. The bulletproof pattern, now used everywhere: `import pg from 'pg'; const { Pool } = pg;`. Same idea for any future CJS-only deps. `src/utils/db.js` already had it; the two auth files needed the fix during execution.
|
||||
|
||||
14. **Frontend Bearer-header gap discovered (drives new Phase F1).** Phase 6 was specified assuming the frontend already sends `Authorization: Bearer` on every API call. It does not — only 7 of ~220 call sites do. Phase 6's `authenticate()` middleware is shipped and ready to enable, but until F1 lands the SPA will 401 on every page. The plan now has Phase F1 to address this explicitly; until then, the Phase 3+6 pm2 reload should not ship unless F1 ships in the same window.
|
||||
|
||||
15. **macOS NFS workflow note.** The `inventory-server/` directory locally is an NFS mount of `/var/www/inventory/` on netcup. Bulk operations (`find`/`grep -r`/mass `node --check`/`npm install`) hang or take minutes locally and pollute file listings with macOS AppleDouble `._*` sidecar files. Default to `ssh netcup` for any sweep across the tree — individual file edits via the editor are fine.
|
||||
|
||||
@@ -1,100 +1,72 @@
|
||||
require('dotenv').config({ path: '../.env' });
|
||||
const bcrypt = require('bcrypt');
|
||||
const { Pool } = require('pg');
|
||||
const inquirer = require('inquirer');
|
||||
import bcrypt from 'bcrypt';
|
||||
import pg from 'pg';
|
||||
import inquirer from 'inquirer';
|
||||
|
||||
// Log connection details for debugging (remove in production)
|
||||
console.log('Attempting to connect with:', {
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT
|
||||
});
|
||||
const { Pool } = pg;
|
||||
import { config as loadEnv } from 'dotenv';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, resolve as resolvePath } from 'node:path';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
loadEnv({ path: resolvePath(__dirname, '../.env') });
|
||||
|
||||
const pool = new Pool({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
password: process.env.DB_PASSWORD,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT,
|
||||
port: Number(process.env.DB_PORT) || 5432,
|
||||
});
|
||||
|
||||
async function promptUser() {
|
||||
const questions = [
|
||||
return inquirer.prompt([
|
||||
{
|
||||
type: 'input',
|
||||
name: 'username',
|
||||
message: 'Enter username:',
|
||||
validate: (input) => {
|
||||
if (input.length < 3) {
|
||||
return 'Username must be at least 3 characters long';
|
||||
}
|
||||
return true;
|
||||
}
|
||||
validate: (input) => input.length >= 3 || 'Username must be at least 3 characters long',
|
||||
},
|
||||
{
|
||||
type: 'password',
|
||||
name: 'password',
|
||||
message: 'Enter password:',
|
||||
mask: '*',
|
||||
validate: (input) => {
|
||||
if (input.length < 8) {
|
||||
return 'Password must be at least 8 characters long';
|
||||
}
|
||||
return true;
|
||||
}
|
||||
validate: (input) => input.length >= 8 || 'Password must be at least 8 characters long',
|
||||
},
|
||||
{
|
||||
type: 'password',
|
||||
name: 'confirmPassword',
|
||||
message: 'Confirm password:',
|
||||
mask: '*',
|
||||
validate: (input, answers) => {
|
||||
if (input !== answers.password) {
|
||||
return 'Passwords do not match';
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
return inquirer.prompt(questions);
|
||||
validate: (input, answers) => input === answers.password || 'Passwords do not match',
|
||||
},
|
||||
]);
|
||||
}
|
||||
|
||||
async function addUser() {
|
||||
try {
|
||||
// Get user input
|
||||
const answers = await promptUser();
|
||||
const { username, password } = answers;
|
||||
const { username, password } = await promptUser();
|
||||
const hashedPassword = await bcrypt.hash(password, 10);
|
||||
|
||||
// Hash password
|
||||
const saltRounds = 10;
|
||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
||||
|
||||
// Check if user already exists
|
||||
const checkResult = await pool.query(
|
||||
'SELECT id FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
|
||||
if (checkResult.rows.length > 0) {
|
||||
console.error('Error: Username already exists');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Insert new user
|
||||
const result = await pool.query(
|
||||
'INSERT INTO users (username, password) VALUES ($1, $2) RETURNING id',
|
||||
[username, hashedPassword]
|
||||
);
|
||||
|
||||
console.log(`User ${username} created successfully with id ${result.rows[0].id}`);
|
||||
} catch (error) {
|
||||
console.error('Error creating user:', error);
|
||||
console.error('Error details:', error.message);
|
||||
if (error.code) {
|
||||
console.error('Error code:', error.code);
|
||||
}
|
||||
if (error.code) console.error('Error code:', error.code);
|
||||
} finally {
|
||||
await pool.end();
|
||||
}
|
||||
|
||||
@@ -2,18 +2,22 @@
|
||||
"name": "inventory-auth-server",
|
||||
"version": "1.0.0",
|
||||
"description": "Authentication server for inventory management system",
|
||||
"type": "module",
|
||||
"main": "server.js",
|
||||
"scripts": {
|
||||
"start": "node server.js"
|
||||
"start": "node server.js",
|
||||
"add-user": "node add-user.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"bcrypt": "^5.1.1",
|
||||
"cors": "^2.8.5",
|
||||
"dotenv": "^16.4.7",
|
||||
"express": "^4.18.2",
|
||||
"express-rate-limit": "^7.4.0",
|
||||
"inquirer": "^8.2.6",
|
||||
"jsonwebtoken": "^9.0.2",
|
||||
"morgan": "^1.10.0",
|
||||
"pg": "^8.11.3"
|
||||
"pg": "^8.11.3",
|
||||
"pino": "^9.5.0",
|
||||
"pino-http": "^10.3.0"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,128 +1,73 @@
|
||||
// Get pool from global or create a new one if not available
|
||||
let pool;
|
||||
if (typeof global.pool !== 'undefined') {
|
||||
pool = global.pool;
|
||||
} else {
|
||||
// If global pool is not available, create a new connection
|
||||
const { Pool } = require('pg');
|
||||
pool = new Pool({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
password: process.env.DB_PASSWORD,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT,
|
||||
});
|
||||
console.log('Created new database pool in permissions.js');
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a user has a specific permission
|
||||
* @param {number} userId - The user ID to check
|
||||
* @param {string} permissionCode - The permission code to check
|
||||
* @returns {Promise<boolean>} - Whether the user has the permission
|
||||
*/
|
||||
async function checkPermission(userId, permissionCode) {
|
||||
try {
|
||||
// First check if the user is an admin
|
||||
const adminResult = await pool.query(
|
||||
'SELECT is_admin FROM users WHERE id = $1',
|
||||
[userId]
|
||||
);
|
||||
|
||||
// If user is admin, automatically grant permission
|
||||
if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Otherwise check for specific permission
|
||||
const result = await pool.query(
|
||||
`SELECT COUNT(*) AS has_permission
|
||||
FROM user_permissions up
|
||||
JOIN permissions p ON up.permission_id = p.id
|
||||
WHERE up.user_id = $1 AND p.code = $2`,
|
||||
[userId, permissionCode]
|
||||
);
|
||||
|
||||
return result.rows[0].has_permission > 0;
|
||||
} catch (error) {
|
||||
console.error('Error checking permission:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Middleware to require a specific permission
|
||||
* @param {string} permissionCode - The permission code required
|
||||
* @returns {Function} - Express middleware function
|
||||
*/
|
||||
function requirePermission(permissionCode) {
|
||||
return async (req, res, next) => {
|
||||
export function createPermissionHelpers({ pool }) {
|
||||
async function checkPermission(userId, permissionCode) {
|
||||
try {
|
||||
// Check if user is authenticated
|
||||
if (!req.user || !req.user.id) {
|
||||
return res.status(401).json({ error: 'Authentication required' });
|
||||
}
|
||||
|
||||
const hasPermission = await checkPermission(req.user.id, permissionCode);
|
||||
|
||||
if (!hasPermission) {
|
||||
return res.status(403).json({
|
||||
error: 'Insufficient permissions',
|
||||
requiredPermission: permissionCode
|
||||
});
|
||||
}
|
||||
|
||||
next();
|
||||
} catch (error) {
|
||||
console.error('Permission middleware error:', error);
|
||||
res.status(500).json({ error: 'Server error checking permissions' });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all permissions for a user
|
||||
* @param {number} userId - The user ID
|
||||
* @returns {Promise<string[]>} - Array of permission codes
|
||||
*/
|
||||
async function getUserPermissions(userId) {
|
||||
try {
|
||||
// Check if user is admin
|
||||
const adminResult = await pool.query(
|
||||
'SELECT is_admin FROM users WHERE id = $1',
|
||||
[userId]
|
||||
);
|
||||
|
||||
if (adminResult.rows.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const isAdmin = adminResult.rows[0].is_admin;
|
||||
|
||||
if (isAdmin) {
|
||||
// Admin gets all permissions
|
||||
const allPermissions = await pool.query('SELECT code FROM permissions');
|
||||
return allPermissions.rows.map(p => p.code);
|
||||
} else {
|
||||
// Get assigned permissions
|
||||
const permissions = await pool.query(
|
||||
`SELECT p.code
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1`,
|
||||
const adminResult = await pool.query(
|
||||
'SELECT is_admin FROM users WHERE id = $1',
|
||||
[userId]
|
||||
);
|
||||
if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) return true;
|
||||
|
||||
return permissions.rows.map(p => p.code);
|
||||
const result = await pool.query(
|
||||
`SELECT COUNT(*) AS has_permission
|
||||
FROM user_permissions up
|
||||
JOIN permissions p ON up.permission_id = p.id
|
||||
WHERE up.user_id = $1 AND p.code = $2`,
|
||||
[userId, permissionCode]
|
||||
);
|
||||
return Number(result.rows[0].has_permission) > 0;
|
||||
} catch (error) {
|
||||
console.error('Error checking permission:', error);
|
||||
return false;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error getting user permissions:', error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
checkPermission,
|
||||
requirePermission,
|
||||
getUserPermissions
|
||||
};
|
||||
function requirePermission(permissionCode) {
|
||||
return async (req, res, next) => {
|
||||
try {
|
||||
if (!req.user?.id) {
|
||||
return res.status(401).json({ error: 'Authentication required' });
|
||||
}
|
||||
const hasPermission = await checkPermission(req.user.id, permissionCode);
|
||||
if (!hasPermission) {
|
||||
return res.status(403).json({
|
||||
error: 'Insufficient permissions',
|
||||
requiredPermission: permissionCode,
|
||||
});
|
||||
}
|
||||
next();
|
||||
} catch (error) {
|
||||
console.error('Permission middleware error:', error);
|
||||
res.status(500).json({ error: 'Server error checking permissions' });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async function getUserPermissions(userId) {
|
||||
try {
|
||||
const adminResult = await pool.query(
|
||||
'SELECT is_admin FROM users WHERE id = $1',
|
||||
[userId]
|
||||
);
|
||||
if (adminResult.rows.length === 0) return [];
|
||||
|
||||
if (adminResult.rows[0].is_admin) {
|
||||
const allPermissions = await pool.query('SELECT code FROM permissions');
|
||||
return allPermissions.rows.map((p) => p.code);
|
||||
}
|
||||
|
||||
const permissions = await pool.query(
|
||||
`SELECT p.code
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1`,
|
||||
[userId]
|
||||
);
|
||||
return permissions.rows.map((p) => p.code);
|
||||
} catch (error) {
|
||||
console.error('Error getting user permissions:', error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
return { checkPermission, requirePermission, getUserPermissions };
|
||||
}
|
||||
|
||||
+296
-512
@@ -1,533 +1,317 @@
|
||||
const express = require('express');
|
||||
const router = express.Router();
|
||||
const bcrypt = require('bcrypt');
|
||||
const jwt = require('jsonwebtoken');
|
||||
const { requirePermission, getUserPermissions } = require('./permissions');
|
||||
import express from 'express';
|
||||
import bcrypt from 'bcrypt';
|
||||
import jwt from 'jsonwebtoken';
|
||||
import { createPermissionHelpers } from './permissions.js';
|
||||
|
||||
// Get pool from global or create a new one if not available
|
||||
let pool;
|
||||
if (typeof global.pool !== 'undefined') {
|
||||
pool = global.pool;
|
||||
} else {
|
||||
// If global pool is not available, create a new connection
|
||||
const { Pool } = require('pg');
|
||||
pool = new Pool({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
password: process.env.DB_PASSWORD,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT,
|
||||
export function createAuthRoutes({ pool }) {
|
||||
const router = express.Router();
|
||||
const { requirePermission, getUserPermissions } = createPermissionHelpers({ pool });
|
||||
|
||||
// Local authenticate(): used by user-management endpoints that need req.user populated
|
||||
// with id/username/email/is_admin. NOT the per-service authenticate() — that lives in
|
||||
// shared/auth/middleware.js and is used by downstream services. Auth-server's surface is
|
||||
// small enough that a local copy is fine; the security boundary is the JWT verify step.
|
||||
async function authenticate(req, res, next) {
|
||||
try {
|
||||
const authHeader = req.headers.authorization;
|
||||
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
||||
return res.status(401).json({ error: 'Authentication required' });
|
||||
}
|
||||
const token = authHeader.split(' ')[1];
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
const result = await pool.query(
|
||||
'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1',
|
||||
[decoded.userId]
|
||||
);
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(401).json({ error: 'User not found' });
|
||||
}
|
||||
req.user = result.rows[0];
|
||||
next();
|
||||
} catch (error) {
|
||||
res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
}
|
||||
|
||||
router.post('/login', async (req, res) => {
|
||||
try {
|
||||
const { username, password } = req.body;
|
||||
const result = await pool.query(
|
||||
'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(401).json({ error: 'Invalid username or password' });
|
||||
}
|
||||
const user = result.rows[0];
|
||||
if (!user.is_active) {
|
||||
return res.status(403).json({ error: 'Account is inactive' });
|
||||
}
|
||||
const validPassword = await bcrypt.compare(password, user.password);
|
||||
if (!validPassword) {
|
||||
return res.status(401).json({ error: 'Invalid username or password' });
|
||||
}
|
||||
await pool.query(
|
||||
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
||||
[user.id]
|
||||
);
|
||||
const token = jwt.sign(
|
||||
{ userId: user.id, username: user.username },
|
||||
process.env.JWT_SECRET,
|
||||
{ expiresIn: '8h' }
|
||||
);
|
||||
const permissions = await getUserPermissions(user.id);
|
||||
res.json({
|
||||
token,
|
||||
user: {
|
||||
id: user.id,
|
||||
username: user.username,
|
||||
is_admin: user.is_admin,
|
||||
rocket_chat_user_id: user.rocket_chat_user_id,
|
||||
permissions,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Login error:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
console.log('Created new database pool in routes.js');
|
||||
}
|
||||
|
||||
// Authentication middleware
|
||||
const authenticate = async (req, res, next) => {
|
||||
try {
|
||||
const authHeader = req.headers.authorization;
|
||||
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
||||
return res.status(401).json({ error: 'Authentication required' });
|
||||
router.get('/me', authenticate, async (req, res) => {
|
||||
try {
|
||||
const permissions = await getUserPermissions(req.user.id);
|
||||
res.json({
|
||||
id: req.user.id,
|
||||
username: req.user.username,
|
||||
email: req.user.email,
|
||||
is_admin: req.user.is_admin,
|
||||
rocket_chat_user_id: req.user.rocket_chat_user_id,
|
||||
permissions,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error getting current user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
const token = authHeader.split(' ')[1];
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
|
||||
// Get user from database
|
||||
const result = await pool.query(
|
||||
'SELECT id, username, email, is_admin, rocket_chat_user_id FROM users WHERE id = $1',
|
||||
[decoded.userId]
|
||||
);
|
||||
|
||||
console.log('Database query result for user', decoded.userId, ':', result.rows[0]);
|
||||
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(401).json({ error: 'User not found' });
|
||||
router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||
FROM users
|
||||
ORDER BY username
|
||||
`);
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting users:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Attach user to request
|
||||
req.user = result.rows[0];
|
||||
next();
|
||||
} catch (error) {
|
||||
console.error('Authentication error:', error);
|
||||
res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
};
|
||||
|
||||
// Login route
|
||||
router.post('/login', async (req, res) => {
|
||||
try {
|
||||
const { username, password } = req.body;
|
||||
|
||||
// Get user from database
|
||||
const result = await pool.query(
|
||||
'SELECT id, username, password, is_admin, is_active, rocket_chat_user_id FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(401).json({ error: 'Invalid username or password' });
|
||||
}
|
||||
|
||||
const user = result.rows[0];
|
||||
|
||||
// Check if user is active
|
||||
if (!user.is_active) {
|
||||
return res.status(403).json({ error: 'Account is inactive' });
|
||||
}
|
||||
|
||||
// Verify password
|
||||
const validPassword = await bcrypt.compare(password, user.password);
|
||||
if (!validPassword) {
|
||||
return res.status(401).json({ error: 'Invalid username or password' });
|
||||
}
|
||||
|
||||
// Update last login
|
||||
await pool.query(
|
||||
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
||||
[user.id]
|
||||
);
|
||||
|
||||
// Generate JWT
|
||||
const token = jwt.sign(
|
||||
{ userId: user.id, username: user.username },
|
||||
process.env.JWT_SECRET,
|
||||
{ expiresIn: '8h' }
|
||||
);
|
||||
|
||||
// Get user permissions
|
||||
const permissions = await getUserPermissions(user.id);
|
||||
|
||||
res.json({
|
||||
token,
|
||||
user: {
|
||||
id: user.id,
|
||||
username: user.username,
|
||||
is_admin: user.is_admin,
|
||||
rocket_chat_user_id: user.rocket_chat_user_id,
|
||||
permissions
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Login error:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get current user
|
||||
router.get('/me', authenticate, async (req, res) => {
|
||||
try {
|
||||
// Get user permissions
|
||||
const permissions = await getUserPermissions(req.user.id);
|
||||
|
||||
res.json({
|
||||
id: req.user.id,
|
||||
username: req.user.username,
|
||||
email: req.user.email,
|
||||
is_admin: req.user.is_admin,
|
||||
rocket_chat_user_id: req.user.rocket_chat_user_id,
|
||||
permissions,
|
||||
// Debug info
|
||||
_debug_raw_user: req.user,
|
||||
_server_identifier: "INVENTORY_AUTH_SERVER_MODIFIED"
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error getting current user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get all users
|
||||
router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||
FROM users
|
||||
ORDER BY username
|
||||
`);
|
||||
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting users:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get user with permissions
|
||||
router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
|
||||
// Get user details
|
||||
const userResult = await pool.query(`
|
||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||
FROM users
|
||||
WHERE id = $1
|
||||
`, [userId]);
|
||||
|
||||
if (userResult.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
|
||||
// Get user permissions
|
||||
const permissionsResult = await pool.query(`
|
||||
SELECT p.id, p.name, p.code, p.category, p.description
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1
|
||||
ORDER BY p.category, p.name
|
||||
`, [userId]);
|
||||
|
||||
// Combine user and permissions
|
||||
const user = {
|
||||
...userResult.rows[0],
|
||||
permissions: permissionsResult.rows
|
||||
};
|
||||
|
||||
res.json(user);
|
||||
} catch (error) {
|
||||
console.error('Error getting user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Create new user
|
||||
router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => {
|
||||
const client = await pool.connect();
|
||||
|
||||
try {
|
||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||
|
||||
console.log("Create user request:", {
|
||||
username,
|
||||
email,
|
||||
is_admin,
|
||||
is_active,
|
||||
rocket_chat_user_id,
|
||||
permissions: permissions || []
|
||||
});
|
||||
|
||||
// Validate required fields
|
||||
if (!username || !password) {
|
||||
return res.status(400).json({ error: 'Username and password are required' });
|
||||
}
|
||||
|
||||
// Check if username is taken
|
||||
const existingUser = await client.query(
|
||||
'SELECT id FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
|
||||
if (existingUser.rows.length > 0) {
|
||||
return res.status(400).json({ error: 'Username already exists' });
|
||||
}
|
||||
|
||||
// Start transaction
|
||||
await client.query('BEGIN');
|
||||
|
||||
// Hash password
|
||||
const saltRounds = 10;
|
||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
||||
|
||||
// Insert new user
|
||||
// Convert rocket_chat_user_id to integer if provided
|
||||
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
||||
|
||||
const userResult = await client.query(`
|
||||
INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP)
|
||||
RETURNING id
|
||||
`, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]);
|
||||
|
||||
const userId = userResult.rows[0].id;
|
||||
|
||||
// Assign permissions if provided and not admin
|
||||
if (!is_admin && Array.isArray(permissions) && permissions.length > 0) {
|
||||
console.log("Adding permissions for new user:", userId);
|
||||
console.log("Permissions received:", permissions);
|
||||
|
||||
// Check permission format
|
||||
const permissionIds = permissions.map(p => {
|
||||
if (typeof p === 'object' && p.id) {
|
||||
console.log("Permission is an object with ID:", p.id);
|
||||
return parseInt(p.id, 10);
|
||||
} else if (typeof p === 'number') {
|
||||
console.log("Permission is a number:", p);
|
||||
return p;
|
||||
} else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
|
||||
console.log("Permission is a string that can be parsed as a number:", p);
|
||||
return parseInt(p, 10);
|
||||
} else {
|
||||
console.log("Unknown permission format:", typeof p, p);
|
||||
// If it's a permission code, we need to look up the ID
|
||||
return null;
|
||||
}
|
||||
}).filter(id => id !== null);
|
||||
|
||||
console.log("Filtered permission IDs:", permissionIds);
|
||||
|
||||
if (permissionIds.length > 0) {
|
||||
const permissionValues = permissionIds
|
||||
.map(permId => `(${userId}, ${permId})`)
|
||||
.join(',');
|
||||
|
||||
console.log("Inserting permission values:", permissionValues);
|
||||
|
||||
try {
|
||||
await client.query(`
|
||||
INSERT INTO user_permissions (user_id, permission_id)
|
||||
VALUES ${permissionValues}
|
||||
ON CONFLICT DO NOTHING
|
||||
`);
|
||||
console.log("Successfully inserted permissions for new user:", userId);
|
||||
} catch (err) {
|
||||
console.error("Error inserting permissions for new user:", err);
|
||||
throw err;
|
||||
}
|
||||
} else {
|
||||
console.log("No valid permission IDs found to insert for new user");
|
||||
}
|
||||
} else {
|
||||
console.log("Not adding permissions: is_admin =", is_admin, "permissions array:", Array.isArray(permissions), "length:", permissions ? permissions.length : 0);
|
||||
}
|
||||
|
||||
await client.query('COMMIT');
|
||||
|
||||
res.status(201).json({
|
||||
id: userId,
|
||||
message: 'User created successfully'
|
||||
});
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK');
|
||||
console.error('Error creating user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
});
|
||||
|
||||
// Update user
|
||||
router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => {
|
||||
const client = await pool.connect();
|
||||
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||
|
||||
console.log("Update user request:", {
|
||||
userId,
|
||||
username,
|
||||
email,
|
||||
is_admin,
|
||||
is_active,
|
||||
rocket_chat_user_id,
|
||||
permissions: permissions || []
|
||||
});
|
||||
|
||||
// Check if user exists
|
||||
const userExists = await client.query(
|
||||
'SELECT id FROM users WHERE id = $1',
|
||||
[userId]
|
||||
);
|
||||
|
||||
if (userExists.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
|
||||
// Start transaction
|
||||
await client.query('BEGIN');
|
||||
|
||||
// Build update fields
|
||||
const updateFields = [];
|
||||
const updateValues = [userId]; // First parameter is the user ID
|
||||
let paramIndex = 2;
|
||||
|
||||
if (username !== undefined) {
|
||||
updateFields.push(`username = $${paramIndex++}`);
|
||||
updateValues.push(username);
|
||||
}
|
||||
|
||||
if (email !== undefined) {
|
||||
updateFields.push(`email = $${paramIndex++}`);
|
||||
updateValues.push(email || null);
|
||||
}
|
||||
|
||||
if (is_admin !== undefined) {
|
||||
updateFields.push(`is_admin = $${paramIndex++}`);
|
||||
updateValues.push(!!is_admin);
|
||||
}
|
||||
|
||||
if (is_active !== undefined) {
|
||||
updateFields.push(`is_active = $${paramIndex++}`);
|
||||
updateValues.push(!!is_active);
|
||||
}
|
||||
|
||||
if (rocket_chat_user_id !== undefined) {
|
||||
updateFields.push(`rocket_chat_user_id = $${paramIndex++}`);
|
||||
// Convert to integer if not null/undefined, otherwise null
|
||||
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
||||
updateValues.push(rcUserId);
|
||||
}
|
||||
|
||||
// Update password if provided
|
||||
if (password) {
|
||||
const saltRounds = 10;
|
||||
const hashedPassword = await bcrypt.hash(password, saltRounds);
|
||||
updateFields.push(`password = $${paramIndex++}`);
|
||||
updateValues.push(hashedPassword);
|
||||
}
|
||||
|
||||
// Update user if there are fields to update
|
||||
if (updateFields.length > 0) {
|
||||
updateFields.push(`updated_at = CURRENT_TIMESTAMP`);
|
||||
|
||||
await client.query(`
|
||||
UPDATE users
|
||||
SET ${updateFields.join(', ')}
|
||||
router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
const userResult = await pool.query(`
|
||||
SELECT id, username, email, is_admin, is_active, rocket_chat_user_id, created_at, last_login
|
||||
FROM users
|
||||
WHERE id = $1
|
||||
`, updateValues);
|
||||
`, [userId]);
|
||||
if (userResult.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
const permissionsResult = await pool.query(`
|
||||
SELECT p.id, p.name, p.code, p.category, p.description
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1
|
||||
ORDER BY p.category, p.name
|
||||
`, [userId]);
|
||||
res.json({
|
||||
...userResult.rows[0],
|
||||
permissions: permissionsResult.rows,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error getting user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Update permissions if provided
|
||||
if (Array.isArray(permissions)) {
|
||||
console.log("Updating permissions for user:", userId);
|
||||
console.log("Permissions received:", permissions);
|
||||
router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => {
|
||||
const client = await pool.connect();
|
||||
try {
|
||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||
if (!username || !password) {
|
||||
return res.status(400).json({ error: 'Username and password are required' });
|
||||
}
|
||||
const existingUser = await client.query(
|
||||
'SELECT id FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
if (existingUser.rows.length > 0) {
|
||||
return res.status(400).json({ error: 'Username already exists' });
|
||||
}
|
||||
await client.query('BEGIN');
|
||||
const hashedPassword = await bcrypt.hash(password, 10);
|
||||
const rcUserId = rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null;
|
||||
const userResult = await client.query(`
|
||||
INSERT INTO users (username, email, password, is_admin, is_active, rocket_chat_user_id, created_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP)
|
||||
RETURNING id
|
||||
`, [username, email || null, hashedPassword, !!is_admin, is_active !== false, rcUserId]);
|
||||
const userId = userResult.rows[0].id;
|
||||
|
||||
// First remove existing permissions
|
||||
await client.query(
|
||||
'DELETE FROM user_permissions WHERE user_id = $1',
|
||||
if (!is_admin && Array.isArray(permissions) && permissions.length > 0) {
|
||||
const permissionIds = normalizePermissionIds(permissions);
|
||||
if (permissionIds.length > 0) {
|
||||
await client.query(
|
||||
`INSERT INTO user_permissions (user_id, permission_id)
|
||||
SELECT $1, unnest($2::int[])
|
||||
ON CONFLICT DO NOTHING`,
|
||||
[userId, permissionIds]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
await client.query('COMMIT');
|
||||
res.status(201).json({ id: userId, message: 'User created successfully' });
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK');
|
||||
console.error('Error creating user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
});
|
||||
|
||||
router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => {
|
||||
const client = await pool.connect();
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
const { username, email, password, is_admin, is_active, rocket_chat_user_id, permissions } = req.body;
|
||||
|
||||
const userExists = await client.query('SELECT id FROM users WHERE id = $1', [userId]);
|
||||
if (userExists.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
|
||||
await client.query('BEGIN');
|
||||
|
||||
const updateFields = [];
|
||||
const updateValues = [userId];
|
||||
let paramIndex = 2;
|
||||
|
||||
if (username !== undefined) { updateFields.push(`username = $${paramIndex++}`); updateValues.push(username); }
|
||||
if (email !== undefined) { updateFields.push(`email = $${paramIndex++}`); updateValues.push(email || null); }
|
||||
if (is_admin !== undefined) { updateFields.push(`is_admin = $${paramIndex++}`); updateValues.push(!!is_admin); }
|
||||
if (is_active !== undefined) { updateFields.push(`is_active = $${paramIndex++}`); updateValues.push(!!is_active); }
|
||||
if (rocket_chat_user_id !== undefined) {
|
||||
updateFields.push(`rocket_chat_user_id = $${paramIndex++}`);
|
||||
updateValues.push(rocket_chat_user_id ? parseInt(rocket_chat_user_id, 10) : null);
|
||||
}
|
||||
if (password) {
|
||||
const hashedPassword = await bcrypt.hash(password, 10);
|
||||
updateFields.push(`password = $${paramIndex++}`);
|
||||
updateValues.push(hashedPassword);
|
||||
}
|
||||
|
||||
if (updateFields.length > 0) {
|
||||
updateFields.push(`updated_at = CURRENT_TIMESTAMP`);
|
||||
await client.query(`
|
||||
UPDATE users SET ${updateFields.join(', ')} WHERE id = $1
|
||||
`, updateValues);
|
||||
}
|
||||
|
||||
if (Array.isArray(permissions)) {
|
||||
await client.query('DELETE FROM user_permissions WHERE user_id = $1', [userId]);
|
||||
const newIsAdmin = is_admin !== undefined
|
||||
? is_admin
|
||||
: (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin;
|
||||
|
||||
if (!newIsAdmin && permissions.length > 0) {
|
||||
const permissionIds = normalizePermissionIds(permissions);
|
||||
if (permissionIds.length > 0) {
|
||||
await client.query(
|
||||
`INSERT INTO user_permissions (user_id, permission_id)
|
||||
SELECT $1, unnest($2::int[])
|
||||
ON CONFLICT DO NOTHING`,
|
||||
[userId, permissionIds]
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
await client.query('COMMIT');
|
||||
res.json({ message: 'User updated successfully' });
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK');
|
||||
console.error('Error updating user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
});
|
||||
|
||||
router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => {
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
if (req.user.id === parseInt(userId, 10)) {
|
||||
return res.status(400).json({ error: 'Cannot delete your own account' });
|
||||
}
|
||||
const result = await pool.query(
|
||||
'DELETE FROM users WHERE id = $1 RETURNING id',
|
||||
[userId]
|
||||
);
|
||||
console.log("Deleted existing permissions for user:", userId);
|
||||
|
||||
// Add new permissions if any and not admin
|
||||
const newIsAdmin = is_admin !== undefined ? is_admin : (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin;
|
||||
|
||||
console.log("User is admin:", newIsAdmin);
|
||||
|
||||
if (!newIsAdmin && permissions.length > 0) {
|
||||
console.log("Adding permissions:", permissions);
|
||||
|
||||
// Check permission format
|
||||
const permissionIds = permissions.map(p => {
|
||||
if (typeof p === 'object' && p.id) {
|
||||
console.log("Permission is an object with ID:", p.id);
|
||||
return parseInt(p.id, 10);
|
||||
} else if (typeof p === 'number') {
|
||||
console.log("Permission is a number:", p);
|
||||
return p;
|
||||
} else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
|
||||
console.log("Permission is a string that can be parsed as a number:", p);
|
||||
return parseInt(p, 10);
|
||||
} else {
|
||||
console.log("Unknown permission format:", typeof p, p);
|
||||
// If it's a permission code, we need to look up the ID
|
||||
return null;
|
||||
}
|
||||
}).filter(id => id !== null);
|
||||
|
||||
console.log("Filtered permission IDs:", permissionIds);
|
||||
|
||||
if (permissionIds.length > 0) {
|
||||
const permissionValues = permissionIds
|
||||
.map(permId => `(${userId}, ${permId})`)
|
||||
.join(',');
|
||||
|
||||
console.log("Inserting permission values:", permissionValues);
|
||||
|
||||
try {
|
||||
await client.query(`
|
||||
INSERT INTO user_permissions (user_id, permission_id)
|
||||
VALUES ${permissionValues}
|
||||
ON CONFLICT DO NOTHING
|
||||
`);
|
||||
console.log("Successfully inserted permissions for user:", userId);
|
||||
} catch (err) {
|
||||
console.error("Error inserting permissions:", err);
|
||||
throw err;
|
||||
}
|
||||
} else {
|
||||
console.log("No valid permission IDs found to insert");
|
||||
}
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
res.json({ message: 'User deleted successfully' });
|
||||
} catch (error) {
|
||||
console.error('Error deleting user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
await client.query('COMMIT');
|
||||
|
||||
res.json({ message: 'User updated successfully' });
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK');
|
||||
console.error('Error updating user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
});
|
||||
|
||||
// Delete user
|
||||
router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => {
|
||||
try {
|
||||
const userId = req.params.id;
|
||||
|
||||
// Check that user is not deleting themselves
|
||||
if (req.user.id === parseInt(userId, 10)) {
|
||||
return res.status(400).json({ error: 'Cannot delete your own account' });
|
||||
router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT category, json_agg(
|
||||
json_build_object(
|
||||
'id', id, 'name', name, 'code', code, 'description', description
|
||||
) ORDER BY name
|
||||
) as permissions
|
||||
FROM permissions
|
||||
GROUP BY category
|
||||
ORDER BY category
|
||||
`);
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting permissions:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Delete user (this will cascade to user_permissions due to FK constraints)
|
||||
const result = await pool.query(
|
||||
'DELETE FROM users WHERE id = $1 RETURNING id',
|
||||
[userId]
|
||||
);
|
||||
|
||||
if (result.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT * FROM permissions ORDER BY category, name
|
||||
`);
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting permissions:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
res.json({ message: 'User deleted successfully' });
|
||||
} catch (error) {
|
||||
console.error('Error deleting user:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
return router;
|
||||
}
|
||||
|
||||
// Get all permissions grouped by category
|
||||
router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT category, json_agg(
|
||||
json_build_object(
|
||||
'id', id,
|
||||
'name', name,
|
||||
'code', code,
|
||||
'description', description
|
||||
) ORDER BY name
|
||||
) as permissions
|
||||
FROM permissions
|
||||
GROUP BY category
|
||||
ORDER BY category
|
||||
`);
|
||||
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting permissions:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get all permissions
|
||||
router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => {
|
||||
try {
|
||||
const result = await pool.query(`
|
||||
SELECT *
|
||||
FROM permissions
|
||||
ORDER BY category, name
|
||||
`);
|
||||
|
||||
res.json(result.rows);
|
||||
} catch (error) {
|
||||
console.error('Error getting permissions:', error);
|
||||
res.status(500).json({ error: 'Server error' });
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
function normalizePermissionIds(permissions) {
|
||||
return permissions
|
||||
.map((p) => {
|
||||
if (typeof p === 'object' && p?.id) return parseInt(p.id, 10);
|
||||
if (typeof p === 'number') return p;
|
||||
if (typeof p === 'string' && !Number.isNaN(parseInt(p, 10))) return parseInt(p, 10);
|
||||
return null;
|
||||
})
|
||||
.filter((id) => id !== null && !Number.isNaN(id));
|
||||
}
|
||||
|
||||
+54
-165
@@ -1,195 +1,84 @@
|
||||
require('dotenv').config({ path: '../.env' });
|
||||
const express = require('express');
|
||||
const cors = require('cors');
|
||||
const bcrypt = require('bcrypt');
|
||||
const jwt = require('jsonwebtoken');
|
||||
const { Pool } = require('pg');
|
||||
const morgan = require('morgan');
|
||||
const authRoutes = require('./routes');
|
||||
import 'dotenv/config';
|
||||
import express from 'express';
|
||||
import cors from 'cors';
|
||||
import pg from 'pg';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// Log startup configuration
|
||||
console.log('Starting auth server with config:', {
|
||||
const { Pool } = pg;
|
||||
import { dirname, resolve as resolvePath } from 'node:path';
|
||||
import { config as loadEnv } from 'dotenv';
|
||||
|
||||
import { corsOptions } from '../shared/cors/policy.js';
|
||||
import { requestLog } from '../shared/logging/request-log.js';
|
||||
import { logger } from '../shared/logging/logger.js';
|
||||
import { errorHandler } from '../shared/errors/handler.js';
|
||||
import { loginLimiter, verifyLimiter } from '../shared/rate-limit/login.js';
|
||||
import { extractBearerToken, verifyToken, TokenError } from '../shared/auth/verify.js';
|
||||
|
||||
import { createAuthRoutes } from './routes.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
// auth/ lives at inventory-server/auth/, so .env one level up
|
||||
loadEnv({ path: resolvePath(__dirname, '../.env') });
|
||||
|
||||
if (!process.env.JWT_SECRET) {
|
||||
logger.error('JWT_SECRET is not set; refusing to start');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
logger.info({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT,
|
||||
auth_port: process.env.AUTH_PORT
|
||||
});
|
||||
auth_port: process.env.AUTH_PORT,
|
||||
}, 'starting auth server');
|
||||
|
||||
const app = express();
|
||||
const port = process.env.AUTH_PORT || 3011;
|
||||
const port = Number(process.env.AUTH_PORT) || 3011;
|
||||
|
||||
// Database configuration
|
||||
const pool = new Pool({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
password: process.env.DB_PASSWORD,
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT,
|
||||
port: Number(process.env.DB_PORT) || 5432,
|
||||
});
|
||||
|
||||
// Make pool available globally
|
||||
global.pool = pool;
|
||||
|
||||
// Middleware
|
||||
app.use(express.json());
|
||||
app.use(morgan('combined'));
|
||||
app.use(cors({
|
||||
origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://acot.site', 'https://tools.acherryontop.com'],
|
||||
credentials: true
|
||||
}));
|
||||
|
||||
// Login endpoint
|
||||
app.post('/login', async (req, res) => {
|
||||
const { username, password } = req.body;
|
||||
|
||||
try {
|
||||
// Get user from database
|
||||
const result = await pool.query(
|
||||
'SELECT id, username, password, is_admin, is_active FROM users WHERE username = $1',
|
||||
[username]
|
||||
);
|
||||
|
||||
const user = result.rows[0];
|
||||
|
||||
// Check if user exists and password is correct
|
||||
if (!user || !(await bcrypt.compare(password, user.password))) {
|
||||
return res.status(401).json({ error: 'Invalid username or password' });
|
||||
}
|
||||
|
||||
// Check if user is active
|
||||
if (!user.is_active) {
|
||||
return res.status(403).json({ error: 'Account is inactive' });
|
||||
}
|
||||
|
||||
// Update last login timestamp
|
||||
await pool.query(
|
||||
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
|
||||
[user.id]
|
||||
);
|
||||
|
||||
// Generate JWT token
|
||||
const token = jwt.sign(
|
||||
{ userId: user.id, username: user.username },
|
||||
process.env.JWT_SECRET,
|
||||
{ expiresIn: '24h' }
|
||||
);
|
||||
|
||||
// Get user permissions for the response
|
||||
const permissionsResult = await pool.query(`
|
||||
SELECT code
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1
|
||||
`, [user.id]);
|
||||
|
||||
const permissions = permissionsResult.rows.map(row => row.code);
|
||||
|
||||
res.json({
|
||||
token,
|
||||
user: {
|
||||
id: user.id,
|
||||
username: user.username,
|
||||
is_admin: user.is_admin,
|
||||
permissions: user.is_admin ? [] : permissions
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Login error:', error);
|
||||
res.status(500).json({ error: 'Internal server error' });
|
||||
}
|
||||
});
|
||||
|
||||
// User info endpoint
|
||||
app.get('/me', async (req, res) => {
|
||||
const authHeader = req.headers.authorization;
|
||||
|
||||
if (!authHeader || !authHeader.startsWith('Bearer ')) {
|
||||
return res.status(401).json({ error: 'No token provided' });
|
||||
}
|
||||
|
||||
try {
|
||||
const token = authHeader.split(' ')[1];
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
|
||||
// Get user details from database
|
||||
const userResult = await pool.query(
|
||||
'SELECT id, username, email, is_admin, rocket_chat_user_id, is_active FROM users WHERE id = $1',
|
||||
[decoded.userId]
|
||||
);
|
||||
|
||||
if (userResult.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'User not found' });
|
||||
}
|
||||
|
||||
const user = userResult.rows[0];
|
||||
|
||||
// Check if user is active
|
||||
if (!user.is_active) {
|
||||
return res.status(403).json({ error: 'Account is inactive' });
|
||||
}
|
||||
|
||||
// Get user permissions
|
||||
let permissions = [];
|
||||
if (!user.is_admin) {
|
||||
const permissionsResult = await pool.query(`
|
||||
SELECT code
|
||||
FROM permissions p
|
||||
JOIN user_permissions up ON p.id = up.permission_id
|
||||
WHERE up.user_id = $1
|
||||
`, [user.id]);
|
||||
|
||||
permissions = permissionsResult.rows.map(row => row.code);
|
||||
}
|
||||
|
||||
res.json({
|
||||
id: user.id,
|
||||
username: user.username,
|
||||
email: user.email,
|
||||
rocket_chat_user_id: user.rocket_chat_user_id,
|
||||
is_admin: user.is_admin,
|
||||
permissions: permissions
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Token verification error:', error);
|
||||
res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
});
|
||||
app.use(requestLog());
|
||||
app.use(express.json({ limit: '1mb' }));
|
||||
app.use(cors(corsOptions));
|
||||
|
||||
// Caddy forward_auth target: JWT signature check only, no DB hit.
|
||||
// Returns 200 with X-User-Id / X-User-Username on success, 401 otherwise.
|
||||
// Per-service middleware re-verifies the token independently; these headers
|
||||
// are informational and must not be trusted by upstreams.
|
||||
app.all('/verify', (req, res) => {
|
||||
const header = req.headers.authorization;
|
||||
if (!header || !header.startsWith('Bearer ')) {
|
||||
return res.status(401).json({ error: 'No token provided' });
|
||||
}
|
||||
// Returns 200 with X-User-Id / X-User-Username on success; 401 otherwise.
|
||||
// Per-service middleware re-verifies independently; these headers are informational.
|
||||
app.all('/verify', verifyLimiter, (req, res) => {
|
||||
try {
|
||||
const decoded = jwt.verify(header.slice(7), process.env.JWT_SECRET);
|
||||
const token = extractBearerToken(req.headers.authorization);
|
||||
const decoded = verifyToken(token, process.env.JWT_SECRET);
|
||||
res.set('X-User-Id', String(decoded.userId));
|
||||
if (decoded.username) res.set('X-User-Username', decoded.username);
|
||||
res.status(200).end();
|
||||
} catch (err) {
|
||||
res.status(401).json({ error: err.name === 'TokenExpiredError' ? 'Token expired' : 'Invalid token' });
|
||||
if (err instanceof TokenError) {
|
||||
return res.status(401).json({ error: err.message });
|
||||
}
|
||||
res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
});
|
||||
|
||||
// Mount all routes from routes.js
|
||||
app.use('/', authRoutes);
|
||||
// Login route gets its own rate limiter to slow credential stuffing.
|
||||
app.use('/login', loginLimiter);
|
||||
|
||||
// Health check endpoint
|
||||
app.get('/health', (req, res) => {
|
||||
res.json({ status: 'healthy' });
|
||||
});
|
||||
// Mount user-management + /login + /me from routes.js
|
||||
app.use('/', createAuthRoutes({ pool }));
|
||||
|
||||
// Error handling middleware
|
||||
app.use((err, req, res, next) => {
|
||||
console.error(err.stack);
|
||||
res.status(500).json({ error: 'Something broke!' });
|
||||
});
|
||||
app.get('/health', (req, res) => res.json({ status: 'healthy' }));
|
||||
|
||||
app.use(errorHandler);
|
||||
|
||||
// Start server
|
||||
app.listen(port, () => {
|
||||
console.log(`Auth server running on port ${port}`);
|
||||
logger.info({ port }, 'auth server listening');
|
||||
});
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
# Phase 6.1 + 6.6 + 6.7: tools.acherryontop.com final form
|
||||
#
|
||||
# Apply on the server with:
|
||||
# curl -X POST http://localhost:2020/load \
|
||||
# -H 'Content-Type: text/caddyfile' \
|
||||
# --data-binary @/home/matt/Caddyfile.new
|
||||
# sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile
|
||||
# sudo cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.bak.$(date +%F)
|
||||
#
|
||||
# Differences from current /etc/caddy/Caddyfile:
|
||||
# 1. forward_auth gate added in front of /api/* and /chat-api/* (Phase 6.1).
|
||||
# 2. /uploads/* moved behind the forward_auth gate (Phase 6.7 — was public).
|
||||
# 3. LAN wildcards / Access-Control-Allow-Origin "*" defaults dropped from /api/* (Phase 6.6).
|
||||
# 4. Removed dead /api/{aircall,gorgias,clarity}/* routes (Phase 1 — already cleaned up here).
|
||||
#
|
||||
# Phase 4 (dashboard-server merge) is NOT yet reflected — klaviyo/meta/google/typeform
|
||||
# still route to their per-vendor PM2 apps in the live Caddyfile. Update those handle
|
||||
# blocks to localhost:3015 when dashboard-server ships.
|
||||
|
||||
tools.acherryontop.com {
|
||||
import security_headers
|
||||
|
||||
# Public: login endpoint
|
||||
handle /auth-inv/* {
|
||||
uri strip_prefix /auth-inv
|
||||
reverse_proxy localhost:3011
|
||||
}
|
||||
|
||||
# Public: static frontend assets (long-cache)
|
||||
@static path *.js *.css *.png *.jpg *.jpeg *.gif *.ico *.svg *.woff *.woff2
|
||||
handle @static {
|
||||
header Cache-Control "public, max-age=2592000"
|
||||
root * /var/www/inventory/frontend/build
|
||||
file_server
|
||||
}
|
||||
|
||||
# ----- Authenticated zone -----
|
||||
# Phase 6.1: forward_auth subrequest to auth-server:/verify. 2xx → proceeds.
|
||||
# 401/403 → Caddy returns auth-server response to client; backend never sees it.
|
||||
@gated path /api/* /chat-api/* /uploads/*
|
||||
handle @gated {
|
||||
forward_auth localhost:3011 {
|
||||
uri /verify
|
||||
copy_headers Authorization
|
||||
}
|
||||
|
||||
# Phase 6.7: /uploads/* now behind the gate (was a public file_server before)
|
||||
handle /uploads/* {
|
||||
root * /var/www/inventory
|
||||
file_server
|
||||
}
|
||||
|
||||
# Vendor dashboard routes
|
||||
# NOTE: pre-Phase-4 these are still on separate ports; updates here when merged.
|
||||
handle /api/klaviyo/* { reverse_proxy localhost:3015 }
|
||||
handle /api/meta/* { reverse_proxy localhost:3015 }
|
||||
handle /api/google-analytics/* { reverse_proxy localhost:3015 }
|
||||
handle /api/typeform/* { reverse_proxy localhost:3015 }
|
||||
|
||||
# ACOT
|
||||
handle /api/acot/* { reverse_proxy localhost:3012 }
|
||||
|
||||
# Chat
|
||||
handle /chat-api/* {
|
||||
uri strip_prefix /chat-api
|
||||
reverse_proxy localhost:3014
|
||||
}
|
||||
|
||||
# Catch-all: inventory-server
|
||||
handle /api/* { reverse_proxy localhost:3010 }
|
||||
}
|
||||
|
||||
# Out-of-band probes (unauthenticated)
|
||||
handle /health { reverse_proxy localhost:3010 }
|
||||
|
||||
# SPA fallback (public assets)
|
||||
handle {
|
||||
root * /var/www/inventory/frontend/build
|
||||
try_files {path} /index.html
|
||||
file_server
|
||||
encode gzip
|
||||
}
|
||||
|
||||
handle_errors {
|
||||
respond "{err.status_code} {err.status_text}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
# Server-side deployment artefacts for Phase 3 + 6
|
||||
|
||||
This directory contains proposed versions of files that live outside the
|
||||
inventory-server tree on production. Each is a recommendation — apply
|
||||
deliberately and only after the Node-side ESM + auth changes are deployed and
|
||||
smoke-tested.
|
||||
|
||||
| Source | Target | Phase |
|
||||
| --------------------------------------- | ------------------------------------- | -------- |
|
||||
| `Caddyfile.proposed` | `/etc/caddy/Caddyfile` | 6.1, 6.6, 6.7 |
|
||||
| `ecosystem.config.cjs.proposed` | `/var/www/ecosystem.config.cjs` | 6.4, 6.10 |
|
||||
|
||||
## Recommended apply order
|
||||
|
||||
1. **Deploy the Node code first** (this repo). PM2 reload picks up the new
|
||||
ESM-mode inventory-server and auth-server. At this point the frontend will
|
||||
start hitting 401s on every API call because the new `authenticate()`
|
||||
middleware is live and the frontend doesn't carry Bearer tokens on most
|
||||
fetches. **This is expected per the discussion in CONSOLIDATION_PLAN.md
|
||||
§6** — the frontend fetch-wrapper work is the next deliverable.
|
||||
|
||||
2. **Apply the ecosystem.cjs change** (Phase 6.4) to fix the `JWT_SECRET`
|
||||
shadow-override before the next pm2 restart silently re-introduces it.
|
||||
|
||||
3. **Apply the Caddyfile change** (Phase 6.1) only after the frontend is
|
||||
sending Bearer tokens. Until then, `forward_auth` will reject every page
|
||||
refresh at the edge.
|
||||
|
||||
## Caddyfile apply pattern
|
||||
|
||||
Caddy admin API is on `:2020` (matt has access). On-disk file needs root.
|
||||
|
||||
```bash
|
||||
# Upload + load atomically into the running Caddy
|
||||
curl -X POST http://localhost:2020/load \
|
||||
-H 'Content-Type: text/caddyfile' \
|
||||
--data-binary @/home/matt/Caddyfile.new
|
||||
|
||||
# Persist to disk (separate sudo step)
|
||||
sudo cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.bak.$(date +%F)
|
||||
sudo cp /home/matt/Caddyfile.new /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
## ecosystem.cjs apply pattern
|
||||
|
||||
```bash
|
||||
sudo cp /var/www/ecosystem.config.cjs /var/www/ecosystem.config.cjs.bak.$(date +%F)
|
||||
sudo cp /home/matt/ecosystem.config.cjs.new /var/www/ecosystem.config.cjs
|
||||
pm2 reload ecosystem.config.cjs --update-env
|
||||
pm2 env new-auth-server | grep -i jwt # JWT_SECRET from .env only
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
Every applied file leaves a `.bak.YYYY-MM-DD` next to it. `sudo cp <bak>
|
||||
<original>` then `caddy reload` / `pm2 reload`.
|
||||
|
||||
Phase 6 changes are *additive* — if `forward_auth` causes problems, comment
|
||||
out the directive in the live Caddyfile and per-server middleware
|
||||
(`authenticate()` in inventory-server, in particular) continues protecting
|
||||
routes.
|
||||
@@ -0,0 +1,114 @@
|
||||
// Phase 6.4 + 6.10: proposed /var/www/ecosystem.config.cjs
|
||||
//
|
||||
// Diffs from the live file (CONSOLIDATION_PLAN.md §6.4 / §6.10):
|
||||
// 1. Drop the `JWT_SECRET: process.env.JWT_SECRET` override in new-auth-server's
|
||||
// env block. That override shadowed the .env value with whatever shell var was
|
||||
// exported when pm2 was last started — causing the silent-divergence footgun
|
||||
// called out in CLAUDE.md memory. With it removed, .env is the single source.
|
||||
// 2. Move ADD_WORD_TOKEN out of the inline env into /opt/lt-wordlist-api/.env.
|
||||
// The PM2 entry below no longer references it.
|
||||
// 3. Rename the placeholder script paths if you have any half-finished ones; this
|
||||
// version only lists the apps that actually run today (post-Phase-1 cleanup).
|
||||
//
|
||||
// To apply:
|
||||
// sudo cp /var/www/ecosystem.config.cjs /var/www/ecosystem.config.cjs.bak.$(date +%F)
|
||||
// sudo cp /home/matt/ecosystem.config.cjs.new /var/www/ecosystem.config.cjs
|
||||
// pm2 reload ecosystem.config.cjs --update-env
|
||||
//
|
||||
// Verify after reload:
|
||||
// pm2 env new-auth-server | grep -i jwt # should show JWT_SECRET from .env only
|
||||
// pm2 env new-auth-server | grep ADD_WORD # should be empty
|
||||
|
||||
const inventoryEnv = require('dotenv').config({ path: '/var/www/inventory/.env' }).parsed;
|
||||
|
||||
module.exports = {
|
||||
apps: [
|
||||
{
|
||||
name: 'new-auth-server', // Phase 8 may rename to 'auth-server' — cosmetic
|
||||
script: './inventory/auth/server.js',
|
||||
cwd: '/var/www',
|
||||
env: {
|
||||
...inventoryEnv,
|
||||
NODE_ENV: 'production',
|
||||
AUTH_PORT: 3011,
|
||||
// PHASE 6.4 FIX: no JWT_SECRET override here. .env wins.
|
||||
},
|
||||
max_memory_restart: '500M',
|
||||
error_file: '/var/log/pm2/new-auth-server-error.log',
|
||||
out_file: '/var/log/pm2/new-auth-server-out.log',
|
||||
},
|
||||
{
|
||||
name: 'inventory-server',
|
||||
script: './inventory/src/server.js',
|
||||
cwd: '/var/www',
|
||||
env: {
|
||||
...inventoryEnv,
|
||||
NODE_ENV: 'production',
|
||||
PORT: 3010,
|
||||
UPLOADS_DIR: '/var/www/inventory/uploads',
|
||||
},
|
||||
max_memory_restart: '1G',
|
||||
error_file: '/var/log/pm2/inventory-server-error.log',
|
||||
out_file: '/var/log/pm2/inventory-server-out.log',
|
||||
},
|
||||
{
|
||||
name: 'chat-server',
|
||||
script: './inventory/chat/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production', PORT: 3014 },
|
||||
max_memory_restart: '500M',
|
||||
},
|
||||
{
|
||||
name: 'acot-server',
|
||||
script: './inventory/dashboard/acot-server/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production', ACOT_PORT: 3012 },
|
||||
max_memory_restart: '1G',
|
||||
},
|
||||
// Per-vendor dashboard apps stay until Phase 4 merge ships.
|
||||
{
|
||||
name: 'klaviyo-server',
|
||||
script: './inventory/dashboard/klaviyo-server/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production' },
|
||||
max_memory_restart: '500M',
|
||||
},
|
||||
{
|
||||
name: 'meta-server',
|
||||
script: './inventory/dashboard/meta-server/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production' },
|
||||
max_memory_restart: '300M',
|
||||
},
|
||||
{
|
||||
name: 'google-server',
|
||||
script: './inventory/dashboard/google-server/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production' },
|
||||
max_memory_restart: '300M',
|
||||
},
|
||||
{
|
||||
name: 'typeform-server',
|
||||
script: './inventory/dashboard/typeform-server/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production' },
|
||||
max_memory_restart: '300M',
|
||||
},
|
||||
// PHASE 6.10: lt-wordlist-api now loads ADD_WORD_TOKEN from /opt/lt-wordlist-api/.env
|
||||
// (no longer hardcoded here). Rotate that token's value when applying this change.
|
||||
{
|
||||
name: 'lt-wordlist-api',
|
||||
script: '/opt/lt-wordlist-api/server.js',
|
||||
cwd: '/opt/lt-wordlist-api',
|
||||
env: { NODE_ENV: 'production' },
|
||||
max_memory_restart: '200M',
|
||||
},
|
||||
{
|
||||
name: 'acot-phone-server',
|
||||
script: './inventory/acot-phone/server.js',
|
||||
cwd: '/var/www',
|
||||
env: { ...inventoryEnv, NODE_ENV: 'production' },
|
||||
max_memory_restart: '300M',
|
||||
},
|
||||
],
|
||||
};
|
||||
@@ -0,0 +1,52 @@
|
||||
-- Phase 6.2: per-route permission codes
|
||||
-- Seeds the permission codes referenced by Phase 6 hardening middleware.
|
||||
-- Safe to run multiple times (ON CONFLICT DO NOTHING).
|
||||
--
|
||||
-- Codes follow the plan's spec (CONSOLIDATION_PLAN.md §6.2):
|
||||
-- product_import — POST/PUT/DELETE on /api/import
|
||||
-- data_management — POST/PUT/DELETE on /api/csv (data-management.js)
|
||||
-- ai_admin — POST/PUT/DELETE on /api/ai-prompts, /api/ai-validation
|
||||
-- templates_write — POST/PUT/DELETE on /api/templates
|
||||
-- image_admin — POST/DELETE on /api/reusable-images
|
||||
-- audit_read — reserved for future read-gating on audit logs
|
||||
-- acot_admin — reserved for acot-server (Phase 5 scope)
|
||||
-- klaviyo_* / meta_* / google_* / typeform_* — reserved for dashboard-server (Phase 4 scope)
|
||||
--
|
||||
-- Admin users (is_admin = true) automatically pass any requirePermission() check,
|
||||
-- so this migration does not auto-grant codes to admins. New non-admin users get
|
||||
-- write access only when explicitly granted via the user-management UI.
|
||||
|
||||
INSERT INTO permissions (code, name, category, description) VALUES
|
||||
('product_import', 'Product Import (write)', 'Imports',
|
||||
'Allows POST/PUT/DELETE on /api/import — uploads, deletes, generate-upc, etc.'),
|
||||
('data_management', 'Data Management (write)', 'Data',
|
||||
'Allows POST/PUT/DELETE on /api/csv — CSV operations, full updates, full resets.'),
|
||||
('ai_admin', 'AI Settings Admin', 'AI',
|
||||
'Allows write access to AI prompts and AI validation endpoints.'),
|
||||
('templates_write', 'Template Editing', 'Templates',
|
||||
'Allows POST/PUT/DELETE on /api/templates.'),
|
||||
('image_admin', 'Image Management', 'Images',
|
||||
'Allows uploads and deletions on /api/reusable-images.'),
|
||||
('audit_read', 'Audit Log Access', 'Audit',
|
||||
'Reserved for future read-gating of import + product-editor audit logs.'),
|
||||
('klaviyo_write', 'Klaviyo Write', 'Dashboard',
|
||||
'Reserved for dashboard-server: mutates Klaviyo lists/segments.'),
|
||||
('klaviyo_admin', 'Klaviyo Admin', 'Dashboard',
|
||||
'Reserved for dashboard-server: triggers campaign syncs.'),
|
||||
('meta_write', 'Meta Write', 'Dashboard',
|
||||
'Reserved for dashboard-server: Meta API write operations.'),
|
||||
('google_write', 'Google Analytics Write', 'Dashboard',
|
||||
'Reserved for dashboard-server: GA write operations.'),
|
||||
('typeform_write', 'Typeform Write', 'Dashboard',
|
||||
'Reserved for dashboard-server: Typeform write operations.'),
|
||||
('acot_admin', 'ACOT Server Admin', 'ACOT',
|
||||
'Reserved for acot-server admin endpoints.')
|
||||
ON CONFLICT (code) DO NOTHING;
|
||||
|
||||
-- Phase 2 deviation #6 cleanup: drop defunct frontend permissions if present.
|
||||
-- These corresponded to the removed Aircall/Gorgias dashboards.
|
||||
DELETE FROM user_permissions
|
||||
WHERE permission_id IN (
|
||||
SELECT id FROM permissions WHERE code IN ('dashboard:gorgias', 'dashboard:calls')
|
||||
);
|
||||
DELETE FROM permissions WHERE code IN ('dashboard:gorgias', 'dashboard:calls');
|
||||
Generated
+367
@@ -18,10 +18,15 @@
|
||||
"diff": "^7.0.0",
|
||||
"dotenv": "^16.4.7",
|
||||
"express": "^4.18.2",
|
||||
"express-rate-limit": "^7.4.0",
|
||||
"ioredis": "^5.10.1",
|
||||
"jsonwebtoken": "^9.0.2",
|
||||
"multer": "^1.4.5-lts.1",
|
||||
"mysql2": "^3.12.0",
|
||||
"openai": "^6.0.0",
|
||||
"pg": "^8.14.1",
|
||||
"pino": "^9.5.0",
|
||||
"pino-http": "^10.3.0",
|
||||
"pm2": "^5.3.0",
|
||||
"sharp": "^0.33.5",
|
||||
"ssh2": "^1.16.0",
|
||||
@@ -409,6 +414,12 @@
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@ioredis/commands": {
|
||||
"version": "1.5.1",
|
||||
"resolved": "https://registry.npmjs.org/@ioredis/commands/-/commands-1.5.1.tgz",
|
||||
"integrity": "sha512-JH8ZL/ywcJyR9MmJ5BNqZllXNZQqQbnVZOqpPQqE1vHiFgAw4NHbvE0FOduNU8IX9babitBT46571OnPTT0Zcw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@mapbox/node-pre-gyp": {
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/@mapbox/node-pre-gyp/-/node-pre-gyp-1.0.11.tgz",
|
||||
@@ -477,6 +488,12 @@
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@pinojs/redact": {
|
||||
"version": "0.4.0",
|
||||
"resolved": "https://registry.npmjs.org/@pinojs/redact/-/redact-0.4.0.tgz",
|
||||
"integrity": "sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@pm2/agent": {
|
||||
"version": "2.0.4",
|
||||
"resolved": "https://registry.npmjs.org/@pm2/agent/-/agent-2.0.4.tgz",
|
||||
@@ -958,6 +975,15 @@
|
||||
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/atomic-sleep": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/atomic-sleep/-/atomic-sleep-1.0.0.tgz",
|
||||
"integrity": "sha512-kNOjDqAh7px0XWNI+4QbzoiR/nTkHAWNud2uvnJquD1/x5a7EQZMJT0AczqK0Qn67oY/TTQ1LbUKajZpp3I9tQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=8.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/aws-ssl-profiles": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/aws-ssl-profiles/-/aws-ssl-profiles-1.1.2.tgz",
|
||||
@@ -1092,6 +1118,12 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/buffer-equal-constant-time": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
|
||||
"integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==",
|
||||
"license": "BSD-3-Clause"
|
||||
},
|
||||
"node_modules/buffer-from": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
|
||||
@@ -1219,6 +1251,15 @@
|
||||
"node": ">=8.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/cluster-key-slot": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
|
||||
"integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/color": {
|
||||
"version": "4.2.3",
|
||||
"resolved": "https://registry.npmjs.org/color/-/color-4.2.3.tgz",
|
||||
@@ -1529,6 +1570,15 @@
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/ecdsa-sig-formatter": {
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/ecdsa-sig-formatter/-/ecdsa-sig-formatter-1.0.11.tgz",
|
||||
"integrity": "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"safe-buffer": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/ee-first": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
|
||||
@@ -1738,6 +1788,21 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/express-rate-limit": {
|
||||
"version": "7.5.1",
|
||||
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-7.5.1.tgz",
|
||||
"integrity": "sha512-7iN8iPMDzOMHPUYllBEsQdWVB6fPDMPqwjBaFrgr4Jgr/+okjvzAy+UHlYYL/Vs0OsOrMkwS6PJDkFlJwoxUnw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 16"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/express-rate-limit"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"express": ">= 4.11"
|
||||
}
|
||||
},
|
||||
"node_modules/extrareqp2": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/extrareqp2/-/extrareqp2-1.0.0.tgz",
|
||||
@@ -1912,6 +1977,15 @@
|
||||
"is-property": "^1.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/get-caller-file": {
|
||||
"version": "2.0.5",
|
||||
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
|
||||
"integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": "6.* || 8.* || >= 10.*"
|
||||
}
|
||||
},
|
||||
"node_modules/get-intrinsic": {
|
||||
"version": "1.2.7",
|
||||
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.7.tgz",
|
||||
@@ -2227,6 +2301,53 @@
|
||||
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/ioredis": {
|
||||
"version": "5.10.1",
|
||||
"resolved": "https://registry.npmjs.org/ioredis/-/ioredis-5.10.1.tgz",
|
||||
"integrity": "sha512-HuEDBTI70aYdx1v6U97SbNx9F1+svQKBDo30o0b9fw055LMepzpOOd0Ccg9Q6tbqmBSJaMuY0fB7yw9/vjBYCA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@ioredis/commands": "1.5.1",
|
||||
"cluster-key-slot": "^1.1.0",
|
||||
"debug": "^4.3.4",
|
||||
"denque": "^2.1.0",
|
||||
"lodash.defaults": "^4.2.0",
|
||||
"lodash.isarguments": "^3.1.0",
|
||||
"redis-errors": "^1.2.0",
|
||||
"redis-parser": "^3.0.0",
|
||||
"standard-as-callback": "^2.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12.22.0"
|
||||
},
|
||||
"funding": {
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/ioredis"
|
||||
}
|
||||
},
|
||||
"node_modules/ioredis/node_modules/debug": {
|
||||
"version": "4.4.3",
|
||||
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
|
||||
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"ms": "^2.1.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"supports-color": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/ioredis/node_modules/ms": {
|
||||
"version": "2.1.3",
|
||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/ip-address": {
|
||||
"version": "9.0.5",
|
||||
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-9.0.5.tgz",
|
||||
@@ -2376,6 +2497,55 @@
|
||||
"license": "ISC",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/jsonwebtoken": {
|
||||
"version": "9.0.3",
|
||||
"resolved": "https://registry.npmjs.org/jsonwebtoken/-/jsonwebtoken-9.0.3.tgz",
|
||||
"integrity": "sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"jws": "^4.0.1",
|
||||
"lodash.includes": "^4.3.0",
|
||||
"lodash.isboolean": "^3.0.3",
|
||||
"lodash.isinteger": "^4.0.4",
|
||||
"lodash.isnumber": "^3.0.3",
|
||||
"lodash.isplainobject": "^4.0.6",
|
||||
"lodash.isstring": "^4.0.1",
|
||||
"lodash.once": "^4.0.0",
|
||||
"ms": "^2.1.1",
|
||||
"semver": "^7.5.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12",
|
||||
"npm": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/jsonwebtoken/node_modules/ms": {
|
||||
"version": "2.1.3",
|
||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/jwa": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/jwa/-/jwa-2.0.1.tgz",
|
||||
"integrity": "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"buffer-equal-constant-time": "^1.0.1",
|
||||
"ecdsa-sig-formatter": "1.0.11",
|
||||
"safe-buffer": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/jws": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/jws/-/jws-4.0.1.tgz",
|
||||
"integrity": "sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"jwa": "^2.0.1",
|
||||
"safe-buffer": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/lazy": {
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/lazy/-/lazy-1.0.11.tgz",
|
||||
@@ -2391,6 +2561,60 @@
|
||||
"integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.defaults": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
|
||||
"integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.includes": {
|
||||
"version": "4.3.0",
|
||||
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
|
||||
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isarguments": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
|
||||
"integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isboolean": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
|
||||
"integrity": "sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isinteger": {
|
||||
"version": "4.0.4",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isinteger/-/lodash.isinteger-4.0.4.tgz",
|
||||
"integrity": "sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isnumber": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isnumber/-/lodash.isnumber-3.0.3.tgz",
|
||||
"integrity": "sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isplainobject": {
|
||||
"version": "4.0.6",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isplainobject/-/lodash.isplainobject-4.0.6.tgz",
|
||||
"integrity": "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.isstring": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/lodash.isstring/-/lodash.isstring-4.0.1.tgz",
|
||||
"integrity": "sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lodash.once": {
|
||||
"version": "4.1.1",
|
||||
"resolved": "https://registry.npmjs.org/lodash.once/-/lodash.once-4.1.1.tgz",
|
||||
"integrity": "sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/long": {
|
||||
"version": "5.2.3",
|
||||
"resolved": "https://registry.npmjs.org/long/-/long-5.2.3.tgz",
|
||||
@@ -2910,6 +3134,15 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/on-exit-leak-free": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/on-exit-leak-free/-/on-exit-leak-free-2.1.2.tgz",
|
||||
"integrity": "sha512-0eJJY6hXLGf1udHwfNftBqH+g73EU4B504nZeKpz1sYRKafAghwxEJunB2O7rDZkL4PGfsMVnTXZ2EjibbqcsA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/on-finished": {
|
||||
"version": "2.4.1",
|
||||
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
|
||||
@@ -3156,6 +3389,55 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/pino": {
|
||||
"version": "9.14.0",
|
||||
"resolved": "https://registry.npmjs.org/pino/-/pino-9.14.0.tgz",
|
||||
"integrity": "sha512-8OEwKp5juEvb/MjpIc4hjqfgCNysrS94RIOMXYvpYCdm/jglrKEiAYmiumbmGhCvs+IcInsphYDFwqrjr7398w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@pinojs/redact": "^0.4.0",
|
||||
"atomic-sleep": "^1.0.0",
|
||||
"on-exit-leak-free": "^2.1.0",
|
||||
"pino-abstract-transport": "^2.0.0",
|
||||
"pino-std-serializers": "^7.0.0",
|
||||
"process-warning": "^5.0.0",
|
||||
"quick-format-unescaped": "^4.0.3",
|
||||
"real-require": "^0.2.0",
|
||||
"safe-stable-stringify": "^2.3.1",
|
||||
"sonic-boom": "^4.0.1",
|
||||
"thread-stream": "^3.0.0"
|
||||
},
|
||||
"bin": {
|
||||
"pino": "bin.js"
|
||||
}
|
||||
},
|
||||
"node_modules/pino-abstract-transport": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/pino-abstract-transport/-/pino-abstract-transport-2.0.0.tgz",
|
||||
"integrity": "sha512-F63x5tizV6WCh4R6RHyi2Ml+M70DNRXt/+HANowMflpgGFMAym/VKm6G7ZOQRjqN7XbGxK1Lg9t6ZrtzOaivMw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"split2": "^4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/pino-http": {
|
||||
"version": "10.5.0",
|
||||
"resolved": "https://registry.npmjs.org/pino-http/-/pino-http-10.5.0.tgz",
|
||||
"integrity": "sha512-hD91XjgaKkSsdn8P7LaebrNzhGTdB086W3pyPihX0EzGPjq5uBJBXo4N5guqNaK6mUjg9aubMF7wDViYek9dRA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"get-caller-file": "^2.0.5",
|
||||
"pino": "^9.0.0",
|
||||
"pino-std-serializers": "^7.0.0",
|
||||
"process-warning": "^5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/pino-std-serializers": {
|
||||
"version": "7.1.0",
|
||||
"resolved": "https://registry.npmjs.org/pino-std-serializers/-/pino-std-serializers-7.1.0.tgz",
|
||||
"integrity": "sha512-BndPH67/JxGExRgiX1dX0w1FvZck5Wa4aal9198SrRhZjH3GxKQUKIBnYJTdj2HDN3UQAS06HlfcSbQj2OHmaw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/pm2": {
|
||||
"version": "5.4.3",
|
||||
"resolved": "https://registry.npmjs.org/pm2/-/pm2-5.4.3.tgz",
|
||||
@@ -3426,6 +3708,22 @@
|
||||
"integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/process-warning": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/process-warning/-/process-warning-5.0.0.tgz",
|
||||
"integrity": "sha512-a39t9ApHNx2L4+HBnQKqxxHNs1r7KF+Intd8Q/g1bUh6q0WIp9voPXJ/x0j+ZL45KF1pJd9+q2jLIRMfvEshkA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fastify"
|
||||
},
|
||||
{
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/fastify"
|
||||
}
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/promptly": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/promptly/-/promptly-2.2.0.tgz",
|
||||
@@ -3518,6 +3816,12 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/quick-format-unescaped": {
|
||||
"version": "4.0.4",
|
||||
"resolved": "https://registry.npmjs.org/quick-format-unescaped/-/quick-format-unescaped-4.0.4.tgz",
|
||||
"integrity": "sha512-tYC1Q1hgyRuHgloV/YXs2w15unPVh8qfu/qCTfhTYamaw7fyhumKa2yGpdSo87vY32rIclj+4fWYQXUMs9EHvg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/range-parser": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz",
|
||||
@@ -3587,6 +3891,36 @@
|
||||
"node": ">=8.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/real-require": {
|
||||
"version": "0.2.0",
|
||||
"resolved": "https://registry.npmjs.org/real-require/-/real-require-0.2.0.tgz",
|
||||
"integrity": "sha512-57frrGM/OCTLqLOAh0mhVA9VBMHd+9U7Zb2THMGdBUoZVOtGbJzjxsYGDJ3A9AYYCP4hn6y1TVbaOfzWtm5GFg==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 12.13.0"
|
||||
}
|
||||
},
|
||||
"node_modules/redis-errors": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
|
||||
"integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/redis-parser": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
|
||||
"integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"redis-errors": "^1.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/require-in-the-middle": {
|
||||
"version": "5.2.0",
|
||||
"resolved": "https://registry.npmjs.org/require-in-the-middle/-/require-in-the-middle-5.2.0.tgz",
|
||||
@@ -3700,6 +4034,15 @@
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/safe-stable-stringify": {
|
||||
"version": "2.5.0",
|
||||
"resolved": "https://registry.npmjs.org/safe-stable-stringify/-/safe-stable-stringify-2.5.0.tgz",
|
||||
"integrity": "sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/safer-buffer": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
|
||||
@@ -4001,6 +4344,15 @@
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/sonic-boom": {
|
||||
"version": "4.2.1",
|
||||
"resolved": "https://registry.npmjs.org/sonic-boom/-/sonic-boom-4.2.1.tgz",
|
||||
"integrity": "sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"atomic-sleep": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/source-map": {
|
||||
"version": "0.6.1",
|
||||
"resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz",
|
||||
@@ -4061,6 +4413,12 @@
|
||||
"nan": "^2.20.0"
|
||||
}
|
||||
},
|
||||
"node_modules/standard-as-callback": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
|
||||
"integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/statuses": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz",
|
||||
@@ -4187,6 +4545,15 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/thread-stream": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/thread-stream/-/thread-stream-3.1.0.tgz",
|
||||
"integrity": "sha512-OqyPZ9u96VohAyMfJykzmivOrY2wfMSf3C5TtFJVgN+Hm6aj+voFhlK+kZEIv2FBh1X6Xp3DlnCOfEQ3B2J86A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"real-require": "^0.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/to-regex-range": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
"name": "inventory-server",
|
||||
"version": "1.0.0",
|
||||
"description": "Backend server for inventory management system",
|
||||
"type": "module",
|
||||
"main": "src/server.js",
|
||||
"scripts": {
|
||||
"start": "node src/server.js",
|
||||
@@ -27,10 +28,15 @@
|
||||
"diff": "^7.0.0",
|
||||
"dotenv": "^16.4.7",
|
||||
"express": "^4.18.2",
|
||||
"express-rate-limit": "^7.4.0",
|
||||
"ioredis": "^5.10.1",
|
||||
"jsonwebtoken": "^9.0.2",
|
||||
"multer": "^1.4.5-lts.1",
|
||||
"mysql2": "^3.12.0",
|
||||
"openai": "^6.0.0",
|
||||
"pg": "^8.14.1",
|
||||
"pino": "^9.5.0",
|
||||
"pino-http": "^10.3.0",
|
||||
"pm2": "^5.3.0",
|
||||
"sharp": "^0.33.5",
|
||||
"ssh2": "^1.16.0",
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "inventory-server-scripts",
|
||||
"private": true,
|
||||
"type": "commonjs",
|
||||
"description": "One-shot maintenance scripts (imports, metrics, forecasting orchestration). Kept CommonJS so the existing `require()` graph still works after the parent server moved to ESM. Convert individual scripts when touching them."
|
||||
}
|
||||
@@ -10,7 +10,7 @@ export function createPool(envPrefix = 'DB', overrides = {}) {
|
||||
user: overrides.user ?? get('USER'),
|
||||
password: overrides.password ?? get('PASSWORD'),
|
||||
database: overrides.database ?? get('NAME'),
|
||||
port: overrides.port ?? Number(get('PORT')) || 5432,
|
||||
port: overrides.port ?? (Number(get('PORT')) || 5432),
|
||||
ssl: (overrides.ssl ?? get('SSL')) === 'true' ? { rejectUnauthorized: false } : false,
|
||||
max: overrides.max ?? 20,
|
||||
idleTimeoutMillis: overrides.idleTimeoutMillis ?? 30_000,
|
||||
|
||||
@@ -19,7 +19,7 @@ export function createRedis(overrides = {}) {
|
||||
|
||||
return new Redis({
|
||||
host: overrides.host ?? process.env.REDIS_HOST ?? 'localhost',
|
||||
port: overrides.port ?? Number(process.env.REDIS_PORT) || 6379,
|
||||
port: overrides.port ?? (Number(process.env.REDIS_PORT) || 6379),
|
||||
username: overrides.username ?? process.env.REDIS_USERNAME,
|
||||
password: overrides.password ?? process.env.REDIS_PASSWORD,
|
||||
...options,
|
||||
|
||||
@@ -1,41 +1,14 @@
|
||||
const cors = require('cors');
|
||||
import cors from 'cors';
|
||||
import { corsOptions } from '../../shared/cors/policy.js';
|
||||
|
||||
// Single CORS middleware for all endpoints
|
||||
const corsMiddleware = cors({
|
||||
origin: [
|
||||
'https://inventory.kent.pw',
|
||||
'http://localhost:5175',
|
||||
'https://acot.site',
|
||||
'https://tools.acherryontop.com',
|
||||
/^http:\/\/192\.168\.\d+\.\d+(:\d+)?$/,
|
||||
/^http:\/\/10\.\d+\.\d+\.\d+(:\d+)?$/
|
||||
],
|
||||
methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
|
||||
allowedHeaders: ['Content-Type', 'Authorization'],
|
||||
exposedHeaders: ['Content-Type'],
|
||||
credentials: true
|
||||
});
|
||||
export const corsMiddleware = cors(corsOptions);
|
||||
|
||||
// Error handler for CORS
|
||||
const corsErrorHandler = (err, req, res, next) => {
|
||||
if (err.message === 'CORS not allowed') {
|
||||
console.error('CORS Error:', {
|
||||
origin: req.get('Origin'),
|
||||
method: req.method,
|
||||
path: req.path,
|
||||
headers: req.headers
|
||||
});
|
||||
res.status(403).json({
|
||||
export function corsErrorHandler(err, req, res, next) {
|
||||
if (err && err.message === 'CORS not allowed') {
|
||||
return res.status(403).json({
|
||||
error: 'CORS not allowed',
|
||||
origin: req.get('Origin'),
|
||||
message: 'Origin not in allowed list: https://inventory.kent.pw, https://acot.site, https://tools.acherryontop.com, localhost:5175, 192.168.x.x, or 10.x.x.x'
|
||||
});
|
||||
} else {
|
||||
next(err);
|
||||
}
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
corsMiddleware,
|
||||
corsErrorHandler
|
||||
};
|
||||
next(err);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Phase 6.2: prompt edits require ai_admin. Reads remain authenticated-only.
|
||||
router.use((req, res, next) => {
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('ai_admin')(req, res, next);
|
||||
});
|
||||
|
||||
// Get all AI prompts
|
||||
router.get('/', async (req, res) => {
|
||||
try {
|
||||
@@ -307,4 +315,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,12 +1,25 @@
|
||||
const express = require("express");
|
||||
import express from "express";
|
||||
import OpenAI from "openai";
|
||||
import { promises as fs } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import dotenv from "dotenv";
|
||||
import mysql from 'mysql2/promise';
|
||||
import { Client } from 'ssh2';
|
||||
import { getDbConnection, closeAllConnections } from '../utils/dbConnection.js'; // Import the optimized connection function
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
const router = express.Router();
|
||||
const OpenAI = require("openai");
|
||||
const fs = require("fs").promises;
|
||||
const path = require("path");
|
||||
const dotenv = require("dotenv");
|
||||
const mysql = require('mysql2/promise');
|
||||
const { Client } = require('ssh2');
|
||||
const { getDbConnection, closeAllConnections } = require('../utils/dbConnection'); // Import the optimized connection function
|
||||
|
||||
// Phase 6.2: AI validation runs (which trigger OpenAI calls + DB writes) require ai_admin.
|
||||
// Status/health reads stay authenticated-only.
|
||||
router.use((req, res, next) => {
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('ai_admin')(req, res, next);
|
||||
});
|
||||
|
||||
// Ensure environment variables are loaded
|
||||
dotenv.config({ path: path.join(__dirname, "../../.env") });
|
||||
@@ -1401,7 +1414,7 @@ router.get("/test-taxonomy", async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
function extractResponseText(response) {
|
||||
if (!response) return "";
|
||||
|
||||
@@ -5,10 +5,11 @@
|
||||
* Provides embedding generation and similarity-based suggestions.
|
||||
*/
|
||||
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import aiService from '../services/ai/index.js';
|
||||
import { getDbConnection, closeAllConnections } from '../utils/dbConnection.js';
|
||||
|
||||
const router = express.Router();
|
||||
const aiService = require('../services/ai');
|
||||
const { getDbConnection, closeAllConnections } = require('../utils/dbConnection');
|
||||
|
||||
// Track initialization state
|
||||
let initializationPromise = null;
|
||||
@@ -440,11 +441,10 @@ router.post('/validate/sanity-check', async (req, res) => {
|
||||
* Call once from server startup so the taxonomy embeddings are ready before
|
||||
* the first user request hits a taxonomy dropdown.
|
||||
*/
|
||||
function initInBackground() {
|
||||
export function initInBackground() {
|
||||
ensureInitialized().catch(err =>
|
||||
console.error('[AI Routes] Background initialization failed:', err)
|
||||
);
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
module.exports.initInBackground = initInBackground;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Forecasting: summarize sales for products received in a period by brand
|
||||
@@ -980,4 +980,4 @@ router.get('/seasonal', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed
|
||||
const router = express.Router();
|
||||
const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed
|
||||
|
||||
// --- Configuration & Helpers ---
|
||||
const DEFAULT_PAGE_LIMIT = 50;
|
||||
@@ -281,4 +281,4 @@ router.get('/', async (req, res) => {
|
||||
// GET /brands-aggregate/:name (Get single brand metric)
|
||||
// Implement if needed, remember to URL-decode the name parameter
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,6 +1,6 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed
|
||||
const router = express.Router();
|
||||
const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed
|
||||
|
||||
// --- Configuration & Helpers ---
|
||||
const DEFAULT_PAGE_LIMIT = 50;
|
||||
@@ -360,4 +360,4 @@ router.get('/', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,10 +1,13 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { requireAdmin } from '../../shared/auth/middleware.js';
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Debug middleware
|
||||
// Phase 6.2: global settings are admin-only on write. Reads pass through to any
|
||||
// authenticated user (the server-level authenticate() already gates that).
|
||||
router.use((req, res, next) => {
|
||||
console.log(`[Config Route] ${req.method} ${req.path}`);
|
||||
next();
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requireAdmin(req, res, next);
|
||||
});
|
||||
|
||||
// ===== GLOBAL SETTINGS =====
|
||||
@@ -322,4 +325,4 @@ router.post('/vendors/:vendor/reset', async (req, res) => {
|
||||
});
|
||||
|
||||
// Export the router
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import db from '../utils/db.js';
|
||||
const router = express.Router();
|
||||
const db = require('../utils/db');
|
||||
|
||||
// Helper function to execute queries using the connection pool
|
||||
async function executeQuery(sql, params = []) {
|
||||
@@ -1288,4 +1288,4 @@ router.get('/replenish/products', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,13 +1,20 @@
|
||||
const express = require('express');
|
||||
const router = express.Router();
|
||||
const { spawn } = require('child_process');
|
||||
const path = require('path');
|
||||
const db = require('../utils/db');
|
||||
import express from 'express';
|
||||
import { spawn } from 'node:child_process';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import db from '../utils/db.js';
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
|
||||
// Debug middleware MUST be first
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Phase 6.2: CSV / full-update / full-reset / metrics-recalc are destructive.
|
||||
// Writes require data_management; reads (status polls, SSE streams) pass through.
|
||||
router.use((req, res, next) => {
|
||||
console.log(`[CSV Route Debug] ${req.method} ${req.path}`);
|
||||
next();
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('data_management')(req, res, next);
|
||||
});
|
||||
|
||||
// Store active processes and their progress
|
||||
@@ -437,4 +444,4 @@ router.get('/status/table-counts', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// GET /api/hts-lookup?search=term
|
||||
@@ -167,4 +167,4 @@ router.get('/', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Create a new audit log entry
|
||||
@@ -190,4 +190,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Get all import sessions for a user (named + unnamed)
|
||||
@@ -334,4 +334,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,14 +1,23 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { Client } from 'ssh2';
|
||||
import mysql from 'mysql2/promise';
|
||||
import multer from 'multer';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import sharp from 'sharp';
|
||||
import axios from 'axios';
|
||||
import net from 'node:net';
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
|
||||
const router = express.Router();
|
||||
const { Client } = require('ssh2');
|
||||
const mysql = require('mysql2/promise');
|
||||
const multer = require('multer');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const fsp = fs.promises;
|
||||
const sharp = require('sharp');
|
||||
const axios = require('axios');
|
||||
const net = require('net');
|
||||
|
||||
// Phase 6.2: imports, uploads, generate-upc and deletions all require product_import.
|
||||
// Reads (list-uploads, status checks) remain authenticated-only.
|
||||
router.use((req, res, next) => {
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('product_import')(req, res, next);
|
||||
});
|
||||
|
||||
// Create uploads directory if it doesn't exist
|
||||
const uploadsDir = path.join('/var/www/inventory/uploads/products');
|
||||
@@ -515,21 +524,30 @@ const storage = multer.diskStorage({
|
||||
|
||||
const MAX_UPLOAD_BYTES = 25 * 1024 * 1024;
|
||||
|
||||
// Phase 6.7: exact-match MIME + extension allowlist. Substring-based regex
|
||||
// matchers (the previous /jpeg|png|.../ approach) accepted MIMEs like
|
||||
// `application/jpeg-payload` because of partial matches; this rejects them.
|
||||
const ALLOWED_MIME_TYPES = new Set([
|
||||
'image/jpeg',
|
||||
'image/png',
|
||||
'image/gif',
|
||||
'image/webp',
|
||||
'image/tiff',
|
||||
]);
|
||||
const ALLOWED_EXTENSIONS = new Set(['.jpg', '.jpeg', '.png', '.gif', '.webp', '.tif', '.tiff']);
|
||||
|
||||
const upload = multer({
|
||||
storage: storage,
|
||||
limits: {
|
||||
fileSize: MAX_UPLOAD_BYTES,
|
||||
files: 1,
|
||||
},
|
||||
fileFilter: function (req, file, cb) {
|
||||
// Accept only image files
|
||||
const filetypes = /jpeg|jpg|png|gif|webp|tiff?/;
|
||||
const mimetype = filetypes.test(file.mimetype);
|
||||
const extname = filetypes.test(path.extname(file.originalname).toLowerCase());
|
||||
|
||||
if (mimetype && extname) {
|
||||
return cb(null, true);
|
||||
const ext = path.extname(file.originalname).toLowerCase();
|
||||
if (!ALLOWED_MIME_TYPES.has(file.mimetype) || !ALLOWED_EXTENSIONS.has(ext)) {
|
||||
return cb(new Error('Only image files are allowed (jpg, png, gif, webp, tiff)'));
|
||||
}
|
||||
cb(new Error('Only image files are allowed'));
|
||||
cb(null, true);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -647,7 +665,7 @@ async function setupSshTunnel() {
|
||||
port: process.env.PROD_SSH_PORT || 22,
|
||||
username: process.env.PROD_SSH_USER,
|
||||
privateKey: process.env.PROD_SSH_KEY_PATH
|
||||
? require('fs').readFileSync(process.env.PROD_SSH_KEY_PATH)
|
||||
? fs.readFileSync(process.env.PROD_SSH_KEY_PATH)
|
||||
: undefined,
|
||||
compress: true
|
||||
};
|
||||
@@ -2872,4 +2890,4 @@ router.get('/query-products', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { parseValue } from '../utils/apiHelpers.js';
|
||||
const router = express.Router();
|
||||
const { parseValue } = require('../utils/apiHelpers');
|
||||
|
||||
// --- Configuration & Helpers ---
|
||||
const DEFAULT_PAGE_LIMIT = 50;
|
||||
@@ -378,4 +378,4 @@ router.get('/:brand/:line/products', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// --- Configuration & Helpers ---
|
||||
@@ -645,4 +645,4 @@ function parseValue(value, type) {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Shared CTE fragment for the reference date.
|
||||
@@ -721,4 +721,4 @@ router.get('/campaigns/links', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Get all orders with pagination, filtering, and sorting
|
||||
@@ -258,4 +258,4 @@ router.get('/:orderNumber', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Create a new audit log entry
|
||||
@@ -192,4 +192,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,27 +1,7 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { PurchaseOrderStatus, ReceivingStatus } from '../types/status-codes.js';
|
||||
|
||||
const router = express.Router();
|
||||
const multer = require('multer');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const { importProductsFromCSV } = require('../utils/csvImporter');
|
||||
const { PurchaseOrderStatus, ReceivingStatus } = require('../types/status-codes');
|
||||
|
||||
// Configure multer for file uploads without silent fallbacks
|
||||
const configuredUploadsDir = process.env.UPLOADS_DIR;
|
||||
const uploadsDir = configuredUploadsDir
|
||||
? (path.isAbsolute(configuredUploadsDir)
|
||||
? configuredUploadsDir
|
||||
: path.resolve(__dirname, '../../', configuredUploadsDir))
|
||||
: path.resolve(__dirname, '../../uploads');
|
||||
|
||||
try {
|
||||
fs.mkdirSync(uploadsDir, { recursive: true });
|
||||
} catch (error) {
|
||||
console.error(`Failed to initialize uploads directory at ${uploadsDir}:`, error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
const upload = multer({ dest: uploadsDir });
|
||||
|
||||
// Get unique brands
|
||||
router.get('/brands', async (req, res) => {
|
||||
@@ -983,4 +963,4 @@ router.get('/:id/forecast', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Status code constants
|
||||
@@ -1277,4 +1277,4 @@ router.get('/pipeline', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
// Stale PO statuses to exclude from our counting — these are the "still active"
|
||||
@@ -388,4 +388,4 @@ router.get('/:pid/history', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,8 +1,16 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import multer from 'multer';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
|
||||
const router = express.Router();
|
||||
const multer = require('multer');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
|
||||
// Phase 6.2: uploads + deletions of reusable images require image_admin.
|
||||
router.use((req, res, next) => {
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('image_admin')(req, res, next);
|
||||
});
|
||||
|
||||
// Create reusable uploads directory if it doesn't exist
|
||||
const uploadsDir = path.join('/var/www/inventory/uploads/reusable');
|
||||
@@ -38,21 +46,24 @@ const storage = multer.diskStorage({
|
||||
}
|
||||
});
|
||||
|
||||
// Phase 6.7: exact-match MIME + extension allowlist.
|
||||
const ALLOWED_MIME_TYPES = new Set([
|
||||
'image/jpeg', 'image/png', 'image/gif', 'image/webp',
|
||||
]);
|
||||
const ALLOWED_EXTENSIONS = new Set(['.jpg', '.jpeg', '.png', '.gif', '.webp']);
|
||||
|
||||
const upload = multer({
|
||||
storage: storage,
|
||||
limits: {
|
||||
fileSize: 5 * 1024 * 1024, // 5MB max file size
|
||||
fileSize: 5 * 1024 * 1024,
|
||||
files: 1,
|
||||
},
|
||||
fileFilter: function (req, file, cb) {
|
||||
// Accept only image files
|
||||
const filetypes = /jpeg|jpg|png|gif|webp/;
|
||||
const mimetype = filetypes.test(file.mimetype);
|
||||
const extname = filetypes.test(path.extname(file.originalname).toLowerCase());
|
||||
|
||||
if (mimetype && extname) {
|
||||
return cb(null, true);
|
||||
const ext = path.extname(file.originalname).toLowerCase();
|
||||
if (!ALLOWED_MIME_TYPES.has(file.mimetype) || !ALLOWED_EXTENSIONS.has(ext)) {
|
||||
return cb(new Error('Only image files are allowed (jpg, png, gif, webp)'));
|
||||
}
|
||||
cb(new Error('Only image files are allowed'));
|
||||
cb(null, true);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -393,4 +404,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
const router = express.Router();
|
||||
|
||||
const MAX_MATCHES = 500;
|
||||
@@ -267,4 +267,4 @@ function descriptionAggregate(products) {
|
||||
return { duplicates, samples };
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,12 +1,23 @@
|
||||
const express = require('express');
|
||||
const { getPool } = require('../utils/db');
|
||||
const dotenv = require('dotenv');
|
||||
const path = require('path');
|
||||
import express from 'express';
|
||||
import dotenv from 'dotenv';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { getPool } from '../utils/db.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
dotenv.config({ path: path.join(__dirname, "../../.env") });
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Phase 6.2: template edits require templates_write. Reads pass through.
|
||||
import { requirePermission } from '../../shared/auth/middleware.js';
|
||||
router.use((req, res, next) => {
|
||||
if (req.method === 'GET' || req.method === 'HEAD' || req.method === 'OPTIONS') return next();
|
||||
return requirePermission('templates_write')(req, res, next);
|
||||
});
|
||||
|
||||
// Get all templates
|
||||
router.get('/', async (req, res) => {
|
||||
try {
|
||||
@@ -278,4 +289,4 @@ router.use((err, req, res, next) => {
|
||||
});
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
const express = require('express');
|
||||
import express from 'express';
|
||||
import { parseValue } from '../utils/apiHelpers.js'; // Adjust path if needed
|
||||
const router = express.Router();
|
||||
const { parseValue } = require('../utils/apiHelpers'); // Adjust path if needed
|
||||
|
||||
// --- Configuration & Helpers ---
|
||||
const DEFAULT_PAGE_LIMIT = 50;
|
||||
@@ -320,4 +320,4 @@ router.get('/', async (req, res) => {
|
||||
// GET /vendors-aggregate/:name (Get single vendor metric)
|
||||
// Implement if needed, remember to URL-decode the name parameter
|
||||
|
||||
module.exports = router;
|
||||
export default router;
|
||||
+76
-150
@@ -1,59 +1,64 @@
|
||||
const express = require('express');
|
||||
const cors = require('cors');
|
||||
const { spawn } = require('child_process');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const { corsMiddleware, corsErrorHandler } = require('./middleware/cors');
|
||||
const { initPool } = require('./utils/db');
|
||||
const productsRouter = require('./routes/products');
|
||||
const dashboardRouter = require('./routes/dashboard');
|
||||
const ordersRouter = require('./routes/orders');
|
||||
const csvRouter = require('./routes/data-management');
|
||||
const analyticsRouter = require('./routes/analytics');
|
||||
const purchaseOrdersRouter = require('./routes/purchase-orders');
|
||||
const configRouter = require('./routes/config');
|
||||
const metricsRouter = require('./routes/metrics');
|
||||
const importRouter = require('./routes/import');
|
||||
const aiValidationRouter = require('./routes/ai-validation');
|
||||
const aiRouter = require('./routes/ai');
|
||||
const templatesRouter = require('./routes/templates');
|
||||
const aiPromptsRouter = require('./routes/ai-prompts');
|
||||
const reusableImagesRouter = require('./routes/reusable-images');
|
||||
const categoriesAggregateRouter = require('./routes/categoriesAggregate');
|
||||
const vendorsAggregateRouter = require('./routes/vendorsAggregate');
|
||||
const brandsAggregateRouter = require('./routes/brandsAggregate');
|
||||
const htsLookupRouter = require('./routes/hts-lookup');
|
||||
const specLookupRouter = require('./routes/spec-lookup');
|
||||
const importSessionsRouter = require('./routes/import-sessions');
|
||||
const importAuditLogRouter = require('./routes/import-audit-log');
|
||||
const productEditorAuditLogRouter = require('./routes/product-editor-audit-log');
|
||||
const newsletterRouter = require('./routes/newsletter');
|
||||
const linesAggregateRouter = require('./routes/linesAggregate');
|
||||
const repeatOrdersRouter = require('./routes/repeat-orders');
|
||||
import { config as loadEnv } from 'dotenv';
|
||||
import express from 'express';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
import { corsMiddleware, corsErrorHandler } from './middleware/cors.js';
|
||||
import { initPool } from './utils/db.js';
|
||||
|
||||
import { authenticate } from '../shared/auth/middleware.js';
|
||||
import { requestLog } from '../shared/logging/request-log.js';
|
||||
import { logger } from '../shared/logging/logger.js';
|
||||
import { errorHandler } from '../shared/errors/handler.js';
|
||||
|
||||
import productsRouter from './routes/products.js';
|
||||
import dashboardRouter from './routes/dashboard.js';
|
||||
import ordersRouter from './routes/orders.js';
|
||||
import csvRouter from './routes/data-management.js';
|
||||
import analyticsRouter from './routes/analytics.js';
|
||||
import purchaseOrdersRouter from './routes/purchase-orders.js';
|
||||
import configRouter from './routes/config.js';
|
||||
import metricsRouter from './routes/metrics.js';
|
||||
import importRouter from './routes/import.js';
|
||||
import aiValidationRouter from './routes/ai-validation.js';
|
||||
import aiRouter, { initInBackground as initAiInBackground } from './routes/ai.js';
|
||||
import templatesRouter from './routes/templates.js';
|
||||
import aiPromptsRouter from './routes/ai-prompts.js';
|
||||
import reusableImagesRouter from './routes/reusable-images.js';
|
||||
import categoriesAggregateRouter from './routes/categoriesAggregate.js';
|
||||
import vendorsAggregateRouter from './routes/vendorsAggregate.js';
|
||||
import brandsAggregateRouter from './routes/brandsAggregate.js';
|
||||
import htsLookupRouter from './routes/hts-lookup.js';
|
||||
import specLookupRouter from './routes/spec-lookup.js';
|
||||
import importSessionsRouter from './routes/import-sessions.js';
|
||||
import importAuditLogRouter from './routes/import-audit-log.js';
|
||||
import productEditorAuditLogRouter from './routes/product-editor-audit-log.js';
|
||||
import newsletterRouter from './routes/newsletter.js';
|
||||
import linesAggregateRouter from './routes/linesAggregate.js';
|
||||
import repeatOrdersRouter from './routes/repeat-orders.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
// Get the absolute path to the .env file
|
||||
const envPath = '/var/www/inventory/.env';
|
||||
console.log('Looking for .env file at:', envPath);
|
||||
console.log('.env file exists:', fs.existsSync(envPath));
|
||||
loadEnv({ path: envPath });
|
||||
logger.info({
|
||||
envPath,
|
||||
envExists: fs.existsSync(envPath),
|
||||
NODE_ENV: process.env.NODE_ENV || 'not set',
|
||||
PORT: process.env.PORT || 'not set',
|
||||
DB_HOST: process.env.DB_HOST || 'not set',
|
||||
DB_NAME: process.env.DB_NAME || 'not set',
|
||||
DB_PASSWORD: process.env.DB_PASSWORD ? '[set]' : 'not set',
|
||||
DB_SSL: process.env.DB_SSL || 'not set',
|
||||
}, 'inventory-server starting');
|
||||
|
||||
try {
|
||||
require('dotenv').config({ path: envPath });
|
||||
console.log('.env file loaded successfully');
|
||||
console.log('Environment check:', {
|
||||
NODE_ENV: process.env.NODE_ENV || 'not set',
|
||||
PORT: process.env.PORT || 'not set',
|
||||
DB_HOST: process.env.DB_HOST || 'not set',
|
||||
DB_USER: process.env.DB_USER || 'not set',
|
||||
DB_NAME: process.env.DB_NAME || 'not set',
|
||||
DB_PASSWORD: process.env.DB_PASSWORD ? '[password set]' : 'not set',
|
||||
DB_PORT: process.env.DB_PORT || 'not set',
|
||||
DB_SSL: process.env.DB_SSL || 'not set'
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error loading .env file:', error);
|
||||
if (!process.env.JWT_SECRET) {
|
||||
logger.error('JWT_SECRET is not set; refusing to start (per Phase 6.4)');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Resolve important directories relative to the project root
|
||||
const serverRoot = path.resolve(__dirname, '..');
|
||||
const configuredUploadsDir = process.env.UPLOADS_DIR;
|
||||
const uploadsDir = configuredUploadsDir
|
||||
@@ -62,12 +67,10 @@ const uploadsDir = configuredUploadsDir
|
||||
: path.resolve(serverRoot, configuredUploadsDir))
|
||||
: path.resolve(serverRoot, 'uploads');
|
||||
|
||||
// Persist the resolved uploads directory so downstream modules share the same path
|
||||
process.env.UPLOADS_DIR = uploadsDir;
|
||||
|
||||
const requiredDirs = [path.resolve(serverRoot, 'logs'), uploadsDir];
|
||||
|
||||
requiredDirs.forEach(dir => {
|
||||
requiredDirs.forEach((dir) => {
|
||||
if (!fs.existsSync(dir)) {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
}
|
||||
@@ -75,28 +78,18 @@ requiredDirs.forEach(dir => {
|
||||
|
||||
const app = express();
|
||||
|
||||
// Debug middleware to log request details
|
||||
app.use((req, res, next) => {
|
||||
console.log('Request details:', {
|
||||
method: req.method,
|
||||
url: req.url,
|
||||
origin: req.get('Origin'),
|
||||
headers: req.headers
|
||||
});
|
||||
next();
|
||||
});
|
||||
// Phase 6.5/6.9: structured access log (replaces the previous header-dumping debug
|
||||
// middleware that wrote raw Authorization values to stdout). Pino redaction strips
|
||||
// `authorization` and `cookie` automatically — see shared/logging/logger.js.
|
||||
app.use(requestLog());
|
||||
|
||||
// Apply CORS middleware first, before any other middleware
|
||||
app.use(corsMiddleware);
|
||||
|
||||
// Body parser middleware
|
||||
app.use(express.json({ limit: '10mb' }));
|
||||
app.use(express.urlencoded({ extended: true, limit: '10mb' }));
|
||||
|
||||
// Initialize database pool and start server
|
||||
async function startServer() {
|
||||
try {
|
||||
// Initialize database pool
|
||||
const pool = await initPool({
|
||||
host: process.env.DB_HOST,
|
||||
user: process.env.DB_USER,
|
||||
@@ -104,17 +97,18 @@ async function startServer() {
|
||||
database: process.env.DB_NAME,
|
||||
port: process.env.DB_PORT || 5432,
|
||||
max: process.env.NODE_ENV === 'production' ? 20 : 10,
|
||||
idleTimeoutMillis: 30000,
|
||||
connectionTimeoutMillis: 2000,
|
||||
ssl: process.env.DB_SSL === 'true' ? {
|
||||
rejectUnauthorized: false
|
||||
} : false
|
||||
idleTimeoutMillis: 30_000,
|
||||
connectionTimeoutMillis: 2_000,
|
||||
ssl: process.env.DB_SSL === 'true' ? { rejectUnauthorized: false } : false,
|
||||
});
|
||||
|
||||
// Make pool available to routes
|
||||
app.locals.pool = pool;
|
||||
|
||||
// Set up routes after pool is initialized
|
||||
// Phase 6.1/6.2: every /api request requires a valid JWT. Defense in depth — Caddy
|
||||
// forward_auth (when enabled) is the first reject; this is the second. Frontend
|
||||
// service files MUST include `Authorization: Bearer <token>` on every fetch.
|
||||
app.use('/api', authenticate({ pool, secret: process.env.JWT_SECRET }));
|
||||
|
||||
app.use('/api/products', productsRouter);
|
||||
app.use('/api/dashboard', dashboardRouter);
|
||||
app.use('/api/orders', ordersRouter);
|
||||
@@ -123,10 +117,8 @@ async function startServer() {
|
||||
app.use('/api/purchase-orders', purchaseOrdersRouter);
|
||||
app.use('/api/config', configRouter);
|
||||
app.use('/api/metrics', metricsRouter);
|
||||
// Use only the aggregate routes for vendors and categories
|
||||
app.use('/api/vendors', vendorsAggregateRouter);
|
||||
app.use('/api/categories', categoriesAggregateRouter);
|
||||
// Keep the aggregate-specific endpoints for backward compatibility
|
||||
app.use('/api/categories-aggregate', categoriesAggregateRouter);
|
||||
app.use('/api/vendors-aggregate', vendorsAggregateRouter);
|
||||
app.use('/api/brands-aggregate', brandsAggregateRouter);
|
||||
@@ -145,101 +137,35 @@ async function startServer() {
|
||||
app.use('/api/lines-aggregate', linesAggregateRouter);
|
||||
app.use('/api/repeat-orders', repeatOrdersRouter);
|
||||
|
||||
// Basic health check route
|
||||
app.get('/health', (req, res) => {
|
||||
res.json({
|
||||
status: 'ok',
|
||||
timestamp: new Date().toISOString(),
|
||||
environment: process.env.NODE_ENV
|
||||
environment: process.env.NODE_ENV,
|
||||
});
|
||||
});
|
||||
|
||||
// CORS error handler - must be before other error handlers
|
||||
app.use(corsErrorHandler);
|
||||
|
||||
// Error handling middleware - MUST be after routes and CORS error handler
|
||||
app.use((err, req, res, next) => {
|
||||
console.error(`[${new Date().toISOString()}] Error:`, err);
|
||||
|
||||
// Send detailed error in development, generic in production
|
||||
const error = process.env.NODE_ENV === 'production'
|
||||
? 'An internal server error occurred'
|
||||
: err.message || err;
|
||||
|
||||
res.status(err.status || 500).json({ error });
|
||||
});
|
||||
app.use(errorHandler);
|
||||
|
||||
const PORT = process.env.PORT || 3000;
|
||||
app.listen(PORT, () => {
|
||||
console.log(`[Server] Running in ${process.env.NODE_ENV || 'development'} mode on port ${PORT}`);
|
||||
// Pre-warm AI service so taxonomy embeddings are ready before first user request
|
||||
aiRouter.initInBackground();
|
||||
logger.info({ port: PORT, mode: process.env.NODE_ENV || 'development' }, 'inventory-server listening');
|
||||
initAiInBackground();
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Failed to start server:', error);
|
||||
logger.error({ err: error }, 'Failed to start server');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle uncaught exceptions
|
||||
process.on('uncaughtException', (err) => {
|
||||
console.error(`[${new Date().toISOString()}] Uncaught Exception:`, err);
|
||||
logger.error({ err }, 'Uncaught Exception');
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
process.on('unhandledRejection', (reason, promise) => {
|
||||
console.error(`[${new Date().toISOString()}] Unhandled Rejection at:`, promise, 'reason:', reason);
|
||||
logger.error({ reason, promise }, 'Unhandled Rejection');
|
||||
});
|
||||
|
||||
// Initialize client sets for SSE
|
||||
const importClients = new Set();
|
||||
const updateClients = new Set();
|
||||
const resetClients = new Set();
|
||||
const resetMetricsClients = new Set();
|
||||
|
||||
// Helper function to send progress to SSE clients
|
||||
const sendProgressToClients = (clients, data) => {
|
||||
clients.forEach(client => {
|
||||
try {
|
||||
client.write(`data: ${JSON.stringify(data)}\n\n`);
|
||||
} catch (error) {
|
||||
console.error('Error sending SSE update:', error);
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// Setup SSE connection
|
||||
const setupSSE = (req, res) => {
|
||||
const { type } = req.params;
|
||||
|
||||
// Set headers for SSE
|
||||
res.writeHead(200, {
|
||||
'Content-Type': 'text/event-stream',
|
||||
'Cache-Control': 'no-cache',
|
||||
'Connection': 'keep-alive',
|
||||
'Access-Control-Allow-Origin': req.headers.origin || '*',
|
||||
'Access-Control-Allow-Credentials': 'true'
|
||||
});
|
||||
|
||||
// Send initial message
|
||||
res.write('data: {"status":"connected"}\n\n');
|
||||
|
||||
// Add client to appropriate set
|
||||
const clientSet = type === 'import' ? importClients :
|
||||
type === 'update' ? updateClients :
|
||||
type === 'reset' ? resetClients :
|
||||
type === 'reset-metrics' ? resetMetricsClients :
|
||||
null;
|
||||
|
||||
if (clientSet) {
|
||||
clientSet.add(res);
|
||||
|
||||
// Remove client when connection closes
|
||||
req.on('close', () => {
|
||||
clientSet.delete(res);
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Start the server
|
||||
startServer();
|
||||
|
||||
@@ -1,82 +1,36 @@
|
||||
/**
|
||||
* Vector similarity utilities
|
||||
*/
|
||||
|
||||
/**
|
||||
* Compute cosine similarity between two vectors
|
||||
* @param {number[]} a
|
||||
* @param {number[]} b
|
||||
* @returns {number} Similarity score between -1 and 1
|
||||
*/
|
||||
function cosineSimilarity(a, b) {
|
||||
if (!a || !b || a.length !== b.length) {
|
||||
return 0;
|
||||
}
|
||||
export function cosineSimilarity(a, b) {
|
||||
if (!a || !b || a.length !== b.length) return 0;
|
||||
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
|
||||
const denominator = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
if (denominator === 0) return 0;
|
||||
|
||||
return dotProduct / denominator;
|
||||
return denominator === 0 ? 0 : dotProduct / denominator;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find top K most similar items from a collection
|
||||
* @param {number[]} queryEmbedding - The embedding to search for
|
||||
* @param {Array<{id: any, embedding: number[]}>} items - Items with embeddings
|
||||
* @param {number} topK - Number of results to return
|
||||
* @returns {Array<{id: any, similarity: number}>}
|
||||
*/
|
||||
function findTopMatches(queryEmbedding, items, topK = 10) {
|
||||
if (!queryEmbedding || !items || items.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const scored = items.map(item => ({
|
||||
export function findTopMatches(queryEmbedding, items, topK = 10) {
|
||||
if (!queryEmbedding || !items || items.length === 0) return [];
|
||||
const scored = items.map((item) => ({
|
||||
id: item.id,
|
||||
similarity: cosineSimilarity(queryEmbedding, item.embedding)
|
||||
similarity: cosineSimilarity(queryEmbedding, item.embedding),
|
||||
}));
|
||||
|
||||
scored.sort((a, b) => b.similarity - a.similarity);
|
||||
|
||||
return scored.slice(0, topK);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find matches above a similarity threshold
|
||||
* @param {number[]} queryEmbedding
|
||||
* @param {Array<{id: any, embedding: number[]}>} items
|
||||
* @param {number} threshold - Minimum similarity (0-1)
|
||||
* @returns {Array<{id: any, similarity: number}>}
|
||||
*/
|
||||
function findMatchesAboveThreshold(queryEmbedding, items, threshold = 0.5) {
|
||||
if (!queryEmbedding || !items || items.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
export function findMatchesAboveThreshold(queryEmbedding, items, threshold = 0.5) {
|
||||
if (!queryEmbedding || !items || items.length === 0) return [];
|
||||
const scored = items
|
||||
.map(item => ({
|
||||
.map((item) => ({
|
||||
id: item.id,
|
||||
similarity: cosineSimilarity(queryEmbedding, item.embedding)
|
||||
similarity: cosineSimilarity(queryEmbedding, item.embedding),
|
||||
}))
|
||||
.filter(item => item.similarity >= threshold);
|
||||
|
||||
.filter((item) => item.similarity >= threshold);
|
||||
scored.sort((a, b) => b.similarity - a.similarity);
|
||||
|
||||
return scored;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
cosineSimilarity,
|
||||
findTopMatches,
|
||||
findMatchesAboveThreshold
|
||||
};
|
||||
|
||||
@@ -1,74 +1,52 @@
|
||||
/**
|
||||
* Taxonomy Embedding Service
|
||||
*
|
||||
* Generates and caches embeddings for categories, themes, and colors.
|
||||
* Excludes "Black Friday", "Gifts", "Deals" categories and their children.
|
||||
*
|
||||
* Disk cache: embeddings are saved to data/taxonomy-embeddings.json and reused
|
||||
* across server restarts. Cache is invalidated by content hash — if the taxonomy
|
||||
* rows in MySQL change, the next check will detect it and regenerate automatically.
|
||||
*
|
||||
* Background check: after initialization, call startBackgroundCheck(getConnectionFn)
|
||||
* to poll for taxonomy changes on a configurable interval (default 1h).
|
||||
* Disk cache at data/taxonomy-embeddings.json; content-hash invalidated.
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const crypto = require('crypto');
|
||||
const { findTopMatches } = require('./similarity');
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import crypto from 'node:crypto';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { findTopMatches } from './similarity.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
// Categories to exclude (and all their children)
|
||||
const EXCLUDED_CATEGORY_NAMES = ['black friday', 'gifts', 'deals'];
|
||||
|
||||
// Disk cache config
|
||||
const CACHE_PATH = path.join(__dirname, '..', '..', '..', '..', 'data', 'taxonomy-embeddings.json');
|
||||
|
||||
class TaxonomyEmbeddings {
|
||||
export class TaxonomyEmbeddings {
|
||||
constructor({ provider, logger }) {
|
||||
this.provider = provider;
|
||||
this.logger = logger || console;
|
||||
|
||||
// Cached taxonomy with embeddings
|
||||
this.categories = [];
|
||||
this.themes = [];
|
||||
this.colors = [];
|
||||
|
||||
// Raw data without embeddings (for lookup)
|
||||
this.categoryMap = new Map();
|
||||
this.themeMap = new Map();
|
||||
this.colorMap = new Map();
|
||||
|
||||
// Content hash of the last successfully built taxonomy (from DB rows)
|
||||
this.contentHash = null;
|
||||
|
||||
this.initialized = false;
|
||||
this.initializing = false;
|
||||
this._checkInterval = null;
|
||||
this._regenerating = false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize embeddings — fetches raw taxonomy rows to compute a content hash,
|
||||
* then either loads the matching disk cache or generates fresh embeddings.
|
||||
*/
|
||||
async initialize(connection) {
|
||||
if (this.initialized) {
|
||||
return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length };
|
||||
}
|
||||
|
||||
if (this.initializing) {
|
||||
// Wait for existing initialization
|
||||
while (this.initializing) {
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length };
|
||||
}
|
||||
|
||||
this.initializing = true;
|
||||
|
||||
try {
|
||||
// Always fetch raw rows first — cheap (~10ms), no OpenAI calls.
|
||||
// Used to compute a content hash for cache validation.
|
||||
const rawRows = await this._fetchRawRows(connection);
|
||||
const freshHash = this._computeContentHash(rawRows);
|
||||
|
||||
@@ -77,9 +55,9 @@ class TaxonomyEmbeddings {
|
||||
this.categories = cached.categories;
|
||||
this.themes = cached.themes;
|
||||
this.colors = cached.colors;
|
||||
this.categoryMap = new Map(this.categories.map(c => [c.id, c]));
|
||||
this.themeMap = new Map(this.themes.map(t => [t.id, t]));
|
||||
this.colorMap = new Map(this.colors.map(c => [c.id, c]));
|
||||
this.categoryMap = new Map(this.categories.map((c) => [c.id, c]));
|
||||
this.themeMap = new Map(this.themes.map((t) => [t.id, t]));
|
||||
this.colorMap = new Map(this.colors.map((c) => [c.id, c]));
|
||||
this.contentHash = freshHash;
|
||||
this.initialized = true;
|
||||
this.logger.info(`[TaxonomyEmbeddings] Loaded from cache: ${this.categories.length} categories, ${this.themes.length} themes, ${this.colors.length} colors`);
|
||||
@@ -95,7 +73,6 @@ class TaxonomyEmbeddings {
|
||||
await this._buildAndEmbed(rawRows, freshHash);
|
||||
this.initialized = true;
|
||||
this.logger.info('[TaxonomyEmbeddings] Initialization complete');
|
||||
|
||||
return { categories: this.categories.length, themes: this.themes.length, colors: this.colors.length };
|
||||
} catch (error) {
|
||||
this.logger.error('[TaxonomyEmbeddings] Initialization failed:', error);
|
||||
@@ -105,26 +82,16 @@ class TaxonomyEmbeddings {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start a background interval that checks for taxonomy changes and regenerates
|
||||
* embeddings automatically if the content hash differs.
|
||||
*
|
||||
* @param {Function} getConnectionFn - async function returning { connection }
|
||||
* @param {number} intervalMs - check interval, default 1 hour
|
||||
*/
|
||||
startBackgroundCheck(getConnectionFn, intervalMs = 60 * 60 * 1000) {
|
||||
if (this._checkInterval) return;
|
||||
|
||||
this.logger.info(`[TaxonomyEmbeddings] Background taxonomy check started (every ${intervalMs / 60000} min)`);
|
||||
|
||||
this._checkInterval = setInterval(async () => {
|
||||
if (this._regenerating) return;
|
||||
|
||||
try {
|
||||
const { connection } = await getConnectionFn();
|
||||
const rawRows = await this._fetchRawRows(connection);
|
||||
const freshHash = this._computeContentHash(rawRows);
|
||||
|
||||
if (freshHash === this.contentHash) return;
|
||||
|
||||
this.logger.info('[TaxonomyEmbeddings] Taxonomy changed, regenerating embeddings in background...');
|
||||
@@ -146,123 +113,79 @@ class TaxonomyEmbeddings {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Find similar categories for a product embedding
|
||||
*/
|
||||
findSimilarCategories(productEmbedding, topK = 10) {
|
||||
if (!this.initialized || !productEmbedding) {
|
||||
return [];
|
||||
}
|
||||
|
||||
if (!this.initialized || !productEmbedding) return [];
|
||||
const matches = findTopMatches(productEmbedding, this.categories, topK);
|
||||
|
||||
return matches.map(match => {
|
||||
return matches.map((match) => {
|
||||
const cat = this.categoryMap.get(match.id);
|
||||
return {
|
||||
id: match.id,
|
||||
name: cat?.name || '',
|
||||
fullPath: cat?.fullPath || '',
|
||||
similarity: match.similarity
|
||||
similarity: match.similarity,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Find similar themes for a product embedding
|
||||
*/
|
||||
findSimilarThemes(productEmbedding, topK = 5) {
|
||||
if (!this.initialized || !productEmbedding) {
|
||||
return [];
|
||||
}
|
||||
|
||||
if (!this.initialized || !productEmbedding) return [];
|
||||
const matches = findTopMatches(productEmbedding, this.themes, topK);
|
||||
|
||||
return matches.map(match => {
|
||||
return matches.map((match) => {
|
||||
const theme = this.themeMap.get(match.id);
|
||||
return {
|
||||
id: match.id,
|
||||
name: theme?.name || '',
|
||||
fullPath: theme?.fullPath || '',
|
||||
similarity: match.similarity
|
||||
similarity: match.similarity,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Find similar colors for a product embedding
|
||||
*/
|
||||
findSimilarColors(productEmbedding, topK = 5) {
|
||||
if (!this.initialized || !productEmbedding) {
|
||||
return [];
|
||||
}
|
||||
|
||||
if (!this.initialized || !productEmbedding) return [];
|
||||
const matches = findTopMatches(productEmbedding, this.colors, topK);
|
||||
|
||||
return matches.map(match => {
|
||||
return matches.map((match) => {
|
||||
const color = this.colorMap.get(match.id);
|
||||
return {
|
||||
id: match.id,
|
||||
name: color?.name || '',
|
||||
similarity: match.similarity
|
||||
similarity: match.similarity,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all taxonomy data (without embeddings) for frontend
|
||||
*/
|
||||
getTaxonomyData() {
|
||||
return {
|
||||
categories: this.categories.map(({ id, name, fullPath, parentId }) => ({ id, name, fullPath, parentId })),
|
||||
themes: this.themes.map(({ id, name, fullPath, parentId }) => ({ id, name, fullPath, parentId })),
|
||||
colors: this.colors.map(({ id, name }) => ({ id, name }))
|
||||
colors: this.colors.map(({ id, name }) => ({ id, name })),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if service is ready
|
||||
*/
|
||||
isReady() {
|
||||
return this.initialized;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Private Methods
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Fetch minimal raw rows from MySQL — used for content hash computation.
|
||||
* This is the cheap path: no path-building, no embeddings, just the raw data.
|
||||
*/
|
||||
async _fetchRawRows(connection) {
|
||||
const [[catRows], [themeRows], [colorRows]] = await Promise.all([
|
||||
connection.query('SELECT cat_id, name, master_cat_id, type FROM product_categories WHERE type IN (10, 11, 12, 13) ORDER BY cat_id'),
|
||||
connection.query('SELECT cat_id, name, master_cat_id, type FROM product_categories WHERE type IN (20, 21) ORDER BY cat_id'),
|
||||
connection.query('SELECT color, name, hex_color FROM product_color_list ORDER BY `order`')
|
||||
connection.query('SELECT color, name, hex_color FROM product_color_list ORDER BY `order`'),
|
||||
]);
|
||||
return { catRows, themeRows, colorRows };
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute a stable SHA-256 hash of the taxonomy row content.
|
||||
* Any change to IDs, names, or parent relationships will produce a different hash.
|
||||
*/
|
||||
_computeContentHash({ catRows, themeRows, colorRows }) {
|
||||
const content = JSON.stringify({
|
||||
cats: catRows.map(r => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]),
|
||||
themes: themeRows.map(r => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]),
|
||||
colors: colorRows.map(r => [r.color, r.name]).sort()
|
||||
cats: catRows.map((r) => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]),
|
||||
themes: themeRows.map((r) => [r.cat_id, r.name, r.master_cat_id]).sort((a, b) => a[0] - b[0]),
|
||||
colors: colorRows.map((r) => [r.color, r.name]).sort(),
|
||||
});
|
||||
return crypto.createHash('sha256').update(content).digest('hex').slice(0, 16);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build full taxonomy objects and generate embeddings, then atomically swap
|
||||
* the in-memory state. Called on cache miss and on background change detection.
|
||||
*/
|
||||
async _buildAndEmbed(rawRows, contentHash) {
|
||||
const { catRows, themeRows, colorRows } = rawRows;
|
||||
|
||||
const categories = this._buildCategories(catRows);
|
||||
const themes = this._buildThemes(themeRows);
|
||||
const colors = this._buildColors(colorRows);
|
||||
@@ -272,23 +195,22 @@ class TaxonomyEmbeddings {
|
||||
const [catEmbeddings, themeEmbeddings, colorEmbeddings] = await Promise.all([
|
||||
this._generateEmbeddings(categories, 'categories'),
|
||||
this._generateEmbeddings(themes, 'themes'),
|
||||
this._generateEmbeddings(colors, 'colors')
|
||||
this._generateEmbeddings(colors, 'colors'),
|
||||
]);
|
||||
|
||||
// Atomic in-memory swap (single-threaded JS — readers always see a consistent state)
|
||||
this.categories = catEmbeddings;
|
||||
this.themes = themeEmbeddings;
|
||||
this.colors = colorEmbeddings;
|
||||
this.categoryMap = new Map(this.categories.map(c => [c.id, c]));
|
||||
this.themeMap = new Map(this.themes.map(t => [t.id, t]));
|
||||
this.colorMap = new Map(this.colors.map(c => [c.id, c]));
|
||||
this.categoryMap = new Map(this.categories.map((c) => [c.id, c]));
|
||||
this.themeMap = new Map(this.themes.map((t) => [t.id, t]));
|
||||
this.colorMap = new Map(this.colors.map((c) => [c.id, c]));
|
||||
this.contentHash = contentHash;
|
||||
|
||||
this._saveCache();
|
||||
}
|
||||
|
||||
_buildCategories(rows) {
|
||||
const byId = new Map(rows.map(r => [r.cat_id, r]));
|
||||
const byId = new Map(rows.map((r) => [r.cat_id, r]));
|
||||
const excludedIds = new Set();
|
||||
|
||||
for (const row of rows) {
|
||||
@@ -297,7 +219,6 @@ class TaxonomyEmbeddings {
|
||||
}
|
||||
}
|
||||
|
||||
// Multiple passes to find all descendants of excluded categories
|
||||
let foundNew = true;
|
||||
while (foundNew) {
|
||||
foundNew = false;
|
||||
@@ -314,98 +235,80 @@ class TaxonomyEmbeddings {
|
||||
const categories = [];
|
||||
for (const row of rows) {
|
||||
if (excludedIds.has(row.cat_id)) continue;
|
||||
|
||||
const pathParts = [];
|
||||
let current = row;
|
||||
while (current) {
|
||||
pathParts.unshift(current.name);
|
||||
current = current.master_cat_id ? byId.get(current.master_cat_id) : null;
|
||||
}
|
||||
|
||||
categories.push({
|
||||
id: row.cat_id,
|
||||
name: row.name,
|
||||
parentId: row.master_cat_id,
|
||||
type: row.type,
|
||||
fullPath: pathParts.join(' > '),
|
||||
embeddingText: pathParts.join(' ')
|
||||
embeddingText: pathParts.join(' '),
|
||||
});
|
||||
}
|
||||
|
||||
return categories;
|
||||
}
|
||||
|
||||
_buildThemes(rows) {
|
||||
const byId = new Map(rows.map(r => [r.cat_id, r]));
|
||||
|
||||
return rows.map(row => {
|
||||
const byId = new Map(rows.map((r) => [r.cat_id, r]));
|
||||
return rows.map((row) => {
|
||||
const pathParts = [];
|
||||
let current = row;
|
||||
while (current) {
|
||||
pathParts.unshift(current.name);
|
||||
current = current.master_cat_id ? byId.get(current.master_cat_id) : null;
|
||||
}
|
||||
|
||||
return {
|
||||
id: row.cat_id,
|
||||
name: row.name,
|
||||
parentId: row.master_cat_id,
|
||||
type: row.type,
|
||||
fullPath: pathParts.join(' > '),
|
||||
embeddingText: pathParts.join(' ')
|
||||
embeddingText: pathParts.join(' '),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
_buildColors(rows) {
|
||||
return rows.map(row => ({
|
||||
return rows.map((row) => ({
|
||||
id: row.color,
|
||||
name: row.name,
|
||||
hexColor: row.hex_color,
|
||||
embeddingText: row.name
|
||||
embeddingText: row.name,
|
||||
}));
|
||||
}
|
||||
|
||||
async _generateEmbeddings(items, label) {
|
||||
if (items.length === 0) {
|
||||
return items;
|
||||
}
|
||||
if (items.length === 0) return items;
|
||||
|
||||
const startTime = Date.now();
|
||||
const texts = items.map(item => item.embeddingText);
|
||||
const texts = items.map((item) => item.embeddingText);
|
||||
const results = [...items];
|
||||
|
||||
// Process in batches
|
||||
for await (const chunk of this.provider.embedBatchChunked(texts, { batchSize: 100 })) {
|
||||
for (let i = 0; i < chunk.embeddings.length; i++) {
|
||||
const globalIndex = chunk.startIndex + i;
|
||||
results[globalIndex] = {
|
||||
...results[globalIndex],
|
||||
embedding: chunk.embeddings[i]
|
||||
};
|
||||
results[globalIndex] = { ...results[globalIndex], embedding: chunk.embeddings[i] };
|
||||
}
|
||||
}
|
||||
|
||||
const elapsed = Date.now() - startTime;
|
||||
this.logger.info(`[TaxonomyEmbeddings] Generated ${items.length} ${label} embeddings in ${elapsed}ms`);
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Disk Cache Methods
|
||||
// ============================================================================
|
||||
|
||||
_loadCache() {
|
||||
try {
|
||||
if (!fs.existsSync(CACHE_PATH)) return null;
|
||||
|
||||
const data = JSON.parse(fs.readFileSync(CACHE_PATH, 'utf8'));
|
||||
if (!data.contentHash || !data.categories?.length || !data.themes?.length || !data.colors?.length) {
|
||||
this.logger.warn('[TaxonomyEmbeddings] Disk cache malformed or missing content hash, will regenerate');
|
||||
return null;
|
||||
}
|
||||
|
||||
return data;
|
||||
} catch (err) {
|
||||
this.logger.warn('[TaxonomyEmbeddings] Failed to load disk cache:', err.message);
|
||||
@@ -429,5 +332,3 @@ class TaxonomyEmbeddings {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { TaxonomyEmbeddings };
|
||||
|
||||
@@ -1,17 +1,16 @@
|
||||
/**
|
||||
* AI Service
|
||||
*
|
||||
* Main entry point for AI functionality including:
|
||||
* - Embeddings for taxonomy suggestions (OpenAI)
|
||||
* - Chat completions for validation tasks (Groq)
|
||||
* - Task registry for AI operations
|
||||
* Main entry point for AI functionality (embeddings + chat completions + task registry).
|
||||
*/
|
||||
|
||||
const { OpenAIProvider } = require('./providers/openaiProvider');
|
||||
const { GroqProvider, MODELS: GROQ_MODELS } = require('./providers/groqProvider');
|
||||
const { TaxonomyEmbeddings } = require('./embeddings/taxonomyEmbeddings');
|
||||
const { cosineSimilarity, findTopMatches } = require('./embeddings/similarity');
|
||||
const { getRegistry, TASK_IDS, registerAllTasks } = require('./tasks');
|
||||
import { OpenAIProvider } from './providers/openaiProvider.js';
|
||||
import { GroqProvider, MODELS as GROQ_MODELS } from './providers/groqProvider.js';
|
||||
import { TaxonomyEmbeddings } from './embeddings/taxonomyEmbeddings.js';
|
||||
import { cosineSimilarity, findTopMatches } from './embeddings/similarity.js';
|
||||
import { getRegistry, TASK_IDS, registerAllTasks } from './tasks/index.js';
|
||||
|
||||
export { TASK_IDS, GROQ_MODELS, cosineSimilarity, findTopMatches };
|
||||
|
||||
let initialized = false;
|
||||
let initializing = false;
|
||||
@@ -19,54 +18,28 @@ let openaiProvider = null;
|
||||
let groqProvider = null;
|
||||
let taxonomyEmbeddings = null;
|
||||
let logger = console;
|
||||
|
||||
// Store pool reference for task access
|
||||
let appPool = null;
|
||||
|
||||
/**
|
||||
* Initialize the AI service
|
||||
* @param {Object} options
|
||||
* @param {string} options.openaiApiKey - OpenAI API key (for embeddings)
|
||||
* @param {string} [options.groqApiKey] - Groq API key (for chat completions)
|
||||
* @param {Object} options.mysqlConnection - MySQL connection for taxonomy data
|
||||
* @param {Object} [options.pool] - PostgreSQL pool for prompt loading
|
||||
* @param {Object} [options.logger] - Logger instance
|
||||
*/
|
||||
async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, logger: customLogger }) {
|
||||
if (initialized) {
|
||||
return { success: true, message: 'Already initialized' };
|
||||
}
|
||||
export async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, logger: customLogger }) {
|
||||
if (initialized) return { success: true, message: 'Already initialized' };
|
||||
|
||||
if (initializing) {
|
||||
// Wait for existing initialization
|
||||
while (initializing) {
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
return { success: initialized, message: initialized ? 'Initialized' : 'Initialization failed' };
|
||||
}
|
||||
|
||||
initializing = true;
|
||||
|
||||
try {
|
||||
if (customLogger) {
|
||||
logger = customLogger;
|
||||
}
|
||||
|
||||
if (!openaiApiKey) {
|
||||
throw new Error('OpenAI API key is required');
|
||||
}
|
||||
if (customLogger) logger = customLogger;
|
||||
if (!openaiApiKey) throw new Error('OpenAI API key is required');
|
||||
|
||||
logger.info('[AI] Initializing AI service...');
|
||||
if (pool) appPool = pool;
|
||||
|
||||
// Store pool reference for tasks
|
||||
if (pool) {
|
||||
appPool = pool;
|
||||
}
|
||||
|
||||
// Create OpenAI provider (for embeddings)
|
||||
openaiProvider = new OpenAIProvider({ apiKey: openaiApiKey });
|
||||
|
||||
// Create Groq provider (for chat completions) if API key provided
|
||||
if (groqApiKey) {
|
||||
groqProvider = new GroqProvider({ apiKey: groqApiKey });
|
||||
logger.info('[AI] Groq provider initialized for chat completions');
|
||||
@@ -74,32 +47,19 @@ async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, log
|
||||
logger.warn('[AI] No Groq API key provided - chat completion tasks will not be available');
|
||||
}
|
||||
|
||||
// Create and initialize taxonomy embeddings
|
||||
taxonomyEmbeddings = new TaxonomyEmbeddings({
|
||||
provider: openaiProvider,
|
||||
logger
|
||||
});
|
||||
|
||||
taxonomyEmbeddings = new TaxonomyEmbeddings({ provider: openaiProvider, logger });
|
||||
const stats = await taxonomyEmbeddings.initialize(mysqlConnection);
|
||||
|
||||
// Register validation tasks if Groq is available
|
||||
if (groqProvider) {
|
||||
registerValidationTasks();
|
||||
}
|
||||
if (groqProvider) registerValidationTasks();
|
||||
|
||||
initialized = true;
|
||||
logger.info('[AI] AI service initialized', {
|
||||
...stats,
|
||||
groqEnabled: !!groqProvider,
|
||||
tasksRegistered: getRegistry().list()
|
||||
tasksRegistered: getRegistry().list(),
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: 'Initialized',
|
||||
stats,
|
||||
groqEnabled: !!groqProvider
|
||||
};
|
||||
return { success: true, message: 'Initialized', stats, groqEnabled: !!groqProvider };
|
||||
} catch (error) {
|
||||
logger.error('[AI] Initialization failed:', error);
|
||||
return { success: false, message: error.message };
|
||||
@@ -108,41 +68,20 @@ async function initialize({ openaiApiKey, groqApiKey, mysqlConnection, pool, log
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Register validation tasks with the task registry
|
||||
* Called during initialization if Groq is available
|
||||
*/
|
||||
function registerValidationTasks() {
|
||||
registerAllTasks(logger);
|
||||
logger.info('[AI] Validation tasks registered');
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if service is ready
|
||||
*/
|
||||
function isReady() {
|
||||
export function isReady() {
|
||||
return initialized && taxonomyEmbeddings?.isReady();
|
||||
}
|
||||
|
||||
/**
|
||||
* Start background taxonomy change detection.
|
||||
* Call once after initialization, passing a function that returns { connection }.
|
||||
* @param {Function} getConnectionFn
|
||||
* @param {number} [intervalMs] - default 1 hour
|
||||
*/
|
||||
function startBackgroundCheck(getConnectionFn, intervalMs) {
|
||||
export function startBackgroundCheck(getConnectionFn, intervalMs) {
|
||||
if (!initialized || !taxonomyEmbeddings) return;
|
||||
taxonomyEmbeddings.startBackgroundCheck(getConnectionFn, intervalMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build weighted product text for embedding.
|
||||
* Weights the product name heavily by repeating it, and truncates long descriptions
|
||||
* to prevent verbose marketing copy from drowning out the product signal.
|
||||
*
|
||||
* @param {Object} product - Product with name, description, company, line
|
||||
* @returns {string} - Combined text for embedding
|
||||
*/
|
||||
function buildProductText(product) {
|
||||
const parts = [];
|
||||
const name = product.name?.trim();
|
||||
@@ -150,20 +89,9 @@ function buildProductText(product) {
|
||||
const company = (product.company_name || product.company)?.trim();
|
||||
const line = (product.line_name || product.line)?.trim();
|
||||
|
||||
// Name is most important - repeat 3x to weight it heavily in the embedding
|
||||
if (name) {
|
||||
parts.push(name, name, name);
|
||||
}
|
||||
|
||||
// Company and line provide context
|
||||
if (company) {
|
||||
parts.push(company);
|
||||
}
|
||||
if (line) {
|
||||
parts.push(line);
|
||||
}
|
||||
|
||||
// Truncate description to prevent it from overwhelming the signal
|
||||
if (name) parts.push(name, name, name);
|
||||
if (company) parts.push(company);
|
||||
if (line) parts.push(line);
|
||||
if (description) {
|
||||
const truncated = description.length > 500
|
||||
? description.substring(0, 500) + '...'
|
||||
@@ -174,74 +102,35 @@ function buildProductText(product) {
|
||||
return parts.join(' ').trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate embedding for a product
|
||||
* @param {Object} product - Product with name, description, company, line
|
||||
* @returns {Promise<{embedding: number[], latencyMs: number}>}
|
||||
*/
|
||||
async function getProductEmbedding(product) {
|
||||
if (!initialized || !openaiProvider) {
|
||||
throw new Error('AI service not initialized');
|
||||
}
|
||||
export async function getProductEmbedding(product) {
|
||||
if (!initialized || !openaiProvider) throw new Error('AI service not initialized');
|
||||
|
||||
const text = buildProductText(product);
|
||||
|
||||
if (!text) {
|
||||
return { embedding: null, latencyMs: 0 };
|
||||
}
|
||||
if (!text) return { embedding: null, latencyMs: 0 };
|
||||
|
||||
const result = await openaiProvider.embed(text);
|
||||
|
||||
return {
|
||||
embedding: result.embeddings[0],
|
||||
latencyMs: result.latencyMs
|
||||
};
|
||||
return { embedding: result.embeddings[0], latencyMs: result.latencyMs };
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate embeddings for multiple products
|
||||
* @param {Object[]} products - Array of products
|
||||
* @returns {Promise<{embeddings: Array<{index: number, embedding: number[]}>, latencyMs: number}>}
|
||||
*/
|
||||
async function getProductEmbeddings(products) {
|
||||
if (!initialized || !openaiProvider) {
|
||||
throw new Error('AI service not initialized');
|
||||
}
|
||||
export async function getProductEmbeddings(products) {
|
||||
if (!initialized || !openaiProvider) throw new Error('AI service not initialized');
|
||||
|
||||
const texts = products.map(buildProductText);
|
||||
const validIndices = texts.map((t, i) => t ? i : -1).filter((i) => i >= 0);
|
||||
const validTexts = texts.filter((t) => t);
|
||||
|
||||
// Track which products have empty text
|
||||
const validIndices = texts.map((t, i) => t ? i : -1).filter(i => i >= 0);
|
||||
const validTexts = texts.filter(t => t);
|
||||
|
||||
if (validTexts.length === 0) {
|
||||
return { embeddings: [], latencyMs: 0 };
|
||||
}
|
||||
if (validTexts.length === 0) return { embeddings: [], latencyMs: 0 };
|
||||
|
||||
const result = await openaiProvider.embed(validTexts);
|
||||
|
||||
// Map embeddings back to original indices
|
||||
const embeddings = validIndices.map((originalIndex, resultIndex) => ({
|
||||
index: originalIndex,
|
||||
embedding: result.embeddings[resultIndex]
|
||||
embedding: result.embeddings[resultIndex],
|
||||
}));
|
||||
|
||||
return {
|
||||
embeddings,
|
||||
latencyMs: result.latencyMs
|
||||
};
|
||||
return { embeddings, latencyMs: result.latencyMs };
|
||||
}
|
||||
|
||||
/**
|
||||
* Find similar taxonomy items for a product embedding
|
||||
* @param {number[]} productEmbedding
|
||||
* @param {Object} options
|
||||
* @returns {{categories: Array, themes: Array, colors: Array}}
|
||||
*/
|
||||
function findSimilarTaxonomy(productEmbedding, options = {}) {
|
||||
if (!initialized || !taxonomyEmbeddings) {
|
||||
throw new Error('AI service not initialized');
|
||||
}
|
||||
export function findSimilarTaxonomy(productEmbedding, options = {}) {
|
||||
if (!initialized || !taxonomyEmbeddings) throw new Error('AI service not initialized');
|
||||
|
||||
const topCategories = options.topCategories ?? 10;
|
||||
const topThemes = options.topThemes ?? 5;
|
||||
@@ -250,25 +139,15 @@ function findSimilarTaxonomy(productEmbedding, options = {}) {
|
||||
return {
|
||||
categories: taxonomyEmbeddings.findSimilarCategories(productEmbedding, topCategories),
|
||||
themes: taxonomyEmbeddings.findSimilarThemes(productEmbedding, topThemes),
|
||||
colors: taxonomyEmbeddings.findSimilarColors(productEmbedding, topColors)
|
||||
colors: taxonomyEmbeddings.findSimilarColors(productEmbedding, topColors),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get product embedding and find similar taxonomy in one call
|
||||
* @param {Object} product
|
||||
* @param {Object} options
|
||||
*/
|
||||
async function getSuggestionsForProduct(product, options = {}) {
|
||||
export async function getSuggestionsForProduct(product, options = {}) {
|
||||
const { embedding, latencyMs: embeddingLatency } = await getProductEmbedding(product);
|
||||
|
||||
if (!embedding) {
|
||||
return {
|
||||
categories: [],
|
||||
themes: [],
|
||||
colors: [],
|
||||
latencyMs: embeddingLatency
|
||||
};
|
||||
return { categories: [], themes: [], colors: [], latencyMs: embeddingLatency };
|
||||
}
|
||||
|
||||
const startSearch = Date.now();
|
||||
@@ -279,27 +158,17 @@ async function getSuggestionsForProduct(product, options = {}) {
|
||||
...suggestions,
|
||||
latencyMs: embeddingLatency + searchLatency,
|
||||
embeddingLatencyMs: embeddingLatency,
|
||||
searchLatencyMs: searchLatency
|
||||
searchLatencyMs: searchLatency,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all taxonomy data (without embeddings) for frontend
|
||||
*/
|
||||
function getTaxonomyData() {
|
||||
if (!initialized || !taxonomyEmbeddings) {
|
||||
throw new Error('AI service not initialized');
|
||||
}
|
||||
|
||||
export function getTaxonomyData() {
|
||||
if (!initialized || !taxonomyEmbeddings) throw new Error('AI service not initialized');
|
||||
return taxonomyEmbeddings.getTaxonomyData();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get service status
|
||||
*/
|
||||
function getStatus() {
|
||||
export function getStatus() {
|
||||
const registry = getRegistry();
|
||||
|
||||
return {
|
||||
initialized,
|
||||
ready: isReady(),
|
||||
@@ -309,90 +178,56 @@ function getStatus() {
|
||||
taxonomyStats: taxonomyEmbeddings ? {
|
||||
categories: taxonomyEmbeddings.categories?.length || 0,
|
||||
themes: taxonomyEmbeddings.themes?.length || 0,
|
||||
colors: taxonomyEmbeddings.colors?.length || 0
|
||||
colors: taxonomyEmbeddings.colors?.length || 0,
|
||||
} : null,
|
||||
tasks: {
|
||||
registered: registry.list(),
|
||||
count: registry.size()
|
||||
}
|
||||
count: registry.size(),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Run an AI task by ID
|
||||
* @param {string} taskId - Task identifier from TASK_IDS
|
||||
* @param {Object} payload - Task-specific input
|
||||
* @returns {Promise<Object>} Task result
|
||||
*/
|
||||
async function runTask(taskId, payload = {}) {
|
||||
if (!initialized) {
|
||||
throw new Error('AI service not initialized');
|
||||
}
|
||||
|
||||
if (!groqProvider) {
|
||||
throw new Error('Groq provider not available - chat completion tasks require GROQ_API_KEY');
|
||||
}
|
||||
export async function runTask(taskId, payload = {}) {
|
||||
if (!initialized) throw new Error('AI service not initialized');
|
||||
if (!groqProvider) throw new Error('Groq provider not available - chat completion tasks require GROQ_API_KEY');
|
||||
|
||||
const registry = getRegistry();
|
||||
return registry.runTask(taskId, {
|
||||
...payload,
|
||||
// Inject dependencies tasks may need
|
||||
provider: groqProvider,
|
||||
// Use pool from payload if provided (from route), fall back to stored appPool
|
||||
pool: payload.pool || appPool,
|
||||
logger
|
||||
logger,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the Groq provider instance (for direct use if needed)
|
||||
* @returns {GroqProvider|null}
|
||||
*/
|
||||
function getGroqProvider() {
|
||||
export function getGroqProvider() {
|
||||
return groqProvider;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the PostgreSQL pool (for tasks that need DB access)
|
||||
* @returns {Object|null}
|
||||
*/
|
||||
function getPool() {
|
||||
export function getPool() {
|
||||
return appPool;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if chat completion tasks are available
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function hasChatCompletion() {
|
||||
export function hasChatCompletion() {
|
||||
return !!groqProvider;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
// Initialization
|
||||
export default {
|
||||
initialize,
|
||||
isReady,
|
||||
getStatus,
|
||||
startBackgroundCheck,
|
||||
|
||||
// Embeddings (OpenAI)
|
||||
getProductEmbedding,
|
||||
getProductEmbeddings,
|
||||
findSimilarTaxonomy,
|
||||
getSuggestionsForProduct,
|
||||
getTaxonomyData,
|
||||
|
||||
// Chat completions (Groq)
|
||||
runTask,
|
||||
hasChatCompletion,
|
||||
getGroqProvider,
|
||||
getPool,
|
||||
|
||||
// Constants
|
||||
TASK_IDS,
|
||||
GROQ_MODELS,
|
||||
|
||||
// Re-export utilities
|
||||
cosineSimilarity,
|
||||
findTopMatches
|
||||
findTopMatches,
|
||||
};
|
||||
|
||||
@@ -1,81 +1,41 @@
|
||||
/**
|
||||
* Description Validation Prompts
|
||||
*
|
||||
* Functions for building and parsing description validation prompts.
|
||||
* System and general prompts are loaded from the database.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Sanitize an issue string from AI response
|
||||
* AI sometimes returns malformed strings with escape sequences
|
||||
*
|
||||
* @param {string} issue - Raw issue string
|
||||
* @returns {string} Cleaned issue string
|
||||
*/
|
||||
function sanitizeIssue(issue) {
|
||||
if (!issue || typeof issue !== 'string') return '';
|
||||
|
||||
let cleaned = issue
|
||||
// Remove trailing backslashes (incomplete escapes)
|
||||
return issue
|
||||
.replace(/\\+$/, '')
|
||||
// Fix malformed escaped quotes at end of string
|
||||
.replace(/\\",?\)?$/, '')
|
||||
// Clean up double-escaped quotes
|
||||
.replace(/\\\\"/g, '"')
|
||||
// Clean up single escaped quotes that aren't needed
|
||||
.replace(/\\"/g, '"')
|
||||
// Remove any remaining trailing punctuation artifacts
|
||||
.replace(/[,\s]+$/, '')
|
||||
// Trim whitespace
|
||||
.trim();
|
||||
|
||||
return cleaned;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the user prompt for description validation
|
||||
* Combines database prompts with product data
|
||||
*
|
||||
* @param {Object} product - Product data
|
||||
* @param {string} product.name - Product name
|
||||
* @param {string} product.description - Current description
|
||||
* @param {string} [product.company_name] - Company name
|
||||
* @param {string} [product.categories] - Product categories
|
||||
* @param {Object} prompts - Prompts loaded from database
|
||||
* @param {string} prompts.general - General description guidelines
|
||||
* @param {string} [prompts.companySpecific] - Company-specific rules
|
||||
* @returns {string} Complete user prompt
|
||||
*/
|
||||
function buildDescriptionUserPrompt(product, prompts) {
|
||||
export function buildDescriptionUserPrompt(product, prompts) {
|
||||
const parts = [];
|
||||
|
||||
// Add general prompt/guidelines if provided
|
||||
if (prompts.general) {
|
||||
parts.push(prompts.general);
|
||||
parts.push(''); // Empty line for separation
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
// Add company-specific rules if provided
|
||||
if (prompts.companySpecific) {
|
||||
parts.push(`COMPANY-SPECIFIC RULES FOR ${product.company_name || 'THIS COMPANY'}:`);
|
||||
parts.push(prompts.companySpecific);
|
||||
parts.push(''); // Empty line for separation
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
// Add product information
|
||||
parts.push('PRODUCT TO VALIDATE:');
|
||||
parts.push(`NAME: "${product.name || ''}"`);
|
||||
parts.push(`COMPANY: ${product.company_name || 'Unknown'}`);
|
||||
|
||||
if (product.categories) {
|
||||
parts.push(`CATEGORIES: ${product.categories}`);
|
||||
}
|
||||
if (product.categories) parts.push(`CATEGORIES: ${product.categories}`);
|
||||
|
||||
parts.push('');
|
||||
parts.push('CURRENT DESCRIPTION:');
|
||||
parts.push(`"${product.description || '(empty)'}"`);
|
||||
|
||||
// Add response format instructions
|
||||
parts.push('');
|
||||
parts.push('CRITICAL RULES:');
|
||||
parts.push('- If isValid is false, you MUST provide a suggestion with the improved description');
|
||||
@@ -86,68 +46,40 @@ function buildDescriptionUserPrompt(product, prompts) {
|
||||
parts.push(JSON.stringify({
|
||||
isValid: 'true if perfect, false if ANY changes needed',
|
||||
suggestion: 'REQUIRED when isValid is false - the complete improved description',
|
||||
issues: ['list each problem found (empty array only if isValid is true)']
|
||||
issues: ['list each problem found (empty array only if isValid is true)'],
|
||||
}, null, 2));
|
||||
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the AI response for description validation
|
||||
*
|
||||
* @param {Object|null} parsed - Parsed JSON from AI
|
||||
* @param {string} content - Raw response content
|
||||
* @returns {Object}
|
||||
*/
|
||||
function parseDescriptionResponse(parsed, content) {
|
||||
// If we got valid parsed JSON, use it
|
||||
export function parseDescriptionResponse(parsed, content) {
|
||||
if (parsed && typeof parsed.isValid === 'boolean') {
|
||||
// Sanitize issues - AI sometimes returns malformed escape sequences
|
||||
const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : [];
|
||||
const issues = rawIssues
|
||||
.map(sanitizeIssue)
|
||||
.filter(issue => issue.length > 0);
|
||||
|
||||
const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0);
|
||||
const suggestion = parsed.suggestion || null;
|
||||
|
||||
// IMPORTANT: LLMs sometimes return contradictory data (isValid: true with issues).
|
||||
// If there are issues, treat as invalid regardless of what the AI said.
|
||||
// Also if there's a suggestion, the AI thought something needed to change.
|
||||
const isValid = parsed.isValid && issues.length === 0 && !suggestion;
|
||||
|
||||
return { isValid, suggestion, issues };
|
||||
}
|
||||
|
||||
// Handle case where isValid is a string "true"/"false" instead of boolean
|
||||
if (parsed && typeof parsed.isValid === 'string') {
|
||||
const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : [];
|
||||
const issues = rawIssues
|
||||
.map(sanitizeIssue)
|
||||
.filter(issue => issue.length > 0);
|
||||
const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0);
|
||||
const suggestion = parsed.suggestion || null;
|
||||
const rawIsValid = parsed.isValid.toLowerCase() !== 'false';
|
||||
|
||||
// Same defensive logic: if there are issues, it's not valid
|
||||
const isValid = rawIsValid && issues.length === 0 && !suggestion;
|
||||
|
||||
return { isValid, suggestion, issues };
|
||||
}
|
||||
|
||||
// Try to extract from content if parsing failed
|
||||
try {
|
||||
// Look for isValid pattern
|
||||
const isValidMatch = content.match(/"isValid"\s*:\s*(true|false)/i);
|
||||
const isValid = isValidMatch ? isValidMatch[1].toLowerCase() === 'true' : true;
|
||||
|
||||
// Look for suggestion (might be multiline)
|
||||
const suggestionMatch = content.match(/"suggestion"\s*:\s*"((?:[^"\\]|\\.)*)"/s);
|
||||
let suggestion = suggestionMatch ? suggestionMatch[1] : null;
|
||||
if (suggestion) {
|
||||
// Unescape common escapes
|
||||
suggestion = suggestion.replace(/\\n/g, '\n').replace(/\\"/g, '"');
|
||||
}
|
||||
|
||||
// Look for issues array
|
||||
const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/);
|
||||
let issues = [];
|
||||
if (issuesMatch) {
|
||||
@@ -155,22 +87,13 @@ function parseDescriptionResponse(parsed, content) {
|
||||
const issueStrings = issuesContent.match(/"([^"]+)"/g);
|
||||
if (issueStrings) {
|
||||
issues = issueStrings
|
||||
.map(s => sanitizeIssue(s.replace(/"/g, '')))
|
||||
.filter(issue => issue.length > 0);
|
||||
.map((s) => sanitizeIssue(s.replace(/"/g, '')))
|
||||
.filter((issue) => issue.length > 0);
|
||||
}
|
||||
}
|
||||
|
||||
// Same logic: if there are issues, it's not valid
|
||||
const finalIsValid = isValid && issues.length === 0 && !suggestion;
|
||||
|
||||
return { isValid: finalIsValid, suggestion, issues };
|
||||
} catch {
|
||||
// Default to valid if we can't parse anything
|
||||
return { isValid: true, suggestion: null, issues: [] };
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
buildDescriptionUserPrompt,
|
||||
parseDescriptionResponse
|
||||
};
|
||||
|
||||
@@ -1,164 +1,94 @@
|
||||
/**
|
||||
* Name Validation Prompts
|
||||
*
|
||||
* Functions for building and parsing name validation prompts.
|
||||
* System and general prompts are loaded from the database.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Sanitize an issue string from AI response
|
||||
* AI sometimes returns malformed strings with escape sequences
|
||||
*
|
||||
* @param {string} issue - Raw issue string
|
||||
* @returns {string} Cleaned issue string
|
||||
*/
|
||||
function sanitizeIssue(issue) {
|
||||
if (!issue || typeof issue !== 'string') return '';
|
||||
|
||||
let cleaned = issue
|
||||
// Remove trailing backslashes (incomplete escapes)
|
||||
return issue
|
||||
.replace(/\\+$/, '')
|
||||
// Fix malformed escaped quotes at end of string
|
||||
.replace(/\\",?\)?$/, '')
|
||||
// Clean up double-escaped quotes
|
||||
.replace(/\\\\"/g, '"')
|
||||
// Clean up single escaped quotes that aren't needed
|
||||
.replace(/\\"/g, '"')
|
||||
// Remove any remaining trailing punctuation artifacts
|
||||
.replace(/[,\s]+$/, '')
|
||||
// Trim whitespace
|
||||
.trim();
|
||||
|
||||
return cleaned;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the user prompt for name validation
|
||||
* Combines database prompts with product data
|
||||
*
|
||||
* @param {Object} product - Product data
|
||||
* @param {string} product.name - Current product name
|
||||
* @param {string} [product.company_name] - Company name
|
||||
* @param {string} [product.line_name] - Product line name
|
||||
* @param {string} [product.subline_name] - Product subline name
|
||||
* @param {string[]} [product.siblingNames] - Names of other products in the same line
|
||||
* @param {Object} prompts - Prompts loaded from database
|
||||
* @param {string} prompts.general - General naming conventions
|
||||
* @param {string} [prompts.companySpecific] - Company-specific rules
|
||||
* @returns {string} Complete user prompt
|
||||
*/
|
||||
function buildNameUserPrompt(product, prompts) {
|
||||
export function buildNameUserPrompt(product, prompts) {
|
||||
const parts = [];
|
||||
|
||||
// Add general prompt/conventions if provided
|
||||
if (prompts.general) {
|
||||
parts.push(prompts.general);
|
||||
parts.push(''); // Empty line for separation
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
// Add company-specific rules if provided
|
||||
if (prompts.companySpecific) {
|
||||
parts.push(`COMPANY-SPECIFIC RULES FOR ${product.company_name || 'THIS COMPANY'}:`);
|
||||
parts.push(prompts.companySpecific);
|
||||
parts.push(''); // Empty line for separation
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
// Add product information
|
||||
parts.push('PRODUCT TO VALIDATE:');
|
||||
parts.push(`NAME: "${product.name || ''}"`);
|
||||
parts.push(`COMPANY: ${product.company_name || 'Unknown'}`);
|
||||
parts.push(`LINE: ${product.line_name || 'None'}`);
|
||||
if (product.subline_name) {
|
||||
parts.push(`SUBLINE: ${product.subline_name}`);
|
||||
}
|
||||
if (product.subline_name) parts.push(`SUBLINE: ${product.subline_name}`);
|
||||
|
||||
// Add sibling context for naming decisions
|
||||
if (product.siblingNames && product.siblingNames.length > 0) {
|
||||
parts.push('');
|
||||
parts.push(`OTHER PRODUCTS IN THIS LINE (${product.siblingNames.length + 1} total including this one):`);
|
||||
product.siblingNames.forEach(name => {
|
||||
parts.push(`- ${name}`);
|
||||
});
|
||||
product.siblingNames.forEach((name) => parts.push(`- ${name}`));
|
||||
}
|
||||
|
||||
// Add response format instructions
|
||||
parts.push('');
|
||||
parts.push('RESPOND WITH JSON:');
|
||||
parts.push(JSON.stringify({
|
||||
isValid: 'true/false',
|
||||
suggestion: 'corrected name if changes needed, or null if valid',
|
||||
issues: ['issue 1', 'issue 2 (empty array if valid)']
|
||||
issues: ['issue 1', 'issue 2 (empty array if valid)'],
|
||||
}, null, 2));
|
||||
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the AI response for name validation
|
||||
*
|
||||
* @param {Object|null} parsed - Parsed JSON from AI
|
||||
* @param {string} content - Raw response content
|
||||
* @returns {Object}
|
||||
*/
|
||||
function parseNameResponse(parsed, content) {
|
||||
// Debug: Log what we're trying to parse
|
||||
export function parseNameResponse(parsed, content) {
|
||||
console.log('[parseNameResponse] Input:', {
|
||||
hasParsed: !!parsed,
|
||||
parsedIsValid: parsed?.isValid,
|
||||
parsedType: typeof parsed?.isValid,
|
||||
contentPreview: content?.substring(0, 3000)
|
||||
contentPreview: content?.substring(0, 3000),
|
||||
});
|
||||
|
||||
// If we got valid parsed JSON, use it
|
||||
if (parsed && typeof parsed.isValid === 'boolean') {
|
||||
// Sanitize issues - AI sometimes returns malformed escape sequences
|
||||
const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : [];
|
||||
const issues = rawIssues
|
||||
.map(sanitizeIssue)
|
||||
.filter(issue => issue.length > 0);
|
||||
const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0);
|
||||
const suggestion = parsed.suggestion || null;
|
||||
|
||||
// IMPORTANT: LLMs sometimes return contradictory data (isValid: true with issues).
|
||||
// If there are issues, treat as invalid regardless of what the AI said.
|
||||
const isValid = parsed.isValid && issues.length === 0 && !suggestion;
|
||||
|
||||
return { isValid, suggestion, issues };
|
||||
}
|
||||
|
||||
// Handle case where isValid is a string "true"/"false" instead of boolean
|
||||
if (parsed && typeof parsed.isValid === 'string') {
|
||||
const rawIssues = Array.isArray(parsed.issues) ? parsed.issues : [];
|
||||
const issues = rawIssues
|
||||
.map(sanitizeIssue)
|
||||
.filter(issue => issue.length > 0);
|
||||
const issues = rawIssues.map(sanitizeIssue).filter((issue) => issue.length > 0);
|
||||
const suggestion = parsed.suggestion || null;
|
||||
const rawIsValid = parsed.isValid.toLowerCase() !== 'false';
|
||||
|
||||
// Same defensive logic: if there are issues, it's not valid
|
||||
const isValid = rawIsValid && issues.length === 0 && !suggestion;
|
||||
|
||||
console.log('[parseNameResponse] Parsed isValid as string:', parsed.isValid, '→', isValid);
|
||||
return { isValid, suggestion, issues };
|
||||
}
|
||||
|
||||
// Try to extract from content if parsing failed
|
||||
try {
|
||||
// Look for isValid pattern - handle both boolean and quoted string
|
||||
// Matches: "isValid": true, "isValid": false, "isValid": "true", "isValid": "false"
|
||||
const isValidMatch = content.match(/"isValid"\s*:\s*"?(true|false)"?/i);
|
||||
const isValid = isValidMatch ? isValidMatch[1].toLowerCase() === 'true' : true;
|
||||
|
||||
console.log('[parseNameResponse] Regex extraction:', {
|
||||
isValidMatch: isValidMatch?.[0],
|
||||
isValidValue: isValidMatch?.[1],
|
||||
resultIsValid: isValid
|
||||
resultIsValid: isValid,
|
||||
});
|
||||
|
||||
// Look for suggestion - handle escaped quotes and null
|
||||
const suggestionMatch = content.match(/"suggestion"\s*:\s*(?:"([^"\\]*(?:\\.[^"\\]*)*)"|null)/);
|
||||
const suggestion = suggestionMatch ? (suggestionMatch[1] || null) : null;
|
||||
|
||||
// Look for issues array
|
||||
const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/);
|
||||
let issues = [];
|
||||
if (issuesMatch) {
|
||||
@@ -166,22 +96,13 @@ function parseNameResponse(parsed, content) {
|
||||
const issueStrings = issuesContent.match(/"([^"]+)"/g);
|
||||
if (issueStrings) {
|
||||
issues = issueStrings
|
||||
.map(s => sanitizeIssue(s.replace(/"/g, '')))
|
||||
.filter(issue => issue.length > 0);
|
||||
.map((s) => sanitizeIssue(s.replace(/"/g, '')))
|
||||
.filter((issue) => issue.length > 0);
|
||||
}
|
||||
}
|
||||
|
||||
// Same defensive logic: if there are issues, it's not valid
|
||||
const finalIsValid = isValid && issues.length === 0 && !suggestion;
|
||||
|
||||
return { isValid: finalIsValid, suggestion, issues };
|
||||
} catch {
|
||||
// Default to valid if we can't parse anything
|
||||
return { isValid: true, suggestion: null, issues: [] };
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
buildNameUserPrompt,
|
||||
parseNameResponse
|
||||
};
|
||||
|
||||
@@ -1,35 +1,18 @@
|
||||
/**
|
||||
* Prompt Loader
|
||||
*
|
||||
* Utilities to load AI prompts from the ai_prompts PostgreSQL table.
|
||||
* Supports loading prompts by base type (e.g., 'name_validation' loads
|
||||
* name_validation_system, name_validation_general, and optionally
|
||||
* name_validation_company_specific).
|
||||
* Prompt Loader — loads AI prompts from the ai_prompts PostgreSQL table.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Load a single prompt by exact type
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string} promptType - Exact prompt type (e.g., 'name_validation_system')
|
||||
* @param {string} [company] - Company identifier (for company_specific types)
|
||||
* @returns {Promise<string|null>} Prompt text or null if not found
|
||||
*/
|
||||
async function loadPromptByType(pool, promptType, company = null) {
|
||||
export async function loadPromptByType(pool, promptType, company = null) {
|
||||
try {
|
||||
let result;
|
||||
|
||||
if (company) {
|
||||
result = await pool.query(
|
||||
'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company = $2',
|
||||
[promptType, company]
|
||||
);
|
||||
} else {
|
||||
result = await pool.query(
|
||||
'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company IS NULL',
|
||||
[promptType]
|
||||
);
|
||||
}
|
||||
|
||||
const result = company
|
||||
? await pool.query(
|
||||
'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company = $2',
|
||||
[promptType, company]
|
||||
)
|
||||
: await pool.query(
|
||||
'SELECT prompt_text FROM ai_prompts WHERE prompt_type = $1 AND company IS NULL',
|
||||
[promptType]
|
||||
);
|
||||
return result.rows[0]?.prompt_text || null;
|
||||
} catch (error) {
|
||||
console.error(`[PromptLoader] Error loading ${promptType} prompt:`, error.message);
|
||||
@@ -37,93 +20,46 @@ async function loadPromptByType(pool, promptType, company = null) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load all prompts for a task type (system, general, and optionally company-specific)
|
||||
*
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string} baseType - Base type name (e.g., 'name_validation', 'description_validation')
|
||||
* @param {string|null} [company] - Optional company ID for company-specific prompts
|
||||
* @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>}
|
||||
*/
|
||||
async function loadPromptsByType(pool, baseType, company = null) {
|
||||
export async function loadPromptsByType(pool, baseType, company = null) {
|
||||
const systemType = `${baseType}_system`;
|
||||
const generalType = `${baseType}_general`;
|
||||
const companyType = `${baseType}_company_specific`;
|
||||
|
||||
// Load system and general prompts in parallel
|
||||
const [system, general] = await Promise.all([
|
||||
loadPromptByType(pool, systemType),
|
||||
loadPromptByType(pool, generalType)
|
||||
loadPromptByType(pool, generalType),
|
||||
]);
|
||||
|
||||
// Load company-specific prompt if company is provided
|
||||
let companySpecific = null;
|
||||
if (company) {
|
||||
companySpecific = await loadPromptByType(pool, companyType, company);
|
||||
}
|
||||
|
||||
return {
|
||||
system,
|
||||
general,
|
||||
companySpecific
|
||||
};
|
||||
return { system, general, companySpecific };
|
||||
}
|
||||
|
||||
/**
|
||||
* Load name validation prompts
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string|null} [company] - Optional company ID
|
||||
* @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>}
|
||||
*/
|
||||
async function loadNameValidationPrompts(pool, company = null) {
|
||||
export function loadNameValidationPrompts(pool, company = null) {
|
||||
return loadPromptsByType(pool, 'name_validation', company);
|
||||
}
|
||||
|
||||
/**
|
||||
* Load description validation prompts
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string|null} [company] - Optional company ID
|
||||
* @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>}
|
||||
*/
|
||||
async function loadDescriptionValidationPrompts(pool, company = null) {
|
||||
export function loadDescriptionValidationPrompts(pool, company = null) {
|
||||
return loadPromptsByType(pool, 'description_validation', company);
|
||||
}
|
||||
|
||||
/**
|
||||
* Load sanity check prompts (no company-specific variant)
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @returns {Promise<{system: string|null, general: string|null, companySpecific: null}>}
|
||||
*/
|
||||
async function loadSanityCheckPrompts(pool) {
|
||||
export function loadSanityCheckPrompts(pool) {
|
||||
return loadPromptsByType(pool, 'sanity_check', null);
|
||||
}
|
||||
|
||||
/**
|
||||
* Load bulk validation prompts (GPT-5 validation)
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string|null} [company] - Optional company ID
|
||||
* @returns {Promise<{system: string|null, general: string|null, companySpecific: string|null}>}
|
||||
*/
|
||||
async function loadBulkValidationPrompts(pool, company = null) {
|
||||
export function loadBulkValidationPrompts(pool, company = null) {
|
||||
return loadPromptsByType(pool, 'bulk_validation', company);
|
||||
}
|
||||
|
||||
/**
|
||||
* Load bulk validation prompts for multiple companies at once
|
||||
* @param {Object} pool - PostgreSQL pool
|
||||
* @param {string[]} companyIds - Array of company IDs
|
||||
* @returns {Promise<{system: string|null, general: string|null, companyPrompts: Map<string, string>}>}
|
||||
*/
|
||||
async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) {
|
||||
// Load system and general prompts
|
||||
export async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) {
|
||||
const [system, general] = await Promise.all([
|
||||
loadPromptByType(pool, 'bulk_validation_system'),
|
||||
loadPromptByType(pool, 'bulk_validation_general')
|
||||
loadPromptByType(pool, 'bulk_validation_general'),
|
||||
]);
|
||||
|
||||
// Load company-specific prompts for all provided companies
|
||||
const companyPrompts = new Map();
|
||||
|
||||
if (companyIds.length > 0) {
|
||||
try {
|
||||
const result = await pool.query(
|
||||
@@ -132,7 +68,6 @@ async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) {
|
||||
AND company = ANY($1)`,
|
||||
[companyIds]
|
||||
);
|
||||
|
||||
for (const row of result.rows) {
|
||||
companyPrompts.set(row.company, row.prompt_text);
|
||||
}
|
||||
@@ -140,35 +75,14 @@ async function loadBulkValidationPromptsForCompanies(pool, companyIds = []) {
|
||||
console.error('[PromptLoader] Error loading company-specific prompts:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
system,
|
||||
general,
|
||||
companyPrompts
|
||||
};
|
||||
return { system, general, companyPrompts };
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate that required prompts exist, throw error if missing
|
||||
* @param {Object} prompts - Prompts object from loadPromptsByType
|
||||
* @param {string} baseType - Base type for error messages
|
||||
* @param {Object} options - Validation options
|
||||
* @param {boolean} [options.requireSystem=true] - Require system prompt
|
||||
* @param {boolean} [options.requireGeneral=true] - Require general prompt
|
||||
* @throws {Error} If required prompts are missing
|
||||
*/
|
||||
function validateRequiredPrompts(prompts, baseType, options = {}) {
|
||||
export function validateRequiredPrompts(prompts, baseType, options = {}) {
|
||||
const { requireSystem = true, requireGeneral = true } = options;
|
||||
const missing = [];
|
||||
|
||||
if (requireSystem && !prompts.system) {
|
||||
missing.push(`${baseType}_system`);
|
||||
}
|
||||
|
||||
if (requireGeneral && !prompts.general) {
|
||||
missing.push(`${baseType}_general`);
|
||||
}
|
||||
|
||||
if (requireSystem && !prompts.system) missing.push(`${baseType}_system`);
|
||||
if (requireGeneral && !prompts.general) missing.push(`${baseType}_general`);
|
||||
if (missing.length > 0) {
|
||||
throw new Error(
|
||||
`Missing required AI prompts: ${missing.join(', ')}. ` +
|
||||
@@ -176,19 +90,3 @@ function validateRequiredPrompts(prompts, baseType, options = {}) {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
// Core loader
|
||||
loadPromptByType,
|
||||
loadPromptsByType,
|
||||
|
||||
// Task-specific loaders
|
||||
loadNameValidationPrompts,
|
||||
loadDescriptionValidationPrompts,
|
||||
loadSanityCheckPrompts,
|
||||
loadBulkValidationPrompts,
|
||||
loadBulkValidationPromptsForCompanies,
|
||||
|
||||
// Validation
|
||||
validateRequiredPrompts
|
||||
};
|
||||
|
||||
@@ -1,21 +1,8 @@
|
||||
/**
|
||||
* Sanity Check Prompts
|
||||
*
|
||||
* Functions for building and parsing batch product consistency validation prompts.
|
||||
* System and general prompts are loaded from the database.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Build the user prompt for sanity check
|
||||
* Combines database prompts with product data
|
||||
*
|
||||
* @param {Object[]} products - Array of product data (limited fields for context)
|
||||
* @param {Object} prompts - Prompts loaded from database
|
||||
* @param {string} prompts.general - General sanity check rules
|
||||
* @returns {string} Complete user prompt
|
||||
*/
|
||||
function buildSanityCheckUserPrompt(products, prompts) {
|
||||
// Build a simplified product list for the prompt
|
||||
export function buildSanityCheckUserPrompt(products, prompts) {
|
||||
const productSummaries = products.map((p, index) => ({
|
||||
index,
|
||||
name: p.name,
|
||||
@@ -33,22 +20,17 @@ function buildSanityCheckUserPrompt(products, prompts) {
|
||||
weight: p.weight,
|
||||
length: p.length,
|
||||
width: p.width,
|
||||
height: p.height
|
||||
height: p.height,
|
||||
}));
|
||||
|
||||
const parts = [];
|
||||
|
||||
// Add general prompt/rules if provided
|
||||
if (prompts.general) {
|
||||
parts.push(prompts.general);
|
||||
parts.push(''); // Empty line for separation
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
// Add products to review
|
||||
parts.push(`PRODUCTS TO REVIEW (${products.length} items):`);
|
||||
parts.push(JSON.stringify(productSummaries, null, 2));
|
||||
|
||||
// Add response format
|
||||
parts.push('');
|
||||
parts.push('RESPOND WITH JSON:');
|
||||
parts.push(JSON.stringify({
|
||||
@@ -57,10 +39,10 @@ function buildSanityCheckUserPrompt(products, prompts) {
|
||||
productIndex: 0,
|
||||
field: 'msrp',
|
||||
issue: 'Description of the issue found',
|
||||
suggestion: 'Suggested fix or verification (optional)'
|
||||
}
|
||||
suggestion: 'Suggested fix or verification (optional)',
|
||||
},
|
||||
],
|
||||
summary: '2-3 sentences summarizing the overall product quality'
|
||||
summary: '2-3 sentences summarizing the overall product quality',
|
||||
}, null, 2));
|
||||
|
||||
parts.push('');
|
||||
@@ -69,60 +51,40 @@ function buildSanityCheckUserPrompt(products, prompts) {
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the AI response for sanity check
|
||||
*
|
||||
* @param {Object|null} parsed - Parsed JSON from AI
|
||||
* @param {string} content - Raw response content
|
||||
* @returns {Object}
|
||||
*/
|
||||
function parseSanityCheckResponse(parsed, content) {
|
||||
// If we got valid parsed JSON, use it
|
||||
export function parseSanityCheckResponse(parsed, content) {
|
||||
if (parsed && Array.isArray(parsed.issues)) {
|
||||
return {
|
||||
issues: parsed.issues.map(issue => ({
|
||||
issues: parsed.issues.map((issue) => ({
|
||||
productIndex: issue.productIndex ?? issue.index ?? 0,
|
||||
field: issue.field || 'unknown',
|
||||
issue: issue.issue || issue.message || '',
|
||||
suggestion: issue.suggestion || null
|
||||
suggestion: issue.suggestion || null,
|
||||
})),
|
||||
summary: parsed.summary || 'Review complete'
|
||||
summary: parsed.summary || 'Review complete',
|
||||
};
|
||||
}
|
||||
|
||||
// Try to extract from content if parsing failed
|
||||
try {
|
||||
// Try to find issues array
|
||||
const issuesMatch = content.match(/"issues"\s*:\s*\[([\s\S]*?)\]/);
|
||||
let issues = [];
|
||||
|
||||
if (issuesMatch) {
|
||||
// Try to parse the array content
|
||||
try {
|
||||
const arrayContent = `[${issuesMatch[1]}]`;
|
||||
const parsedIssues = JSON.parse(arrayContent);
|
||||
issues = parsedIssues.map(issue => ({
|
||||
issues = parsedIssues.map((issue) => ({
|
||||
productIndex: issue.productIndex ?? issue.index ?? 0,
|
||||
field: issue.field || 'unknown',
|
||||
issue: issue.issue || issue.message || '',
|
||||
suggestion: issue.suggestion || null
|
||||
suggestion: issue.suggestion || null,
|
||||
}));
|
||||
} catch {
|
||||
// Couldn't parse the array
|
||||
/* fall through */
|
||||
}
|
||||
}
|
||||
|
||||
// Try to find summary
|
||||
const summaryMatch = content.match(/"summary"\s*:\s*"([^"]+)"/);
|
||||
const summary = summaryMatch ? summaryMatch[1] : 'Review complete';
|
||||
|
||||
return { issues, summary };
|
||||
} catch {
|
||||
return { issues: [], summary: 'Could not parse review results' };
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
buildSanityCheckUserPrompt,
|
||||
parseSanityCheckResponse
|
||||
};
|
||||
|
||||
@@ -1,25 +1,15 @@
|
||||
/**
|
||||
* Groq Provider - Handles chat completions via Groq's OpenAI-compatible API
|
||||
*
|
||||
* Uses Groq's fast inference for real-time AI validation tasks.
|
||||
* Supports models like openai/gpt-oss-120b (complex) and openai/gpt-oss-20b (simple).
|
||||
* Groq Provider - chat completions via Groq's OpenAI-compatible API
|
||||
*/
|
||||
|
||||
const GROQ_BASE_URL = 'https://api.groq.com/openai/v1';
|
||||
export const GROQ_BASE_URL = 'https://api.groq.com/openai/v1';
|
||||
|
||||
// Default models
|
||||
const MODELS = {
|
||||
LARGE: 'openai/gpt-oss-120b', // For complex tasks (descriptions, sanity checks)
|
||||
SMALL: 'openai/gpt-oss-20b' // For simple tasks (name validation)
|
||||
export const MODELS = {
|
||||
LARGE: 'openai/gpt-oss-120b',
|
||||
SMALL: 'openai/gpt-oss-20b',
|
||||
};
|
||||
|
||||
class GroqProvider {
|
||||
/**
|
||||
* @param {Object} options
|
||||
* @param {string} options.apiKey - Groq API key
|
||||
* @param {string} [options.baseUrl] - Override base URL
|
||||
* @param {number} [options.timeoutMs=30000] - Default timeout
|
||||
*/
|
||||
export class GroqProvider {
|
||||
constructor({ apiKey, baseUrl = GROQ_BASE_URL, timeoutMs = 30000 }) {
|
||||
if (!apiKey) {
|
||||
throw new Error('Groq API key is required');
|
||||
@@ -29,41 +19,25 @@ class GroqProvider {
|
||||
this.timeoutMs = timeoutMs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Send a chat completion request
|
||||
*
|
||||
* @param {Object} params
|
||||
* @param {Array<{role: string, content: string}>} params.messages - Conversation messages
|
||||
* @param {string} [params.model] - Model to use (defaults to LARGE)
|
||||
* @param {number} [params.temperature=0.3] - Response randomness (0-2)
|
||||
* @param {number} [params.maxTokens=500] - Max tokens in response
|
||||
* @param {Object} [params.responseFormat] - For JSON mode: { type: 'json_object' }
|
||||
* @param {number} [params.timeoutMs] - Request timeout override
|
||||
* @returns {Promise<{content: string, parsed: Object|null, usage: Object, latencyMs: number, model: string}>}
|
||||
*/
|
||||
async chatCompletion({
|
||||
messages,
|
||||
model = MODELS.LARGE,
|
||||
temperature = 0.3,
|
||||
maxTokens = 500,
|
||||
responseFormat = null,
|
||||
timeoutMs = this.timeoutMs
|
||||
timeoutMs = this.timeoutMs,
|
||||
}) {
|
||||
const started = Date.now();
|
||||
|
||||
const body = {
|
||||
model,
|
||||
messages,
|
||||
temperature,
|
||||
max_completion_tokens: maxTokens
|
||||
max_completion_tokens: maxTokens,
|
||||
};
|
||||
|
||||
// Enable JSON mode if requested
|
||||
if (responseFormat?.type === 'json_object') {
|
||||
body.response_format = { type: 'json_object' };
|
||||
}
|
||||
|
||||
// Debug: Log request being sent
|
||||
console.log('[Groq] Request:', {
|
||||
model: body.model,
|
||||
temperature: body.temperature,
|
||||
@@ -71,12 +45,11 @@ class GroqProvider {
|
||||
hasResponseFormat: !!body.response_format,
|
||||
messageCount: body.messages?.length,
|
||||
systemPromptLength: body.messages?.[0]?.content?.length,
|
||||
userPromptLength: body.messages?.[1]?.content?.length
|
||||
userPromptLength: body.messages?.[1]?.content?.length,
|
||||
});
|
||||
|
||||
const response = await this._makeRequest('chat/completions', body, timeoutMs);
|
||||
|
||||
// Debug: Log raw response structure
|
||||
console.log('[Groq] Raw response:', {
|
||||
hasChoices: !!response.choices,
|
||||
choicesLength: response.choices?.length,
|
||||
@@ -84,22 +57,20 @@ class GroqProvider {
|
||||
finishReason: response.choices[0].finish_reason,
|
||||
hasMessage: !!response.choices[0].message,
|
||||
contentLength: response.choices[0].message?.content?.length,
|
||||
contentPreview: response.choices[0].message?.content?.substring(0, 200)
|
||||
contentPreview: response.choices[0].message?.content?.substring(0, 200),
|
||||
} : null,
|
||||
usage: response.usage,
|
||||
model: response.model
|
||||
model: response.model,
|
||||
});
|
||||
|
||||
const content = response.choices?.[0]?.message?.content || '';
|
||||
const usage = response.usage || {};
|
||||
|
||||
// Attempt to parse JSON if response format was requested
|
||||
let parsed = null;
|
||||
if (responseFormat && content) {
|
||||
try {
|
||||
parsed = JSON.parse(content);
|
||||
} catch {
|
||||
// Content isn't valid JSON - try to extract JSON from markdown
|
||||
parsed = this._extractJson(content);
|
||||
}
|
||||
}
|
||||
@@ -110,74 +81,50 @@ class GroqProvider {
|
||||
usage: {
|
||||
promptTokens: usage.prompt_tokens || 0,
|
||||
completionTokens: usage.completion_tokens || 0,
|
||||
totalTokens: usage.total_tokens || 0
|
||||
totalTokens: usage.total_tokens || 0,
|
||||
},
|
||||
latencyMs: Date.now() - started,
|
||||
model: response.model || model
|
||||
model: response.model || model,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract JSON from content that might be wrapped in markdown code blocks
|
||||
* @private
|
||||
*/
|
||||
_extractJson(content) {
|
||||
// Try to find JSON in code blocks
|
||||
const codeBlockMatch = content.match(/```(?:json)?\s*([\s\S]*?)```/);
|
||||
if (codeBlockMatch) {
|
||||
try {
|
||||
return JSON.parse(codeBlockMatch[1].trim());
|
||||
} catch {
|
||||
// Fall through
|
||||
}
|
||||
try { return JSON.parse(codeBlockMatch[1].trim()); } catch { /* fall through */ }
|
||||
}
|
||||
|
||||
// Try to find JSON object/array directly
|
||||
const jsonMatch = content.match(/(\{[\s\S]*\}|\[[\s\S]*\])/);
|
||||
if (jsonMatch) {
|
||||
try {
|
||||
return JSON.parse(jsonMatch[1]);
|
||||
} catch {
|
||||
// Fall through
|
||||
}
|
||||
try { return JSON.parse(jsonMatch[1]); } catch { /* fall through */ }
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Make an HTTP request to Groq API
|
||||
* @private
|
||||
*/
|
||||
async _makeRequest(endpoint, body, timeoutMs) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
try {
|
||||
const response = await fetch(`${this.baseUrl}/${endpoint}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${this.apiKey}`
|
||||
'Authorization': `Bearer ${this.apiKey}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
signal: controller.signal
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.json().catch(() => ({}));
|
||||
const message = error.error?.message || `Groq API error: ${response.status}`;
|
||||
const err = new Error(message);
|
||||
err.status = response.status;
|
||||
err.code = error.error?.code;
|
||||
// Include failed_generation if available (for JSON mode failures)
|
||||
if (error.error?.failed_generation) {
|
||||
err.failedGeneration = error.error.failed_generation;
|
||||
console.error('[Groq] JSON validation failed. Model output:', error.error.failed_generation);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
||||
return response.json();
|
||||
} catch (error) {
|
||||
if (error.name === 'AbortError') {
|
||||
@@ -191,13 +138,7 @@ class GroqProvider {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the provider is properly configured
|
||||
* @returns {boolean}
|
||||
*/
|
||||
isConfigured() {
|
||||
return !!this.apiKey;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { GroqProvider, MODELS, GROQ_BASE_URL };
|
||||
|
||||
@@ -2,11 +2,11 @@
|
||||
* OpenAI Provider - Handles embedding generation
|
||||
*/
|
||||
|
||||
const EMBEDDING_MODEL = 'text-embedding-3-small';
|
||||
const EMBEDDING_DIMENSIONS = 1536;
|
||||
export const EMBEDDING_MODEL = 'text-embedding-3-small';
|
||||
export const EMBEDDING_DIMENSIONS = 1536;
|
||||
const MAX_BATCH_SIZE = 2048;
|
||||
|
||||
class OpenAIProvider {
|
||||
export class OpenAIProvider {
|
||||
constructor({ apiKey, baseUrl = 'https://api.openai.com/v1', timeoutMs = 60000 }) {
|
||||
if (!apiKey) {
|
||||
throw new Error('OpenAI API key is required');
|
||||
@@ -16,12 +16,6 @@ class OpenAIProvider {
|
||||
this.timeoutMs = timeoutMs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate embeddings for one or more texts
|
||||
* @param {string|string[]} input - Text or array of texts
|
||||
* @param {Object} options
|
||||
* @returns {Promise<{embeddings: number[][], usage: Object, model: string, latencyMs: number}>}
|
||||
*/
|
||||
async embed(input, options = {}) {
|
||||
const texts = Array.isArray(input) ? input : [input];
|
||||
const model = options.model || EMBEDDING_MODEL;
|
||||
@@ -33,56 +27,39 @@ class OpenAIProvider {
|
||||
}
|
||||
|
||||
const started = Date.now();
|
||||
|
||||
// Clean and truncate input texts
|
||||
const cleanedTexts = texts.map(t =>
|
||||
const cleanedTexts = texts.map((t) =>
|
||||
(t || '').replace(/\n+/g, ' ').trim().substring(0, 8000)
|
||||
);
|
||||
|
||||
const body = {
|
||||
input: cleanedTexts,
|
||||
model,
|
||||
encoding_format: 'float'
|
||||
};
|
||||
|
||||
// Only embedding-3 models support dimensions parameter
|
||||
if (model.includes('embedding-3')) {
|
||||
body.dimensions = dimensions;
|
||||
}
|
||||
const body = { input: cleanedTexts, model, encoding_format: 'float' };
|
||||
if (model.includes('embedding-3')) body.dimensions = dimensions;
|
||||
|
||||
const response = await this._makeRequest('embeddings', body, timeoutMs);
|
||||
|
||||
// Sort by index to ensure order matches input
|
||||
const sortedData = response.data.sort((a, b) => a.index - b.index);
|
||||
|
||||
return {
|
||||
embeddings: sortedData.map(item => item.embedding),
|
||||
embeddings: sortedData.map((item) => item.embedding),
|
||||
usage: {
|
||||
promptTokens: response.usage?.prompt_tokens || 0,
|
||||
totalTokens: response.usage?.total_tokens || 0
|
||||
totalTokens: response.usage?.total_tokens || 0,
|
||||
},
|
||||
model: response.model || model,
|
||||
latencyMs: Date.now() - started
|
||||
latencyMs: Date.now() - started,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generator for processing large batches in chunks
|
||||
*/
|
||||
async *embedBatchChunked(texts, options = {}) {
|
||||
const batchSize = Math.min(options.batchSize || 100, MAX_BATCH_SIZE);
|
||||
|
||||
for (let i = 0; i < texts.length; i += batchSize) {
|
||||
const chunk = texts.slice(i, i + batchSize);
|
||||
const result = await this.embed(chunk, options);
|
||||
|
||||
yield {
|
||||
embeddings: result.embeddings,
|
||||
startIndex: i,
|
||||
endIndex: i + chunk.length,
|
||||
usage: result.usage,
|
||||
model: result.model,
|
||||
latencyMs: result.latencyMs
|
||||
latencyMs: result.latencyMs,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -90,28 +67,23 @@ class OpenAIProvider {
|
||||
async _makeRequest(endpoint, body, timeoutMs) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
try {
|
||||
const response = await fetch(`${this.baseUrl}/${endpoint}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${this.apiKey}`
|
||||
'Authorization': `Bearer ${this.apiKey}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
signal: controller.signal
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.json().catch(() => ({}));
|
||||
throw new Error(error.error?.message || `OpenAI API error: ${response.status}`);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { OpenAIProvider, EMBEDDING_MODEL, EMBEDDING_DIMENSIONS };
|
||||
|
||||
@@ -1,132 +1,84 @@
|
||||
/**
|
||||
* Description Validation Task
|
||||
*
|
||||
* Validates a product description for quality, accuracy, and guideline compliance.
|
||||
* Uses Groq with the larger model for better reasoning about content quality.
|
||||
* Loads all prompts from the database (no hardcoded prompts).
|
||||
*/
|
||||
|
||||
const { MODELS } = require('../providers/groqProvider');
|
||||
const {
|
||||
import { MODELS } from '../providers/groqProvider.js';
|
||||
import {
|
||||
loadDescriptionValidationPrompts,
|
||||
validateRequiredPrompts
|
||||
} = require('../prompts/promptLoader');
|
||||
const {
|
||||
validateRequiredPrompts,
|
||||
} from '../prompts/promptLoader.js';
|
||||
import {
|
||||
buildDescriptionUserPrompt,
|
||||
parseDescriptionResponse
|
||||
} = require('../prompts/descriptionPrompts');
|
||||
parseDescriptionResponse,
|
||||
} from '../prompts/descriptionPrompts.js';
|
||||
|
||||
const TASK_ID = 'validate.description';
|
||||
export const TASK_ID = 'validate.description';
|
||||
|
||||
/**
|
||||
* Create the description validation task
|
||||
*
|
||||
* @returns {Object} Task definition
|
||||
*/
|
||||
function createDescriptionValidationTask() {
|
||||
export function createDescriptionValidationTask() {
|
||||
return {
|
||||
id: TASK_ID,
|
||||
description: 'Validate product description for quality and guideline compliance',
|
||||
|
||||
/**
|
||||
* Run the description validation
|
||||
*
|
||||
* @param {Object} payload
|
||||
* @param {Object} payload.product - Product data
|
||||
* @param {string} payload.product.name - Product name (for context)
|
||||
* @param {string} payload.product.description - Description to validate
|
||||
* @param {string} [payload.product.company_name] - Company name
|
||||
* @param {string} [payload.product.company_id] - Company ID for loading specific rules
|
||||
* @param {string} [payload.product.categories] - Product categories
|
||||
* @param {Object} payload.provider - Groq provider instance
|
||||
* @param {Object} payload.pool - PostgreSQL pool
|
||||
* @param {Object} [payload.logger] - Logger instance
|
||||
* @returns {Promise<Object>}
|
||||
*/
|
||||
async run(payload) {
|
||||
const { product, provider, pool, logger } = payload;
|
||||
const log = logger || console;
|
||||
|
||||
// Validate required input
|
||||
if (!product?.name && !product?.description) {
|
||||
return {
|
||||
isValid: true,
|
||||
suggestion: null,
|
||||
issues: [],
|
||||
skipped: true,
|
||||
reason: 'No name or description provided'
|
||||
};
|
||||
}
|
||||
|
||||
if (!provider) {
|
||||
throw new Error('Groq provider not available');
|
||||
}
|
||||
|
||||
if (!pool) {
|
||||
throw new Error('Database pool not available');
|
||||
return { isValid: true, suggestion: null, issues: [], skipped: true, reason: 'No name or description provided' };
|
||||
}
|
||||
if (!provider) throw new Error('Groq provider not available');
|
||||
if (!pool) throw new Error('Database pool not available');
|
||||
|
||||
try {
|
||||
// Load prompts from database
|
||||
const companyKey = product.company_id || product.company_name || product.company;
|
||||
const prompts = await loadDescriptionValidationPrompts(pool, companyKey);
|
||||
|
||||
// Validate required prompts exist
|
||||
validateRequiredPrompts(prompts, 'description_validation');
|
||||
|
||||
// Build the user prompt with database-loaded prompts
|
||||
const userPrompt = buildDescriptionUserPrompt(product, prompts);
|
||||
|
||||
let response;
|
||||
let result;
|
||||
|
||||
try {
|
||||
// Try with JSON mode first
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.LARGE, // openai/gpt-oss-120b - better for content analysis
|
||||
temperature: 0.3, // Slightly higher for creative suggestions
|
||||
maxTokens: 2000, // Reasoning models need extra tokens for thinking
|
||||
responseFormat: { type: 'json_object' }
|
||||
model: MODELS.LARGE,
|
||||
temperature: 0.3,
|
||||
maxTokens: 2000,
|
||||
responseFormat: { type: 'json_object' },
|
||||
});
|
||||
|
||||
// Log full raw response for debugging
|
||||
log.info('[DescriptionValidation] Raw AI response:', {
|
||||
parsed: response.parsed,
|
||||
content: response.content,
|
||||
contentLength: response.content?.length
|
||||
contentLength: response.content?.length,
|
||||
});
|
||||
|
||||
// Parse the response
|
||||
result = parseDescriptionResponse(response.parsed, response.content);
|
||||
} catch (jsonError) {
|
||||
// If JSON mode failed, check if we have failedGeneration to parse
|
||||
if (jsonError.failedGeneration) {
|
||||
log.warn('[DescriptionValidation] JSON mode failed, attempting to parse failed_generation:', {
|
||||
failedGeneration: jsonError.failedGeneration
|
||||
failedGeneration: jsonError.failedGeneration,
|
||||
});
|
||||
result = parseDescriptionResponse(null, jsonError.failedGeneration);
|
||||
response = { latencyMs: 0, usage: {}, model: MODELS.LARGE };
|
||||
} else {
|
||||
// Retry without JSON mode
|
||||
log.warn('[DescriptionValidation] JSON mode failed, retrying without JSON mode');
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.LARGE,
|
||||
temperature: 0.3,
|
||||
maxTokens: 2000 // Reasoning models need extra tokens for thinking
|
||||
// No responseFormat - let the model respond freely
|
||||
maxTokens: 2000,
|
||||
});
|
||||
log.info('[DescriptionValidation] Raw AI response (no JSON mode):', {
|
||||
parsed: response.parsed,
|
||||
content: response.content,
|
||||
contentLength: response.content?.length
|
||||
contentLength: response.content?.length,
|
||||
});
|
||||
result = parseDescriptionResponse(response.parsed, response.content);
|
||||
}
|
||||
@@ -135,24 +87,19 @@ function createDescriptionValidationTask() {
|
||||
log.info(`[DescriptionValidation] Validated description for "${product.name}" in ${response.latencyMs}ms`, {
|
||||
isValid: result.isValid,
|
||||
hasSuggestion: !!result.suggestion,
|
||||
issueCount: result.issues.length
|
||||
issueCount: result.issues.length,
|
||||
});
|
||||
|
||||
return {
|
||||
...result,
|
||||
latencyMs: response.latencyMs,
|
||||
usage: response.usage,
|
||||
model: response.model
|
||||
model: response.model,
|
||||
};
|
||||
} catch (error) {
|
||||
log.error('[DescriptionValidation] Error:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
TASK_ID,
|
||||
createDescriptionValidationTask
|
||||
};
|
||||
|
||||
@@ -1,166 +1,87 @@
|
||||
/**
|
||||
* AI Task Registry
|
||||
*
|
||||
* Simple registry pattern for AI tasks. Each task has:
|
||||
* - id: Unique identifier
|
||||
* - run: Async function that executes the task
|
||||
*
|
||||
* This allows adding new AI capabilities without modifying core code.
|
||||
*/
|
||||
|
||||
const { createNameValidationTask, TASK_ID: NAME_TASK_ID } = require('./nameValidationTask');
|
||||
const { createDescriptionValidationTask, TASK_ID: DESC_TASK_ID } = require('./descriptionValidationTask');
|
||||
const { createSanityCheckTask, TASK_ID: SANITY_TASK_ID } = require('./sanityCheckTask');
|
||||
import { createNameValidationTask, TASK_ID as NAME_TASK_ID } from './nameValidationTask.js';
|
||||
import { createDescriptionValidationTask, TASK_ID as DESC_TASK_ID } from './descriptionValidationTask.js';
|
||||
import { createSanityCheckTask, TASK_ID as SANITY_TASK_ID } from './sanityCheckTask.js';
|
||||
|
||||
/**
|
||||
* Task IDs - frozen constants for type safety
|
||||
*/
|
||||
const TASK_IDS = Object.freeze({
|
||||
// Inline validation (triggered on field blur)
|
||||
export { createNameValidationTask, createDescriptionValidationTask, createSanityCheckTask };
|
||||
|
||||
export const TASK_IDS = Object.freeze({
|
||||
VALIDATE_NAME: NAME_TASK_ID,
|
||||
VALIDATE_DESCRIPTION: DESC_TASK_ID,
|
||||
|
||||
// Batch operations (triggered on user action)
|
||||
SANITY_CHECK: SANITY_TASK_ID
|
||||
SANITY_CHECK: SANITY_TASK_ID,
|
||||
});
|
||||
|
||||
/**
|
||||
* Task Registry
|
||||
*/
|
||||
class TaskRegistry {
|
||||
export class TaskRegistry {
|
||||
constructor() {
|
||||
this.tasks = new Map();
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a task
|
||||
* @param {Object} task
|
||||
* @param {string} task.id - Unique task identifier
|
||||
* @param {Function} task.run - Async function: (payload) => result
|
||||
* @param {string} [task.description] - Human-readable description
|
||||
*/
|
||||
register(task) {
|
||||
if (!task?.id) {
|
||||
throw new Error('Task must have an id');
|
||||
}
|
||||
if (typeof task.run !== 'function') {
|
||||
throw new Error(`Task ${task.id} must have a run function`);
|
||||
}
|
||||
if (this.tasks.has(task.id)) {
|
||||
throw new Error(`Task ${task.id} is already registered`);
|
||||
}
|
||||
|
||||
if (!task?.id) throw new Error('Task must have an id');
|
||||
if (typeof task.run !== 'function') throw new Error(`Task ${task.id} must have a run function`);
|
||||
if (this.tasks.has(task.id)) throw new Error(`Task ${task.id} is already registered`);
|
||||
this.tasks.set(task.id, task);
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a task by ID
|
||||
* @param {string} taskId
|
||||
* @returns {Object|null}
|
||||
*/
|
||||
get(taskId) {
|
||||
return this.tasks.get(taskId) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a task exists
|
||||
* @param {string} taskId
|
||||
* @returns {boolean}
|
||||
*/
|
||||
has(taskId) {
|
||||
return this.tasks.has(taskId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a task by ID
|
||||
* @param {string} taskId
|
||||
* @param {Object} payload - Task-specific input
|
||||
* @returns {Promise<Object>} Task result
|
||||
*/
|
||||
async runTask(taskId, payload = {}) {
|
||||
const task = this.get(taskId);
|
||||
if (!task) {
|
||||
throw new Error(`Unknown task: ${taskId}`);
|
||||
}
|
||||
|
||||
if (!task) throw new Error(`Unknown task: ${taskId}`);
|
||||
try {
|
||||
const result = await task.run(payload);
|
||||
return {
|
||||
success: true,
|
||||
taskId,
|
||||
...result
|
||||
};
|
||||
return { success: true, taskId, ...result };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
taskId,
|
||||
error: error.message,
|
||||
code: error.code
|
||||
code: error.code,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* List all registered task IDs
|
||||
* @returns {string[]}
|
||||
*/
|
||||
list() {
|
||||
return Array.from(this.tasks.keys());
|
||||
}
|
||||
|
||||
/**
|
||||
* Get count of registered tasks
|
||||
* @returns {number}
|
||||
*/
|
||||
size() {
|
||||
return this.tasks.size;
|
||||
}
|
||||
}
|
||||
|
||||
// Singleton instance
|
||||
let registry = null;
|
||||
|
||||
/**
|
||||
* Get or create the task registry
|
||||
* @returns {TaskRegistry}
|
||||
*/
|
||||
function getRegistry() {
|
||||
if (!registry) {
|
||||
registry = new TaskRegistry();
|
||||
}
|
||||
export function getRegistry() {
|
||||
if (!registry) registry = new TaskRegistry();
|
||||
return registry;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset the registry (mainly for testing)
|
||||
*/
|
||||
function resetRegistry() {
|
||||
export function resetRegistry() {
|
||||
registry = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Register all validation tasks with the registry
|
||||
* Call this during initialization after the registry is created
|
||||
*
|
||||
* @param {Object} [logger] - Optional logger
|
||||
*/
|
||||
function registerAllTasks(logger = console) {
|
||||
export function registerAllTasks(logger = console) {
|
||||
const reg = getRegistry();
|
||||
|
||||
// Register name validation
|
||||
if (!reg.has(TASK_IDS.VALIDATE_NAME)) {
|
||||
reg.register(createNameValidationTask());
|
||||
logger.info(`[Tasks] Registered: ${TASK_IDS.VALIDATE_NAME}`);
|
||||
}
|
||||
|
||||
// Register description validation
|
||||
if (!reg.has(TASK_IDS.VALIDATE_DESCRIPTION)) {
|
||||
reg.register(createDescriptionValidationTask());
|
||||
logger.info(`[Tasks] Registered: ${TASK_IDS.VALIDATE_DESCRIPTION}`);
|
||||
}
|
||||
|
||||
// Register sanity check
|
||||
if (!reg.has(TASK_IDS.SANITY_CHECK)) {
|
||||
reg.register(createSanityCheckTask());
|
||||
logger.info(`[Tasks] Registered: ${TASK_IDS.SANITY_CHECK}`);
|
||||
@@ -168,19 +89,3 @@ function registerAllTasks(logger = console) {
|
||||
|
||||
return reg;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
// Constants
|
||||
TASK_IDS,
|
||||
|
||||
// Registry
|
||||
TaskRegistry,
|
||||
getRegistry,
|
||||
resetRegistry,
|
||||
registerAllTasks,
|
||||
|
||||
// Task factories (for custom registration)
|
||||
createNameValidationTask,
|
||||
createDescriptionValidationTask,
|
||||
createSanityCheckTask
|
||||
};
|
||||
|
||||
@@ -1,77 +1,38 @@
|
||||
/**
|
||||
* Name Validation Task
|
||||
*
|
||||
* Validates a product name for spelling, grammar, and naming conventions.
|
||||
* Uses Groq with the smaller model for fast response times.
|
||||
* Loads all prompts from the database (no hardcoded prompts).
|
||||
*/
|
||||
|
||||
const { MODELS } = require('../providers/groqProvider');
|
||||
const {
|
||||
import { MODELS } from '../providers/groqProvider.js';
|
||||
import {
|
||||
loadNameValidationPrompts,
|
||||
validateRequiredPrompts
|
||||
} = require('../prompts/promptLoader');
|
||||
const {
|
||||
validateRequiredPrompts,
|
||||
} from '../prompts/promptLoader.js';
|
||||
import {
|
||||
buildNameUserPrompt,
|
||||
parseNameResponse
|
||||
} = require('../prompts/namePrompts');
|
||||
parseNameResponse,
|
||||
} from '../prompts/namePrompts.js';
|
||||
|
||||
const TASK_ID = 'validate.name';
|
||||
export const TASK_ID = 'validate.name';
|
||||
|
||||
/**
|
||||
* Create the name validation task
|
||||
*
|
||||
* @returns {Object} Task definition
|
||||
*/
|
||||
function createNameValidationTask() {
|
||||
export function createNameValidationTask() {
|
||||
return {
|
||||
id: TASK_ID,
|
||||
description: 'Validate product name for spelling, grammar, and conventions',
|
||||
|
||||
/**
|
||||
* Run the name validation
|
||||
*
|
||||
* @param {Object} payload
|
||||
* @param {Object} payload.product - Product data
|
||||
* @param {string} payload.product.name - Product name to validate
|
||||
* @param {string} [payload.product.company_name] - Company name
|
||||
* @param {string} [payload.product.company_id] - Company ID for loading specific rules
|
||||
* @param {string} [payload.product.line_name] - Product line
|
||||
* @param {string} [payload.product.description] - Description for context
|
||||
* @param {Object} payload.provider - Groq provider instance
|
||||
* @param {Object} payload.pool - PostgreSQL pool
|
||||
* @param {Object} [payload.logger] - Logger instance
|
||||
* @returns {Promise<Object>}
|
||||
*/
|
||||
async run(payload) {
|
||||
const { product, provider, pool, logger } = payload;
|
||||
const log = logger || console;
|
||||
|
||||
// Validate required input
|
||||
if (!product?.name) {
|
||||
return {
|
||||
isValid: true,
|
||||
suggestion: null,
|
||||
issues: [],
|
||||
skipped: true,
|
||||
reason: 'No name provided'
|
||||
};
|
||||
}
|
||||
|
||||
if (!provider) {
|
||||
throw new Error('Groq provider not available');
|
||||
}
|
||||
|
||||
if (!pool) {
|
||||
throw new Error('Database pool not available');
|
||||
return { isValid: true, suggestion: null, issues: [], skipped: true, reason: 'No name provided' };
|
||||
}
|
||||
if (!provider) throw new Error('Groq provider not available');
|
||||
if (!pool) throw new Error('Database pool not available');
|
||||
|
||||
try {
|
||||
// Load prompts from database
|
||||
const companyKey = product.company_id || product.company_name || product.company;
|
||||
const prompts = await loadNameValidationPrompts(pool, companyKey);
|
||||
|
||||
// Debug: Log loaded prompts
|
||||
log.info('[NameValidation] Loaded prompts:', {
|
||||
hasSystem: !!prompts.system,
|
||||
systemLength: prompts.system?.length || 0,
|
||||
@@ -79,68 +40,57 @@ function createNameValidationTask() {
|
||||
generalLength: prompts.general?.length || 0,
|
||||
generalPreview: prompts.general?.substring(0, 100) || '(empty)',
|
||||
hasCompanySpecific: !!prompts.companySpecific,
|
||||
companyKey
|
||||
companyKey,
|
||||
});
|
||||
|
||||
// Validate required prompts exist
|
||||
validateRequiredPrompts(prompts, 'name_validation');
|
||||
|
||||
// Build the user prompt with database-loaded prompts
|
||||
const userPrompt = buildNameUserPrompt(product, prompts);
|
||||
|
||||
// Debug: Log the full user prompt being sent
|
||||
log.info('[NameValidation] User prompt:', userPrompt.substring(0, 500));
|
||||
|
||||
let response;
|
||||
let result;
|
||||
|
||||
try {
|
||||
// Try with JSON mode first
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.LARGE, // openai/gpt-oss-120b - reasoning model
|
||||
temperature: 0.2, // Low temperature for consistent results
|
||||
maxTokens: 3000, // Reasoning models need extra tokens for thinking
|
||||
responseFormat: { type: 'json_object' }
|
||||
model: MODELS.LARGE,
|
||||
temperature: 0.2,
|
||||
maxTokens: 3000,
|
||||
responseFormat: { type: 'json_object' },
|
||||
});
|
||||
|
||||
// Log full raw response for debugging
|
||||
log.info('[NameValidation] Raw AI response:', {
|
||||
parsed: response.parsed,
|
||||
content: response.content,
|
||||
contentLength: response.content?.length
|
||||
contentLength: response.content?.length,
|
||||
});
|
||||
|
||||
// Parse the response
|
||||
result = parseNameResponse(response.parsed, response.content);
|
||||
} catch (jsonError) {
|
||||
// If JSON mode failed, check if we have failedGeneration to parse
|
||||
if (jsonError.failedGeneration) {
|
||||
log.warn('[NameValidation] JSON mode failed, attempting to parse failed_generation:', {
|
||||
failedGeneration: jsonError.failedGeneration
|
||||
failedGeneration: jsonError.failedGeneration,
|
||||
});
|
||||
result = parseNameResponse(null, jsonError.failedGeneration);
|
||||
response = { latencyMs: 0, usage: {}, model: MODELS.SMALL };
|
||||
} else {
|
||||
// Retry without JSON mode
|
||||
log.warn('[NameValidation] JSON mode failed, retrying without JSON mode');
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.SMALL,
|
||||
temperature: 0.2,
|
||||
maxTokens: 1500 // Reasoning models need extra tokens for thinking
|
||||
// No responseFormat - let the model respond freely
|
||||
maxTokens: 1500,
|
||||
});
|
||||
log.info('[NameValidation] Raw AI response (no JSON mode):', {
|
||||
parsed: response.parsed,
|
||||
content: response.content,
|
||||
contentLength: response.content?.length
|
||||
contentLength: response.content?.length,
|
||||
});
|
||||
result = parseNameResponse(response.parsed, response.content);
|
||||
}
|
||||
@@ -149,24 +99,19 @@ function createNameValidationTask() {
|
||||
log.info(`[NameValidation] Validated "${product.name}" in ${response.latencyMs}ms`, {
|
||||
isValid: result.isValid,
|
||||
hassuggestion: !!result.suggestion,
|
||||
issueCount: result.issues.length
|
||||
issueCount: result.issues.length,
|
||||
});
|
||||
|
||||
return {
|
||||
...result,
|
||||
latencyMs: response.latencyMs,
|
||||
usage: response.usage,
|
||||
model: response.model
|
||||
model: response.model,
|
||||
};
|
||||
} catch (error) {
|
||||
log.error('[NameValidation] Error:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
TASK_ID,
|
||||
createNameValidationTask
|
||||
};
|
||||
|
||||
@@ -1,96 +1,55 @@
|
||||
/**
|
||||
* Sanity Check Task
|
||||
*
|
||||
* Reviews a batch of products for consistency and appropriateness.
|
||||
* Uses Groq with the larger model for complex batch analysis.
|
||||
* Loads all prompts from the database (no hardcoded prompts).
|
||||
*/
|
||||
|
||||
const { MODELS } = require('../providers/groqProvider');
|
||||
const {
|
||||
import { MODELS } from '../providers/groqProvider.js';
|
||||
import {
|
||||
loadSanityCheckPrompts,
|
||||
validateRequiredPrompts
|
||||
} = require('../prompts/promptLoader');
|
||||
const {
|
||||
validateRequiredPrompts,
|
||||
} from '../prompts/promptLoader.js';
|
||||
import {
|
||||
buildSanityCheckUserPrompt,
|
||||
parseSanityCheckResponse
|
||||
} = require('../prompts/sanityCheckPrompts');
|
||||
parseSanityCheckResponse,
|
||||
} from '../prompts/sanityCheckPrompts.js';
|
||||
|
||||
const TASK_ID = 'sanity.check';
|
||||
export const TASK_ID = 'sanity.check';
|
||||
export const MAX_PRODUCTS_PER_REQUEST = 50;
|
||||
|
||||
// Maximum products to send in a single request (to avoid token limits)
|
||||
const MAX_PRODUCTS_PER_REQUEST = 50;
|
||||
|
||||
/**
|
||||
* Create the sanity check task
|
||||
*
|
||||
* @returns {Object} Task definition
|
||||
*/
|
||||
function createSanityCheckTask() {
|
||||
export function createSanityCheckTask() {
|
||||
return {
|
||||
id: TASK_ID,
|
||||
description: 'Review batch of products for consistency and appropriateness',
|
||||
|
||||
/**
|
||||
* Run the sanity check
|
||||
*
|
||||
* @param {Object} payload
|
||||
* @param {Object[]} payload.products - Array of products to check
|
||||
* @param {Object} payload.provider - Groq provider instance
|
||||
* @param {Object} payload.pool - PostgreSQL pool
|
||||
* @param {Object} [payload.logger] - Logger instance
|
||||
* @returns {Promise<Object>}
|
||||
*/
|
||||
async run(payload) {
|
||||
const { products, provider, pool, logger } = payload;
|
||||
const log = logger || console;
|
||||
|
||||
// Validate required input
|
||||
if (!Array.isArray(products) || products.length === 0) {
|
||||
return {
|
||||
issues: [],
|
||||
summary: 'No products to check',
|
||||
skipped: true
|
||||
};
|
||||
}
|
||||
|
||||
if (!provider) {
|
||||
throw new Error('Groq provider not available');
|
||||
}
|
||||
|
||||
if (!pool) {
|
||||
throw new Error('Database pool not available');
|
||||
return { issues: [], summary: 'No products to check', skipped: true };
|
||||
}
|
||||
if (!provider) throw new Error('Groq provider not available');
|
||||
if (!pool) throw new Error('Database pool not available');
|
||||
|
||||
try {
|
||||
// Load prompts from database
|
||||
const prompts = await loadSanityCheckPrompts(pool);
|
||||
|
||||
// Validate required prompts exist
|
||||
validateRequiredPrompts(prompts, 'sanity_check');
|
||||
|
||||
// If batch is small enough, process in one request
|
||||
if (products.length <= MAX_PRODUCTS_PER_REQUEST) {
|
||||
return await checkBatch(products, prompts, provider, log);
|
||||
}
|
||||
|
||||
// Otherwise, process in chunks and combine results
|
||||
log.info(`[SanityCheck] Processing ${products.length} products in chunks`);
|
||||
const allIssues = [];
|
||||
const summaries = [];
|
||||
|
||||
for (let i = 0; i < products.length; i += MAX_PRODUCTS_PER_REQUEST) {
|
||||
const chunk = products.slice(i, i + MAX_PRODUCTS_PER_REQUEST);
|
||||
const chunkOffset = i; // To adjust product indices in results
|
||||
|
||||
const chunkOffset = i;
|
||||
const result = await checkBatch(chunk, prompts, provider, log);
|
||||
|
||||
// Adjust product indices to match original array
|
||||
const adjustedIssues = result.issues.map(issue => ({
|
||||
const adjustedIssues = result.issues.map((issue) => ({
|
||||
...issue,
|
||||
productIndex: issue.productIndex + chunkOffset
|
||||
productIndex: issue.productIndex + chunkOffset,
|
||||
}));
|
||||
|
||||
allIssues.push(...adjustedIssues);
|
||||
summaries.push(result.summary);
|
||||
}
|
||||
@@ -101,82 +60,61 @@ function createSanityCheckTask() {
|
||||
? `Reviewed ${products.length} products in ${summaries.length} batches. ${allIssues.length} issues found.`
|
||||
: summaries[0],
|
||||
totalProducts: products.length,
|
||||
issueCount: allIssues.length
|
||||
issueCount: allIssues.length,
|
||||
};
|
||||
} catch (error) {
|
||||
log.error('[SanityCheck] Error:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Check a single batch of products
|
||||
*
|
||||
* @param {Object[]} products - Products to check
|
||||
* @param {Object} prompts - Loaded prompts from database
|
||||
* @param {Object} provider - Groq provider
|
||||
* @param {Object} log - Logger
|
||||
* @returns {Promise<Object>}
|
||||
*/
|
||||
async function checkBatch(products, prompts, provider, log) {
|
||||
const userPrompt = buildSanityCheckUserPrompt(products, prompts);
|
||||
|
||||
let response;
|
||||
let result;
|
||||
|
||||
try {
|
||||
// Try with JSON mode first
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.LARGE, // openai/gpt-oss-120b - needed for complex batch analysis
|
||||
temperature: 0.2, // Low temperature for consistent analysis
|
||||
maxTokens: 2000, // More tokens for batch results
|
||||
responseFormat: { type: 'json_object' }
|
||||
model: MODELS.LARGE,
|
||||
temperature: 0.2,
|
||||
maxTokens: 2000,
|
||||
responseFormat: { type: 'json_object' },
|
||||
});
|
||||
|
||||
result = parseSanityCheckResponse(response.parsed, response.content);
|
||||
} catch (jsonError) {
|
||||
// If JSON mode failed, check if we have failedGeneration to parse
|
||||
if (jsonError.failedGeneration) {
|
||||
log.warn('[SanityCheck] JSON mode failed, attempting to parse failed_generation');
|
||||
result = parseSanityCheckResponse(null, jsonError.failedGeneration);
|
||||
response = { latencyMs: 0, usage: {}, model: MODELS.LARGE };
|
||||
} else {
|
||||
// Retry without JSON mode
|
||||
log.warn('[SanityCheck] JSON mode failed, retrying without JSON mode');
|
||||
response = await provider.chatCompletion({
|
||||
messages: [
|
||||
{ role: 'system', content: prompts.system },
|
||||
{ role: 'user', content: userPrompt }
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
model: MODELS.LARGE,
|
||||
temperature: 0.2,
|
||||
maxTokens: 2000
|
||||
// No responseFormat - let the model respond freely
|
||||
maxTokens: 2000,
|
||||
});
|
||||
result = parseSanityCheckResponse(response.parsed, response.content);
|
||||
}
|
||||
}
|
||||
|
||||
log.info(`[SanityCheck] Checked ${products.length} products in ${response.latencyMs}ms`, {
|
||||
issueCount: result.issues.length
|
||||
issueCount: result.issues.length,
|
||||
});
|
||||
|
||||
return {
|
||||
...result,
|
||||
latencyMs: response.latencyMs,
|
||||
usage: response.usage,
|
||||
model: response.model
|
||||
model: response.model,
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
TASK_ID,
|
||||
createSanityCheckTask,
|
||||
MAX_PRODUCTS_PER_REQUEST
|
||||
};
|
||||
|
||||
@@ -1,79 +1,63 @@
|
||||
// Purchase Order Status Codes
|
||||
const PurchaseOrderStatus = {
|
||||
Canceled: 0,
|
||||
Created: 1,
|
||||
ElectronicallyReadySend: 10,
|
||||
Ordered: 11,
|
||||
Preordered: 12,
|
||||
ElectronicallySent: 13,
|
||||
ReceivingStarted: 15,
|
||||
Done: 50
|
||||
export const PurchaseOrderStatus = {
|
||||
Canceled: 0,
|
||||
Created: 1,
|
||||
ElectronicallyReadySend: 10,
|
||||
Ordered: 11,
|
||||
Preordered: 12,
|
||||
ElectronicallySent: 13,
|
||||
ReceivingStarted: 15,
|
||||
Done: 50,
|
||||
};
|
||||
|
||||
// Receiving Status Codes
|
||||
const ReceivingStatus = {
|
||||
Canceled: 0,
|
||||
Created: 1,
|
||||
PartialReceived: 30,
|
||||
FullReceived: 40,
|
||||
Paid: 50
|
||||
export const ReceivingStatus = {
|
||||
Canceled: 0,
|
||||
Created: 1,
|
||||
PartialReceived: 30,
|
||||
FullReceived: 40,
|
||||
Paid: 50,
|
||||
};
|
||||
|
||||
// Status Code Display Names
|
||||
const PurchaseOrderStatusLabels = {
|
||||
[PurchaseOrderStatus.Canceled]: 'Canceled',
|
||||
[PurchaseOrderStatus.Created]: 'Created',
|
||||
[PurchaseOrderStatus.ElectronicallyReadySend]: 'Ready to Send',
|
||||
[PurchaseOrderStatus.Ordered]: 'Ordered',
|
||||
[PurchaseOrderStatus.Preordered]: 'Preordered',
|
||||
[PurchaseOrderStatus.ElectronicallySent]: 'Sent',
|
||||
[PurchaseOrderStatus.ReceivingStarted]: 'Receiving Started',
|
||||
[PurchaseOrderStatus.Done]: 'Done'
|
||||
export const PurchaseOrderStatusLabels = {
|
||||
[PurchaseOrderStatus.Canceled]: 'Canceled',
|
||||
[PurchaseOrderStatus.Created]: 'Created',
|
||||
[PurchaseOrderStatus.ElectronicallyReadySend]: 'Ready to Send',
|
||||
[PurchaseOrderStatus.Ordered]: 'Ordered',
|
||||
[PurchaseOrderStatus.Preordered]: 'Preordered',
|
||||
[PurchaseOrderStatus.ElectronicallySent]: 'Sent',
|
||||
[PurchaseOrderStatus.ReceivingStarted]: 'Receiving Started',
|
||||
[PurchaseOrderStatus.Done]: 'Done',
|
||||
};
|
||||
|
||||
const ReceivingStatusLabels = {
|
||||
[ReceivingStatus.Canceled]: 'Canceled',
|
||||
[ReceivingStatus.Created]: 'Created',
|
||||
[ReceivingStatus.PartialReceived]: 'Partially Received',
|
||||
[ReceivingStatus.FullReceived]: 'Fully Received',
|
||||
[ReceivingStatus.Paid]: 'Paid'
|
||||
export const ReceivingStatusLabels = {
|
||||
[ReceivingStatus.Canceled]: 'Canceled',
|
||||
[ReceivingStatus.Created]: 'Created',
|
||||
[ReceivingStatus.PartialReceived]: 'Partially Received',
|
||||
[ReceivingStatus.FullReceived]: 'Fully Received',
|
||||
[ReceivingStatus.Paid]: 'Paid',
|
||||
};
|
||||
|
||||
// Helper functions
|
||||
function getPurchaseOrderStatusLabel(status) {
|
||||
return PurchaseOrderStatusLabels[status] || 'Unknown';
|
||||
export function getPurchaseOrderStatusLabel(status) {
|
||||
return PurchaseOrderStatusLabels[status] || 'Unknown';
|
||||
}
|
||||
|
||||
function getReceivingStatusLabel(status) {
|
||||
return ReceivingStatusLabels[status] || 'Unknown';
|
||||
export function getReceivingStatusLabel(status) {
|
||||
return ReceivingStatusLabels[status] || 'Unknown';
|
||||
}
|
||||
|
||||
// Status checks
|
||||
function isReceivingComplete(status) {
|
||||
return status >= ReceivingStatus.PartialReceived;
|
||||
export function isReceivingComplete(status) {
|
||||
return status >= ReceivingStatus.PartialReceived;
|
||||
}
|
||||
|
||||
function isPurchaseOrderComplete(status) {
|
||||
return status === PurchaseOrderStatus.Done;
|
||||
export function isPurchaseOrderComplete(status) {
|
||||
return status === PurchaseOrderStatus.Done;
|
||||
}
|
||||
|
||||
function isPurchaseOrderCanceled(status) {
|
||||
return status === PurchaseOrderStatus.Canceled;
|
||||
export function isPurchaseOrderCanceled(status) {
|
||||
return status === PurchaseOrderStatus.Canceled;
|
||||
}
|
||||
|
||||
function isReceivingCanceled(status) {
|
||||
return status === ReceivingStatus.Canceled;
|
||||
export function isReceivingCanceled(status) {
|
||||
return status === ReceivingStatus.Canceled;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
PurchaseOrderStatus,
|
||||
ReceivingStatus,
|
||||
PurchaseOrderStatusLabels,
|
||||
ReceivingStatusLabels,
|
||||
getPurchaseOrderStatusLabel,
|
||||
getReceivingStatusLabel,
|
||||
isReceivingComplete,
|
||||
isPurchaseOrderComplete,
|
||||
isPurchaseOrderCanceled,
|
||||
isReceivingCanceled
|
||||
};
|
||||
@@ -1,45 +1,29 @@
|
||||
/**
|
||||
* Parses a query parameter value based on its expected type.
|
||||
* Throws error for invalid formats. Adjust date handling as needed.
|
||||
* Throws on invalid formats.
|
||||
*/
|
||||
function parseValue(value, type) {
|
||||
if (value === null || value === undefined || value === '') return null;
|
||||
export function parseValue(value, type) {
|
||||
if (value === null || value === undefined || value === '') return null;
|
||||
|
||||
console.log(`Parsing value: "${value}" as type: "${type}"`);
|
||||
|
||||
switch (type) {
|
||||
case 'number':
|
||||
const num = parseFloat(value);
|
||||
if (isNaN(num)) {
|
||||
console.error(`Invalid number format: "${value}"`);
|
||||
throw new Error(`Invalid number format: "${value}"`);
|
||||
}
|
||||
return num;
|
||||
case 'integer': // Specific type for integer IDs etc.
|
||||
const int = parseInt(value, 10);
|
||||
if (isNaN(int)) {
|
||||
console.error(`Invalid integer format: "${value}"`);
|
||||
throw new Error(`Invalid integer format: "${value}"`);
|
||||
}
|
||||
console.log(`Successfully parsed integer: ${int}`);
|
||||
return int;
|
||||
case 'boolean':
|
||||
if (String(value).toLowerCase() === 'true') return true;
|
||||
if (String(value).toLowerCase() === 'false') return false;
|
||||
console.error(`Invalid boolean format: "${value}"`);
|
||||
throw new Error(`Invalid boolean format: "${value}"`);
|
||||
case 'date':
|
||||
// Basic ISO date format validation (YYYY-MM-DD)
|
||||
if (!String(value).match(/^\d{4}-\d{2}-\d{2}$/)) {
|
||||
console.warn(`Potentially invalid date format passed: "${value}"`);
|
||||
// Optionally throw an error or return null depending on strictness
|
||||
// throw new Error(`Invalid date format (YYYY-MM-DD expected): "${value}"`);
|
||||
}
|
||||
return String(value); // Send as string, let DB handle casting/comparison
|
||||
case 'string':
|
||||
default:
|
||||
return String(value);
|
||||
switch (type) {
|
||||
case 'number': {
|
||||
const num = parseFloat(value);
|
||||
if (Number.isNaN(num)) throw new Error(`Invalid number format: "${value}"`);
|
||||
return num;
|
||||
}
|
||||
case 'integer': {
|
||||
const int = parseInt(value, 10);
|
||||
if (Number.isNaN(int)) throw new Error(`Invalid integer format: "${value}"`);
|
||||
return int;
|
||||
}
|
||||
case 'boolean':
|
||||
if (String(value).toLowerCase() === 'true') return true;
|
||||
if (String(value).toLowerCase() === 'false') return false;
|
||||
throw new Error(`Invalid boolean format: "${value}"`);
|
||||
case 'date':
|
||||
return String(value);
|
||||
case 'string':
|
||||
default:
|
||||
return String(value);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { parseValue };
|
||||
@@ -1,45 +1,37 @@
|
||||
const fs = require('fs');
|
||||
const { parse } = require('csv-parse');
|
||||
const { v4: uuidv4 } = require('uuid');
|
||||
import fs from 'node:fs';
|
||||
import { parse } from 'csv-parse';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
|
||||
async function importProductsFromCSV(filePath, pool) {
|
||||
export async function importProductsFromCSV(filePath, pool) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const products = [];
|
||||
|
||||
fs.createReadStream(filePath)
|
||||
.pipe(parse({
|
||||
columns: true,
|
||||
skip_empty_lines: true
|
||||
}))
|
||||
.on('data', async (row) => {
|
||||
.pipe(parse({ columns: true, skip_empty_lines: true }))
|
||||
.on('data', (row) => {
|
||||
products.push({
|
||||
id: uuidv4(),
|
||||
sku: row.sku,
|
||||
name: row.name,
|
||||
description: row.description || null,
|
||||
category: row.category || null
|
||||
category: row.category || null,
|
||||
});
|
||||
})
|
||||
.on('end', async () => {
|
||||
try {
|
||||
const connection = await pool.getConnection();
|
||||
|
||||
try {
|
||||
await connection.beginTransaction();
|
||||
|
||||
for (const product of products) {
|
||||
await connection.query(
|
||||
'INSERT INTO products (id, sku, name, description, category) VALUES (?, ?, ?, ?, ?)',
|
||||
[product.id, product.sku, product.name, product.description, product.category]
|
||||
);
|
||||
|
||||
// Initialize inventory level for the product
|
||||
await connection.query(
|
||||
'INSERT INTO inventory_levels (id, product_id, quantity) VALUES (?, ?, 0)',
|
||||
[uuidv4(), product.id]
|
||||
);
|
||||
}
|
||||
|
||||
await connection.commit();
|
||||
resolve({ imported: products.length });
|
||||
} catch (error) {
|
||||
@@ -52,12 +44,6 @@ async function importProductsFromCSV(filePath, pool) {
|
||||
reject(error);
|
||||
}
|
||||
})
|
||||
.on('error', (error) => {
|
||||
reject(error);
|
||||
});
|
||||
.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
importProductsFromCSV
|
||||
};
|
||||
@@ -1,21 +1,23 @@
|
||||
const { Pool } = require('pg');
|
||||
import pg from 'pg';
|
||||
|
||||
const { Pool } = pg;
|
||||
|
||||
let pool;
|
||||
|
||||
function initPool(config) {
|
||||
export function initPool(config) {
|
||||
pool = new Pool(config);
|
||||
return pool;
|
||||
}
|
||||
|
||||
async function getConnection() {
|
||||
export async function getConnection() {
|
||||
if (!pool) {
|
||||
throw new Error('Database pool not initialized');
|
||||
}
|
||||
return pool.connect();
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
initPool,
|
||||
getConnection,
|
||||
getPool: () => pool
|
||||
};
|
||||
export function getPool() {
|
||||
return pool;
|
||||
}
|
||||
|
||||
export default { initPool, getConnection, getPool };
|
||||
|
||||
@@ -1,158 +1,94 @@
|
||||
const { Client } = require('ssh2');
|
||||
const mysql = require('mysql2/promise');
|
||||
const fs = require('fs');
|
||||
import { Client } from 'ssh2';
|
||||
import mysql from 'mysql2/promise';
|
||||
import fs from 'node:fs';
|
||||
|
||||
// Connection pooling and cache configuration
|
||||
const connectionCache = {
|
||||
ssh: null,
|
||||
dbConnection: null,
|
||||
lastUsed: 0,
|
||||
isConnecting: false,
|
||||
connectionPromise: null,
|
||||
// Cache expiration time in milliseconds (5 minutes)
|
||||
expirationTime: 5 * 60 * 1000,
|
||||
// Cache for query results (key: query string, value: {data, timestamp})
|
||||
queryCache: new Map(),
|
||||
// Cache duration for different query types in milliseconds
|
||||
cacheDuration: {
|
||||
'field-options': 30 * 60 * 1000, // 30 minutes for field options
|
||||
'product-lines': 10 * 60 * 1000, // 10 minutes for product lines
|
||||
'sublines': 10 * 60 * 1000, // 10 minutes for sublines
|
||||
'taxonomy': 30 * 60 * 1000, // 30 minutes for taxonomy data
|
||||
'default': 60 * 1000 // 1 minute default
|
||||
}
|
||||
'field-options': 30 * 60 * 1000,
|
||||
'product-lines': 10 * 60 * 1000,
|
||||
'sublines': 10 * 60 * 1000,
|
||||
'taxonomy': 30 * 60 * 1000,
|
||||
'default': 60 * 1000,
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Get a database connection with connection pooling
|
||||
* @returns {Promise<{ssh: object, connection: object}>} The SSH and database connection
|
||||
*/
|
||||
async function getDbConnection() {
|
||||
export async function getDbConnection() {
|
||||
const now = Date.now();
|
||||
const needsRefresh = !connectionCache.ssh
|
||||
|| !connectionCache.dbConnection
|
||||
|| (now - connectionCache.lastUsed > connectionCache.expirationTime);
|
||||
|
||||
// Check if we need to refresh the connection due to inactivity
|
||||
const needsRefresh = !connectionCache.ssh ||
|
||||
!connectionCache.dbConnection ||
|
||||
(now - connectionCache.lastUsed > connectionCache.expirationTime);
|
||||
|
||||
// If connection is still valid, update last used time and return existing connection
|
||||
if (!needsRefresh) {
|
||||
connectionCache.lastUsed = now;
|
||||
return {
|
||||
ssh: connectionCache.ssh,
|
||||
connection: connectionCache.dbConnection
|
||||
};
|
||||
return { ssh: connectionCache.ssh, connection: connectionCache.dbConnection };
|
||||
}
|
||||
|
||||
// If another request is already establishing a connection, wait for that promise
|
||||
if (connectionCache.isConnecting && connectionCache.connectionPromise) {
|
||||
try {
|
||||
await connectionCache.connectionPromise;
|
||||
return {
|
||||
ssh: connectionCache.ssh,
|
||||
connection: connectionCache.dbConnection
|
||||
};
|
||||
return { ssh: connectionCache.ssh, connection: connectionCache.dbConnection };
|
||||
} catch (error) {
|
||||
// If that connection attempt failed, we'll try again below
|
||||
console.error('Error waiting for existing connection:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Close existing connections if they exist
|
||||
if (connectionCache.dbConnection) {
|
||||
try {
|
||||
await connectionCache.dbConnection.end();
|
||||
} catch (error) {
|
||||
console.error('Error closing existing database connection:', error);
|
||||
}
|
||||
try { await connectionCache.dbConnection.end(); }
|
||||
catch (error) { console.error('Error closing existing database connection:', error); }
|
||||
}
|
||||
|
||||
if (connectionCache.ssh) {
|
||||
try {
|
||||
connectionCache.ssh.end();
|
||||
} catch (error) {
|
||||
console.error('Error closing existing SSH connection:', error);
|
||||
}
|
||||
try { connectionCache.ssh.end(); }
|
||||
catch (error) { console.error('Error closing existing SSH connection:', error); }
|
||||
}
|
||||
|
||||
// Mark that we're establishing a new connection
|
||||
connectionCache.isConnecting = true;
|
||||
|
||||
// Create a new promise for this connection attempt
|
||||
connectionCache.connectionPromise = setupSshTunnel().then(tunnel => {
|
||||
connectionCache.connectionPromise = setupSshTunnel().then((tunnel) => {
|
||||
const { ssh, stream, dbConfig } = tunnel;
|
||||
|
||||
return mysql.createConnection({
|
||||
...dbConfig,
|
||||
stream
|
||||
}).then(connection => {
|
||||
// Store the new connections
|
||||
return mysql.createConnection({ ...dbConfig, stream }).then((connection) => {
|
||||
connectionCache.ssh = ssh;
|
||||
connectionCache.dbConnection = connection;
|
||||
connectionCache.lastUsed = Date.now();
|
||||
connectionCache.isConnecting = false;
|
||||
|
||||
return {
|
||||
ssh,
|
||||
connection
|
||||
};
|
||||
return { ssh, connection };
|
||||
});
|
||||
}).catch(error => {
|
||||
}).catch((error) => {
|
||||
connectionCache.isConnecting = false;
|
||||
throw error;
|
||||
});
|
||||
|
||||
// Wait for the connection to be established
|
||||
return connectionCache.connectionPromise;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get cached query results or execute query if not cached
|
||||
* @param {string} cacheKey - Unique key to identify the query
|
||||
* @param {string} queryType - Type of query (field-options, product-lines, etc.)
|
||||
* @param {Function} queryFn - Function to execute if cache miss
|
||||
* @returns {Promise<any>} The query result
|
||||
*/
|
||||
async function getCachedQuery(cacheKey, queryType, queryFn) {
|
||||
// Get cache duration based on query type
|
||||
export async function getCachedQuery(cacheKey, queryType, queryFn) {
|
||||
const cacheDuration = connectionCache.cacheDuration[queryType] || connectionCache.cacheDuration.default;
|
||||
|
||||
// Check if we have a valid cached result
|
||||
const cachedResult = connectionCache.queryCache.get(cacheKey);
|
||||
const now = Date.now();
|
||||
|
||||
if (cachedResult && (now - cachedResult.timestamp < cacheDuration)) {
|
||||
console.log(`Cache hit for ${queryType} query: ${cacheKey}`);
|
||||
return cachedResult.data;
|
||||
}
|
||||
|
||||
// No valid cache found, execute the query
|
||||
console.log(`Cache miss for ${queryType} query: ${cacheKey}`);
|
||||
const result = await queryFn();
|
||||
|
||||
// Cache the result
|
||||
connectionCache.queryCache.set(cacheKey, {
|
||||
data: result,
|
||||
timestamp: now
|
||||
});
|
||||
|
||||
connectionCache.queryCache.set(cacheKey, { data: result, timestamp: now });
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Setup SSH tunnel to production database
|
||||
* @private - Should only be used by getDbConnection
|
||||
* @returns {Promise<{ssh: object, stream: object, dbConfig: object}>}
|
||||
*/
|
||||
async function setupSshTunnel() {
|
||||
const sshConfig = {
|
||||
host: process.env.PROD_SSH_HOST,
|
||||
port: process.env.PROD_SSH_PORT || 22,
|
||||
port: Number(process.env.PROD_SSH_PORT) || 22,
|
||||
username: process.env.PROD_SSH_USER,
|
||||
privateKey: process.env.PROD_SSH_KEY_PATH
|
||||
? fs.readFileSync(process.env.PROD_SSH_KEY_PATH)
|
||||
: undefined,
|
||||
compress: true
|
||||
compress: true,
|
||||
};
|
||||
|
||||
const dbConfig = {
|
||||
@@ -160,80 +96,42 @@ async function setupSshTunnel() {
|
||||
user: process.env.PROD_DB_USER,
|
||||
password: process.env.PROD_DB_PASSWORD,
|
||||
database: process.env.PROD_DB_NAME,
|
||||
port: process.env.PROD_DB_PORT || 3306,
|
||||
timezone: 'Z'
|
||||
port: Number(process.env.PROD_DB_PORT) || 3306,
|
||||
timezone: 'Z',
|
||||
};
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const ssh = new Client();
|
||||
|
||||
ssh.on('error', (err) => {
|
||||
console.error('SSH connection error:', err);
|
||||
reject(err);
|
||||
});
|
||||
|
||||
ssh.on('ready', () => {
|
||||
ssh.forwardOut(
|
||||
'127.0.0.1',
|
||||
0,
|
||||
dbConfig.host,
|
||||
dbConfig.port,
|
||||
(err, stream) => {
|
||||
if (err) reject(err);
|
||||
resolve({ ssh, stream, dbConfig });
|
||||
}
|
||||
);
|
||||
ssh.forwardOut('127.0.0.1', 0, dbConfig.host, dbConfig.port, (err, stream) => {
|
||||
if (err) reject(err);
|
||||
else resolve({ ssh, stream, dbConfig });
|
||||
});
|
||||
}).connect(sshConfig);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear cached query results
|
||||
* @param {string} [cacheKey] - Specific cache key to clear (clears all if not provided)
|
||||
*/
|
||||
function clearQueryCache(cacheKey) {
|
||||
if (cacheKey) {
|
||||
connectionCache.queryCache.delete(cacheKey);
|
||||
console.log(`Cleared cache for key: ${cacheKey}`);
|
||||
} else {
|
||||
connectionCache.queryCache.clear();
|
||||
console.log('Cleared all query cache');
|
||||
}
|
||||
export function clearQueryCache(cacheKey) {
|
||||
if (cacheKey) connectionCache.queryCache.delete(cacheKey);
|
||||
else connectionCache.queryCache.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Force close all active connections
|
||||
* Useful for server shutdown or manual connection reset
|
||||
*/
|
||||
async function closeAllConnections() {
|
||||
export async function closeAllConnections() {
|
||||
if (connectionCache.dbConnection) {
|
||||
try {
|
||||
await connectionCache.dbConnection.end();
|
||||
console.log('Closed database connection');
|
||||
} catch (error) {
|
||||
console.error('Error closing database connection:', error);
|
||||
}
|
||||
try { await connectionCache.dbConnection.end(); }
|
||||
catch (error) { console.error('Error closing database connection:', error); }
|
||||
connectionCache.dbConnection = null;
|
||||
}
|
||||
|
||||
if (connectionCache.ssh) {
|
||||
try {
|
||||
connectionCache.ssh.end();
|
||||
console.log('Closed SSH connection');
|
||||
} catch (error) {
|
||||
console.error('Error closing SSH connection:', error);
|
||||
}
|
||||
try { connectionCache.ssh.end(); }
|
||||
catch (error) { console.error('Error closing SSH connection:', error); }
|
||||
connectionCache.ssh = null;
|
||||
}
|
||||
|
||||
connectionCache.lastUsed = 0;
|
||||
connectionCache.isConnecting = false;
|
||||
connectionCache.connectionPromise = null;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
getDbConnection,
|
||||
getCachedQuery,
|
||||
clearQueryCache,
|
||||
closeAllConnections
|
||||
};
|
||||
Reference in New Issue
Block a user