Product editor tweaks

.claude/CLAUDE.md

@@ -0,0 +1,172 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+This is a full-stack inventory management system with a React + TypeScript frontend and Node.js/Express backend using PostgreSQL. The system includes product management, analytics, forecasting, purchase orders, and a comprehensive dashboard for business metrics.
+
+**Monorepo Structure:**
+- `inventory/` - Vite-based React frontend with TypeScript
+- `inventory-server/` - Express backend API server
+- Root `package.json` contains shared dependencies
+
+## Development Commands
+
+### Frontend (inventory/)
+```bash
+cd inventory
+npm run dev        # Start dev server on port 5175
+npm run build      # Build for production (outputs to build/ then copies to ../inventory-server/frontend/build)
+npm run lint       # Run ESLint
+npm run preview    # Preview production build
+```
+
+### Backend (inventory-server/)
+```bash
+cd inventory-server
+npm run dev        # Start with nodemon (auto-reload)
+npm start          # Start server (production)
+npm run prod       # Start with PM2 for production
+npm run prod:stop  # Stop PM2 instance
+npm run prod:restart  # Restart PM2 instance
+npm run prod:logs  # View PM2 logs
+npm run setup      # Create required directories (logs, uploads)
+```
+
+## Architecture
+
+### Frontend Architecture
+
+**Router Structure:** React Router with lazy loading for code splitting:
+- Main chunks: Core inventory, Dashboard, Product Import, Chat Archive
+- Authentication flow uses `RequireAuth` and `Protected` components with permission-based access
+- All routes except `/login` and `/small` require authentication
+
+**Key Directories:**
+- `src/pages/` - Top-level page components (Overview, Products, Analytics, Dashboard, etc.)
+- `src/components/` - Organized by feature (dashboard/, products/, analytics/, etc.)
+- `src/components/ui/` - shadcn/ui components
+- `src/types/` - TypeScript type definitions
+- `src/contexts/` - React contexts (AuthContext, DashboardScrollContext)
+- `src/hooks/` - Custom React hooks (use-toast, useDebounce, use-mobile)
+- `src/utils/` - Utility functions (emojiUtils, productUtils, naturalLanguagePeriod)
+- `src/services/` - API service layer
+- `src/config/` - Configuration files
+
+**State Management:**
+- React Context for auth and global state
+- @tanstack/react-query for server state management
+- zustand for client state management
+- Local storage for auth tokens, session storage for login state
+
+**Key Dependencies:**
+- UI: Radix UI primitives, shadcn/ui, Tailwind CSS, Framer Motion
+- Data: @tanstack/react-table, react-data-grid, @tanstack/react-virtual
+- Forms: react-hook-form, zod
+- Charts: recharts, chart.js, react-chartjs-2
+- File handling: xlsx for Excel export, react-dropzone for uploads
+- Other: axios for HTTP, date-fns/luxon for dates
+
+**Path Alias:** `@/` maps to `./src/`
+
+### Backend Architecture
+
+**Entry Point:** `inventory-server/src/server.js`
+
+**Key Directories:**
+- `src/routes/` - Express route handlers (products, dashboard, analytics, import, etc.)
+- `src/middleware/` - Express middleware (CORS, auth, etc.)
+- `src/utils/` - Utility functions (database connection, API helpers)
+- `src/types/` - Type definitions (e.g., status-codes)
+
+**Database:**
+- PostgreSQL with connection pooling (pg library)
+- Pool initialized in `utils/db.js` via `initPool()`
+- Pool attached to `app.locals.pool` for route access
+- Environment variables loaded from `/var/www/html/inventory/.env` (production path)
+
+**API Routes:** All prefixed with `/api/`
+- `/api/products` - Product CRUD operations
+- `/api/dashboard` - Dashboard metrics and data
+- `/api/analytics` - Analytics and reporting
+- `/api/orders` - Order management
+- `/api/purchase-orders` - Purchase order management
+- `/api/csv` - CSV import/export (data management)
+- `/api/import` - Product import workflows
+- `/api/config` - Configuration management
+- `/api/metrics` - System metrics
+- `/api/ai-validation` - AI-powered validation
+- `/api/ai-prompts` - AI prompt management
+- `/api/templates` - Template management
+- `/api/reusable-images` - Image management
+- `/api/categoriesAggregate`, `/api/vendorsAggregate`, `/api/brandsAggregate` - Aggregate data endpoints
+
+**Authentication:**
+- External auth service at `/auth-inv` endpoint
+- Token-based authentication (Bearer tokens)
+- Frontend stores tokens in localStorage
+- Protected routes verify tokens via auth service `/me` endpoint
+
+**File Uploads:**
+- Multer middleware for file handling
+- Uploads directory: `inventory-server/uploads/`
+
+### Development Proxy Setup
+
+The Vite dev server (port 5175) proxies API requests to `https://inventory.kent.pw`:
+- `/api/*` → production API
+- `/auth-inv/*` → authentication service
+- `/chat-api/*` → chat service
+- `/uploads/*` → uploaded files
+- Various third-party services (Aircall, Klaviyo, Meta, Gorgias, Typeform, ACOT, Clarity)
+
+### Build Process
+
+When building the frontend:
+1. TypeScript compilation (`tsc -b`)
+2. Vite build (outputs to `inventory/build/`)
+3. Custom Vite plugin copies build to `inventory-server/frontend/build/`
+4. Manual chunks for vendor splitting (react-vendor, ui-vendor, query-vendor)
+
+## Testing
+
+Run tests for individual components or features:
+```bash
+# No test suite currently configured
+# Tests would typically use Jest or Vitest with React Testing Library
+```
+
+## Common Development Workflows
+
+### Adding a New Page
+1. Create page component in `inventory/src/pages/YourPage.tsx`
+2. Add lazy import in `inventory/src/App.tsx`
+3. Add route with `<Protected>` wrapper and permission check
+4. Add corresponding backend route in `inventory-server/src/routes/`
+5. Update permission system if needed
+
+### Adding a New API Endpoint
+1. Create or update route file in `inventory-server/src/routes/`
+2. Use `executeQuery()` helper for database queries
+3. Register router in `inventory-server/src/server.js`
+4. Frontend can access at `/api/{route-name}`
+
+### Working with Database
+- Use parameterized queries: `executeQuery(sql, [param1, param2])`
+- Pool is accessed via `db.getPool()` or `app.locals.pool`
+- Connection helper: `db.getConnection()` returns a client for transactions
+
+### Permissions System
+- User permissions stored in `user.permissions` array (permission codes)
+- Check permissions in `<Protected page="permission_code">` component
+- Admin users (`is_admin: true`) have access to all pages
+
+## Important Notes
+
+- Environment variables must be configured in `/var/www/html/inventory/.env` for production
+- The frontend expects the backend at `/api` (proxied in dev, served together in production)
+- PM2 is used for production process management
+- Database uses PostgreSQL with SSL support (configurable via `DB_SSL` env var)
+- File uploads stored in `inventory-server/uploads/` directory
+- Build artifacts in `inventory/build/` are copied to `inventory-server/frontend/build/`

.gitignore

@@ -68,3 +68,15 @@ inventory-server/scripts/.fuse_hidden00000fa20000000a
 .VSCodeCounter/
 .VSCodeCounter/*
 .VSCodeCounter/**/*
+
+*/chat/db-convert/db/*
+*/chat/db-convert/mongo_converter_env/*
+
+# Ignore compiled Vite config to avoid duplication
+vite.config.js
+inventory-server/inventory_backup.sql
+chat-files.tar.gz
+chat-migration*/
+**/chat-migration*/
+chat-migration*/**
+**/chat-migration*/**

CLAUDE.md

@@ -0,0 +1,2 @@
+* Avoid using glob tool for search as it may not work properly on this codebase. Search using bash instead.
+* If you use the task tool to have an agent investigate something, make sure to let it know to avoid using glob

PRODUCT_IMPORT_ENHANCEMENTS.md

@@ -0,0 +1,375 @@
+# Product Import Module - Enhancement & Issues Outline
+
+This document outlines the investigation and implementation requirements for each requested enhancement to the product import module.
+
+---
+
+## 1. UPC Import - Strip Quotes and Spaces ✅ IMPLEMENTED
+
+**Issue:** When importing UPCs, strip `'`, `"` characters and any spaces, leaving only numbers.
+
+**Implementation (Completed):**
+- Modified `normalizeUpcValue()` in [Import.tsx:661-667](inventory/src/pages/Import.tsx#L661-L667)
+- Strips single quotes, double quotes, smart quotes (`'"`), and whitespace before processing
+- Then handles scientific notation and extracts only digits
+
+**Files Modified:**
+- `inventory/src/pages/Import.tsx` - `normalizeUpcValue()` function
+
+---
+
+## 2. AI Context Columns in Validation Payloads ✅ IMPLEMENTED
+
+**Issue:** The match columns step has a setting to use a field only for AI context (`isAiSupplemental`). Update AI description validation to include any columns selected with this option in the payload. Also include in sanity check payload. Not needed for names.
+
+**Current Implementation:**
+- AI Supplemental toggle: [MatchColumnsStep.tsx:102-118](inventory/src/components/product-import/steps/MatchColumnsStep/MatchColumnsStep.tsx#L102-L118)
+- AI supplemental data stored in `__aiSupplemental` field on each row
+- Description payload builder: [inlineAiPayload.ts:183-195](inventory/src/components/product-import/steps/ValidationStep/utils/inlineAiPayload.ts#L183-L195)
+
+**Implementation:**
+1. **Update `buildDescriptionValidationPayload()` in `inlineAiPayload.ts`** to include AI supplemental data:
+   ```typescript
+   export const buildDescriptionValidationPayload = (
+     row: Data<string>,
+     fieldOptions: FieldOptionsMap,
+     productLinesCache: Map<string, SelectOption[]>,
+     sublinesCache: Map<string, SelectOption[]>
+   ) => {
+     const payload: Record<string, unknown> = {
+       name: row.name,
+       description: row.description,
+       company_name: getFieldOptionLabel(row.company, fieldOptions, 'company'),
+       company_id: row.company,
+       categories: getFieldOptionLabel(row.category, fieldOptions, 'category'),
+     };
+
+     // Add AI supplemental context if present
+     if (row.__aiSupplemental && typeof row.__aiSupplemental === 'object') {
+       payload.additional_context = row.__aiSupplemental;
+     }
+
+     return payload;
+   };
+   ```
+
+2. **Update sanity check payload** - Locate sanity check submission logic and include `__aiSupplemental` data
+
+3. **Verify `__aiSupplemental` is properly populated** from MatchColumnsStep when columns are marked as AI context only
+
+**Files to Modify:**
+- `inventory/src/components/product-import/steps/ValidationStep/utils/inlineAiPayload.ts`
+- Backend sanity check endpoint (if separate from description validation)
+- Verify data flow in `MatchColumnsStep.tsx` → `ValidationStep`
+
+---
+
+## 3. Fresh Taxonomy Data Per Session ✅ IMPLEMENTED
+
+**Issue:** Ensure taxonomy data is brought in fresh with each session - cache should be invalidated if we exit the import flow and start again.
+
+**Current Implementation:**
+- Field options cached 5 minutes: [ValidationStep/index.tsx:128-133](inventory/src/components/product-import/steps/ValidationStep/index.tsx#L128-L133)
+- Product lines cache: `productLinesCache` in Zustand store
+- Sublines cache: `sublinesCache` in Zustand store
+- Caches set to 10-minute stale time
+
+**Implementation:**
+1. **Add cache invalidation on import flow mount/unmount** in `UploadFlow.tsx`:
+   ```typescript
+   useEffect(() => {
+     // On mount - invalidate import-related query cache
+     queryClient.invalidateQueries({ queryKey: ['import-field-options'] });
+
+     return () => {
+       // On unmount - clear caches
+       queryClient.removeQueries({ queryKey: ['import-field-options'] });
+       queryClient.removeQueries({ queryKey: ['product-lines'] });
+       queryClient.removeQueries({ queryKey: ['sublines'] });
+     };
+   }, []);
+   ```
+
+2. **Clear Zustand store caches** when exiting import flow:
+   - Add action to `validationStore.ts` to clear `productLinesCache` and `sublinesCache`
+   - Call this action on unmount of `UploadFlow` or when navigating away
+
+3. **Consider adding a `sessionId`** that changes on each import flow start, used as part of cache keys
+
+**Files to Modify:**
+- `inventory/src/components/product-import/steps/UploadFlow.tsx` - Add cleanup effect
+- `inventory/src/components/product-import/steps/ValidationStep/store/validationStore.ts` - Add cache clear action
+- Potentially `inventory/src/components/product-import/steps/ValidationStep/index.tsx` - Query key updates
+
+---
+
+## 4. Save Template from Confirmation Page ✅ IMPLEMENTED
+
+**Issue:** Add option to save rows of submitted data as a new template on the confirmation page after completing the import flow. Verify this works with new validation step changes.
+
+**Current Implementation:**
+- **Import Results section already exists** inline in [Import.tsx:968-1150](inventory/src/pages/Import.tsx#L968-L1150)
+  - Shows created products (lines 1021-1097) with image, name, UPC, item number
+  - Shows errored products (lines 1100-1138) with error details
+  - "Fix products with errors" button resumes validation flow for failed items
+- Template saving logic in ValidationStep: [useTemplateManagement.ts:204-266](inventory/src/components/product-import/steps/ValidationStep/hooks/useTemplateManagement.ts#L204-L266)
+- Saves via `POST /api/templates`
+- `importOutcome.submittedProducts` contains the full product data for each row
+
+**Implementation:**
+1. **Add "Save as Template" button** to each created product row in the results section (around line 1087-1092 in Import.tsx):
+   ```typescript
+   // Add button after the item number display
+   <Button
+     variant="ghost"
+     size="sm"
+     onClick={() => handleSaveAsTemplate(index)}
+   >
+     <BookmarkPlus className="h-4 w-4" />
+   </Button>
+   ```
+
+2. **Add state and dialog** for template saving in Import.tsx:
+   ```typescript
+   const [templateSaveDialogOpen, setTemplateSaveDialogOpen] = useState(false);
+   const [selectedProductForTemplate, setSelectedProductForTemplate] = useState<NormalizedProduct | null>(null);
+   ```
+
+3. **Extract/reuse template save logic** from `useTemplateManagement.ts`:
+   - The `saveNewTemplate()` function (lines 204-266) can be extracted into a shared utility
+   - Or create a `SaveTemplateDialog` component that can be used in both places
+   - Key fields needed: `company` (for template name), `product_type`, and all product field values
+
+4. **Data mapping consideration:**
+   - `importOutcome.submittedProducts` uses `NormalizedProduct` type
+   - Templates expect raw field values - may need to map back from normalized format
+   - Exclude metadata fields: `['id', '__index', '__meta', '__template', '__original', '__corrected', '__changes', '__aiSupplemental']`
+
+**Files to Modify:**
+- `inventory/src/pages/Import.tsx` - Add save template button, state, and dialog
+- Consider creating `inventory/src/components/product-import/SaveTemplateDialog.tsx` for reusability
+- Potentially extract core save logic from `useTemplateManagement.ts` into shared utility
+
+---
+
+## 5. Sheet Preview on Select Sheet Step ✅ IMPLEMENTED
+
+**Issue:** On the select sheet step, show a preview of the first 10 lines or so of each sheet underneath the options.
+
+**Implementation (Completed):**
+- Added `workbook` prop to `SelectSheetStep` component
+- Added `sheetPreviews` memoized computation using `XLSXLib.utils.sheet_to_json()`
+- Shows first 10 rows, 8 columns max per sheet
+- Added `truncateCell()` helper to limit cell content to 30 characters with ellipsis
+- Each sheet option is now a clickable card with:
+  - Radio button and sheet name
+  - Row count indicator
+  - Scrollable preview table with horizontal scroll
+  - Selected state highlighted with primary border
+- Updated `UploadFlow.tsx` to pass workbook prop
+
+**Files Modified:**
+- `inventory/src/components/product-import/steps/SelectSheetStep/SelectSheetStep.tsx`
+- `inventory/src/components/product-import/steps/UploadFlow.tsx`
+
+---
+
+## 6. Empty Row Removal ✅ IMPLEMENTED
+
+**Issue:** When importing a sheet, automatically remove completely empty rows.
+
+**Current Implementation:**
+- Empty columns are filtered: [MatchColumnsStep.tsx:616-634](inventory/src/components/product-import/steps/MatchColumnsStep/MatchColumnsStep.tsx#L616-L634)
+- A "Remove empty/duplicates" button exists that removes empty rows, single-value rows, AND duplicates
+- The automatic removal should ONLY remove completely empty rows, not duplicates or single-value rows
+
+**Implementation (Completed):**
+- Added `isRowCompletelyEmpty()` helper function to [SelectHeaderStep.tsx](inventory/src/components/product-import/steps/SelectHeaderStep/SelectHeaderStep.tsx)
+- Added `useMemo` to filter empty rows on initial data load
+- Uses `Object.values(row)` to check all cell values (matches existing button logic)
+- Only removes rows where ALL values are undefined, null, or whitespace-only strings
+- Manual "Remove Empty/Duplicates" button still available for additional cleanup (duplicates, single-value rows)
+
+**Files Modified:**
+- `inventory/src/components/product-import/steps/SelectHeaderStep/SelectHeaderStep.tsx`
+
+---
+
+## 7. Unit Conversion for Weight/Dimensions ✅ IMPLEMENTED
+
+**Issue:** Add unit conversion feature for weight and dimensions columns - similar to calculator button on cost/msrp, add button that opens popover with options to convert grams → oz, lbs → oz for the whole column at once.
+
+**Current Implementation:**
+- Calculator button on price columns: [ValidationTable.tsx:1491-1627](inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx#L1491-L1627)
+- `PriceColumnHeader` component shows calculator icon on hover
+- Weight field defined in config with validation
+
+**Implementation:**
+1. **Create `UnitConversionColumnHeader` component** (similar to `PriceColumnHeader`):
+   ```typescript
+   const UnitConversionColumnHeader = ({ field, table }) => {
+     const [showPopover, setShowPopover] = useState(false);
+
+     const conversions = {
+       weight: [
+         { label: 'Grams → Ounces', factor: 0.035274 },
+         { label: 'Pounds → Ounces', factor: 16 },
+         { label: 'Kilograms → Ounces', factor: 35.274 },
+       ],
+       dimensions: [
+         { label: 'Centimeters → Inches', factor: 0.393701 },
+         { label: 'Millimeters → Inches', factor: 0.0393701 },
+       ]
+     };
+
+     const applyConversion = (factor: number) => {
+       // Batch update all cells in column
+       table.rows.forEach((row, index) => {
+         const currentValue = parseFloat(row[field.key]);
+         if (!isNaN(currentValue)) {
+           updateCell(index, field.key, (currentValue * factor).toFixed(2));
+         }
+       });
+     };
+
+     return (
+       <Popover open={showPopover} onOpenChange={setShowPopover}>
+         <PopoverTrigger>
+           <Scale className="h-4 w-4" /> {/* or similar icon */}
+         </PopoverTrigger>
+         <PopoverContent>
+           {conversions[fieldType].map(conv => (
+             <Button key={conv.label} onClick={() => applyConversion(conv.factor)}>
+               {conv.label}
+             </Button>
+           ))}
+         </PopoverContent>
+       </Popover>
+     );
+   };
+   ```
+
+2. **Identify weight/dimension fields** in config:
+   - `weight_oz`, `length_in`, `width_in`, `height_in` (check actual field keys)
+
+3. **Add to column header render logic** in ValidationTable
+
+**Files to Modify:**
+- `inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx`
+- Potentially create new component file for `UnitConversionColumnHeader`
+- Update column header rendering to use new component for weight/dimension fields
+
+---
+
+## 8. Expanded MSRP Auto-Fill from Cost ✅ IMPLEMENTED
+
+**Issue:** Expand auto-fill functionality for MSRP from cost - open small popover with options for 2x, 2.1x, 2.2x, 2.3x, 2.4x, 2.5x multipliers, plus checkbox to round up to nearest 9.
+
+**Current Implementation:**
+- Calculator on MSRP column: [ValidationTable.tsx:1540-1584](inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx#L1540-L1584)
+- Currently only does `Cost × 2` then subtracts 0.01 if whole number
+
+**Implementation:**
+1. **Replace simple click with popover** in `PriceColumnHeader`:
+   ```typescript
+   const [selectedMultiplier, setSelectedMultiplier] = useState(2.0);
+   const [roundToNine, setRoundToNine] = useState(false);
+   const multipliers = [2.0, 2.1, 2.2, 2.3, 2.4, 2.5];
+
+   const roundUpToNine = (value: number): number => {
+     // 1.41 → 1.49, 2.78 → 2.79, 12.32 → 12.39
+     const wholePart = Math.floor(value);
+     const decimal = value - wholePart;
+     if (decimal <= 0.09) return wholePart + 0.09;
+     if (decimal <= 0.19) return wholePart + 0.19;
+     // ... continue pattern, or:
+     const lastDigit = Math.floor(decimal * 10);
+     return wholePart + (lastDigit / 10) + 0.09;
+   };
+
+   const calculateMsrp = (cost: number): number => {
+     let result = cost * selectedMultiplier;
+     if (roundToNine) {
+       result = roundUpToNine(result);
+     }
+     return result;
+   };
+   ```
+
+2. **Create popover UI**:
+   ```tsx
+   <Popover>
+     <PopoverTrigger><Calculator className="h-4 w-4" /></PopoverTrigger>
+     <PopoverContent className="w-48">
+       <div className="space-y-2">
+         <Label>Multiplier</Label>
+         <div className="grid grid-cols-3 gap-1">
+           {multipliers.map(m => (
+             <Button
+               key={m}
+               variant={selectedMultiplier === m ? 'default' : 'outline'}
+               size="sm"
+               onClick={() => setSelectedMultiplier(m)}
+             >
+               {m}x
+             </Button>
+           ))}
+         </div>
+         <div className="flex items-center gap-2">
+           <Checkbox checked={roundToNine} onCheckedChange={setRoundToNine} />
+           <Label>Round to .X9</Label>
+         </div>
+         <Button onClick={applyCalculation} className="w-full">
+           Apply
+         </Button>
+       </div>
+     </PopoverContent>
+   </Popover>
+   ```
+
+**Files to Modify:**
+- `inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx` - `PriceColumnHeader` component
+
+---
+
+## 9. Debug Mode - Skip API Submission ✅ IMPLEMENTED
+
+**Issue:** Add a third switch in the footer of image upload step (visible only to users with `admin:debug` permission) that will not submit data to any API, only complete the process and show results page as if it had worked.
+
+**Implementation (Completed):**
+- Added `skipApiSubmission` state to `ImageUploadStep.tsx`
+- Added amber-colored "Skip API (Debug)" switch (visible only with `admin:debug` permission)
+- When skip is active, "Use Test API" and "Use Test Database" switches are hidden
+- Added `skipApiSubmission?: boolean` to `SubmitOptions` type in `types.ts`
+- In `Import.tsx`, when `skipApiSubmission` is true:
+  - Skips the actual API call entirely
+  - Generates mock success response with mock PIDs
+  - Shows `[DEBUG]` prefix in toast and result message
+  - Displays results page as if submission succeeded
+
+**Files Modified:**
+- `inventory/src/components/product-import/types.ts` - Added `skipApiSubmission` to `SubmitOptions`
+- `inventory/src/components/product-import/steps/ImageUploadStep/ImageUploadStep.tsx` - Added switch UI
+- `inventory/src/pages/Import.tsx` - Added skip logic in `handleData()`
+
+---
+
+## Summary
+
+| # | Enhancement | Complexity | Status |
+|---|-------------|------------|--------|
+| 1 | Strip UPC quotes/spaces | Low | ✅ Implemented |
+| 2 | AI context in validation | Medium | ✅ Implemented |
+| 3 | Fresh taxonomy per session | Medium | ✅ Implemented |
+| 4 | Save template from confirmation | Medium-High | ✅ Implemented |
+| 5 | Sheet preview | Low-Medium | ✅ Implemented |
+| 6 | Remove empty rows | Low | ✅ Implemented |
+| 7 | Unit conversion | Medium | ✅ Implemented |
+| 8 | MSRP multiplier options | Medium | ✅ Implemented |
+| 9 | Debug skip API | Low-Medium | ✅ Implemented |
+
+**Implemented:** 9 of 9 items - All enhancements complete!
+
+---
+
+*Document generated: 2026-01-25*

docs/PERMISSIONS.md

@@ -7,12 +7,13 @@ This document outlines the permission system implemented in the Inventory Manage
 Permissions follow this naming convention:
 
 - Page access: `access:{page_name}`
-- Actions: `{action}:{resource}`
+- Settings sections: `settings:{section_name}`
+- Admin features: `admin:{feature}`
 
 Examples:
 - `access:products` - Can access the Products page
-- `create:products` - Can create new products
-- `edit:users` - Can edit user accounts
+- `settings:user_management` - Can access User Management settings
+- `admin:debug` - Can see debug information
 
 ## Permission Components
 
@@ -22,10 +23,10 @@ The core component that conditionally renders content based on permissions.
 
 ```tsx
 <PermissionGuard 
-  permission="create:products"
+  permission="settings:user_management"
   fallback={<p>No permission</p>}
 >
-  <button>Create Product</button>
+  <button>Manage Users</button>
 </PermissionGuard>
 ```
 
@@ -81,7 +82,7 @@ Specific component for settings with built-in permission checks.
 <SettingsSection
   title="System Settings"
   description="Configure global settings"
-  permission="edit:system_settings"
+  permission="settings:global"
 >
   {/* Settings content */}
 </SettingsSection>
@@ -95,8 +96,8 @@ Core hook for checking any permission.
 
 ```tsx
 const { hasPermission, hasPageAccess, isAdmin } = usePermissions();
-if (hasPermission('delete:products')) {
-  // Can delete products
+if (hasPermission('settings:user_management')) {
+  // Can access user management
 }
 ```
 
@@ -106,8 +107,8 @@ Specialized hook for page-level permissions.
 
 ```tsx
 const { canView, canCreate, canEdit, canDelete } = usePagePermission('products');
-if (canEdit()) {
-  // Can edit products
+if (canView()) {
+  // Can view products
 }
 ```
 
@@ -119,18 +120,43 @@ Permissions are stored in the database:
 
 Admin users automatically have all permissions.
 
-## Common Permission Codes
+## Implemented Permission Codes
 
+### Page Access Permissions
 | Code | Description |
 |------|-------------|
 | `access:dashboard` | Access to Dashboard page |
+| `access:overview` | Access to Overview page |
 | `access:products` | Access to Products page |
-| `create:products` | Create new products |
-| `edit:products` | Edit existing products |
-| `delete:products` | Delete products |
-| `view:users` | View user accounts |
-| `edit:users` | Edit user accounts |
-| `manage:permissions` | Assign permissions to users |
+| `access:categories` | Access to Categories page |
+| `access:brands` | Access to Brands page |
+| `access:vendors` | Access to Vendors page |
+| `access:purchase_orders` | Access to Purchase Orders page |
+| `access:analytics` | Access to Analytics page |
+| `access:forecasting` | Access to Forecasting page |
+| `access:import` | Access to Import page |
+| `access:settings` | Access to Settings page |
+| `access:chat` | Access to Chat Archive page |
+
+### Settings Permissions
+| Code | Description |
+|------|-------------|
+| `settings:global` | Access to Global Settings section |
+| `settings:products` | Access to Product Settings section |
+| `settings:vendors` | Access to Vendor Settings section |
+| `settings:data_management` | Access to Data Management settings |
+| `settings:calculation_settings` | Access to Calculation Settings |
+| `settings:library_management` | Access to Image Library Management |
+| `settings:performance_metrics` | Access to Performance Metrics |
+| `settings:prompt_management` | Access to AI Prompt Management |
+| `settings:stock_management` | Access to Stock Management |
+| `settings:templates` | Access to Template Management |
+| `settings:user_management` | Access to User Management |
+
+### Admin Permissions
+| Code | Description |
+|------|-------------|
+| `admin:debug` | Can see debug information and features |
 
 ## Implementation Examples
 
@@ -148,25 +174,31 @@ In `App.tsx`:
 ### Component Level Protection
 
 ```tsx
-const { canEdit } = usePagePermission('products');
+const { hasPermission } = usePermissions();
 
-function handleEdit() {
-  if (!canEdit()) {
+function handleAction() {
+  if (!hasPermission('settings:user_management')) {
     toast.error("You don't have permission");
     return;
   }
-  // Edit logic
+  // Action logic
 }
 ```
 
 ### UI Element Protection
 
 ```tsx
-<PermissionButton 
-  page="products" 
-  action="delete"
-  onClick={handleDelete}
->
-  Delete
-</PermissionButton>
+<PermissionGuard permission="settings:user_management">
+  <button onClick={handleManageUsers}>
+    Manage Users
+  </button>
+</PermissionGuard>
 ```
+
+## Notes
+
+- **Page Access**: These permissions control which pages a user can navigate to
+- **Settings Access**: These permissions control access to different sections within the Settings page
+- **Admin Features**: Special permissions for administrative functions
+- **CRUD Operations**: The application currently focuses on viewing and managing data rather than creating/editing/deleting individual records
+- **User Management**: User CRUD operations are handled through the settings interface rather than dedicated user management pages 

docs/import-from-prod-data-mapping.md

@@ -0,0 +1,342 @@
+# MySQL to PostgreSQL Import Process Documentation
+
+This document outlines the data import process from the production MySQL database to the local PostgreSQL database, focusing on column mappings, data transformations, and the overall import architecture.
+
+## Table of Contents
+1. [Overview](#overview)
+2. [Import Architecture](#import-architecture)
+3. [Column Mappings](#column-mappings)
+    - [Categories](#categories)
+    - [Products](#products)
+    - [Product Categories (Relationship)](#product-categories-relationship)
+    - [Orders](#orders)
+    - [Purchase Orders](#purchase-orders)
+    - [Metadata Tables](#metadata-tables)
+4. [Special Calculations](#special-calculations)
+5. [Implementation Notes](#implementation-notes)
+
+## Overview
+
+The import process extracts data from a MySQL 5.7 production database and imports it into a PostgreSQL database. It can operate in two modes:
+
+- **Full Import**: Imports all data regardless of last sync time
+- **Incremental Import**: Only imports data that has changed since the last import
+
+The process handles four main data types:
+- Categories (product categorization hierarchy)
+- Products (inventory items)
+- Orders (sales records)
+- Purchase Orders (vendor orders)
+
+## Import Architecture
+
+The import process follows these steps:
+
+1. **Establish Connection**: Creates a SSH tunnel to the production server and establishes database connections
+2. **Setup Import History**: Creates a record of the current import operation
+3. **Import Categories**: Processes product categories in hierarchical order
+4. **Import Products**: Processes products with their attributes and category relationships
+5. **Import Orders**: Processes customer orders with line items, taxes, and discounts
+6. **Import Purchase Orders**: Processes vendor purchase orders with line items
+7. **Record Results**: Updates the import history with results
+8. **Close Connections**: Cleans up connections and resources
+
+Each import step uses temporary tables for processing and wraps operations in transactions to ensure data consistency.
+
+## Column Mappings
+
+### Categories
+| PostgreSQL Column | MySQL Source                    | Transformation                               |
+|-------------------|---------------------------------|----------------------------------------------|
+| cat_id            | product_categories.cat_id       | Direct mapping                               |
+| name              | product_categories.name         | Direct mapping                               |
+| type              | product_categories.type         | Direct mapping                               |
+| parent_id         | product_categories.master_cat_id| NULL for top-level categories (types 10, 20) |
+| description       | product_categories.combined_name| Direct mapping                               |
+| status            | N/A                             | Hard-coded 'active'                          |
+| created_at        | N/A                             | Current timestamp                            |
+| updated_at        | N/A                             | Current timestamp                            |
+
+**Notes:**
+- Categories are processed in hierarchical order by type: [10, 20, 11, 21, 12, 13]
+- Type 10/20 are top-level categories with no parent
+- Types 11/21/12/13 are child categories that reference parent categories
+
+### Products
+| PostgreSQL Column    | MySQL Source                     | Transformation                                                |
+|----------------------|----------------------------------|---------------------------------------------------------------|
+| pid                  | products.pid                     | Direct mapping                                                |
+| title                | products.description             | Direct mapping                                                |
+| description          | products.notes                   | Direct mapping                                                |
+| sku                  | products.itemnumber              | Fallback to 'NO-SKU' if empty                                 |
+| stock_quantity       | shop_inventory.available_local   | Capped at 5000, minimum 0                                     |
+| preorder_count       | current_inventory.onpreorder     | Default 0                                                     |
+| notions_inv_count    | product_notions_b2b.inventory    | Default 0                                                     |
+| price                | product_current_prices.price_each| Default 0, filtered on active=1                              |
+| regular_price        | products.sellingprice            | Default 0                                                     |
+| cost_price           | product_inventory                | Weighted average: SUM(costeach * count) / SUM(count) when count > 0, or latest costeach |
+| vendor               | suppliers.companyname            | Via supplier_item_data.supplier_id                            |
+| vendor_reference     | supplier_item_data               | supplier_itemnumber or notions_itemnumber based on vendor     |
+| notions_reference    | supplier_item_data.notions_itemnumber | Direct mapping                                           |
+| brand                | product_categories.name          | Linked via products.company                                   |
+| line                 | product_categories.name          | Linked via products.line                                      |
+| subline              | product_categories.name          | Linked via products.subline                                   |
+| artist               | product_categories.name          | Linked via products.artist                                    |
+| categories           | product_category_index           | Comma-separated list of category IDs                          |
+| created_at           | products.date_created            | Validated date, NULL if invalid                               |
+| first_received       | products.datein                  | Validated date, NULL if invalid                               |
+| landing_cost_price   | NULL                             | Not set                                                       |
+| barcode              | products.upc                     | Direct mapping                                                |
+| harmonized_tariff_code| products.harmonized_tariff_code | Direct mapping                                                |
+| updated_at           | products.stamp                   | Validated date, NULL if invalid                               |
+| visible              | shop_inventory                   | Calculated from show + buyable > 0                            |
+| managing_stock       | N/A                              | Hard-coded true                                               |
+| replenishable        | Multiple fields                  | Complex calculation based on reorder, dates, etc.             |
+| permalink            | N/A                              | Constructed URL with product ID                               |
+| moq                  | supplier_item_data               | notions_qty_per_unit or supplier_qty_per_unit, minimum 1      |
+| uom                  | N/A                              | Hard-coded 1                                                  |
+| rating               | products.rating                  | Direct mapping                                                |
+| reviews              | products.rating_votes            | Direct mapping                                                |
+| weight               | products.weight                  | Direct mapping                                                |
+| length               | products.length                  | Direct mapping                                                |
+| width                | products.width                   | Direct mapping                                                |
+| height               | products.height                  | Direct mapping                                                |
+| country_of_origin    | products.country_of_origin       | Direct mapping                                                |
+| location             | products.location                | Direct mapping                                                |
+| total_sold           | order_items                      | SUM(qty_ordered) for all order_items where prod_pid = pid     |
+| baskets              | mybasket                         | COUNT of records where mb.item = pid and qty > 0              |
+| notifies             | product_notify                   | COUNT of records where pn.pid = pid                           |
+| date_last_sold       | product_last_sold.date_sold      | Validated date, NULL if invalid                               |
+| image                | N/A                              | Constructed from pid and image URL pattern                    |
+| image_175            | N/A                              | Constructed from pid and image URL pattern                    |
+| image_full           | N/A                              | Constructed from pid and image URL pattern                    |
+| options              | NULL                             | Not set                                                       |
+| tags                 | NULL                             | Not set                                                       |
+
+**Notes:**
+- Replenishable calculation: 
+  ```javascript
+  CASE 
+    WHEN p.reorder < 0 THEN 0
+    WHEN (
+      (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
+      AND (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+      AND (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+    ) THEN 0
+    ELSE 1
+  END
+  ```
+
+In business terms, a product is considered NOT replenishable only if:
+- It was manually flagged as not replenishable (negative reorder value)
+- OR it shows no activity across ALL metrics (no sales AND no receipts AND no refills in the past 5 years)
+- Image URLs are constructed using this pattern:
+  ```javascript
+  const paddedPid = pid.toString().padStart(6, '0');
+  const prefix = paddedPid.slice(0, 3);
+  const basePath = `${imageUrlBase}${prefix}/${pid}`;
+  return {
+    image: `${basePath}-t-${iid}.jpg`,
+    image_175: `${basePath}-175x175-${iid}.jpg`,
+    image_full: `${basePath}-o-${iid}.jpg`
+  };
+  ```
+
+### Product Categories (Relationship)
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| pid               | products.pid                      | Direct mapping                                                |
+| cat_id            | product_category_index.cat_id     | Direct mapping, filtered by category types                    |
+
+**Notes:**
+- Only categories of types 10, 20, 11, 21, 12, 13 are imported
+- Categories 16 and 17 are explicitly excluded
+
+### Orders
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| order_number      | order_items.order_id              | Direct mapping                                                |
+| pid               | order_items.prod_pid              | Direct mapping                                                |
+| sku               | order_items.prod_itemnumber       | Fallback to 'NO-SKU' if empty                                 |
+| date              | _order.date_placed_onlydate       | Via join to _order table                                      |
+| price             | order_items.prod_price            | Direct mapping                                                |
+| quantity          | order_items.qty_ordered           | Direct mapping                                                |
+| discount          | Multiple sources                  | Complex calculation (see notes)                               |
+| tax               | order_tax_info_products.item_taxes_to_collect | Via latest order_tax_info record                  |
+| tax_included      | N/A                               | Hard-coded false                                              |
+| shipping          | N/A                               | Hard-coded 0                                                  |
+| customer          | _order.order_cid                  | Direct mapping                                                |
+| customer_name     | users                             | CONCAT(users.firstname, ' ', users.lastname)                  |
+| status            | _order.order_status               | Direct mapping                                                |
+| canceled          | _order.date_cancelled             | Boolean: true if date_cancelled is not '0000-00-00 00:00:00'  |
+| costeach          | order_costs                       | From latest record or fallback to price * 0.5                 |
+
+**Notes:**
+- Only orders with order_status >= 15 and with a valid date_placed are processed
+- For incremental imports, only orders modified since last sync are processed
+- Discount calculation combines three sources:
+  1. Base discount: order_items.prod_price_reg - order_items.prod_price
+  2. Promo discount: SUM of order_discount_items.amount
+  3. Proportional order discount: Calculation based on order subtotal proportion
+  ```javascript
+  (oi.base_discount + 
+   COALESCE(ot.promo_discount, 0) + 
+   CASE 
+    WHEN om.summary_discount > 0 AND om.summary_subtotal > 0 THEN 
+      ROUND((om.summary_discount * (oi.price * oi.quantity)) / NULLIF(om.summary_subtotal, 0), 2)
+    ELSE 0 
+   END)::DECIMAL(10,2)
+  ```
+- Taxes are taken from the latest tax record for an order
+- Cost data is taken from the latest non-pending cost record
+
+### Purchase Orders
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| po_id             | po.po_id                          | Default 0 if NULL                                             |
+| pid               | po_products.pid                   | Direct mapping                                                |
+| sku               | products.itemnumber               | Fallback to 'NO-SKU' if empty                                 |
+| name              | products.description              | Fallback to 'Unknown Product'                                 |
+| cost_price        | po_products.cost_each             | Direct mapping                                                |
+| po_cost_price     | po_products.cost_each             | Duplicate of cost_price                                       |
+| vendor            | suppliers.companyname             | Fallback to 'Unknown Vendor' if empty                         |
+| date              | po.date_ordered                   | Fallback to po.date_created if NULL                           |
+| expected_date     | po.date_estin                     | Direct mapping                                                |
+| status            | po.status                         | Default 1 if NULL                                             |
+| notes             | po.short_note                     | Fallback to po.notes if NULL                                  |
+| ordered           | po_products.qty_each              | Direct mapping                                                |
+| received          | N/A                               | Hard-coded 0                                                  |
+| receiving_status  | N/A                               | Hard-coded 1                                                  |
+
+**Notes:**
+- Only POs created within last 1 year (incremental) or 5 years (full) are processed
+- For incremental imports, only POs modified since last sync are processed
+
+### Metadata Tables
+
+#### import_history
+
+| PostgreSQL Column | Source                            | Notes                                                         |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| id                | Auto-increment                    | Primary key                                                   |
+| table_name        | Code                              | 'all_tables' for overall import                               |
+| start_time        | NOW()                             | Import start time                                             |
+| end_time          | NOW()                             | Import completion time                                        |
+| duration_seconds  | Calculation                       | Elapsed seconds                                               |
+| is_incremental    | INCREMENTAL_UPDATE                | Flag from config                                              |
+| records_added     | Calculation                       | Sum from all imports                                          |
+| records_updated   | Calculation                       | Sum from all imports                                          |
+| status            | Code                              | 'running', 'completed', 'failed', or 'cancelled'              |
+| error_message     | Exception                         | Error message if failed                                       |
+| additional_info   | JSON                              | Configuration and results                                     |
+
+#### sync_status
+
+| PostgreSQL Column    | Source                         | Notes                                                         |
+|----------------------|--------------------------------|---------------------------------------------------------------|
+| table_name           | Code                           | Name of imported table                                        |
+| last_sync_timestamp  | NOW()                          | Timestamp of successful sync                                  |
+| last_sync_id         | NULL                           | Not used currently                                            |
+
+## Special Calculations
+
+### Date Validation
+
+MySQL dates are validated before insertion into PostgreSQL:
+
+```javascript
+function validateDate(mysqlDate) {
+  if (!mysqlDate || mysqlDate === '0000-00-00' || mysqlDate === '0000-00-00 00:00:00') {
+    return null;
+  }
+  // Check if the date is valid
+  const date = new Date(mysqlDate);
+  return isNaN(date.getTime()) ? null : mysqlDate;
+}
+```
+
+### Retry Mechanism
+
+Operations that might fail temporarily are retried with exponential backoff:
+
+```javascript
+async function withRetry(operation, errorMessage) {
+  let lastError;
+  for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error;
+      console.error(`${errorMessage} (Attempt ${attempt}/${MAX_RETRIES}):`, error);
+      if (attempt < MAX_RETRIES) {
+        const backoffTime = RETRY_DELAY * Math.pow(2, attempt - 1);
+        await new Promise(resolve => setTimeout(resolve, backoffTime));
+      }
+    }
+  }
+  throw lastError;
+}
+```
+
+### Progress Tracking
+
+Progress is tracked with estimated time remaining:
+
+```javascript
+function estimateRemaining(startTime, current, total) {
+  if (current === 0) return "Calculating...";
+  const elapsedSeconds = (Date.now() - startTime) / 1000;
+  const itemsPerSecond = current / elapsedSeconds;
+  const remainingItems = total - current;
+  const remainingSeconds = remainingItems / itemsPerSecond;
+  return formatElapsedTime(remainingSeconds);
+}
+```
+
+## Implementation Notes
+
+### Transaction Management
+
+All imports use transactions to ensure data consistency:
+
+- **Categories**: Uses savepoints for each category type
+- **Products**: Uses a single transaction for the entire import
+- **Orders**: Uses a single transaction with temporary tables
+- **Purchase Orders**: Uses a single transaction with temporary tables
+
+### Memory Usage Optimization
+
+To minimize memory usage when processing large datasets:
+
+1. Data is processed in batches (100-5000 records per batch)
+2. Temporary tables are used for intermediate data
+3. Some queries use cursors to avoid loading all results at once
+
+### MySQL vs PostgreSQL Compatibility
+
+The scripts handle differences between MySQL and PostgreSQL:
+
+1. MySQL-specific syntax like `USE INDEX` is removed for PostgreSQL
+2. `GROUP_CONCAT` in MySQL becomes string operations in PostgreSQL
+3. Transaction syntax differences are abstracted in the connection wrapper
+4. PostgreSQL's `ON CONFLICT` replaces MySQL's `ON DUPLICATE KEY UPDATE`
+
+### SSH Tunnel
+
+Database connections go through an SSH tunnel for security:
+
+```javascript
+ssh.forwardOut(
+  "127.0.0.1",
+  0,
+  sshConfig.prodDbConfig.host,
+  sshConfig.prodDbConfig.port,
+  async (err, stream) => {
+    if (err) reject(err);
+    resolve({ ssh, stream });
+  }
+);
+``` 

docs/routes-cleanup.md

@@ -0,0 +1,271 @@
+**Analysis of Potential Issues**
+
+1.  **Obsolete Functionality:**
+    *   **`config.js` Legacy Endpoints:** The endpoints `GET /config/`, `PUT /config/stock-thresholds/:id`, `PUT /config/lead-time-thresholds/:id`, `PUT /config/sales-velocity/:id`, `PUT /config/abc-classification/:id`, `PUT /config/safety-stock/:id`, and `PUT /config/turnover/:id` appear **highly likely to be obsolete**. They reference older, single-row config tables (`stock_thresholds`, etc.) while newer endpoints (`/config/global`, `/config/products`, `/config/vendors`) manage settings in more structured tables (`settings_global`, `settings_product`, `settings_vendor`). Unless specifically required for backward compatibility, these legacy endpoints should be removed to avoid confusion and potential data conflicts.
+    *   **`analytics.js` Forecast Endpoint (`GET /analytics/forecast`):** This endpoint uses **MySQL syntax** (`DATEDIFF`, `DATE_FORMAT`, `JSON_OBJECT`, `?` placeholders) but seems intended to run within the analytics module which otherwise uses PostgreSQL (`req.app.locals.pool`, `date_trunc`, `::text`, `$1` placeholders). This endpoint is likely **obsolete or misplaced** and will not function correctly against the PostgreSQL database.
+    *   **`csv.js` Redundant Actions:**
+        *   `POST /csv/update` seems redundant with `POST /csv/full-update`. The latter uses the `runScript` helper and dedicated state (`activeFullUpdate`), appearing more robust. `/csv/update` might be older or incomplete.
+        *   `POST /csv/reset` seems redundant with `POST /csv/full-reset`. Similar reasoning applies; `/csv/full-reset` appears preferred.
+    *   **`products.js` Import Endpoint (`POST /products/import`):** This is **dangerous duplication**. The `/csv` module handles imports (`/csv/import`, `/csv/import-from-prod`) with locking (`activeImport`) to prevent concurrent operations. This endpoint lacks such locking and could corrupt data if run simultaneously with other CSV/reset operations. It should likely be removed.
+    *   **`products.js` Metrics Endpoint (`GET /products/:id/metrics`):** This is redundant. The `/metrics/:pid` endpoint provides the same, possibly more comprehensive, data directly from the `product_metrics` table. Clients should use `/metrics/:pid` instead.
+
+2.  **Overlap or Inappropriate Duplication of Effort:**
+    *   **AI Prompt Getters:** `GET /ai-prompts/type/general` and `GET /ai-prompts/type/system` could potentially be handled by adding a query parameter filter to `GET /ai-prompts/` (e.g., `GET /ai-prompts?prompt_type=general`). However, dedicated endpoints for single, specific items can sometimes be simpler. This is more of a design choice than a major issue.
+    *   **Vendor Performance/Metrics:** There are multiple ways to get vendor performance data:
+        *   `GET /analytics/vendors` (uses `vendor_metrics`)
+        *   `GET /dashboard/vendor/performance` (uses `purchase_orders`)
+        *   `GET /purchase-orders/vendor-metrics` (uses `purchase_orders`)
+        *   `GET /vendors-aggregate/` (uses `vendor_metrics`, augmented with `purchase_orders`)
+        This suggests significant overlap. The `/vendors-aggregate` endpoint seems the most comprehensive, combining pre-aggregated data with some real-time info. The others, especially `/dashboard/vendor/performance` and `/purchase-orders/vendor-metrics` which calculate directly from `purchase_orders`, might be redundant or less performant.
+    *   **Product Listing:**
+        *   `GET /products/` lists products joining `products`, `product_metrics`, and `categories`.
+        *   `GET /metrics/` lists products primarily from `product_metrics`.
+        They offer similar filtering/sorting. If `product_metrics` contains all necessary display fields, `GET /products/` might be partly redundant for simple listing views, although it does provide aggregated category names. Evaluate if both full list endpoints are necessary.
+    *   **Image Uploads/Management:** Image handling is split:
+        *   `products-import.js`: Uploads temporary images for product import to `/uploads/products/`, schedules deletion.
+        *   `reusable-images.js`: Uploads persistent images to `/uploads/reusable/`, stores metadata in DB.
+        *   `products-import.js` has `/check-file` and `/list-uploads` that can see *both* directories, while `reusable-images.js` has a `/check-file` that only sees its own. This separation could be confusing. Clarify the purpose and lifecycle of images in each directory.
+    *   **Background Task Management (`csv.js`):** The use of `activeImport` for multiple unrelated tasks (import, reset, metrics calc) prevents concurrency, which might be too restrictive. The cancellation logic (`/cancel`) only targets `full-update`/`full-reset`, not tasks locked by `activeImport`. This needs unification.
+    *   **Analytics/Dashboard Base Table Queries:** Several endpoints in `analytics.js` (`/pricing`, `/categories`) and `dashboard.js` (`/best-sellers`, `/sales/metrics`, `/trending/products`, `/key-metrics`, `/inventory-health`, `/sales-overview`) query base tables (`orders`, `products`, `purchase_orders`) directly, while many others leverage pre-aggregated `_metrics` tables. This inconsistency can lead to performance differences and suggests potential for optimization by using aggregates where possible.
+
+3.  **Obvious Mistakes / Data Issues:**
+    *   **AI Prompt Fetching:** `GET /ai-prompts/company/:companyId`, `/type/general`, `/type/system` return `result.rows[0]`. This assumes uniqueness. If the underlying DB constraints (`unique_company_prompt`, etc.) fail or aren't present, this could silently hide data if multiple rows match. The use of unique constraint handling in POST/PUT suggests this is likely intended and safe *if* DB constraints are solid.
+    *   **Mixed Databases & SSH Tunnels:** The heavy reliance in `ai_validation.js` and `products-import.js` on connecting to a production MySQL DB via SSH tunnel while also using a local PostgreSQL DB adds significant architectural complexity.
+        *   **Inefficiency:** In `ai_validation.js` (`generateDebugResponse`), an SSH tunnel and MySQL connection (`promptTunnel`, `promptConnection`) are established but seem unused when fetching prompts (which correctly come from the PG pool `res.app.locals.pool`). This is wasted effort.
+        *   **Improvement:** The `getDbConnection` function in `products-import.js` implements caching/pooling for the SSH/MySQL connection – this is much better and should ideally be used consistently wherever the production DB is accessed (e.g., in `ai_validation.js`).
+    *   **`products.js` Brand Filtering:** `GET /products/brands` filters brands based on having associated purchase orders with a cost >= 500. This seems arbitrary for a general list of brands and might return incomplete results depending on the use case.
+    *   **Type Handling:** Ensure `parseValue` handles all required types and edge cases correctly, especially for filtering complex queries in `*-aggregate` and `metrics` routes. Explicit type casting in SQL (`::numeric`, `::text`, etc.) is generally good practice in PostgreSQL.
+    *   **Dummy Data:** Several `dashboard.js` endpoints return hardcoded dummy data on errors or when no data is found. While this prevents UI crashes, it can mask real issues. Ensure logging is robust when fallbacks are used.
+
+**Summary of Endpoints**
+
+Here's a summary of the available endpoints, grouped by their likely file/module:
+
+**1. AI Prompts (`ai_prompts.js`)**
+    *   `GET /`: Get all AI prompts.
+    *   `GET /:id`: Get a specific AI prompt by its ID.
+    *   `GET /company/:companyId`: Get the AI prompt for a specific company (expects one). **(Deprecated)**
+    *   `GET /type/general`: Get the general AI prompt (expects one). **(Deprecated)**
+    *   `GET /type/system`: Get the system AI prompt (expects one). **(Deprecated)**
+    *   `GET /by-type`: Get AI prompt by type (general, system, company_specific) with optional company parameter. **(New Consolidated Endpoint)**
+    *   `POST /`: Create a new AI prompt.
+    *   `PUT /:id`: Update an existing AI prompt.
+    *   `DELETE /:id`: Delete an AI prompt.
+
+**2. AI Validation (`ai_validation.js`)**
+    *   `POST /debug`: Generate and view the structure of prompts and taxonomy data (for debugging, doesn't call OpenAI). Connects to Prod MySQL (taxonomy) and Local PG (prompts, performance).
+    *   `POST /validate`: Validate product data using OpenAI. Connects to Prod MySQL (taxonomy) and Local PG (prompts, performance).
+    *   `GET /test-taxonomy`: Test endpoint to query sample taxonomy data from Prod MySQL.
+
+**3. Analytics (`analytics.js`)**
+    *   `GET /stats`: Get overall business statistics from metrics tables.
+    *   `GET /profit`: Get profit analysis data (by category, over time, top products) from metrics tables.
+    *   `GET /vendors`: Get vendor performance analysis from `vendor_metrics`.
+    *   `GET /stock`: Get stock analysis data (turnover, levels, critical items) from metrics tables.
+    *   `GET /pricing`: Get pricing analysis (price points, elasticity, recommendations) - **uses `orders` table**.
+    *   `GET /categories`: Get category performance analysis (revenue, profit, growth, distribution, trends) - **uses `orders` and `products` tables**.
+    *   `GET /forecast`: (**Likely Obsolete/Broken**) Attempts to get forecast data using MySQL syntax.
+
+**4. Brands Aggregate (`brands-aggregate.js`)**
+    *   `GET /filter-options`: Get distinct brand names and statuses for UI filters (from `brand_metrics`).
+    *   `GET /stats`: Get overall statistics related to brands (from `brand_metrics`).
+    *   `GET /`: List brands with aggregated metrics, supporting filtering, sorting, pagination (from `brand_metrics`).
+
+**5. Categories Aggregate (`categories-aggregate.js`)**
+    *   `GET /filter-options`: Get distinct category types, statuses, and counts for UI filters (from `category_metrics` & `categories`).
+    *   `GET /stats`: Get overall statistics related to categories (from `category_metrics` & `categories`).
+    *   `GET /`: List categories with aggregated metrics, supporting filtering, sorting (incl. hierarchy), pagination (from `category_metrics` & `categories`).
+
+**6. Configuration (`config.js`)**
+    *   **(New)** `GET /global`: Get all global settings.
+    *   **(New)** `PUT /global`: Update global settings.
+    *   **(New)** `GET /products`: List product-specific settings with pagination/search.
+    *   **(New)** `PUT /products/:pid`: Update/Create product-specific settings.
+    *   **(New)** `POST /products/:pid/reset`: Reset product settings to defaults.
+    *   **(New)** `GET /vendors`: List vendor-specific settings with pagination/search.
+    *   **(New)** `PUT /vendors/:vendor`: Update/Create vendor-specific settings.
+    *   **(New)** `POST /vendors/:vendor/reset`: Reset vendor settings to defaults.
+    *   **(Legacy/Obsolete)** `GET /`: Get all config from old single-row tables.
+    *   **(Legacy/Obsolete)** `PUT /stock-thresholds/:id`: Update old stock thresholds.
+    *   **(Legacy/Obsolete)** `PUT /lead-time-thresholds/:id`: Update old lead time thresholds.
+    *   **(Legacy/Obsolete)** `PUT /sales-velocity/:id`: Update old sales velocity config.
+    *   **(Legacy/Obsolete)** `PUT /abc-classification/:id`: Update old ABC config.
+    *   **(Legacy/Obsolete)** `PUT /safety-stock/:id`: Update old safety stock config.
+    *   **(Legacy/Obsolete)** `PUT /turnover/:id`: Update old turnover config.
+
+**7. CSV Operations & Background Tasks (`csv.js`)**
+    *   `GET /:type/progress`: SSE endpoint for full update/reset progress.
+    *   `GET /test`: Simple test endpoint.
+    *   `GET /status`: Check status of the generic background task lock (`activeImport`).
+    *   `GET /calculate-metrics/status`: Check status of metrics calculation.
+    *   `GET /history/import`: Get recent import history.
+    *   `GET /history/calculate`: Get recent metrics calculation history.
+    *   `GET /status/modules`: Get last calculation time per module.
+    *   `GET /status/tables`: Get last sync time per table.
+    *   `GET /status/table-counts`: Get record counts for key tables.
+    *   `POST /update`: (**Potentially Obsolete**) Trigger `update-csv.js` script.
+    *   `POST /import`: Trigger `import-csv.js` script.
+    *   `POST /cancel`: Cancel `/full-update` or `/full-reset` task.
+    *   `POST /reset`: (**Potentially Obsolete**) Trigger `reset-db.js` script.
+    *   `POST /reset-metrics`: Trigger `reset-metrics.js` script.
+    *   `POST /calculate-metrics`: Trigger `calculate-metrics.js` script.
+    *   `POST /import-from-prod`: Trigger `import-from-prod.js` script.
+    *   `POST /full-update`: Trigger `full-update.js` script (preferred update).
+    *   `POST /full-reset`: Trigger `full-reset.js` script (preferred reset).
+
+**8. Dashboard (`dashboard.js`)**
+    *   `GET /stock/metrics`: Get dashboard stock summary metrics & brand breakdown.
+    *   `GET /purchase/metrics`: Get dashboard purchase order summary metrics & vendor breakdown.
+    *   `GET /replenishment/metrics`: Get dashboard replenishment summary & top variants.
+    *   `GET /forecast/metrics`: Get dashboard forecast summary, daily, and category breakdown.
+    *   `GET /overstock/metrics`: Get dashboard overstock summary & category breakdown.
+    *   `GET /overstock/products`: Get list of top overstocked products.
+    *   `GET /best-sellers`: Get dashboard best-selling products, brands, categories - **uses `orders`, `products`**.
+    *   `GET /sales/metrics`: Get dashboard sales summary for a period - **uses `orders`**.
+    *   `GET /low-stock/products`: Get list of top low stock/critical products.
+    *   `GET /trending/products`: Get list of trending products - **uses `orders`, `products`**.
+    *   `GET /vendor/performance`: Get dashboard vendor performance details - **uses `purchase_orders`**.
+    *   `GET /key-metrics`: Get dashboard summary KPIs - **uses multiple base tables**.
+    *   `GET /inventory-health`: Get dashboard inventory health overview - **uses `products`, `product_metrics`**.
+    *   `GET /replenish/products`: Get list of products needing replenishment (overlaps `/low-stock/products`).
+    *   `GET /sales-overview`: Get daily sales totals for chart - **uses `orders`**.
+
+**9. Product Import Utilities (`products-import.js`)**
+    *   `POST /upload-image`: Upload temporary product image, schedule deletion.
+    *   `DELETE /delete-image`: Delete temporary product image.
+    *   `GET /field-options`: Get dropdown options for product fields from Prod MySQL (cached).
+    *   `GET /product-lines/:companyId`: Get product lines for a company from Prod MySQL (cached).
+    *   `GET /sublines/:lineId`: Get sublines for a line from Prod MySQL (cached).
+    *   `GET /check-file/:filename`: Check existence/permissions of uploaded file (temp or reusable).
+    *   `GET /list-uploads`: List files in upload directories.
+    *   `GET /search-products`: Search products in Prod MySQL DB.
+    *   `GET /check-upc-and-generate-sku`: Check UPC existence and generate SKU suggestion based on Prod MySQL data.
+    *   `GET /product-categories/:pid`: Get assigned categories for a product from Prod MySQL.
+
+**10. Product Metrics (`product-metrics.js`)**
+    *   `GET /filter-options`: Get distinct filter values (vendor, brand, abcClass) from `product_metrics`.
+    *   `GET /`: List detailed product metrics with filtering, sorting, pagination (primary data access).
+    *   `GET /:pid`: Get full metrics record for a single product.
+
+**11. Orders (`orders.js`)**
+    *   `GET /`: List orders with summary info, filtering, sorting, pagination, and stats.
+    *   `GET /:orderNumber`: Get details for a single order, including items.
+
+**12. Products (`products.js`)**
+    *   `GET /brands`: Get distinct brands (filtered by PO value).
+    *   `GET /`: List products with core data + metrics, filtering, sorting, pagination.
+    *   `GET /trending`: Get trending products based on `product_metrics`.
+    *   `GET /:id`: Get details for a single product (core data + metrics).
+    *   `POST /import`: (**Likely Obsolete/Dangerous**) Import products from CSV.
+    *   `PUT /:id`: Update core product data.
+    *   `GET /:id/metrics`: (**Redundant**) Get metrics for a single product.
+    *   `GET /:id/time-series`: Get sales/PO history for a single product.
+
+**13. Purchase Orders (`purchase-orders.js`)**
+    *   `GET /`: List purchase orders with summary info, filtering, sorting, pagination, and summary stats.
+    *   `GET /vendor-metrics`: Calculate vendor performance metrics from `purchase_orders`.
+    *   `GET /cost-analysis`: Calculate cost analysis by category from `purchase_orders`.
+    *   `GET /receiving-status`: Get summary counts based on PO receiving status.
+    *   `GET /order-vs-received`: List product ordered vs. received quantities.
+
+**14. Reusable Images (`reusable-images.js`)**
+    *   `GET /`: List all reusable images.
+    *   `GET /by-company/:companyId`: List global and company-specific images.
+    *   `GET /global`: List only global images.
+    *   `GET /:id`: Get a single reusable image record.
+    *   `POST /upload`: Upload a new reusable image and create DB record.
+    *   `PUT /:id`: Update reusable image metadata (name, global, company).
+    *   `DELETE /:id`: Delete reusable image record and file.
+    *   `GET /check-file/:filename`: Check existence/permissions of a reusable image file.
+
+**15. Templates (`templates.js`)**
+    *   `GET /`: List all product data templates.
+    *   `GET /:company/:productType`: Get a specific template.
+    *   `POST /`: Create a new template.
+    *   `PUT /:id`: Update an existing template.
+    *   `DELETE /:id`: Delete a template.
+
+**16. Vendors Aggregate (`vendors-aggregate.js`)**
+    *   `GET /filter-options`: Get distinct vendor names and statuses for UI filters (from `vendor_metrics`).
+    *   `GET /stats`: Get overall statistics related to vendors (from `vendor_metrics` & `purchase_orders`).
+    *   `GET /`: List vendors with aggregated metrics, supporting filtering, sorting, pagination (from `vendor_metrics` & `purchase_orders`).
+
+**Recommendations:**
+
+1.  **Address Obsolete Endpoints:** Prioritize removing or confirming the necessity of the endpoints marked as obsolete/redundant (legacy config, `/analytics/forecast`, `/csv/update`, `/csv/reset`, `/products/import`, `/products/:id/metrics`).
+2.  **Consolidate Overlapping Functionality:** Review the multiple vendor performance and product listing endpoints. Decide on the primary method (e.g., using aggregate tables via `/vendors-aggregate` and `/metrics`) and refactor or remove the others. Clarify the image upload strategies.
+3.  **Standardize Data Access:** Decide whether `dashboard` and `analytics` endpoints should primarily use aggregate tables (like `/metrics`, `/brands-aggregate`, etc.) or if direct access to base tables is sometimes necessary. Aim for consistency and document the reasoning. Optimize queries hitting base tables if they must remain.
+4.  **Improve Background Task Management:** Refactor `csv.js` to use a unified locking mechanism (maybe separate locks per task type?) and a consistent cancellation strategy for all spawned/managed processes. Clarify the purpose of `update` vs `full-update` and `reset` vs `full-reset`.
+5.  **Optimize DB Connections:** Ensure the `getDbConnection` pooling/caching helper from `products-import.js` is used *consistently* across all modules interacting with the production MySQL database (especially `ai_validation.js`). Remove unnecessary tunnel creations.
+6.  **Review Data Integrity:** Double-check the assumptions made (e.g., uniqueness of AI prompts) and ensure database constraints enforce them. Review the `GET /products/brands` filtering logic.
+
+## Changes Made
+
+1. **Removed Obsolete Legacy Endpoints in `config.js`**:
+   - Removed `GET /config/` endpoint
+   - Removed `PUT /config/stock-thresholds/:id` endpoint
+   - Removed `PUT /config/lead-time-thresholds/:id` endpoint
+   - Removed `PUT /config/sales-velocity/:id` endpoint
+   - Removed `PUT /config/abc-classification/:id` endpoint
+   - Removed `PUT /config/safety-stock/:id` endpoint
+   - Removed `PUT /config/turnover/:id` endpoint
+   
+   These endpoints were obsolete as they referenced older, single-row config tables that have been replaced by newer endpoints using the structured tables `settings_global`, `settings_product`, and `settings_vendor`.
+
+2. **Removed MySQL Syntax `/forecast` Endpoint in `analytics.js`**:
+   - Removed `GET /analytics/forecast` endpoint that was using MySQL-specific syntax incompatible with the PostgreSQL database used elsewhere in the application.
+
+3. **Renamed and Removed Redundant Endpoints**:
+   - Renamed `csv.js` to `data-management.js` while maintaining the same `/csv/*` endpoint paths for consistency
+   - Removed deprecated `/csv/update` endpoint (now fully replaced by `/csv/full-update`)
+   - Removed deprecated `/csv/reset` endpoint (now fully replaced by `/csv/full-reset`)
+   - Removed deprecated `/products/import` endpoint (now handled by `/csv/import`)
+   - Removed deprecated `/products/:id/metrics` endpoint (now handled by `/metrics/:pid`)
+
+4. **Fixed Data Integrity Issues**:
+   - Improved `GET /products/brands` endpoint by removing the arbitrary filtering logic that was only showing brands with purchase orders that had a total cost of at least $500
+   - The updated endpoint now returns all distinct brands from visible products, providing more complete data
+
+5. **Optimized Database Connections**:
+   - Created a new `dbConnection.js` utility file that encapsulates the optimized database connection management logic
+   - Improved the `ai-validation.js` file to use this shared connection management, eliminating unnecessary repeated tunnel creation
+   - Added proper connection pooling with timeout-based connection reuse, reducing the overhead of repeatedly creating SSH tunnels
+   - Added query result caching for frequently accessed data to improve performance
+
+These changes improve maintainability by removing duplicate code, enhance consistency by standardizing on the newer endpoint patterns, and optimize performance by reducing redundant database connections.
+
+## Additional Improvements
+
+1. **Further Database Connection Optimizations**:
+   - Extended the use of the optimized database connection utility to additional endpoints in `ai-validation.js`
+   - Updated the `/validate` endpoint and `/test-taxonomy` endpoint to use `getDbConnection`
+   - Ensured consistent connection management across all routes that access the production database
+
+2. **AI Prompts Data Integrity Verification**:
+   - Confirmed proper uniqueness constraints are in place in the database schema for AI prompts
+   - The schema includes:
+     - `unique_company_prompt` constraint ensuring only one prompt per company
+     - `idx_unique_general_prompt` index ensuring only one general prompt in the system
+     - `idx_unique_system_prompt` index ensuring only one system prompt in the system
+   - Endpoint handlers properly handle uniqueness constraint violations with appropriate 409 Conflict responses
+   - Validation ensures company-specific prompts have company IDs, while general/system prompts do not
+
+3. **AI Prompts Endpoint Consolidation**:
+   - Added a new consolidated `/by-type` endpoint that handles all types of prompts (general, system, company_specific)
+   - Marked the existing separate endpoints as deprecated with console warnings
+   - Maintained backward compatibility while providing a cleaner API moving forward
+
+## Completed Items
+
+✅ Removed obsolete legacy endpoints in `config.js`  
+✅ Removed MySQL syntax `/forecast` endpoint in `analytics.js`  
+✅ Fixed `GET /products/brands` endpoint filtering logic  
+✅ Created reusable database connection utility (`dbConnection.js`)  
+✅ Optimized database connections in `ai-validation.js`  
+✅ Verified data integrity in AI prompts handling  
+✅ Consolidated AI prompts endpoints with a unified `/by-type` endpoint  
+
+## Remaining Items
+
+- Consider adding additional error handling and logging for database connections
+- Perform load testing on the optimized database connections to ensure they handle high traffic properly

docs/setup-chat.md

@@ -0,0 +1,23 @@
+This portion of the application is going to be a read only chat archive. It will pull data from a rocketchat export converted to postgresql. This is a separate database than the rest of the inventory application uses, but it will still use users and permissions from the inventory database. Both databases are on the same postgres instance.
+
+For now, let's add a select to the top of the page that allows me to "view as" any of the users in the rocketchat database. We'll connect this to the authorization in the main application later. 
+
+The db connection info is stored in the .env file in the inventory-server root. It contains these variables
+DB_HOST=localhost
+DB_USER=rocketchat_user
+DB_PASSWORD=password
+DB_NAME=rocketchat_converted
+DB_PORT=5432
+
+Not all of the information in this database is relevant as it's a direct export from another app with more features. You can use the query tool to examine the structure and data available.
+
+Server-side files should use similar conventions and the same technologies as the inventory-server (inventor-server root) and auth-server (inventory-server/auth). I will provide my current pm2 ecosystem file upon request for you to add the configuration for the new "chat-server". I use Caddy on the server and can provide my caddyfile to assist with configuring the api routes. All configuration and routes for the chat-server should go in the inventory-server/chat folder or subfolders you create.
+
+The folder you see as inventory-server is actually a direct mount of the /var/www/html/inventory folder on the server. You can read and write files from there like usual, but any terminal commands for the server I will have to run myself.
+
+The "Chat" page should be added to the main application sidebar and a similar page to the others should be created in inventory/src/pages. All other frontend pages should go in inventory/src/components/chat. 
+
+The application uses shadcn components and those should be used for all ui elements where possible (located in inventory/src/components/ui). The UI should match existing pages and components. 
+
+
+

docs/split-up-pos.md

@@ -0,0 +1,112 @@
+Okay, I understand completely now. The core issue is that the previous approaches tried too hard to reconcile every receipt back to a specific PO line within the `purchase_orders` table structure, which doesn't reflect the reality where receipts can be independent events. Your downstream scripts, especially `daily_snapshots` and `product_metrics`, rely on having a complete picture of *all* receivings.
+
+Let's pivot to a model that respects both distinct data streams: **Orders (Intent)** and **Receivings (Actuals)**.
+
+**Proposed Solution: Separate `purchase_orders` and `receivings` Tables**
+
+This is the cleanest way to model the reality you've described.
+
+1.  **`purchase_orders` Table:**
+    *   **Purpose:** Tracks the status and details of purchase *orders* placed. Represents the *intent* to receive goods.
+    *   **Key Columns:** `po_id`, `pid`, `ordered` (quantity ordered), `po_cost_price`, `date` (order/created date), `expected_date`, `status` (PO lifecycle: 'ordered', 'canceled', 'done'), `vendor`, `notes`, etc.
+    *   **Crucially:** This table *does not* need a `received` column or a `receiving_history` column derived from complex allocations. It focuses solely on the PO itself.
+
+2.  **`receivings` Table (New or Refined):**
+    *   **Purpose:** Tracks every single line item received, regardless of whether it was linked to a PO during the receiving process. Represents the *actual* goods that arrived.
+    *   **Key Columns:**
+        *   `receiving_id` (Identifier for the overall receiving document/batch)
+        *   `pid` (Product ID received)
+        *   `received_qty` (Quantity received for this specific line)
+        *   `cost_each` (Actual cost paid for this item on this receiving)
+        *   `received_date` (Actual date the item was received)
+        *   `received_by` (Employee ID/Name)
+        *   `source_po_id` (The `po_id` entered on the receiving screen, *nullable*. Stores the original link attempt, even if it was wrong or missing)
+        *   `source_receiving_status` (The status from the source `receivings` table: 'partial_received', 'full_received', 'paid', 'canceled')
+
+**How the Import Script Changes:**
+
+1.  **Fetch POs:** Fetch data from `po` and `po_products`.
+2.  **Populate `purchase_orders`:**
+    *   Insert/Update rows into `purchase_orders` based directly on the fetched PO data.
+    *   Set `po_id`, `pid`, `ordered`, `po_cost_price`, `date` (`COALESCE(date_ordered, date_created)`), `expected_date`.
+    *   Set `status` by mapping the source `po.status` code directly ('ordered', 'canceled', 'done', etc.).
+    *   **No complex allocation needed here.**
+3.  **Fetch Receivings:** Fetch data from `receivings` and `receivings_products`.
+4.  **Populate `receivings`:**
+    *   For *every* line item fetched from `receivings_products`:
+        *   Perform necessary data validation (dates, numbers).
+        *   Insert a new row into `receivings` with all the relevant details (`receiving_id`, `pid`, `received_qty`, `cost_each`, `received_date`, `received_by`, `source_po_id`, `source_receiving_status`).
+    *   Use `ON CONFLICT (receiving_id, pid)` (or similar unique key based on your source data) `DO UPDATE SET ...` for incremental updates if necessary, or simply delete/re-insert based on `receiving_id` for simplicity if performance allows.
+
+**Impact on Downstream Scripts (and how to adapt):**
+
+*   **Initial Query (Active POs):**
+    *   `SELECT ... FROM purchase_orders po WHERE po.status NOT IN ('canceled', 'done', 'paid_equivalent_status?') AND po.date >= ...`
+    *   `active_pos`: `COUNT(DISTINCT po.po_id)` based on the filtered POs.
+    *   `overdue_pos`: Add `AND po.expected_date < CURRENT_DATE`.
+    *   `total_units`: `SUM(po.ordered)`. Represents total units *ordered* on active POs.
+    *   `total_cost`: `SUM(po.ordered * po.po_cost_price)`. Cost of units *ordered*.
+    *   `total_retail`: `SUM(po.ordered * pm.current_price)`. Retail value of units *ordered*.
+    *   **Result:** This query now cleanly reports on the status of *orders* placed, which seems closer to its original intent. The filter `po.receiving_status NOT IN ('partial_received', 'full_received', 'paid')` is replaced by `po.status NOT IN ('canceled', 'done', 'paid_equivalent?')`. The 90% received check is removed as `received` is not reliably tracked *on the PO* anymore.
+
+*   **`daily_product_snapshots`:**
+    *   **`SalesData` CTE:** No change needed.
+    *   **`ReceivingData` CTE:** **Must be changed.** Query the **`receivings`** table instead of `purchase_orders`.
+        ```sql
+        ReceivingData AS (
+            SELECT
+                rl.pid,
+                COUNT(DISTINCT rl.receiving_id) as receiving_doc_count, 
+                SUM(rl.received_qty) AS units_received,
+                SUM(rl.received_qty * rl.cost_each) AS cost_received
+            FROM public.receivings rl
+            WHERE rl.received_date::date = _date
+              -- Optional: Filter out canceled receivings if needed
+              -- AND rl.source_receiving_status <> 'canceled' 
+            GROUP BY rl.pid
+        ),
+        ```
+    *   **Result:** This now accurately reflects *all* units received on a given day from the definitive source.
+
+*   **`update_product_metrics`:**
+    *   **`CurrentInfo` CTE:** No change needed (pulls from `products`).
+    *   **`OnOrderInfo` CTE:** Needs re-evaluation. How do you want to define "On Order"?
+        *   **Option A (Strict PO View):** `SUM(po.ordered)` from `purchase_orders po WHERE po.status NOT IN ('canceled', 'done', 'paid_equivalent?')`. This is quantity on *open orders*, ignoring fulfillment state. Simple, but might overestimate if items arrived unlinked.
+        *   **Option B (Approximate Fulfillment):** `SUM(po.ordered)` from open POs MINUS `SUM(rl.received_qty)` from `receivings rl` where `rl.source_po_id = po.po_id` (summing only directly linked receivings). Better, but still misses fulfillment via unlinked receivings.
+        *   **Option C (Heuristic):** `SUM(po.ordered)` from open POs MINUS `SUM(rl.received_qty)` from `receivings rl` where `rl.pid = po.pid` and `rl.received_date >= po.date`. This *tries* to account for unlinked receivings but is imprecise.
+        *   **Recommendation:** Start with **Option A** for simplicity, clearly labeling it "Quantity on Open POs". You might need a separate process or metric for a more nuanced view of expected vs. actual pipeline.
+        ```sql
+         -- Example for Option A
+         OnOrderInfo AS (
+             SELECT
+                 pid,
+                 SUM(ordered) AS on_order_qty, -- Total qty on open POs
+                 SUM(ordered * po_cost_price) AS on_order_cost -- Cost of qty on open POs
+             FROM public.purchase_orders
+             WHERE status NOT IN ('canceled', 'done', 'paid_equivalent?') -- Define your open statuses
+             GROUP BY pid
+         ),
+        ```
+    *   **`HistoricalDates` CTE:**
+        *   `date_first_sold`, `max_order_date`: No change (queries `orders`).
+        *   `date_first_received_calc`, `date_last_received_calc`: **Must be changed.** Query `MIN(rl.received_date)` and `MAX(rl.received_date)` from the **`receivings`** table grouped by `pid`.
+    *   **`SnapshotAggregates` CTE:**
+        *   `received_qty_30d`, `received_cost_30d`: These are calculated from `daily_product_snapshots`, which are now correctly sourced from `receivings`, so this part is fine.
+    *   **Forecasting Calculations:** Will use the chosen definition of `on_order_qty`. Be aware of the implications of Option A (potentially inflated if unlinked receivings fulfill orders).
+    *   **Result:** Metrics are calculated based on distinct order data and complete receiving data. The definition of "on order" needs careful consideration.
+
+**Summary of this Approach:**
+
+*   **Pros:**
+    *   Accurately models distinct order and receiving events.
+    *   Provides a definitive source (`receivings`) for all received inventory.
+    *   Simplifies the `purchase_orders` table and its import logic.
+    *   Avoids complex/potentially inaccurate allocation logic for unlinked receivings within the main tables.
+    *   Avoids synthetic records.
+    *   Fixes downstream reporting (`daily_snapshots` receiving data).
+*   **Cons:**
+    *   Requires creating/managing the `receivings` table.
+    *   Requires modifying downstream queries (`ReceivingData`, `OnOrderInfo`, `HistoricalDates`).
+    *   Calculating a precise "net quantity still expected to arrive" (true on-order minus all relevant fulfillment) becomes more complex and may require specific business rules or heuristics outside the basic table structure if Option A for `OnOrderInfo` isn't sufficient.
+
+This two-table approach (`purchase_orders` + `receivings`) seems the most robust and accurate way to handle your requirement for complete receiving records independent of potentially flawed PO linking. It directly addresses the shortcomings of the previous attempts.

ecosystem.config.js

@@ -1,222 +0,0 @@
-// ecosystem.config.js
-const path = require('path');
-const dotenv = require('dotenv');
-
-// Load environment variables safely with error handling
-const loadEnvFile = (envPath) => {
-  try {
-    console.log('Loading env from:', envPath);
-    const result = dotenv.config({ path: envPath });
-    if (result.error) {
-      console.warn(`Warning: .env file not found or invalid at ${envPath}:`, result.error.message);
-      return {};
-    }
-    console.log('Env variables loaded from', envPath, ':', Object.keys(result.parsed || {}));
-    return result.parsed || {};
-  } catch (error) {
-    console.warn(`Warning: Error loading .env file at ${envPath}:`, error.message);
-    return {};
-  }
-}
-
-// Load environment variables for each server
-const authEnv = loadEnvFile(path.resolve(__dirname, 'dashboard/auth-server/.env'));
-const aircallEnv = loadEnvFile(path.resolve(__dirname, 'dashboard/aircall-server/.env'));
-const klaviyoEnv = loadEnvFile(path.resolve(__dirname, 'dashboard/klaviyo-server/.env'));
-const metaEnv = loadEnvFile(path.resolve(__dirname, 'dashboard/meta-server/.env'));
-const googleAnalyticsEnv = require('dotenv').config({ 
-  path: path.resolve(__dirname, 'dashboard/google-server/.env') 
-}).parsed || {};
-const typeformEnv = loadEnvFile(path.resolve(__dirname, 'dashboard/typeform-server/.env'));
-const inventoryEnv = loadEnvFile(path.resolve(__dirname, 'inventory/.env'));
-
-// Common log settings for all apps
-const logSettings = {
-  log_rotate: true,
-  max_size: '10M',
-  retain: '10',
-  log_date_format: 'YYYY-MM-DD HH:mm:ss'
-};
-
-// Common app settings
-const commonSettings = {
-  instances: 1,
-  exec_mode: 'fork',
-  autorestart: true,
-  watch: false,
-  max_memory_restart: '1G',
-  time: true,
-  ...logSettings,
-  ignore_watch: [
-    'node_modules',
-    'logs',
-    '.git',
-    '*.log'
-  ],
-  min_uptime: 5000,
-  max_restarts: 5,
-  restart_delay: 4000,
-  listen_timeout: 50000,
-  kill_timeout: 5000,
-  node_args: '--max-old-space-size=1536'
-};
-
-module.exports = {
-  apps: [
-    {
-      ...commonSettings,
-      name: 'auth-server',
-      script: './dashboard/auth-server/index.js',
-      env: {
-        NODE_ENV: 'production',
-        PORT: 3003,
-        ...authEnv
-      },
-      error_file: 'dashboard/auth-server/logs/pm2/err.log',
-      out_file: 'dashboard/auth-server/logs/pm2/out.log',
-      log_file: 'dashboard/auth-server/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        PORT: 3003
-      },
-      env_development: {
-        NODE_ENV: 'development',
-        PORT: 3003
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'aircall-server',
-      script: './dashboard/aircall-server/server.js',
-      env: {
-        NODE_ENV: 'production',
-        AIRCALL_PORT: 3002,
-        ...aircallEnv
-      },
-      error_file: 'dashboard/aircall-server/logs/pm2/err.log',
-      out_file: 'dashboard/aircall-server/logs/pm2/out.log',
-      log_file: 'dashboard/aircall-server/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        AIRCALL_PORT: 3002
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'klaviyo-server',
-      script: './dashboard/klaviyo-server/server.js',
-      env: {
-        NODE_ENV: 'production',
-        KLAVIYO_PORT: 3004,
-        ...klaviyoEnv
-      },
-      error_file: 'dashboard/klaviyo-server/logs/pm2/err.log',
-      out_file: 'dashboard/klaviyo-server/logs/pm2/out.log',
-      log_file: 'dashboard/klaviyo-server/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        KLAVIYO_PORT: 3004
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'meta-server',
-      script: './dashboard/meta-server/server.js',
-      env: {
-        NODE_ENV: 'production',
-        PORT: 3005,
-        ...metaEnv
-      },
-      error_file: 'dashboard/meta-server/logs/pm2/err.log',
-      out_file: 'dashboard/meta-server/logs/pm2/out.log',
-      log_file: 'dashboard/meta-server/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        PORT: 3005
-      }
-    },
-    {
-      name: "gorgias-server",
-      script: "./dashboard/gorgias-server/server.js",
-      env: {
-        NODE_ENV: "development",
-        PORT: 3006
-      },
-      env_production: {
-        NODE_ENV: "production",
-        PORT: 3006
-      },
-      error_file: "dashboard/logs/gorgias-server-error.log",
-      out_file: "dashboard/logs/gorgias-server-out.log",
-      log_file: "dashboard/logs/gorgias-server-combined.log",
-      time: true
-    },
-    {
-      ...commonSettings,
-      name: 'google-server',
-      script: path.resolve(__dirname, 'dashboard/google-server/server.js'),
-      watch: false,
-      env: {
-        NODE_ENV: 'production',
-        GOOGLE_ANALYTICS_PORT: 3007,
-        ...googleAnalyticsEnv
-      },
-      error_file: path.resolve(__dirname, 'dashboard/google-server/logs/pm2/err.log'),
-      out_file: path.resolve(__dirname, 'dashboard/google-server/logs/pm2/out.log'),
-      log_file: path.resolve(__dirname, 'dashboard/google-server/logs/pm2/combined.log'),
-      env_production: {
-        NODE_ENV: 'production',
-        GOOGLE_ANALYTICS_PORT: 3007
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'typeform-server',
-      script: './dashboard/typeform-server/server.js',
-      env: {
-        NODE_ENV: 'production',
-        TYPEFORM_PORT: 3008,
-        ...typeformEnv
-      },
-      error_file: 'dashboard/typeform-server/logs/pm2/err.log',
-      out_file: 'dashboard/typeform-server/logs/pm2/out.log',
-      log_file: 'dashboard/typeform-server/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        TYPEFORM_PORT: 3008
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'inventory-server',
-      script: './inventory/src/server.js',
-      env: {
-        NODE_ENV: 'production',
-        PORT: 3010,
-        ...inventoryEnv
-      },
-      error_file: 'inventory/logs/pm2/err.log',
-      out_file: 'inventory/logs/pm2/out.log',
-      log_file: 'inventory/logs/pm2/combined.log',
-      env_production: {
-        NODE_ENV: 'production',
-        PORT: 3010,
-        ...inventoryEnv
-      }
-    },
-    {
-      ...commonSettings,
-      name: 'new-auth-server',
-      script: './inventory-server/auth/server.js',
-      env: {
-        NODE_ENV: 'production',
-        AUTH_PORT: 3011,
-        ...inventoryEnv,
-        JWT_SECRET: process.env.JWT_SECRET
-      },
-      error_file: 'inventory-server/auth/logs/pm2/err.log',
-      out_file: 'inventory-server/auth/logs/pm2/out.log',
-      log_file: 'inventory-server/auth/logs/pm2/combined.log'
-    }
-  ]
-};

inventory-server/auth/add-user.js

@@ -1,103 +0,0 @@
-require('dotenv').config({ path: '../.env' });
-const bcrypt = require('bcrypt');
-const { Pool } = require('pg');
-const inquirer = require('inquirer');
-
-// Log connection details for debugging (remove in production)
-console.log('Attempting to connect with:', {
-  host: process.env.DB_HOST,
-  user: process.env.DB_USER,
-  database: process.env.DB_NAME,
-  port: process.env.DB_PORT
-});
-
-const pool = new Pool({
-  host: process.env.DB_HOST,
-  user: process.env.DB_USER,
-  password: process.env.DB_PASSWORD,
-  database: process.env.DB_NAME,
-  port: process.env.DB_PORT,
-});
-
-async function promptUser() {
-  const questions = [
-    {
-      type: 'input',
-      name: 'username',
-      message: 'Enter username:',
-      validate: (input) => {
-        if (input.length < 3) {
-          return 'Username must be at least 3 characters long';
-        }
-        return true;
-      }
-    },
-    {
-      type: 'password',
-      name: 'password',
-      message: 'Enter password:',
-      mask: '*',
-      validate: (input) => {
-        if (input.length < 8) {
-          return 'Password must be at least 8 characters long';
-        }
-        return true;
-      }
-    },
-    {
-      type: 'password',
-      name: 'confirmPassword',
-      message: 'Confirm password:',
-      mask: '*',
-      validate: (input, answers) => {
-        if (input !== answers.password) {
-          return 'Passwords do not match';
-        }
-        return true;
-      }
-    }
-  ];
-
-  return inquirer.prompt(questions);
-}
-
-async function addUser() {
-  try {
-    // Get user input
-    const answers = await promptUser();
-    const { username, password } = answers;
-
-    // Hash password
-    const saltRounds = 10;
-    const hashedPassword = await bcrypt.hash(password, saltRounds);
-
-    // Check if user already exists
-    const checkResult = await pool.query(
-      'SELECT id FROM users WHERE username = $1',
-      [username]
-    );
-
-    if (checkResult.rows.length > 0) {
-      console.error('Error: Username already exists');
-      process.exit(1);
-    }
-
-    // Insert new user
-    const result = await pool.query(
-      'INSERT INTO users (username, password) VALUES ($1, $2) RETURNING id',
-      [username, hashedPassword]
-    );
-
-    console.log(`User ${username} created successfully with id ${result.rows[0].id}`);
-  } catch (error) {
-    console.error('Error creating user:', error);
-    console.error('Error details:', error.message);
-    if (error.code) {
-      console.error('Error code:', error.code);
-    }
-  } finally {
-    await pool.end();
-  }
-}
-
-addUser(); 

inventory-server/auth/package.json

@@ -1,19 +0,0 @@
-{
-  "name": "inventory-auth-server",
-  "version": "1.0.0",
-  "description": "Authentication server for inventory management system",
-  "main": "server.js",
-  "scripts": {
-    "start": "node server.js"
-  },
-  "dependencies": {
-    "bcrypt": "^5.1.1",
-    "cors": "^2.8.5",
-    "dotenv": "^16.4.7",
-    "express": "^4.18.2",
-    "inquirer": "^8.2.6",
-    "jsonwebtoken": "^9.0.2",
-    "morgan": "^1.10.0",
-    "pg": "^8.11.3"
-  }
-}

inventory-server/auth/permissions.js

@@ -1,128 +0,0 @@
-// Get pool from global or create a new one if not available
-let pool;
-if (typeof global.pool !== 'undefined') {
-  pool = global.pool;
-} else {
-  // If global pool is not available, create a new connection
-  const { Pool } = require('pg');
-  pool = new Pool({
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    port: process.env.DB_PORT,
-  });
-  console.log('Created new database pool in permissions.js');
-}
-
-/**
- * Check if a user has a specific permission
- * @param {number} userId - The user ID to check
- * @param {string} permissionCode - The permission code to check
- * @returns {Promise<boolean>} - Whether the user has the permission
- */
-async function checkPermission(userId, permissionCode) {
-  try {
-    // First check if the user is an admin
-    const adminResult = await pool.query(
-      'SELECT is_admin FROM users WHERE id = $1',
-      [userId]
-    );
-    
-    // If user is admin, automatically grant permission
-    if (adminResult.rows.length > 0 && adminResult.rows[0].is_admin) {
-      return true;
-    }
-    
-    // Otherwise check for specific permission
-    const result = await pool.query(
-      `SELECT COUNT(*) AS has_permission
-       FROM user_permissions up
-       JOIN permissions p ON up.permission_id = p.id
-       WHERE up.user_id = $1 AND p.code = $2`,
-      [userId, permissionCode]
-    );
-    
-    return result.rows[0].has_permission > 0;
-  } catch (error) {
-    console.error('Error checking permission:', error);
-    return false;
-  }
-}
-
-/**
- * Middleware to require a specific permission
- * @param {string} permissionCode - The permission code required
- * @returns {Function} - Express middleware function
- */
-function requirePermission(permissionCode) {
-  return async (req, res, next) => {
-    try {
-      // Check if user is authenticated
-      if (!req.user || !req.user.id) {
-        return res.status(401).json({ error: 'Authentication required' });
-      }
-      
-      const hasPermission = await checkPermission(req.user.id, permissionCode);
-      
-      if (!hasPermission) {
-        return res.status(403).json({ 
-          error: 'Insufficient permissions',
-          requiredPermission: permissionCode
-        });
-      }
-      
-      next();
-    } catch (error) {
-      console.error('Permission middleware error:', error);
-      res.status(500).json({ error: 'Server error checking permissions' });
-    }
-  };
-}
-
-/**
- * Get all permissions for a user
- * @param {number} userId - The user ID
- * @returns {Promise<string[]>} - Array of permission codes
- */
-async function getUserPermissions(userId) {
-  try {
-    // Check if user is admin
-    const adminResult = await pool.query(
-      'SELECT is_admin FROM users WHERE id = $1',
-      [userId]
-    );
-    
-    if (adminResult.rows.length === 0) {
-      return [];
-    }
-    
-    const isAdmin = adminResult.rows[0].is_admin;
-    
-    if (isAdmin) {
-      // Admin gets all permissions
-      const allPermissions = await pool.query('SELECT code FROM permissions');
-      return allPermissions.rows.map(p => p.code);
-    } else {
-      // Get assigned permissions
-      const permissions = await pool.query(
-        `SELECT p.code
-         FROM permissions p
-         JOIN user_permissions up ON p.id = up.permission_id
-         WHERE up.user_id = $1`,
-        [userId]
-      );
-      
-      return permissions.rows.map(p => p.code);
-    }
-  } catch (error) {
-    console.error('Error getting user permissions:', error);
-    return [];
-  }
-}
-
-module.exports = {
-  checkPermission,
-  requirePermission,
-  getUserPermissions
-}; 

inventory-server/auth/routes.js

@@ -1,513 +0,0 @@
-const express = require('express');
-const router = express.Router();
-const bcrypt = require('bcrypt');
-const jwt = require('jsonwebtoken');
-const { requirePermission, getUserPermissions } = require('./permissions');
-
-// Get pool from global or create a new one if not available
-let pool;
-if (typeof global.pool !== 'undefined') {
-  pool = global.pool;
-} else {
-  // If global pool is not available, create a new connection
-  const { Pool } = require('pg');
-  pool = new Pool({
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    port: process.env.DB_PORT,
-  });
-  console.log('Created new database pool in routes.js');
-}
-
-// Authentication middleware
-const authenticate = async (req, res, next) => {
-  try {
-    const authHeader = req.headers.authorization;
-    if (!authHeader || !authHeader.startsWith('Bearer ')) {
-      return res.status(401).json({ error: 'Authentication required' });
-    }
-
-    const token = authHeader.split(' ')[1];
-    const decoded = jwt.verify(token, process.env.JWT_SECRET);
-    
-    // Get user from database
-    const result = await pool.query(
-      'SELECT id, username, is_admin FROM users WHERE id = $1',
-      [decoded.userId]
-    );
-    
-    if (result.rows.length === 0) {
-      return res.status(401).json({ error: 'User not found' });
-    }
-    
-    // Attach user to request
-    req.user = result.rows[0];
-    next();
-  } catch (error) {
-    console.error('Authentication error:', error);
-    res.status(401).json({ error: 'Invalid token' });
-  }
-};
-
-// Login route
-router.post('/login', async (req, res) => {
-  try {
-    const { username, password } = req.body;
-    
-    // Get user from database
-    const result = await pool.query(
-      'SELECT id, username, password, is_admin, is_active FROM users WHERE username = $1',
-      [username]
-    );
-    
-    if (result.rows.length === 0) {
-      return res.status(401).json({ error: 'Invalid username or password' });
-    }
-    
-    const user = result.rows[0];
-    
-    // Check if user is active
-    if (!user.is_active) {
-      return res.status(403).json({ error: 'Account is inactive' });
-    }
-    
-    // Verify password
-    const validPassword = await bcrypt.compare(password, user.password);
-    if (!validPassword) {
-      return res.status(401).json({ error: 'Invalid username or password' });
-    }
-    
-    // Update last login
-    await pool.query(
-      'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
-      [user.id]
-    );
-    
-    // Generate JWT
-    const token = jwt.sign(
-      { userId: user.id, username: user.username },
-      process.env.JWT_SECRET,
-      { expiresIn: '8h' }
-    );
-    
-    // Get user permissions
-    const permissions = await getUserPermissions(user.id);
-    
-    res.json({
-      token,
-      user: {
-        id: user.id,
-        username: user.username,
-        is_admin: user.is_admin,
-        permissions
-      }
-    });
-  } catch (error) {
-    console.error('Login error:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Get current user
-router.get('/me', authenticate, async (req, res) => {
-  try {
-    // Get user permissions
-    const permissions = await getUserPermissions(req.user.id);
-    
-    res.json({
-      id: req.user.id,
-      username: req.user.username,
-      is_admin: req.user.is_admin,
-      permissions
-    });
-  } catch (error) {
-    console.error('Error getting current user:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Get all users
-router.get('/users', authenticate, requirePermission('view:users'), async (req, res) => {
-  try {
-    const result = await pool.query(`
-      SELECT id, username, email, is_admin, is_active, created_at, last_login
-      FROM users
-      ORDER BY username
-    `);
-    
-    res.json(result.rows);
-  } catch (error) {
-    console.error('Error getting users:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Get user with permissions
-router.get('/users/:id', authenticate, requirePermission('view:users'), async (req, res) => {
-  try {
-    const userId = req.params.id;
-    
-    // Get user details
-    const userResult = await pool.query(`
-      SELECT id, username, email, is_admin, is_active, created_at, last_login
-      FROM users
-      WHERE id = $1
-    `, [userId]);
-    
-    if (userResult.rows.length === 0) {
-      return res.status(404).json({ error: 'User not found' });
-    }
-    
-    // Get user permissions
-    const permissionsResult = await pool.query(`
-      SELECT p.id, p.name, p.code, p.category, p.description
-      FROM permissions p
-      JOIN user_permissions up ON p.id = up.permission_id
-      WHERE up.user_id = $1
-      ORDER BY p.category, p.name
-    `, [userId]);
-    
-    // Combine user and permissions
-    const user = {
-      ...userResult.rows[0],
-      permissions: permissionsResult.rows
-    };
-    
-    res.json(user);
-  } catch (error) {
-    console.error('Error getting user:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Create new user
-router.post('/users', authenticate, requirePermission('create:users'), async (req, res) => {
-  const client = await pool.connect();
-  
-  try {
-    const { username, email, password, is_admin, is_active, permissions } = req.body;
-    
-    console.log("Create user request:", {
-      username,
-      email,
-      is_admin,
-      is_active,
-      permissions: permissions || []
-    });
-    
-    // Validate required fields
-    if (!username || !password) {
-      return res.status(400).json({ error: 'Username and password are required' });
-    }
-    
-    // Check if username is taken
-    const existingUser = await client.query(
-      'SELECT id FROM users WHERE username = $1',
-      [username]
-    );
-    
-    if (existingUser.rows.length > 0) {
-      return res.status(400).json({ error: 'Username already exists' });
-    }
-    
-    // Start transaction
-    await client.query('BEGIN');
-    
-    // Hash password
-    const saltRounds = 10;
-    const hashedPassword = await bcrypt.hash(password, saltRounds);
-    
-    // Insert new user
-    const userResult = await client.query(`
-      INSERT INTO users (username, email, password, is_admin, is_active, created_at)
-      VALUES ($1, $2, $3, $4, $5, CURRENT_TIMESTAMP)
-      RETURNING id
-    `, [username, email || null, hashedPassword, !!is_admin, is_active !== false]);
-    
-    const userId = userResult.rows[0].id;
-    
-    // Assign permissions if provided and not admin
-    if (!is_admin && Array.isArray(permissions) && permissions.length > 0) {
-      console.log("Adding permissions for new user:", userId);
-      console.log("Permissions received:", permissions);
-      
-      // Check permission format
-      const permissionIds = permissions.map(p => {
-        if (typeof p === 'object' && p.id) {
-          console.log("Permission is an object with ID:", p.id);
-          return parseInt(p.id, 10);
-        } else if (typeof p === 'number') {
-          console.log("Permission is a number:", p);
-          return p;
-        } else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
-          console.log("Permission is a string that can be parsed as a number:", p);
-          return parseInt(p, 10);
-        } else {
-          console.log("Unknown permission format:", typeof p, p);
-          // If it's a permission code, we need to look up the ID
-          return null;
-        }
-      }).filter(id => id !== null);
-      
-      console.log("Filtered permission IDs:", permissionIds);
-      
-      if (permissionIds.length > 0) {
-        const permissionValues = permissionIds
-          .map(permId => `(${userId}, ${permId})`)
-          .join(',');
-        
-        console.log("Inserting permission values:", permissionValues);
-        
-        try {
-          await client.query(`
-            INSERT INTO user_permissions (user_id, permission_id)
-            VALUES ${permissionValues}
-            ON CONFLICT DO NOTHING
-          `);
-          console.log("Successfully inserted permissions for new user:", userId);
-        } catch (err) {
-          console.error("Error inserting permissions for new user:", err);
-          throw err;
-        }
-      } else {
-        console.log("No valid permission IDs found to insert for new user");
-      }
-    } else {
-      console.log("Not adding permissions: is_admin =", is_admin, "permissions array:", Array.isArray(permissions), "length:", permissions ? permissions.length : 0);
-    }
-    
-    await client.query('COMMIT');
-    
-    res.status(201).json({ 
-      id: userId, 
-      message: 'User created successfully' 
-    });
-  } catch (error) {
-    await client.query('ROLLBACK');
-    console.error('Error creating user:', error);
-    res.status(500).json({ error: 'Server error' });
-  } finally {
-    client.release();
-  }
-});
-
-// Update user
-router.put('/users/:id', authenticate, requirePermission('edit:users'), async (req, res) => {
-  const client = await pool.connect();
-  
-  try {
-    const userId = req.params.id;
-    const { username, email, password, is_admin, is_active, permissions } = req.body;
-    
-    console.log("Update user request:", {
-      userId,
-      username,
-      email,
-      is_admin,
-      is_active,
-      permissions: permissions || []
-    });
-    
-    // Check if user exists
-    const userExists = await client.query(
-      'SELECT id FROM users WHERE id = $1',
-      [userId]
-    );
-    
-    if (userExists.rows.length === 0) {
-      return res.status(404).json({ error: 'User not found' });
-    }
-    
-    // Start transaction
-    await client.query('BEGIN');
-    
-    // Build update fields
-    const updateFields = [];
-    const updateValues = [userId]; // First parameter is the user ID
-    let paramIndex = 2;
-    
-    if (username !== undefined) {
-      updateFields.push(`username = $${paramIndex++}`);
-      updateValues.push(username);
-    }
-    
-    if (email !== undefined) {
-      updateFields.push(`email = $${paramIndex++}`);
-      updateValues.push(email || null);
-    }
-    
-    if (is_admin !== undefined) {
-      updateFields.push(`is_admin = $${paramIndex++}`);
-      updateValues.push(!!is_admin);
-    }
-    
-    if (is_active !== undefined) {
-      updateFields.push(`is_active = $${paramIndex++}`);
-      updateValues.push(!!is_active);
-    }
-    
-    // Update password if provided
-    if (password) {
-      const saltRounds = 10;
-      const hashedPassword = await bcrypt.hash(password, saltRounds);
-      updateFields.push(`password = $${paramIndex++}`);
-      updateValues.push(hashedPassword);
-    }
-    
-    // Update user if there are fields to update
-    if (updateFields.length > 0) {
-      updateFields.push(`updated_at = CURRENT_TIMESTAMP`);
-      
-      await client.query(`
-        UPDATE users
-        SET ${updateFields.join(', ')}
-        WHERE id = $1
-      `, updateValues);
-    }
-    
-    // Update permissions if provided
-    if (Array.isArray(permissions)) {
-      console.log("Updating permissions for user:", userId);
-      console.log("Permissions received:", permissions);
-      
-      // First remove existing permissions
-      await client.query(
-        'DELETE FROM user_permissions WHERE user_id = $1',
-        [userId]
-      );
-      console.log("Deleted existing permissions for user:", userId);
-      
-      // Add new permissions if any and not admin
-      const newIsAdmin = is_admin !== undefined ? is_admin : (await client.query('SELECT is_admin FROM users WHERE id = $1', [userId])).rows[0].is_admin;
-      
-      console.log("User is admin:", newIsAdmin);
-      
-      if (!newIsAdmin && permissions.length > 0) {
-        console.log("Adding permissions:", permissions);
-        
-        // Check permission format
-        const permissionIds = permissions.map(p => {
-          if (typeof p === 'object' && p.id) {
-            console.log("Permission is an object with ID:", p.id);
-            return parseInt(p.id, 10);
-          } else if (typeof p === 'number') {
-            console.log("Permission is a number:", p);
-            return p;
-          } else if (typeof p === 'string' && !isNaN(parseInt(p, 10))) {
-            console.log("Permission is a string that can be parsed as a number:", p);
-            return parseInt(p, 10);
-          } else {
-            console.log("Unknown permission format:", typeof p, p);
-            // If it's a permission code, we need to look up the ID
-            return null;
-          }
-        }).filter(id => id !== null);
-        
-        console.log("Filtered permission IDs:", permissionIds);
-        
-        if (permissionIds.length > 0) {
-          const permissionValues = permissionIds
-            .map(permId => `(${userId}, ${permId})`)
-            .join(',');
-          
-          console.log("Inserting permission values:", permissionValues);
-          
-          try {
-            await client.query(`
-              INSERT INTO user_permissions (user_id, permission_id)
-              VALUES ${permissionValues}
-              ON CONFLICT DO NOTHING
-            `);
-            console.log("Successfully inserted permissions for user:", userId);
-          } catch (err) {
-            console.error("Error inserting permissions:", err);
-            throw err;
-          }
-        } else {
-          console.log("No valid permission IDs found to insert");
-        }
-      }
-    }
-    
-    await client.query('COMMIT');
-    
-    res.json({ message: 'User updated successfully' });
-  } catch (error) {
-    await client.query('ROLLBACK');
-    console.error('Error updating user:', error);
-    res.status(500).json({ error: 'Server error' });
-  } finally {
-    client.release();
-  }
-});
-
-// Delete user
-router.delete('/users/:id', authenticate, requirePermission('delete:users'), async (req, res) => {
-  try {
-    const userId = req.params.id;
-    
-    // Check that user is not deleting themselves
-    if (req.user.id === parseInt(userId, 10)) {
-      return res.status(400).json({ error: 'Cannot delete your own account' });
-    }
-    
-    // Delete user (this will cascade to user_permissions due to FK constraints)
-    const result = await pool.query(
-      'DELETE FROM users WHERE id = $1 RETURNING id',
-      [userId]
-    );
-    
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'User not found' });
-    }
-    
-    res.json({ message: 'User deleted successfully' });
-  } catch (error) {
-    console.error('Error deleting user:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Get all permissions grouped by category
-router.get('/permissions/categories', authenticate, requirePermission('view:users'), async (req, res) => {
-  try {
-    const result = await pool.query(`
-      SELECT category, json_agg(
-        json_build_object(
-          'id', id,
-          'name', name,
-          'code', code,
-          'description', description
-        ) ORDER BY name
-      ) as permissions
-      FROM permissions
-      GROUP BY category
-      ORDER BY category
-    `);
-    
-    res.json(result.rows);
-  } catch (error) {
-    console.error('Error getting permissions:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-// Get all permissions
-router.get('/permissions', authenticate, requirePermission('view:users'), async (req, res) => {
-  try {
-    const result = await pool.query(`
-      SELECT *
-      FROM permissions
-      ORDER BY category, name
-    `);
-    
-    res.json(result.rows);
-  } catch (error) {
-    console.error('Error getting permissions:', error);
-    res.status(500).json({ error: 'Server error' });
-  }
-});
-
-module.exports = router; 

inventory-server/auth/schema.sql

@@ -1,89 +0,0 @@
-CREATE TABLE users (
-  id SERIAL PRIMARY KEY,
-  username VARCHAR(255) NOT NULL UNIQUE,
-  password VARCHAR(255) NOT NULL,
-  email VARCHAR UNIQUE,
-  is_admin BOOLEAN DEFAULT FALSE,
-  is_active BOOLEAN DEFAULT TRUE,
-  last_login TIMESTAMP WITH TIME ZONE,
-  updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-); 
-
--- Function to update the updated_at timestamp
-CREATE OR REPLACE FUNCTION update_updated_at_column()
-RETURNS TRIGGER AS $$
-BEGIN
-    NEW.updated_at = CURRENT_TIMESTAMP;
-    RETURN NEW;
-END;
-$$ language 'plpgsql';
-
--- Sequence and defined type for users table if not exists
-CREATE SEQUENCE IF NOT EXISTS users_id_seq;
-
--- Create permissions table
-CREATE TABLE IF NOT EXISTS "public"."permissions" (
-    "id" SERIAL PRIMARY KEY,
-    "name" varchar NOT NULL UNIQUE,
-    "code" varchar NOT NULL UNIQUE,
-    "description" text,
-    "category" varchar NOT NULL,
-    "created_at" timestamp with time zone DEFAULT CURRENT_TIMESTAMP,
-    "updated_at" timestamp with time zone DEFAULT CURRENT_TIMESTAMP
-);
-
--- Create user_permissions junction table
-CREATE TABLE IF NOT EXISTS "public"."user_permissions" (
-    "user_id" int4 NOT NULL REFERENCES "public"."users"("id") ON DELETE CASCADE,
-    "permission_id" int4 NOT NULL REFERENCES "public"."permissions"("id") ON DELETE CASCADE,
-    "created_at" timestamp with time zone DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY ("user_id", "permission_id")
-);
-
--- Add triggers for updated_at on users and permissions
-DROP TRIGGER IF EXISTS update_users_updated_at ON users;
-CREATE TRIGGER update_users_updated_at
-    BEFORE UPDATE ON users
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
-DROP TRIGGER IF EXISTS update_permissions_updated_at ON permissions;
-CREATE TRIGGER update_permissions_updated_at
-    BEFORE UPDATE ON permissions
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Insert default permissions by page - only the ones used in application
-INSERT INTO permissions (name, code, description, category) VALUES 
-    ('Dashboard Access', 'access:dashboard', 'Can access the Dashboard page', 'Pages'),
-    ('Products Access', 'access:products', 'Can access the Products page', 'Pages'),
-    ('Categories Access', 'access:categories', 'Can access the Categories page', 'Pages'),
-    ('Vendors Access', 'access:vendors', 'Can access the Vendors page', 'Pages'),
-    ('Analytics Access', 'access:analytics', 'Can access the Analytics page', 'Pages'),
-    ('Forecasting Access', 'access:forecasting', 'Can access the Forecasting page', 'Pages'),
-    ('Purchase Orders Access', 'access:purchase_orders', 'Can access the Purchase Orders page', 'Pages'),
-    ('Import Access', 'access:import', 'Can access the Import page', 'Pages'),
-    ('Settings Access', 'access:settings', 'Can access the Settings page', 'Pages'),
-    ('AI Validation Debug Access', 'access:ai_validation_debug', 'Can access the AI Validation Debug page', 'Pages')
-ON CONFLICT (code) DO NOTHING;
-
--- Settings section permissions
-INSERT INTO permissions (name, code, description, category) VALUES
-    ('Data Management', 'settings:data_management', 'Access to the Data Management settings section', 'Settings'),
-    ('Stock Management', 'settings:stock_management', 'Access to the Stock Management settings section', 'Settings'),
-    ('Performance Metrics', 'settings:performance_metrics', 'Access to the Performance Metrics settings section', 'Settings'),
-    ('Calculation Settings', 'settings:calculation_settings', 'Access to the Calculation Settings section', 'Settings'),
-    ('Template Management', 'settings:templates', 'Access to the Template Management settings section', 'Settings'),
-    ('User Management', 'settings:user_management', 'Access to the User Management settings section', 'Settings')
-ON CONFLICT (code) DO NOTHING;
-
--- Set any existing users as admin
-UPDATE users SET is_admin = TRUE WHERE is_admin IS NULL;
-
--- Grant all permissions to admin users
-INSERT INTO user_permissions (user_id, permission_id)
-SELECT u.id, p.id
-FROM users u, permissions p
-WHERE u.is_admin = TRUE
-ON CONFLICT DO NOTHING; 

inventory-server/auth/server.js

@@ -35,7 +35,7 @@ global.pool = pool;
 app.use(express.json());
 app.use(morgan('combined'));
 app.use(cors({
-  origin: ['http://localhost:5173', 'http://localhost:5174', 'https://inventory.kent.pw'],
+  origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://tools.acherryontop.com', 'https://tools.acherryontop.com'],
   credentials: true
 }));
 
@@ -62,6 +62,12 @@ app.post('/login', async (req, res) => {
       return res.status(403).json({ error: 'Account is inactive' });
     }
 
+    // Update last login timestamp
+    await pool.query(
+      'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = $1',
+      [user.id]
+    );
+
     // Generate JWT token
     const token = jwt.sign(
       { userId: user.id, username: user.username },
@@ -108,7 +114,7 @@ app.get('/me', async (req, res) => {
     
     // Get user details from database
     const userResult = await pool.query(
-      'SELECT id, username, email, is_admin, is_active FROM users WHERE id = $1',
+      'SELECT id, username, email, is_admin, rocket_chat_user_id, is_active FROM users WHERE id = $1',
       [decoded.userId]
     );
     
@@ -135,6 +141,7 @@ app.get('/me', async (req, res) => {
       id: user.id,
       username: user.username,
       email: user.email,
+      rocket_chat_user_id: user.rocket_chat_user_id,
       is_admin: user.is_admin,
       permissions: permissions
     });

inventory-server/chat/server.js

@@ -0,0 +1,83 @@
+require('dotenv').config({ path: '../.env' });
+const express = require('express');
+const cors = require('cors');
+const { Pool } = require('pg');
+const morgan = require('morgan');
+const chatRoutes = require('./routes');
+
+// Log startup configuration
+console.log('Starting chat server with config:', {
+  host: process.env.CHAT_DB_HOST,
+  user: process.env.CHAT_DB_USER,
+  database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
+  port: process.env.CHAT_DB_PORT,
+  chat_port: process.env.CHAT_PORT || 3014
+});
+
+const app = express();
+const port = process.env.CHAT_PORT || 3014;
+
+// Database configuration for rocketchat_converted database
+const pool = new Pool({
+  host: process.env.CHAT_DB_HOST,
+  user: process.env.CHAT_DB_USER,
+  password: process.env.CHAT_DB_PASSWORD,
+  database: process.env.CHAT_DB_NAME || 'rocketchat_converted',
+  port: process.env.CHAT_DB_PORT,
+});
+
+// Make pool available globally
+global.pool = pool;
+
+// Middleware
+app.use(express.json());
+app.use(morgan('combined'));
+app.use(cors({
+  origin: ['http://localhost:5175', 'http://localhost:5174', 'https://inventory.kent.pw', 'https://tools.acherryontop.com', 'https://tools.acherryontop.com'],
+  credentials: true
+}));
+
+// Test database connection endpoint
+app.get('/test-db', async (req, res) => {
+  try {
+    const result = await pool.query('SELECT COUNT(*) as user_count FROM users WHERE active = true');
+    const messageResult = await pool.query('SELECT COUNT(*) as message_count FROM message');
+    const roomResult = await pool.query('SELECT COUNT(*) as room_count FROM room');
+    
+    res.json({
+      status: 'success',
+      database: 'rocketchat_converted',
+      stats: {
+        active_users: parseInt(result.rows[0].user_count),
+        total_messages: parseInt(messageResult.rows[0].message_count),
+        total_rooms: parseInt(roomResult.rows[0].room_count)
+      }
+    });
+  } catch (error) {
+    console.error('Database test error:', error);
+    res.status(500).json({ 
+      status: 'error',
+      error: 'Database connection failed',
+      details: error.message
+    });
+  }
+});
+
+// Mount all routes from routes.js
+app.use('/', chatRoutes);
+
+// Health check endpoint
+app.get('/health', (req, res) => {
+  res.json({ status: 'healthy' });
+});
+
+// Error handling middleware
+app.use((err, req, res, next) => {
+  console.error(err.stack);
+  res.status(500).json({ error: 'Something broke!' });
+});
+
+// Start server
+app.listen(port, () => {
+  console.log(`Chat server running on port ${port}`);
+}); 

inventory-server/dashboard/acot-server/routes/events.js

@@ -0,0 +1,973 @@
+const express = require('express');
+const router = express.Router();
+const { getDbConnection, getPoolStatus } = require('../db/connection');
+const { getTimeRangeConditions, formatBusinessDate, getBusinessDayBounds } = require('../utils/timeUtils');
+
+// Image URL generation utility
+const getImageUrls = (pid, iid = 1) => {
+  const imageUrlBase = 'https://sbing.com/i/products/0000/';
+  const paddedPid = pid.toString().padStart(6, '0');
+  const prefix = paddedPid.slice(0, 3);
+  const basePath = `${imageUrlBase}${prefix}/${pid}`;
+  return {
+    image: `${basePath}-t-${iid}.jpg`,
+    image_175: `${basePath}-175x175-${iid}.jpg`,
+    image_full: `${basePath}-o-${iid}.jpg`,
+    ImgThumb: `${basePath}-175x175-${iid}.jpg` // For ProductGrid component
+  };
+};
+
+// Main stats endpoint - replaces /api/klaviyo/events/stats
+router.get('/stats', async (req, res) => {
+  const startTime = Date.now();
+  console.log(`[STATS] Starting request for timeRange: ${req.query.timeRange}`);
+  
+  // Set a timeout for the entire operation
+  const timeoutPromise = new Promise((_, reject) => {
+    setTimeout(() => reject(new Error('Request timeout after 15 seconds')), 15000);
+  });
+  
+  try {
+    const mainOperation = async () => {
+      const { timeRange, startDate, endDate } = req.query;
+      console.log(`[STATS] Getting DB connection...`);
+      const { connection, release } = await getDbConnection();
+      console.log(`[STATS] DB connection obtained in ${Date.now() - startTime}ms`);
+    
+    const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
+    
+    // Main order stats query
+    const mainStatsQuery = `
+      SELECT 
+        COUNT(*) as orderCount,
+        SUM(summary_total) as revenue,
+        SUM(stats_prod_pieces) as itemCount,
+        AVG(summary_total) as averageOrderValue,
+        AVG(stats_prod_pieces) as averageItemsPerOrder,
+        SUM(CASE WHEN stats_waiting_preorder > 0 THEN 1 ELSE 0 END) as preOrderCount,
+        SUM(CASE WHEN ship_method_selected = 'localpickup' THEN 1 ELSE 0 END) as localPickupCount,
+        SUM(CASE WHEN ship_method_selected = 'holdit' THEN 1 ELSE 0 END) as onHoldCount,
+        SUM(CASE WHEN order_status IN (100, 92) THEN 1 ELSE 0 END) as shippedCount,
+        SUM(CASE WHEN order_status = 15 THEN 1 ELSE 0 END) as cancelledCount,
+        SUM(CASE WHEN order_status = 15 THEN summary_total ELSE 0 END) as cancelledTotal
+      FROM _order 
+      WHERE order_status > 15 AND ${whereClause}
+    `;
+    
+    const [mainStats] = await connection.execute(mainStatsQuery, params);
+    const stats = mainStats[0];
+    
+    // Refunds query
+    const refundsQuery = `
+      SELECT 
+        COUNT(*) as refundCount,
+        ABS(SUM(payment_amount)) as refundTotal
+      FROM order_payment op
+      JOIN _order o ON op.order_id = o.order_id
+      WHERE payment_amount < 0 AND o.order_status > 15 AND ${whereClause.replace('date_placed', 'o.date_placed')}
+    `;
+    
+    const [refundStats] = await connection.execute(refundsQuery, params);
+    
+    // Best revenue day query
+    const bestDayQuery = `
+      SELECT 
+        DATE(date_placed) as date,
+        SUM(summary_total) as revenue,
+        COUNT(*) as orders
+      FROM _order 
+      WHERE order_status > 15 AND ${whereClause}
+      GROUP BY DATE(date_placed)
+      ORDER BY revenue DESC
+      LIMIT 1
+    `;
+    
+    const [bestDayResult] = await connection.execute(bestDayQuery, params);
+    
+    // Peak hour query (for single day periods)
+    let peakHour = null;
+    if (['today', 'yesterday'].includes(timeRange)) {
+      const peakHourQuery = `
+        SELECT 
+          HOUR(date_placed) as hour,
+          COUNT(*) as count
+        FROM _order 
+        WHERE order_status > 15 AND ${whereClause}
+        GROUP BY HOUR(date_placed)
+        ORDER BY count DESC
+        LIMIT 1
+      `;
+      
+      const [peakHourResult] = await connection.execute(peakHourQuery, params);
+      if (peakHourResult.length > 0) {
+        const hour = peakHourResult[0].hour;
+        const date = new Date();
+        date.setHours(hour, 0, 0);
+        peakHour = {
+          hour,
+          count: peakHourResult[0].count,
+          displayHour: date.toLocaleString("en-US", { hour: "numeric", hour12: true })
+        };
+      }
+    }
+    
+    // Brands and categories query - simplified for now since we don't have the category tables
+    // We'll use a simple approach without company table for now
+    const brandsQuery = `
+      SELECT 
+        'Various Brands' as brandName,
+        COUNT(DISTINCT oi.order_id) as orderCount,
+        SUM(oi.qty_ordered) as itemCount,
+        SUM(oi.qty_ordered * oi.prod_price) as revenue
+      FROM order_items oi
+      JOIN _order o ON oi.order_id = o.order_id
+      JOIN products p ON oi.prod_pid = p.pid
+      WHERE o.order_status > 15 AND ${whereClause.replace('date_placed', 'o.date_placed')}
+      HAVING revenue > 0
+    `;
+    
+    const [brandsResult] = await connection.execute(brandsQuery, params);
+    
+    // For categories, we'll use a simplified approach
+    const categoriesQuery = `
+      SELECT 
+        'General' as categoryName,
+        COUNT(DISTINCT oi.order_id) as orderCount,
+        SUM(oi.qty_ordered) as itemCount,
+        SUM(oi.qty_ordered * oi.prod_price) as revenue
+      FROM order_items oi
+      JOIN _order o ON oi.order_id = o.order_id
+      JOIN products p ON oi.prod_pid = p.pid
+      WHERE o.order_status > 15 AND ${whereClause.replace('date_placed', 'o.date_placed')}
+      HAVING revenue > 0
+    `;
+    
+    const [categoriesResult] = await connection.execute(categoriesQuery, params);
+    
+    // Shipping locations query
+    const shippingQuery = `
+      SELECT 
+        ship_country,
+        ship_state,
+        ship_method_selected,
+        COUNT(*) as count
+      FROM _order 
+      WHERE order_status IN (100, 92) AND ${whereClause}
+      GROUP BY ship_country, ship_state, ship_method_selected
+    `;
+    
+    const [shippingResult] = await connection.execute(shippingQuery, params);
+    
+    // Process shipping data
+    const shippingStats = processShippingData(shippingResult, stats.shippedCount);
+    
+    // Order value range query
+    const orderRangeQuery = `
+      SELECT 
+        MIN(summary_total) as smallest,
+        MAX(summary_total) as largest
+      FROM _order 
+      WHERE order_status > 15 AND ${whereClause}
+    `;
+    
+    const [orderRangeResult] = await connection.execute(orderRangeQuery, params);
+    
+    // Calculate period progress for incomplete periods
+    let periodProgress = 100;
+    if (['today', 'thisWeek', 'thisMonth'].includes(timeRange)) {
+      periodProgress = calculatePeriodProgress(timeRange);
+    }
+    
+    // Previous period comparison data
+    const prevPeriodData = await getPreviousPeriodData(connection, timeRange, startDate, endDate);
+    
+    const response = {
+      timeRange: dateRange,
+      stats: {
+        revenue: parseFloat(stats.revenue || 0),
+        orderCount: parseInt(stats.orderCount || 0),
+        itemCount: parseInt(stats.itemCount || 0),
+        averageOrderValue: parseFloat(stats.averageOrderValue || 0),
+        averageItemsPerOrder: parseFloat(stats.averageItemsPerOrder || 0),
+        
+        // Order types
+        orderTypes: {
+          preOrders: {
+            count: parseInt(stats.preOrderCount || 0),
+            percentage: stats.orderCount > 0 ? (stats.preOrderCount / stats.orderCount) * 100 : 0
+          },
+          localPickup: {
+            count: parseInt(stats.localPickupCount || 0),
+            percentage: stats.orderCount > 0 ? (stats.localPickupCount / stats.orderCount) * 100 : 0
+          },
+          heldItems: {
+            count: parseInt(stats.onHoldCount || 0),
+            percentage: stats.orderCount > 0 ? (stats.onHoldCount / stats.orderCount) * 100 : 0
+          }
+        },
+        
+        // Shipping
+        shipping: {
+          shippedCount: parseInt(stats.shippedCount || 0),
+          locations: shippingStats.locations,
+          methodStats: shippingStats.methods
+        },
+        
+        // Brands and categories
+        brands: {
+          total: brandsResult.length,
+          list: brandsResult.slice(0, 50).map(brand => ({
+            name: brand.brandName,
+            count: parseInt(brand.itemCount),
+            revenue: parseFloat(brand.revenue)
+          }))
+        },
+        
+        categories: {
+          total: categoriesResult.length,
+          list: categoriesResult.slice(0, 50).map(category => ({
+            name: category.categoryName,
+            count: parseInt(category.itemCount),
+            revenue: parseFloat(category.revenue)
+          }))
+        },
+        
+        // Refunds and cancellations
+        refunds: {
+          total: parseFloat(refundStats[0]?.refundTotal || 0),
+          count: parseInt(refundStats[0]?.refundCount || 0)
+        },
+        
+        canceledOrders: {
+          total: parseFloat(stats.cancelledTotal || 0),
+          count: parseInt(stats.cancelledCount || 0)
+        },
+        
+        // Best day
+        bestRevenueDay: bestDayResult.length > 0 ? {
+          amount: parseFloat(bestDayResult[0].revenue),
+          displayDate: bestDayResult[0].date,
+          orders: parseInt(bestDayResult[0].orders)
+        } : null,
+        
+        // Peak hour (for single days)
+        peakOrderHour: peakHour,
+        
+        // Order value range
+        orderValueRange: orderRangeResult.length > 0 ? {
+          smallest: parseFloat(orderRangeResult[0].smallest || 0),
+          largest: parseFloat(orderRangeResult[0].largest || 0)
+        } : { smallest: 0, largest: 0 },
+        
+        // Period progress and projections
+        periodProgress,
+        projectedRevenue: periodProgress < 100 ? (stats.revenue / (periodProgress / 100)) : stats.revenue,
+        
+        // Previous period comparison
+        prevPeriodRevenue: prevPeriodData.revenue,
+        prevPeriodOrders: prevPeriodData.orderCount,
+        prevPeriodAOV: prevPeriodData.averageOrderValue
+      }
+    };
+    
+    return { response, release };
+    };
+    
+    // Race between the main operation and timeout
+    let result;
+    try {
+      result = await Promise.race([mainOperation(), timeoutPromise]);
+    } catch (error) {
+      // If it's a timeout, we don't have a release function to call
+      if (error.message.includes('timeout')) {
+        console.log(`[STATS] Request timed out in ${Date.now() - startTime}ms`);
+        throw error;
+      }
+      // For other errors, re-throw
+      throw error;
+    }
+    
+    const { response, release } = result;
+    
+    // Release connection back to pool
+    if (release) release();
+    
+    console.log(`[STATS] Request completed in ${Date.now() - startTime}ms`);
+    res.json(response);
+    
+  } catch (error) {
+    console.error('Error in /stats:', error);
+    console.log(`[STATS] Request failed in ${Date.now() - startTime}ms`);
+    res.status(500).json({ error: error.message });
+  }
+});
+
+// Daily details endpoint - replaces /api/klaviyo/events/stats/details
+router.get('/stats/details', async (req, res) => {
+  let release;
+  try {
+    const { timeRange, startDate, endDate, metric, daily } = req.query;
+    const { connection, release: releaseConn } = await getDbConnection();
+    release = releaseConn;
+
+    const { whereClause, params } = getTimeRangeConditions(timeRange, startDate, endDate);
+
+    // Daily breakdown query
+    const dailyQuery = `
+      SELECT 
+        DATE(date_placed) as date,
+        COUNT(*) as orders,
+        SUM(summary_total) as revenue,
+        AVG(summary_total) as averageOrderValue,
+        SUM(stats_prod_pieces) as itemCount
+      FROM _order 
+      WHERE order_status > 15 AND ${whereClause}
+      GROUP BY DATE(date_placed)
+      ORDER BY DATE(date_placed)
+    `;
+    
+    const [dailyResults] = await connection.execute(dailyQuery, params);
+    
+    // Get previous period data using the same logic as main stats endpoint
+    let prevWhereClause, prevParams;
+    
+    if (timeRange && timeRange !== 'custom') {
+      const prevTimeRange = getPreviousTimeRange(timeRange);
+      const result = getTimeRangeConditions(prevTimeRange);
+      prevWhereClause = result.whereClause;
+      prevParams = result.params;
+    } else {
+      // Custom date range - go back by the same duration
+      const start = new Date(startDate);
+      const end = new Date(endDate);
+      const duration = end.getTime() - start.getTime();
+      
+      const prevEnd = new Date(start.getTime() - 1);
+      const prevStart = new Date(prevEnd.getTime() - duration);
+      
+      prevWhereClause = 'date_placed >= ? AND date_placed <= ?';
+      prevParams = [prevStart.toISOString(), prevEnd.toISOString()];
+    }
+
+    // Get previous period daily data
+    const prevQuery = `
+      SELECT 
+        DATE(date_placed) as date,
+        COUNT(*) as prevOrders,
+        SUM(summary_total) as prevRevenue,
+        AVG(summary_total) as prevAvgOrderValue
+      FROM _order 
+      WHERE order_status > 15 AND ${prevWhereClause}
+      GROUP BY DATE(date_placed)
+    `;
+    
+    const [prevResults] = await connection.execute(prevQuery, prevParams);
+    
+    // Create a map for quick lookup of previous period data
+    const prevMap = new Map();
+    prevResults.forEach(prev => {
+      const key = new Date(prev.date).toISOString().split('T')[0];
+      prevMap.set(key, prev);
+    });
+    
+    // For period-to-period comparison, we need to map days by relative position
+    // since dates won't match exactly (e.g., current week vs previous week)
+    const dailyArray = dailyResults.map(day => ({
+      timestamp: day.date,
+      date: day.date,
+      orders: parseInt(day.orders),
+      revenue: parseFloat(day.revenue),
+      averageOrderValue: parseFloat(day.averageOrderValue || 0),
+      itemCount: parseInt(day.itemCount)
+    }));
+
+    const prevArray = prevResults.map(day => ({
+      orders: parseInt(day.prevOrders),
+      revenue: parseFloat(day.prevRevenue),
+      averageOrderValue: parseFloat(day.prevAvgOrderValue || 0)
+    }));
+
+    // Combine current and previous period data by matching relative positions
+    const statsWithComparison = dailyArray.map((day, index) => {
+      const prev = prevArray[index] || { orders: 0, revenue: 0, averageOrderValue: 0 };
+      
+      return {
+        ...day,
+        prevOrders: prev.orders,
+        prevRevenue: prev.revenue,
+        prevAvgOrderValue: prev.averageOrderValue
+      };
+    });
+    
+    res.json({ stats: statsWithComparison });
+    
+  } catch (error) {
+    console.error('Error in /stats/details:', error);
+    res.status(500).json({ error: error.message });
+  } finally {
+    // Release connection back to pool
+    if (release) release();
+  }
+});
+
+// Financial performance endpoint
+router.get('/financials', async (req, res) => {
+  let release;
+  try {
+    const { timeRange, startDate, endDate } = req.query;
+    const { connection, release: releaseConn } = await getDbConnection();
+    release = releaseConn;
+
+    const { whereClause, params, dateRange } = getTimeRangeConditions(timeRange, startDate, endDate);
+    const financialWhere = whereClause.replace(/date_placed/g, 'DATE_SUB(date_change, INTERVAL 1 HOUR)');
+
+    const [totalsRows] = await connection.execute(
+      buildFinancialTotalsQuery(financialWhere),
+      params
+    );
+
+    const totals = normalizeFinancialTotals(totalsRows[0]);
+
+    const [trendRows] = await connection.execute(
+      buildFinancialTrendQuery(financialWhere),
+      params
+    );
+
+    const trend = trendRows.map(normalizeFinancialTrendRow);
+
+    let previousTotals = null;
+    let comparison = null;
+
+    const previousRange = getPreviousPeriodRange(timeRange, startDate, endDate);
+    if (previousRange) {
+      const prevWhere = previousRange.whereClause.replace(/date_placed/g, 'DATE_SUB(date_change, INTERVAL 1 HOUR)');
+      const [previousRows] = await connection.execute(
+        buildFinancialTotalsQuery(prevWhere),
+        previousRange.params
+      );
+      previousTotals = normalizeFinancialTotals(previousRows[0]);
+      comparison = {
+        grossSales: calculateComparison(totals.grossSales, previousTotals.grossSales),
+        refunds: calculateComparison(totals.refunds, previousTotals.refunds),
+        taxCollected: calculateComparison(totals.taxCollected, previousTotals.taxCollected),
+        discounts: calculateComparison(totals.discounts, previousTotals.discounts),
+        cogs: calculateComparison(totals.cogs, previousTotals.cogs),
+        income: calculateComparison(totals.income, previousTotals.income),
+        profit: calculateComparison(totals.profit, previousTotals.profit),
+        margin: calculateComparison(totals.margin, previousTotals.margin),
+      };
+    }
+
+    res.json({
+      dateRange,
+      totals,
+      previousTotals,
+      comparison,
+      trend,
+    });
+  } catch (error) {
+    console.error('Error in /financials:', error);
+    res.status(500).json({ error: error.message });
+  } finally {
+    if (release) release();
+  }
+});
+
+// Products endpoint - replaces /api/klaviyo/events/products
+router.get('/products', async (req, res) => {
+  let release;
+  try {
+    const { timeRange, startDate, endDate } = req.query;
+    const { connection, release: releaseConn } = await getDbConnection();
+    release = releaseConn;
+    
+    const { whereClause, params } = getTimeRangeConditions(timeRange, startDate, endDate);
+    
+    const productsQuery = `
+      SELECT 
+        p.pid,
+        p.description as name,
+        SUM(oi.qty_ordered) as totalQuantity,
+        SUM(oi.qty_ordered * oi.prod_price) as totalRevenue,
+        COUNT(DISTINCT oi.order_id) as orderCount,
+        (SELECT pi.iid FROM product_images pi WHERE pi.pid = p.pid AND pi.order = 255 LIMIT 1) as primary_iid
+      FROM order_items oi
+      JOIN _order o ON oi.order_id = o.order_id
+      JOIN products p ON oi.prod_pid = p.pid
+      WHERE o.order_status > 15 AND ${whereClause.replace('date_placed', 'o.date_placed')}
+      GROUP BY p.pid, p.description
+      ORDER BY totalRevenue DESC
+      LIMIT 500
+    `;
+    
+    const [productsResult] = await connection.execute(productsQuery, params);
+    
+    // Add image URLs to each product
+    const productsWithImages = productsResult.map(product => {
+      const imageUrls = getImageUrls(product.pid, product.primary_iid || 1);
+      return {
+        id: product.pid,
+        name: product.name,
+        totalQuantity: parseInt(product.totalQuantity),
+        totalRevenue: parseFloat(product.totalRevenue),
+        orderCount: parseInt(product.orderCount),
+        ...imageUrls
+      };
+    });
+    
+    res.json({
+      stats: {
+        products: {
+          total: productsWithImages.length,
+          list: productsWithImages
+        }
+      }
+    });
+    
+  } catch (error) {
+    console.error('Error in /products:', error);
+    res.status(500).json({ error: error.message });
+  } finally {
+    // Release connection back to pool
+    if (release) release();
+  }
+});
+
+// Projection endpoint - replaces /api/klaviyo/events/projection
+router.get('/projection', async (req, res) => {
+  let release;
+  try {
+    const { timeRange, startDate, endDate } = req.query;
+    
+    // Only provide projections for incomplete periods
+    if (!['today', 'thisWeek', 'thisMonth'].includes(timeRange)) {
+      return res.json({ projectedRevenue: 0, confidence: 0 });
+    }
+    
+    const { connection, release: releaseConn } = await getDbConnection();
+    release = releaseConn;
+    
+    // Get current period data
+    const { whereClause, params } = getTimeRangeConditions(timeRange, startDate, endDate);
+    
+    const currentQuery = `
+      SELECT 
+        SUM(summary_total) as currentRevenue,
+        COUNT(*) as currentOrders
+      FROM _order 
+      WHERE order_status > 15 AND ${whereClause}
+    `;
+    
+    const [currentResult] = await connection.execute(currentQuery, params);
+    const current = currentResult[0];
+    
+    // Get historical data for the same period type
+    const historicalQuery = await getHistoricalProjectionData(connection, timeRange);
+    
+    // Calculate projection based on current progress and historical patterns
+    const periodProgress = calculatePeriodProgress(timeRange);
+    const projection = calculateSmartProjection(
+      parseFloat(current.currentRevenue || 0),
+      parseInt(current.currentOrders || 0),
+      periodProgress,
+      historicalQuery
+    );
+    
+    res.json(projection);
+    
+  } catch (error) {
+    console.error('Error in /projection:', error);
+    res.status(500).json({ error: error.message });
+  } finally {
+    // Release connection back to pool
+    if (release) release();
+  }
+});
+
+// Debug endpoint to check connection pool status
+router.get('/debug/pool', (req, res) => {
+  res.json(getPoolStatus());
+});
+
+// Health check endpoint
+router.get('/health', async (req, res) => {
+  try {
+    const { connection, release } = await getDbConnection();
+    
+    // Simple query to test connection
+    const [result] = await connection.execute('SELECT 1 as test');
+    release();
+    
+    res.json({ 
+      status: 'healthy', 
+      timestamp: new Date().toISOString(),
+      pool: getPoolStatus(),
+      dbTest: result[0]
+    });
+  } catch (error) {
+    res.status(500).json({ 
+      status: 'unhealthy', 
+      error: error.message,
+      timestamp: new Date().toISOString(),
+      pool: getPoolStatus()
+    });
+  }
+});
+
+// Helper functions
+function processShippingData(shippingResult, totalShipped) {
+  const countries = {};
+  const states = {};
+  const methods = {};
+  
+  shippingResult.forEach(row => {
+    // Countries
+    if (row.ship_country) {
+      countries[row.ship_country] = (countries[row.ship_country] || 0) + row.count;
+    }
+    
+    // States
+    if (row.ship_state) {
+      states[row.ship_state] = (states[row.ship_state] || 0) + row.count;
+    }
+    
+    // Methods
+    if (row.ship_method_selected) {
+      methods[row.ship_method_selected] = (methods[row.ship_method_selected] || 0) + row.count;
+    }
+  });
+  
+  return {
+    locations: {
+      total: totalShipped,
+      byCountry: Object.entries(countries)
+        .map(([country, count]) => ({
+          country,
+          count,
+          percentage: (count / totalShipped) * 100
+        }))
+        .sort((a, b) => b.count - a.count),
+      byState: Object.entries(states)
+        .map(([state, count]) => ({
+          state,
+          count,
+          percentage: (count / totalShipped) * 100
+        }))
+        .sort((a, b) => b.count - a.count)
+    },
+    methods: Object.entries(methods)
+      .map(([name, value]) => ({ name, value }))
+      .sort((a, b) => b.value - a.value)
+  };
+}
+
+function calculatePeriodProgress(timeRange) {
+  const now = new Date();
+  const easternTime = new Date(now.getTime() - (5 * 60 * 60 * 1000)); // UTC-5
+  
+  switch (timeRange) {
+    case 'today': {
+      const { start } = getBusinessDayBounds('today');
+      const businessStart = new Date(start);
+      const businessEnd = new Date(businessStart);
+      businessEnd.setDate(businessEnd.getDate() + 1);
+      businessEnd.setHours(0, 59, 59, 999); // 12:59 AM next day
+      
+      const elapsed = easternTime.getTime() - businessStart.getTime();
+      const total = businessEnd.getTime() - businessStart.getTime();
+      return Math.min(100, Math.max(0, (elapsed / total) * 100));
+    }
+    case 'thisWeek': {
+      const startOfWeek = new Date(easternTime);
+      startOfWeek.setDate(easternTime.getDate() - easternTime.getDay()); // Sunday
+      startOfWeek.setHours(1, 0, 0, 0); // 1 AM business day start
+      
+      const endOfWeek = new Date(startOfWeek);
+      endOfWeek.setDate(endOfWeek.getDate() + 7);
+      
+      const elapsed = easternTime.getTime() - startOfWeek.getTime();
+      const total = endOfWeek.getTime() - startOfWeek.getTime();
+      return Math.min(100, Math.max(0, (elapsed / total) * 100));
+    }
+    case 'thisMonth': {
+      const startOfMonth = new Date(easternTime.getFullYear(), easternTime.getMonth(), 1, 1, 0, 0, 0);
+      const endOfMonth = new Date(easternTime.getFullYear(), easternTime.getMonth() + 1, 1, 0, 59, 59, 999);
+      
+      const elapsed = easternTime.getTime() - startOfMonth.getTime();
+      const total = endOfMonth.getTime() - startOfMonth.getTime();
+      return Math.min(100, Math.max(0, (elapsed / total) * 100));
+    }
+    default:
+      return 100;
+  }
+}
+
+function buildFinancialTotalsQuery(whereClause) {
+  return `
+    SELECT 
+      COALESCE(SUM(sale_amount), 0) as grossSales,
+      COALESCE(SUM(refund_amount), 0) as refunds,
+      COALESCE(SUM(shipping_collected_amount + small_order_fee_amount + rush_fee_amount), 0) as shippingFees,
+      COALESCE(SUM(tax_collected_amount), 0) as taxCollected,
+      COALESCE(SUM(discount_total_amount), 0) as discounts,
+      COALESCE(SUM(cogs_amount), 0) as cogs
+    FROM report_sales_data
+    WHERE ${whereClause}
+      AND action IN (1, 2, 3)
+  `;
+}
+
+function buildFinancialTrendQuery(whereClause) {
+  return `
+    SELECT 
+      DATE(DATE_SUB(date_change, INTERVAL 1 HOUR)) as date,
+      SUM(sale_amount) as grossSales,
+      SUM(refund_amount) as refunds,
+      SUM(shipping_collected_amount + small_order_fee_amount + rush_fee_amount) as shippingFees,
+      SUM(tax_collected_amount) as taxCollected,
+      SUM(discount_total_amount) as discounts,
+      SUM(cogs_amount) as cogs
+    FROM report_sales_data
+    WHERE ${whereClause}
+      AND action IN (1, 2, 3)
+    GROUP BY DATE(DATE_SUB(date_change, INTERVAL 1 HOUR))
+    ORDER BY date ASC
+  `;
+}
+
+function normalizeFinancialTotals(row = {}) {
+  const grossSales = parseFloat(row.grossSales || 0);
+  const refunds = parseFloat(row.refunds || 0);
+  const shippingFees = parseFloat(row.shippingFees || 0);
+  const taxCollected = parseFloat(row.taxCollected || 0);
+  const discounts = parseFloat(row.discounts || 0);
+  const cogs = parseFloat(row.cogs || 0);
+  const productNet = grossSales - refunds - discounts;
+  const income = productNet + shippingFees;
+  const profit = income - cogs;
+  const margin = income !== 0 ? (profit / income) * 100 : 0;
+
+  return {
+    grossSales,
+    refunds,
+    shippingFees,
+    taxCollected,
+    discounts,
+    cogs,
+    income,
+    profit,
+    margin,
+  };
+}
+
+function normalizeFinancialTrendRow(row = {}) {
+  const grossSales = parseFloat(row.grossSales || 0);
+  const refunds = parseFloat(row.refunds || 0);
+  const shippingFees = parseFloat(row.shippingFees || 0);
+  const taxCollected = parseFloat(row.taxCollected || 0);
+  const discounts = parseFloat(row.discounts || 0);
+  const cogs = parseFloat(row.cogs || 0);
+  const productNet = grossSales - refunds - discounts;
+  const income = productNet + shippingFees;
+  const profit = income - cogs;
+  const margin = income !== 0 ? (profit / income) * 100 : 0;
+  let timestamp = null;
+  let dateValue = null;
+
+  if (row.date instanceof Date) {
+    dateValue = row.date.toISOString().slice(0, 10);
+  } else if (typeof row.date === 'string') {
+    dateValue = row.date;
+  }
+
+  if (typeof dateValue === 'string') {
+    timestamp = new Date(`${dateValue}T06:00:00.000Z`).toISOString();
+  }
+
+  return {
+    date: dateValue,
+    grossSales,
+    refunds,
+    shippingFees,
+    taxCollected,
+    discounts,
+    cogs,
+    income,
+    profit,
+    margin,
+    timestamp,
+  };
+}
+
+function calculateComparison(currentValue, previousValue) {
+  if (typeof previousValue !== 'number') {
+    return { absolute: null, percentage: null };
+  }
+
+  const absolute = typeof currentValue === 'number' ? currentValue - previousValue : null;
+  const percentage =
+    absolute !== null && previousValue !== 0
+      ? (absolute / Math.abs(previousValue)) * 100
+      : null;
+
+  return { absolute, percentage };
+}
+
+function getPreviousPeriodRange(timeRange, startDate, endDate) {
+  if (timeRange && timeRange !== 'custom') {
+    const prevTimeRange = getPreviousTimeRange(timeRange);
+    if (!prevTimeRange || prevTimeRange === timeRange) {
+      return null;
+    }
+    return getTimeRangeConditions(prevTimeRange);
+  }
+
+  const hasCustomDates = (timeRange === 'custom' || !timeRange) && startDate && endDate;
+  if (!hasCustomDates) {
+    return null;
+  }
+
+  const start = new Date(startDate);
+  const end = new Date(endDate);
+
+  if (Number.isNaN(start.getTime()) || Number.isNaN(end.getTime())) {
+    return null;
+  }
+
+  const duration = end.getTime() - start.getTime();
+  if (!Number.isFinite(duration) || duration <= 0) {
+    return null;
+  }
+
+  const prevEnd = new Date(start.getTime() - 1);
+  const prevStart = new Date(prevEnd.getTime() - duration);
+
+  return getTimeRangeConditions('custom', prevStart.toISOString(), prevEnd.toISOString());
+}
+
+async function getPreviousPeriodData(connection, timeRange, startDate, endDate) {
+  // Calculate previous period dates
+  let prevWhereClause, prevParams;
+  
+  if (timeRange && timeRange !== 'custom') {
+    const prevTimeRange = getPreviousTimeRange(timeRange);
+    const result = getTimeRangeConditions(prevTimeRange);
+    prevWhereClause = result.whereClause;
+    prevParams = result.params;
+  } else {
+    // Custom date range - go back by the same duration
+    const start = new Date(startDate);
+    const end = new Date(endDate);
+    const duration = end.getTime() - start.getTime();
+    
+    const prevEnd = new Date(start.getTime() - 1);
+    const prevStart = new Date(prevEnd.getTime() - duration);
+    
+    prevWhereClause = 'date_placed >= ? AND date_placed <= ?';
+    prevParams = [prevStart.toISOString(), prevEnd.toISOString()];
+  }
+  
+  const prevQuery = `
+    SELECT 
+      COUNT(*) as orderCount,
+      SUM(summary_total) as revenue,
+      AVG(summary_total) as averageOrderValue
+    FROM _order 
+    WHERE order_status > 15 AND ${prevWhereClause}
+  `;
+  
+  const [prevResult] = await connection.execute(prevQuery, prevParams);
+  const prev = prevResult[0] || { orderCount: 0, revenue: 0, averageOrderValue: 0 };
+  
+  return {
+    orderCount: parseInt(prev.orderCount || 0),
+    revenue: parseFloat(prev.revenue || 0),
+    averageOrderValue: parseFloat(prev.averageOrderValue || 0)
+  };
+}
+
+function getPreviousTimeRange(timeRange) {
+  const map = {
+    today: 'yesterday',
+    thisWeek: 'lastWeek',
+    thisMonth: 'lastMonth',
+    last7days: 'previous7days',
+    last30days: 'previous30days',
+    last90days: 'previous90days',
+    yesterday: 'twoDaysAgo'
+  };
+  return map[timeRange] || timeRange;
+}
+
+async function getHistoricalProjectionData(connection, timeRange) {
+  // Get historical data for projection calculations
+  // This is a simplified version - you could make this more sophisticated
+  const historicalQuery = `
+    SELECT 
+      SUM(summary_total) as revenue,
+      COUNT(*) as orders
+    FROM _order 
+    WHERE order_status > 15 
+      AND date_placed >= DATE_SUB(NOW(), INTERVAL 30 DAY)
+      AND date_placed < DATE_SUB(NOW(), INTERVAL 1 DAY)
+  `;
+  
+  const [result] = await connection.execute(historicalQuery);
+  return result;
+}
+
+function calculateSmartProjection(currentRevenue, currentOrders, periodProgress, historicalData) {
+  if (periodProgress >= 100) {
+    return { projectedRevenue: currentRevenue, projectedOrders: currentOrders, confidence: 1.0 };
+  }
+  
+  // Simple linear projection with confidence based on how much of the period has elapsed
+  const projectedRevenue = currentRevenue / (periodProgress / 100);
+  const projectedOrders = Math.round(currentOrders / (periodProgress / 100));
+  
+  // Confidence increases with more data (higher period progress)
+  const confidence = Math.min(0.95, Math.max(0.1, periodProgress / 100));
+  
+  return {
+    projectedRevenue,
+    projectedOrders,
+    confidence
+  };
+}
+
+// Health check endpoint
+router.get('/health', async (req, res) => {
+  try {
+    const poolStatus = getPoolStatus();
+    
+    // Test database connectivity
+    const { connection, release } = await getDbConnection();
+    await connection.execute('SELECT 1 as test');
+    release();
+    
+    res.json({
+      status: 'healthy',
+      timestamp: new Date().toISOString(),
+      pool: poolStatus,
+      database: 'connected'
+    });
+  } catch (error) {
+    console.error('Health check failed:', error);
+    res.status(500).json({
+      status: 'unhealthy',
+      timestamp: new Date().toISOString(),
+      error: error.message,
+      pool: getPoolStatus()
+    });
+  }
+});
+
+// Debug endpoint for pool status
+router.get('/debug/pool', (req, res) => {
+  res.json({
+    timestamp: new Date().toISOString(),
+    pool: getPoolStatus()
+  });
+});
+
+module.exports = router; 

inventory-server/db/config-schema.sql

@@ -1,278 +0,0 @@
--- Configuration tables schema
-
--- Create function for updating timestamps if it doesn't exist
-CREATE OR REPLACE FUNCTION update_updated_column()
-RETURNS TRIGGER AS $$
-BEGIN
-    NEW.updated = CURRENT_TIMESTAMP;
-    RETURN NEW;
-END;
-$$ language 'plpgsql';
-
--- Create function for updating updated_at timestamps
-CREATE OR REPLACE FUNCTION update_updated_at_column()
-RETURNS TRIGGER AS $$
-BEGIN
-    NEW.updated_at = CURRENT_TIMESTAMP;
-    RETURN NEW;
-END;
-$$ language 'plpgsql';
-
--- Stock threshold configurations
-CREATE TABLE stock_thresholds (
-    id INTEGER NOT NULL,
-    category_id BIGINT,  -- NULL means default/global threshold
-    vendor VARCHAR(100), -- NULL means applies to all vendors
-    critical_days INTEGER NOT NULL DEFAULT 7,
-    reorder_days INTEGER NOT NULL DEFAULT 14,
-    overstock_days INTEGER NOT NULL DEFAULT 90,
-    low_stock_threshold INTEGER NOT NULL DEFAULT 5,
-    min_reorder_quantity INTEGER NOT NULL DEFAULT 1,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE,
-    UNIQUE (category_id, vendor)
-);
-
-CREATE TRIGGER update_stock_thresholds_updated
-    BEFORE UPDATE ON stock_thresholds
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
-CREATE INDEX idx_st_metrics ON stock_thresholds(category_id, vendor);
-
--- Lead time threshold configurations
-CREATE TABLE lead_time_thresholds (
-    id INTEGER NOT NULL,
-    category_id BIGINT,  -- NULL means default/global threshold
-    vendor VARCHAR(100), -- NULL means applies to all vendors
-    target_days INTEGER NOT NULL DEFAULT 14,
-    warning_days INTEGER NOT NULL DEFAULT 21,
-    critical_days INTEGER NOT NULL DEFAULT 30,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE,
-    UNIQUE (category_id, vendor)
-);
-
-CREATE TRIGGER update_lead_time_thresholds_updated
-    BEFORE UPDATE ON lead_time_thresholds
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Sales velocity window configurations
-CREATE TABLE sales_velocity_config (
-    id INTEGER NOT NULL,
-    category_id BIGINT,  -- NULL means default/global threshold
-    vendor VARCHAR(100), -- NULL means applies to all vendors
-    daily_window_days INTEGER NOT NULL DEFAULT 30,
-    weekly_window_days INTEGER NOT NULL DEFAULT 7,
-    monthly_window_days INTEGER NOT NULL DEFAULT 90,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE,
-    UNIQUE (category_id, vendor)
-);
-
-CREATE TRIGGER update_sales_velocity_config_updated
-    BEFORE UPDATE ON sales_velocity_config
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
-CREATE INDEX idx_sv_metrics ON sales_velocity_config(category_id, vendor);
-
--- ABC Classification configurations
-CREATE TABLE abc_classification_config (
-    id INTEGER NOT NULL PRIMARY KEY,
-    a_threshold DECIMAL(5,2) NOT NULL DEFAULT 20.0,
-    b_threshold DECIMAL(5,2) NOT NULL DEFAULT 50.0,
-    classification_period_days INTEGER NOT NULL DEFAULT 90,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
-);
-
-CREATE TRIGGER update_abc_classification_config_updated
-    BEFORE UPDATE ON abc_classification_config
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Safety stock configurations
-CREATE TABLE safety_stock_config (
-    id INTEGER NOT NULL,
-    category_id BIGINT,  -- NULL means default/global threshold
-    vendor VARCHAR(100), -- NULL means applies to all vendors
-    coverage_days INTEGER NOT NULL DEFAULT 14,
-    service_level DECIMAL(5,2) NOT NULL DEFAULT 95.0,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE,
-    UNIQUE (category_id, vendor)
-);
-
-CREATE TRIGGER update_safety_stock_config_updated
-    BEFORE UPDATE ON safety_stock_config
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
-CREATE INDEX idx_ss_metrics ON safety_stock_config(category_id, vendor);
-
--- Turnover rate configurations
-CREATE TABLE turnover_config (
-    id INTEGER NOT NULL,
-    category_id BIGINT,  -- NULL means default/global threshold
-    vendor VARCHAR(100), -- NULL means applies to all vendors
-    calculation_period_days INTEGER NOT NULL DEFAULT 30,
-    target_rate DECIMAL(10,2) NOT NULL DEFAULT 1.0,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE,
-    UNIQUE (category_id, vendor)
-);
-
-CREATE TRIGGER update_turnover_config_updated
-    BEFORE UPDATE ON turnover_config
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Create table for sales seasonality factors
-CREATE TABLE sales_seasonality (
-    month INTEGER NOT NULL,
-    seasonality_factor DECIMAL(5,3) DEFAULT 0,
-    last_updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (month),
-    CONSTRAINT month_range CHECK (month BETWEEN 1 AND 12),
-    CONSTRAINT seasonality_range CHECK (seasonality_factor BETWEEN -1.0 AND 1.0)
-);
-
-CREATE TRIGGER update_sales_seasonality_updated
-    BEFORE UPDATE ON sales_seasonality
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Insert default global thresholds
-INSERT INTO stock_thresholds (id, category_id, vendor, critical_days, reorder_days, overstock_days)
-VALUES (1, NULL, NULL, 7, 14, 90)
-ON CONFLICT (id) DO UPDATE SET
-    critical_days = EXCLUDED.critical_days,
-    reorder_days = EXCLUDED.reorder_days,
-    overstock_days = EXCLUDED.overstock_days;
-
-INSERT INTO lead_time_thresholds (id, category_id, vendor, target_days, warning_days, critical_days)
-VALUES (1, NULL, NULL, 14, 21, 30)
-ON CONFLICT (id) DO UPDATE SET
-    target_days = EXCLUDED.target_days,
-    warning_days = EXCLUDED.warning_days,
-    critical_days = EXCLUDED.critical_days;
-
-INSERT INTO sales_velocity_config (id, category_id, vendor, daily_window_days, weekly_window_days, monthly_window_days)
-VALUES (1, NULL, NULL, 30, 7, 90)
-ON CONFLICT (id) DO UPDATE SET
-    daily_window_days = EXCLUDED.daily_window_days,
-    weekly_window_days = EXCLUDED.weekly_window_days,
-    monthly_window_days = EXCLUDED.monthly_window_days;
-
-INSERT INTO abc_classification_config (id, a_threshold, b_threshold, classification_period_days)
-VALUES (1, 20.0, 50.0, 90)
-ON CONFLICT (id) DO UPDATE SET
-    a_threshold = EXCLUDED.a_threshold,
-    b_threshold = EXCLUDED.b_threshold,
-    classification_period_days = EXCLUDED.classification_period_days;
-
-INSERT INTO safety_stock_config (id, category_id, vendor, coverage_days, service_level)
-VALUES (1, NULL, NULL, 14, 95.0)
-ON CONFLICT (id) DO UPDATE SET
-    coverage_days = EXCLUDED.coverage_days,
-    service_level = EXCLUDED.service_level;
-
-INSERT INTO turnover_config (id, category_id, vendor, calculation_period_days, target_rate)
-VALUES (1, NULL, NULL, 30, 1.0)
-ON CONFLICT (id) DO UPDATE SET
-    calculation_period_days = EXCLUDED.calculation_period_days,
-    target_rate = EXCLUDED.target_rate;
-
--- Insert default seasonality factors (neutral)
-INSERT INTO sales_seasonality (month, seasonality_factor)
-VALUES 
-    (1, 0), (2, 0), (3, 0), (4, 0), (5, 0), (6, 0),
-    (7, 0), (8, 0), (9, 0), (10, 0), (11, 0), (12, 0)
-ON CONFLICT (month) DO UPDATE SET
-    last_updated = CURRENT_TIMESTAMP;
-
--- View to show thresholds with category names
-CREATE OR REPLACE VIEW stock_thresholds_view AS
-SELECT 
-    st.*,
-    c.name as category_name,
-    CASE 
-        WHEN st.category_id IS NULL AND st.vendor IS NULL THEN 'Global Default'
-        WHEN st.category_id IS NULL THEN 'Vendor: ' || st.vendor
-        WHEN st.vendor IS NULL THEN 'Category: ' || c.name
-        ELSE 'Category: ' || c.name || ' / Vendor: ' || st.vendor
-    END as threshold_scope
-FROM 
-    stock_thresholds st
-LEFT JOIN 
-    categories c ON st.category_id = c.cat_id
-ORDER BY 
-    CASE 
-        WHEN st.category_id IS NULL AND st.vendor IS NULL THEN 1
-        WHEN st.category_id IS NULL THEN 2
-        WHEN st.vendor IS NULL THEN 3
-        ELSE 4
-    END,
-    c.name,
-    st.vendor;
-
--- History and status tables
-CREATE TABLE IF NOT EXISTS calculate_history (
-    id BIGSERIAL PRIMARY KEY,
-    start_time TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    end_time TIMESTAMP WITH TIME ZONE NULL,
-    duration_seconds INTEGER,
-    duration_minutes DECIMAL(10,2) GENERATED ALWAYS AS (duration_seconds::decimal / 60.0) STORED,
-    total_products INTEGER DEFAULT 0,
-    total_orders INTEGER DEFAULT 0,
-    total_purchase_orders INTEGER DEFAULT 0,
-    processed_products INTEGER DEFAULT 0,
-    processed_orders INTEGER DEFAULT 0,
-    processed_purchase_orders INTEGER DEFAULT 0,
-    status calculation_status DEFAULT 'running',
-    error_message TEXT,
-    additional_info JSONB
-);
-
-CREATE TABLE IF NOT EXISTS calculate_status (
-    module_name module_name PRIMARY KEY,
-    last_calculation_timestamp TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
-);
-
-CREATE TABLE IF NOT EXISTS sync_status (
-    table_name VARCHAR(50) PRIMARY KEY,
-    last_sync_timestamp TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    last_sync_id BIGINT
-);
-
-CREATE TABLE IF NOT EXISTS import_history (
-    id BIGSERIAL PRIMARY KEY,
-    table_name VARCHAR(50) NOT NULL,
-    start_time TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    end_time TIMESTAMP WITH TIME ZONE NULL,
-    duration_seconds INTEGER,
-    duration_minutes DECIMAL(10,2) GENERATED ALWAYS AS (duration_seconds::decimal / 60.0) STORED,
-    records_added INTEGER DEFAULT 0,
-    records_updated INTEGER DEFAULT 0,
-    is_incremental BOOLEAN DEFAULT FALSE,
-    status calculation_status DEFAULT 'running',
-    error_message TEXT,
-    additional_info JSONB
-);
-
--- Create all indexes after tables are fully created
-CREATE INDEX IF NOT EXISTS idx_last_calc ON calculate_status(last_calculation_timestamp);
-CREATE INDEX IF NOT EXISTS idx_last_sync ON sync_status(last_sync_timestamp);
-CREATE INDEX IF NOT EXISTS idx_table_time ON import_history(table_name, start_time); 

inventory-server/db/metrics-schema.sql

@@ -1,375 +0,0 @@
--- Disable foreign key checks
-SET session_replication_role = 'replica';
-
--- Temporary tables for batch metrics processing
-CREATE TABLE temp_sales_metrics (
-    pid BIGINT NOT NULL,
-    daily_sales_avg DECIMAL(10,3),
-    weekly_sales_avg DECIMAL(10,3),
-    monthly_sales_avg DECIMAL(10,3),
-    total_revenue DECIMAL(10,3),
-    avg_margin_percent DECIMAL(10,3),
-    first_sale_date DATE,
-    last_sale_date DATE,
-    PRIMARY KEY (pid)
-);
-
-CREATE TABLE temp_purchase_metrics (
-    pid BIGINT NOT NULL,
-    avg_lead_time_days INTEGER,
-    last_purchase_date DATE,
-    first_received_date DATE,
-    last_received_date DATE,
-    PRIMARY KEY (pid)
-);
-
--- New table for product metrics
-CREATE TABLE product_metrics (
-    pid BIGINT NOT NULL,
-    last_calculated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    -- Sales velocity metrics
-    daily_sales_avg DECIMAL(10,3),
-    weekly_sales_avg DECIMAL(10,3),
-    monthly_sales_avg DECIMAL(10,3),
-    avg_quantity_per_order DECIMAL(10,3),
-    number_of_orders INTEGER,
-    first_sale_date DATE,
-    last_sale_date DATE,
-    -- Stock metrics
-    days_of_inventory INTEGER,
-    weeks_of_inventory INTEGER,
-    reorder_point INTEGER,
-    safety_stock INTEGER,
-    reorder_qty INTEGER DEFAULT 0,
-    overstocked_amt INTEGER DEFAULT 0,
-    -- Financial metrics
-    avg_margin_percent DECIMAL(10,3),
-    total_revenue DECIMAL(10,3),
-    inventory_value DECIMAL(10,3),
-    cost_of_goods_sold DECIMAL(10,3),
-    gross_profit DECIMAL(10,3),
-    gmroi DECIMAL(10,3),
-    -- Purchase metrics
-    avg_lead_time_days INTEGER,
-    last_purchase_date DATE,
-    first_received_date DATE,
-    last_received_date DATE,
-    -- Classification metrics
-    abc_class CHAR(1),
-    stock_status VARCHAR(20),
-    -- Turnover metrics
-    turnover_rate DECIMAL(12,3),
-    -- Lead time metrics
-    current_lead_time INTEGER,
-    target_lead_time INTEGER,
-    lead_time_status VARCHAR(20),
-    -- Forecast metrics
-    forecast_accuracy DECIMAL(5,2) DEFAULT NULL,
-    forecast_bias DECIMAL(5,2) DEFAULT NULL,
-    last_forecast_date DATE DEFAULT NULL,
-    PRIMARY KEY (pid),
-    FOREIGN KEY (pid) REFERENCES products(pid) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_metrics_revenue ON product_metrics(total_revenue);
-CREATE INDEX idx_metrics_stock_status ON product_metrics(stock_status);
-CREATE INDEX idx_metrics_lead_time ON product_metrics(lead_time_status);
-CREATE INDEX idx_metrics_turnover ON product_metrics(turnover_rate);
-CREATE INDEX idx_metrics_last_calculated ON product_metrics(last_calculated_at);
-CREATE INDEX idx_metrics_abc ON product_metrics(abc_class);
-CREATE INDEX idx_metrics_sales ON product_metrics(daily_sales_avg, weekly_sales_avg, monthly_sales_avg);
-CREATE INDEX idx_metrics_forecast ON product_metrics(forecast_accuracy, forecast_bias);
-
--- New table for time-based aggregates
-CREATE TABLE product_time_aggregates (
-    pid BIGINT NOT NULL,
-    year INTEGER NOT NULL,
-    month INTEGER NOT NULL,
-    -- Sales metrics
-    total_quantity_sold INTEGER DEFAULT 0,
-    total_revenue DECIMAL(10,3) DEFAULT 0,
-    total_cost DECIMAL(10,3) DEFAULT 0,
-    order_count INTEGER DEFAULT 0,
-    -- Stock changes
-    stock_received INTEGER DEFAULT 0,
-    stock_ordered INTEGER DEFAULT 0,
-    -- Calculated fields
-    avg_price DECIMAL(10,3),
-    profit_margin DECIMAL(10,3),
-    inventory_value DECIMAL(10,3),
-    gmroi DECIMAL(10,3),
-    PRIMARY KEY (pid, year, month),
-    FOREIGN KEY (pid) REFERENCES products(pid) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_date ON product_time_aggregates(year, month);
-
--- Create vendor_details table
-CREATE TABLE vendor_details (
-    vendor VARCHAR(100) PRIMARY KEY,
-    contact_name VARCHAR(100),
-    email VARCHAR(255),
-    phone VARCHAR(50),
-    status VARCHAR(20) DEFAULT 'active',
-    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-);
-
-CREATE INDEX idx_vendor_details_status ON vendor_details(status);
-
--- New table for vendor metrics
-CREATE TABLE vendor_metrics (
-    vendor VARCHAR(100) NOT NULL,
-    last_calculated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    -- Performance metrics
-    avg_lead_time_days DECIMAL(10,3),
-    on_time_delivery_rate DECIMAL(5,2),
-    order_fill_rate DECIMAL(5,2),
-    total_orders INTEGER DEFAULT 0,
-    total_late_orders INTEGER DEFAULT 0,
-    total_purchase_value DECIMAL(10,3) DEFAULT 0,
-    avg_order_value DECIMAL(10,3),
-    -- Product metrics
-    active_products INTEGER DEFAULT 0,
-    total_products INTEGER DEFAULT 0,
-    -- Financial metrics
-    total_revenue DECIMAL(10,3) DEFAULT 0,
-    avg_margin_percent DECIMAL(5,2),
-    -- Status
-    status VARCHAR(20) DEFAULT 'active',
-    PRIMARY KEY (vendor),
-    FOREIGN KEY (vendor) REFERENCES vendor_details(vendor) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_vendor_performance ON vendor_metrics(on_time_delivery_rate);
-CREATE INDEX idx_vendor_status ON vendor_metrics(status);
-CREATE INDEX idx_vendor_metrics_last_calculated ON vendor_metrics(last_calculated_at);
-CREATE INDEX idx_vendor_metrics_orders ON vendor_metrics(total_orders, total_late_orders);
-
--- New table for category metrics
-CREATE TABLE category_metrics (
-    category_id BIGINT NOT NULL,
-    last_calculated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    -- Product metrics
-    product_count INTEGER DEFAULT 0,
-    active_products INTEGER DEFAULT 0,
-    -- Financial metrics
-    total_value DECIMAL(15,3) DEFAULT 0,
-    avg_margin DECIMAL(5,2),
-    turnover_rate DECIMAL(12,3),
-    growth_rate DECIMAL(5,2),
-    -- Status
-    status VARCHAR(20) DEFAULT 'active',
-    PRIMARY KEY (category_id),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_category_status ON category_metrics(status);
-CREATE INDEX idx_category_growth ON category_metrics(growth_rate);
-CREATE INDEX idx_metrics_last_calculated_cat ON category_metrics(last_calculated_at);
-CREATE INDEX idx_category_metrics_products ON category_metrics(product_count, active_products);
-
--- New table for vendor time-based metrics
-CREATE TABLE vendor_time_metrics (
-    vendor VARCHAR(100) NOT NULL,
-    year INTEGER NOT NULL,
-    month INTEGER NOT NULL,
-    -- Order metrics
-    total_orders INTEGER DEFAULT 0,
-    late_orders INTEGER DEFAULT 0,
-    avg_lead_time_days DECIMAL(10,3),
-    -- Financial metrics
-    total_purchase_value DECIMAL(10,3) DEFAULT 0,
-    total_revenue DECIMAL(10,3) DEFAULT 0,
-    avg_margin_percent DECIMAL(5,2),
-    PRIMARY KEY (vendor, year, month),
-    FOREIGN KEY (vendor) REFERENCES vendor_details(vendor) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_vendor_date ON vendor_time_metrics(year, month);
-
--- New table for category time-based metrics
-CREATE TABLE category_time_metrics (
-    category_id BIGINT NOT NULL,
-    year INTEGER NOT NULL,
-    month INTEGER NOT NULL,
-    -- Product metrics
-    product_count INTEGER DEFAULT 0,
-    active_products INTEGER DEFAULT 0,
-    -- Financial metrics
-    total_value DECIMAL(15,3) DEFAULT 0,
-    total_revenue DECIMAL(15,3) DEFAULT 0,
-    avg_margin DECIMAL(5,2),
-    turnover_rate DECIMAL(12,3),
-    PRIMARY KEY (category_id, year, month),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_category_date ON category_time_metrics(year, month);
-
--- New table for category-based sales metrics
-CREATE TABLE category_sales_metrics (
-    category_id BIGINT NOT NULL,
-    brand VARCHAR(100) NOT NULL,
-    period_start DATE NOT NULL,
-    period_end DATE NOT NULL,
-    avg_daily_sales DECIMAL(10,3) DEFAULT 0,
-    total_sold INTEGER DEFAULT 0,
-    num_products INTEGER DEFAULT 0,
-    avg_price DECIMAL(10,3) DEFAULT 0,
-    last_calculated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (category_id, brand, period_start, period_end),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_category_brand ON category_sales_metrics(category_id, brand);
-CREATE INDEX idx_period ON category_sales_metrics(period_start, period_end);
-
--- New table for brand metrics
-CREATE TABLE brand_metrics (
-    brand VARCHAR(100) NOT NULL,
-    last_calculated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    -- Product metrics
-    product_count INTEGER DEFAULT 0,
-    active_products INTEGER DEFAULT 0,
-    -- Stock metrics
-    total_stock_units INTEGER DEFAULT 0,
-    total_stock_cost DECIMAL(15,2) DEFAULT 0,
-    total_stock_retail DECIMAL(15,2) DEFAULT 0,
-    -- Sales metrics
-    total_revenue DECIMAL(15,2) DEFAULT 0,
-    avg_margin DECIMAL(5,2) DEFAULT 0,
-    growth_rate DECIMAL(5,2) DEFAULT 0,
-    PRIMARY KEY (brand)
-);
-
-CREATE INDEX idx_brand_metrics_last_calculated ON brand_metrics(last_calculated_at);
-CREATE INDEX idx_brand_metrics_revenue ON brand_metrics(total_revenue);
-CREATE INDEX idx_brand_metrics_growth ON brand_metrics(growth_rate);
-
--- New table for brand time-based metrics
-CREATE TABLE brand_time_metrics (
-    brand VARCHAR(100) NOT NULL,
-    year INTEGER NOT NULL,
-    month INTEGER NOT NULL,
-    -- Product metrics
-    product_count INTEGER DEFAULT 0,
-    active_products INTEGER DEFAULT 0,
-    -- Stock metrics
-    total_stock_units INTEGER DEFAULT 0,
-    total_stock_cost DECIMAL(15,2) DEFAULT 0,
-    total_stock_retail DECIMAL(15,2) DEFAULT 0,
-    -- Sales metrics
-    total_revenue DECIMAL(15,2) DEFAULT 0,
-    avg_margin DECIMAL(5,2) DEFAULT 0,
-    growth_rate DECIMAL(5,2) DEFAULT 0,
-    PRIMARY KEY (brand, year, month)
-);
-
-CREATE INDEX idx_brand_time_date ON brand_time_metrics(year, month);
-
--- New table for sales forecasts
-CREATE TABLE sales_forecasts (
-    pid BIGINT NOT NULL,
-    forecast_date DATE NOT NULL,
-    forecast_quantity INTEGER,
-    confidence_level DECIMAL(5,2),
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (pid, forecast_date),
-    FOREIGN KEY (pid) REFERENCES products(pid) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_forecast_date ON sales_forecasts(forecast_date);
-
--- New table for category forecasts
-CREATE TABLE category_forecasts (
-    category_id BIGINT NOT NULL,
-    forecast_date DATE NOT NULL,
-    forecast_revenue DECIMAL(15,2),
-    forecast_units INTEGER,
-    confidence_level DECIMAL(5,2),
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (category_id, forecast_date),
-    FOREIGN KEY (category_id) REFERENCES categories(cat_id) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_cat_forecast_date ON category_forecasts(forecast_date);
-
--- Create views for common calculations
-CREATE OR REPLACE VIEW inventory_health AS
-WITH stock_levels AS (
-    SELECT 
-        p.pid,
-        p.title,
-        p.SKU,
-        p.stock_quantity,
-        p.preorder_count,
-        pm.daily_sales_avg,
-        pm.weekly_sales_avg,
-        pm.monthly_sales_avg,
-        pm.reorder_point,
-        pm.safety_stock,
-        pm.days_of_inventory,
-        pm.weeks_of_inventory,
-        pm.stock_status,
-        pm.abc_class,
-        pm.turnover_rate,
-        pm.avg_lead_time_days,
-        pm.current_lead_time,
-        pm.target_lead_time,
-        pm.lead_time_status,
-        p.cost_price,
-        p.price,
-        pm.inventory_value,
-        pm.gmroi
-    FROM products p
-    LEFT JOIN product_metrics pm ON p.pid = pm.pid
-    WHERE p.managing_stock = true AND p.visible = true
-)
-SELECT 
-    *,
-    CASE 
-        WHEN stock_quantity <= safety_stock THEN 'Critical'
-        WHEN stock_quantity <= reorder_point THEN 'Low'
-        WHEN stock_quantity > (reorder_point * 3) THEN 'Excess'
-        ELSE 'Healthy'
-    END as inventory_status,
-    CASE 
-        WHEN lead_time_status = 'delayed' AND stock_status = 'low' THEN 'High'
-        WHEN lead_time_status = 'delayed' OR stock_status = 'low' THEN 'Medium'
-        ELSE 'Low'
-    END as risk_level
-FROM stock_levels;
-
--- Create view for category performance trends
-CREATE OR REPLACE VIEW category_performance_trends AS
-WITH monthly_trends AS (
-    SELECT 
-        c.cat_id,
-        c.name as category_name,
-        ctm.year,
-        ctm.month,
-        ctm.product_count,
-        ctm.active_products,
-        ctm.total_value,
-        ctm.total_revenue,
-        ctm.avg_margin,
-        ctm.turnover_rate,
-        LAG(ctm.total_revenue) OVER (PARTITION BY c.cat_id ORDER BY ctm.year, ctm.month) as prev_month_revenue,
-        LAG(ctm.turnover_rate) OVER (PARTITION BY c.cat_id ORDER BY ctm.year, ctm.month) as prev_month_turnover
-    FROM categories c
-    JOIN category_time_metrics ctm ON c.cat_id = ctm.category_id
-)
-SELECT 
-    *,
-    CASE 
-        WHEN prev_month_revenue IS NULL THEN 0
-        ELSE ((total_revenue - prev_month_revenue) / prev_month_revenue) * 100 
-    END as revenue_growth_percent,
-    CASE 
-        WHEN prev_month_turnover IS NULL THEN 0
-        ELSE ((turnover_rate - prev_month_turnover) / prev_month_turnover) * 100 
-    END as turnover_growth_percent
-FROM monthly_trends;
-
-SET session_replication_role = 'origin';

inventory-server/db/schema.sql

@@ -1,206 +0,0 @@
--- Enable strict error reporting
-SET session_replication_role = 'replica';  -- Disable foreign key checks temporarily
-
--- Create function for updating timestamps
-CREATE OR REPLACE FUNCTION update_updated_column() RETURNS TRIGGER AS $func$
-BEGIN
-    NEW.updated = CURRENT_TIMESTAMP;
-    RETURN NEW;
-END;
-$func$ language plpgsql;
-
--- Create tables
-CREATE TABLE products (
-    pid BIGINT NOT NULL,
-    title VARCHAR(255) NOT NULL,
-    description TEXT,
-    SKU VARCHAR(50) NOT NULL,
-    created_at TIMESTAMP WITH TIME ZONE,
-    first_received TIMESTAMP WITH TIME ZONE,
-    stock_quantity INTEGER DEFAULT 0,
-    preorder_count INTEGER DEFAULT 0,
-    notions_inv_count INTEGER DEFAULT 0,
-    price DECIMAL(10, 3) NOT NULL,
-    regular_price DECIMAL(10, 3) NOT NULL,
-    cost_price DECIMAL(10, 3),
-    landing_cost_price DECIMAL(10, 3),
-    barcode VARCHAR(50),
-    harmonized_tariff_code VARCHAR(20),
-    updated_at TIMESTAMP WITH TIME ZONE,
-    visible BOOLEAN DEFAULT true,
-    managing_stock BOOLEAN DEFAULT true,
-    replenishable BOOLEAN DEFAULT true,
-    vendor VARCHAR(100),
-    vendor_reference VARCHAR(100),
-    notions_reference VARCHAR(100),
-    permalink VARCHAR(255),
-    categories TEXT,
-    image VARCHAR(255),
-    image_175 VARCHAR(255),
-    image_full VARCHAR(255),
-    brand VARCHAR(100),
-    line VARCHAR(100),
-    subline VARCHAR(100),
-    artist VARCHAR(100),
-    options TEXT,
-    tags TEXT,
-    moq INTEGER DEFAULT 1,
-    uom INTEGER DEFAULT 1,
-    rating DECIMAL(10,2) DEFAULT 0.00,
-    reviews INTEGER DEFAULT 0,
-    weight DECIMAL(10,3),
-    length DECIMAL(10,3),
-    width DECIMAL(10,3),
-    height DECIMAL(10,3),
-    country_of_origin VARCHAR(5),
-    location VARCHAR(50),
-    total_sold INTEGER DEFAULT 0,
-    baskets INTEGER DEFAULT 0,
-    notifies INTEGER DEFAULT 0,
-    date_last_sold DATE,
-    updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (pid)
-);
-
--- Create trigger for products
-CREATE TRIGGER update_products_updated
-    BEFORE UPDATE ON products
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_column();
-
--- Create indexes for products table
-CREATE INDEX idx_products_sku ON products(SKU);
-CREATE INDEX idx_products_vendor ON products(vendor);
-CREATE INDEX idx_products_brand ON products(brand);
-CREATE INDEX idx_products_location ON products(location);
-CREATE INDEX idx_products_total_sold ON products(total_sold);
-CREATE INDEX idx_products_date_last_sold ON products(date_last_sold);
-CREATE INDEX idx_products_updated ON products(updated);
-
--- Create categories table with hierarchy support
-CREATE TABLE categories (
-    cat_id BIGINT PRIMARY KEY,
-    name VARCHAR(100) NOT NULL,
-    type SMALLINT NOT NULL,
-    parent_id BIGINT,
-    description TEXT,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    status VARCHAR(20) DEFAULT 'active',
-    FOREIGN KEY (parent_id) REFERENCES categories(cat_id)
-);
-
--- Create trigger for categories
-CREATE TRIGGER update_categories_updated_at
-    BEFORE UPDATE ON categories
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_column();
-
-COMMENT ON COLUMN categories.type IS '10=section, 11=category, 12=subcategory, 13=subsubcategory, 1=company, 2=line, 3=subline, 40=artist';
-
-CREATE INDEX idx_categories_parent ON categories(parent_id);
-CREATE INDEX idx_categories_type ON categories(type);
-CREATE INDEX idx_categories_status ON categories(status);
-CREATE INDEX idx_categories_name_type ON categories(name, type);
-
--- Create product_categories junction table
-CREATE TABLE product_categories (
-    cat_id BIGINT NOT NULL,
-    pid BIGINT NOT NULL,
-    PRIMARY KEY (pid, cat_id),
-    FOREIGN KEY (pid) REFERENCES products(pid) ON DELETE CASCADE,
-    FOREIGN KEY (cat_id) REFERENCES categories(cat_id) ON DELETE CASCADE
-);
-
-CREATE INDEX idx_product_categories_category ON product_categories(cat_id);
-CREATE INDEX idx_product_categories_product ON product_categories(pid);
-
--- Create orders table with its indexes
-CREATE TABLE orders (
-    id BIGSERIAL PRIMARY KEY,
-    order_number VARCHAR(50) NOT NULL,
-    pid BIGINT NOT NULL,
-    SKU VARCHAR(50) NOT NULL,
-    date DATE NOT NULL,
-    price DECIMAL(10,3) NOT NULL,
-    quantity INTEGER NOT NULL,
-    discount DECIMAL(10,3) DEFAULT 0.000,
-    tax DECIMAL(10,3) DEFAULT 0.000,
-    tax_included BOOLEAN DEFAULT false,
-    shipping DECIMAL(10,3) DEFAULT 0.000,
-    costeach DECIMAL(10,3) DEFAULT 0.000,
-    customer VARCHAR(50) NOT NULL,
-    customer_name VARCHAR(100),
-    status VARCHAR(20) DEFAULT 'pending',
-    canceled BOOLEAN DEFAULT false,
-    updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    UNIQUE (order_number, pid)
-);
-
--- Create trigger for orders
-CREATE TRIGGER update_orders_updated
-    BEFORE UPDATE ON orders
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_column();
-
-CREATE INDEX idx_orders_number ON orders(order_number);
-CREATE INDEX idx_orders_pid ON orders(pid);
-CREATE INDEX idx_orders_customer ON orders(customer);
-CREATE INDEX idx_orders_date ON orders(date);
-CREATE INDEX idx_orders_status ON orders(status);
-CREATE INDEX idx_orders_metrics ON orders(pid, date, canceled);
-CREATE INDEX idx_orders_updated ON orders(updated);
-
--- Create purchase_orders table with its indexes
-CREATE TABLE purchase_orders (
-    id BIGSERIAL PRIMARY KEY,
-    po_id VARCHAR(50) NOT NULL,
-    vendor VARCHAR(100) NOT NULL,
-    date DATE NOT NULL,
-    expected_date DATE,
-    pid BIGINT NOT NULL,
-    sku VARCHAR(50) NOT NULL,
-    name VARCHAR(100) NOT NULL,
-    cost_price DECIMAL(10, 3) NOT NULL,
-    po_cost_price DECIMAL(10, 3) NOT NULL,
-    status SMALLINT DEFAULT 1,
-    receiving_status SMALLINT DEFAULT 1,
-    notes TEXT,
-    long_note TEXT,
-    ordered INTEGER NOT NULL,
-    received INTEGER DEFAULT 0,
-    received_date DATE,
-    last_received_date DATE,
-    received_by VARCHAR(100),
-    receiving_history JSONB,
-    updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    FOREIGN KEY (pid) REFERENCES products(pid),
-    UNIQUE (po_id, pid)
-);
-
--- Create trigger for purchase_orders
-CREATE TRIGGER update_purchase_orders_updated
-    BEFORE UPDATE ON purchase_orders
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_column();
-
-COMMENT ON COLUMN purchase_orders.name IS 'Product name from products.description';
-COMMENT ON COLUMN purchase_orders.po_cost_price IS 'Original cost from PO, before receiving adjustments';
-COMMENT ON COLUMN purchase_orders.status IS '0=canceled,1=created,10=electronically_ready_send,11=ordered,12=preordered,13=electronically_sent,15=receiving_started,50=done';
-COMMENT ON COLUMN purchase_orders.receiving_status IS '0=canceled,1=created,30=partial_received,40=full_received,50=paid';
-COMMENT ON COLUMN purchase_orders.receiving_history IS 'Array of receiving records with qty, date, cost, receiving_id, and alt_po flag';
-
-CREATE INDEX idx_po_id ON purchase_orders(po_id);
-CREATE INDEX idx_po_vendor ON purchase_orders(vendor);
-CREATE INDEX idx_po_status ON purchase_orders(status);
-CREATE INDEX idx_po_receiving_status ON purchase_orders(receiving_status);
-CREATE INDEX idx_po_metrics ON purchase_orders(pid, date, status, ordered, received);
-CREATE INDEX idx_po_metrics_receiving ON purchase_orders(pid, date, receiving_status, received_date);
-CREATE INDEX idx_po_product_date ON purchase_orders(pid, date);
-CREATE INDEX idx_po_product_status ON purchase_orders(pid, status);
-CREATE INDEX idx_po_updated ON purchase_orders(updated);
-
-SET session_replication_role = 'origin';  -- Re-enable foreign key checks
-
--- Create views for common calculations
--- product_sales_trends view moved to metrics-schema.sql 

inventory-server/db/setup-schema.sql

@@ -1,85 +0,0 @@
--- Templates table for storing import templates
-CREATE TABLE IF NOT EXISTS templates (
-    id SERIAL PRIMARY KEY,
-    company TEXT NOT NULL,
-    product_type TEXT NOT NULL,
-    supplier TEXT,
-    msrp DECIMAL(10,2),
-    cost_each DECIMAL(10,2),
-    qty_per_unit INTEGER,
-    case_qty INTEGER,
-    hts_code TEXT,
-    description TEXT,
-    weight DECIMAL(10,2),
-    length DECIMAL(10,2),
-    width DECIMAL(10,2),
-    height DECIMAL(10,2),
-    tax_cat TEXT,
-    size_cat TEXT,
-    categories TEXT[],
-    ship_restrictions TEXT[],
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    UNIQUE(company, product_type)
-);
-
--- AI Prompts table for storing validation prompts
-CREATE TABLE IF NOT EXISTS ai_prompts (
-    id SERIAL PRIMARY KEY,
-    prompt_text TEXT NOT NULL,
-    prompt_type TEXT NOT NULL CHECK (prompt_type IN ('general', 'company_specific', 'system')),
-    company TEXT,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-    CONSTRAINT unique_company_prompt UNIQUE (company),
-    CONSTRAINT company_required_for_specific CHECK (
-        (prompt_type = 'general' AND company IS NULL) OR 
-        (prompt_type = 'system' AND company IS NULL) OR 
-        (prompt_type = 'company_specific' AND company IS NOT NULL)
-    )
-);
-
--- Create a unique partial index to ensure only one general prompt
-CREATE UNIQUE INDEX IF NOT EXISTS idx_unique_general_prompt 
-ON ai_prompts (prompt_type) 
-WHERE prompt_type = 'general';
-
--- Create a unique partial index to ensure only one system prompt
-CREATE UNIQUE INDEX IF NOT EXISTS idx_unique_system_prompt 
-ON ai_prompts (prompt_type) 
-WHERE prompt_type = 'system';
-
--- AI Validation Performance Tracking
-CREATE TABLE IF NOT EXISTS ai_validation_performance (
-    id SERIAL PRIMARY KEY,
-    prompt_length INTEGER NOT NULL,
-    product_count INTEGER NOT NULL,
-    start_time TIMESTAMP WITH TIME ZONE NOT NULL,
-    end_time TIMESTAMP WITH TIME ZONE NOT NULL,
-    duration_seconds DECIMAL(10,2) GENERATED ALWAYS AS (EXTRACT(EPOCH FROM (end_time - start_time))) STORED,
-    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
-);
-
--- Create index on prompt_length for efficient querying
-CREATE INDEX IF NOT EXISTS idx_ai_validation_prompt_length ON ai_validation_performance(prompt_length);
-
--- Function to update the updated_at timestamp
-CREATE OR REPLACE FUNCTION update_updated_at_column()
-RETURNS TRIGGER AS $$
-BEGIN
-    NEW.updated_at = CURRENT_TIMESTAMP;
-    RETURN NEW;
-END;
-$$ language 'plpgsql';
-
--- Trigger to automatically update the updated_at column
-CREATE TRIGGER update_templates_updated_at
-    BEFORE UPDATE ON templates
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column();
-
--- Trigger to automatically update the updated_at column for ai_prompts
-CREATE TRIGGER update_ai_prompts_updated_at
-    BEFORE UPDATE ON ai_prompts
-    FOR EACH ROW
-    EXECUTE FUNCTION update_updated_at_column(); 

inventory-server/old/old_csv/update-csv.js

@@ -1,180 +0,0 @@
-const path = require('path');
-const fs = require('fs');
-const axios = require('axios');
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate } = require('../metrics/utils/progress');
-
-// Change working directory to script directory
-process.chdir(path.dirname(__filename));
-
-require('dotenv').config({ path: path.resolve(__dirname, '..', '.env') });
-
-const FILES = [
-    {
-        name: '39f2x83-products.csv',
-        url: process.env.PRODUCTS_CSV_URL
-    },
-    {
-        name: '39f2x83-orders.csv',
-        url: process.env.ORDERS_CSV_URL
-    },
-    {
-        name: '39f2x83-purchase_orders.csv',
-        url: process.env.PURCHASE_ORDERS_CSV_URL
-    }
-];
-
-let isCancelled = false;
-
-function cancelUpdate() {
-    isCancelled = true;
-    outputProgress({
-        status: 'cancelled',
-        operation: 'CSV update cancelled',
-        current: 0,
-        total: FILES.length,
-        elapsed: null,
-        remaining: null,
-        rate: 0
-    });
-}
-
-async function downloadFile(file, index, startTime) {
-    if (isCancelled) return;
-
-    const csvDir = path.join(__dirname, '../csv');
-    if (!fs.existsSync(csvDir)) {
-        fs.mkdirSync(csvDir, { recursive: true });
-    }
-
-    const writer = fs.createWriteStream(path.join(csvDir, file.name));
-    
-    try {
-        const response = await axios({
-            url: file.url,
-            method: 'GET',
-            responseType: 'stream'
-        });
-
-        const totalLength = response.headers['content-length'];
-        let downloadedLength = 0;
-        let lastProgressUpdate = Date.now();
-        const PROGRESS_INTERVAL = 1000; // Update progress every second
-
-        response.data.on('data', (chunk) => {
-            if (isCancelled) {
-                writer.end();
-                return;
-            }
-
-            downloadedLength += chunk.length;
-            
-            // Update progress based on time interval
-            const now = Date.now();
-            if (now - lastProgressUpdate >= PROGRESS_INTERVAL) {
-                const progress = (downloadedLength / totalLength) * 100;
-                outputProgress({
-                    status: 'running',
-                    operation: `Downloading ${file.name}`,
-                    current: index + (downloadedLength / totalLength),
-                    total: FILES.length,
-                    elapsed: formatElapsedTime(startTime),
-                    remaining: estimateRemaining(startTime, index + (downloadedLength / totalLength), FILES.length),
-                    rate: calculateRate(startTime, index + (downloadedLength / totalLength)),
-                    percentage: progress.toFixed(1),
-                    file_progress: {
-                        name: file.name,
-                        downloaded: downloadedLength,
-                        total: totalLength,
-                        percentage: progress.toFixed(1)
-                    }
-                });
-                lastProgressUpdate = now;
-            }
-        });
-
-        response.data.pipe(writer);
-
-        return new Promise((resolve, reject) => {
-            writer.on('finish', resolve);
-            writer.on('error', reject);
-        });
-    } catch (error) {
-        fs.unlinkSync(path.join(csvDir, file.name));
-        throw error;
-    }
-}
-
-// Main function to update all files
-async function updateFiles() {
-    const startTime = Date.now();
-    
-    outputProgress({
-        status: 'running',
-        operation: 'Starting CSV update',
-        current: 0,
-        total: FILES.length,
-        elapsed: '0s',
-        remaining: null,
-        rate: 0,
-        percentage: '0'
-    });
-
-    try {
-        for (let i = 0; i < FILES.length; i++) {
-            if (isCancelled) {
-                return;
-            }
-
-            const file = FILES[i];
-            await downloadFile(file, i, startTime);
-
-            outputProgress({
-                status: 'running',
-                operation: 'CSV update in progress',
-                current: i + 1,
-                total: FILES.length,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, i + 1, FILES.length),
-                rate: calculateRate(startTime, i + 1),
-                percentage: (((i + 1) / FILES.length) * 100).toFixed(1)
-            });
-        }
-
-        outputProgress({
-            status: 'complete',
-            operation: 'CSV update complete',
-            current: FILES.length,
-            total: FILES.length,
-            elapsed: formatElapsedTime(startTime),
-            remaining: '0s',
-            rate: calculateRate(startTime, FILES.length),
-            percentage: '100'
-        });
-    } catch (error) {
-        outputProgress({
-            status: 'error',
-            operation: 'CSV update failed',
-            error: error.message,
-            current: 0,
-            total: FILES.length,
-            elapsed: formatElapsedTime(startTime),
-            remaining: null,
-            rate: 0
-        });
-        throw error;
-    }
-}
-
-// Run the update only if this is the main module
-if (require.main === module) {
-    updateFiles().catch((error) => {
-        console.error('Error updating CSV files:', error);
-        process.exit(1);
-    });
-}
-
-// Export the functions needed by the route
-module.exports = {
-    updateFiles,
-    cancelUpdate
-}; 

inventory-server/old/scripts.js

@@ -1,180 +0,0 @@
-const readline = require('readline');
-
-const rl = readline.createInterface({
-  input: process.stdin,
-  output: process.stdout
-});
-
-const question = (query) => new Promise((resolve) => rl.question(query, resolve));
-
-async function loadScript(name) {
-  try {
-    return await require(name);
-  } catch (error) {
-    console.error(`Failed to load script ${name}:`, error);
-    return null;
-  }
-}
-
-async function runWithTimeout(fn) {
-  return new Promise((resolve, reject) => {
-    // Create a child process for the script
-    const child = require('child_process').fork(fn.toString(), [], {
-      stdio: 'inherit'
-    });
-
-    child.on('exit', (code) => {
-      if (code === 0) {
-        resolve();
-      } else {
-        reject(new Error(`Script exited with code ${code}`));
-      }
-    });
-
-    child.on('error', (err) => {
-      reject(err);
-    });
-  });
-}
-
-function clearScreen() {
-  process.stdout.write('\x1Bc');
-}
-
-const scripts = {
-  'Import Scripts': {
-    '1': { name: 'Full Import From Production', path: './import-from-prod' },
-    '2': { name: 'Individual Import Scripts ▸', submenu: {
-      '1': { name: 'Import Orders', path: './import/orders', key: 'importOrders' },
-      '2': { name: 'Import Products', path: './import/products', key: 'importProducts' },
-      '3': { name: 'Import Purchase Orders', path: './import/purchase-orders' },
-      '4': { name: 'Import Categories', path: './import/categories' },
-      'b': { name: 'Back to Main Menu' }
-    }}
-  },
-  'Metrics': {
-    '3': { name: 'Calculate All Metrics', path: './calculate-metrics' },
-    '4': { name: 'Individual Metric Scripts ▸', submenu: {
-      '1': { name: 'Brand Metrics', path: './metrics/brand-metrics' },
-      '2': { name: 'Category Metrics', path: './metrics/category-metrics' },
-      '3': { name: 'Financial Metrics', path: './metrics/financial-metrics' },
-      '4': { name: 'Product Metrics', path: './metrics/product-metrics' },
-      '5': { name: 'Sales Forecasts', path: './metrics/sales-forecasts' },
-      '6': { name: 'Time Aggregates', path: './metrics/time-aggregates' },
-      '7': { name: 'Vendor Metrics', path: './metrics/vendor-metrics' },
-      'b': { name: 'Back to Main Menu' }
-    }}
-  },
-  'Database Management': {
-    '5': { name: 'Test Production Connection', path: './test-prod-connection' }
-  },
-  'Reset Scripts': {
-    '6': { name: 'Reset Database', path: './reset-db' },
-    '7': { name: 'Reset Metrics', path: './reset-metrics' }
-  }
-};
-
-let lastRun = null;
-
-async function displayMenu(menuItems, title = 'Inventory Management Script Runner') {
-  clearScreen();
-  console.log(`\n${title}\n`);
-  
-  for (const [category, items] of Object.entries(menuItems)) {
-    console.log(`\n${category}:`);
-    Object.entries(items).forEach(([key, script]) => {
-      console.log(`${key}. ${script.name}`);
-    });
-  }
-
-  if (lastRun) {
-    console.log('\nQuick Access:');
-    console.log(`r. Repeat Last Script (${lastRun.name})`);
-  }
-  
-  console.log('\nq. Quit\n');
-}
-
-async function handleSubmenu(submenu, title) {
-  while (true) {
-    await displayMenu({"Individual Scripts": submenu}, title);
-    const choice = await question('Select an option (or b to go back): ');
-    
-    if (choice.toLowerCase() === 'b') {
-      return null;
-    }
-
-    if (submenu[choice]) {
-      return submenu[choice];
-    }
-
-    console.log('Invalid selection. Please try again.');
-    await new Promise(resolve => setTimeout(resolve, 1000));
-  }
-}
-
-async function runScript(script) {
-  console.log(`\nRunning: ${script.name}`);
-  try {
-    const scriptPath = require.resolve(script.path);
-    await runWithTimeout(scriptPath);
-    console.log('\nScript completed successfully');
-    lastRun = script;
-  } catch (error) {
-    console.error('\nError running script:', error);
-  }
-  await question('\nPress Enter to continue...');
-}
-
-async function main() {
-  while (true) {
-    await displayMenu(scripts);
-    
-    const choice = await question('Select an option: ');
-    
-    if (choice.toLowerCase() === 'q') {
-      break;
-    }
-
-    if (choice.toLowerCase() === 'r' && lastRun) {
-      await runScript(lastRun);
-      continue;
-    }
-
-    let selectedScript = null;
-    for (const category of Object.values(scripts)) {
-      if (category[choice]) {
-        selectedScript = category[choice];
-        break;
-      }
-    }
-
-    if (!selectedScript) {
-      console.log('Invalid selection. Please try again.');
-      await new Promise(resolve => setTimeout(resolve, 1000));
-      continue;
-    }
-
-    if (selectedScript.submenu) {
-      const submenuChoice = await handleSubmenu(
-        selectedScript.submenu,
-        selectedScript.name
-      );
-      if (submenuChoice && submenuChoice.path) {
-        await runScript(submenuChoice);
-      }
-    } else if (selectedScript.path) {
-      await runScript(selectedScript);
-    }
-  }
-
-  rl.close();
-  process.exit(0);
-}
-
-if (require.main === module) {
-  main().catch(error => {
-    console.error('Fatal error:', error);
-    process.exit(1);
-  });
-} 

inventory-server/old/test-connection.js

@@ -1,22 +0,0 @@
-const express = require('express');
-const router = express.Router();
-const { testConnection } = require('../../scripts/test-prod-connection');
-
-router.get('/test-prod-connection', async (req, res) => {
-  try {
-    const productCount = await testConnection();
-    res.json({ 
-      success: true, 
-      message: 'Successfully connected to production database',
-      productCount 
-    });
-  } catch (error) {
-    console.error('Production connection test failed:', error);
-    res.status(500).json({ 
-      success: false, 
-      error: error.message || 'Failed to connect to production database' 
-    });
-  }
-});
-
-module.exports = router; 

inventory-server/old/test-prod-connection.js

@@ -1,89 +0,0 @@
-const mysql = require('mysql2/promise');
-const { Client } = require('ssh2');
-const dotenv = require('dotenv');
-const path = require('path');
-
-dotenv.config({ path: path.join(__dirname, '../.env') });
-
-// SSH configuration
-const sshConfig = {
-  host: process.env.PROD_SSH_HOST,
-  port: process.env.PROD_SSH_PORT || 22,
-  username: process.env.PROD_SSH_USER,
-  privateKey: process.env.PROD_SSH_KEY_PATH ? require('fs').readFileSync(process.env.PROD_SSH_KEY_PATH) : undefined
-};
-
-// Database configuration
-const dbConfig = {
-  host: process.env.PROD_DB_HOST || 'localhost', // Usually localhost when tunneling
-  user: process.env.PROD_DB_USER,
-  password: process.env.PROD_DB_PASSWORD,
-  database: process.env.PROD_DB_NAME,
-  port: process.env.PROD_DB_PORT || 3306
-};
-
-async function testConnection() {
-  const ssh = new Client();
-
-  try {
-    // Create new Promise for SSH connection
-    await new Promise((resolve, reject) => {
-      ssh.on('ready', resolve)
-         .on('error', reject)
-         .connect(sshConfig);
-    });
-
-    console.log('SSH Connection successful!');
-
-    // Forward local port to remote MySQL port
-    const tunnel = await new Promise((resolve, reject) => {
-      ssh.forwardOut(
-        '127.0.0.1',
-        0,
-        dbConfig.host,
-        dbConfig.port,
-        (err, stream) => {
-          if (err) reject(err);
-          resolve(stream);
-        }
-      );
-    });
-
-    console.log('Port forwarding established');
-
-    // Create MySQL connection over SSH tunnel
-    const connection = await mysql.createConnection({
-      ...dbConfig,
-      stream: tunnel
-    });
-
-    console.log('MySQL Connection successful!');
-
-    // Test query
-    const [rows] = await connection.query('SELECT COUNT(*) as count FROM products');
-    console.log('Query successful! Product count:', rows[0].count);
-
-    // Clean up
-    await connection.end();
-    ssh.end();
-    console.log('Connections closed successfully');
-    return rows[0].count;
-
-  } catch (error) {
-    console.error('Error:', error);
-    if (ssh) ssh.end();
-    throw error;
-  }
-}
-
-// If running directly (not imported)
-if (require.main === module) {
-  testConnection()
-    .then(() => process.exit(0))
-    .catch(error => {
-      console.error('Test failed:', error);
-      process.exit(1);
-    });
-}
-
-module.exports = { testConnection }; 

inventory-server/package.json

@@ -1,40 +0,0 @@
-{
-  "name": "inventory-server",
-  "version": "1.0.0",
-  "description": "Backend server for inventory management system",
-  "main": "src/server.js",
-  "scripts": {
-    "start": "node src/server.js",
-    "dev": "nodemon src/server.js",
-    "prod": "pm2 start ecosystem.config.js",
-    "prod:stop": "pm2 stop inventory-server",
-    "prod:restart": "pm2 restart inventory-server",
-    "prod:logs": "pm2 logs inventory-server",
-    "prod:status": "pm2 status inventory-server",
-    "setup": "mkdir -p logs uploads",
-    "test": "echo \"Error: no test specified\" && exit 1"
-  },
-  "keywords": [],
-  "author": "",
-  "license": "ISC",
-  "dependencies": {
-    "@types/diff": "^7.0.1",
-    "axios": "^1.8.1",
-    "bcrypt": "^5.1.1",
-    "cors": "^2.8.5",
-    "csv-parse": "^5.6.0",
-    "diff": "^7.0.0",
-    "dotenv": "^16.4.7",
-    "express": "^4.18.2",
-    "multer": "^1.4.5-lts.1",
-    "mysql2": "^3.12.0",
-    "openai": "^4.85.3",
-    "pg": "^8.13.3",
-    "pm2": "^5.3.0",
-    "ssh2": "^1.16.0",
-    "uuid": "^9.0.1"
-  },
-  "devDependencies": {
-    "nodemon": "^3.0.2"
-  }
-}

inventory-server/scripts/calculate-metrics.js

@@ -1,710 +0,0 @@
-const path = require('path');
-
-// Change working directory to script directory
-process.chdir(path.dirname(__filename));
-
-require('dotenv').config({ path: path.resolve(__dirname, '..', '.env') });
-
-// Configuration flags for controlling which metrics to calculate
-// Set to 1 to skip the corresponding calculation, 0 to run it
-const SKIP_PRODUCT_METRICS = 0;
-const SKIP_TIME_AGGREGATES = 0;
-const SKIP_FINANCIAL_METRICS = 0;
-const SKIP_VENDOR_METRICS = 0;
-const SKIP_CATEGORY_METRICS = 0;
-const SKIP_BRAND_METRICS = 0;
-const SKIP_SALES_FORECASTS = 0;
-
-// Add error handler for uncaught exceptions
-process.on('uncaughtException', (error) => {
-    console.error('Uncaught Exception:', error);
-    process.exit(1);
-});
-
-// Add error handler for unhandled promise rejections
-process.on('unhandledRejection', (reason, promise) => {
-    console.error('Unhandled Rejection at:', promise, 'reason:', reason);
-    process.exit(1);
-});
-
-const progress = require('./metrics/utils/progress');
-console.log('Progress module loaded:', {
-    modulePath: require.resolve('./metrics/utils/progress'),
-    exports: Object.keys(progress),
-    currentDir: process.cwd(),
-    scriptDir: __dirname
-});
-
-// Store progress functions in global scope to ensure availability
-global.formatElapsedTime = progress.formatElapsedTime;
-global.estimateRemaining = progress.estimateRemaining;
-global.calculateRate = progress.calculateRate;
-global.outputProgress = progress.outputProgress;
-global.clearProgress = progress.clearProgress;
-global.getProgress = progress.getProgress;
-global.logError = progress.logError;
-
-// List of temporary tables used in the calculation process
-const TEMP_TABLES = [
-    'temp_revenue_ranks',
-    'temp_sales_metrics',
-    'temp_purchase_metrics',
-    'temp_product_metrics',
-    'temp_vendor_metrics',
-    'temp_category_metrics',
-    'temp_brand_metrics',
-    'temp_forecast_dates',
-    'temp_daily_sales',
-    'temp_product_stats',
-    'temp_category_sales',
-    'temp_category_stats'
-];
-
-// Add cleanup function for temporary tables
-async function cleanupTemporaryTables(connection) {
-    try {
-        for (const table of TEMP_TABLES) {
-            await connection.query(`DROP TEMPORARY TABLE IF EXISTS ${table}`);
-        }
-    } catch (error) {
-        logError(error, 'Error cleaning up temporary tables');
-        throw error; // Re-throw to be handled by the caller
-    }
-}
-
-const { getConnection, closePool } = require('./metrics/utils/db');
-const calculateProductMetrics = require('./metrics/product-metrics');
-const calculateTimeAggregates = require('./metrics/time-aggregates');
-const calculateFinancialMetrics = require('./metrics/financial-metrics');
-const calculateVendorMetrics = require('./metrics/vendor-metrics');
-const calculateCategoryMetrics = require('./metrics/category-metrics');
-const calculateBrandMetrics = require('./metrics/brand-metrics');
-const calculateSalesForecasts = require('./metrics/sales-forecasts');
-
-// Add cancel handler
-let isCancelled = false;
-
-function cancelCalculation() {
-    isCancelled = true;
-    global.clearProgress();
-    // Format as SSE event
-    const event = {
-        progress: {
-            status: 'cancelled',
-            operation: 'Calculation cancelled',
-            current: 0,
-            total: 0,
-            elapsed: null,
-            remaining: null,
-            rate: 0,
-            timestamp: Date.now()
-        }
-    };
-    process.stdout.write(JSON.stringify(event) + '\n');
-    process.exit(0);
-}
-
-// Handle SIGTERM signal for cancellation
-process.on('SIGTERM', cancelCalculation);
-
-// Update the main calculation function to use the new modular structure
-async function calculateMetrics() {
-    let connection;
-    const startTime = Date.now();
-    let processedProducts = 0;
-    let processedOrders = 0;
-    let processedPurchaseOrders = 0;
-    let totalProducts = 0;
-    let totalOrders = 0;
-    let totalPurchaseOrders = 0;
-    let calculateHistoryId;
-    
-    try {
-        // Clean up any previously running calculations
-        connection = await getConnection();
-        await connection.query(`
-            UPDATE calculate_history 
-            SET 
-                status = 'cancelled',
-                end_time = NOW(),
-                duration_seconds = TIMESTAMPDIFF(SECOND, start_time, NOW()),
-                error_message = 'Previous calculation was not completed properly'
-            WHERE status = 'running'
-        `);
-
-        // Get counts from all relevant tables
-        const [[productCount], [orderCount], [poCount]] = await Promise.all([
-            connection.query('SELECT COUNT(*) as total FROM products'),
-            connection.query('SELECT COUNT(*) as total FROM orders'),
-            connection.query('SELECT COUNT(*) as total FROM purchase_orders')
-        ]);
-
-        totalProducts = productCount.total;
-        totalOrders = orderCount.total;
-        totalPurchaseOrders = poCount.total;
-
-        // Create history record for this calculation
-        const [historyResult] = await connection.query(`
-            INSERT INTO calculate_history (
-                start_time,
-                status,
-                total_products,
-                total_orders,
-                total_purchase_orders,
-                additional_info
-            ) VALUES (
-                NOW(),
-                'running',
-                ?,
-                ?,
-                ?,
-                JSON_OBJECT(
-                    'skip_product_metrics', ?,
-                    'skip_time_aggregates', ?,
-                    'skip_financial_metrics', ?,
-                    'skip_vendor_metrics', ?,
-                    'skip_category_metrics', ?,
-                    'skip_brand_metrics', ?,
-                    'skip_sales_forecasts', ?
-                )
-            )
-        `, [
-            totalProducts,
-            totalOrders,
-            totalPurchaseOrders,
-            SKIP_PRODUCT_METRICS,
-            SKIP_TIME_AGGREGATES,
-            SKIP_FINANCIAL_METRICS, 
-            SKIP_VENDOR_METRICS,
-            SKIP_CATEGORY_METRICS,
-            SKIP_BRAND_METRICS,
-            SKIP_SALES_FORECASTS
-        ]);
-        calculateHistoryId = historyResult.insertId;
-        connection.release();
-
-        // Add debug logging for the progress functions
-        console.log('Debug - Progress functions:', {
-            formatElapsedTime: typeof global.formatElapsedTime,
-            estimateRemaining: typeof global.estimateRemaining,
-            calculateRate: typeof global.calculateRate,
-            startTime: startTime
-        });
-
-        try {
-            const elapsed = global.formatElapsedTime(startTime);
-            console.log('Debug - formatElapsedTime test successful:', elapsed);
-        } catch (err) {
-            console.error('Debug - Error testing formatElapsedTime:', err);
-            throw err;
-        }
-
-        isCancelled = false;
-        connection = await getConnection();
-
-        try {
-            global.outputProgress({
-                status: 'running',
-                operation: 'Starting metrics calculation',
-                current: 0,
-                total: 100,
-                elapsed: '0s',
-                remaining: 'Calculating...',
-                rate: 0,
-                percentage: '0',
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            // Update progress periodically
-            const updateProgress = async (products = null, orders = null, purchaseOrders = null) => {
-                // Ensure all values are valid numbers or default to previous value
-                if (products !== null) processedProducts = Number(products) || processedProducts || 0;
-                if (orders !== null) processedOrders = Number(orders) || processedOrders || 0;
-                if (purchaseOrders !== null) processedPurchaseOrders = Number(purchaseOrders) || processedPurchaseOrders || 0;
-
-                // Ensure we never send NaN to the database
-                const safeProducts = Number(processedProducts) || 0;
-                const safeOrders = Number(processedOrders) || 0;
-                const safePurchaseOrders = Number(processedPurchaseOrders) || 0;
-
-                await connection.query(`
-                    UPDATE calculate_history 
-                    SET 
-                        processed_products = ?,
-                        processed_orders = ?,
-                        processed_purchase_orders = ?
-                    WHERE id = ?
-                `, [safeProducts, safeOrders, safePurchaseOrders, calculateHistoryId]);
-            };
-
-            // Helper function to ensure valid progress numbers
-            const ensureValidProgress = (current, total) => ({
-                current: Number(current) || 0,
-                total: Number(total) || 1,  // Default to 1 to avoid division by zero
-                percentage: (((Number(current) || 0) / (Number(total) || 1)) * 100).toFixed(1)
-            });
-
-            // Initial progress
-            const initialProgress = ensureValidProgress(0, totalProducts);
-            global.outputProgress({
-                status: 'running',
-                operation: 'Starting metrics calculation',
-                current: initialProgress.current,
-                total: initialProgress.total,
-                elapsed: '0s',
-                remaining: 'Calculating...',
-                rate: 0,
-                percentage: initialProgress.percentage,
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            if (!SKIP_PRODUCT_METRICS) {
-                const result = await calculateProductMetrics(startTime, totalProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Product metrics calculation failed');
-                }
-            } else {
-                console.log('Skipping product metrics calculation...');
-                processedProducts = Math.floor(totalProducts * 0.6);
-                await updateProgress(processedProducts);
-                global.outputProgress({
-                    status: 'running',
-                    operation: 'Skipping product metrics calculation',
-                    current: processedProducts,
-                    total: totalProducts,
-                    elapsed: global.formatElapsedTime(startTime),
-                    remaining: global.estimateRemaining(startTime, processedProducts, totalProducts),
-                    rate: global.calculateRate(startTime, processedProducts),
-                    percentage: '60',
-                    timing: {
-                        start_time: new Date(startTime).toISOString(),
-                        end_time: new Date().toISOString(),
-                        elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                    }
-                });
-            }
-
-            // Calculate time-based aggregates
-            if (!SKIP_TIME_AGGREGATES) {
-                const result = await calculateTimeAggregates(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Time aggregates calculation failed');
-                }
-            } else {
-                console.log('Skipping time aggregates calculation');
-            }
-
-            // Calculate financial metrics
-            if (!SKIP_FINANCIAL_METRICS) {
-                const result = await calculateFinancialMetrics(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Financial metrics calculation failed');
-                }
-            } else {
-                console.log('Skipping financial metrics calculation');
-            }
-            
-            // Calculate vendor metrics
-            if (!SKIP_VENDOR_METRICS) {
-                const result = await calculateVendorMetrics(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Vendor metrics calculation failed');
-                }
-            } else {
-                console.log('Skipping vendor metrics calculation');
-            }
-
-            // Calculate category metrics
-            if (!SKIP_CATEGORY_METRICS) {
-                const result = await calculateCategoryMetrics(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Category metrics calculation failed');
-                }
-            } else {
-                console.log('Skipping category metrics calculation');
-            }
-
-            // Calculate brand metrics
-            if (!SKIP_BRAND_METRICS) {
-                const result = await calculateBrandMetrics(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Brand metrics calculation failed');
-                }
-            } else {
-                console.log('Skipping brand metrics calculation');
-            }
-
-            // Calculate sales forecasts
-            if (!SKIP_SALES_FORECASTS) {
-                const result = await calculateSalesForecasts(startTime, totalProducts, processedProducts);
-                await updateProgress(result.processedProducts, result.processedOrders, result.processedPurchaseOrders);
-                if (!result.success) {
-                    throw new Error('Sales forecasts calculation failed');
-                }
-            } else {
-                console.log('Skipping sales forecasts calculation');
-            }
-
-            // Calculate ABC classification
-            outputProgress({
-                status: 'running',
-                operation: 'Starting ABC classification',
-                current: processedProducts || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedProducts || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedProducts || 0),
-                percentage: (((processedProducts || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            if (isCancelled) return {
-                processedProducts: processedProducts || 0,
-                processedOrders: processedOrders || 0,
-                processedPurchaseOrders: 0,
-                success: false
-            };
-
-            const [abcConfig] = await connection.query('SELECT a_threshold, b_threshold FROM abc_classification_config WHERE id = 1');
-            const abcThresholds = abcConfig[0] || { a_threshold: 20, b_threshold: 50 };
-
-            // First, create and populate the rankings table with an index
-            await connection.query('DROP TEMPORARY TABLE IF EXISTS temp_revenue_ranks');
-            await connection.query(`
-                CREATE TEMPORARY TABLE temp_revenue_ranks (
-                    pid BIGINT NOT NULL,
-                    total_revenue DECIMAL(10,3),
-                    rank_num INT,
-                    total_count INT,
-                    PRIMARY KEY (pid),
-                    INDEX (rank_num)
-                ) ENGINE=MEMORY
-            `);
-
-            outputProgress({
-                status: 'running',
-                operation: 'Creating revenue rankings',
-                current: processedProducts || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedProducts || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedProducts || 0),
-                percentage: (((processedProducts || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            if (isCancelled) return {
-                processedProducts: processedProducts || 0,
-                processedOrders: processedOrders || 0,
-                processedPurchaseOrders: 0,
-                success: false
-            };
-
-            await connection.query(`
-                INSERT INTO temp_revenue_ranks
-                SELECT 
-                    pid,
-                    total_revenue,
-                    @rank := @rank + 1 as rank_num,
-                    @total_count := @rank as total_count
-                FROM (
-                    SELECT pid, total_revenue 
-                    FROM product_metrics 
-                    WHERE total_revenue > 0 
-                    ORDER BY total_revenue DESC
-                ) ranked, 
-                (SELECT @rank := 0) r
-            `);
-
-            // Get total count for percentage calculation
-            const [rankingCount] = await connection.query('SELECT MAX(rank_num) as total_count FROM temp_revenue_ranks');
-            const totalCount = rankingCount[0].total_count || 1;
-            const max_rank = totalCount; // Store max_rank for use in classification
-
-            outputProgress({
-                status: 'running',
-                operation: 'Updating ABC classifications',
-                current: processedProducts || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedProducts || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedProducts || 0),
-                percentage: (((processedProducts || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            if (isCancelled) return {
-                processedProducts: processedProducts || 0,
-                processedOrders: processedOrders || 0,
-                processedPurchaseOrders: 0,
-                success: false
-            };
-
-            // ABC classification progress tracking
-            let abcProcessedCount = 0;
-            const batchSize = 5000;
-            let lastProgressUpdate = Date.now();
-            const progressUpdateInterval = 1000; // Update every second
-
-            while (true) {
-                if (isCancelled) return {
-                    processedProducts: Number(processedProducts) || 0,
-                    processedOrders: Number(processedOrders) || 0,
-                    processedPurchaseOrders: 0,
-                    success: false
-                };
-
-                // First get a batch of PIDs that need updating
-                const [pids] = await connection.query(`
-                    SELECT pm.pid 
-                    FROM product_metrics pm
-                    LEFT JOIN temp_revenue_ranks tr ON pm.pid = tr.pid
-                    WHERE pm.abc_class IS NULL 
-                    OR pm.abc_class != 
-                        CASE 
-                            WHEN tr.rank_num IS NULL THEN 'C'
-                            WHEN (tr.rank_num / ?) * 100 <= ? THEN 'A'
-                            WHEN (tr.rank_num / ?) * 100 <= ? THEN 'B'
-                            ELSE 'C'
-                        END
-                    LIMIT ?
-                `, [max_rank, abcThresholds.a_threshold,
-                    max_rank, abcThresholds.b_threshold,
-                    batchSize]);
-
-                if (pids.length === 0) {
-                    break;
-                }
-
-                // Then update just those PIDs
-                const [result] = await connection.query(`
-                    UPDATE product_metrics pm
-                    LEFT JOIN temp_revenue_ranks tr ON pm.pid = tr.pid
-                    SET pm.abc_class = 
-                        CASE 
-                            WHEN tr.rank_num IS NULL THEN 'C'
-                            WHEN (tr.rank_num / ?) * 100 <= ? THEN 'A'
-                            WHEN (tr.rank_num / ?) * 100 <= ? THEN 'B'
-                            ELSE 'C'
-                        END,
-                        pm.last_calculated_at = NOW()
-                    WHERE pm.pid IN (?)
-                `, [max_rank, abcThresholds.a_threshold,
-                    max_rank, abcThresholds.b_threshold,
-                    pids.map(row => row.pid)]);
-
-                abcProcessedCount += result.affectedRows;
-                
-                // Calculate progress ensuring valid numbers
-                const currentProgress = Math.floor(totalProducts * (0.99 + (abcProcessedCount / (totalCount || 1)) * 0.01));
-                processedProducts = Number(currentProgress) || processedProducts || 0;
-
-                // Only update progress at most once per second
-                const now = Date.now();
-                if (now - lastProgressUpdate >= progressUpdateInterval) {
-                    const progress = ensureValidProgress(processedProducts, totalProducts);
-                    
-                    outputProgress({
-                        status: 'running',
-                        operation: 'ABC classification progress',
-                        current: progress.current,
-                        total: progress.total,
-                        elapsed: formatElapsedTime(startTime),
-                        remaining: estimateRemaining(startTime, progress.current, progress.total),
-                        rate: calculateRate(startTime, progress.current),
-                        percentage: progress.percentage,
-                        timing: {
-                            start_time: new Date(startTime).toISOString(),
-                            end_time: new Date().toISOString(),
-                            elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                        }
-                    });
-                    
-                    lastProgressUpdate = now;
-                }
-
-                // Update database progress
-                await updateProgress(processedProducts, processedOrders, processedPurchaseOrders);
-
-                // Small delay between batches to allow other transactions
-                await new Promise(resolve => setTimeout(resolve, 100));
-            }
-
-            // Clean up
-            await connection.query('DROP TEMPORARY TABLE IF EXISTS temp_revenue_ranks');
-
-            const endTime = Date.now();
-            const totalElapsedSeconds = Math.round((endTime - startTime) / 1000);
-
-            // Update calculate_status for ABC classification
-            await connection.query(`
-                INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-                VALUES ('abc_classification', NOW())
-                ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-            `);
-
-            // Final progress update with guaranteed valid numbers
-            const finalProgress = ensureValidProgress(totalProducts, totalProducts);
-            
-            // Final success message
-            outputProgress({
-                status: 'complete',
-                operation: 'Metrics calculation complete',
-                current: finalProgress.current,
-                total: finalProgress.total,
-                elapsed: formatElapsedTime(startTime),
-                remaining: '0s',
-                rate: calculateRate(startTime, finalProgress.current),
-                percentage: '100',
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: totalElapsedSeconds
-                }
-            });
-
-            // Ensure all values are valid numbers before final update
-            const finalStats = {
-                processedProducts: Number(processedProducts) || 0,
-                processedOrders: Number(processedOrders) || 0,
-                processedPurchaseOrders: Number(processedPurchaseOrders) || 0
-            };
-
-            // Update history with completion
-            await connection.query(`
-                UPDATE calculate_history 
-                SET 
-                    end_time = NOW(),
-                    duration_seconds = ?,
-                    processed_products = ?,
-                    processed_orders = ?,
-                    processed_purchase_orders = ?,
-                    status = 'completed'
-                WHERE id = ?
-            `, [totalElapsedSeconds, 
-                finalStats.processedProducts,
-                finalStats.processedOrders, 
-                finalStats.processedPurchaseOrders,
-                calculateHistoryId]);
-
-            // Clear progress file on successful completion
-            global.clearProgress();
-
-        } catch (error) {
-            const endTime = Date.now();
-            const totalElapsedSeconds = Math.round((endTime - startTime) / 1000);
-
-            // Update history with error
-            await connection.query(`
-                UPDATE calculate_history 
-                SET 
-                    end_time = NOW(),
-                    duration_seconds = ?,
-                    processed_products = ?,
-                    processed_orders = ?,
-                    processed_purchase_orders = ?,
-                    status = ?,
-                    error_message = ?
-                WHERE id = ?
-            `, [
-                totalElapsedSeconds,
-                processedProducts || 0,  // Ensure we have a valid number
-                processedOrders || 0,    // Ensure we have a valid number
-                processedPurchaseOrders || 0,  // Ensure we have a valid number
-                isCancelled ? 'cancelled' : 'failed',
-                error.message,
-                calculateHistoryId
-            ]);
-
-            if (isCancelled) {
-                global.outputProgress({
-                    status: 'cancelled',
-                    operation: 'Calculation cancelled',
-                    current: processedProducts,
-                    total: totalProducts || 0,
-                    elapsed: global.formatElapsedTime(startTime),
-                    remaining: null,
-                    rate: global.calculateRate(startTime, processedProducts),
-                    percentage: ((processedProducts / (totalProducts || 1)) * 100).toFixed(1),
-                    timing: {
-                        start_time: new Date(startTime).toISOString(),
-                        end_time: new Date().toISOString(),
-                        elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                    }
-                });
-            } else {
-                global.outputProgress({
-                    status: 'error',
-                    operation: 'Error: ' + error.message,
-                    current: processedProducts,
-                    total: totalProducts || 0,
-                    elapsed: global.formatElapsedTime(startTime),
-                    remaining: null,
-                    rate: global.calculateRate(startTime, processedProducts),
-                    percentage: ((processedProducts / (totalProducts || 1)) * 100).toFixed(1),
-                    timing: {
-                        start_time: new Date(startTime).toISOString(),
-                        end_time: new Date().toISOString(),
-                        elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                    }
-                });
-            }
-            throw error;
-        } finally {
-            if (connection) {
-                // Ensure temporary tables are cleaned up
-                await cleanupTemporaryTables(connection);
-                connection.release();
-            }
-            // Close the connection pool when we're done
-            await closePool();
-        }
-    } catch (error) {
-        success = false;
-        logError(error, 'Error in metrics calculation');
-        throw error;
-    }
-}
-
-// Export as a module with all necessary functions
-module.exports = {
-    calculateMetrics,
-    cancelCalculation,
-    getProgress: global.getProgress
-};
-
-// Run directly if called from command line
-if (require.main === module) {
-    calculateMetrics().catch(error => {
-        if (!error.message.includes('Operation cancelled')) {
-            console.error('Error:', error);
-        }
-        process.exit(1);
-    });
-} 

inventory-server/scripts/full-reset.js

@@ -1,115 +0,0 @@
-const path = require('path');
-const { spawn } = require('child_process');
-
-function outputProgress(data) {
-    if (!data.status) {
-        data = {
-            status: 'running',
-            ...data
-        };
-    }
-    console.log(JSON.stringify(data));
-}
-
-function runScript(scriptPath) {
-    return new Promise((resolve, reject) => {
-        const child = spawn('node', [scriptPath], {
-            stdio: ['inherit', 'pipe', 'pipe'],
-            env: {
-                ...process.env,
-                PGHOST: process.env.DB_HOST,
-                PGUSER: process.env.DB_USER,
-                PGPASSWORD: process.env.DB_PASSWORD,
-                PGDATABASE: process.env.DB_NAME,
-                PGPORT: process.env.DB_PORT || '5432'
-            }
-        });
-
-        let output = '';
-
-        child.stdout.on('data', (data) => {
-            const lines = data.toString().split('\n');
-            lines.filter(line => line.trim()).forEach(line => {
-                try {
-                    console.log(line); // Pass through the JSON output
-                    output += line + '\n';
-                } catch (e) {
-                    console.log(line); // If not JSON, just log it directly
-                }
-            });
-        });
-
-        child.stderr.on('data', (data) => {
-            console.error(data.toString());
-        });
-
-        child.on('close', (code) => {
-            if (code !== 0) {
-                reject(new Error(`Script ${scriptPath} exited with code ${code}`));
-            } else {
-                resolve(output);
-            }
-        });
-
-        child.on('error', (err) => {
-            reject(err);
-        });
-    });
-}
-
-async function fullReset() {
-    try {
-        // Step 1: Reset Database
-        outputProgress({
-            operation: 'Starting full reset',
-            message: 'Step 1/3: Resetting database...'
-        });
-        await runScript(path.join(__dirname, 'reset-db.js'));
-        outputProgress({
-            status: 'complete',
-            operation: 'Database reset step complete',
-            message: 'Database reset finished, moving to import...'
-        });
-
-        // Step 2: Import from Production
-        outputProgress({
-            operation: 'Starting import',
-            message: 'Step 2/3: Importing from production...'
-        });
-        await runScript(path.join(__dirname, 'import-from-prod.js'));
-        outputProgress({
-            status: 'complete',
-            operation: 'Import step complete',
-            message: 'Import finished, moving to metrics calculation...'
-        });
-
-        // Step 3: Calculate Metrics
-        outputProgress({
-            operation: 'Starting metrics calculation',
-            message: 'Step 3/3: Calculating metrics...'
-        });
-        await runScript(path.join(__dirname, 'calculate-metrics.js'));
-        
-        // Final completion message
-        outputProgress({
-            status: 'complete',
-            operation: 'Full reset complete',
-            message: 'Successfully completed all steps: database reset, import, and metrics calculation'
-        });
-    } catch (error) {
-        outputProgress({
-            status: 'error',
-            operation: 'Full reset failed',
-            error: error.message,
-            stack: error.stack
-        });
-        process.exit(1);
-    }
-}
-
-// Run if called directly
-if (require.main === module) {
-    fullReset();
-}
-
-module.exports = fullReset; 

inventory-server/scripts/full-update.js

@@ -1,100 +0,0 @@
-const path = require('path');
-const { spawn } = require('child_process');
-
-function outputProgress(data) {
-    if (!data.status) {
-        data = {
-            status: 'running',
-            ...data
-        };
-    }
-    console.log(JSON.stringify(data));
-}
-
-function runScript(scriptPath) {
-    return new Promise((resolve, reject) => {
-        const child = spawn('node', [scriptPath], {
-            stdio: ['inherit', 'pipe', 'pipe']
-        });
-
-        let output = '';
-
-        child.stdout.on('data', (data) => {
-            const lines = data.toString().split('\n');
-            lines.filter(line => line.trim()).forEach(line => {
-                try {
-                    console.log(line); // Pass through the JSON output
-                    output += line + '\n';
-                } catch (e) {
-                    console.log(line); // If not JSON, just log it directly
-                }
-            });
-        });
-
-        child.stderr.on('data', (data) => {
-            console.error(data.toString());
-        });
-
-        child.on('close', (code) => {
-            if (code !== 0) {
-                reject(new Error(`Script ${scriptPath} exited with code ${code}`));
-            } else {
-                resolve(output);
-            }
-        });
-
-        child.on('error', (err) => {
-            reject(err);
-        });
-    });
-}
-
-async function fullUpdate() {
-    try {
-        // Step 1: Import from Production
-        outputProgress({
-            operation: 'Starting full update',
-            message: 'Step 1/2: Importing from production...'
-        });
-        await runScript(path.join(__dirname, 'import-from-prod.js'));
-        outputProgress({
-            status: 'complete',
-            operation: 'Import step complete',
-            message: 'Import finished, moving to metrics calculation...'
-        });
-
-        // Step 2: Calculate Metrics
-        outputProgress({
-            operation: 'Starting metrics calculation',
-            message: 'Step 2/2: Calculating metrics...'
-        });
-        await runScript(path.join(__dirname, 'calculate-metrics.js'));
-        outputProgress({
-            status: 'complete',
-            operation: 'Metrics step complete',
-            message: 'Metrics calculation finished'
-        });
-
-        // Final completion message
-        outputProgress({
-            status: 'complete',
-            operation: 'Full update complete',
-            message: 'Successfully completed all steps: import and metrics calculation'
-        });
-    } catch (error) {
-        outputProgress({
-            status: 'error',
-            operation: 'Full update failed',
-            error: error.message,
-            stack: error.stack
-        });
-        process.exit(1);
-    }
-}
-
-// Run if called directly
-if (require.main === module) {
-    fullUpdate();
-}
-
-module.exports = fullUpdate; 

inventory-server/scripts/import-from-prod.js

@@ -1,311 +0,0 @@
-const dotenv = require("dotenv");
-const path = require("path");
-const { outputProgress, formatElapsedTime } = require('./metrics/utils/progress');
-const { setupConnections, closeConnections } = require('./import/utils');
-const importCategories = require('./import/categories');
-const { importProducts } = require('./import/products');
-const importOrders = require('./import/orders');
-const importPurchaseOrders = require('./import/purchase-orders');
-
-dotenv.config({ path: path.join(__dirname, "../.env") });
-
-// Constants to control which imports run
-const IMPORT_CATEGORIES = true;
-const IMPORT_PRODUCTS = true;
-const IMPORT_ORDERS = true;
-const IMPORT_PURCHASE_ORDERS = true;
-
-// Add flag for incremental updates
-const INCREMENTAL_UPDATE = process.env.INCREMENTAL_UPDATE !== 'false';  // Default to true unless explicitly set to false
-
-// SSH configuration
-const sshConfig = {
-  ssh: {
-    host: process.env.PROD_SSH_HOST,
-    port: process.env.PROD_SSH_PORT || 22,
-    username: process.env.PROD_SSH_USER,
-    privateKey: process.env.PROD_SSH_KEY_PATH
-      ? require("fs").readFileSync(process.env.PROD_SSH_KEY_PATH)
-      : undefined,
-    compress: true, // Enable SSH compression
-  },
-  prodDbConfig: {
-    // MySQL config for production
-    host: process.env.PROD_DB_HOST || "localhost",
-    user: process.env.PROD_DB_USER,
-    password: process.env.PROD_DB_PASSWORD,
-    database: process.env.PROD_DB_NAME,
-    port: process.env.PROD_DB_PORT || 3306,
-    timezone: 'Z',
-  },
-  localDbConfig: {
-    // PostgreSQL config for local
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    port: process.env.DB_PORT || 5432,
-    ssl: process.env.DB_SSL === 'true',
-    connectionTimeoutMillis: 60000,
-    idleTimeoutMillis: 30000,
-    max: 10 // connection pool max size
-  }
-};
-
-let isImportCancelled = false;
-
-// Add cancel function
-function cancelImport() {
-    isImportCancelled = true;
-    outputProgress({
-        status: 'cancelled',
-        operation: 'Import process',
-        message: 'Import cancelled by user',
-        current: 0,
-        total: 0,
-        elapsed: null,
-        remaining: null,
-        rate: 0
-    });
-}
-
-async function main() {
-  const startTime = Date.now();
-  let connections;
-  let completedSteps = 0;
-  let importHistoryId;
-  const totalSteps = [
-    IMPORT_CATEGORIES,
-    IMPORT_PRODUCTS,
-    IMPORT_ORDERS,
-    IMPORT_PURCHASE_ORDERS
-  ].filter(Boolean).length;
-
-  try {
-    // Initial progress update
-    outputProgress({
-      status: "running",
-      operation: "Import process",
-      message: `Initializing SSH tunnel for ${INCREMENTAL_UPDATE ? 'incremental' : 'full'} import...`,
-      current: completedSteps,
-      total: totalSteps,
-      elapsed: formatElapsedTime(startTime)
-    });
-
-    connections = await setupConnections(sshConfig);
-    const { prodConnection, localConnection } = connections;
-
-    if (isImportCancelled) throw new Error("Import cancelled");
-
-    // Clean up any previously running imports that weren't completed
-    await localConnection.query(`
-      UPDATE import_history 
-      SET 
-        status = 'cancelled',
-        end_time = NOW(),
-        duration_seconds = EXTRACT(EPOCH FROM (NOW() - start_time))::INTEGER,
-        error_message = 'Previous import was not completed properly'
-      WHERE status = 'running'
-    `);
-
-    // Initialize sync_status table if it doesn't exist
-    await localConnection.query(`
-      CREATE TABLE IF NOT EXISTS sync_status (
-        table_name VARCHAR(50) PRIMARY KEY,
-        last_sync_timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-        last_sync_id BIGINT
-      );
-
-      CREATE INDEX IF NOT EXISTS idx_last_sync ON sync_status (last_sync_timestamp);
-    `);
-
-    // Create import history record for the overall session
-    const [historyResult] = await localConnection.query(`
-      INSERT INTO import_history (
-        table_name, 
-        start_time, 
-        is_incremental,
-        status,
-        additional_info
-      ) VALUES (
-        'all_tables',
-        NOW(),
-        $1::boolean,
-        'running',
-        jsonb_build_object(
-          'categories_enabled', $2::boolean,
-          'products_enabled', $3::boolean,
-          'orders_enabled', $4::boolean,
-          'purchase_orders_enabled', $5::boolean
-        )
-      ) RETURNING id
-    `, [INCREMENTAL_UPDATE, IMPORT_CATEGORIES, IMPORT_PRODUCTS, IMPORT_ORDERS, IMPORT_PURCHASE_ORDERS]);
-    importHistoryId = historyResult.rows[0].id;
-
-    const results = {
-      categories: null,
-      products: null,
-      orders: null,
-      purchaseOrders: null
-    };
-
-    let totalRecordsAdded = 0;
-    let totalRecordsUpdated = 0;
-
-    // Run each import based on constants
-    if (IMPORT_CATEGORIES) {
-      results.categories = await importCategories(prodConnection, localConnection);
-      if (isImportCancelled) throw new Error("Import cancelled");
-      completedSteps++;
-      console.log('Categories import result:', results.categories);
-      totalRecordsAdded += parseInt(results.categories?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.categories?.recordsUpdated || 0);
-    }
-
-    if (IMPORT_PRODUCTS) {
-      results.products = await importProducts(prodConnection, localConnection, INCREMENTAL_UPDATE);
-      if (isImportCancelled) throw new Error("Import cancelled");
-      completedSteps++;
-      console.log('Products import result:', results.products);
-      totalRecordsAdded += parseInt(results.products?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.products?.recordsUpdated || 0);
-    }
-
-    if (IMPORT_ORDERS) {
-      results.orders = await importOrders(prodConnection, localConnection, INCREMENTAL_UPDATE);
-      if (isImportCancelled) throw new Error("Import cancelled");
-      completedSteps++;
-      console.log('Orders import result:', results.orders);
-      totalRecordsAdded += parseInt(results.orders?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.orders?.recordsUpdated || 0);
-    }
-
-    if (IMPORT_PURCHASE_ORDERS) {
-      results.purchaseOrders = await importPurchaseOrders(prodConnection, localConnection, INCREMENTAL_UPDATE);
-      if (isImportCancelled) throw new Error("Import cancelled");
-      completedSteps++;
-      console.log('Purchase orders import result:', results.purchaseOrders);
-      totalRecordsAdded += parseInt(results.purchaseOrders?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.purchaseOrders?.recordsUpdated || 0);
-    }
-
-    const endTime = Date.now();
-    const totalElapsedSeconds = Math.round((endTime - startTime) / 1000);
-
-    // Update import history with final stats
-    await localConnection.query(`
-      UPDATE import_history 
-      SET 
-        end_time = NOW(),
-        duration_seconds = $1,
-        records_added = $2,
-        records_updated = $3,
-        status = 'completed',
-        additional_info = jsonb_build_object(
-          'categories_enabled', $4::boolean,
-          'products_enabled', $5::boolean,
-          'orders_enabled', $6::boolean,
-          'purchase_orders_enabled', $7::boolean,
-          'categories_result', COALESCE($8::jsonb, 'null'::jsonb),
-          'products_result', COALESCE($9::jsonb, 'null'::jsonb),
-          'orders_result', COALESCE($10::jsonb, 'null'::jsonb),
-          'purchase_orders_result', COALESCE($11::jsonb, 'null'::jsonb)
-        )
-      WHERE id = $12
-    `, [
-      totalElapsedSeconds,
-      totalRecordsAdded,
-      totalRecordsUpdated,
-      IMPORT_CATEGORIES,
-      IMPORT_PRODUCTS,
-      IMPORT_ORDERS,
-      IMPORT_PURCHASE_ORDERS,
-      JSON.stringify(results.categories),
-      JSON.stringify(results.products),
-      JSON.stringify(results.orders),
-      JSON.stringify(results.purchaseOrders),
-      importHistoryId
-    ]);
-
-    outputProgress({
-      status: "complete",
-      operation: "Import process",
-      message: `${INCREMENTAL_UPDATE ? 'Incremental' : 'Full'} import completed successfully in ${formatElapsedTime(totalElapsedSeconds)}`,
-      current: completedSteps,
-      total: totalSteps,
-      elapsed: formatElapsedTime(startTime),
-      timing: {
-        start_time: new Date(startTime).toISOString(),
-        end_time: new Date(endTime).toISOString(),
-        elapsed_time: formatElapsedTime(startTime),
-        elapsed_seconds: totalElapsedSeconds,
-        total_duration: formatElapsedTime(totalElapsedSeconds)
-      },
-      results
-    });
-
-    return results;
-  } catch (error) {
-    const endTime = Date.now();
-    const totalElapsedSeconds = Math.round((endTime - startTime) / 1000);
-
-    // Update import history with error
-    if (importHistoryId && connections?.localConnection) {
-      await connections.localConnection.query(`
-        UPDATE import_history 
-        SET 
-          end_time = NOW(),
-          duration_seconds = $1,
-          status = $2,
-          error_message = $3
-        WHERE id = $4
-      `, [totalElapsedSeconds, error.message === "Import cancelled" ? 'cancelled' : 'failed', error.message, importHistoryId]);
-    }
-
-    console.error("Error during import process:", error);
-    outputProgress({
-      status: error.message === "Import cancelled" ? "cancelled" : "error",
-      operation: "Import process",
-      message: error.message === "Import cancelled" 
-        ? `${INCREMENTAL_UPDATE ? 'Incremental' : 'Full'} import cancelled by user after ${formatElapsedTime(totalElapsedSeconds)}`
-        : `${INCREMENTAL_UPDATE ? 'Incremental' : 'Full'} import failed after ${formatElapsedTime(totalElapsedSeconds)}`,
-      error: error.message,
-      current: completedSteps,
-      total: totalSteps,
-      elapsed: formatElapsedTime(startTime),
-      timing: {
-        start_time: new Date(startTime).toISOString(),
-        end_time: new Date(endTime).toISOString(),
-        elapsed_time: formatElapsedTime(startTime),
-        elapsed_seconds: totalElapsedSeconds,
-        total_duration: formatElapsedTime(totalElapsedSeconds)
-      }
-    });
-    throw error;
-  } finally {
-    if (connections) {
-      await closeConnections(connections).catch(err => {
-        console.error("Error closing connections:", err);
-      });
-    }
-  }
-}
-
-// Run the import only if this is the main module
-if (require.main === module) {
-  main().then((results) => {
-    console.log('Import completed successfully:', results);
-    // Force exit after a small delay to ensure all logs are written
-    setTimeout(() => process.exit(0), 500);
-  }).catch((error) => {
-    console.error("Unhandled error in main process:", error);
-    // Force exit with error code after a small delay
-    setTimeout(() => process.exit(1), 500);
-  });
-}
-
-// Export the functions needed by the route
-module.exports = {
-  main,
-  cancelImport,
-};

inventory-server/scripts/import/categories.js

@@ -1,218 +0,0 @@
-const { outputProgress, formatElapsedTime } = require('../metrics/utils/progress');
-
-async function importCategories(prodConnection, localConnection) {
-  outputProgress({
-    operation: "Starting categories import",
-    status: "running",
-  });
-
-  const startTime = Date.now();
-  const typeOrder = [10, 20, 11, 21, 12, 13];
-  let totalInserted = 0;
-  let totalUpdated = 0;
-  let skippedCategories = [];
-
-  try {
-    // Start a single transaction for the entire import
-    await localConnection.query('BEGIN');
-
-    // Process each type in order with its own savepoint
-    for (const type of typeOrder) {
-      try {
-        // Create a savepoint for this type
-        await localConnection.query(`SAVEPOINT category_type_${type}`);
-
-        // Production query remains MySQL compatible
-        const [categories] = await prodConnection.query(
-          `
-          SELECT 
-            pc.cat_id,
-            pc.name,
-            pc.type,
-            CASE 
-              WHEN pc.type IN (10, 20) THEN NULL  -- Top level categories should have no parent
-              WHEN pc.master_cat_id IS NULL THEN NULL
-              ELSE pc.master_cat_id
-            END as parent_id,
-            pc.combined_name as description
-          FROM product_categories pc
-          WHERE pc.type = ?
-          ORDER BY pc.cat_id
-        `,
-          [type]
-        );
-
-        if (categories.length === 0) {
-          await localConnection.query(`RELEASE SAVEPOINT category_type_${type}`);
-          continue;
-        }
-
-        console.log(`\nProcessing ${categories.length} type ${type} categories`);
-        if (type === 10) {
-          console.log("Type 10 categories:", JSON.stringify(categories, null, 2));
-        }
-
-        // For types that can have parents (11, 21, 12, 13), verify parent existence
-        let categoriesToInsert = categories;
-        if (![10, 20].includes(type)) {
-          // Get all parent IDs
-          const parentIds = [
-            ...new Set(
-              categories
-                .filter(c => c && c.parent_id !== null)
-                .map(c => c.parent_id)
-            ),
-          ];
-
-          console.log(`Processing ${categories.length} type ${type} categories with ${parentIds.length} unique parent IDs`);
-          console.log('Parent IDs:', parentIds);
-
-          // No need to check for parent existence - we trust they exist since they were just inserted
-          categoriesToInsert = categories;
-        }
-
-        if (categoriesToInsert.length === 0) {
-          console.log(
-            `No valid categories of type ${type} to insert`
-          );
-          await localConnection.query(`RELEASE SAVEPOINT category_type_${type}`);
-          continue;
-        }
-
-        console.log(
-          `Inserting ${categoriesToInsert.length} type ${type} categories`
-        );
-
-        // PostgreSQL upsert query with parameterized values
-        const values = categoriesToInsert.flatMap((cat) => [
-          cat.cat_id,
-          cat.name,
-          cat.type,
-          cat.parent_id,
-          cat.description,
-          'active',
-          new Date(),
-          new Date()
-        ]);
-
-        console.log('Attempting to insert/update with values:', JSON.stringify(values, null, 2));
-
-        const placeholders = categoriesToInsert
-          .map((_, i) => `($${i * 8 + 1}, $${i * 8 + 2}, $${i * 8 + 3}, $${i * 8 + 4}, $${i * 8 + 5}, $${i * 8 + 6}, $${i * 8 + 7}, $${i * 8 + 8})`)
-          .join(',');
-
-        console.log('Using placeholders:', placeholders);
-
-        // Insert categories with ON CONFLICT clause for PostgreSQL
-        const query = `
-          WITH inserted_categories AS (
-            INSERT INTO categories (
-              cat_id, name, type, parent_id, description, status, created_at, updated_at
-            )
-            VALUES ${placeholders}
-            ON CONFLICT (cat_id) DO UPDATE SET
-              name = EXCLUDED.name,
-              type = EXCLUDED.type,
-              parent_id = EXCLUDED.parent_id,
-              description = EXCLUDED.description,
-              status = EXCLUDED.status,
-              updated_at = EXCLUDED.updated_at
-            RETURNING 
-              cat_id,
-              CASE 
-                WHEN xmax = 0 THEN true
-                ELSE false
-              END as is_insert
-          )
-          SELECT 
-            COUNT(*) as total,
-            COUNT(*) FILTER (WHERE is_insert) as inserted,
-            COUNT(*) FILTER (WHERE NOT is_insert) as updated
-          FROM inserted_categories`;
-
-        console.log('Executing query:', query);
-        
-        const result = await localConnection.query(query, values);
-        console.log('Query result:', result);
-
-        // Get the first result since query returns an array
-        const queryResult = Array.isArray(result) ? result[0] : result;
-
-        if (!queryResult || !queryResult.rows || !queryResult.rows[0]) {
-          console.error('Query failed to return results. Result:', queryResult);
-          throw new Error('Query did not return expected results');
-        }
-
-        const total = parseInt(queryResult.rows[0].total) || 0;
-        const inserted = parseInt(queryResult.rows[0].inserted) || 0;
-        const updated = parseInt(queryResult.rows[0].updated) || 0;
-
-        console.log(`Total: ${total}, Inserted: ${inserted}, Updated: ${updated}`);
-        
-        totalInserted += inserted;
-        totalUpdated += updated;
-
-        // Release the savepoint for this type
-        await localConnection.query(`RELEASE SAVEPOINT category_type_${type}`);
-
-        outputProgress({
-          status: "running",
-          operation: "Categories import",
-          message: `Imported ${inserted} (updated ${updated}) categories of type ${type}`,
-          current: totalInserted + totalUpdated,
-          total: categories.length,
-          elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-        });
-      } catch (error) {
-        // Rollback to the savepoint for this type
-        await localConnection.query(`ROLLBACK TO SAVEPOINT category_type_${type}`);
-        throw error;
-      }
-    }
-
-    // Commit the entire transaction - we'll do this even if we have skipped categories
-    await localConnection.query('COMMIT');
-
-    outputProgress({
-      status: "complete",
-      operation: "Categories import completed",
-      current: totalInserted + totalUpdated,
-      total: totalInserted + totalUpdated,
-      duration: formatElapsedTime((Date.now() - startTime) / 1000),
-      warnings: skippedCategories.length > 0 ? {
-        message: "Some categories were skipped due to missing parents",
-        skippedCategories
-      } : undefined
-    });
-
-    return {
-      status: "complete",
-      recordsAdded: totalInserted,
-      recordsUpdated: totalUpdated,
-      totalRecords: totalInserted + totalUpdated,
-      warnings: skippedCategories.length > 0 ? {
-        message: "Some categories were skipped due to missing parents",
-        skippedCategories
-      } : undefined
-    };
-  } catch (error) {
-    console.error("Error importing categories:", error);
-    
-    // Only rollback if we haven't committed yet
-    try {
-      await localConnection.query('ROLLBACK');
-    } catch (rollbackError) {
-      console.error("Error during rollback:", rollbackError);
-    }
-    
-    outputProgress({
-      status: "error",
-      operation: "Categories import failed",
-      error: error.message
-    });
-
-    throw error;
-  }
-}
-
-module.exports = importCategories; 

inventory-server/scripts/import/orders.js

@@ -1,575 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate } = require('../metrics/utils/progress');
-const { importMissingProducts, setupTemporaryTables, cleanupTemporaryTables, materializeCalculations } = require('./products');
-
-/**
- * Imports orders from a production MySQL database to a local PostgreSQL database.
- * It can run in two modes:
- *   1. Incremental update mode (default): Only fetch orders that have changed since the last sync time.
- *   2. Full update mode: Fetch all eligible orders within the last 5 years regardless of timestamp.
- *
- * @param {object} prodConnection - A MySQL connection to production DB (MySQL 5.7).
- * @param {object} localConnection - A MySQL connection to local DB (MySQL 8.0).
- * @param {boolean} incrementalUpdate - Set to false for a full sync; true for incremental.
- *
- * @returns {object} Information about the sync operation.
- */
-async function importOrders(prodConnection, localConnection, incrementalUpdate = true) {
-  const startTime = Date.now();
-  const skippedOrders = new Set();
-  const missingProducts = new Set();
-  let recordsAdded = 0;
-  let recordsUpdated = 0;
-  let processedCount = 0;
-  let importedCount = 0;
-  let totalOrderItems = 0;
-  let totalUniqueOrders = 0;
-  let cumulativeProcessedOrders = 0;
-
-  try {
-    // Get last sync info
-    const [syncInfo] = await localConnection.query(
-      "SELECT last_sync_timestamp FROM sync_status WHERE table_name = 'orders'"
-    );
-    const lastSyncTime = syncInfo?.rows?.[0]?.last_sync_timestamp || '1970-01-01';
-
-    console.log('Orders: Using last sync time:', lastSyncTime);
-
-    // First get count of order items - Keep MySQL compatible for production
-    const [[{ total }]] = await prodConnection.query(`
-      SELECT COUNT(*) as total
-      FROM order_items oi
-      USE INDEX (PRIMARY)
-      JOIN _order o ON oi.order_id = o.order_id
-      WHERE o.order_status >= 15
-        AND o.date_placed_onlydate >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-        AND o.date_placed_onlydate IS NOT NULL
-        ${incrementalUpdate ? `
-          AND (
-            o.stamp > ? 
-            OR oi.stamp > ?
-            OR EXISTS (
-              SELECT 1 FROM order_discount_items odi 
-              WHERE odi.order_id = o.order_id 
-              AND odi.pid = oi.prod_pid
-            )
-            OR EXISTS (
-              SELECT 1 FROM order_tax_info oti 
-              JOIN order_tax_info_products otip ON oti.taxinfo_id = otip.taxinfo_id
-              WHERE oti.order_id = o.order_id 
-              AND otip.pid = oi.prod_pid
-              AND oti.stamp > ?
-            )
-          )
-        ` : ''}
-    `, incrementalUpdate ? [lastSyncTime, lastSyncTime, lastSyncTime] : []);
-
-    totalOrderItems = total;
-    console.log('Orders: Found changes:', totalOrderItems);
-
-    // Get order items - Keep MySQL compatible for production
-    console.log('Orders: Starting MySQL query...');
-    const [orderItems] = await prodConnection.query(`
-      SELECT 
-        oi.order_id,
-        oi.prod_pid,
-        COALESCE(NULLIF(TRIM(oi.prod_itemnumber), ''), 'NO-SKU') as SKU,
-        oi.prod_price as price,
-        oi.qty_ordered as quantity,
-        COALESCE(oi.prod_price_reg - oi.prod_price, 0) as base_discount,
-        oi.stamp as last_modified
-      FROM order_items oi
-      USE INDEX (PRIMARY)
-      JOIN _order o ON oi.order_id = o.order_id
-      WHERE o.order_status >= 15
-        AND o.date_placed_onlydate >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-        AND o.date_placed_onlydate IS NOT NULL
-        ${incrementalUpdate ? `
-          AND (
-            o.stamp > ? 
-            OR oi.stamp > ?
-            OR EXISTS (
-              SELECT 1 FROM order_discount_items odi 
-              WHERE odi.order_id = o.order_id 
-              AND odi.pid = oi.prod_pid
-            )
-            OR EXISTS (
-              SELECT 1 FROM order_tax_info oti 
-              JOIN order_tax_info_products otip ON oti.taxinfo_id = otip.taxinfo_id
-              WHERE oti.order_id = o.order_id 
-              AND otip.pid = oi.prod_pid
-              AND oti.stamp > ?
-            )
-          )
-        ` : ''}
-    `, incrementalUpdate ? [lastSyncTime, lastSyncTime, lastSyncTime] : []);
-
-    console.log('Orders: Found', orderItems.length, 'order items to process');
-
-    // Create tables in PostgreSQL for debugging
-    await localConnection.query(`
-      DROP TABLE IF EXISTS debug_order_items;
-      DROP TABLE IF EXISTS debug_order_meta;
-      DROP TABLE IF EXISTS debug_order_discounts;
-      DROP TABLE IF EXISTS debug_order_taxes;
-      DROP TABLE IF EXISTS debug_order_costs;
-
-      CREATE TABLE debug_order_items (
-        order_id INTEGER NOT NULL,
-        pid INTEGER NOT NULL,
-        SKU VARCHAR(50) NOT NULL,
-        price DECIMAL(10,2) NOT NULL,
-        quantity INTEGER NOT NULL,
-        base_discount DECIMAL(10,2) DEFAULT 0,
-        PRIMARY KEY (order_id, pid)
-      );
-
-      CREATE TABLE debug_order_meta (
-        order_id INTEGER NOT NULL,
-        date DATE NOT NULL,
-        customer VARCHAR(100) NOT NULL,
-        customer_name VARCHAR(150) NOT NULL,
-        status INTEGER,
-        canceled BOOLEAN,
-        summary_discount DECIMAL(10,2) DEFAULT 0.00,
-        summary_subtotal DECIMAL(10,2) DEFAULT 0.00,
-        PRIMARY KEY (order_id)
-      );
-
-      CREATE TABLE debug_order_discounts (
-        order_id INTEGER NOT NULL,
-        pid INTEGER NOT NULL,
-        discount DECIMAL(10,2) NOT NULL,
-        PRIMARY KEY (order_id, pid)
-      );
-
-      CREATE TABLE debug_order_taxes (
-        order_id INTEGER NOT NULL,
-        pid INTEGER NOT NULL,
-        tax DECIMAL(10,2) NOT NULL,
-        PRIMARY KEY (order_id, pid)
-      );
-
-      CREATE TABLE debug_order_costs (
-        order_id INTEGER NOT NULL,
-        pid INTEGER NOT NULL,
-        costeach DECIMAL(10,3) DEFAULT 0.000,
-        PRIMARY KEY (order_id, pid)
-      );
-    `);
-
-    // Insert order items in batches
-    for (let i = 0; i < orderItems.length; i += 5000) {
-      const batch = orderItems.slice(i, Math.min(i + 5000, orderItems.length));
-      const placeholders = batch.map((_, idx) => 
-        `($${idx * 6 + 1}, $${idx * 6 + 2}, $${idx * 6 + 3}, $${idx * 6 + 4}, $${idx * 6 + 5}, $${idx * 6 + 6})`
-      ).join(",");
-      const values = batch.flatMap(item => [
-        item.order_id, item.prod_pid, item.SKU, item.price, item.quantity, item.base_discount
-      ]);
-
-      await localConnection.query(`
-        INSERT INTO debug_order_items (order_id, pid, SKU, price, quantity, base_discount)
-        VALUES ${placeholders}
-        ON CONFLICT (order_id, pid) DO UPDATE SET
-          SKU = EXCLUDED.SKU,
-          price = EXCLUDED.price,
-          quantity = EXCLUDED.quantity,
-          base_discount = EXCLUDED.base_discount
-      `, values);
-
-      processedCount = i + batch.length;
-      outputProgress({
-        status: "running",
-        operation: "Orders import",
-        message: `Loading order items: ${processedCount} of ${totalOrderItems}`,
-        current: processedCount,
-        total: totalOrderItems,
-        elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-        remaining: estimateRemaining(startTime, processedCount, totalOrderItems),
-        rate: calculateRate(startTime, processedCount)
-      });
-    }
-
-    // Get unique order IDs
-    const orderIds = [...new Set(orderItems.map(item => item.order_id))];
-    totalUniqueOrders = orderIds.length;
-    console.log('Orders: Processing', totalUniqueOrders, 'unique orders');
-
-    // Reset processed count for order processing phase
-    processedCount = 0;
-
-    // Process metadata, discounts, taxes, and costs in parallel
-    const METADATA_BATCH_SIZE = 2000;
-    const PG_BATCH_SIZE = 200;
-    
-    const processMetadataBatch = async (batchIds) => {
-      const [orders] = await prodConnection.query(`
-        SELECT 
-          o.order_id,
-          o.date_placed_onlydate as date,
-          o.order_cid as customer,
-          CONCAT(COALESCE(u.firstname, ''), ' ', COALESCE(u.lastname, '')) as customer_name,
-          o.order_status as status,
-          CASE WHEN o.date_cancelled != '0000-00-00 00:00:00' THEN 1 ELSE 0 END as canceled,
-          o.summary_discount,
-          o.summary_subtotal
-        FROM _order o
-        LEFT JOIN users u ON o.order_cid = u.cid
-        WHERE o.order_id IN (?)
-      `, [batchIds]);
-
-      // Process in sub-batches for PostgreSQL
-      for (let j = 0; j < orders.length; j += PG_BATCH_SIZE) {
-        const subBatch = orders.slice(j, j + PG_BATCH_SIZE);
-        if (subBatch.length === 0) continue;
-
-        const placeholders = subBatch.map((_, idx) => 
-          `($${idx * 8 + 1}, $${idx * 8 + 2}, $${idx * 8 + 3}, $${idx * 8 + 4}, $${idx * 8 + 5}, $${idx * 8 + 6}, $${idx * 8 + 7}, $${idx * 8 + 8})`
-        ).join(",");
-        
-        const values = subBatch.flatMap(order => [
-          order.order_id,
-          order.date,
-          order.customer,
-          order.customer_name || '',
-          order.status,
-          order.canceled,
-          order.summary_discount || 0,
-          order.summary_subtotal || 0
-        ]);
-
-        await localConnection.query(`
-          INSERT INTO debug_order_meta (
-            order_id, date, customer, customer_name, status, canceled,
-            summary_discount, summary_subtotal
-          )
-          VALUES ${placeholders}
-          ON CONFLICT (order_id) DO UPDATE SET
-            date = EXCLUDED.date,
-            customer = EXCLUDED.customer,
-            customer_name = EXCLUDED.customer_name,
-            status = EXCLUDED.status,
-            canceled = EXCLUDED.canceled,
-            summary_discount = EXCLUDED.summary_discount,
-            summary_subtotal = EXCLUDED.summary_subtotal
-        `, values);
-      }
-    };
-
-    const processDiscountsBatch = async (batchIds) => {
-      const [discounts] = await prodConnection.query(`
-        SELECT order_id, pid, SUM(amount) as discount
-        FROM order_discount_items
-        WHERE order_id IN (?)
-        GROUP BY order_id, pid
-      `, [batchIds]);
-
-      if (discounts.length === 0) return;
-
-      for (let j = 0; j < discounts.length; j += PG_BATCH_SIZE) {
-        const subBatch = discounts.slice(j, j + PG_BATCH_SIZE);
-        if (subBatch.length === 0) continue;
-
-        const placeholders = subBatch.map((_, idx) => 
-          `($${idx * 3 + 1}, $${idx * 3 + 2}, $${idx * 3 + 3})`
-        ).join(",");
-        
-        const values = subBatch.flatMap(d => [
-          d.order_id,
-          d.pid,
-          d.discount || 0
-        ]);
-
-        await localConnection.query(`
-          INSERT INTO debug_order_discounts (order_id, pid, discount)
-          VALUES ${placeholders}
-          ON CONFLICT (order_id, pid) DO UPDATE SET
-            discount = EXCLUDED.discount
-        `, values);
-      }
-    };
-
-    const processTaxesBatch = async (batchIds) => {
-      // Optimized tax query to avoid subquery
-      const [taxes] = await prodConnection.query(`
-        SELECT oti.order_id, otip.pid, otip.item_taxes_to_collect as tax
-        FROM (
-          SELECT order_id, MAX(taxinfo_id) as latest_taxinfo_id
-          FROM order_tax_info
-          WHERE order_id IN (?)
-          GROUP BY order_id
-        ) latest_info
-        JOIN order_tax_info oti ON oti.order_id = latest_info.order_id 
-          AND oti.taxinfo_id = latest_info.latest_taxinfo_id
-        JOIN order_tax_info_products otip ON oti.taxinfo_id = otip.taxinfo_id
-      `, [batchIds]);
-
-      if (taxes.length === 0) return;
-
-      for (let j = 0; j < taxes.length; j += PG_BATCH_SIZE) {
-        const subBatch = taxes.slice(j, j + PG_BATCH_SIZE);
-        if (subBatch.length === 0) continue;
-
-        const placeholders = subBatch.map((_, idx) => 
-          `($${idx * 3 + 1}, $${idx * 3 + 2}, $${idx * 3 + 3})`
-        ).join(",");
-        
-        const values = subBatch.flatMap(t => [
-          t.order_id,
-          t.pid,
-          t.tax || 0
-        ]);
-
-        await localConnection.query(`
-          INSERT INTO debug_order_taxes (order_id, pid, tax)
-          VALUES ${placeholders}
-          ON CONFLICT (order_id, pid) DO UPDATE SET
-            tax = EXCLUDED.tax
-        `, values);
-      }
-    };
-
-    const processCostsBatch = async (batchIds) => {
-      const [costs] = await prodConnection.query(`
-        SELECT 
-          oc.orderid as order_id,
-          oc.pid,
-          oc.costeach
-        FROM order_costs oc
-        WHERE oc.orderid IN (?)
-          AND oc.pending = 0
-      `, [batchIds]);
-
-      if (costs.length === 0) return;
-
-      for (let j = 0; j < costs.length; j += PG_BATCH_SIZE) {
-        const subBatch = costs.slice(j, j + PG_BATCH_SIZE);
-        if (subBatch.length === 0) continue;
-
-        const placeholders = subBatch.map((_, idx) => 
-          `($${idx * 3 + 1}, $${idx * 3 + 2}, $${idx * 3 + 3})`
-        ).join(",");
-        
-        const values = subBatch.flatMap(c => [
-          c.order_id,
-          c.pid,
-          c.costeach || 0
-        ]);
-
-        await localConnection.query(`
-          INSERT INTO debug_order_costs (order_id, pid, costeach)
-          VALUES ${placeholders}
-          ON CONFLICT (order_id, pid) DO UPDATE SET
-            costeach = EXCLUDED.costeach
-        `, values);
-      }
-    };
-
-    // Process all data types in parallel for each batch
-    for (let i = 0; i < orderIds.length; i += METADATA_BATCH_SIZE) {
-      const batchIds = orderIds.slice(i, i + METADATA_BATCH_SIZE);
-      
-      await Promise.all([
-        processMetadataBatch(batchIds),
-        processDiscountsBatch(batchIds),
-        processTaxesBatch(batchIds),
-        processCostsBatch(batchIds)
-      ]);
-
-      processedCount = i + batchIds.length;
-      outputProgress({
-        status: "running",
-        operation: "Orders import",
-        message: `Loading order data: ${processedCount} of ${totalUniqueOrders}`,
-        current: processedCount,
-        total: totalUniqueOrders,
-        elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-        remaining: estimateRemaining(startTime, processedCount, totalUniqueOrders),
-        rate: calculateRate(startTime, processedCount)
-      });
-    }
-
-    // Pre-check all products at once
-    const allOrderPids = [...new Set(orderItems.map(item => item.prod_pid))];
-    console.log('Orders: Checking', allOrderPids.length, 'unique products');
-    
-    const [existingProducts] = allOrderPids.length > 0 ? await localConnection.query(
-      "SELECT pid FROM products WHERE pid = ANY($1::bigint[])",
-      [allOrderPids]
-    ) : [[]];
-    
-    const existingPids = new Set(existingProducts.rows.map(p => p.pid));
-    
-    // Process in smaller batches
-    for (let i = 0; i < orderIds.length; i += 1000) {
-      const batchIds = orderIds.slice(i, i + 1000);
-
-      // Get combined data for this batch in sub-batches
-      const PG_BATCH_SIZE = 100; // Process 100 records at a time
-      for (let j = 0; j < batchIds.length; j += PG_BATCH_SIZE) {
-        const subBatchIds = batchIds.slice(j, j + PG_BATCH_SIZE);
-        
-        const [orders] = await localConnection.query(`
-          WITH order_totals AS (
-            SELECT 
-              oi.order_id,
-              oi.pid,
-              SUM(COALESCE(od.discount, 0)) as promo_discount,
-              COALESCE(ot.tax, 0) as total_tax,
-              COALESCE(oi.price * 0.5, 0) as costeach
-            FROM debug_order_items oi
-            LEFT JOIN debug_order_discounts od ON oi.order_id = od.order_id AND oi.pid = od.pid
-            LEFT JOIN debug_order_taxes ot ON oi.order_id = ot.order_id AND oi.pid = ot.pid
-            GROUP BY oi.order_id, oi.pid, ot.tax
-          )
-          SELECT 
-            oi.order_id as order_number,
-            oi.pid::bigint as pid,
-            oi.SKU as sku,
-            om.date,
-            oi.price,
-            oi.quantity,
-            (oi.base_discount + 
-             COALESCE(ot.promo_discount, 0) + 
-             CASE 
-              WHEN om.summary_discount > 0 AND om.summary_subtotal > 0 THEN 
-                ROUND((om.summary_discount * (oi.price * oi.quantity)) / NULLIF(om.summary_subtotal, 0), 2)
-              ELSE 0 
-             END)::DECIMAL(10,2) as discount,
-            COALESCE(ot.total_tax, 0)::DECIMAL(10,2) as tax,
-            false as tax_included,
-            0 as shipping,
-            om.customer,
-            om.customer_name,
-            om.status,
-            om.canceled,
-            COALESCE(ot.costeach, oi.price * 0.5)::DECIMAL(10,3) as costeach
-          FROM (
-            SELECT DISTINCT ON (order_id, pid)
-              order_id, pid, SKU, price, quantity, base_discount
-            FROM debug_order_items
-            WHERE order_id = ANY($1)
-            ORDER BY order_id, pid
-          ) oi
-          JOIN debug_order_meta om ON oi.order_id = om.order_id
-          LEFT JOIN order_totals ot ON oi.order_id = ot.order_id AND oi.pid = ot.pid
-          ORDER BY oi.order_id, oi.pid
-        `, [subBatchIds]);
-
-        // Filter orders and track missing products
-        const validOrders = [];
-        const processedOrderItems = new Set();
-        const processedOrders = new Set();
-        
-        for (const order of orders.rows) {
-          if (!existingPids.has(order.pid)) {
-            missingProducts.add(order.pid);
-            skippedOrders.add(order.order_number);
-            continue;
-          }
-          validOrders.push(order);
-          processedOrderItems.add(`${order.order_number}-${order.pid}`);
-          processedOrders.add(order.order_number);
-        }
-
-        // Process valid orders in smaller sub-batches
-        const FINAL_BATCH_SIZE = 50;
-        for (let k = 0; k < validOrders.length; k += FINAL_BATCH_SIZE) {
-          const subBatch = validOrders.slice(k, k + FINAL_BATCH_SIZE);
-          
-          const placeholders = subBatch.map((_, idx) => {
-            const base = idx * 14;  // 14 columns (removed updated)
-            return `($${base + 1}, $${base + 2}, $${base + 3}, $${base + 4}, $${base + 5}, $${base + 6}, $${base + 7}, $${base + 8}, $${base + 9}, $${base + 10}, $${base + 11}, $${base + 12}, $${base + 13}, $${base + 14})`;
-          }).join(',');
-
-          const batchValues = subBatch.flatMap(o => [
-            o.order_number,
-            o.pid,
-            o.sku || 'NO-SKU',
-            o.date,
-            o.price,
-            o.quantity,
-            o.discount,
-            o.tax,
-            o.tax_included,
-            o.shipping,
-            o.customer,
-            o.customer_name,
-            o.status,
-            o.canceled
-          ]);
-
-          const [result] = await localConnection.query(`
-            WITH inserted_orders AS (
-              INSERT INTO orders (
-                order_number, pid, sku, date, price, quantity, discount,
-                tax, tax_included, shipping, customer, customer_name,
-                status, canceled
-              )
-              VALUES ${placeholders}
-              ON CONFLICT (order_number, pid) DO UPDATE SET
-                sku = EXCLUDED.sku,
-                date = EXCLUDED.date,
-                price = EXCLUDED.price,
-                quantity = EXCLUDED.quantity,
-                discount = EXCLUDED.discount,
-                tax = EXCLUDED.tax,
-                tax_included = EXCLUDED.tax_included,
-                shipping = EXCLUDED.shipping,
-                customer = EXCLUDED.customer,
-                customer_name = EXCLUDED.customer_name,
-                status = EXCLUDED.status,
-                canceled = EXCLUDED.canceled
-              RETURNING xmax = 0 as inserted
-            )
-            SELECT 
-              COUNT(*) FILTER (WHERE inserted) as inserted,
-              COUNT(*) FILTER (WHERE NOT inserted) as updated
-            FROM inserted_orders
-          `, batchValues);
-          
-          const { inserted, updated } = result.rows[0];
-          recordsAdded += inserted;
-          recordsUpdated += updated;
-          importedCount += subBatch.length;
-        }
-
-        cumulativeProcessedOrders += processedOrders.size;
-        outputProgress({
-          status: "running",
-          operation: "Orders import",
-          message: `Importing orders: ${cumulativeProcessedOrders} of ${totalUniqueOrders}`,
-          current: cumulativeProcessedOrders,
-          total: totalUniqueOrders,
-          elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-          remaining: estimateRemaining(startTime, cumulativeProcessedOrders, totalUniqueOrders),
-          rate: calculateRate(startTime, cumulativeProcessedOrders)
-        });
-      }
-    }
-
-    // Update sync status
-    await localConnection.query(`
-      INSERT INTO sync_status (table_name, last_sync_timestamp)
-      VALUES ('orders', NOW())
-      ON CONFLICT (table_name) DO UPDATE SET
-        last_sync_timestamp = NOW()
-    `);
-
-    return {
-      status: "complete",
-      totalImported: Math.floor(importedCount),
-      recordsAdded: recordsAdded || 0,
-      recordsUpdated: Math.floor(recordsUpdated),
-      totalSkipped: skippedOrders.size,
-      missingProducts: missingProducts.size,
-      incrementalUpdate,
-      lastSyncTime
-    };
-  } catch (error) {
-    console.error("Error during orders import:", error);
-    throw error;
-  }
-}
-
-module.exports = importOrders;

inventory-server/scripts/import/products.js

@@ -1,843 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate } = require('../metrics/utils/progress');
-const BATCH_SIZE = 100; // Smaller batch size for better progress tracking
-const MAX_RETRIES = 3;
-const RETRY_DELAY = 5000; // 5 seconds
-
-// Utility functions
-const imageUrlBase = 'https://sbing.com/i/products/0000/';
-const getImageUrls = (pid, iid = 1) => {
-  const paddedPid = pid.toString().padStart(6, '0');
-  // Use padded PID only for the first 3 digits
-  const prefix = paddedPid.slice(0, 3);
-  // Use the actual pid for the rest of the URL
-  const basePath = `${imageUrlBase}${prefix}/${pid}`;
-  return {
-    image: `${basePath}-t-${iid}.jpg`,
-    image_175: `${basePath}-175x175-${iid}.jpg`,
-    image_full: `${basePath}-o-${iid}.jpg`
-  };
-};
-
-// Add helper function for retrying operations
-async function withRetry(operation, errorMessage) {
-  let lastError;
-  for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      lastError = error;
-      console.error(`${errorMessage} (Attempt ${attempt}/${MAX_RETRIES}):`, error);
-      if (attempt < MAX_RETRIES) {
-        await new Promise(resolve => setTimeout(resolve, RETRY_DELAY));
-      }
-    }
-  }
-  throw lastError;
-}
-
-// Add helper function at the top of the file
-function validateDate(mysqlDate) {
-  if (!mysqlDate || mysqlDate === '0000-00-00' || mysqlDate === '0000-00-00 00:00:00') {
-    return null;
-  }
-  // Check if the date is valid
-  const date = new Date(mysqlDate);
-  return isNaN(date.getTime()) ? null : mysqlDate;
-}
-
-async function setupTemporaryTables(connection) {
-  // Drop the table if it exists
-  await connection.query('DROP TABLE IF EXISTS temp_products');
-
-  // Create the temporary table
-  await connection.query(`
-    CREATE TEMP TABLE temp_products (
-      pid BIGINT NOT NULL,
-      title VARCHAR(255),
-      description TEXT,
-      sku VARCHAR(50),
-      stock_quantity INTEGER DEFAULT 0,
-      preorder_count INTEGER DEFAULT 0,
-      notions_inv_count INTEGER DEFAULT 0,
-      price DECIMAL(10,3) NOT NULL DEFAULT 0,
-      regular_price DECIMAL(10,3) NOT NULL DEFAULT 0,
-      cost_price DECIMAL(10,3),
-      vendor VARCHAR(100),
-      vendor_reference VARCHAR(100),
-      notions_reference VARCHAR(100),
-      brand VARCHAR(100),
-      line VARCHAR(100),
-      subline VARCHAR(100),
-      artist VARCHAR(100),
-      categories TEXT,
-      created_at TIMESTAMP,
-      first_received TIMESTAMP,
-      landing_cost_price DECIMAL(10,3),
-      barcode VARCHAR(50),
-      harmonized_tariff_code VARCHAR(50),
-      updated_at TIMESTAMP,
-      visible BOOLEAN,
-      managing_stock BOOLEAN DEFAULT true,
-      replenishable BOOLEAN,
-      permalink VARCHAR(255),
-      moq INTEGER DEFAULT 1,
-      uom INTEGER DEFAULT 1,
-      rating DECIMAL(10,2),
-      reviews INTEGER,
-      weight DECIMAL(10,3),
-      length DECIMAL(10,3),
-      width DECIMAL(10,3),
-      height DECIMAL(10,3),
-      country_of_origin VARCHAR(100),
-      location VARCHAR(100),
-      total_sold INTEGER,
-      baskets INTEGER,
-      notifies INTEGER,
-      date_last_sold TIMESTAMP,
-      image VARCHAR(255),
-      image_175 VARCHAR(255),
-      image_full VARCHAR(255),
-      options TEXT,
-      tags TEXT,
-      needs_update BOOLEAN DEFAULT TRUE,
-      PRIMARY KEY (pid)
-    )`);
-
-  // Create the index
-  await connection.query('CREATE INDEX idx_temp_products_needs_update ON temp_products (needs_update)');
-}
-
-async function cleanupTemporaryTables(connection) {
-  await connection.query('DROP TABLE IF EXISTS temp_products');
-}
-
-async function importMissingProducts(prodConnection, localConnection, missingPids) {
-  if (!missingPids || missingPids.length === 0) {
-    return {
-      status: "complete",
-      recordsAdded: 0,
-      message: "No missing products to import"
-    };
-  }
-
-  try {
-    // Setup temporary tables
-    await setupTemporaryTables(localConnection);
-
-    // Get product data from production - Keep MySQL compatible
-    const [prodData] = await prodConnection.query(`
-      SELECT 
-        p.pid,
-        p.description AS title,
-        p.notes AS description,
-        p.itemnumber AS sku,
-        p.date_created,
-        p.datein AS first_received,
-        p.location,
-        p.upc AS barcode,
-        p.harmonized_tariff_code,
-        p.stamp AS updated_at,
-        CASE WHEN si.show + si.buyable > 0 THEN 1 ELSE 0 END AS visible,
-        CASE 
-          WHEN p.reorder < 0 THEN 0
-          WHEN (
-            (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
-            OR (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-            OR (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-          ) THEN 0
-          ELSE 1
-        END AS replenishable,
-        COALESCE(si.available_local, 0) as stock_quantity,
-        0 as pending_qty,
-        COALESCE(ci.onpreorder, 0) as preorder_count,
-        COALESCE(pnb.inventory, 0) as notions_inv_count,
-        COALESCE(pcp.price_each, 0) as price,
-        COALESCE(p.sellingprice, 0) AS regular_price,
-        CASE 
-          WHEN EXISTS (SELECT 1 FROM product_inventory WHERE pid = p.pid AND count > 0) 
-          THEN (SELECT ROUND(AVG(costeach), 5) FROM product_inventory WHERE pid = p.pid AND count > 0)
-          ELSE (SELECT costeach FROM product_inventory WHERE pid = p.pid ORDER BY daterec DESC LIMIT 1)
-        END AS cost_price,
-        NULL as landing_cost_price,
-        s.companyname AS vendor,
-        CASE 
-          WHEN s.companyname = 'Notions' THEN sid.notions_itemnumber 
-          ELSE sid.supplier_itemnumber 
-        END AS vendor_reference,
-        sid.notions_itemnumber AS notions_reference,
-        CONCAT('https://www.acherryontop.com/shop/product/', p.pid) AS permalink,
-        pc1.name AS brand,
-        pc2.name AS line,
-        pc3.name AS subline,
-        pc4.name AS artist,
-        COALESCE(CASE 
-          WHEN sid.supplier_id = 92 THEN sid.notions_qty_per_unit
-          ELSE sid.supplier_qty_per_unit
-        END, sid.notions_qty_per_unit) AS moq,
-        p.rating,
-        p.rating_votes AS reviews,
-        p.weight,
-        p.length,
-        p.width,
-        p.height,
-        p.country_of_origin,
-        (SELECT COUNT(*) FROM mybasket mb WHERE mb.item = p.pid AND mb.qty > 0) AS baskets,
-        (SELECT COUNT(*) FROM product_notify pn WHERE pn.pid = p.pid) AS notifies,
-        p.totalsold AS total_sold,
-        pls.date_sold as date_last_sold,
-        GROUP_CONCAT(DISTINCT CASE 
-          WHEN pc.cat_id IS NOT NULL 
-          AND pc.type IN (10, 20, 11, 21, 12, 13)
-          AND pci.cat_id NOT IN (16, 17)
-          THEN pci.cat_id 
-        END) as category_ids
-      FROM products p
-      LEFT JOIN shop_inventory si ON p.pid = si.pid AND si.store = 0
-      LEFT JOIN current_inventory ci ON p.pid = ci.pid
-      LEFT JOIN product_notions_b2b pnb ON p.pid = pnb.pid
-      LEFT JOIN product_current_prices pcp ON p.pid = pcp.pid AND pcp.active = 1
-      LEFT JOIN supplier_item_data sid ON p.pid = sid.pid
-      LEFT JOIN suppliers s ON sid.supplier_id = s.supplierid
-      LEFT JOIN product_category_index pci ON p.pid = pci.pid
-      LEFT JOIN product_categories pc ON pci.cat_id = pc.cat_id
-      LEFT JOIN product_categories pc1 ON p.company = pc1.cat_id
-      LEFT JOIN product_categories pc2 ON p.line = pc2.cat_id
-      LEFT JOIN product_categories pc3 ON p.subline = pc3.cat_id
-      LEFT JOIN product_categories pc4 ON p.artist = pc4.cat_id
-      LEFT JOIN product_last_sold pls ON p.pid = pls.pid
-      WHERE p.pid IN (?)
-      GROUP BY p.pid
-    `, [missingPids]);
-
-    if (!prodData || prodData.length === 0) {
-      return {
-        status: "complete",
-        recordsAdded: 0,
-        message: "No products found in production database"
-      };
-    }
-
-    // Process in batches
-    let recordsAdded = 0;
-    for (let i = 0; i < prodData.length; i += BATCH_SIZE) {
-      const batch = prodData.slice(i, i + BATCH_SIZE);
-      
-      const placeholders = batch.map((_, idx) => {
-        const base = idx * 47; // 47 columns
-        return `(${Array.from({ length: 47 }, (_, i) => `$${base + i + 1}`).join(', ')})`;
-      }).join(',');
-
-      const values = batch.flatMap(row => {
-        const imageUrls = getImageUrls(row.pid);
-        return [
-          row.pid,
-          row.title,
-          row.description,
-          row.itemnumber || '',
-          row.stock_quantity > 5000 ? 0 : Math.max(0, row.stock_quantity),
-          row.preorder_count,
-          row.notions_inv_count,
-          row.price,
-          row.regular_price,
-          row.cost_price,
-          row.vendor,
-          row.vendor_reference,
-          row.notions_reference,
-          row.brand,
-          row.line,
-          row.subline,
-          row.artist,
-          row.category_ids,
-          validateDate(row.date_created),
-          validateDate(row.first_received),
-          row.landing_cost_price,
-          row.barcode,
-          row.harmonized_tariff_code,
-          validateDate(row.updated_at),
-          row.visible,
-          true,
-          row.replenishable,
-          row.permalink,
-          Math.max(1, Math.round(row.moq || 1)),
-          1,
-          row.rating,
-          row.reviews,
-          row.weight,
-          row.length,
-          row.width,
-          row.height,
-          row.country_of_origin,
-          row.location,
-          row.total_sold,
-          row.baskets,
-          row.notifies,
-          validateDate(row.date_last_sold),
-          imageUrls.image,
-          imageUrls.image_175,
-          imageUrls.image_full,
-          null,
-          null
-        ];
-      });
-
-      const [result] = await localConnection.query(`
-        WITH inserted_products AS (
-          INSERT INTO products (
-            pid, title, description, sku, stock_quantity, preorder_count, notions_inv_count,
-            price, regular_price, cost_price, vendor, vendor_reference, notions_reference,
-            brand, line, subline, artist, categories, created_at, first_received,
-            landing_cost_price, barcode, harmonized_tariff_code, updated_at, visible,
-            managing_stock, replenishable, permalink, moq, uom, rating, reviews,
-            weight, length, width, height, country_of_origin, location, total_sold,
-            baskets, notifies, date_last_sold, image, image_175, image_full, options, tags
-          )
-          VALUES ${placeholders}
-          ON CONFLICT (pid) DO NOTHING
-          RETURNING pid
-        )
-        SELECT COUNT(*) as inserted FROM inserted_products
-      `, values);
-
-      recordsAdded += parseInt(result.rows[0].inserted, 10) || 0;
-    }
-
-    return {
-      status: "complete",
-      recordsAdded,
-      message: `Successfully imported ${recordsAdded} missing products`
-    };
-  } catch (error) {
-    console.error('Error importing missing products:', error);
-    throw error;
-  }
-}
-
-async function materializeCalculations(prodConnection, localConnection, incrementalUpdate = true, lastSyncTime = '1970-01-01', startTime = Date.now()) {
-  outputProgress({
-    status: "running",
-    operation: "Products import",
-    message: "Fetching product data from production"
-  });
-
-  // Get all product data in a single optimized query - Keep MySQL compatible
-  const [prodData] = await prodConnection.query(`
-    SELECT 
-      p.pid,
-      p.description AS title,
-      p.notes AS description,
-      p.itemnumber AS sku,
-      p.date_created,
-      p.datein AS first_received,
-      p.location,
-      p.upc AS barcode,
-      p.harmonized_tariff_code,
-      p.stamp AS updated_at,
-      CASE WHEN si.show + si.buyable > 0 THEN 1 ELSE 0 END AS visible,
-      CASE 
-        WHEN p.reorder < 0 THEN 0
-        WHEN (
-          (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
-          OR (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-          OR (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-        ) THEN 0
-        ELSE 1
-      END AS replenishable,
-      COALESCE(si.available_local, 0) as stock_quantity,
-      0 as pending_qty,
-      COALESCE(ci.onpreorder, 0) as preorder_count,
-      COALESCE(pnb.inventory, 0) as notions_inv_count,
-      COALESCE(pcp.price_each, 0) as price,
-      COALESCE(p.sellingprice, 0) AS regular_price,
-      CASE 
-        WHEN EXISTS (SELECT 1 FROM product_inventory WHERE pid = p.pid AND count > 0) 
-        THEN (SELECT ROUND(AVG(costeach), 5) FROM product_inventory WHERE pid = p.pid AND count > 0)
-        ELSE (SELECT costeach FROM product_inventory WHERE pid = p.pid ORDER BY daterec DESC LIMIT 1)
-      END AS cost_price,
-      NULL as landing_cost_price,
-      s.companyname AS vendor,
-      CASE 
-        WHEN s.companyname = 'Notions' THEN sid.notions_itemnumber 
-        ELSE sid.supplier_itemnumber 
-      END AS vendor_reference,
-      sid.notions_itemnumber AS notions_reference,
-      CONCAT('https://www.acherryontop.com/shop/product/', p.pid) AS permalink,
-      pc1.name AS brand,
-      pc2.name AS line,
-      pc3.name AS subline,
-      pc4.name AS artist,
-      COALESCE(CASE 
-        WHEN sid.supplier_id = 92 THEN sid.notions_qty_per_unit
-        ELSE sid.supplier_qty_per_unit
-      END, sid.notions_qty_per_unit) AS moq,
-      p.rating,
-      p.rating_votes AS reviews,
-      p.weight,
-      p.length,
-      p.width,
-      p.height,
-      p.country_of_origin,
-      (SELECT COUNT(*) FROM mybasket mb WHERE mb.item = p.pid AND mb.qty > 0) AS baskets,
-      (SELECT COUNT(*) FROM product_notify pn WHERE pn.pid = p.pid) AS notifies,
-      p.totalsold AS total_sold,
-      pls.date_sold as date_last_sold,
-      GROUP_CONCAT(DISTINCT CASE 
-        WHEN pc.cat_id IS NOT NULL 
-        AND pc.type IN (10, 20, 11, 21, 12, 13)
-        AND pci.cat_id NOT IN (16, 17)
-        THEN pci.cat_id 
-      END) as category_ids
-    FROM products p
-    LEFT JOIN shop_inventory si ON p.pid = si.pid AND si.store = 0
-    LEFT JOIN current_inventory ci ON p.pid = ci.pid
-    LEFT JOIN product_notions_b2b pnb ON p.pid = pnb.pid
-    LEFT JOIN product_current_prices pcp ON p.pid = pcp.pid AND pcp.active = 1
-    LEFT JOIN supplier_item_data sid ON p.pid = sid.pid
-    LEFT JOIN suppliers s ON sid.supplier_id = s.supplierid
-    LEFT JOIN product_category_index pci ON p.pid = pci.pid
-    LEFT JOIN product_categories pc ON pci.cat_id = pc.cat_id
-    LEFT JOIN product_categories pc1 ON p.company = pc1.cat_id
-    LEFT JOIN product_categories pc2 ON p.line = pc2.cat_id
-    LEFT JOIN product_categories pc3 ON p.subline = pc3.cat_id
-    LEFT JOIN product_categories pc4 ON p.artist = pc4.cat_id
-    LEFT JOIN product_last_sold pls ON p.pid = pls.pid
-    WHERE ${incrementalUpdate ? `
-      p.stamp > ? OR
-      ci.stamp > ? OR
-      pcp.date_deactive > ? OR
-      pcp.date_active > ? OR
-      pnb.date_updated > ?
-    ` : 'TRUE'}
-    GROUP BY p.pid
-  `, incrementalUpdate ? [lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime] : []);
-
-  outputProgress({
-    status: "running",
-    operation: "Products import",
-    message: `Processing ${prodData.length} product records`
-  });
-
-  // Insert all product data into temp table in batches
-  for (let i = 0; i < prodData.length; i += BATCH_SIZE) {
-    const batch = prodData.slice(i, Math.min(i + BATCH_SIZE, prodData.length));
-    
-    await withRetry(async () => {
-      const placeholders = batch.map((_, idx) => {
-        const base = idx * 47; // 47 columns
-        return `(${Array.from({ length: 47 }, (_, i) => `$${base + i + 1}`).join(', ')})`;
-      }).join(',');
-
-      const values = batch.flatMap(row => {
-        const imageUrls = getImageUrls(row.pid);
-        return [
-          row.pid,
-          row.title,
-          row.description,
-          row.itemnumber || '',
-          row.stock_quantity > 5000 ? 0 : Math.max(0, row.stock_quantity),
-          row.preorder_count,
-          row.notions_inv_count,
-          row.price,
-          row.regular_price,
-          row.cost_price,
-          row.vendor,
-          row.vendor_reference,
-          row.notions_reference,
-          row.brand,
-          row.line,
-          row.subline,
-          row.artist,
-          row.category_ids,
-          validateDate(row.date_created),
-          validateDate(row.first_received),
-          row.landing_cost_price,
-          row.barcode,
-          row.harmonized_tariff_code,
-          validateDate(row.updated_at),
-          row.visible,
-          true,
-          row.replenishable,
-          row.permalink,
-          Math.max(1, Math.round(row.moq || 1)),
-          1,
-          row.rating,
-          row.reviews,
-          row.weight,
-          row.length,
-          row.width,
-          row.height,
-          row.country_of_origin,
-          row.location,
-          row.total_sold,
-          row.baskets,
-          row.notifies,
-          validateDate(row.date_last_sold),
-          imageUrls.image,
-          imageUrls.image_175,
-          imageUrls.image_full,
-          null,
-          null
-        ];
-      });
-
-      await localConnection.query(`
-        INSERT INTO temp_products (
-          pid, title, description, sku, stock_quantity, preorder_count, notions_inv_count,
-          price, regular_price, cost_price, vendor, vendor_reference, notions_reference,
-          brand, line, subline, artist, categories, created_at, first_received,
-          landing_cost_price, barcode, harmonized_tariff_code, updated_at, visible,
-          managing_stock, replenishable, permalink, moq, uom, rating, reviews,
-          weight, length, width, height, country_of_origin, location, total_sold,
-          baskets, notifies, date_last_sold, image, image_175, image_full, options, tags
-        ) VALUES ${placeholders}
-        ON CONFLICT (pid) DO UPDATE SET
-          title = EXCLUDED.title,
-          description = EXCLUDED.description,
-          sku = EXCLUDED.sku,
-          stock_quantity = EXCLUDED.stock_quantity,
-          preorder_count = EXCLUDED.preorder_count,
-          notions_inv_count = EXCLUDED.notions_inv_count,
-          price = EXCLUDED.price,
-          regular_price = EXCLUDED.regular_price,
-          cost_price = EXCLUDED.cost_price,
-          vendor = EXCLUDED.vendor,
-          vendor_reference = EXCLUDED.vendor_reference,
-          notions_reference = EXCLUDED.notions_reference,
-          brand = EXCLUDED.brand,
-          line = EXCLUDED.line,
-          subline = EXCLUDED.subline,
-          artist = EXCLUDED.artist,
-          created_at = EXCLUDED.created_at,
-          first_received = EXCLUDED.first_received,
-          landing_cost_price = EXCLUDED.landing_cost_price,
-          barcode = EXCLUDED.barcode,
-          harmonized_tariff_code = EXCLUDED.harmonized_tariff_code,
-          updated_at = EXCLUDED.updated_at,
-          visible = EXCLUDED.visible,
-          managing_stock = EXCLUDED.managing_stock,
-          replenishable = EXCLUDED.replenishable,
-          permalink = EXCLUDED.permalink,
-          moq = EXCLUDED.moq,
-          uom = EXCLUDED.uom,
-          rating = EXCLUDED.rating,
-          reviews = EXCLUDED.reviews,
-          weight = EXCLUDED.weight,
-          length = EXCLUDED.length,
-          width = EXCLUDED.width,
-          height = EXCLUDED.height,
-          country_of_origin = EXCLUDED.country_of_origin,
-          location = EXCLUDED.location,
-          total_sold = EXCLUDED.total_sold,
-          baskets = EXCLUDED.baskets,
-          notifies = EXCLUDED.notifies,
-          date_last_sold = EXCLUDED.date_last_sold,
-          image = EXCLUDED.image,
-          image_175 = EXCLUDED.image_175,
-          image_full = EXCLUDED.image_full,
-          options = EXCLUDED.options,
-          tags = EXCLUDED.tags
-        RETURNING 
-          xmax = 0 as inserted
-      `, values);
-    }, `Error inserting batch ${i} to ${i + batch.length}`);
-
-    outputProgress({
-      status: "running",
-      operation: "Products import",
-      message: `Imported ${i + batch.length} of ${prodData.length} products`,
-      current: i + batch.length,
-      total: prodData.length,
-      elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-      remaining: estimateRemaining(startTime, i + batch.length, prodData.length),
-      rate: calculateRate(startTime, i + batch.length)
-    });
-  }
-
-  outputProgress({
-    status: "running",
-    operation: "Products import",
-    message: "Finished materializing calculations"
-  });
-}
-
-async function importProducts(prodConnection, localConnection, incrementalUpdate = true) {
-  const startTime = Date.now();
-  let lastSyncTime = '1970-01-01';
-
-  try {
-    // Get last sync time if doing incremental update
-    if (incrementalUpdate) {
-      const [syncResult] = await localConnection.query(
-        "SELECT last_sync_timestamp FROM sync_status WHERE table_name = 'products'"
-      );
-      if (syncResult.rows.length > 0) {
-        lastSyncTime = syncResult.rows[0].last_sync_timestamp;
-      }
-    }
-
-    // Start a transaction to ensure temporary tables persist
-    await localConnection.beginTransaction();
-
-    try {
-      // Setup temporary tables
-      await setupTemporaryTables(localConnection);
-
-      // Materialize calculations into temp table
-      await materializeCalculations(prodConnection, localConnection, incrementalUpdate, lastSyncTime, startTime);
-
-      // Get the list of products that need updating
-      const [products] = await localConnection.query(`
-        SELECT 
-          t.pid,
-          t.title,
-          t.description,
-          t.sku,
-          t.stock_quantity,
-          t.preorder_count,
-          t.notions_inv_count,
-          t.price,
-          t.regular_price,
-          t.cost_price,
-          t.vendor,
-          t.vendor_reference,
-          t.notions_reference,
-          t.brand,
-          t.line,
-          t.subline,
-          t.artist,
-          t.categories,
-          t.created_at,
-          t.first_received,
-          t.landing_cost_price,
-          t.barcode,
-          t.harmonized_tariff_code,
-          t.updated_at,
-          t.visible,
-          t.managing_stock,
-          t.replenishable,
-          t.permalink,
-          t.moq,
-          t.rating,
-          t.reviews,
-          t.weight,
-          t.length,
-          t.width,
-          t.height,
-          t.country_of_origin,
-          t.location,
-          t.total_sold,
-          t.baskets,
-          t.notifies,
-          t.date_last_sold,
-          t.image,
-          t.image_175,
-          t.image_full,
-          t.options,
-          t.tags
-        FROM temp_products t
-        WHERE t.needs_update = true
-      `);
-
-      // Process products in batches
-      let recordsAdded = 0;
-      let recordsUpdated = 0;
-
-      for (let i = 0; i < products.rows.length; i += BATCH_SIZE) {
-        const batch = products.rows.slice(i, i + BATCH_SIZE);
-        
-        const placeholders = batch.map((_, idx) => {
-          const base = idx * 47; // 47 columns
-          return `(${Array.from({ length: 47 }, (_, i) => `$${base + i + 1}`).join(', ')})`;
-        }).join(',');
-
-        const values = batch.flatMap(row => {
-          const imageUrls = getImageUrls(row.pid);
-          return [
-            row.pid,
-            row.title,
-            row.description,
-            row.sku || '',
-            row.stock_quantity > 5000 ? 0 : Math.max(0, row.stock_quantity),
-            row.preorder_count,
-            row.notions_inv_count,
-            row.price,
-            row.regular_price,
-            row.cost_price,
-            row.vendor,
-            row.vendor_reference,
-            row.notions_reference,
-            row.brand,
-            row.line,
-            row.subline,
-            row.artist,
-            row.categories,
-            validateDate(row.created_at),
-            validateDate(row.first_received),
-            row.landing_cost_price,
-            row.barcode,
-            row.harmonized_tariff_code,
-            validateDate(row.updated_at),
-            row.visible,
-            row.managing_stock,
-            row.replenishable,
-            row.permalink,
-            row.moq,
-            1,
-            row.rating,
-            row.reviews,
-            row.weight,
-            row.length,
-            row.width,
-            row.height,
-            row.country_of_origin,
-            row.location,
-            row.total_sold,
-            row.baskets,
-            row.notifies,
-            validateDate(row.date_last_sold),
-            imageUrls.image,
-            imageUrls.image_175,
-            imageUrls.image_full,
-            row.options,
-            row.tags
-          ];
-        });
-
-        const [result] = await localConnection.query(`
-          WITH upserted AS (
-            INSERT INTO products (
-              pid, title, description, sku, stock_quantity, preorder_count, notions_inv_count,
-              price, regular_price, cost_price, vendor, vendor_reference, notions_reference,
-              brand, line, subline, artist, categories, created_at, first_received,
-              landing_cost_price, barcode, harmonized_tariff_code, updated_at, visible,
-              managing_stock, replenishable, permalink, moq, uom, rating, reviews,
-              weight, length, width, height, country_of_origin, location, total_sold,
-              baskets, notifies, date_last_sold, image, image_175, image_full, options, tags
-            )
-            VALUES ${placeholders}
-            ON CONFLICT (pid) DO UPDATE SET
-              title = EXCLUDED.title,
-              description = EXCLUDED.description,
-              sku = EXCLUDED.sku,
-              stock_quantity = EXCLUDED.stock_quantity,
-              preorder_count = EXCLUDED.preorder_count,
-              notions_inv_count = EXCLUDED.notions_inv_count,
-              price = EXCLUDED.price,
-              regular_price = EXCLUDED.regular_price,
-              cost_price = EXCLUDED.cost_price,
-              vendor = EXCLUDED.vendor,
-              vendor_reference = EXCLUDED.vendor_reference,
-              notions_reference = EXCLUDED.notions_reference,
-              brand = EXCLUDED.brand,
-              line = EXCLUDED.line,
-              subline = EXCLUDED.subline,
-              artist = EXCLUDED.artist,
-              created_at = EXCLUDED.created_at,
-              first_received = EXCLUDED.first_received,
-              landing_cost_price = EXCLUDED.landing_cost_price,
-              barcode = EXCLUDED.barcode,
-              harmonized_tariff_code = EXCLUDED.harmonized_tariff_code,
-              updated_at = EXCLUDED.updated_at,
-              visible = EXCLUDED.visible,
-              managing_stock = EXCLUDED.managing_stock,
-              replenishable = EXCLUDED.replenishable,
-              permalink = EXCLUDED.permalink,
-              moq = EXCLUDED.moq,
-              uom = EXCLUDED.uom,
-              rating = EXCLUDED.rating,
-              reviews = EXCLUDED.reviews,
-              weight = EXCLUDED.weight,
-              length = EXCLUDED.length,
-              width = EXCLUDED.width,
-              height = EXCLUDED.height,
-              country_of_origin = EXCLUDED.country_of_origin,
-              location = EXCLUDED.location,
-              total_sold = EXCLUDED.total_sold,
-              baskets = EXCLUDED.baskets,
-              notifies = EXCLUDED.notifies,
-              date_last_sold = EXCLUDED.date_last_sold,
-              image = EXCLUDED.image,
-              image_175 = EXCLUDED.image_175,
-              image_full = EXCLUDED.image_full,
-              options = EXCLUDED.options,
-              tags = EXCLUDED.tags
-            RETURNING 
-              xmax = 0 as inserted
-          )
-          SELECT 
-            COUNT(*) FILTER (WHERE inserted) as inserted,
-            COUNT(*) FILTER (WHERE NOT inserted) as updated
-          FROM upserted
-        `, values);
-
-        recordsAdded += parseInt(result.rows[0].inserted, 10) || 0;
-        recordsUpdated += parseInt(result.rows[0].updated, 10) || 0;
-
-        // Process category relationships for each product in the batch
-        for (const row of batch) {
-          if (row.categories) {
-            const categoryIds = row.categories.split(',').filter(id => id && id.trim());
-            if (categoryIds.length > 0) {
-              const catPlaceholders = categoryIds.map((_, idx) => 
-                `($${idx * 2 + 1}, $${idx * 2 + 2})`
-              ).join(',');
-              const catValues = categoryIds.flatMap(catId => [row.pid, parseInt(catId.trim(), 10)]);
-              
-              // First delete existing relationships for this product
-              await localConnection.query(
-                'DELETE FROM product_categories WHERE pid = $1',
-                [row.pid]
-              );
-              
-              // Then insert the new relationships
-              await localConnection.query(`
-                INSERT INTO product_categories (pid, cat_id)
-                VALUES ${catPlaceholders}
-                ON CONFLICT (pid, cat_id) DO NOTHING
-              `, catValues);
-            }
-          }
-        }
-
-        outputProgress({
-          status: "running",
-          operation: "Products import",
-          message: `Processing products: ${i + batch.length} of ${products.rows.length}`,
-          current: i + batch.length,
-          total: products.rows.length,
-          elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-          remaining: estimateRemaining(startTime, i + batch.length, products.rows.length),
-          rate: calculateRate(startTime, i + batch.length)
-        });
-      }
-
-      // Cleanup temporary tables
-      await cleanupTemporaryTables(localConnection);
-
-      // Commit the transaction
-      await localConnection.commit();
-
-      return {
-        status: 'complete',
-        recordsAdded,
-        recordsUpdated,
-        totalRecords: products.rows.length,
-        duration: formatElapsedTime(Date.now() - startTime)
-      };
-    } catch (error) {
-      // Rollback on error
-      await localConnection.rollback();
-      throw error;
-    }
-  } catch (error) {
-    console.error('Error in importProducts:', error);
-    throw error;
-  }
-}
-
-module.exports = {
-  importProducts,
-  importMissingProducts,
-  setupTemporaryTables,
-  cleanupTemporaryTables,
-  materializeCalculations
-};

inventory-server/scripts/import/purchase-orders.js

@@ -1,649 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate } = require('../metrics/utils/progress');
-
-async function importPurchaseOrders(prodConnection, localConnection, incrementalUpdate = true) {
-  const startTime = Date.now();
-  let recordsAdded = 0;
-  let recordsUpdated = 0;
-
-  try {
-    // Get last sync info
-    const [syncInfo] = await localConnection.query(
-      "SELECT last_sync_timestamp FROM sync_status WHERE table_name = 'purchase_orders'"
-    );
-    const lastSyncTime = syncInfo?.rows?.[0]?.last_sync_timestamp || '1970-01-01';
-
-    console.log('Purchase Orders: Using last sync time:', lastSyncTime);
-
-    // Create temporary tables with PostgreSQL syntax
-    await localConnection.query(`
-      DROP TABLE IF EXISTS temp_purchase_orders;
-      DROP TABLE IF EXISTS temp_po_receivings;
-
-      CREATE TEMP TABLE temp_purchase_orders (
-        po_id INTEGER NOT NULL,
-        pid INTEGER NOT NULL,
-        sku VARCHAR(50),
-        name VARCHAR(255),
-        vendor VARCHAR(255),
-        date TIMESTAMP WITH TIME ZONE,
-        expected_date TIMESTAMP WITH TIME ZONE,
-        status INTEGER,
-        notes TEXT,
-        ordered INTEGER,
-        cost_price DECIMAL(10,3),
-        PRIMARY KEY (po_id, pid)
-      );
-
-      CREATE TEMP TABLE temp_po_receivings (
-        po_id INTEGER,
-        pid INTEGER NOT NULL,
-        receiving_id INTEGER NOT NULL,
-        qty_each INTEGER,
-        cost_each DECIMAL(10,3),
-        received_date TIMESTAMP WITH TIME ZONE,
-        received_by INTEGER,
-        received_by_name VARCHAR(255),
-        is_alt_po INTEGER,
-        PRIMARY KEY (receiving_id, pid)
-      );
-    `);
-
-    outputProgress({
-      operation: `Starting ${incrementalUpdate ? 'incremental' : 'full'} purchase orders import`,
-      status: "running",
-    });
-
-    // Get column names - Keep MySQL compatible for production
-    const [columns] = await prodConnection.query(`
-      SELECT COLUMN_NAME 
-      FROM INFORMATION_SCHEMA.COLUMNS 
-      WHERE TABLE_NAME = 'purchase_orders'
-        AND COLUMN_NAME != 'updated'  -- Exclude the updated column
-      ORDER BY ORDINAL_POSITION
-    `);
-    const columnNames = columns.map(col => col.COLUMN_NAME);
-
-    // Build incremental conditions
-    const incrementalWhereClause = incrementalUpdate
-      ? `AND (
-          p.date_updated > ? 
-          OR p.date_ordered > ?
-          OR p.date_estin > ?
-          OR r.date_updated > ?
-          OR r.date_created > ?
-          OR r.date_checked > ?
-          OR rp.stamp > ?
-          OR rp.received_date > ?
-        )`
-      : "";
-    const incrementalParams = incrementalUpdate
-      ? [lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime]
-      : [];
-
-    // First get all relevant PO IDs with basic info - Keep MySQL compatible for production
-    const [[{ total }]] = await prodConnection.query(`
-      SELECT COUNT(*) as total
-      FROM (
-        SELECT DISTINCT pop.po_id, pop.pid
-        FROM po p 
-        USE INDEX (idx_date_created)
-        JOIN po_products pop ON p.po_id = pop.po_id
-        JOIN suppliers s ON p.supplier_id = s.supplierid
-        WHERE p.date_ordered >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-          ${incrementalUpdate ? `
-            AND (
-              p.date_updated > ? 
-              OR p.date_ordered > ?
-              OR p.date_estin > ?
-            )
-          ` : ''}
-        UNION
-        SELECT DISTINCT r.receiving_id as po_id, rp.pid
-        FROM receivings_products rp
-        USE INDEX (received_date)
-        LEFT JOIN receivings r ON r.receiving_id = rp.receiving_id
-        WHERE rp.received_date >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-          ${incrementalUpdate ? `
-            AND (
-              r.date_created > ?
-              OR r.date_checked > ?
-              OR rp.stamp > ?
-              OR rp.received_date > ?
-            )
-          ` : ''}
-      ) all_items
-    `, incrementalUpdate ? [
-      lastSyncTime, lastSyncTime, lastSyncTime,  // PO conditions
-      lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime  // Receiving conditions
-    ] : []);
-
-    console.log('Purchase Orders: Found changes:', total);
-
-    // Get PO list - Keep MySQL compatible for production
-    const [poList] = await prodConnection.query(`
-      SELECT DISTINCT 
-        COALESCE(p.po_id, r.receiving_id) as po_id,
-        COALESCE(
-          NULLIF(s1.companyname, ''),
-          NULLIF(s2.companyname, ''),
-          'Unknown Vendor'
-        ) as vendor,
-        CASE 
-          WHEN p.po_id IS NOT NULL THEN 
-            COALESCE(
-              NULLIF(p.date_ordered, '0000-00-00 00:00:00'),
-              p.date_created
-            )
-          WHEN r.receiving_id IS NOT NULL THEN
-            r.date_created
-        END as date,
-        CASE 
-          WHEN p.date_estin = '0000-00-00' THEN NULL
-          WHEN p.date_estin IS NULL THEN NULL
-          WHEN p.date_estin NOT REGEXP '^[0-9]{4}-[0-9]{2}-[0-9]{2}$' THEN NULL
-          ELSE p.date_estin
-        END as expected_date,
-        COALESCE(p.status, 50) as status,
-        p.short_note as notes,
-        p.notes as long_note
-      FROM (
-        SELECT po_id FROM po 
-        USE INDEX (idx_date_created)
-        WHERE date_ordered >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-          ${incrementalUpdate ? `
-            AND (
-              date_ordered > ? 
-              OR date_updated > ?
-              OR date_estin > ?
-            )
-          ` : ''}
-        UNION
-        SELECT DISTINCT r.receiving_id as po_id
-        FROM receivings r
-        JOIN receivings_products rp USE INDEX (received_date) ON r.receiving_id = rp.receiving_id
-        WHERE rp.received_date >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
-          ${incrementalUpdate ? `
-            AND (
-              r.date_created > ?
-              OR r.date_checked > ?
-              OR rp.stamp > ?
-              OR rp.received_date > ?
-            )
-          ` : ''}
-      ) ids
-      LEFT JOIN po p ON ids.po_id = p.po_id
-      LEFT JOIN suppliers s1 ON p.supplier_id = s1.supplierid
-      LEFT JOIN receivings r ON ids.po_id = r.receiving_id
-      LEFT JOIN suppliers s2 ON r.supplier_id = s2.supplierid
-      ORDER BY po_id
-    `, incrementalUpdate ? [
-      lastSyncTime, lastSyncTime, lastSyncTime,  // PO conditions
-      lastSyncTime, lastSyncTime, lastSyncTime, lastSyncTime  // Receiving conditions
-    ] : []);
-
-    console.log('Sample PO dates:', poList.slice(0, 5).map(po => ({
-      po_id: po.po_id,
-      raw_date_ordered: po.raw_date_ordered,
-      raw_date_created: po.raw_date_created,
-      raw_date_estin: po.raw_date_estin,
-      computed_date: po.date,
-      expected_date: po.expected_date
-    })));
-
-    const totalItems = total;
-    let processed = 0;
-
-    const BATCH_SIZE = 5000;
-    const PROGRESS_INTERVAL = 500;
-    let lastProgressUpdate = Date.now();
-
-    outputProgress({
-      operation: `Starting purchase orders import - Processing ${totalItems} purchase order items`,
-      status: "running",
-    });
-
-    for (let i = 0; i < poList.length; i += BATCH_SIZE) {
-      const batch = poList.slice(i, Math.min(i + BATCH_SIZE, poList.length));
-      const poIds = batch.map(po => po.po_id);
-
-      // Get all products for these POs in one query - Keep MySQL compatible for production
-      const [poProducts] = await prodConnection.query(`
-        SELECT 
-          pop.po_id,
-          pop.pid,
-          pr.itemnumber as sku,
-          pr.description as name,
-          pop.cost_each as cost_price,
-          pop.qty_each as ordered
-        FROM po_products pop
-        USE INDEX (PRIMARY)
-        JOIN products pr ON pop.pid = pr.pid
-        WHERE pop.po_id IN (?)
-      `, [poIds]);
-
-      // Process PO products in smaller sub-batches to avoid packet size issues
-      const SUB_BATCH_SIZE = 5000;
-      for (let j = 0; j < poProducts.length; j += SUB_BATCH_SIZE) {
-        const productBatch = poProducts.slice(j, j + SUB_BATCH_SIZE);
-        const productPids = [...new Set(productBatch.map(p => p.pid))];
-        const batchPoIds = [...new Set(productBatch.map(p => p.po_id))];
-
-        // Get receivings for this batch with employee names
-        const [receivings] = await prodConnection.query(`
-          SELECT 
-            r.po_id,
-            rp.pid,
-            rp.receiving_id,
-            rp.qty_each,
-            rp.cost_each,
-            COALESCE(rp.received_date, r.date_created) as received_date,
-            rp.received_by,
-            CONCAT(e.firstname, ' ', e.lastname) as received_by_name,
-            CASE 
-              WHEN r.po_id IS NULL THEN 2  -- No PO
-              WHEN r.po_id IN (?) THEN 0   -- Original PO
-              ELSE 1                       -- Different PO
-            END as is_alt_po
-          FROM receivings_products rp
-          USE INDEX (received_date)
-          LEFT JOIN receivings r ON r.receiving_id = rp.receiving_id
-          LEFT JOIN employees e ON rp.received_by = e.employeeid
-          WHERE rp.pid IN (?)
-            AND rp.received_date >= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR)
-          ORDER BY r.po_id, rp.pid, rp.received_date
-        `, [batchPoIds, productPids]);
-
-        // Insert receivings into temp table
-        if (receivings.length > 0) {
-          // Process in smaller chunks to avoid parameter limits
-          const CHUNK_SIZE = 100; // Reduce chunk size to avoid parameter limits
-          for (let i = 0; i < receivings.length; i += CHUNK_SIZE) {
-            const chunk = receivings.slice(i, Math.min(i + CHUNK_SIZE, receivings.length));
-            
-            const values = [];
-            const placeholders = [];
-            
-            chunk.forEach((r, idx) => {
-              values.push(
-                r.po_id,
-                r.pid,
-                r.receiving_id,
-                r.qty_each,
-                r.cost_each,
-                r.received_date,
-                r.received_by,
-                r.received_by_name || null,
-                r.is_alt_po
-              );
-              
-              const offset = idx * 9;
-              placeholders.push(`($${offset + 1}, $${offset + 2}, $${offset + 3}, $${offset + 4}, $${offset + 5}, $${offset + 6}, $${offset + 7}, $${offset + 8}, $${offset + 9})`);
-            });
-
-            await localConnection.query(`
-              INSERT INTO temp_po_receivings (
-                po_id, pid, receiving_id, qty_each, cost_each, received_date,
-                received_by, received_by_name, is_alt_po
-              )
-              VALUES ${placeholders.join(',')}
-              ON CONFLICT (receiving_id, pid) DO UPDATE SET
-                po_id = EXCLUDED.po_id,
-                qty_each = EXCLUDED.qty_each,
-                cost_each = EXCLUDED.cost_each,
-                received_date = EXCLUDED.received_date,
-                received_by = EXCLUDED.received_by,
-                received_by_name = EXCLUDED.received_by_name,
-                is_alt_po = EXCLUDED.is_alt_po
-            `, values);
-          }
-        }
-
-        // Process each PO product in chunks
-        const PRODUCT_CHUNK_SIZE = 100;
-        for (let i = 0; i < productBatch.length; i += PRODUCT_CHUNK_SIZE) {
-          const chunk = productBatch.slice(i, Math.min(i + PRODUCT_CHUNK_SIZE, productBatch.length));
-          const values = [];
-          const placeholders = [];
-          
-          chunk.forEach((product, idx) => {
-            const po = batch.find(p => p.po_id === product.po_id);
-            if (!po) return;
-            
-            values.push(
-              product.po_id,
-              product.pid,
-              product.sku,
-              product.name,
-              po.vendor,
-              po.date,
-              po.expected_date,
-              po.status,
-              po.notes || po.long_note,
-              product.ordered,
-              product.cost_price
-            );
-            
-            const offset = idx * 11;  // Updated to match 11 fields
-            placeholders.push(`($${offset + 1}, $${offset + 2}, $${offset + 3}, $${offset + 4}, $${offset + 5}, $${offset + 6}, $${offset + 7}, $${offset + 8}, $${offset + 9}, $${offset + 10}, $${offset + 11})`);
-          });
-
-          if (placeholders.length > 0) {
-            await localConnection.query(`
-              INSERT INTO temp_purchase_orders (
-                po_id, pid, sku, name, vendor, date, expected_date, 
-                status, notes, ordered, cost_price
-              )
-              VALUES ${placeholders.join(',')}
-              ON CONFLICT (po_id, pid) DO UPDATE SET
-                sku = EXCLUDED.sku,
-                name = EXCLUDED.name,
-                vendor = EXCLUDED.vendor,
-                date = EXCLUDED.date,
-                expected_date = EXCLUDED.expected_date,
-                status = EXCLUDED.status,
-                notes = EXCLUDED.notes,
-                ordered = EXCLUDED.ordered,
-                cost_price = EXCLUDED.cost_price
-            `, values);
-          }
-
-          processed += chunk.length;
-          
-          // Update progress based on time interval
-          const now = Date.now();
-          if (now - lastProgressUpdate >= PROGRESS_INTERVAL || processed === totalItems) {
-            outputProgress({
-              status: "running",
-              operation: "Purchase orders import",
-              current: processed,
-              total: totalItems,
-              elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-              remaining: estimateRemaining(startTime, processed, totalItems),
-              rate: calculateRate(startTime, processed)
-            });
-            lastProgressUpdate = now;
-          }
-        }
-      }
-    }
-
-    // Insert final data into purchase_orders table in chunks
-    const FINAL_CHUNK_SIZE = 1000;
-    let totalProcessed = 0;
-    const totalPosResult = await localConnection.query('SELECT COUNT(*) as total_pos FROM temp_purchase_orders');
-    const total_pos = parseInt(totalPosResult.rows?.[0]?.total_pos || '0', 10);
-
-    outputProgress({
-      status: "running",
-      operation: "Purchase orders final import",
-      message: `Processing ${total_pos} purchase orders for final import`,
-      current: 0,
-      total: total_pos
-    });
-
-    // Process in chunks using cursor-based pagination
-    let lastPoId = 0;
-    let lastPid = 0;
-    let recordsAdded = 0;
-    let recordsUpdated = 0;
-
-    while (true) {
-      console.log('Fetching next chunk with lastPoId:', lastPoId, 'lastPid:', lastPid);
-      const chunkResult = await localConnection.query(`
-        SELECT po_id, pid FROM temp_purchase_orders 
-        WHERE (po_id, pid) > ($1, $2)
-        ORDER BY po_id, pid
-        LIMIT $3
-      `, [lastPoId, lastPid, FINAL_CHUNK_SIZE]);
-
-      if (!chunkResult?.rows) {
-        console.error('No rows returned from chunk query:', chunkResult);
-        break;
-      }
-
-      const chunk = chunkResult.rows;
-      console.log('Got chunk of size:', chunk.length);
-      if (chunk.length === 0) break;
-
-      const result = await localConnection.query(`
-        WITH inserted_pos AS (
-          INSERT INTO purchase_orders (
-            po_id, pid, sku, name, cost_price, po_cost_price,
-            vendor, date, expected_date, status, notes,
-            ordered, received, receiving_status,
-            received_date, last_received_date, received_by,
-            receiving_history
-          )
-          SELECT 
-            po.po_id,
-            po.pid,
-            po.sku,
-            po.name,
-            COALESCE(
-              (
-                SELECT cost_each 
-                FROM temp_po_receivings r2 
-                WHERE r2.pid = po.pid 
-                  AND r2.po_id = po.po_id
-                  AND r2.is_alt_po = 0 
-                  AND r2.cost_each > 0
-                ORDER BY r2.received_date 
-                LIMIT 1
-              ),
-              po.cost_price
-            ) as cost_price,
-            po.cost_price as po_cost_price,
-            po.vendor,
-            po.date,
-            po.expected_date,
-            po.status,
-            po.notes,
-            po.ordered,
-            COALESCE(SUM(CASE WHEN r.is_alt_po = 0 THEN r.qty_each END), 0) as received,
-            CASE 
-              WHEN COUNT(r.receiving_id) = 0 THEN 1  -- created
-              WHEN SUM(CASE WHEN r.is_alt_po = 0 THEN r.qty_each END) < po.ordered THEN 30  -- partial
-              ELSE 40  -- full
-            END as receiving_status,
-            MIN(CASE WHEN r.is_alt_po = 0 THEN r.received_date END) as received_date,
-            MAX(CASE WHEN r.is_alt_po = 0 THEN r.received_date END) as last_received_date,
-            (
-              SELECT r2.received_by_name 
-              FROM temp_po_receivings r2 
-              WHERE r2.pid = po.pid 
-                AND r2.is_alt_po = 0 
-              ORDER BY r2.received_date 
-              LIMIT 1
-            ) as received_by,
-            jsonb_build_object(
-              'ordered_qty', po.ordered,
-              'total_received', COALESCE(SUM(CASE WHEN r.is_alt_po = 0 THEN r.qty_each END), 0),
-              'remaining_unfulfilled', GREATEST(0, po.ordered - COALESCE(SUM(CASE WHEN r.is_alt_po = 0 THEN r.qty_each END), 0)),
-              'excess_received', GREATEST(0, COALESCE(SUM(CASE WHEN r.is_alt_po = 0 THEN r.qty_each END), 0) - po.ordered),
-              'po_cost', po.cost_price,
-              'actual_cost', COALESCE(
-                (
-                  SELECT cost_each 
-                  FROM temp_po_receivings r2 
-                  WHERE r2.pid = po.pid 
-                    AND r2.is_alt_po = 0 
-                    AND r2.cost_each > 0
-                  ORDER BY r2.received_date 
-                  LIMIT 1
-                ),
-                po.cost_price
-              ),
-              'fulfillment', (
-                SELECT jsonb_agg(
-                  jsonb_build_object(
-                    'receiving_id', r2.receiving_id,
-                    'qty_applied', CASE 
-                      WHEN r2.running_total <= po.ordered THEN r2.qty_each
-                      WHEN r2.running_total - r2.qty_each < po.ordered THEN po.ordered - (r2.running_total - r2.qty_each)
-                      ELSE 0
-                    END,
-                    'qty_total', r2.qty_each,
-                    'cost', r2.cost_each,
-                    'date', r2.received_date,
-                    'received_by', r2.received_by,
-                    'received_by_name', r2.received_by_name,
-                    'type', CASE r2.is_alt_po 
-                      WHEN 0 THEN 'original'
-                      WHEN 1 THEN 'alternate'
-                      ELSE 'no_po'
-                    END,
-                    'remaining_qty', CASE 
-                      WHEN r2.running_total <= po.ordered THEN 0
-                      WHEN r2.running_total - r2.qty_each < po.ordered THEN r2.running_total - po.ordered
-                      ELSE r2.qty_each
-                    END,
-                    'is_excess', r2.running_total > po.ordered
-                  )
-                  ORDER BY r2.received_date
-                ) 
-                FROM (
-                  SELECT 
-                    r2.*,
-                    SUM(r2.qty_each) OVER (
-                      PARTITION BY r2.pid 
-                      ORDER BY r2.received_date
-                      ROWS UNBOUNDED PRECEDING
-                    ) as running_total
-                  FROM temp_po_receivings r2
-                  WHERE r2.pid = po.pid
-                ) r2
-              ),
-              'alternate_po_receivings', (
-                SELECT jsonb_agg(
-                  jsonb_build_object(
-                    'receiving_id', r2.receiving_id,
-                    'qty', r2.qty_each,
-                    'cost', r2.cost_each,
-                    'date', r2.received_date,
-                    'received_by', r2.received_by,
-                    'received_by_name', r2.received_by_name
-                  )
-                  ORDER BY r2.received_date
-                )
-                FROM temp_po_receivings r2
-                WHERE r2.pid = po.pid AND r2.is_alt_po = 1
-              ),
-              'no_po_receivings', (
-                SELECT jsonb_agg(
-                  jsonb_build_object(
-                    'receiving_id', r2.receiving_id,
-                    'qty', r2.qty_each,
-                    'cost', r2.cost_each,
-                    'date', r2.received_date,
-                    'received_by', r2.received_by,
-                    'received_by_name', r2.received_by_name
-                  )
-                  ORDER BY r2.received_date
-                )
-                FROM temp_po_receivings r2
-                WHERE r2.pid = po.pid AND r2.is_alt_po = 2
-              )
-            ) as receiving_history
-          FROM temp_purchase_orders po
-          LEFT JOIN temp_po_receivings r ON po.pid = r.pid
-          WHERE (po.po_id, po.pid) IN (
-            SELECT po_id, pid FROM UNNEST($1::int[], $2::int[])
-          )
-          GROUP BY po.po_id, po.pid, po.sku, po.name, po.vendor, po.date, 
-                  po.expected_date, po.status, po.notes, po.ordered, po.cost_price
-          ON CONFLICT (po_id, pid) DO UPDATE SET
-            vendor = EXCLUDED.vendor,
-            date = EXCLUDED.date,
-            expected_date = EXCLUDED.expected_date,
-            status = EXCLUDED.status,
-            notes = EXCLUDED.notes,
-            ordered = EXCLUDED.ordered,
-            received = EXCLUDED.received,
-            receiving_status = EXCLUDED.receiving_status,
-            received_date = EXCLUDED.received_date,
-            last_received_date = EXCLUDED.last_received_date,
-            received_by = EXCLUDED.received_by,
-            receiving_history = EXCLUDED.receiving_history,
-            cost_price = EXCLUDED.cost_price,
-            po_cost_price = EXCLUDED.po_cost_price
-          RETURNING xmax
-        )
-        SELECT 
-          COUNT(*) FILTER (WHERE xmax = 0) as inserted,
-          COUNT(*) FILTER (WHERE xmax <> 0) as updated
-        FROM inserted_pos
-      `, [
-        chunk.map(r => r.po_id),
-        chunk.map(r => r.pid)
-      ]);
-
-      // Add debug logging
-      console.log('Insert result:', result?.rows?.[0]);
-
-      // Handle the result properly for PostgreSQL with more defensive coding
-      const resultRow = result?.rows?.[0] || {};
-      const insertCount = parseInt(resultRow.inserted || '0', 10);
-      const updateCount = parseInt(resultRow.updated || '0', 10);
-
-      recordsAdded += insertCount;
-      recordsUpdated += updateCount;
-      totalProcessed += chunk.length;
-
-      // Update progress
-      outputProgress({
-        status: "running",
-        operation: "Purchase orders final import",
-        message: `Processed ${totalProcessed} of ${total_pos} purchase orders`,
-        current: totalProcessed,
-        total: total_pos,
-        elapsed: formatElapsedTime((Date.now() - startTime) / 1000),
-        remaining: estimateRemaining(startTime, totalProcessed, total_pos),
-        rate: calculateRate(startTime, totalProcessed)
-      });
-
-      // Update last processed IDs for next chunk with safety check
-      if (chunk.length > 0) {
-        const lastItem = chunk[chunk.length - 1];
-        if (lastItem) {
-          lastPoId = lastItem.po_id;
-          lastPid = lastItem.pid;
-        }
-      }
-    }
-
-    // Update sync status
-    await localConnection.query(`
-      INSERT INTO sync_status (table_name, last_sync_timestamp)
-      VALUES ('purchase_orders', NOW())
-      ON CONFLICT (table_name) DO UPDATE SET
-        last_sync_timestamp = NOW()
-    `);
-
-    // Clean up temporary tables
-    await localConnection.query(`
-      DROP TABLE IF EXISTS temp_purchase_orders;
-      DROP TABLE IF EXISTS temp_po_receivings;
-    `);
-
-    return {
-      status: "complete",
-      recordsAdded,
-      recordsUpdated,
-      totalRecords: processed
-    };
-  } catch (error) {
-    console.error("Error during purchase orders import:", error);
-    // Attempt cleanup on error
-    try {
-      await localConnection.query(`
-        DROP TABLE IF EXISTS temp_purchase_orders;
-        DROP TABLE IF EXISTS temp_po_receivings;
-      `);
-    } catch (cleanupError) {
-      console.error('Error during cleanup:', cleanupError);
-    }
-    throw error;
-  }
-}
-
-module.exports = importPurchaseOrders;

inventory-server/scripts/import/utils.js

@@ -1,156 +0,0 @@
-const mysql = require("mysql2/promise");
-const { Client } = require("ssh2");
-const { Pool } = require('pg');
-const dotenv = require("dotenv");
-const path = require("path");
-
-// Helper function to setup SSH tunnel
-async function setupSshTunnel(sshConfig) {
-  return new Promise((resolve, reject) => {
-    const ssh = new Client();
-
-    ssh.on('error', (err) => {
-      console.error('SSH connection error:', err);
-    });
-
-    ssh.on('end', () => {
-      console.log('SSH connection ended normally');
-    });
-
-    ssh.on('close', () => {
-      console.log('SSH connection closed');
-    });
-
-    ssh
-      .on("ready", () => {
-        ssh.forwardOut(
-          "127.0.0.1",
-          0,
-          sshConfig.prodDbConfig.host,
-          sshConfig.prodDbConfig.port,
-          async (err, stream) => {
-            if (err) reject(err);
-            resolve({ ssh, stream });
-          }
-        );
-      })
-      .connect(sshConfig.ssh);
-  });
-}
-
-// Helper function to setup database connections
-async function setupConnections(sshConfig) {
-  const tunnel = await setupSshTunnel(sshConfig);
-  
-  // Setup MySQL connection for production
-  const prodConnection = await mysql.createConnection({
-    ...sshConfig.prodDbConfig,
-    stream: tunnel.stream,
-  });
-
-  // Setup PostgreSQL connection pool for local
-  const localPool = new Pool(sshConfig.localDbConfig);
-
-  // Test the PostgreSQL connection
-  try {
-    const client = await localPool.connect();
-    await client.query('SELECT NOW()');
-    client.release();
-    console.log('PostgreSQL connection successful');
-  } catch (err) {
-    console.error('PostgreSQL connection error:', err);
-    throw err;
-  }
-
-  // Create a wrapper for the PostgreSQL pool to match MySQL interface
-  const localConnection = {
-    _client: null,
-    _transactionActive: false,
-
-    query: async (text, params) => {
-      // If we're not in a transaction, use the pool directly
-      if (!localConnection._transactionActive) {
-        const client = await localPool.connect();
-        try {
-          const result = await client.query(text, params);
-          return [result];
-        } finally {
-          client.release();
-        }
-      }
-      
-      // If we're in a transaction, use the dedicated client
-      if (!localConnection._client) {
-        throw new Error('No active transaction client');
-      }
-      const result = await localConnection._client.query(text, params);
-      return [result];
-    },
-
-    beginTransaction: async () => {
-      if (localConnection._transactionActive) {
-        throw new Error('Transaction already active');
-      }
-      localConnection._client = await localPool.connect();
-      await localConnection._client.query('BEGIN');
-      localConnection._transactionActive = true;
-    },
-
-    commit: async () => {
-      if (!localConnection._transactionActive) {
-        throw new Error('No active transaction to commit');
-      }
-      await localConnection._client.query('COMMIT');
-      localConnection._client.release();
-      localConnection._client = null;
-      localConnection._transactionActive = false;
-    },
-
-    rollback: async () => {
-      if (!localConnection._transactionActive) {
-        throw new Error('No active transaction to rollback');
-      }
-      await localConnection._client.query('ROLLBACK');
-      localConnection._client.release();
-      localConnection._client = null;
-      localConnection._transactionActive = false;
-    },
-
-    end: async () => {
-      if (localConnection._client) {
-        localConnection._client.release();
-        localConnection._client = null;
-      }
-      await localPool.end();
-    }
-  };
-
-  return { prodConnection, localConnection, tunnel };
-}
-
-// Helper function to close connections
-async function closeConnections(connections) {
-  const { ssh, prodConnection, localConnection } = connections;
-  
-  try {
-    if (prodConnection) await prodConnection.end();
-    if (localConnection) await localConnection.end();
-    
-    // Wait a bit for any pending data to be written before closing SSH
-    await new Promise(resolve => setTimeout(resolve, 100));
-    
-    if (ssh) {
-      ssh.on('close', () => {
-        console.log('SSH connection closed cleanly');
-      });
-      ssh.end();
-    }
-  } catch (err) {
-    console.error('Error during cleanup:', err);
-  }
-}
-
-module.exports = {
-  setupConnections,
-  closeConnections
-}; 

inventory-server/scripts/metrics-new/calculate_brand_metrics.sql

@@ -0,0 +1,135 @@
+-- Description: Calculates and updates aggregated metrics per brand.
+-- Dependencies: product_metrics, products, calculate_status table.
+-- Frequency: Daily (after product_metrics update).
+
+DO $$
+DECLARE
+    _module_name VARCHAR := 'brand_metrics';
+    _start_time TIMESTAMPTZ := clock_timestamp();
+    _min_revenue NUMERIC := 50.00; -- Minimum revenue threshold for margin calculation
+BEGIN
+    RAISE NOTICE 'Running % calculation...', _module_name;
+
+    WITH BrandAggregates AS (
+        -- Aggregate metrics from product_metrics table per brand
+        SELECT
+            COALESCE(p.brand, 'Unbranded') AS brand_group, -- Group NULL/empty brands together
+            COUNT(DISTINCT pm.pid) AS product_count,
+            COUNT(DISTINCT CASE WHEN pm.is_visible THEN pm.pid END) AS active_product_count,
+            COUNT(DISTINCT CASE WHEN pm.is_replenishable THEN pm.pid END) AS replenishable_product_count,
+            SUM(pm.current_stock) AS current_stock_units,
+            SUM(pm.current_stock_cost) AS current_stock_cost,
+            SUM(pm.current_stock_retail) AS current_stock_retail,
+            -- Only include products with valid sales data in each time period
+            COUNT(DISTINCT CASE WHEN pm.sales_7d > 0 THEN pm.pid END) AS products_with_sales_7d,
+            SUM(CASE WHEN pm.sales_7d > 0 THEN pm.sales_7d ELSE 0 END) AS sales_7d,
+            SUM(CASE WHEN pm.revenue_7d > 0 THEN pm.revenue_7d ELSE 0 END) AS revenue_7d,
+            
+            COUNT(DISTINCT CASE WHEN pm.sales_30d > 0 THEN pm.pid END) AS products_with_sales_30d,
+            SUM(CASE WHEN pm.sales_30d > 0 THEN pm.sales_30d ELSE 0 END) AS sales_30d,
+            SUM(CASE WHEN pm.revenue_30d > 0 THEN pm.revenue_30d ELSE 0 END) AS revenue_30d,
+            SUM(CASE WHEN pm.cogs_30d > 0 THEN pm.cogs_30d ELSE 0 END) AS cogs_30d,
+            SUM(CASE WHEN pm.profit_30d != 0 THEN pm.profit_30d ELSE 0 END) AS profit_30d,
+            
+            COUNT(DISTINCT CASE WHEN pm.sales_365d > 0 THEN pm.pid END) AS products_with_sales_365d,
+            SUM(CASE WHEN pm.sales_365d > 0 THEN pm.sales_365d ELSE 0 END) AS sales_365d,
+            SUM(CASE WHEN pm.revenue_365d > 0 THEN pm.revenue_365d ELSE 0 END) AS revenue_365d,
+            
+            COUNT(DISTINCT CASE WHEN pm.lifetime_sales > 0 THEN pm.pid END) AS products_with_lifetime_sales,
+            SUM(CASE WHEN pm.lifetime_sales > 0 THEN pm.lifetime_sales ELSE 0 END) AS lifetime_sales,
+            SUM(CASE WHEN pm.lifetime_revenue > 0 THEN pm.lifetime_revenue ELSE 0 END) AS lifetime_revenue
+        FROM public.product_metrics pm
+        JOIN public.products p ON pm.pid = p.pid
+        GROUP BY brand_group
+    ),
+    AllBrands AS (
+         -- Ensure all brands from products table are included, mapping NULL/empty to 'Unbranded'
+        SELECT DISTINCT COALESCE(brand, 'Unbranded') as brand_group
+        FROM public.products
+    )
+    INSERT INTO public.brand_metrics (
+        brand_name, last_calculated,
+        product_count, active_product_count, replenishable_product_count,
+        current_stock_units, current_stock_cost, current_stock_retail,
+        sales_7d, revenue_7d, sales_30d, revenue_30d, profit_30d, cogs_30d,
+        sales_365d, revenue_365d, lifetime_sales, lifetime_revenue,
+        avg_margin_30d
+    )
+    SELECT
+        b.brand_group,
+        _start_time,
+        -- Base Aggregates
+        COALESCE(ba.product_count, 0),
+        COALESCE(ba.active_product_count, 0),
+        COALESCE(ba.replenishable_product_count, 0),
+        COALESCE(ba.current_stock_units, 0),
+        COALESCE(ba.current_stock_cost, 0.00),
+        COALESCE(ba.current_stock_retail, 0.00),
+        -- Sales Aggregates
+        COALESCE(ba.sales_7d, 0), COALESCE(ba.revenue_7d, 0.00),
+        COALESCE(ba.sales_30d, 0), COALESCE(ba.revenue_30d, 0.00),
+        COALESCE(ba.profit_30d, 0.00), COALESCE(ba.cogs_30d, 0.00),
+        COALESCE(ba.sales_365d, 0), COALESCE(ba.revenue_365d, 0.00),
+        COALESCE(ba.lifetime_sales, 0), COALESCE(ba.lifetime_revenue, 0.00),
+        -- KPIs - Calculate margin only for brands with significant revenue
+        CASE
+            WHEN COALESCE(ba.revenue_30d, 0) >= _min_revenue THEN
+                -- Directly calculate margin from revenue and cogs for consistency
+                -- This is mathematically equivalent to profit/revenue but more explicit
+                ((COALESCE(ba.revenue_30d, 0) - COALESCE(ba.cogs_30d, 0)) / COALESCE(ba.revenue_30d, 1)) * 100.0
+            ELSE NULL -- No margin for low/no revenue brands
+        END
+    FROM AllBrands b
+    LEFT JOIN BrandAggregates ba ON b.brand_group = ba.brand_group
+
+    ON CONFLICT (brand_name) DO UPDATE SET
+        last_calculated = EXCLUDED.last_calculated,
+        product_count = EXCLUDED.product_count,
+        active_product_count = EXCLUDED.active_product_count,
+        replenishable_product_count = EXCLUDED.replenishable_product_count,
+        current_stock_units = EXCLUDED.current_stock_units,
+        current_stock_cost = EXCLUDED.current_stock_cost,
+        current_stock_retail = EXCLUDED.current_stock_retail,
+        sales_7d = EXCLUDED.sales_7d, revenue_7d = EXCLUDED.revenue_7d,
+        sales_30d = EXCLUDED.sales_30d, revenue_30d = EXCLUDED.revenue_30d,
+        profit_30d = EXCLUDED.profit_30d, cogs_30d = EXCLUDED.cogs_30d,
+        sales_365d = EXCLUDED.sales_365d, revenue_365d = EXCLUDED.revenue_365d,
+        lifetime_sales = EXCLUDED.lifetime_sales, lifetime_revenue = EXCLUDED.lifetime_revenue,
+        avg_margin_30d = EXCLUDED.avg_margin_30d
+    WHERE -- Only update if at least one value has changed
+        brand_metrics.product_count IS DISTINCT FROM EXCLUDED.product_count OR
+        brand_metrics.active_product_count IS DISTINCT FROM EXCLUDED.active_product_count OR
+        brand_metrics.current_stock_units IS DISTINCT FROM EXCLUDED.current_stock_units OR
+        brand_metrics.sales_30d IS DISTINCT FROM EXCLUDED.sales_30d OR
+        brand_metrics.revenue_30d IS DISTINCT FROM EXCLUDED.revenue_30d OR
+        brand_metrics.lifetime_sales IS DISTINCT FROM EXCLUDED.lifetime_sales;
+
+    -- Update calculate_status
+    INSERT INTO public.calculate_status (module_name, last_calculation_timestamp)
+    VALUES (_module_name, _start_time)
+    ON CONFLICT (module_name) DO UPDATE SET last_calculation_timestamp = _start_time;
+
+    RAISE NOTICE 'Finished % calculation. Duration: %', _module_name, clock_timestamp() - _start_time;
+END $$;
+
+-- Return metrics about the update operation for tracking
+WITH update_stats AS (
+    SELECT 
+        COUNT(*) as total_brands,
+        COUNT(*) FILTER (WHERE last_calculated >= NOW() - INTERVAL '5 minutes') as rows_processed,
+        SUM(product_count) as total_products,
+        SUM(active_product_count) as total_active_products,
+        SUM(sales_30d) as total_sales_30d,
+        SUM(revenue_30d) as total_revenue_30d,
+        AVG(avg_margin_30d) as overall_avg_margin_30d
+    FROM public.brand_metrics
+)
+SELECT 
+    rows_processed,
+    total_brands,
+    total_products::int,
+    total_active_products::int,
+    total_sales_30d::int,
+    ROUND(total_revenue_30d, 2) as total_revenue_30d,
+    ROUND(overall_avg_margin_30d, 2) as overall_avg_margin_30d
+FROM update_stats;

inventory-server/scripts/metrics-new/calculate_category_metrics.sql

@@ -0,0 +1,309 @@
+-- Description: Calculates and updates aggregated metrics per category.
+-- Dependencies: product_metrics, products, categories, product_categories, calculate_status table.
+-- Frequency: Daily (after product_metrics update).
+
+DO $$
+DECLARE
+    _module_name VARCHAR := 'category_metrics';
+    _start_time TIMESTAMPTZ := clock_timestamp();
+    _min_revenue NUMERIC := 50.00; -- Minimum revenue threshold for margin calculation
+BEGIN
+    RAISE NOTICE 'Running % calculation...', _module_name;
+
+    WITH 
+    -- Identify the hierarchy depth for each category
+    CategoryDepth AS (
+        WITH RECURSIVE CategoryTree AS (
+            -- Base case: Start with categories without parents (root categories)
+            SELECT cat_id, name, parent_id, 0 AS depth
+            FROM public.categories
+            WHERE parent_id IS NULL
+            
+            UNION ALL
+            
+            -- Recursive step: Add child categories with incremented depth
+            SELECT c.cat_id, c.name, c.parent_id, ct.depth + 1
+            FROM public.categories c
+            JOIN CategoryTree ct ON c.parent_id = ct.cat_id
+        )
+        SELECT cat_id, depth
+        FROM CategoryTree
+    ),
+    -- For each product, find the most specific (deepest) category it belongs to
+    ProductDeepestCategory AS (
+        SELECT 
+            pc.pid,
+            pc.cat_id
+        FROM public.product_categories pc
+        JOIN CategoryDepth cd ON pc.cat_id = cd.cat_id
+        -- This is the key part: for each product, select only the category with maximum depth
+        WHERE (pc.pid, cd.depth) IN (
+            SELECT pc2.pid, MAX(cd2.depth)
+            FROM public.product_categories pc2
+            JOIN CategoryDepth cd2 ON pc2.cat_id = cd2.cat_id
+            GROUP BY pc2.pid
+        )
+    ),
+    -- Calculate metrics only at the most specific category level for each product
+    -- These are the direct metrics (only products directly in this category)
+    DirectCategoryMetrics AS (
+        SELECT
+            pdc.cat_id,
+            -- Counts
+            COUNT(DISTINCT pm.pid) AS product_count,
+            COUNT(DISTINCT CASE WHEN pm.is_visible THEN pm.pid END) AS active_product_count,
+            COUNT(DISTINCT CASE WHEN pm.is_replenishable THEN pm.pid END) AS replenishable_product_count,
+            -- Current Stock
+            SUM(pm.current_stock) AS current_stock_units,
+            SUM(pm.current_stock_cost) AS current_stock_cost,
+            SUM(pm.current_stock_retail) AS current_stock_retail,
+            -- Rolling Periods - Only include products with actual sales in each period
+            SUM(CASE WHEN pm.sales_7d > 0 THEN pm.sales_7d ELSE 0 END) AS sales_7d,
+            SUM(CASE WHEN pm.revenue_7d > 0 THEN pm.revenue_7d ELSE 0 END) AS revenue_7d,
+            SUM(CASE WHEN pm.sales_30d > 0 THEN pm.sales_30d ELSE 0 END) AS sales_30d,
+            SUM(CASE WHEN pm.revenue_30d > 0 THEN pm.revenue_30d ELSE 0 END) AS revenue_30d,
+            SUM(CASE WHEN pm.cogs_30d > 0 THEN pm.cogs_30d ELSE 0 END) AS cogs_30d,
+            SUM(CASE WHEN pm.profit_30d != 0 THEN pm.profit_30d ELSE 0 END) AS profit_30d,
+            SUM(CASE WHEN pm.sales_365d > 0 THEN pm.sales_365d ELSE 0 END) AS sales_365d,
+            SUM(CASE WHEN pm.revenue_365d > 0 THEN pm.revenue_365d ELSE 0 END) AS revenue_365d,
+            SUM(CASE WHEN pm.lifetime_sales > 0 THEN pm.lifetime_sales ELSE 0 END) AS lifetime_sales,
+            SUM(CASE WHEN pm.lifetime_revenue > 0 THEN pm.lifetime_revenue ELSE 0 END) AS lifetime_revenue,
+            -- Data for KPIs - Only average stock for products with stock
+            SUM(CASE WHEN pm.avg_stock_units_30d > 0 THEN pm.avg_stock_units_30d ELSE 0 END) AS total_avg_stock_units_30d
+        FROM public.product_metrics pm
+        JOIN ProductDeepestCategory pdc ON pm.pid = pdc.pid
+        GROUP BY pdc.cat_id
+    ),
+    -- Build a category lookup table for parent relationships
+    CategoryHierarchyPaths AS (
+        WITH RECURSIVE ParentPaths AS (
+            -- Base case: All categories with their immediate parents
+            SELECT 
+                cat_id,
+                cat_id as leaf_id, -- Every category is its own leaf initially
+                ARRAY[cat_id] as path
+            FROM public.categories
+            
+            UNION ALL
+            
+            -- Recursive step: Walk up the parent chain
+            SELECT
+                c.parent_id as cat_id,
+                pp.leaf_id, -- Keep the original leaf_id
+                c.parent_id || pp.path as path
+            FROM ParentPaths pp
+            JOIN public.categories c ON pp.cat_id = c.cat_id
+            WHERE c.parent_id IS NOT NULL -- Stop at root categories
+        )
+        -- Select distinct paths to avoid duplication
+        SELECT DISTINCT cat_id, leaf_id
+        FROM ParentPaths
+    ),
+    -- Aggregate metrics from leaf categories to their ancestors without duplication
+    -- These are the rolled-up metrics (including all child categories)
+    RollupMetrics AS (
+        SELECT
+            chp.cat_id,
+            -- For each parent category, count distinct products to avoid duplication
+            COUNT(DISTINCT dcm.cat_id) AS child_categories_count,
+            SUM(dcm.product_count) AS rollup_product_count,
+            SUM(dcm.active_product_count) AS rollup_active_product_count,
+            SUM(dcm.replenishable_product_count) AS rollup_replenishable_product_count,
+            SUM(dcm.current_stock_units) AS rollup_current_stock_units,
+            SUM(dcm.current_stock_cost) AS rollup_current_stock_cost,
+            SUM(dcm.current_stock_retail) AS rollup_current_stock_retail,
+            SUM(dcm.sales_7d) AS rollup_sales_7d,
+            SUM(dcm.revenue_7d) AS rollup_revenue_7d,
+            SUM(dcm.sales_30d) AS rollup_sales_30d,
+            SUM(dcm.revenue_30d) AS rollup_revenue_30d,
+            SUM(dcm.cogs_30d) AS rollup_cogs_30d,
+            SUM(dcm.profit_30d) AS rollup_profit_30d,
+            SUM(dcm.sales_365d) AS rollup_sales_365d,
+            SUM(dcm.revenue_365d) AS rollup_revenue_365d,
+            SUM(dcm.lifetime_sales) AS rollup_lifetime_sales,
+            SUM(dcm.lifetime_revenue) AS rollup_lifetime_revenue,
+            SUM(dcm.total_avg_stock_units_30d) AS rollup_total_avg_stock_units_30d
+        FROM CategoryHierarchyPaths chp
+        JOIN DirectCategoryMetrics dcm ON chp.leaf_id = dcm.cat_id
+        GROUP BY chp.cat_id
+    ),
+    -- Combine direct and rollup metrics
+    CombinedMetrics AS (
+        SELECT
+            c.cat_id,
+            c.name,
+            c.type,
+            c.parent_id,
+            -- Direct metrics (just this category)
+            COALESCE(dcm.product_count, 0) AS direct_product_count,
+            COALESCE(dcm.active_product_count, 0) AS direct_active_product_count,
+            COALESCE(dcm.replenishable_product_count, 0) AS direct_replenishable_product_count,
+            COALESCE(dcm.current_stock_units, 0) AS direct_current_stock_units,
+            COALESCE(dcm.current_stock_cost, 0) AS direct_current_stock_cost,
+            COALESCE(dcm.current_stock_retail, 0) AS direct_current_stock_retail,
+            COALESCE(dcm.sales_7d, 0) AS direct_sales_7d,
+            COALESCE(dcm.revenue_7d, 0) AS direct_revenue_7d,
+            COALESCE(dcm.sales_30d, 0) AS direct_sales_30d,
+            COALESCE(dcm.revenue_30d, 0) AS direct_revenue_30d,
+            COALESCE(dcm.cogs_30d, 0) AS direct_cogs_30d,
+            COALESCE(dcm.profit_30d, 0) AS direct_profit_30d,
+            COALESCE(dcm.sales_365d, 0) AS direct_sales_365d,
+            COALESCE(dcm.revenue_365d, 0) AS direct_revenue_365d,
+            COALESCE(dcm.lifetime_sales, 0) AS direct_lifetime_sales,
+            COALESCE(dcm.lifetime_revenue, 0) AS direct_lifetime_revenue,
+            COALESCE(dcm.total_avg_stock_units_30d, 0) AS direct_avg_stock_units_30d,
+            
+            -- Rolled up metrics (this category + all children)
+            COALESCE(rm.rollup_product_count, 0) AS product_count,
+            COALESCE(rm.rollup_active_product_count, 0) AS active_product_count,
+            COALESCE(rm.rollup_replenishable_product_count, 0) AS replenishable_product_count,
+            COALESCE(rm.rollup_current_stock_units, 0) AS current_stock_units,
+            COALESCE(rm.rollup_current_stock_cost, 0) AS current_stock_cost,
+            COALESCE(rm.rollup_current_stock_retail, 0) AS current_stock_retail,
+            COALESCE(rm.rollup_sales_7d, 0) AS sales_7d,
+            COALESCE(rm.rollup_revenue_7d, 0) AS revenue_7d,
+            COALESCE(rm.rollup_sales_30d, 0) AS sales_30d,
+            COALESCE(rm.rollup_revenue_30d, 0) AS revenue_30d,
+            COALESCE(rm.rollup_cogs_30d, 0) AS cogs_30d,
+            COALESCE(rm.rollup_profit_30d, 0) AS profit_30d,
+            COALESCE(rm.rollup_sales_365d, 0) AS sales_365d,
+            COALESCE(rm.rollup_revenue_365d, 0) AS revenue_365d,
+            COALESCE(rm.rollup_lifetime_sales, 0) AS lifetime_sales,
+            COALESCE(rm.rollup_lifetime_revenue, 0) AS lifetime_revenue,
+            COALESCE(rm.rollup_total_avg_stock_units_30d, 0) AS total_avg_stock_units_30d
+        FROM public.categories c
+        LEFT JOIN DirectCategoryMetrics dcm ON c.cat_id = dcm.cat_id
+        LEFT JOIN RollupMetrics rm ON c.cat_id = rm.cat_id
+    )
+    INSERT INTO public.category_metrics (
+        category_id, category_name, category_type, parent_id, last_calculated,
+        -- Store all direct and rolled up metrics
+        product_count, active_product_count, replenishable_product_count,
+        current_stock_units, current_stock_cost, current_stock_retail,
+        sales_7d, revenue_7d, sales_30d, revenue_30d, profit_30d, cogs_30d,
+        sales_365d, revenue_365d, lifetime_sales, lifetime_revenue,
+        -- Also store direct metrics with direct_ prefix
+        direct_product_count, direct_active_product_count, direct_replenishable_product_count,
+        direct_current_stock_units, direct_stock_cost, direct_stock_retail,
+        direct_sales_7d, direct_revenue_7d, direct_sales_30d, direct_revenue_30d, 
+        direct_profit_30d, direct_cogs_30d, direct_sales_365d, direct_revenue_365d,
+        direct_lifetime_sales, direct_lifetime_revenue,
+        -- KPIs
+        avg_margin_30d, stock_turn_30d
+    )
+    SELECT
+        cm.cat_id,
+        cm.name,
+        cm.type,
+        cm.parent_id,
+        _start_time,
+        -- Rolled-up metrics (total including children)
+        cm.product_count,
+        cm.active_product_count,
+        cm.replenishable_product_count,
+        cm.current_stock_units,
+        cm.current_stock_cost,
+        cm.current_stock_retail,
+        cm.sales_7d, cm.revenue_7d,
+        cm.sales_30d, cm.revenue_30d, cm.profit_30d, cm.cogs_30d,
+        cm.sales_365d, cm.revenue_365d,
+        cm.lifetime_sales, cm.lifetime_revenue,
+        -- Direct metrics (just this category)
+        cm.direct_product_count,
+        cm.direct_active_product_count,
+        cm.direct_replenishable_product_count,
+        cm.direct_current_stock_units,
+        cm.direct_current_stock_cost,
+        cm.direct_current_stock_retail,
+        cm.direct_sales_7d, cm.direct_revenue_7d,
+        cm.direct_sales_30d, cm.direct_revenue_30d, cm.direct_profit_30d, cm.direct_cogs_30d,
+        cm.direct_sales_365d, cm.direct_revenue_365d,
+        cm.direct_lifetime_sales, cm.direct_lifetime_revenue,
+        -- KPIs - Calculate margin only for categories with significant revenue
+        CASE
+            WHEN cm.revenue_30d >= _min_revenue THEN
+                ((cm.revenue_30d - cm.cogs_30d) / cm.revenue_30d) * 100.0
+            ELSE NULL -- No margin for low/no revenue categories
+        END,
+        -- Stock Turn calculation
+        CASE
+            WHEN cm.total_avg_stock_units_30d > 0 THEN
+                cm.sales_30d / cm.total_avg_stock_units_30d
+            ELSE NULL -- No stock turn if no average stock
+        END
+    FROM CombinedMetrics cm
+
+    ON CONFLICT (category_id) DO UPDATE SET
+        category_name = EXCLUDED.category_name,
+        category_type = EXCLUDED.category_type,
+        parent_id = EXCLUDED.parent_id,
+        last_calculated = EXCLUDED.last_calculated,
+        
+        -- ROLLED-UP METRICS (includes this category + all descendants)
+        product_count = EXCLUDED.product_count,
+        active_product_count = EXCLUDED.active_product_count,
+        replenishable_product_count = EXCLUDED.replenishable_product_count,
+        current_stock_units = EXCLUDED.current_stock_units,
+        current_stock_cost = EXCLUDED.current_stock_cost,
+        current_stock_retail = EXCLUDED.current_stock_retail,
+        sales_7d = EXCLUDED.sales_7d, revenue_7d = EXCLUDED.revenue_7d,
+        sales_30d = EXCLUDED.sales_30d, revenue_30d = EXCLUDED.revenue_30d,
+        profit_30d = EXCLUDED.profit_30d, cogs_30d = EXCLUDED.cogs_30d,
+        sales_365d = EXCLUDED.sales_365d, revenue_365d = EXCLUDED.revenue_365d,
+        lifetime_sales = EXCLUDED.lifetime_sales, lifetime_revenue = EXCLUDED.lifetime_revenue,
+        
+        -- DIRECT METRICS (only products directly in this category)
+        direct_product_count = EXCLUDED.direct_product_count,
+        direct_active_product_count = EXCLUDED.direct_active_product_count,
+        direct_replenishable_product_count = EXCLUDED.direct_replenishable_product_count,
+        direct_current_stock_units = EXCLUDED.direct_current_stock_units,
+        direct_stock_cost = EXCLUDED.direct_stock_cost,
+        direct_stock_retail = EXCLUDED.direct_stock_retail,
+        direct_sales_7d = EXCLUDED.direct_sales_7d, direct_revenue_7d = EXCLUDED.direct_revenue_7d,
+        direct_sales_30d = EXCLUDED.direct_sales_30d, direct_revenue_30d = EXCLUDED.direct_revenue_30d,
+        direct_profit_30d = EXCLUDED.direct_profit_30d, direct_cogs_30d = EXCLUDED.direct_cogs_30d,
+        direct_sales_365d = EXCLUDED.direct_sales_365d, direct_revenue_365d = EXCLUDED.direct_revenue_365d,
+        direct_lifetime_sales = EXCLUDED.direct_lifetime_sales, direct_lifetime_revenue = EXCLUDED.direct_lifetime_revenue,
+        
+        -- Calculated KPIs
+        avg_margin_30d = EXCLUDED.avg_margin_30d,
+        stock_turn_30d = EXCLUDED.stock_turn_30d
+    WHERE -- Only update if at least one value has changed
+        category_metrics.product_count IS DISTINCT FROM EXCLUDED.product_count OR
+        category_metrics.active_product_count IS DISTINCT FROM EXCLUDED.active_product_count OR
+        category_metrics.current_stock_units IS DISTINCT FROM EXCLUDED.current_stock_units OR
+        category_metrics.sales_30d IS DISTINCT FROM EXCLUDED.sales_30d OR
+        category_metrics.revenue_30d IS DISTINCT FROM EXCLUDED.revenue_30d OR
+        category_metrics.lifetime_sales IS DISTINCT FROM EXCLUDED.lifetime_sales OR
+        category_metrics.direct_product_count IS DISTINCT FROM EXCLUDED.direct_product_count OR
+        category_metrics.direct_sales_30d IS DISTINCT FROM EXCLUDED.direct_sales_30d;
+
+    -- Update calculate_status
+    INSERT INTO public.calculate_status (module_name, last_calculation_timestamp)
+    VALUES (_module_name, _start_time)
+    ON CONFLICT (module_name) DO UPDATE SET last_calculation_timestamp = _start_time;
+
+    RAISE NOTICE 'Finished % calculation. Duration: %', _module_name, clock_timestamp() - _start_time;
+END $$;
+
+-- Return metrics about the update operation for tracking
+WITH update_stats AS (
+    SELECT 
+        COUNT(*) as total_categories,
+        COUNT(*) FILTER (WHERE last_calculated >= NOW() - INTERVAL '5 minutes') as rows_processed,
+        COUNT(*) FILTER (WHERE category_type = 11) as main_categories, -- 11 = category
+        COUNT(*) FILTER (WHERE category_type = 12) as subcategories,  -- 12 = subcategory
+        SUM(product_count) as total_products,
+        SUM(active_product_count) as total_active_products,
+        SUM(current_stock_units) as total_stock_units
+    FROM public.category_metrics
+)
+SELECT 
+    rows_processed,
+    total_categories,
+    main_categories,
+    subcategories,
+    total_products::int,
+    total_active_products::int,
+    total_stock_units::int
+FROM update_stats;

inventory-server/scripts/metrics/brand-metrics.js

@@ -1,318 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateBrandMetrics(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Brand metrics calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders: 0,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // Get order count that will be processed
-        const [orderCount] = await connection.query(`
-            SELECT COUNT(*) as count 
-            FROM orders o 
-            WHERE o.canceled = false
-        `);
-        processedOrders = orderCount[0].count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting brand metrics calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // Calculate brand metrics with optimized queries
-        await connection.query(`
-            INSERT INTO brand_metrics (
-                brand,
-                product_count,
-                active_products,
-                total_stock_units,
-                total_stock_cost,
-                total_stock_retail,
-                total_revenue,
-                avg_margin,
-                growth_rate
-            )
-            WITH filtered_products AS (
-                SELECT 
-                    p.*,
-                    CASE 
-                        WHEN p.stock_quantity <= 5000 AND p.stock_quantity >= 0 
-                        THEN p.pid 
-                    END as valid_pid,
-                    CASE 
-                        WHEN p.visible = true 
-                        AND p.stock_quantity <= 5000 
-                        AND p.stock_quantity >= 0 
-                        THEN p.pid 
-                    END as active_pid,
-                    CASE 
-                        WHEN p.stock_quantity IS NULL 
-                        OR p.stock_quantity < 0 
-                        OR p.stock_quantity > 5000 
-                        THEN 0 
-                        ELSE p.stock_quantity 
-                    END as valid_stock
-                FROM products p
-                WHERE p.brand IS NOT NULL
-            ),
-            sales_periods AS (
-                SELECT 
-                    p.brand,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0))) as period_revenue,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0) - p.cost_price)) as period_margin,
-                    COUNT(DISTINCT DATE(o.date)) as period_days,
-                    CASE 
-                        WHEN o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 3 MONTH) THEN 'current'
-                        WHEN o.date BETWEEN DATE_SUB(CURRENT_DATE, INTERVAL 15 MONTH) 
-                            AND DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH) THEN 'previous'
-                    END as period_type
-                FROM filtered_products p
-                JOIN orders o ON p.pid = o.pid 
-                WHERE o.canceled = false
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 15 MONTH)
-                GROUP BY p.brand, period_type
-            ),
-            brand_data AS (
-                SELECT 
-                    p.brand,
-                    COUNT(DISTINCT p.valid_pid) as product_count,
-                    COUNT(DISTINCT p.active_pid) as active_products,
-                    SUM(p.valid_stock) as total_stock_units,
-                    SUM(p.valid_stock * p.cost_price) as total_stock_cost,
-                    SUM(p.valid_stock * p.price) as total_stock_retail,
-                    COALESCE(SUM(o.quantity * (o.price - COALESCE(o.discount, 0))), 0) as total_revenue,
-                    CASE 
-                        WHEN SUM(o.quantity * o.price) > 0 
-                        THEN GREATEST(
-                            -100.0,
-                            LEAST(
-                                100.0,
-                                (
-                                    SUM(o.quantity * o.price) -  -- Use gross revenue (before discounts)
-                                    SUM(o.quantity * COALESCE(p.cost_price, 0))  -- Total costs
-                                ) * 100.0 / 
-                                NULLIF(SUM(o.quantity * o.price), 0)  -- Divide by gross revenue
-                            )
-                        )
-                        ELSE 0 
-                    END as avg_margin
-                FROM filtered_products p
-                LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
-                GROUP BY p.brand
-            )
-            SELECT 
-                bd.brand,
-                bd.product_count,
-                bd.active_products,
-                bd.total_stock_units,
-                bd.total_stock_cost,
-                bd.total_stock_retail,
-                bd.total_revenue,
-                bd.avg_margin,
-                CASE
-                    WHEN MAX(CASE WHEN sp.period_type = 'previous' THEN sp.period_revenue END) = 0 
-                         AND MAX(CASE WHEN sp.period_type = 'current' THEN sp.period_revenue END) > 0 
-                    THEN 100.0
-                    WHEN MAX(CASE WHEN sp.period_type = 'previous' THEN sp.period_revenue END) = 0 
-                    THEN 0.0
-                    ELSE GREATEST(
-                        -100.0,
-                        LEAST(
-                            ((MAX(CASE WHEN sp.period_type = 'current' THEN sp.period_revenue END) - 
-                              MAX(CASE WHEN sp.period_type = 'previous' THEN sp.period_revenue END)) / 
-                             NULLIF(ABS(MAX(CASE WHEN sp.period_type = 'previous' THEN sp.period_revenue END)), 0)) * 100.0,
-                            999.99
-                        )
-                    )
-                END as growth_rate
-            FROM brand_data bd
-            LEFT JOIN sales_periods sp ON bd.brand = sp.brand
-            GROUP BY bd.brand, bd.product_count, bd.active_products, bd.total_stock_units, 
-                     bd.total_stock_cost, bd.total_stock_retail, bd.total_revenue, bd.avg_margin
-            ON DUPLICATE KEY UPDATE
-                product_count = VALUES(product_count),
-                active_products = VALUES(active_products),
-                total_stock_units = VALUES(total_stock_units),
-                total_stock_cost = VALUES(total_stock_cost),
-                total_stock_retail = VALUES(total_stock_retail),
-                total_revenue = VALUES(total_revenue),
-                avg_margin = VALUES(avg_margin),
-                growth_rate = VALUES(growth_rate),
-                last_calculated_at = CURRENT_TIMESTAMP
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.97);
-        outputProgress({
-            status: 'running',
-            operation: 'Brand metrics calculated, starting time-based metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Calculate brand time-based metrics with optimized query
-        await connection.query(`
-            INSERT INTO brand_time_metrics (
-                brand,
-                year,
-                month,
-                product_count,
-                active_products,
-                total_stock_units,
-                total_stock_cost,
-                total_stock_retail,
-                total_revenue,
-                avg_margin
-            )
-            WITH filtered_products AS (
-                SELECT 
-                    p.*,
-                    CASE WHEN p.stock_quantity <= 5000 THEN p.pid END as valid_pid,
-                    CASE WHEN p.visible = true AND p.stock_quantity <= 5000 THEN p.pid END as active_pid,
-                    CASE 
-                        WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 THEN 0 
-                        ELSE p.stock_quantity 
-                    END as valid_stock
-                FROM products p
-                WHERE p.brand IS NOT NULL
-            ),
-            monthly_metrics AS (
-                SELECT 
-                    p.brand,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    COUNT(DISTINCT p.valid_pid) as product_count,
-                    COUNT(DISTINCT p.active_pid) as active_products,
-                    SUM(p.valid_stock) as total_stock_units,
-                    SUM(p.valid_stock * p.cost_price) as total_stock_cost,
-                    SUM(p.valid_stock * p.price) as total_stock_retail,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    CASE 
-                        WHEN SUM(o.quantity * o.price) > 0 
-                        THEN GREATEST(
-                            -100.0,
-                            LEAST(
-                                100.0,
-                                (
-                                    SUM(o.quantity * o.price) -  -- Use gross revenue (before discounts)
-                                    SUM(o.quantity * COALESCE(p.cost_price, 0))  -- Total costs
-                                ) * 100.0 / 
-                                NULLIF(SUM(o.quantity * o.price), 0)  -- Divide by gross revenue
-                            )
-                        )
-                        ELSE 0 
-                    END as avg_margin
-                FROM filtered_products p
-                LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
-                WHERE o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                GROUP BY p.brand, YEAR(o.date), MONTH(o.date)
-            )
-            SELECT *
-            FROM monthly_metrics
-            ON DUPLICATE KEY UPDATE
-                product_count = VALUES(product_count),
-                active_products = VALUES(active_products),
-                total_stock_units = VALUES(total_stock_units),
-                total_stock_cost = VALUES(total_stock_cost),
-                total_stock_retail = VALUES(total_stock_retail),
-                total_revenue = VALUES(total_revenue),
-                avg_margin = VALUES(avg_margin)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.99);
-        outputProgress({
-            status: 'running',
-            operation: 'Brand time-based metrics calculated',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('brand_metrics', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating brand metrics');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateBrandMetrics; 

inventory-server/scripts/metrics/category-metrics.js

@@ -1,522 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateCategoryMetrics(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Category metrics calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders: 0,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // Get order count that will be processed
-        const [orderCount] = await connection.query(`
-            SELECT COUNT(*) as count 
-            FROM orders o 
-            WHERE o.canceled = false
-        `);
-        processedOrders = orderCount[0].count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting category metrics calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-        
-        // First, calculate base category metrics
-        await connection.query(`
-            INSERT INTO category_metrics (
-                category_id,
-                product_count,
-                active_products,
-                total_value,
-                status,
-                last_calculated_at
-            )
-            SELECT 
-                c.cat_id,
-                COUNT(DISTINCT p.pid) as product_count,
-                COUNT(DISTINCT CASE WHEN p.visible = true THEN p.pid END) as active_products,
-                COALESCE(SUM(p.stock_quantity * p.cost_price), 0) as total_value,
-                c.status,
-                NOW() as last_calculated_at
-            FROM categories c
-            LEFT JOIN product_categories pc ON c.cat_id = pc.cat_id
-            LEFT JOIN products p ON pc.pid = p.pid
-            GROUP BY c.cat_id, c.status
-            ON DUPLICATE KEY UPDATE
-                product_count = VALUES(product_count),
-                active_products = VALUES(active_products),
-                total_value = VALUES(total_value),
-                status = VALUES(status),
-                last_calculated_at = VALUES(last_calculated_at)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.90);
-        outputProgress({
-            status: 'running',
-            operation: 'Base category metrics calculated, updating with margin data',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Then update with margin and turnover data
-        await connection.query(`
-            WITH category_sales AS (
-                SELECT 
-                    pc.cat_id,
-                    SUM(o.quantity * o.price) as total_sales,
-                    SUM(o.quantity * (o.price - p.cost_price)) as total_margin,
-                    SUM(o.quantity) as units_sold,
-                    AVG(GREATEST(p.stock_quantity, 0)) as avg_stock,
-                    COUNT(DISTINCT DATE(o.date)) as active_days
-                FROM product_categories pc
-                JOIN products p ON pc.pid = p.pid
-                JOIN orders o ON p.pid = o.pid
-                LEFT JOIN turnover_config tc ON 
-                    (tc.category_id = pc.cat_id AND tc.vendor = p.vendor) OR
-                    (tc.category_id = pc.cat_id AND tc.vendor IS NULL) OR
-                    (tc.category_id IS NULL AND tc.vendor = p.vendor) OR
-                    (tc.category_id IS NULL AND tc.vendor IS NULL)
-                WHERE o.canceled = false 
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL COALESCE(tc.calculation_period_days, 30) DAY)
-                GROUP BY pc.cat_id
-            )
-            UPDATE category_metrics cm
-            JOIN category_sales cs ON cm.category_id = cs.cat_id
-            LEFT JOIN turnover_config tc ON 
-                (tc.category_id = cm.category_id AND tc.vendor IS NULL) OR
-                (tc.category_id IS NULL AND tc.vendor IS NULL)
-            SET 
-                cm.avg_margin = COALESCE(cs.total_margin * 100.0 / NULLIF(cs.total_sales, 0), 0),
-                cm.turnover_rate = CASE 
-                    WHEN cs.avg_stock > 0 AND cs.active_days > 0 
-                    THEN LEAST(
-                        (cs.units_sold / cs.avg_stock) * (365.0 / cs.active_days),
-                        999.99
-                    )
-                    ELSE 0 
-                END,
-                cm.last_calculated_at = NOW()
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.95);
-        outputProgress({
-            status: 'running',
-            operation: 'Margin data updated, calculating growth rates',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Finally update growth rates
-        await connection.query(`
-            WITH current_period AS (
-                SELECT 
-                    pc.cat_id,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0)) / 
-                        (1 + COALESCE(ss.seasonality_factor, 0))) as revenue,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0) - p.cost_price)) as gross_profit,
-                    COUNT(DISTINCT DATE(o.date)) as days
-                FROM product_categories pc
-                JOIN products p ON pc.pid = p.pid
-                JOIN orders o ON p.pid = o.pid
-                LEFT JOIN sales_seasonality ss ON MONTH(o.date) = ss.month
-                WHERE o.canceled = false 
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 3 MONTH)
-                GROUP BY pc.cat_id
-            ),
-            previous_period AS (
-                SELECT 
-                    pc.cat_id,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0)) / 
-                        (1 + COALESCE(ss.seasonality_factor, 0))) as revenue,
-                    COUNT(DISTINCT DATE(o.date)) as days
-                FROM product_categories pc
-                JOIN products p ON pc.pid = p.pid
-                JOIN orders o ON p.pid = o.pid
-                LEFT JOIN sales_seasonality ss ON MONTH(o.date) = ss.month
-                WHERE o.canceled = false 
-                    AND o.date BETWEEN DATE_SUB(CURRENT_DATE, INTERVAL 15 MONTH) 
-                    AND DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                GROUP BY pc.cat_id
-            ),
-            trend_data AS (
-                SELECT 
-                    pc.cat_id,
-                    MONTH(o.date) as month,
-                    SUM(o.quantity * (o.price - COALESCE(o.discount, 0)) / 
-                        (1 + COALESCE(ss.seasonality_factor, 0))) as revenue,
-                    COUNT(DISTINCT DATE(o.date)) as days_in_month
-                FROM product_categories pc
-                JOIN products p ON pc.pid = p.pid
-                JOIN orders o ON p.pid = o.pid
-                LEFT JOIN sales_seasonality ss ON MONTH(o.date) = ss.month
-                WHERE o.canceled = false 
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 15 MONTH)
-                GROUP BY pc.cat_id, MONTH(o.date)
-            ),
-            trend_stats AS (
-                SELECT 
-                    cat_id,
-                    COUNT(*) as n,
-                    AVG(month) as avg_x,
-                    AVG(revenue / NULLIF(days_in_month, 0)) as avg_y,
-                    SUM(month * (revenue / NULLIF(days_in_month, 0))) as sum_xy,
-                    SUM(month * month) as sum_xx
-                FROM trend_data
-                GROUP BY cat_id
-                HAVING COUNT(*) >= 6
-            ),
-            trend_analysis AS (
-                SELECT 
-                    cat_id,
-                    ((n * sum_xy) - (avg_x * n * avg_y)) / 
-                        NULLIF((n * sum_xx) - (n * avg_x * avg_x), 0) as trend_slope,
-                    avg_y as avg_daily_revenue
-                FROM trend_stats
-            ),
-            margin_calc AS (
-                SELECT 
-                    pc.cat_id,
-                    CASE 
-                        WHEN SUM(o.quantity * o.price) > 0 THEN
-                            GREATEST(
-                                -100.0,
-                                LEAST(
-                                    100.0,
-                                    (
-                                        SUM(o.quantity * o.price) -  -- Use gross revenue (before discounts)
-                                        SUM(o.quantity * COALESCE(p.cost_price, 0))  -- Total costs
-                                    ) * 100.0 / 
-                                    NULLIF(SUM(o.quantity * o.price), 0)  -- Divide by gross revenue
-                                )
-                            )
-                        ELSE NULL
-                    END as avg_margin
-                FROM product_categories pc
-                JOIN products p ON pc.pid = p.pid
-                JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false 
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 3 MONTH)
-                GROUP BY pc.cat_id
-            )
-            UPDATE category_metrics cm
-            LEFT JOIN current_period cp ON cm.category_id = cp.cat_id
-            LEFT JOIN previous_period pp ON cm.category_id = pp.cat_id
-            LEFT JOIN trend_analysis ta ON cm.category_id = ta.cat_id
-            LEFT JOIN margin_calc mc ON cm.category_id = mc.cat_id
-            SET 
-                cm.growth_rate = CASE
-                    WHEN pp.revenue = 0 AND COALESCE(cp.revenue, 0) > 0 THEN 100.0
-                    WHEN pp.revenue = 0 OR cp.revenue IS NULL THEN 0.0
-                    WHEN ta.trend_slope IS NOT NULL THEN
-                        GREATEST(
-                            -100.0,
-                            LEAST(
-                                (ta.trend_slope / NULLIF(ta.avg_daily_revenue, 0)) * 365 * 100,
-                                999.99
-                            )
-                        )
-                    ELSE
-                        GREATEST(
-                            -100.0,
-                            LEAST(
-                                ((COALESCE(cp.revenue, 0) - pp.revenue) / 
-                                    NULLIF(ABS(pp.revenue), 0)) * 100.0,
-                                999.99
-                            )
-                        )
-                END,
-                cm.avg_margin = COALESCE(mc.avg_margin, cm.avg_margin),
-                cm.last_calculated_at = NOW()
-            WHERE cp.cat_id IS NOT NULL OR pp.cat_id IS NOT NULL
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.97);
-        outputProgress({
-            status: 'running',
-            operation: 'Growth rates calculated, updating time-based metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Calculate time-based metrics
-        await connection.query(`
-            INSERT INTO category_time_metrics (
-                category_id,
-                year,
-                month,
-                product_count,
-                active_products,
-                total_value,
-                total_revenue,
-                avg_margin,
-                turnover_rate
-            )
-            SELECT 
-                pc.cat_id,
-                YEAR(o.date) as year,
-                MONTH(o.date) as month,
-                COUNT(DISTINCT p.pid) as product_count,
-                COUNT(DISTINCT CASE WHEN p.visible = true THEN p.pid END) as active_products,
-                SUM(p.stock_quantity * p.cost_price) as total_value,
-                SUM(o.quantity * o.price) as total_revenue,
-                CASE
-                    WHEN SUM(o.quantity * o.price) > 0 THEN
-                        LEAST(
-                            GREATEST(
-                                SUM(o.quantity * (o.price - GREATEST(p.cost_price, 0))) * 100.0 / 
-                                SUM(o.quantity * o.price),
-                                -100
-                            ),
-                            100
-                        )
-                    ELSE 0
-                END as avg_margin,
-                COALESCE(
-                    LEAST(
-                        SUM(o.quantity) / NULLIF(AVG(GREATEST(p.stock_quantity, 0)), 0),
-                        999.99
-                    ),
-                    0
-                ) as turnover_rate
-            FROM product_categories pc
-            JOIN products p ON pc.pid = p.pid
-            JOIN orders o ON p.pid = o.pid
-            WHERE o.canceled = false
-                AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-            GROUP BY pc.cat_id, YEAR(o.date), MONTH(o.date)
-            ON DUPLICATE KEY UPDATE
-                product_count = VALUES(product_count),
-                active_products = VALUES(active_products),
-                total_value = VALUES(total_value),
-                total_revenue = VALUES(total_revenue),
-                avg_margin = VALUES(avg_margin),
-                turnover_rate = VALUES(turnover_rate)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.99);
-        outputProgress({
-            status: 'running',
-            operation: 'Time-based metrics calculated, updating category-sales metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Calculate category-sales metrics
-        await connection.query(`
-            INSERT INTO category_sales_metrics (
-                category_id,
-                brand,
-                period_start,
-                period_end,
-                avg_daily_sales,
-                total_sold,
-                num_products,
-                avg_price,
-                last_calculated_at
-            )
-            WITH date_ranges AS (
-                SELECT 
-                    DATE_SUB(CURRENT_DATE, INTERVAL 30 DAY) as period_start,
-                    CURRENT_DATE as period_end
-                UNION ALL
-                SELECT 
-                    DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY),
-                    DATE_SUB(CURRENT_DATE, INTERVAL 31 DAY)
-                UNION ALL
-                SELECT 
-                    DATE_SUB(CURRENT_DATE, INTERVAL 180 DAY),
-                    DATE_SUB(CURRENT_DATE, INTERVAL 91 DAY)
-                UNION ALL
-                SELECT 
-                    DATE_SUB(CURRENT_DATE, INTERVAL 365 DAY),
-                    DATE_SUB(CURRENT_DATE, INTERVAL 181 DAY)
-            ),
-            sales_data AS (
-                SELECT 
-                    pc.cat_id,
-                    COALESCE(p.brand, 'Unknown') as brand,
-                    dr.period_start,
-                    dr.period_end,
-                    COUNT(DISTINCT p.pid) as num_products,
-                    SUM(o.quantity) as total_sold,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    COUNT(DISTINCT DATE(o.date)) as num_days
-                FROM products p
-                JOIN product_categories pc ON p.pid = pc.pid
-                JOIN orders o ON p.pid = o.pid
-                CROSS JOIN date_ranges dr
-                WHERE o.canceled = false
-                    AND o.date BETWEEN dr.period_start AND dr.period_end
-                GROUP BY pc.cat_id, p.brand, dr.period_start, dr.period_end
-            )
-            SELECT 
-                cat_id as category_id,
-                brand,
-                period_start,
-                period_end,
-                CASE 
-                    WHEN num_days > 0 
-                    THEN total_sold / num_days 
-                    ELSE 0 
-                END as avg_daily_sales,
-                total_sold,
-                num_products,
-                CASE 
-                    WHEN total_sold > 0 
-                    THEN total_revenue / total_sold 
-                    ELSE 0 
-                END as avg_price,
-                NOW() as last_calculated_at
-            FROM sales_data
-            ON DUPLICATE KEY UPDATE
-                avg_daily_sales = VALUES(avg_daily_sales),
-                total_sold = VALUES(total_sold),
-                num_products = VALUES(num_products),
-                avg_price = VALUES(avg_price),
-                last_calculated_at = VALUES(last_calculated_at)
-        `);
-
-        processedCount = Math.floor(totalProducts * 1.0);
-        outputProgress({
-            status: 'running',
-            operation: 'Category-sales metrics calculated',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('category_metrics', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating category metrics');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateCategoryMetrics; 

inventory-server/scripts/metrics/financial-metrics.js

@@ -1,192 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateFinancialMetrics(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Financial metrics calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders: 0,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // Get order count that will be processed
-        const [orderCount] = await connection.query(`
-            SELECT COUNT(*) as count 
-            FROM orders o 
-            WHERE o.canceled = false
-                AND DATE(o.date) >= DATE_SUB(CURDATE(), INTERVAL 12 MONTH)
-        `);
-        processedOrders = orderCount[0].count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting financial metrics calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // Calculate financial metrics with optimized query
-        await connection.query(`
-            WITH product_financials AS (
-                SELECT 
-                    p.pid,
-                    p.cost_price * p.stock_quantity as inventory_value,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    SUM(o.quantity * p.cost_price) as cost_of_goods_sold,
-                    SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
-                    MIN(o.date) as first_sale_date,
-                    MAX(o.date) as last_sale_date,
-                    DATEDIFF(MAX(o.date), MIN(o.date)) + 1 as calculation_period_days,
-                    COUNT(DISTINCT DATE(o.date)) as active_days
-                FROM products p
-                LEFT JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false
-                    AND DATE(o.date) >= DATE_SUB(CURDATE(), INTERVAL 12 MONTH)
-                GROUP BY p.pid
-            )
-            UPDATE product_metrics pm
-            JOIN product_financials pf ON pm.pid = pf.pid
-            SET 
-                pm.inventory_value = COALESCE(pf.inventory_value, 0),
-                pm.total_revenue = COALESCE(pf.total_revenue, 0),
-                pm.cost_of_goods_sold = COALESCE(pf.cost_of_goods_sold, 0),
-                pm.gross_profit = COALESCE(pf.gross_profit, 0),
-                pm.gmroi = CASE 
-                    WHEN COALESCE(pf.inventory_value, 0) > 0 AND pf.active_days > 0 THEN 
-                        (COALESCE(pf.gross_profit, 0) * (365.0 / pf.active_days)) / COALESCE(pf.inventory_value, 0)
-                    ELSE 0 
-                END,
-                pm.last_calculated_at = CURRENT_TIMESTAMP
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.65);
-        outputProgress({
-            status: 'running',
-            operation: 'Base financial metrics calculated, updating time aggregates',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Update time-based aggregates with optimized query
-        await connection.query(`
-            WITH monthly_financials AS (
-                SELECT 
-                    p.pid,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    p.cost_price * p.stock_quantity as inventory_value,
-                    SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
-                    COUNT(DISTINCT DATE(o.date)) as active_days,
-                    MIN(o.date) as period_start,
-                    MAX(o.date) as period_end
-                FROM products p
-                LEFT JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false
-                GROUP BY p.pid, YEAR(o.date), MONTH(o.date)
-            )
-            UPDATE product_time_aggregates pta
-            JOIN monthly_financials mf ON pta.pid = mf.pid 
-                AND pta.year = mf.year 
-                AND pta.month = mf.month
-            SET 
-                pta.inventory_value = COALESCE(mf.inventory_value, 0),
-                pta.gmroi = CASE 
-                    WHEN COALESCE(mf.inventory_value, 0) > 0 AND mf.active_days > 0 THEN 
-                        (COALESCE(mf.gross_profit, 0) * (365.0 / mf.active_days)) / COALESCE(mf.inventory_value, 0)
-                    ELSE 0 
-                END
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.70);
-        outputProgress({
-            status: 'running',
-            operation: 'Time-based aggregates updated',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('financial_metrics', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating financial metrics');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateFinancialMetrics; 

inventory-server/scripts/metrics/product-metrics.js

@@ -1,642 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-// Helper function to handle NaN and undefined values
-function sanitizeValue(value) {
-    if (value === undefined || value === null || Number.isNaN(value)) {
-        return null;
-    }
-    return value;
-}
-
-async function calculateProductMetrics(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    const BATCH_SIZE = 5000;
-    
-    try {
-        // Skip flags are inherited from the parent scope
-        const SKIP_PRODUCT_BASE_METRICS = 0;
-        const SKIP_PRODUCT_TIME_AGGREGATES = 0;
-
-        // Get total product count if not provided
-        if (!totalProducts) {
-            const [productCount] = await connection.query('SELECT COUNT(*) as count FROM products');
-            totalProducts = productCount[0].count;
-        }
-
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Product metrics calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // First ensure all products have a metrics record
-        await connection.query(`
-            INSERT IGNORE INTO product_metrics (pid, last_calculated_at)
-            SELECT pid, NOW()
-            FROM products
-        `);
-
-        // Get threshold settings once
-        const [thresholds] = await connection.query(`
-            SELECT critical_days, reorder_days, overstock_days, low_stock_threshold
-            FROM stock_thresholds 
-            WHERE category_id IS NULL AND vendor IS NULL
-            LIMIT 1
-        `);
-        const defaultThresholds = thresholds[0];
-
-        // Calculate base product metrics
-        if (!SKIP_PRODUCT_BASE_METRICS) {
-            outputProgress({
-                status: 'running',
-                operation: 'Starting base product metrics calculation',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedCount, totalProducts),
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            // Get order count that will be processed
-            const [orderCount] = await connection.query(`
-                SELECT COUNT(*) as count 
-                FROM orders o 
-                WHERE o.canceled = false
-            `);
-            processedOrders = orderCount[0].count;
-
-            // Clear temporary tables
-            await connection.query('TRUNCATE TABLE temp_sales_metrics');
-            await connection.query('TRUNCATE TABLE temp_purchase_metrics');
-
-            // Populate temp_sales_metrics with base stats and sales averages
-            await connection.query(`
-                INSERT INTO temp_sales_metrics
-                SELECT 
-                    p.pid,
-                    COALESCE(SUM(o.quantity) / NULLIF(COUNT(DISTINCT DATE(o.date)), 0), 0) as daily_sales_avg,
-                    COALESCE(SUM(o.quantity) / NULLIF(CEIL(COUNT(DISTINCT DATE(o.date)) / 7), 0), 0) as weekly_sales_avg,
-                    COALESCE(SUM(o.quantity) / NULLIF(CEIL(COUNT(DISTINCT DATE(o.date)) / 30), 0), 0) as monthly_sales_avg,
-                    COALESCE(SUM(o.quantity * o.price), 0) as total_revenue,
-                    CASE 
-                        WHEN SUM(o.quantity * o.price) > 0 
-                        THEN ((SUM(o.quantity * o.price) - SUM(o.quantity * p.cost_price)) / SUM(o.quantity * o.price)) * 100 
-                        ELSE 0 
-                    END as avg_margin_percent,
-                    MIN(o.date) as first_sale_date,
-                    MAX(o.date) as last_sale_date
-                FROM products p
-                LEFT JOIN orders o ON p.pid = o.pid 
-                    AND o.canceled = false
-                    AND o.date >= DATE_SUB(CURDATE(), INTERVAL 90 DAY)
-                GROUP BY p.pid
-            `);
-
-            // Populate temp_purchase_metrics
-            await connection.query(`
-                INSERT INTO temp_purchase_metrics
-                SELECT 
-                    p.pid,
-                    AVG(DATEDIFF(po.received_date, po.date)) as avg_lead_time_days,
-                    MAX(po.date) as last_purchase_date,
-                    MIN(po.received_date) as first_received_date,
-                    MAX(po.received_date) as last_received_date
-                FROM products p
-                LEFT JOIN purchase_orders po ON p.pid = po.pid 
-                    AND po.received_date IS NOT NULL
-                    AND po.date >= DATE_SUB(CURDATE(), INTERVAL 365 DAY)
-                GROUP BY p.pid
-            `);
-
-            // Process updates in batches
-            let lastPid = 0;
-            while (true) {
-                if (isCancelled) break;
-
-                const [batch] = await connection.query(
-                    'SELECT pid FROM products WHERE pid > ? ORDER BY pid LIMIT ?',
-                    [lastPid, BATCH_SIZE]
-                );
-
-                if (batch.length === 0) break;
-
-                await connection.query(`
-                    UPDATE product_metrics pm
-                    JOIN products p ON pm.pid = p.pid
-                    LEFT JOIN temp_sales_metrics sm ON pm.pid = sm.pid
-                    LEFT JOIN temp_purchase_metrics lm ON pm.pid = lm.pid
-                    SET 
-                        pm.inventory_value = p.stock_quantity * NULLIF(p.cost_price, 0),
-                        pm.daily_sales_avg = COALESCE(sm.daily_sales_avg, 0),
-                        pm.weekly_sales_avg = COALESCE(sm.weekly_sales_avg, 0),
-                        pm.monthly_sales_avg = COALESCE(sm.monthly_sales_avg, 0),
-                        pm.total_revenue = COALESCE(sm.total_revenue, 0),
-                        pm.avg_margin_percent = COALESCE(sm.avg_margin_percent, 0),
-                        pm.first_sale_date = sm.first_sale_date,
-                        pm.last_sale_date = sm.last_sale_date,
-                        pm.avg_lead_time_days = COALESCE(lm.avg_lead_time_days, 30),
-                        pm.days_of_inventory = CASE 
-                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / NULLIF(sm.daily_sales_avg, 0))
-                            ELSE NULL 
-                        END,
-                        pm.weeks_of_inventory = CASE 
-                            WHEN COALESCE(sm.weekly_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / NULLIF(sm.weekly_sales_avg, 0))
-                            ELSE NULL 
-                        END,
-                        pm.stock_status = CASE
-                            WHEN p.stock_quantity <= 0 THEN 'Out of Stock'
-                            WHEN COALESCE(sm.daily_sales_avg, 0) = 0 AND p.stock_quantity <= ? THEN 'Low Stock'
-                            WHEN COALESCE(sm.daily_sales_avg, 0) = 0 THEN 'In Stock'
-                            WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) <= ? THEN 'Critical'
-                            WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) <= ? THEN 'Reorder'
-                            WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) > ? THEN 'Overstocked'
-                            ELSE 'Healthy'
-                        END,
-                        pm.safety_stock = CASE
-                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
-                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
-                            ELSE ?
-                        END,
-                        pm.reorder_point = CASE
-                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
-                                CEIL(sm.daily_sales_avg * COALESCE(lm.avg_lead_time_days, 30)) + 
-                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
-                            ELSE ?
-                        END,
-                        pm.reorder_qty = CASE
-                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 AND NULLIF(p.cost_price, 0) IS NOT NULL THEN
-                                GREATEST(
-                                    CEIL(SQRT((2 * (sm.daily_sales_avg * 365) * 25) / (NULLIF(p.cost_price, 0) * 0.25))),
-                                    ?
-                                )
-                            ELSE ?
-                        END,
-                        pm.overstocked_amt = CASE
-                            WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) > ? 
-                            THEN GREATEST(0, p.stock_quantity - CEIL(sm.daily_sales_avg * ?))
-                            ELSE 0
-                        END,
-                        pm.last_calculated_at = NOW()
-                    WHERE p.pid IN (${batch.map(() => '?').join(',')})
-                `,
-                [
-                    defaultThresholds.low_stock_threshold,
-                    defaultThresholds.critical_days,
-                    defaultThresholds.reorder_days,
-                    defaultThresholds.overstock_days,
-                    defaultThresholds.low_stock_threshold,
-                    defaultThresholds.low_stock_threshold,
-                    defaultThresholds.low_stock_threshold,
-                    defaultThresholds.low_stock_threshold,
-                    defaultThresholds.overstock_days,
-                    defaultThresholds.overstock_days,
-                    ...batch.map(row => row.pid)
-                ]
-                );
-
-                lastPid = batch[batch.length - 1].pid;
-                processedCount += batch.length;
-
-                outputProgress({
-                    status: 'running',
-                    operation: 'Processing base metrics batch',
-                    current: processedCount,
-                    total: totalProducts,
-                    elapsed: formatElapsedTime(startTime),
-                    remaining: estimateRemaining(startTime, processedCount, totalProducts),
-                    rate: calculateRate(startTime, processedCount),
-                    percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                    timing: {
-                        start_time: new Date(startTime).toISOString(),
-                        end_time: new Date().toISOString(),
-                        elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                    }
-                });
-            }
-
-            // Calculate forecast accuracy and bias in batches
-            lastPid = 0;
-            while (true) {
-                if (isCancelled) break;
-
-                const [batch] = await connection.query(
-                    'SELECT pid FROM products WHERE pid > ? ORDER BY pid LIMIT ?',
-                    [lastPid, BATCH_SIZE]
-                );
-
-                if (batch.length === 0) break;
-
-                await connection.query(`
-                    UPDATE product_metrics pm
-                    JOIN (
-                        SELECT 
-                            sf.pid,
-                            AVG(CASE 
-                                WHEN o.quantity > 0 
-                                THEN ABS(sf.forecast_units - o.quantity) / o.quantity * 100
-                                ELSE 100 
-                            END) as avg_forecast_error,
-                            AVG(CASE 
-                                WHEN o.quantity > 0 
-                                THEN (sf.forecast_units - o.quantity) / o.quantity * 100
-                                ELSE 0 
-                            END) as avg_forecast_bias,
-                            MAX(sf.forecast_date) as last_forecast_date
-                        FROM sales_forecasts sf
-                        JOIN orders o ON sf.pid = o.pid 
-                            AND DATE(o.date) = sf.forecast_date
-                        WHERE o.canceled = false
-                            AND sf.forecast_date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
-                            AND sf.pid IN (?)
-                        GROUP BY sf.pid
-                    ) fa ON pm.pid = fa.pid
-                    SET 
-                        pm.forecast_accuracy = GREATEST(0, 100 - LEAST(fa.avg_forecast_error, 100)),
-                        pm.forecast_bias = GREATEST(-100, LEAST(fa.avg_forecast_bias, 100)),
-                        pm.last_forecast_date = fa.last_forecast_date,
-                        pm.last_calculated_at = NOW()
-                    WHERE pm.pid IN (?)
-                `, [batch.map(row => row.pid), batch.map(row => row.pid)]);
-
-                lastPid = batch[batch.length - 1].pid;
-            }
-        }
-
-        // Calculate product time aggregates
-        if (!SKIP_PRODUCT_TIME_AGGREGATES) {
-            outputProgress({
-                status: 'running',
-                operation: 'Starting product time aggregates calculation',
-                current: processedCount || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedCount || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedCount || 0),
-                percentage: (((processedCount || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-
-            // Calculate time-based aggregates
-            await connection.query(`
-                INSERT INTO product_time_aggregates (
-                    pid,
-                    year,
-                    month,
-                    total_quantity_sold,
-                    total_revenue,
-                    total_cost,
-                    order_count,
-                    avg_price,
-                    profit_margin,
-                    inventory_value,
-                    gmroi
-                )
-                SELECT 
-                    p.pid,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    SUM(o.quantity) as total_quantity_sold,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    SUM(o.quantity * p.cost_price) as total_cost,
-                    COUNT(DISTINCT o.order_number) as order_count,
-                    AVG(o.price) as avg_price,
-                    CASE 
-                        WHEN SUM(o.quantity * o.price) > 0 
-                        THEN ((SUM(o.quantity * o.price) - SUM(o.quantity * p.cost_price)) / SUM(o.quantity * o.price)) * 100
-                        ELSE 0 
-                    END as profit_margin,
-                    p.cost_price * p.stock_quantity as inventory_value,
-                    CASE 
-                        WHEN p.cost_price * p.stock_quantity > 0 
-                        THEN (SUM(o.quantity * (o.price - p.cost_price))) / (p.cost_price * p.stock_quantity)
-                        ELSE 0 
-                    END as gmroi
-                FROM products p
-                LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
-                WHERE o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                GROUP BY p.pid, YEAR(o.date), MONTH(o.date)
-                ON DUPLICATE KEY UPDATE
-                    total_quantity_sold = VALUES(total_quantity_sold),
-                    total_revenue = VALUES(total_revenue),
-                    total_cost = VALUES(total_cost),
-                    order_count = VALUES(order_count),
-                    avg_price = VALUES(avg_price),
-                    profit_margin = VALUES(profit_margin),
-                    inventory_value = VALUES(inventory_value),
-                    gmroi = VALUES(gmroi)
-            `);
-
-            processedCount = Math.floor(totalProducts * 0.6);
-            outputProgress({
-                status: 'running',
-                operation: 'Product time aggregates calculated',
-                current: processedCount || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedCount || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedCount || 0),
-                percentage: (((processedCount || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-        } else {
-            processedCount = Math.floor(totalProducts * 0.6);
-            outputProgress({
-                status: 'running',
-                operation: 'Skipping product time aggregates calculation',
-                current: processedCount || 0,
-                total: totalProducts || 0,
-                elapsed: formatElapsedTime(startTime),
-                remaining: estimateRemaining(startTime, processedCount || 0, totalProducts || 0),
-                rate: calculateRate(startTime, processedCount || 0),
-                percentage: (((processedCount || 0) / (totalProducts || 1)) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-        }
-
-        // Calculate ABC classification
-        outputProgress({
-            status: 'running',
-            operation: 'Starting ABC classification',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,  // This module doesn't process POs
-            success
-        };
-
-        const [abcConfig] = await connection.query('SELECT a_threshold, b_threshold FROM abc_classification_config WHERE id = 1');
-        const abcThresholds = abcConfig[0] || { a_threshold: 20, b_threshold: 50 };
-
-        // First, create and populate the rankings table with an index
-        await connection.query('DROP TEMPORARY TABLE IF EXISTS temp_revenue_ranks');
-        await connection.query(`
-            CREATE TEMPORARY TABLE temp_revenue_ranks (
-                pid BIGINT NOT NULL,
-                total_revenue DECIMAL(10,3),
-                rank_num INT,
-                dense_rank_num INT,
-                percentile DECIMAL(5,2),
-                total_count INT,
-                PRIMARY KEY (pid),
-                INDEX (rank_num),
-                INDEX (dense_rank_num),
-                INDEX (percentile)
-            ) ENGINE=MEMORY
-        `);
-
-        // Calculate rankings with proper tie handling
-        await connection.query(`
-            INSERT INTO temp_revenue_ranks
-            WITH revenue_data AS (
-                SELECT 
-                    pid,
-                    total_revenue,
-                    COUNT(*) OVER () as total_count,
-                    PERCENT_RANK() OVER (ORDER BY total_revenue DESC) * 100 as percentile,
-                    RANK() OVER (ORDER BY total_revenue DESC) as rank_num,
-                    DENSE_RANK() OVER (ORDER BY total_revenue DESC) as dense_rank_num
-                FROM product_metrics 
-                WHERE total_revenue > 0
-            )
-            SELECT 
-                pid,
-                total_revenue,
-                rank_num,
-                dense_rank_num,
-                percentile,
-                total_count
-            FROM revenue_data
-        `);
-
-        // Get total count for percentage calculation
-        const [rankingCount] = await connection.query('SELECT MAX(rank_num) as total_count FROM temp_revenue_ranks');
-        const totalCount = rankingCount[0].total_count || 1;
-        const max_rank = totalCount;
-
-        // Process updates in batches
-        let abcProcessedCount = 0;
-        const batchSize = 5000;
-
-        while (true) {
-            if (isCancelled) return {
-                processedProducts: processedCount,
-                processedOrders,
-                processedPurchaseOrders: 0,  // This module doesn't process POs
-                success
-            };
-
-            // Get a batch of PIDs that need updating
-            const [pids] = await connection.query(`
-                SELECT pm.pid 
-                FROM product_metrics pm
-                LEFT JOIN temp_revenue_ranks tr ON pm.pid = tr.pid
-                WHERE pm.abc_class IS NULL 
-                OR pm.abc_class != 
-                    CASE 
-                        WHEN tr.pid IS NULL THEN 'C'
-                        WHEN tr.percentile <= ? THEN 'A'
-                        WHEN tr.percentile <= ? THEN 'B'
-                        ELSE 'C'
-                    END
-                LIMIT ?
-            `, [abcThresholds.a_threshold, abcThresholds.b_threshold, batchSize]);
-
-            if (pids.length === 0) break;
-
-            await connection.query(`
-                UPDATE product_metrics pm
-                LEFT JOIN temp_revenue_ranks tr ON pm.pid = tr.pid
-                SET pm.abc_class = 
-                    CASE 
-                        WHEN tr.pid IS NULL THEN 'C'
-                        WHEN tr.percentile <= ? THEN 'A'
-                        WHEN tr.percentile <= ? THEN 'B'
-                        ELSE 'C'
-                    END,
-                    pm.last_calculated_at = NOW()
-                WHERE pm.pid IN (?)
-            `, [abcThresholds.a_threshold, abcThresholds.b_threshold, pids.map(row => row.pid)]);
-
-            // Now update turnover rate with proper handling of zero inventory periods
-            await connection.query(`
-                UPDATE product_metrics pm
-                JOIN (
-                    SELECT 
-                        o.pid,
-                        SUM(o.quantity) as total_sold,
-                        COUNT(DISTINCT DATE(o.date)) as active_days,
-                        AVG(CASE 
-                            WHEN p.stock_quantity > 0 THEN p.stock_quantity 
-                            ELSE NULL 
-                        END) as avg_nonzero_stock
-                    FROM orders o
-                    JOIN products p ON o.pid = p.pid
-                    WHERE o.canceled = false 
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
-                    AND o.pid IN (?)
-                    GROUP BY o.pid
-                ) sales ON pm.pid = sales.pid
-                SET 
-                    pm.turnover_rate = CASE 
-                        WHEN sales.avg_nonzero_stock > 0 AND sales.active_days > 0 
-                        THEN LEAST(
-                            (sales.total_sold / sales.avg_nonzero_stock) * (365.0 / sales.active_days),
-                            999.99
-                        )
-                        ELSE 0 
-                    END,
-                    pm.last_calculated_at = NOW()
-                WHERE pm.pid IN (?)
-            `, [pids.map(row => row.pid), pids.map(row => row.pid)]);
-        }
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('product_metrics', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount || 0,
-            processedOrders: processedOrders || 0,
-            processedPurchaseOrders: 0,  // This module doesn't process POs
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating product metrics');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-function calculateStockStatus(stock, config, daily_sales_avg, weekly_sales_avg, monthly_sales_avg) {
-    if (stock <= 0) {
-        return 'Out of Stock';
-    }
-
-    // Use the most appropriate sales average based on data quality
-    let sales_avg = daily_sales_avg;
-    if (sales_avg === 0) {
-        sales_avg = weekly_sales_avg / 7;
-    }
-    if (sales_avg === 0) {
-        sales_avg = monthly_sales_avg / 30;
-    }
-
-    if (sales_avg === 0) {
-        return stock <= config.low_stock_threshold ? 'Low Stock' : 'In Stock';
-    }
-
-    const days_of_stock = stock / sales_avg;
-
-    if (days_of_stock <= config.critical_days) {
-        return 'Critical';
-    } else if (days_of_stock <= config.reorder_days) {
-        return 'Reorder';
-    } else if (days_of_stock > config.overstock_days) {
-        return 'Overstocked';
-    }
-
-    return 'Healthy';
-}
-
-function calculateReorderQuantities(stock, stock_status, daily_sales_avg, avg_lead_time, config) {
-    // Calculate safety stock based on service level and lead time
-    const z_score = 1.96; // 95% service level
-    const lead_time = avg_lead_time || config.target_days;
-    const safety_stock = Math.ceil(daily_sales_avg * Math.sqrt(lead_time) * z_score);
-
-    // Calculate reorder point
-    const lead_time_demand = daily_sales_avg * lead_time;
-    const reorder_point = Math.ceil(lead_time_demand + safety_stock);
-
-    // Calculate reorder quantity using EOQ formula if we have the necessary data
-    let reorder_qty = 0;
-    if (daily_sales_avg > 0) {
-        const annual_demand = daily_sales_avg * 365;
-        const order_cost = 25; // Fixed cost per order
-        const holding_cost = config.cost_price * 0.25; // 25% of unit cost as annual holding cost
-        
-        reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost));
-    } else {
-        // If no sales data, use a basic calculation
-        reorder_qty = Math.max(safety_stock, config.low_stock_threshold);
-    }
-
-    // Calculate overstocked amount
-    const overstocked_amt = stock_status === 'Overstocked' ? 
-        stock - Math.ceil(daily_sales_avg * config.overstock_days) : 
-        0;
-
-    return {
-        safety_stock,
-        reorder_point,
-        reorder_qty,
-        overstocked_amt
-    };
-}
-
-module.exports = calculateProductMetrics; 

inventory-server/scripts/metrics/sales-forecasts.js

@@ -1,447 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateSalesForecasts(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Sales forecasts calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders: 0,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // Get order count that will be processed
-        const [orderCount] = await connection.query(`
-            SELECT COUNT(*) as count 
-            FROM orders o 
-            WHERE o.canceled = false
-                AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
-        `);
-        processedOrders = orderCount[0].count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting sales forecasts calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // First, create a temporary table for forecast dates
-        await connection.query(`
-            CREATE TEMPORARY TABLE IF NOT EXISTS temp_forecast_dates (
-                forecast_date DATE,
-                day_of_week INT,
-                month INT,
-                PRIMARY KEY (forecast_date)
-            )
-        `);
-
-        await connection.query(`
-            INSERT INTO temp_forecast_dates
-            SELECT 
-                DATE_ADD(CURRENT_DATE, INTERVAL n DAY) as forecast_date,
-                DAYOFWEEK(DATE_ADD(CURRENT_DATE, INTERVAL n DAY)) as day_of_week,
-                MONTH(DATE_ADD(CURRENT_DATE, INTERVAL n DAY)) as month
-            FROM (
-                SELECT a.N + b.N * 10 as n 
-                FROM 
-                    (SELECT 0 as N UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
-                     SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) a,
-                    (SELECT 0 as N UNION SELECT 1 UNION SELECT 2) b
-                ORDER BY n
-                LIMIT 31
-            ) numbers
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.92);
-        outputProgress({
-            status: 'running',
-            operation: 'Forecast dates prepared, calculating daily sales stats',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Create temporary table for daily sales stats
-        await connection.query(`
-            CREATE TEMPORARY TABLE IF NOT EXISTS temp_daily_sales AS
-            SELECT 
-                o.pid,
-                DAYOFWEEK(o.date) as day_of_week,
-                SUM(o.quantity) as daily_quantity,
-                SUM(o.price * o.quantity) as daily_revenue,
-                COUNT(DISTINCT DATE(o.date)) as day_count
-            FROM orders o
-            WHERE o.canceled = false
-                AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
-            GROUP BY o.pid, DAYOFWEEK(o.date)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.94);
-        outputProgress({
-            status: 'running',
-            operation: 'Daily sales stats calculated, preparing product stats',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Create temporary table for product stats
-        await connection.query(`
-            CREATE TEMPORARY TABLE IF NOT EXISTS temp_product_stats AS
-            SELECT 
-                pid,
-                AVG(daily_revenue) as overall_avg_revenue,
-                SUM(day_count) as total_days
-            FROM temp_daily_sales
-            GROUP BY pid
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.96);
-        outputProgress({
-            status: 'running',
-            operation: 'Product stats prepared, calculating product-level forecasts',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Calculate product-level forecasts
-        await connection.query(`
-            INSERT INTO sales_forecasts (
-                pid,
-                forecast_date,
-                forecast_units,
-                forecast_revenue,
-                confidence_level,
-                last_calculated_at
-            )
-            WITH daily_stats AS (
-                SELECT 
-                    ds.pid,
-                    AVG(ds.daily_quantity) as avg_daily_qty,
-                    STDDEV(ds.daily_quantity) as std_daily_qty,
-                    COUNT(DISTINCT ds.day_count) as data_points,
-                    SUM(ds.day_count) as total_days,
-                    AVG(ds.daily_revenue) as avg_daily_revenue,
-                    STDDEV(ds.daily_revenue) as std_daily_revenue,
-                    MIN(ds.daily_quantity) as min_daily_qty,
-                    MAX(ds.daily_quantity) as max_daily_qty,
-                    -- Calculate variance without using LAG
-                    COALESCE(
-                        STDDEV(ds.daily_quantity) / NULLIF(AVG(ds.daily_quantity), 0),
-                        0
-                    ) as daily_variance_ratio
-                FROM temp_daily_sales ds
-                GROUP BY ds.pid
-                HAVING AVG(ds.daily_quantity) > 0
-            )
-            SELECT 
-                ds.pid,
-                fd.forecast_date,
-                GREATEST(0, 
-                    ROUND(
-                        ds.avg_daily_qty * 
-                        (1 + COALESCE(sf.seasonality_factor, 0)) *
-                        CASE
-                            WHEN ds.std_daily_qty / NULLIF(ds.avg_daily_qty, 0) > 1.5 THEN 0.85
-                            WHEN ds.std_daily_qty / NULLIF(ds.avg_daily_qty, 0) > 1.0 THEN 0.9
-                            WHEN ds.std_daily_qty / NULLIF(ds.avg_daily_qty, 0) > 0.5 THEN 0.95
-                            ELSE 1.0
-                        END,
-                        2
-                    )
-                ) as forecast_units,
-                GREATEST(0, 
-                    ROUND(
-                        COALESCE(
-                            CASE 
-                                WHEN ds.data_points >= 4 THEN ds.avg_daily_revenue
-                                ELSE ps.overall_avg_revenue
-                            END * 
-                            (1 + COALESCE(sf.seasonality_factor, 0)) *
-                            CASE
-                                WHEN ds.std_daily_revenue / NULLIF(ds.avg_daily_revenue, 0) > 1.5 THEN 0.85
-                                WHEN ds.std_daily_revenue / NULLIF(ds.avg_daily_revenue, 0) > 1.0 THEN 0.9
-                                WHEN ds.std_daily_revenue / NULLIF(ds.avg_daily_revenue, 0) > 0.5 THEN 0.95
-                                ELSE 1.0
-                            END,
-                            0
-                        ),
-                        2
-                    )
-                ) as forecast_revenue,
-                CASE
-                    WHEN ds.total_days >= 60 AND ds.daily_variance_ratio < 0.5 THEN 90
-                    WHEN ds.total_days >= 60 THEN 85
-                    WHEN ds.total_days >= 30 AND ds.daily_variance_ratio < 0.5 THEN 80
-                    WHEN ds.total_days >= 30 THEN 75
-                    WHEN ds.total_days >= 14 AND ds.daily_variance_ratio < 0.5 THEN 70
-                    WHEN ds.total_days >= 14 THEN 65
-                    ELSE 60
-                END as confidence_level,
-                NOW() as last_calculated_at
-            FROM daily_stats ds
-            JOIN temp_product_stats ps ON ds.pid = ps.pid
-            CROSS JOIN temp_forecast_dates fd
-            LEFT JOIN sales_seasonality sf ON fd.month = sf.month
-            GROUP BY ds.pid, fd.forecast_date, ps.overall_avg_revenue, sf.seasonality_factor
-            ON DUPLICATE KEY UPDATE
-                forecast_units = VALUES(forecast_units),
-                forecast_revenue = VALUES(forecast_revenue),
-                confidence_level = VALUES(confidence_level),
-                last_calculated_at = NOW()
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.98);
-        outputProgress({
-            status: 'running',
-            operation: 'Product forecasts calculated, preparing category stats',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Create temporary table for category stats
-        await connection.query(`
-            CREATE TEMPORARY TABLE IF NOT EXISTS temp_category_sales AS
-            SELECT 
-                pc.cat_id,
-                DAYOFWEEK(o.date) as day_of_week,
-                SUM(o.quantity) as daily_quantity,
-                SUM(o.price * o.quantity) as daily_revenue,
-                COUNT(DISTINCT DATE(o.date)) as day_count
-            FROM orders o
-            JOIN product_categories pc ON o.pid = pc.pid
-            WHERE o.canceled = false
-                AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
-            GROUP BY pc.cat_id, DAYOFWEEK(o.date)
-        `);
-
-        await connection.query(`
-            CREATE TEMPORARY TABLE IF NOT EXISTS temp_category_stats AS
-            SELECT 
-                cat_id,
-                AVG(daily_revenue) as overall_avg_revenue,
-                SUM(day_count) as total_days
-            FROM temp_category_sales
-            GROUP BY cat_id
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.99);
-        outputProgress({
-            status: 'running',
-            operation: 'Category stats prepared, calculating category-level forecasts',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Calculate category-level forecasts
-        await connection.query(`
-            INSERT INTO category_forecasts (
-                category_id,
-                forecast_date,
-                forecast_units,
-                forecast_revenue,
-                confidence_level,
-                last_calculated_at
-            )
-            SELECT 
-                cs.cat_id as category_id,
-                fd.forecast_date,
-                GREATEST(0, 
-                    AVG(cs.daily_quantity) * 
-                    (1 + COALESCE(sf.seasonality_factor, 0))
-                ) as forecast_units,
-                GREATEST(0, 
-                    COALESCE(
-                        CASE 
-                            WHEN SUM(cs.day_count) >= 4 THEN AVG(cs.daily_revenue)
-                            ELSE ct.overall_avg_revenue
-                        END * 
-                        (1 + COALESCE(sf.seasonality_factor, 0)) *
-                        (0.95 + (RAND() * 0.1)),
-                        0
-                    )
-                ) as forecast_revenue,
-                CASE
-                    WHEN ct.total_days >= 60 THEN 90
-                    WHEN ct.total_days >= 30 THEN 80
-                    WHEN ct.total_days >= 14 THEN 70
-                    ELSE 60
-                END as confidence_level,
-                NOW() as last_calculated_at
-            FROM temp_category_sales cs
-            JOIN temp_category_stats ct ON cs.cat_id = ct.cat_id
-            CROSS JOIN temp_forecast_dates fd
-            LEFT JOIN sales_seasonality sf ON fd.month = sf.month
-            GROUP BY cs.cat_id, fd.forecast_date, ct.overall_avg_revenue, ct.total_days, sf.seasonality_factor
-            HAVING AVG(cs.daily_quantity) > 0
-            ON DUPLICATE KEY UPDATE
-                forecast_units = VALUES(forecast_units),
-                forecast_revenue = VALUES(forecast_revenue),
-                confidence_level = VALUES(confidence_level),
-                last_calculated_at = NOW()
-        `);
-
-        // Clean up temporary tables
-        await connection.query(`
-            DROP TEMPORARY TABLE IF EXISTS temp_forecast_dates;
-            DROP TEMPORARY TABLE IF EXISTS temp_daily_sales;
-            DROP TEMPORARY TABLE IF EXISTS temp_product_stats;
-            DROP TEMPORARY TABLE IF EXISTS temp_category_sales;
-            DROP TEMPORARY TABLE IF EXISTS temp_category_stats;
-        `);
-
-        processedCount = Math.floor(totalProducts * 1.0);
-        outputProgress({
-            status: 'running',
-            operation: 'Category forecasts calculated and temporary tables cleaned up',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('sales_forecasts', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating sales forecasts');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateSalesForecasts; 

inventory-server/scripts/metrics/time-aggregates.js

@@ -1,304 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateTimeAggregates(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Time aggregates calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders: 0,
-                processedPurchaseOrders: 0,
-                success
-            };
-        }
-
-        // Get order count that will be processed
-        const [orderCount] = await connection.query(`
-            SELECT COUNT(*) as count 
-            FROM orders o 
-            WHERE o.canceled = false
-        `);
-        processedOrders = orderCount[0].count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting time aggregates calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // Initial insert of time-based aggregates
-        await connection.query(`
-            INSERT INTO product_time_aggregates (
-                pid,
-                year,
-                month,
-                total_quantity_sold,
-                total_revenue,
-                total_cost,
-                order_count,
-                stock_received,
-                stock_ordered,
-                avg_price,
-                profit_margin,
-                inventory_value,
-                gmroi
-            )
-            WITH monthly_sales AS (
-                SELECT 
-                    o.pid,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    SUM(o.quantity) as total_quantity_sold,
-                    SUM((o.price - COALESCE(o.discount, 0)) * o.quantity) as total_revenue,
-                    SUM(COALESCE(p.cost_price, 0) * o.quantity) as total_cost,
-                    COUNT(DISTINCT o.order_number) as order_count,
-                    AVG(o.price - COALESCE(o.discount, 0)) as avg_price,
-                    CASE 
-                        WHEN SUM((o.price - COALESCE(o.discount, 0)) * o.quantity) > 0 
-                        THEN ((SUM((o.price - COALESCE(o.discount, 0)) * o.quantity) - SUM(COALESCE(p.cost_price, 0) * o.quantity)) 
-                            / SUM((o.price - COALESCE(o.discount, 0)) * o.quantity)) * 100
-                        ELSE 0 
-                    END as profit_margin,
-                    p.cost_price * p.stock_quantity as inventory_value,
-                    COUNT(DISTINCT DATE(o.date)) as active_days
-                FROM orders o
-                JOIN products p ON o.pid = p.pid
-                WHERE o.canceled = false
-                GROUP BY o.pid, YEAR(o.date), MONTH(o.date)
-            ),
-            monthly_stock AS (
-                SELECT 
-                    pid,
-                    YEAR(date) as year,
-                    MONTH(date) as month,
-                    SUM(received) as stock_received,
-                    SUM(ordered) as stock_ordered
-                FROM purchase_orders
-                GROUP BY pid, YEAR(date), MONTH(date)
-            ),
-            base_products AS (
-                SELECT 
-                    p.pid,
-                    p.cost_price * p.stock_quantity as inventory_value
-                FROM products p
-            )
-            SELECT 
-                COALESCE(s.pid, ms.pid) as pid,
-                COALESCE(s.year, ms.year) as year,
-                COALESCE(s.month, ms.month) as month,
-                COALESCE(s.total_quantity_sold, 0) as total_quantity_sold,
-                COALESCE(s.total_revenue, 0) as total_revenue,
-                COALESCE(s.total_cost, 0) as total_cost,
-                COALESCE(s.order_count, 0) as order_count,
-                COALESCE(ms.stock_received, 0) as stock_received,
-                COALESCE(ms.stock_ordered, 0) as stock_ordered,
-                COALESCE(s.avg_price, 0) as avg_price,
-                COALESCE(s.profit_margin, 0) as profit_margin,
-                COALESCE(s.inventory_value, bp.inventory_value, 0) as inventory_value,
-                CASE 
-                    WHEN COALESCE(s.inventory_value, bp.inventory_value, 0) > 0 
-                        AND COALESCE(s.active_days, 0) > 0 
-                    THEN (COALESCE(s.total_revenue - s.total_cost, 0) * (365.0 / s.active_days)) 
-                        / COALESCE(s.inventory_value, bp.inventory_value)
-                    ELSE 0 
-                END as gmroi
-            FROM (
-                SELECT * FROM monthly_sales s
-                UNION ALL
-                SELECT 
-                    ms.pid,
-                    ms.year,
-                    ms.month,
-                    0 as total_quantity_sold,
-                    0 as total_revenue,
-                    0 as total_cost,
-                    0 as order_count,
-                    NULL as avg_price,
-                    0 as profit_margin,
-                    NULL as inventory_value,
-                    0 as active_days
-                FROM monthly_stock ms
-                WHERE NOT EXISTS (
-                    SELECT 1 FROM monthly_sales s2
-                    WHERE s2.pid = ms.pid
-                    AND s2.year = ms.year
-                    AND s2.month = ms.month
-                )
-            ) s
-            LEFT JOIN monthly_stock ms 
-                ON s.pid = ms.pid 
-                AND s.year = ms.year 
-                AND s.month = ms.month
-            JOIN base_products bp ON COALESCE(s.pid, ms.pid) = bp.pid
-            UNION
-            SELECT 
-                ms.pid,
-                ms.year,
-                ms.month,
-                0 as total_quantity_sold,
-                0 as total_revenue,
-                0 as total_cost,
-                0 as order_count,
-                ms.stock_received,
-                ms.stock_ordered,
-                0 as avg_price,
-                0 as profit_margin,
-                bp.inventory_value,
-                0 as gmroi
-            FROM monthly_stock ms
-            JOIN base_products bp ON ms.pid = bp.pid
-            WHERE NOT EXISTS (
-                SELECT 1 FROM (
-                    SELECT * FROM monthly_sales
-                    UNION ALL
-                    SELECT 
-                        ms2.pid,
-                        ms2.year,
-                        ms2.month,
-                        0, 0, 0, 0, NULL, 0, NULL, 0
-                    FROM monthly_stock ms2
-                    WHERE NOT EXISTS (
-                        SELECT 1 FROM monthly_sales s2
-                        WHERE s2.pid = ms2.pid
-                        AND s2.year = ms2.year
-                        AND s2.month = ms2.month
-                    )
-                ) s
-                WHERE s.pid = ms.pid
-                AND s.year = ms.year
-                AND s.month = ms.month
-            )
-            ON DUPLICATE KEY UPDATE
-                total_quantity_sold = VALUES(total_quantity_sold),
-                total_revenue = VALUES(total_revenue),
-                total_cost = VALUES(total_cost),
-                order_count = VALUES(order_count),
-                stock_received = VALUES(stock_received),
-                stock_ordered = VALUES(stock_ordered),
-                avg_price = VALUES(avg_price),
-                profit_margin = VALUES(profit_margin),
-                inventory_value = VALUES(inventory_value),
-                gmroi = VALUES(gmroi)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.60);
-        outputProgress({
-            status: 'running',
-            operation: 'Base time aggregates calculated, updating financial metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-        // Update with financial metrics
-        await connection.query(`
-            UPDATE product_time_aggregates pta
-            JOIN (
-                SELECT 
-                    p.pid,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    p.cost_price * p.stock_quantity as inventory_value,
-                    SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
-                    COUNT(DISTINCT DATE(o.date)) as active_days
-                FROM products p
-                LEFT JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false
-                GROUP BY p.pid, YEAR(o.date), MONTH(o.date)
-            ) fin ON pta.pid = fin.pid 
-                AND pta.year = fin.year 
-                AND pta.month = fin.month
-            SET 
-                pta.inventory_value = COALESCE(fin.inventory_value, 0)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.65);
-        outputProgress({
-            status: 'running',
-            operation: 'Financial metrics updated',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('time_aggregates', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders: 0,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating time aggregates');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateTimeAggregates; 

inventory-server/scripts/metrics/utils/db.js

@@ -1,51 +0,0 @@
-const mysql = require('mysql2/promise');
-const path = require('path');
-require('dotenv').config({ path: path.resolve(__dirname, '../../..', '.env') });
-
-// Database configuration
-const dbConfig = {
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    waitForConnections: true,
-    connectionLimit: 10,
-    queueLimit: 0,
-    // Add performance optimizations
-    namedPlaceholders: true,
-    maxPreparedStatements: 256,
-    enableKeepAlive: true,
-    keepAliveInitialDelay: 0,
-    // Add memory optimizations
-    flags: [
-        'FOUND_ROWS',
-        'LONG_PASSWORD',
-        'PROTOCOL_41',
-        'TRANSACTIONS',
-        'SECURE_CONNECTION',
-        'MULTI_RESULTS',
-        'PS_MULTI_RESULTS',
-        'PLUGIN_AUTH',
-        'CONNECT_ATTRS',
-        'PLUGIN_AUTH_LENENC_CLIENT_DATA',
-        'SESSION_TRACK',
-        'MULTI_STATEMENTS'
-    ]
-};
-
-// Create a single pool instance to be reused
-const pool = mysql.createPool(dbConfig);
-
-async function getConnection() {
-    return await pool.getConnection();
-}
-
-async function closePool() {
-    await pool.end();
-}
-
-module.exports = {
-    dbConfig,
-    getConnection,
-    closePool
-}; 

inventory-server/scripts/metrics/utils/progress.js

@@ -1,158 +0,0 @@
-const fs = require('fs');
-const path = require('path');
-
-// Helper function to format elapsed time
-function formatElapsedTime(elapsed) {
-    // If elapsed is a timestamp, convert to elapsed milliseconds
-    if (elapsed instanceof Date || elapsed > 1000000000000) {
-        elapsed = Date.now() - elapsed;
-    } else {
-        // If elapsed is in seconds, convert to milliseconds
-        elapsed = elapsed * 1000;
-    }
-    
-    const seconds = Math.floor(elapsed / 1000);
-    const minutes = Math.floor(seconds / 60);
-    const hours = Math.floor(minutes / 60);
-    
-    if (hours > 0) {
-        return `${hours}h ${minutes % 60}m`;
-    } else if (minutes > 0) {
-        return `${minutes}m ${seconds % 60}s`;
-    } else {
-        return `${seconds}s`;
-    }
-}
-
-// Helper function to estimate remaining time
-function estimateRemaining(startTime, current, total) {
-    if (current === 0) return null;
-    const elapsed = Date.now() - startTime;
-    const rate = current / elapsed;
-    const remaining = (total - current) / rate;
-    
-    const minutes = Math.floor(remaining / 60000);
-    const seconds = Math.floor((remaining % 60000) / 1000);
-    
-    if (minutes > 0) {
-        return `${minutes}m ${seconds}s`;
-    } else {
-        return `${seconds}s`;
-    }
-}
-
-// Helper function to calculate rate
-function calculateRate(startTime, current) {
-    const elapsed = (Date.now() - startTime) / 1000; // Convert to seconds
-    return elapsed > 0 ? Math.round(current / elapsed) : 0;
-}
-
-// Set up logging
-const LOG_DIR = path.join(__dirname, '../../../logs');
-const ERROR_LOG = path.join(LOG_DIR, 'import-errors.log');
-const IMPORT_LOG = path.join(LOG_DIR, 'import.log');
-const STATUS_FILE = path.join(LOG_DIR, 'metrics-status.json');
-
-// Ensure log directory exists
-if (!fs.existsSync(LOG_DIR)) {
-    fs.mkdirSync(LOG_DIR, { recursive: true });
-}
-
-// Helper function to log errors
-function logError(error, context = '') {
-    const timestamp = new Date().toISOString();
-    const errorMessage = `[${timestamp}] ${context}\nError: ${error.message}\nStack: ${error.stack}\n\n`;
-    
-    // Log to error file
-    fs.appendFileSync(ERROR_LOG, errorMessage);
-    
-    // Also log to console
-    console.error(`\n${context}\nError: ${error.message}`);
-}
-
-// Helper function to log import progress
-function logImport(message) {
-    const timestamp = new Date().toISOString();
-    const logMessage = `[${timestamp}] ${message}\n`;
-    fs.appendFileSync(IMPORT_LOG, logMessage);
-}
-
-// Helper function to output progress
-function outputProgress(data) {
-    // Save progress to file for resumption
-    saveProgress(data);
-    // Format as SSE event
-    const event = {
-        progress: data
-    };
-    // Always send to stdout for frontend
-    process.stdout.write(JSON.stringify(event) + '\n');
-    
-    // Log significant events to disk
-    const isSignificant = 
-        // Operation starts
-        (data.operation && !data.current) ||
-        // Operation completions and errors
-        data.status === 'complete' || 
-        data.status === 'error' ||
-        // Major phase changes
-        data.operation?.includes('Starting ABC classification') ||
-        data.operation?.includes('Starting time-based aggregates') ||
-        data.operation?.includes('Starting vendor metrics');
-    
-    if (isSignificant) {
-        logImport(`${data.operation || 'Operation'}${data.message ? ': ' + data.message : ''}${data.error ? ' Error: ' + data.error : ''}${data.status ? ' Status: ' + data.status : ''}`);
-    }
-}
-
-function saveProgress(progress) {
-    try {
-        fs.writeFileSync(STATUS_FILE, JSON.stringify({
-            ...progress,
-            timestamp: Date.now()
-        }));
-    } catch (err) {
-        console.error('Failed to save progress:', err);
-    }
-}
-
-function clearProgress() {
-    try {
-        if (fs.existsSync(STATUS_FILE)) {
-            fs.unlinkSync(STATUS_FILE);
-        }
-    } catch (err) {
-        console.error('Failed to clear progress:', err);
-    }
-}
-
-function getProgress() {
-    try {
-        if (fs.existsSync(STATUS_FILE)) {
-            const progress = JSON.parse(fs.readFileSync(STATUS_FILE, 'utf8'));
-            // Check if the progress is still valid (less than 1 hour old)
-            if (progress.timestamp && Date.now() - progress.timestamp < 3600000) {
-                return progress;
-            } else {
-                // Clear old progress
-                clearProgress();
-            }
-        }
-    } catch (err) {
-        console.error('Failed to read progress:', err);
-        clearProgress();
-    }
-    return null;
-}
-
-module.exports = {
-    formatElapsedTime,
-    estimateRemaining,
-    calculateRate,
-    logError,
-    logImport,
-    outputProgress,
-    saveProgress,
-    clearProgress,
-    getProgress
-}; 

inventory-server/scripts/metrics/vendor-metrics.js

@@ -1,368 +0,0 @@
-const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate, logError } = require('./utils/progress');
-const { getConnection } = require('./utils/db');
-
-async function calculateVendorMetrics(startTime, totalProducts, processedCount = 0, isCancelled = false) {
-    const connection = await getConnection();
-    let success = false;
-    let processedOrders = 0;
-    let processedPurchaseOrders = 0;
-    
-    try {
-        if (isCancelled) {
-            outputProgress({
-                status: 'cancelled',
-                operation: 'Vendor metrics calculation cancelled',
-                current: processedCount,
-                total: totalProducts,
-                elapsed: formatElapsedTime(startTime),
-                remaining: null,
-                rate: calculateRate(startTime, processedCount),
-                percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-                timing: {
-                    start_time: new Date(startTime).toISOString(),
-                    end_time: new Date().toISOString(),
-                    elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-                }
-            });
-            return {
-                processedProducts: processedCount,
-                processedOrders,
-                processedPurchaseOrders,
-                success
-            };
-        }
-
-        // Get counts of records that will be processed
-        const [[orderCount], [poCount]] = await Promise.all([
-            connection.query(`
-                SELECT COUNT(*) as count 
-                FROM orders o 
-                WHERE o.canceled = false
-            `),
-            connection.query(`
-                SELECT COUNT(*) as count 
-                FROM purchase_orders po 
-                WHERE po.status != 0
-            `)
-        ]);
-        processedOrders = orderCount.count;
-        processedPurchaseOrders = poCount.count;
-
-        outputProgress({
-            status: 'running',
-            operation: 'Starting vendor metrics calculation',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // First ensure all vendors exist in vendor_details
-        await connection.query(`
-            INSERT IGNORE INTO vendor_details (vendor, status, created_at, updated_at)
-            SELECT DISTINCT 
-                vendor,
-                'active' as status,
-                NOW() as created_at,
-                NOW() as updated_at
-            FROM products
-            WHERE vendor IS NOT NULL
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.8);
-        outputProgress({
-            status: 'running',
-            operation: 'Vendor details updated, calculating metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders,
-            success
-        };
-
-        // Now calculate vendor metrics
-        await connection.query(`
-            INSERT INTO vendor_metrics (
-                vendor,
-                total_revenue,
-                total_orders,
-                total_late_orders,
-                avg_lead_time_days,
-                on_time_delivery_rate,
-                order_fill_rate,
-                avg_order_value,
-                active_products,
-                total_products,
-                total_purchase_value,
-                avg_margin_percent,
-                status,
-                last_calculated_at
-            )
-            WITH vendor_sales AS (
-                SELECT 
-                    p.vendor,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    COUNT(DISTINCT o.id) as total_orders,
-                    COUNT(DISTINCT p.pid) as active_products,
-                    SUM(o.quantity * (o.price - p.cost_price)) as total_margin
-                FROM products p
-                JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                GROUP BY p.vendor
-            ),
-            vendor_po AS (
-                SELECT 
-                    p.vendor,
-                    COUNT(DISTINCT CASE WHEN po.receiving_status = 40 THEN po.id END) as received_orders,
-                    COUNT(DISTINCT po.id) as total_orders,
-                    AVG(CASE 
-                        WHEN po.receiving_status = 40 
-                        THEN DATEDIFF(po.received_date, po.date) 
-                    END) as avg_lead_time_days,
-                    SUM(po.ordered * po.po_cost_price) as total_purchase_value
-                FROM products p
-                JOIN purchase_orders po ON p.pid = po.pid
-                WHERE po.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                GROUP BY p.vendor
-            ),
-            vendor_products AS (
-                SELECT 
-                    vendor,
-                    COUNT(DISTINCT pid) as total_products
-                FROM products
-                GROUP BY vendor
-            )
-            SELECT 
-                vs.vendor,
-                COALESCE(vs.total_revenue, 0) as total_revenue,
-                COALESCE(vp.total_orders, 0) as total_orders,
-                COALESCE(vp.total_orders - vp.received_orders, 0) as total_late_orders,
-                COALESCE(vp.avg_lead_time_days, 0) as avg_lead_time_days,
-                CASE 
-                    WHEN vp.total_orders > 0 
-                    THEN (vp.received_orders / vp.total_orders) * 100 
-                    ELSE 0 
-                END as on_time_delivery_rate,
-                CASE 
-                    WHEN vp.total_orders > 0 
-                    THEN (vp.received_orders / vp.total_orders) * 100 
-                    ELSE 0 
-                END as order_fill_rate,
-                CASE 
-                    WHEN vs.total_orders > 0 
-                    THEN vs.total_revenue / vs.total_orders 
-                    ELSE 0 
-                END as avg_order_value,
-                COALESCE(vs.active_products, 0) as active_products,
-                COALESCE(vpr.total_products, 0) as total_products,
-                COALESCE(vp.total_purchase_value, 0) as total_purchase_value,
-                CASE 
-                    WHEN vs.total_revenue > 0 
-                    THEN (vs.total_margin / vs.total_revenue) * 100 
-                    ELSE 0 
-                END as avg_margin_percent,
-                'active' as status,
-                NOW() as last_calculated_at
-            FROM vendor_sales vs
-            LEFT JOIN vendor_po vp ON vs.vendor = vp.vendor
-            LEFT JOIN vendor_products vpr ON vs.vendor = vpr.vendor
-            WHERE vs.vendor IS NOT NULL
-            ON DUPLICATE KEY UPDATE
-                total_revenue = VALUES(total_revenue),
-                total_orders = VALUES(total_orders),
-                total_late_orders = VALUES(total_late_orders),
-                avg_lead_time_days = VALUES(avg_lead_time_days),
-                on_time_delivery_rate = VALUES(on_time_delivery_rate),
-                order_fill_rate = VALUES(order_fill_rate),
-                avg_order_value = VALUES(avg_order_value),
-                active_products = VALUES(active_products),
-                total_products = VALUES(total_products),
-                total_purchase_value = VALUES(total_purchase_value),
-                avg_margin_percent = VALUES(avg_margin_percent),
-                status = VALUES(status),
-                last_calculated_at = VALUES(last_calculated_at)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.9);
-        outputProgress({
-            status: 'running',
-            operation: 'Vendor metrics calculated, updating time-based metrics',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        if (isCancelled) return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders,
-            success
-        };
-
-        // Calculate time-based metrics
-        await connection.query(`
-            INSERT INTO vendor_time_metrics (
-                vendor,
-                year,
-                month,
-                total_orders,
-                late_orders,
-                avg_lead_time_days,
-                total_purchase_value,
-                total_revenue,
-                avg_margin_percent
-            )
-            WITH monthly_orders AS (
-                SELECT 
-                    p.vendor,
-                    YEAR(o.date) as year,
-                    MONTH(o.date) as month,
-                    COUNT(DISTINCT o.id) as total_orders,
-                    SUM(o.quantity * o.price) as total_revenue,
-                    SUM(o.quantity * (o.price - p.cost_price)) as total_margin
-                FROM products p
-                JOIN orders o ON p.pid = o.pid
-                WHERE o.canceled = false
-                    AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                    AND p.vendor IS NOT NULL
-                GROUP BY p.vendor, YEAR(o.date), MONTH(o.date)
-            ),
-            monthly_po AS (
-                SELECT 
-                    p.vendor,
-                    YEAR(po.date) as year,
-                    MONTH(po.date) as month,
-                    COUNT(DISTINCT po.id) as total_po,
-                    COUNT(DISTINCT CASE 
-                        WHEN po.receiving_status = 40 AND po.received_date > po.expected_date 
-                        THEN po.id 
-                    END) as late_orders,
-                    AVG(CASE 
-                        WHEN po.receiving_status = 40 
-                        THEN DATEDIFF(po.received_date, po.date)
-                    END) as avg_lead_time_days,
-                    SUM(po.ordered * po.po_cost_price) as total_purchase_value
-                FROM products p
-                JOIN purchase_orders po ON p.pid = po.pid
-                WHERE po.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
-                    AND p.vendor IS NOT NULL
-                GROUP BY p.vendor, YEAR(po.date), MONTH(po.date)
-            )
-            SELECT 
-                mo.vendor,
-                mo.year,
-                mo.month,
-                COALESCE(mp.total_po, 0) as total_orders,
-                COALESCE(mp.late_orders, 0) as late_orders,
-                COALESCE(mp.avg_lead_time_days, 0) as avg_lead_time_days,
-                COALESCE(mp.total_purchase_value, 0) as total_purchase_value,
-                mo.total_revenue,
-                CASE 
-                    WHEN mo.total_revenue > 0 
-                    THEN (mo.total_margin / mo.total_revenue) * 100 
-                    ELSE 0 
-                END as avg_margin_percent
-            FROM monthly_orders mo
-            LEFT JOIN monthly_po mp ON mo.vendor = mp.vendor 
-                AND mo.year = mp.year 
-                AND mo.month = mp.month
-            UNION
-            SELECT 
-                mp.vendor,
-                mp.year,
-                mp.month,
-                mp.total_po as total_orders,
-                mp.late_orders,
-                mp.avg_lead_time_days,
-                mp.total_purchase_value,
-                0 as total_revenue,
-                0 as avg_margin_percent
-            FROM monthly_po mp
-            LEFT JOIN monthly_orders mo ON mp.vendor = mo.vendor 
-                AND mp.year = mo.year 
-                AND mp.month = mo.month
-            WHERE mo.vendor IS NULL
-            ON DUPLICATE KEY UPDATE
-                total_orders = VALUES(total_orders),
-                late_orders = VALUES(late_orders),
-                avg_lead_time_days = VALUES(avg_lead_time_days),
-                total_purchase_value = VALUES(total_purchase_value),
-                total_revenue = VALUES(total_revenue),
-                avg_margin_percent = VALUES(avg_margin_percent)
-        `);
-
-        processedCount = Math.floor(totalProducts * 0.95);
-        outputProgress({
-            status: 'running',
-            operation: 'Time-based vendor metrics calculated',
-            current: processedCount,
-            total: totalProducts,
-            elapsed: formatElapsedTime(startTime),
-            remaining: estimateRemaining(startTime, processedCount, totalProducts),
-            rate: calculateRate(startTime, processedCount),
-            percentage: ((processedCount / totalProducts) * 100).toFixed(1),
-            timing: {
-                start_time: new Date(startTime).toISOString(),
-                end_time: new Date().toISOString(),
-                elapsed_seconds: Math.round((Date.now() - startTime) / 1000)
-            }
-        });
-
-        // If we get here, everything completed successfully
-        success = true;
-        
-        // Update calculate_status
-        await connection.query(`
-            INSERT INTO calculate_status (module_name, last_calculation_timestamp)
-            VALUES ('vendor_metrics', NOW())
-            ON DUPLICATE KEY UPDATE last_calculation_timestamp = NOW()
-        `);
-
-        return {
-            processedProducts: processedCount,
-            processedOrders,
-            processedPurchaseOrders,
-            success
-        };
-
-    } catch (error) {
-        success = false;
-        logError(error, 'Error calculating vendor metrics');
-        throw error;
-    } finally {
-        if (connection) {
-            connection.release();
-        }
-    }
-}
-
-module.exports = calculateVendorMetrics;

inventory-server/scripts/reset-db.js

@@ -1,508 +0,0 @@
-const { Client } = require('pg');
-const path = require('path');
-const dotenv = require('dotenv');
-const fs = require('fs');
-
-dotenv.config({ path: path.join(__dirname, '../.env') });
-
-const dbConfig = {
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    port: process.env.DB_PORT || 5432
-};
-
-// Helper function to output progress in JSON format
-function outputProgress(data) {
-    if (!data.status) {
-        data = {
-            status: 'running',
-            ...data
-        };
-    }
-    console.log(JSON.stringify(data));
-}
-
-// Core tables that must be created
-const CORE_TABLES = [
-    'products',
-    'orders',
-    'purchase_orders',
-    'categories',
-    'product_categories'
-];
-
-// Config tables that must be created
-const CONFIG_TABLES = [
-    'stock_thresholds',
-    'lead_time_thresholds',
-    'sales_velocity_config',
-    'abc_classification_config',
-    'safety_stock_config',
-    'sales_seasonality',
-    'turnover_config'
-];
-
-// Split SQL into individual statements
-function splitSQLStatements(sql) {
-    // First, normalize line endings
-    sql = sql.replace(/\r\n/g, '\n');
-    
-    // Track statement boundaries
-    let statements = [];
-    let currentStatement = '';
-    let inString = false;
-    let stringChar = '';
-    let inDollarQuote = false;
-    let dollarQuoteTag = '';
-    
-    // Process character by character
-    for (let i = 0; i < sql.length; i++) {
-        const char = sql[i];
-        const nextChar = sql[i + 1] || '';
-        
-        // Handle dollar quotes
-        if (char === '$' && !inString) {
-            // Look ahead to find the dollar quote tag
-            let tag = '$';
-            let j = i + 1;
-            while (j < sql.length && sql[j] !== '$') {
-                tag += sql[j];
-                j++;
-            }
-            tag += '$';
-            
-            if (j < sql.length) { // Found closing $
-                if (!inDollarQuote) {
-                    inDollarQuote = true;
-                    dollarQuoteTag = tag;
-                    currentStatement += tag;
-                    i = j;
-                    continue;
-                } else if (sql.substring(i, j + 1) === dollarQuoteTag) {
-                    inDollarQuote = false;
-                    dollarQuoteTag = '';
-                    currentStatement += tag;
-                    i = j;
-                    continue;
-                }
-            }
-        }
-        
-        // Handle string literals (only if not in dollar quote)
-        if (!inDollarQuote && (char === "'" || char === '"') && sql[i - 1] !== '\\') {
-            if (!inString) {
-                inString = true;
-                stringChar = char;
-            } else if (char === stringChar) {
-                inString = false;
-            }
-        }
-        
-        // Handle comments (only if not in string or dollar quote)
-        if (!inString && !inDollarQuote) {
-            if (char === '-' && nextChar === '-') {
-                // Skip to end of line
-                while (i < sql.length && sql[i] !== '\n') i++;
-                continue;
-            }
-            
-            if (char === '/' && nextChar === '*') {
-                // Skip until closing */
-                i += 2;
-                while (i < sql.length && (sql[i] !== '*' || sql[i + 1] !== '/')) i++;
-                i++; // Skip the closing /
-                continue;
-            }
-        }
-        
-        // Handle statement boundaries (only if not in string or dollar quote)
-        if (!inString && !inDollarQuote && char === ';') {
-            if (currentStatement.trim()) {
-                statements.push(currentStatement.trim());
-            }
-            currentStatement = '';
-        } else {
-            currentStatement += char;
-        }
-    }
-    
-    // Add the last statement if it exists
-    if (currentStatement.trim()) {
-        statements.push(currentStatement.trim());
-    }
-    
-    return statements;
-}
-
-async function resetDatabase() {
-    outputProgress({
-        operation: 'Starting database reset',
-        message: 'Connecting to database...'
-    });
-
-    // Debug: Log current directory and file paths
-    outputProgress({
-        operation: 'Debug paths',
-        message: {
-            currentDir: process.cwd(),
-            __dirname: __dirname,
-            schemaPath: path.join(__dirname, '../db/schema.sql')
-        }
-    });
-
-    const client = new Client(dbConfig);
-    await client.connect();
-    
-    try {
-        // Check PostgreSQL version and user
-        outputProgress({
-            operation: 'Checking database',
-            message: 'Verifying PostgreSQL version and user privileges...'
-        });
-
-        const versionResult = await client.query('SELECT version()');
-        const userResult = await client.query('SELECT current_user, current_database()');
-        
-        outputProgress({
-            operation: 'Database info',
-            message: {
-                version: versionResult.rows[0].version,
-                user: userResult.rows[0].current_user,
-                database: userResult.rows[0].current_database
-            }
-        });
-
-        // Get list of all tables in the current database
-        outputProgress({
-            operation: 'Getting table list',
-            message: 'Retrieving all table names...'
-        });
-
-        const tablesResult = await client.query(`
-            SELECT string_agg(tablename, ', ') as tables
-            FROM pg_tables
-            WHERE schemaname = 'public'
-            AND tablename NOT IN ('users', 'permissions', 'user_permissions', 'calculate_history', 'import_history');
-        `);
-
-        if (!tablesResult.rows[0].tables) {
-            outputProgress({
-                operation: 'No tables found',
-                message: 'Database is already empty'
-            });
-        } else {
-            outputProgress({
-                operation: 'Dropping tables',
-                message: 'Dropping all existing tables...'
-            });
-
-            // Disable triggers/foreign key checks
-            await client.query('SET session_replication_role = \'replica\';');
-            
-            // Drop all tables except users
-            const tables = tablesResult.rows[0].tables.split(', ');
-            for (const table of tables) {
-                if (!['users'].includes(table)) {
-                    await client.query(`DROP TABLE IF EXISTS "${table}" CASCADE`);
-                }
-            }
-
-            // Only drop types if we're not preserving history tables
-            const historyTablesExist = await client.query(`
-                SELECT EXISTS (
-                    SELECT FROM pg_tables 
-                    WHERE schemaname = 'public' 
-                    AND tablename IN ('calculate_history', 'import_history')
-                );
-            `);
-
-            if (!historyTablesExist.rows[0].exists) {
-                await client.query('DROP TYPE IF EXISTS calculation_status CASCADE;');
-                await client.query('DROP TYPE IF EXISTS module_name CASCADE;');
-            }
-            
-            // Re-enable triggers/foreign key checks
-            await client.query('SET session_replication_role = \'origin\';');
-        }
-
-        // Create enum types if they don't exist
-        outputProgress({
-            operation: 'Creating enum types',
-            message: 'Setting up required enum types...'
-        });
-
-        // Check if types exist before creating
-        const typesExist = await client.query(`
-            SELECT EXISTS (
-                SELECT 1 FROM pg_type 
-                WHERE typname = 'calculation_status'
-            ) as calc_status_exists,
-            EXISTS (
-                SELECT 1 FROM pg_type 
-                WHERE typname = 'module_name'
-            ) as module_name_exists;
-        `);
-
-        if (!typesExist.rows[0].calc_status_exists) {
-            await client.query(`CREATE TYPE calculation_status AS ENUM ('running', 'completed', 'failed', 'cancelled')`);
-        }
-
-        if (!typesExist.rows[0].module_name_exists) {
-            await client.query(`
-                CREATE TYPE module_name AS ENUM (
-                    'product_metrics',
-                    'time_aggregates',
-                    'financial_metrics',
-                    'vendor_metrics',
-                    'category_metrics',
-                    'brand_metrics',
-                    'sales_forecasts',
-                    'abc_classification'
-                )
-            `);
-        }
-
-        // Read and execute main schema first (core tables)
-        outputProgress({
-            operation: 'Running database setup',
-            message: 'Creating core tables...'
-        });
-        const schemaPath = path.join(__dirname, '../db/schema.sql');
-        
-        // Verify file exists
-        if (!fs.existsSync(schemaPath)) {
-            throw new Error(`Schema file not found at: ${schemaPath}`);
-        }
-
-        const schemaSQL = fs.readFileSync(schemaPath, 'utf8');
-        
-        outputProgress({
-            operation: 'Schema file',
-            message: {
-                path: schemaPath,
-                exists: fs.existsSync(schemaPath),
-                size: fs.statSync(schemaPath).size,
-                firstFewLines: schemaSQL.split('\n').slice(0, 5).join('\n')
-            }
-        });
-
-        // Execute schema statements one at a time
-        const statements = splitSQLStatements(schemaSQL);
-        outputProgress({
-            operation: 'SQL Execution',
-            message: {
-                totalStatements: statements.length,
-                statements: statements.map((stmt, i) => ({
-                    number: i + 1,
-                    preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : '')
-                }))
-            }
-        });
-
-        for (let i = 0; i < statements.length; i++) {
-            const stmt = statements[i];
-            try {
-                const result = await client.query(stmt);
-                
-                // Verify if table was created (if this was a CREATE TABLE statement)
-                if (stmt.trim().toLowerCase().startsWith('create table')) {
-                    const tableName = stmt.match(/create\s+table\s+(?:if\s+not\s+exists\s+)?["]?(\w+)["]?/i)?.[1];
-                    if (tableName) {
-                        const tableExists = await client.query(`
-                            SELECT COUNT(*) as count 
-                            FROM information_schema.tables 
-                            WHERE table_schema = 'public' 
-                            AND table_name = $1
-                        `, [tableName]);
-                        
-                        outputProgress({
-                            operation: 'Table Creation Verification',
-                            message: {
-                                table: tableName,
-                                exists: tableExists.rows[0].count > 0
-                            }
-                        });
-                    }
-                }
-                
-                outputProgress({
-                    operation: 'SQL Progress',
-                    message: {
-                        statement: i + 1,
-                        total: statements.length,
-                        preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : ''),
-                        rowCount: result.rowCount
-                    }
-                });
-            } catch (sqlError) {
-                outputProgress({
-                    status: 'error',
-                    operation: 'SQL Error',
-                    error: sqlError.message,
-                    statement: stmt,
-                    statementNumber: i + 1
-                });
-                throw sqlError;
-            }
-        }
-
-        // Verify core tables were created
-        const existingTables = (await client.query(`
-            SELECT table_name
-            FROM information_schema.tables 
-            WHERE table_schema = 'public'
-        `)).rows.map(t => t.table_name);
-
-        outputProgress({
-            operation: 'Core tables verification',
-            message: {
-                found: existingTables,
-                expected: CORE_TABLES
-            }
-        });
-
-        const missingCoreTables = CORE_TABLES.filter(
-            t => !existingTables.includes(t)
-        );
-
-        if (missingCoreTables.length > 0) {
-            throw new Error(
-                `Failed to create core tables: ${missingCoreTables.join(', ')}`
-            );
-        }
-
-        outputProgress({
-            operation: 'Core tables created',
-            message: `Successfully created tables: ${CORE_TABLES.join(', ')}`
-        });
-
-        // Now read and execute config schema (since core tables exist)
-        outputProgress({
-            operation: 'Running config setup',
-            message: 'Creating configuration tables...'
-        });
-        const configSchemaSQL = fs.readFileSync(
-            path.join(__dirname, '../db/config-schema.sql'),
-            'utf8'
-        );
-        
-        // Execute config schema statements one at a time
-        const configStatements = splitSQLStatements(configSchemaSQL);
-        outputProgress({
-            operation: 'Config SQL Execution',
-            message: {
-                totalStatements: configStatements.length,
-                statements: configStatements.map((stmt, i) => ({
-                    number: i + 1,
-                    preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : '')
-                }))
-            }
-        });
-
-        for (let i = 0; i < configStatements.length; i++) {
-            const stmt = configStatements[i];
-            try {
-                const result = await client.query(stmt);
-                
-                outputProgress({
-                    operation: 'Config SQL Progress',
-                    message: {
-                        statement: i + 1,
-                        total: configStatements.length,
-                        preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : ''),
-                        rowCount: result.rowCount
-                    }
-                });
-            } catch (sqlError) {
-                outputProgress({
-                    status: 'error',
-                    operation: 'Config SQL Error',
-                    error: sqlError.message,
-                    statement: stmt,
-                    statementNumber: i + 1
-                });
-                throw sqlError;
-            }
-        }
-
-        // Read and execute metrics schema (metrics tables)
-        outputProgress({
-            operation: 'Running metrics setup',
-            message: 'Creating metrics tables...'
-        });
-        const metricsSchemaSQL = fs.readFileSync(
-            path.join(__dirname, '../db/metrics-schema.sql'),
-            'utf8'
-        );
-        
-        // Execute metrics schema statements one at a time
-        const metricsStatements = splitSQLStatements(metricsSchemaSQL);
-        outputProgress({
-            operation: 'Metrics SQL Execution',
-            message: {
-                totalStatements: metricsStatements.length,
-                statements: metricsStatements.map((stmt, i) => ({
-                    number: i + 1,
-                    preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : '')
-                }))
-            }
-        });
-
-        for (let i = 0; i < metricsStatements.length; i++) {
-            const stmt = metricsStatements[i];
-            try {
-                const result = await client.query(stmt);
-                
-                outputProgress({
-                    operation: 'Metrics SQL Progress',
-                    message: {
-                        statement: i + 1,
-                        total: metricsStatements.length,
-                        preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : ''),
-                        rowCount: result.rowCount
-                    }
-                });
-            } catch (sqlError) {
-                outputProgress({
-                    status: 'error',
-                    operation: 'Metrics SQL Error',
-                    error: sqlError.message,
-                    statement: stmt,
-                    statementNumber: i + 1
-                });
-                throw sqlError;
-            }
-        }
-
-        outputProgress({
-            status: 'complete',
-            operation: 'Database reset complete',
-            message: 'Database has been reset and all tables recreated'
-        });
-    } catch (error) {
-        outputProgress({
-            status: 'error',
-            operation: 'Failed to reset database',
-            error: error.message,
-            stack: error.stack
-        });
-        process.exit(1);
-    } finally {
-        await client.end();
-    }
-}
-
-// Export if required as a module
-if (typeof module !== 'undefined' && module.exports) {
-    module.exports = resetDatabase;
-}
-
-// Run if called directly
-if (require.main === module) {
-    resetDatabase().catch(error => {
-        console.error('Error:', error);
-        process.exit(1);
-    });
-} 

inventory-server/scripts/reset-metrics.js

@@ -1,316 +0,0 @@
-const { Client } = require('pg');
-const path = require('path');
-const fs = require('fs');
-require('dotenv').config({ path: path.resolve(__dirname, '../.env') });
-
-const dbConfig = {
-    host: process.env.DB_HOST,
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: process.env.DB_NAME,
-    port: process.env.DB_PORT || 5432
-};
-
-function outputProgress(data) {
-    if (!data.status) {
-        data = {
-            status: 'running',
-            ...data
-        };
-    }
-    console.log(JSON.stringify(data));
-}
-
-// Explicitly define all metrics-related tables in dependency order
-const METRICS_TABLES = [
-    'brand_metrics',
-    'brand_time_metrics',
-    'category_forecasts',
-    'category_metrics',
-    'category_sales_metrics',
-    'category_time_metrics',
-    'product_metrics',
-    'product_time_aggregates',
-    'sales_forecasts',
-    'temp_purchase_metrics',
-    'temp_sales_metrics',
-    'vendor_metrics',
-    'vendor_time_metrics',
-    'vendor_details'
-];
-
-// Split SQL into individual statements
-function splitSQLStatements(sql) {
-    sql = sql.replace(/\r\n/g, '\n');
-    let statements = [];
-    let currentStatement = '';
-    let inString = false;
-    let stringChar = '';
-    
-    for (let i = 0; i < sql.length; i++) {
-        const char = sql[i];
-        const nextChar = sql[i + 1] || '';
-        
-        if ((char === "'" || char === '"') && sql[i - 1] !== '\\') {
-            if (!inString) {
-                inString = true;
-                stringChar = char;
-            } else if (char === stringChar) {
-                inString = false;
-            }
-        }
-        
-        if (!inString && char === '-' && nextChar === '-') {
-            while (i < sql.length && sql[i] !== '\n') i++;
-            continue;
-        }
-        
-        if (!inString && char === '/' && nextChar === '*') {
-            i += 2;
-            while (i < sql.length && (sql[i] !== '*' || sql[i + 1] !== '/')) i++;
-            i++;
-            continue;
-        }
-        
-        if (!inString && char === ';') {
-            if (currentStatement.trim()) {
-                statements.push(currentStatement.trim());
-            }
-            currentStatement = '';
-        } else {
-            currentStatement += char;
-        }
-    }
-    
-    if (currentStatement.trim()) {
-        statements.push(currentStatement.trim());
-    }
-    
-    return statements;
-}
-
-async function resetMetrics() {
-    let client;
-    try {
-        outputProgress({
-            operation: 'Starting metrics reset',
-            message: 'Connecting to database...'
-        });
-
-        client = new Client(dbConfig);
-        await client.connect();
-
-        // First verify current state
-        const initialTables = await client.query(`
-            SELECT tablename as name
-            FROM pg_tables 
-            WHERE schemaname = 'public'
-            AND tablename = ANY($1)
-        `, [METRICS_TABLES]);
-
-        outputProgress({
-            operation: 'Initial state',
-            message: `Found ${initialTables.rows.length} existing metrics tables: ${initialTables.rows.map(t => t.name).join(', ')}`
-        });
-
-        // Disable foreign key checks at the start
-        await client.query('SET session_replication_role = \'replica\'');
-
-        // Drop all metrics tables in reverse order to handle dependencies
-        outputProgress({
-            operation: 'Dropping metrics tables',
-            message: 'Removing existing metrics tables...'
-        });
-
-        for (const table of [...METRICS_TABLES].reverse()) {
-            try {
-                await client.query(`DROP TABLE IF EXISTS "${table}" CASCADE`);
-                
-                // Verify the table was actually dropped
-                const checkDrop = await client.query(`
-                    SELECT COUNT(*) as count 
-                    FROM pg_tables 
-                    WHERE schemaname = 'public' 
-                    AND tablename = $1
-                `, [table]);
-                
-                if (parseInt(checkDrop.rows[0].count) > 0) {
-                    throw new Error(`Failed to drop table ${table} - table still exists`);
-                }
-
-                outputProgress({
-                    operation: 'Table dropped',
-                    message: `Successfully dropped table: ${table}`
-                });
-            } catch (err) {
-                outputProgress({
-                    status: 'error',
-                    operation: 'Drop table error',
-                    message: `Error dropping table ${table}: ${err.message}`
-                });
-                throw err;
-            }
-        }
-
-        // Verify all tables were dropped
-        const afterDrop = await client.query(`
-            SELECT tablename as name
-            FROM pg_tables 
-            WHERE schemaname = 'public'
-            AND tablename = ANY($1)
-        `, [METRICS_TABLES]);
-
-        if (afterDrop.rows.length > 0) {
-            throw new Error(`Failed to drop all tables. Remaining tables: ${afterDrop.rows.map(t => t.name).join(', ')}`);
-        }
-
-        // Read metrics schema
-        outputProgress({
-            operation: 'Reading schema',
-            message: 'Loading metrics schema file...'
-        });
-
-        const schemaPath = path.resolve(__dirname, '../db/metrics-schema.sql');
-        if (!fs.existsSync(schemaPath)) {
-            throw new Error(`Schema file not found at: ${schemaPath}`);
-        }
-
-        const schemaSQL = fs.readFileSync(schemaPath, 'utf8');
-        const statements = splitSQLStatements(schemaSQL);
-
-        outputProgress({
-            operation: 'Schema loaded',
-            message: `Found ${statements.length} SQL statements to execute`
-        });
-
-        // Execute schema statements
-        for (let i = 0; i < statements.length; i++) {
-            const stmt = statements[i];
-            try {
-                const result = await client.query(stmt);
-
-                // If this is a CREATE TABLE statement, verify the table was created
-                if (stmt.trim().toLowerCase().startsWith('create table')) {
-                    const tableName = stmt.match(/create\s+table\s+(?:if\s+not\s+exists\s+)?["]?(\w+)["]?/i)?.[1];
-                    if (tableName) {
-                        const checkCreate = await client.query(`
-                            SELECT tablename as name
-                            FROM pg_tables 
-                            WHERE schemaname = 'public' 
-                            AND tablename = $1
-                        `, [tableName]);
-                        
-                        if (checkCreate.rows.length === 0) {
-                            throw new Error(`Failed to create table ${tableName} - table does not exist after CREATE statement`);
-                        }
-
-                        outputProgress({
-                            operation: 'Table created',
-                            message: `Successfully created table: ${tableName}`
-                        });
-                    }
-                }
-                
-                outputProgress({
-                    operation: 'SQL Progress',
-                    message: {
-                        statement: i + 1,
-                        total: statements.length,
-                        preview: stmt.substring(0, 100) + (stmt.length > 100 ? '...' : ''),
-                        rowCount: result.rowCount
-                    }
-                });
-            } catch (sqlError) {
-                outputProgress({
-                    status: 'error',
-                    operation: 'SQL Error',
-                    message: {
-                        error: sqlError.message,
-                        statement: stmt,
-                        statementNumber: i + 1
-                    }
-                });
-                throw sqlError;
-            }
-        }
-
-        // Re-enable foreign key checks after all tables are created
-        await client.query('SET session_replication_role = \'origin\'');
-
-        // Verify metrics tables were created
-        outputProgress({
-            operation: 'Verifying metrics tables',
-            message: 'Checking all metrics tables were created...'
-        });
-
-        const metricsTablesResult = await client.query(`
-            SELECT tablename as name
-            FROM pg_tables 
-            WHERE schemaname = 'public'
-            AND tablename = ANY($1)
-        `, [METRICS_TABLES]);
-
-        outputProgress({
-            operation: 'Tables found',
-            message: `Found ${metricsTablesResult.rows.length} tables: ${metricsTablesResult.rows.map(t => t.name).join(', ')}`
-        });
-
-        const existingMetricsTables = metricsTablesResult.rows.map(t => t.name);
-        const missingMetricsTables = METRICS_TABLES.filter(t => !existingMetricsTables.includes(t));
-
-        if (missingMetricsTables.length > 0) {
-            // Do one final check of the actual tables
-            const finalCheck = await client.query(`
-                SELECT tablename as name
-                FROM pg_tables 
-                WHERE schemaname = 'public'
-            `);
-            outputProgress({
-                operation: 'Final table check',
-                message: `All database tables: ${finalCheck.rows.map(t => t.name).join(', ')}`
-            });
-            throw new Error(`Failed to create metrics tables: ${missingMetricsTables.join(', ')}`);
-        }
-
-        await client.query('COMMIT');
-        
-        outputProgress({
-            status: 'complete',
-            operation: 'Reset complete',
-            message: 'All metrics tables have been reset successfully'
-        });
-    } catch (error) {
-        outputProgress({
-            status: 'error',
-            operation: 'Reset failed',
-            message: error.message,
-            stack: error.stack
-        });
-        
-        if (client) {
-            await client.query('ROLLBACK');
-            // Make sure to re-enable foreign key checks even if there's an error
-            await client.query('SET session_replication_role = \'origin\'').catch(() => {});
-        }
-        throw error;
-    } finally {
-        if (client) {
-            // One final attempt to ensure foreign key checks are enabled
-            await client.query('SET session_replication_role = \'origin\'').catch(() => {});
-            await client.end();
-        }
-    }
-}
-
-// Export if required as a module
-if (typeof module !== 'undefined' && module.exports) {
-    module.exports = resetMetrics;
-}
-
-// Run if called from command line
-if (require.main === module) {
-    resetMetrics().catch(error => {
-        console.error('Error:', error);
-        process.exit(1);
-    });
-} 

inventory-server/src/middleware/cors.js

@@ -4,7 +4,9 @@ const cors = require('cors');
 const corsMiddleware = cors({
   origin: [
     'https://inventory.kent.pw',
-    'http://localhost:5173',
+    'http://localhost:5175',
+    'https://tools.acherryontop.com',
+    'https://tools.acherryontop.com',
     /^http:\/\/192\.168\.\d+\.\d+(:\d+)?$/,
     /^http:\/\/10\.\d+\.\d+\.\d+(:\d+)?$/
   ],
@@ -26,7 +28,7 @@ const corsErrorHandler = (err, req, res, next) => {
     res.status(403).json({
       error: 'CORS not allowed',
       origin: req.get('Origin'),
-      message: 'Origin not in allowed list: https://inventory.kent.pw, localhost:5173, 192.168.x.x, or 10.x.x.x'
+      message: 'Origin not in allowed list: https://inventory.kent.pw, https://tools.acherryontop.com, https://tools.acherryontop.com, localhost:5175, 192.168.x.x, or 10.x.x.x'
     });
   } else {
     next(err);

inventory-server/src/routes/ai-prompts.js

@@ -1,335 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Get all AI prompts
-router.get('/', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM ai_prompts 
-      ORDER BY prompt_type ASC, company ASC
-    `);
-    res.json(result.rows);
-  } catch (error) {
-    console.error('Error fetching AI prompts:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch AI prompts',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Get prompt by ID
-router.get('/:id', async (req, res) => {
-  try {
-    const { id } = req.params;
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM ai_prompts 
-      WHERE id = $1
-    `, [id]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'AI prompt not found' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error fetching AI prompt:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Get prompt by company
-router.get('/company/:companyId', async (req, res) => {
-  try {
-    const { companyId } = req.params;
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM ai_prompts 
-      WHERE company = $1
-    `, [companyId]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'AI prompt not found for this company' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error fetching AI prompt by company:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch AI prompt by company',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Get general prompt
-router.get('/type/general', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM ai_prompts 
-      WHERE prompt_type = 'general'
-    `);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'General AI prompt not found' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error fetching general AI prompt:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch general AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Get system prompt
-router.get('/type/system', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM ai_prompts 
-      WHERE prompt_type = 'system'
-    `);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'System AI prompt not found' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error fetching system AI prompt:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch system AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Create new AI prompt
-router.post('/', async (req, res) => {
-  try {
-    const {
-      prompt_text,
-      prompt_type,
-      company
-    } = req.body;
-
-    // Validate required fields
-    if (!prompt_text || !prompt_type) {
-      return res.status(400).json({ error: 'Prompt text and type are required' });
-    }
-    
-    // Validate prompt type
-    if (!['general', 'company_specific', 'system'].includes(prompt_type)) {
-      return res.status(400).json({ error: 'Prompt type must be either "general", "company_specific", or "system"' });
-    }
-    
-    // Validate company is provided for company-specific prompts
-    if (prompt_type === 'company_specific' && !company) {
-      return res.status(400).json({ error: 'Company is required for company-specific prompts' });
-    }
-
-    // Validate company is not provided for general or system prompts
-    if ((prompt_type === 'general' || prompt_type === 'system') && company) {
-      return res.status(400).json({ error: 'Company should not be provided for general or system prompts' });
-    }
-
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      INSERT INTO ai_prompts (
-        prompt_text,
-        prompt_type,
-        company
-      ) VALUES ($1, $2, $3)
-      RETURNING *
-    `, [
-      prompt_text,
-      prompt_type,
-      company
-    ]);
-
-    res.status(201).json(result.rows[0]);
-  } catch (error) {
-    console.error('Error creating AI prompt:', error);
-    
-    // Check for unique constraint violations
-    if (error instanceof Error && error.message.includes('unique constraint')) {
-      if (error.message.includes('unique_company_prompt')) {
-        return res.status(409).json({ 
-          error: 'A prompt already exists for this company',
-          details: error.message
-        });
-      } else if (error.message.includes('idx_unique_general_prompt')) {
-        return res.status(409).json({ 
-          error: 'A general prompt already exists',
-          details: error.message
-        });
-      } else if (error.message.includes('idx_unique_system_prompt')) {
-        return res.status(409).json({ 
-          error: 'A system prompt already exists',
-          details: error.message
-        });
-      }
-    }
-    
-    res.status(500).json({ 
-      error: 'Failed to create AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Update AI prompt
-router.put('/:id', async (req, res) => {
-  try {
-    const { id } = req.params;
-    const {
-      prompt_text,
-      prompt_type,
-      company
-    } = req.body;
-
-    // Validate required fields
-    if (!prompt_text || !prompt_type) {
-      return res.status(400).json({ error: 'Prompt text and type are required' });
-    }
-    
-    // Validate prompt type
-    if (!['general', 'company_specific', 'system'].includes(prompt_type)) {
-      return res.status(400).json({ error: 'Prompt type must be either "general", "company_specific", or "system"' });
-    }
-    
-    // Validate company is provided for company-specific prompts
-    if (prompt_type === 'company_specific' && !company) {
-      return res.status(400).json({ error: 'Company is required for company-specific prompts' });
-    }
-    
-    // Validate company is not provided for general or system prompts
-    if ((prompt_type === 'general' || prompt_type === 'system') && company) {
-      return res.status(400).json({ error: 'Company should not be provided for general or system prompts' });
-    }
-
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    // Check if the prompt exists
-    const checkResult = await pool.query('SELECT * FROM ai_prompts WHERE id = $1', [id]);
-    if (checkResult.rows.length === 0) {
-      return res.status(404).json({ error: 'AI prompt not found' });
-    }
-    
-    const result = await pool.query(`
-      UPDATE ai_prompts 
-      SET 
-        prompt_text = $1,
-        prompt_type = $2,
-        company = $3
-      WHERE id = $4
-      RETURNING *
-    `, [
-      prompt_text,
-      prompt_type,
-      company,
-      id
-    ]);
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error updating AI prompt:', error);
-    
-    // Check for unique constraint violations
-    if (error instanceof Error && error.message.includes('unique constraint')) {
-      if (error.message.includes('unique_company_prompt')) {
-        return res.status(409).json({ 
-          error: 'A prompt already exists for this company',
-          details: error.message
-        });
-      } else if (error.message.includes('idx_unique_general_prompt')) {
-        return res.status(409).json({ 
-          error: 'A general prompt already exists',
-          details: error.message
-        });
-      } else if (error.message.includes('idx_unique_system_prompt')) {
-        return res.status(409).json({ 
-          error: 'A system prompt already exists',
-          details: error.message
-        });
-      }
-    }
-    
-    res.status(500).json({ 
-      error: 'Failed to update AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Delete AI prompt
-router.delete('/:id', async (req, res) => {
-  try {
-    const { id } = req.params;
-    const pool = req.app.locals.pool;
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query('DELETE FROM ai_prompts WHERE id = $1 RETURNING *', [id]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'AI prompt not found' });
-    }
-
-    res.json({ message: 'AI prompt deleted successfully' });
-  } catch (error) {
-    console.error('Error deleting AI prompt:', error);
-    res.status(500).json({ 
-      error: 'Failed to delete AI prompt',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Error handling middleware
-router.use((err, req, res, next) => {
-  console.error('AI prompts route error:', err);
-  res.status(500).json({ 
-    error: 'Internal server error',
-    details: err.message
-  });
-});
-
-module.exports = router; 

inventory-server/src/routes/analytics.js

@@ -1,654 +1 @@
-const express = require('express');
-const router = express.Router();
  
-// Get overall analytics stats
-router.get('/stats', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    const { rows: [results] } = await pool.query(`
-      SELECT 
-        COALESCE(
-          ROUND(
-            (SUM(o.price * o.quantity - p.cost_price * o.quantity) / 
-             NULLIF(SUM(o.price * o.quantity), 0) * 100)::numeric, 1
-          ),
-          0
-        ) as profitMargin,
-        COALESCE(
-          ROUND(
-            (AVG(p.price / NULLIF(p.cost_price, 0) - 1) * 100)::numeric, 1
-          ),
-          0
-        ) as averageMarkup,
-        COALESCE(
-          ROUND(
-            (SUM(o.quantity) / NULLIF(AVG(p.stock_quantity), 0))::numeric, 2
-          ),
-          0
-        ) as stockTurnoverRate,
-        COALESCE(COUNT(DISTINCT p.vendor), 0) as vendorCount,
-        COALESCE(COUNT(DISTINCT p.categories), 0) as categoryCount,
-        COALESCE(
-          ROUND(
-            AVG(o.price * o.quantity)::numeric, 2
-          ),
-          0
-        ) as averageOrderValue
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      WHERE o.date >= CURRENT_DATE - INTERVAL '30 days'
-    `);
-
-    // Ensure all values are numbers
-    const stats = {
-      profitMargin: Number(results.profitmargin) || 0,
-      averageMarkup: Number(results.averagemarkup) || 0,
-      stockTurnoverRate: Number(results.stockturnoverrate) || 0,
-      vendorCount: Number(results.vendorcount) || 0,
-      categoryCount: Number(results.categorycount) || 0,
-      averageOrderValue: Number(results.averageordervalue) || 0
-    };
-
-    res.json(stats);
-  } catch (error) {
-    console.error('Error fetching analytics stats:', error);
-    res.status(500).json({ error: 'Failed to fetch analytics stats' });
-  }
-});
-
-// Get profit analysis data
-router.get('/profit', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    // Get profit margins by category with full path
-    const { rows: byCategory } = await pool.query(`
-      WITH RECURSIVE category_path AS (
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          c.name::text as path
-        FROM categories c
-        WHERE c.parent_id IS NULL
-        
-        UNION ALL
-        
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          cp.path || ' > ' || c.name
-        FROM categories c
-        JOIN category_path cp ON c.parent_id = cp.cat_id
-      )
-      SELECT 
-        c.name as category,
-        cp.path as categoryPath,
-        ROUND(
-          (SUM(o.price * o.quantity - p.cost_price * o.quantity) / 
-           NULLIF(SUM(o.price * o.quantity), 0) * 100)::numeric, 1
-        ) as profitMargin,
-        ROUND(SUM(o.price * o.quantity)::numeric, 3) as revenue,
-        ROUND(SUM(p.cost_price * o.quantity)::numeric, 3) as cost
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      WHERE o.date >= CURRENT_DATE - INTERVAL '30 days'
-      GROUP BY c.name, cp.path
-      ORDER BY profitMargin DESC
-      LIMIT 10
-    `);
-
-    // Get profit margin trend over time
-    const { rows: overTime } = await pool.query(`
-      SELECT 
-        to_char(o.date, 'YYYY-MM-DD') as date,
-        ROUND(
-          (SUM(o.price * o.quantity - p.cost_price * o.quantity) / 
-           NULLIF(SUM(o.price * o.quantity), 0) * 100)::numeric, 1
-        ) as profitMargin,
-        ROUND(SUM(o.price * o.quantity)::numeric, 3) as revenue,
-        ROUND(SUM(p.cost_price * o.quantity)::numeric, 3) as cost
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      WHERE o.date >= CURRENT_DATE - INTERVAL '30 days'
-      GROUP BY to_char(o.date, 'YYYY-MM-DD')
-      ORDER BY date
-    `);
-
-    // Get top performing products with category paths
-    const { rows: topProducts } = await pool.query(`
-      WITH RECURSIVE category_path AS (
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          c.name::text as path
-        FROM categories c
-        WHERE c.parent_id IS NULL
-        
-        UNION ALL
-        
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          cp.path || ' > ' || c.name
-        FROM categories c
-        JOIN category_path cp ON c.parent_id = cp.cat_id
-      )
-      SELECT 
-        p.title as product,
-        c.name as category,
-        cp.path as categoryPath,
-        ROUND(
-          (SUM(o.price * o.quantity - p.cost_price * o.quantity) / 
-           NULLIF(SUM(o.price * o.quantity), 0) * 100)::numeric, 1
-        ) as profitMargin,
-        ROUND(SUM(o.price * o.quantity)::numeric, 3) as revenue,
-        ROUND(SUM(p.cost_price * o.quantity)::numeric, 3) as cost
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      WHERE o.date >= CURRENT_DATE - INTERVAL '30 days'
-      GROUP BY p.pid, p.title, c.name, cp.path
-      HAVING SUM(o.price * o.quantity) > 0
-      ORDER BY profitMargin DESC
-      LIMIT 10
-    `);
-
-    res.json({ byCategory, overTime, topProducts });
-  } catch (error) {
-    console.error('Error fetching profit analysis:', error);
-    res.status(500).json({ error: 'Failed to fetch profit analysis' });
-  }
-});
-
-// Get vendor performance data
-router.get('/vendors', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    console.log('Fetching vendor performance data...');
-
-    // First check if we have any vendors with sales
-    const { rows: [checkData] } = await pool.query(`
-      SELECT COUNT(DISTINCT p.vendor) as vendor_count,
-             COUNT(DISTINCT o.order_number) as order_count
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      WHERE p.vendor IS NOT NULL
-    `);
-    
-    console.log('Vendor data check:', checkData);
-    
-    // Get vendor performance metrics
-    const { rows: performance } = await pool.query(`
-      WITH monthly_sales AS (
-        SELECT 
-          p.vendor,
-          ROUND(SUM(CASE 
-            WHEN o.date >= CURRENT_DATE - INTERVAL '30 days'
-            THEN o.price * o.quantity 
-            ELSE 0 
-          END)::numeric, 3) as current_month,
-          ROUND(SUM(CASE 
-            WHEN o.date >= CURRENT_DATE - INTERVAL '60 days'
-            AND o.date < CURRENT_DATE - INTERVAL '30 days'
-            THEN o.price * o.quantity 
-            ELSE 0 
-          END)::numeric, 3) as previous_month
-        FROM products p
-        LEFT JOIN orders o ON p.pid = o.pid
-        WHERE p.vendor IS NOT NULL
-          AND o.date >= CURRENT_DATE - INTERVAL '60 days'
-        GROUP BY p.vendor
-      )
-      SELECT 
-        p.vendor,
-        ROUND(SUM(o.price * o.quantity)::numeric, 3) as salesVolume,
-        COALESCE(ROUND(
-          (SUM(o.price * o.quantity - p.cost_price * o.quantity) / 
-           NULLIF(SUM(o.price * o.quantity), 0) * 100)::numeric, 1
-        ), 0) as profitMargin,
-        COALESCE(ROUND(
-          (SUM(o.quantity) / NULLIF(AVG(p.stock_quantity), 0))::numeric, 1
-        ), 0) as stockTurnover,
-        COUNT(DISTINCT p.pid) as productCount,
-        ROUND(
-          ((ms.current_month / NULLIF(ms.previous_month, 0)) - 1) * 100,
-          1
-        ) as growth
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      LEFT JOIN monthly_sales ms ON p.vendor = ms.vendor
-      WHERE p.vendor IS NOT NULL
-        AND o.date >= CURRENT_DATE - INTERVAL '30 days'
-      GROUP BY p.vendor, ms.current_month, ms.previous_month
-      ORDER BY salesVolume DESC
-      LIMIT 10
-    `);
-
-    console.log('Performance data:', performance);
-
-    res.json({ performance });
-  } catch (error) {
-    console.error('Error fetching vendor performance:', error);
-    res.status(500).json({ error: 'Failed to fetch vendor performance' });
-  }
-});
-
-// Get stock analysis data
-router.get('/stock', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    // Get global configuration values
-    const [configs] = await pool.query(`
-      SELECT 
-        st.low_stock_threshold,
-        tc.calculation_period_days as turnover_period
-      FROM stock_thresholds st
-      CROSS JOIN turnover_config tc
-      WHERE st.id = 1 AND tc.id = 1
-    `);
-    
-    const config = configs[0] || {
-      low_stock_threshold: 5,
-      turnover_period: 30
-    };
-    
-    // Get turnover by category
-    const [turnoverByCategory] = await pool.query(`
-      SELECT 
-        c.name as category,
-        ROUND(SUM(o.quantity) / NULLIF(AVG(p.stock_quantity), 0), 1) as turnoverRate,
-        ROUND(AVG(p.stock_quantity), 0) as averageStock,
-        SUM(o.quantity) as totalSales
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-      GROUP BY c.name
-      HAVING turnoverRate > 0
-      ORDER BY turnoverRate DESC
-      LIMIT 10
-    `, [config.turnover_period]);
-
-    // Get stock levels over time
-    const [stockLevels] = await pool.query(`
-      SELECT 
-        DATE_FORMAT(o.date, '%Y-%m-%d') as date,
-        SUM(CASE WHEN p.stock_quantity > ? THEN 1 ELSE 0 END) as inStock,
-        SUM(CASE WHEN p.stock_quantity <= ? AND p.stock_quantity > 0 THEN 1 ELSE 0 END) as lowStock,
-        SUM(CASE WHEN p.stock_quantity = 0 THEN 1 ELSE 0 END) as outOfStock
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-      GROUP BY DATE_FORMAT(o.date, '%Y-%m-%d')
-      ORDER BY date
-    `, [
-      config.low_stock_threshold,
-      config.low_stock_threshold,
-      config.turnover_period
-    ]);
-
-    // Get critical stock items
-    const [criticalItems] = await pool.query(`
-      WITH product_thresholds AS (
-        SELECT 
-          p.pid,
-          COALESCE(
-            (SELECT reorder_days 
-             FROM stock_thresholds st 
-             WHERE st.vendor = p.vendor LIMIT 1),
-            (SELECT reorder_days 
-             FROM stock_thresholds st 
-             WHERE st.vendor IS NULL LIMIT 1),
-            14
-          ) as reorder_days
-        FROM products p
-      )
-      SELECT 
-        p.title as product,
-        p.SKU as sku,
-        p.stock_quantity as stockQuantity,
-        GREATEST(ROUND(AVG(o.quantity) * pt.reorder_days), ?) as reorderPoint,
-        ROUND(SUM(o.quantity) / NULLIF(p.stock_quantity, 0), 1) as turnoverRate,
-        CASE 
-          WHEN p.stock_quantity = 0 THEN 0
-          ELSE ROUND(p.stock_quantity / NULLIF((SUM(o.quantity) / ?), 0))
-        END as daysUntilStockout
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_thresholds pt ON p.pid = pt.pid
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-        AND p.managing_stock = true
-      GROUP BY p.pid
-      HAVING daysUntilStockout < ? AND daysUntilStockout >= 0
-      ORDER BY daysUntilStockout
-      LIMIT 10
-    `, [
-      config.low_stock_threshold,
-      config.turnover_period,
-      config.turnover_period,
-      config.turnover_period
-    ]);
-
-    res.json({ turnoverByCategory, stockLevels, criticalItems });
-  } catch (error) {
-    console.error('Error fetching stock analysis:', error);
-    res.status(500).json({ error: 'Failed to fetch stock analysis' });
-  }
-});
-
-// Get price analysis data
-router.get('/pricing', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    // Get price points analysis
-    const [pricePoints] = await pool.query(`
-      SELECT 
-        CAST(p.price AS DECIMAL(15,3)) as price,
-        CAST(SUM(o.quantity) AS DECIMAL(15,3)) as salesVolume,
-        CAST(SUM(o.price * o.quantity) AS DECIMAL(15,3)) as revenue,
-        c.name as category
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-      GROUP BY p.price, c.name
-      HAVING salesVolume > 0
-      ORDER BY revenue DESC
-      LIMIT 50
-    `);
-
-    // Get price elasticity data (price changes vs demand)
-    const [elasticity] = await pool.query(`
-      SELECT 
-        DATE_FORMAT(o.date, '%Y-%m-%d') as date,
-        CAST(AVG(o.price) AS DECIMAL(15,3)) as price,
-        CAST(SUM(o.quantity) AS DECIMAL(15,3)) as demand
-      FROM orders o
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-      GROUP BY DATE_FORMAT(o.date, '%Y-%m-%d')
-      ORDER BY date
-    `);
-
-    // Get price optimization recommendations
-    const [recommendations] = await pool.query(`
-      SELECT 
-        p.title as product,
-        CAST(p.price AS DECIMAL(15,3)) as currentPrice,
-        CAST(
-          ROUND(
-            CASE 
-              WHEN AVG(o.quantity) > 10 THEN p.price * 1.1
-              WHEN AVG(o.quantity) < 2 THEN p.price * 0.9
-              ELSE p.price
-            END, 2
-          ) AS DECIMAL(15,3)
-        ) as recommendedPrice,
-        CAST(
-          ROUND(
-            SUM(o.price * o.quantity) * 
-            CASE 
-              WHEN AVG(o.quantity) > 10 THEN 1.15
-              WHEN AVG(o.quantity) < 2 THEN 0.95
-              ELSE 1
-            END, 2
-          ) AS DECIMAL(15,3)
-        ) as potentialRevenue,
-        CASE 
-          WHEN AVG(o.quantity) > 10 THEN 85
-          WHEN AVG(o.quantity) < 2 THEN 75
-          ELSE 65
-        END as confidence
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-      GROUP BY p.pid, p.price
-      HAVING ABS(recommendedPrice - currentPrice) > 0
-      ORDER BY potentialRevenue - CAST(SUM(o.price * o.quantity) AS DECIMAL(15,3)) DESC
-      LIMIT 10
-    `);
-
-    res.json({ pricePoints, elasticity, recommendations });
-  } catch (error) {
-    console.error('Error fetching price analysis:', error);
-    res.status(500).json({ error: 'Failed to fetch price analysis' });
-  }
-});
-
-// Get category performance data
-router.get('/categories', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    // Common CTE for category paths
-    const categoryPathCTE = `
-      WITH RECURSIVE category_path AS (
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CAST(c.name AS CHAR(1000)) as path
-        FROM categories c
-        WHERE c.parent_id IS NULL
-        
-        UNION ALL
-        
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CONCAT(cp.path, ' > ', c.name)
-        FROM categories c
-        JOIN category_path cp ON c.parent_id = cp.cat_id
-      )
-    `;
-    
-    // Get category performance metrics with full path
-    const [performance] = await pool.query(`
-      ${categoryPathCTE},
-      monthly_sales AS (
-        SELECT 
-          c.name,
-          cp.path,
-          SUM(CASE 
-            WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY) 
-            THEN o.price * o.quantity 
-            ELSE 0 
-          END) as current_month,
-          SUM(CASE 
-            WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY) 
-            AND o.date < DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-            THEN o.price * o.quantity 
-            ELSE 0 
-          END) as previous_month
-        FROM products p
-        LEFT JOIN orders o ON p.pid = o.pid
-        JOIN product_categories pc ON p.pid = pc.pid
-        JOIN categories c ON pc.cat_id = c.cat_id
-        JOIN category_path cp ON c.cat_id = cp.cat_id
-        WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY)
-        GROUP BY c.name, cp.path
-      )
-      SELECT 
-        c.name as category,
-        cp.path as categoryPath,
-        SUM(o.price * o.quantity) as revenue,
-        SUM(o.price * o.quantity - p.cost_price * o.quantity) as profit,
-        ROUND(
-          ((ms.current_month / NULLIF(ms.previous_month, 0)) - 1) * 100,
-          1
-        ) as growth,
-        COUNT(DISTINCT p.pid) as productCount
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      LEFT JOIN monthly_sales ms ON c.name = ms.name AND cp.path = ms.path
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY)
-      GROUP BY c.name, cp.path, ms.current_month, ms.previous_month
-      HAVING revenue > 0
-      ORDER BY revenue DESC
-      LIMIT 10
-    `);
-
-    // Get category revenue distribution with full path
-    const [distribution] = await pool.query(`
-      ${categoryPathCTE}
-      SELECT 
-        c.name as category,
-        cp.path as categoryPath,
-        SUM(o.price * o.quantity) as value
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-      GROUP BY c.name, cp.path
-      HAVING value > 0
-      ORDER BY value DESC
-      LIMIT 6
-    `);
-
-    // Get category sales trends with full path
-    const [trends] = await pool.query(`
-      ${categoryPathCTE}
-      SELECT 
-        c.name as category,
-        cp.path as categoryPath,
-        DATE_FORMAT(o.date, '%b %Y') as month,
-        SUM(o.price * o.quantity) as sales
-      FROM products p
-      LEFT JOIN orders o ON p.pid = o.pid
-      JOIN product_categories pc ON p.pid = pc.pid
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 6 MONTH)
-      GROUP BY 
-        c.name,
-        cp.path,
-        DATE_FORMAT(o.date, '%b %Y'),
-        DATE_FORMAT(o.date, '%Y-%m')
-      ORDER BY 
-        c.name,
-        DATE_FORMAT(o.date, '%Y-%m')
-    `);
-
-    res.json({ performance, distribution, trends });
-  } catch (error) {
-    console.error('Error fetching category performance:', error);
-    res.status(500).json({ error: 'Failed to fetch category performance' });
-  }
-});
-
-// Forecast endpoint
-router.get('/forecast', async (req, res) => {
-  try {
-    const { brand, startDate, endDate } = req.query;
-    const pool = req.app.locals.pool;
-
-    const [results] = await pool.query(`
-      WITH RECURSIVE category_path AS (
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CAST(c.name AS CHAR(1000)) as path
-        FROM categories c
-        WHERE c.parent_id IS NULL
-        
-        UNION ALL
-        
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CONCAT(cp.path, ' > ', c.name)
-        FROM categories c
-        JOIN category_path cp ON c.parent_id = cp.cat_id
-      ),
-      category_metrics AS (
-        SELECT 
-          c.cat_id,
-          c.name as category_name,
-          cp.path,
-          p.brand,
-          COUNT(DISTINCT p.pid) as num_products,
-          CAST(COALESCE(ROUND(SUM(o.quantity) / DATEDIFF(?, ?), 2), 0) AS DECIMAL(15,3)) as avg_daily_sales,
-          COALESCE(SUM(o.quantity), 0) as total_sold,
-          CAST(COALESCE(ROUND(SUM(o.quantity) / COUNT(DISTINCT p.pid), 2), 0) AS DECIMAL(15,3)) as avgTotalSold,
-          CAST(COALESCE(ROUND(AVG(o.price), 2), 0) AS DECIMAL(15,3)) as avg_price
-        FROM categories c
-        JOIN product_categories pc ON c.cat_id = pc.cat_id
-        JOIN products p ON pc.pid = p.pid
-        JOIN category_path cp ON c.cat_id = cp.cat_id
-        LEFT JOIN product_metrics pmet ON p.pid = pmet.pid
-        LEFT JOIN orders o ON p.pid = o.pid 
-          AND o.date BETWEEN ? AND ?
-          AND o.canceled = false
-        WHERE p.brand = ?
-          AND pmet.first_received_date BETWEEN ? AND ?
-        GROUP BY c.cat_id, c.name, cp.path, p.brand
-      ),
-      product_details AS (
-        SELECT 
-          p.pid,
-          p.title,
-          p.SKU,
-          p.stock_quantity,
-          pc.cat_id,
-          pmet.first_received_date,
-          COALESCE(SUM(o.quantity), 0) as total_sold,
-          CAST(COALESCE(ROUND(AVG(o.price), 2), 0) AS DECIMAL(15,3)) as avg_price
-        FROM products p
-        JOIN product_categories pc ON p.pid = pc.pid
-        JOIN product_metrics pmet ON p.pid = pmet.pid
-        LEFT JOIN orders o ON p.pid = o.pid 
-          AND o.date BETWEEN ? AND ?
-          AND o.canceled = false
-        WHERE p.brand = ?
-          AND pmet.first_received_date BETWEEN ? AND ?
-        GROUP BY p.pid, p.title, p.SKU, p.stock_quantity, pc.cat_id, pmet.first_received_date
-      )
-      SELECT 
-        cm.*,
-        JSON_ARRAYAGG(
-          JSON_OBJECT(
-            'pid', pd.pid,
-            'title', pd.title,
-            'SKU', pd.SKU,
-            'stock_quantity', pd.stock_quantity,
-            'total_sold', pd.total_sold,
-            'avg_price', pd.avg_price,
-            'first_received_date', DATE_FORMAT(pd.first_received_date, '%Y-%m-%d')
-          )
-        ) as products
-      FROM category_metrics cm
-      JOIN product_details pd ON cm.cat_id = pd.cat_id
-      GROUP BY cm.cat_id, cm.category_name, cm.path, cm.brand, cm.num_products, cm.avg_daily_sales, cm.total_sold, cm.avgTotalSold, cm.avg_price
-      ORDER BY cm.total_sold DESC
-    `, [endDate, startDate, startDate, endDate, brand, startDate, endDate, startDate, endDate, brand, startDate, endDate]);
-
-    res.json(results);
-  } catch (error) {
-    console.error('Error fetching forecast data:', error);
-    res.status(500).json({ error: 'Failed to fetch forecast data' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/brandsAggregate.js

@@ -0,0 +1 @@
+ 

inventory-server/src/routes/categories.js

@@ -1,100 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Get all categories
-router.get('/', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    // Get all categories with metrics and hierarchy info
-    const { rows: categories } = await pool.query(`
-      SELECT 
-        c.cat_id,
-        c.name,
-        c.type,
-        c.parent_id,
-        c.description,
-        c.status,
-        p.name as parent_name,
-        p.type as parent_type,
-        COALESCE(cm.product_count, 0) as product_count,
-        COALESCE(cm.active_products, 0) as active_products,
-        ROUND(COALESCE(cm.total_value, 0)::numeric, 3) as total_value,
-        COALESCE(cm.avg_margin, 0) as avg_margin,
-        COALESCE(cm.turnover_rate, 0) as turnover_rate,
-        COALESCE(cm.growth_rate, 0) as growth_rate
-      FROM categories c
-      LEFT JOIN categories p ON c.parent_id = p.cat_id
-      LEFT JOIN category_metrics cm ON c.cat_id = cm.category_id
-      ORDER BY 
-        CASE 
-          WHEN c.type = 10 THEN 1  -- sections first
-          WHEN c.type = 11 THEN 2  -- categories second
-          WHEN c.type = 12 THEN 3  -- subcategories third
-          WHEN c.type = 13 THEN 4  -- subsubcategories fourth
-          WHEN c.type = 20 THEN 5  -- themes fifth
-          WHEN c.type = 21 THEN 6  -- subthemes last
-          ELSE 7
-        END,
-        c.name ASC
-    `);
-
-    // Get overall stats
-    const { rows: [stats] } = await pool.query(`
-      SELECT 
-        COUNT(DISTINCT c.cat_id) as totalCategories,
-        COUNT(DISTINCT CASE WHEN c.status = 'active' THEN c.cat_id END) as activeCategories,
-        ROUND(COALESCE(SUM(cm.total_value), 0)::numeric, 3) as totalValue,
-        COALESCE(ROUND(AVG(NULLIF(cm.avg_margin, 0))::numeric, 1), 0) as avgMargin,
-        COALESCE(ROUND(AVG(NULLIF(cm.growth_rate, 0))::numeric, 1), 0) as avgGrowth
-      FROM categories c
-      LEFT JOIN category_metrics cm ON c.cat_id = cm.category_id
-    `);
-
-    // Get type counts for filtering
-    const { rows: typeCounts } = await pool.query(`
-      SELECT 
-        type,
-        COUNT(*)::integer as count
-      FROM categories
-      GROUP BY type
-      ORDER BY type
-    `);
-
-    res.json({
-      categories: categories.map(cat => ({
-        cat_id: cat.cat_id,
-        name: cat.name,
-        type: cat.type,
-        parent_id: cat.parent_id,
-        parent_name: cat.parent_name,
-        parent_type: cat.parent_type,
-        description: cat.description,
-        status: cat.status,
-        metrics: {
-          product_count: parseInt(cat.product_count),
-          active_products: parseInt(cat.active_products),
-          total_value: parseFloat(cat.total_value),
-          avg_margin: parseFloat(cat.avg_margin),
-          turnover_rate: parseFloat(cat.turnover_rate),
-          growth_rate: parseFloat(cat.growth_rate)
-        }
-      })),
-      typeCounts: typeCounts.map(tc => ({
-        type: tc.type,
-        count: tc.count // Already cast to integer in the query
-      })),
-      stats: {
-        totalCategories: parseInt(stats.totalcategories),
-        activeCategories: parseInt(stats.activecategories),
-        totalValue: parseFloat(stats.totalvalue),
-        avgMargin: parseFloat(stats.avgmargin),
-        avgGrowth: parseFloat(stats.avggrowth)
-      }
-    });
-  } catch (error) {
-    console.error('Error fetching categories:', error);
-    res.status(500).json({ error: 'Failed to fetch categories' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/config.js

@@ -1,172 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Debug middleware
-router.use((req, res, next) => {
-  console.log(`[Config Route] ${req.method} ${req.path}`);
-  next();
-});
-
-// Get all configuration values
-router.get('/', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    console.log('[Config Route] Fetching configuration values...');
-    
-    const { rows: stockThresholds } = await pool.query('SELECT * FROM stock_thresholds WHERE id = 1');
-    console.log('[Config Route] Stock thresholds:', stockThresholds);
-    
-    const { rows: leadTimeThresholds } = await pool.query('SELECT * FROM lead_time_thresholds WHERE id = 1');
-    console.log('[Config Route] Lead time thresholds:', leadTimeThresholds);
-    
-    const { rows: salesVelocityConfig } = await pool.query('SELECT * FROM sales_velocity_config WHERE id = 1');
-    console.log('[Config Route] Sales velocity config:', salesVelocityConfig);
-    
-    const { rows: abcConfig } = await pool.query('SELECT * FROM abc_classification_config WHERE id = 1');
-    console.log('[Config Route] ABC config:', abcConfig);
-    
-    const { rows: safetyStockConfig } = await pool.query('SELECT * FROM safety_stock_config WHERE id = 1');
-    console.log('[Config Route] Safety stock config:', safetyStockConfig);
-    
-    const { rows: turnoverConfig } = await pool.query('SELECT * FROM turnover_config WHERE id = 1');
-    console.log('[Config Route] Turnover config:', turnoverConfig);
-
-    const response = {
-      stockThresholds: stockThresholds[0],
-      leadTimeThresholds: leadTimeThresholds[0],
-      salesVelocityConfig: salesVelocityConfig[0],
-      abcConfig: abcConfig[0],
-      safetyStockConfig: safetyStockConfig[0],
-      turnoverConfig: turnoverConfig[0]
-    };
-    
-    console.log('[Config Route] Sending response:', response);
-    res.json(response);
-  } catch (error) {
-    console.error('[Config Route] Error fetching configuration:', error);
-    res.status(500).json({ error: 'Failed to fetch configuration', details: error.message });
-  }
-});
-
-// Update stock thresholds
-router.put('/stock-thresholds/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { critical_days, reorder_days, overstock_days, low_stock_threshold, min_reorder_quantity } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE stock_thresholds 
-       SET critical_days = $1,
-           reorder_days = $2,
-           overstock_days = $3,
-           low_stock_threshold = $4,
-           min_reorder_quantity = $5
-       WHERE id = $6`,
-      [critical_days, reorder_days, overstock_days, low_stock_threshold, min_reorder_quantity, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating stock thresholds:', error);
-    res.status(500).json({ error: 'Failed to update stock thresholds' });
-  }
-});
-
-// Update lead time thresholds
-router.put('/lead-time-thresholds/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { target_days, warning_days, critical_days } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE lead_time_thresholds 
-       SET target_days = $1,
-           warning_days = $2,
-           critical_days = $3
-       WHERE id = $4`,
-      [target_days, warning_days, critical_days, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating lead time thresholds:', error);
-    res.status(500).json({ error: 'Failed to update lead time thresholds' });
-  }
-});
-
-// Update sales velocity config
-router.put('/sales-velocity/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { daily_window_days, weekly_window_days, monthly_window_days } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE sales_velocity_config 
-       SET daily_window_days = $1,
-           weekly_window_days = $2,
-           monthly_window_days = $3
-       WHERE id = $4`,
-      [daily_window_days, weekly_window_days, monthly_window_days, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating sales velocity config:', error);
-    res.status(500).json({ error: 'Failed to update sales velocity config' });
-  }
-});
-
-// Update ABC classification config
-router.put('/abc-classification/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { a_threshold, b_threshold, classification_period_days } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE abc_classification_config 
-       SET a_threshold = $1,
-           b_threshold = $2,
-           classification_period_days = $3
-       WHERE id = $4`,
-      [a_threshold, b_threshold, classification_period_days, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating ABC classification config:', error);
-    res.status(500).json({ error: 'Failed to update ABC classification config' });
-  }
-});
-
-// Update safety stock config
-router.put('/safety-stock/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { coverage_days, service_level } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE safety_stock_config 
-       SET coverage_days = $1,
-           service_level = $2
-       WHERE id = $3`,
-      [coverage_days, service_level, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating safety stock config:', error);
-    res.status(500).json({ error: 'Failed to update safety stock config' });
-  }
-});
-
-// Update turnover config
-router.put('/turnover/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { calculation_period_days, target_rate } = req.body;
-    const { rows } = await pool.query(
-      `UPDATE turnover_config 
-       SET calculation_period_days = $1,
-           target_rate = $2
-       WHERE id = $3`,
-      [calculation_period_days, target_rate, req.params.id]
-    );
-    res.json({ success: true });
-  } catch (error) {
-    console.error('[Config Route] Error updating turnover config:', error);
-    res.status(500).json({ error: 'Failed to update turnover config' });
-  }
-});
-
-// Export the router
-module.exports = router; 

inventory-server/src/routes/csv.js

@@ -1,833 +0,0 @@
-const express = require('express');
-const router = express.Router();
-const { spawn } = require('child_process');
-const path = require('path');
-const db = require('../utils/db');
-
-// Debug middleware MUST be first
-router.use((req, res, next) => {
-    console.log(`[CSV Route Debug] ${req.method} ${req.path}`);
-    next();
-});
-
-// Store active processes and their progress
-let activeImport = null;
-let importProgress = null;
-let activeFullUpdate = null;
-let activeFullReset = null;
-
-// SSE clients for progress updates
-const updateClients = new Set();
-const importClients = new Set();
-const resetClients = new Set();
-const resetMetricsClients = new Set();
-const calculateMetricsClients = new Set();
-const fullUpdateClients = new Set();
-const fullResetClients = new Set();
-
-// Helper to send progress to specific clients
-function sendProgressToClients(clients, data) {
-    // If data is a string, send it directly
-    // If it's an object, convert it to JSON
-    const message = typeof data === 'string' 
-        ? `data: ${data}\n\n`
-        : `data: ${JSON.stringify(data)}\n\n`;
-    
-    clients.forEach(client => {
-        try {
-            client.write(message);
-            // Immediately flush the response
-            if (typeof client.flush === 'function') {
-                client.flush();
-            }
-        } catch (error) {
-            // Silently remove failed client
-            clients.delete(client);
-        }
-    });
-}
-
-// Helper to run a script and stream progress
-function runScript(scriptPath, type, clients) {
-    return new Promise((resolve, reject) => {
-        // Kill any existing process of this type
-        let activeProcess;
-        switch (type) {
-            case 'update':
-                if (activeFullUpdate) {
-                    try { activeFullUpdate.kill(); } catch (e) { }
-                }
-                activeProcess = activeFullUpdate;
-                break;
-            case 'reset':
-                if (activeFullReset) {
-                    try { activeFullReset.kill(); } catch (e) { }
-                }
-                activeProcess = activeFullReset;
-                break;
-        }
-
-        const child = spawn('node', [scriptPath], {
-            stdio: ['inherit', 'pipe', 'pipe']
-        });
-
-        switch (type) {
-            case 'update':
-                activeFullUpdate = child;
-                break;
-            case 'reset':
-                activeFullReset = child;
-                break;
-        }
-
-        let output = '';
-
-        child.stdout.on('data', (data) => {
-            const text = data.toString();
-            output += text;
-            
-            // Split by lines to handle multiple JSON outputs
-            const lines = text.split('\n');
-            lines.filter(line => line.trim()).forEach(line => {
-                try {
-                    // Try to parse as JSON but don't let it affect the display
-                    const jsonData = JSON.parse(line);
-                    // Only end the process if we get a final status
-                    if (jsonData.status === 'complete' || jsonData.status === 'error' || jsonData.status === 'cancelled') {
-                        if (jsonData.status === 'complete' && !jsonData.operation?.includes('complete')) {
-                            // Don't close for intermediate completion messages
-                            sendProgressToClients(clients, line);
-                            return;
-                        }
-                        // Close only on final completion/error/cancellation
-                        switch (type) {
-                            case 'update':
-                                activeFullUpdate = null;
-                                break;
-                            case 'reset':
-                                activeFullReset = null;
-                                break;
-                        }
-                        if (jsonData.status === 'error') {
-                            reject(new Error(jsonData.error || 'Unknown error'));
-                        } else {
-                            resolve({ output });
-                        }
-                    }
-                } catch (e) {
-                    // Not JSON, just display as is
-                }
-                // Always send the raw line
-                sendProgressToClients(clients, line);
-            });
-        });
-
-        child.stderr.on('data', (data) => {
-            const text = data.toString();
-            console.error(text);
-            // Send stderr output directly too
-            sendProgressToClients(clients, text);
-        });
-
-        child.on('close', (code) => {
-            switch (type) {
-                case 'update':
-                    activeFullUpdate = null;
-                    break;
-                case 'reset':
-                    activeFullReset = null;
-                    break;
-            }
-
-            if (code !== 0) {
-                const error = `Script ${scriptPath} exited with code ${code}`;
-                sendProgressToClients(clients, error);
-                reject(new Error(error));
-            }
-            // Don't resolve here - let the completion message from the script trigger the resolve
-        });
-
-        child.on('error', (err) => {
-            switch (type) {
-                case 'update':
-                    activeFullUpdate = null;
-                    break;
-                case 'reset':
-                    activeFullReset = null;
-                    break;
-            }
-            sendProgressToClients(clients, err.message);
-            reject(err);
-        });
-    });
-}
-
-// Progress endpoints
-router.get('/:type/progress', (req, res) => {
-    const { type } = req.params;
-    if (!['update', 'reset'].includes(type)) {
-        return res.status(400).json({ error: 'Invalid operation type' });
-    }
-
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Add this client to the correct set
-    const clients = type === 'update' ? fullUpdateClients : fullResetClients;
-    clients.add(res);
-
-    // Send initial connection message
-    sendProgressToClients(new Set([res]), JSON.stringify({
-        status: 'running',
-        operation: 'Initializing connection...'
-    }));
-
-    // Handle client disconnect
-    req.on('close', () => {
-        clients.delete(res);
-    });
-});
-
-// Debug endpoint to verify route registration
-router.get('/test', (req, res) => {
-    console.log('CSV test endpoint hit');
-    res.json({ message: 'CSV routes are working' });
-});
-
-// Route to check import status
-router.get('/status', (req, res) => {
-    console.log('CSV status endpoint hit');
-    res.json({
-        active: !!activeImport,
-        progress: importProgress
-    });
-});
-
-// Add calculate-metrics status endpoint
-router.get('/calculate-metrics/status', (req, res) => {
-    const calculateMetrics = require('../../scripts/calculate-metrics');
-    const progress = calculateMetrics.getProgress();
-    
-    // Only consider it active if both the process is running and we have progress
-    const isActive = !!activeImport && !!progress;
-    
-    res.json({
-        active: isActive,
-        progress: isActive ? progress : null
-    });
-});
-
-// Route to update CSV files
-router.post('/update', async (req, res, next) => {
-    if (activeImport) {
-        return res.status(409).json({ error: 'Import already in progress' });
-    }
-
-    try {
-        const scriptPath = path.join(__dirname, '..', '..', 'scripts', 'update-csv.js');
-        
-        if (!require('fs').existsSync(scriptPath)) {
-            return res.status(500).json({ error: 'Update script not found' });
-        }
-        
-        activeImport = spawn('node', [scriptPath]);
-        
-        activeImport.stdout.on('data', (data) => {
-            const output = data.toString().trim();
-            
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(output);
-                sendProgressToClients(updateClients, {
-                    status: 'running',
-                    ...jsonData
-                });
-            } catch (e) {
-                // If not JSON, send as plain progress
-                sendProgressToClients(updateClients, { 
-                    status: 'running',
-                    progress: output
-                });
-            }
-        });
-
-        activeImport.stderr.on('data', (data) => {
-            const error = data.toString().trim();
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(error);
-                sendProgressToClients(updateClients, { 
-                    status: 'error',
-                    ...jsonData
-                });
-            } catch {
-                sendProgressToClients(updateClients, { 
-                    status: 'error',
-                    error
-                });
-            }
-        });
-
-        await new Promise((resolve, reject) => {
-            activeImport.on('close', (code) => {
-                // Don't treat cancellation (code 143/SIGTERM) as an error
-                if (code === 0 || code === 143) {
-                    sendProgressToClients(updateClients, { 
-                        status: 'complete',
-                        operation: code === 143 ? 'Operation cancelled' : 'Update complete'
-                    });
-                    resolve();
-                } else {
-                    const errorMsg = `Update process exited with code ${code}`;
-                    sendProgressToClients(updateClients, { 
-                        status: 'error',
-                        error: errorMsg
-                    });
-                    reject(new Error(errorMsg));
-                }
-                activeImport = null;
-                importProgress = null;
-            });
-        });
-
-        res.json({ success: true });
-    } catch (error) {
-        console.error('Error updating CSV files:', error);
-        activeImport = null;
-        importProgress = null;
-        sendProgressToClients(updateClients, { 
-            status: 'error',
-            error: error.message
-        });
-        next(error);
-    }
-});
-
-// Route to import CSV files
-router.post('/import', async (req, res) => {
-    if (activeImport) {
-        return res.status(409).json({ error: 'Import already in progress' });
-    }
-
-    try {
-        const scriptPath = path.join(__dirname, '..', '..', 'scripts', 'import-csv.js');
-        
-        if (!require('fs').existsSync(scriptPath)) {
-            return res.status(500).json({ error: 'Import script not found' });
-        }
-        
-        // Get test limits from request body
-        const { products = 0, orders = 10000, purchaseOrders = 10000 } = req.body;
-        
-        // Create environment variables for the script
-        const env = {
-            ...process.env,
-            PRODUCTS_TEST_LIMIT: products.toString(),
-            ORDERS_TEST_LIMIT: orders.toString(),
-            PURCHASE_ORDERS_TEST_LIMIT: purchaseOrders.toString()
-        };
-        
-        activeImport = spawn('node', [scriptPath], { env });
-        
-        activeImport.stdout.on('data', (data) => {
-            const output = data.toString().trim();
-            
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(output);
-                sendProgressToClients(importClients, {
-                    status: 'running',
-                    ...jsonData
-                });
-            } catch {
-                // If not JSON, send as plain progress
-                sendProgressToClients(importClients, { 
-                    status: 'running',
-                    progress: output
-                });
-            }
-        });
-
-        activeImport.stderr.on('data', (data) => {
-            const error = data.toString().trim();
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(error);
-                sendProgressToClients(importClients, { 
-                    status: 'error',
-                    ...jsonData
-                });
-            } catch {
-                sendProgressToClients(importClients, { 
-                    status: 'error',
-                    error
-                });
-            }
-        });
-
-        await new Promise((resolve, reject) => {
-            activeImport.on('close', (code) => {
-                // Don't treat cancellation (code 143/SIGTERM) as an error
-                if (code === 0 || code === 143) {
-                    sendProgressToClients(importClients, { 
-                        status: 'complete',
-                        operation: code === 143 ? 'Operation cancelled' : 'Import complete'
-                    });
-                    resolve();
-                } else {
-                    sendProgressToClients(importClients, { 
-                        status: 'error',
-                        error: `Process exited with code ${code}`
-                    });
-                    reject(new Error(`Import process exited with code ${code}`));
-                }
-                activeImport = null;
-                importProgress = null;
-            });
-        });
-
-        res.json({ success: true });
-    } catch (error) {
-        console.error('Error importing CSV files:', error);
-        activeImport = null;
-        importProgress = null;
-        sendProgressToClients(importClients, { 
-            status: 'error',
-            error: error.message
-        });
-        res.status(500).json({ error: 'Failed to import CSV files', details: error.message });
-    }
-});
-
-// Route to cancel active process
-router.post('/cancel', (req, res) => {
-    let killed = false;
-
-    // Get the operation type from the request
-    const { type } = req.query;
-    const clients = type === 'update' ? fullUpdateClients : fullResetClients;
-    const activeProcess = type === 'update' ? activeFullUpdate : activeFullReset;
-
-    if (activeProcess) {
-        try {
-            activeProcess.kill('SIGTERM');
-            if (type === 'update') {
-                activeFullUpdate = null;
-            } else {
-                activeFullReset = null;
-            }
-            killed = true;
-            sendProgressToClients(clients, JSON.stringify({
-                status: 'cancelled',
-                operation: 'Operation cancelled'
-            }));
-        } catch (err) {
-            console.error(`Error killing ${type} process:`, err);
-        }
-    }
-
-    if (killed) {
-        res.json({ success: true });
-    } else {
-        res.status(404).json({ error: 'No active process to cancel' });
-    }
-});
-
-// Route to reset database
-router.post('/reset', async (req, res) => {
-    if (activeImport) {
-        return res.status(409).json({ error: 'Import already in progress' });
-    }
-
-    try {
-        const scriptPath = path.join(__dirname, '..', '..', 'scripts', 'reset-db.js');
-        
-        if (!require('fs').existsSync(scriptPath)) {
-            return res.status(500).json({ error: 'Reset script not found' });
-        }
-        
-        activeImport = spawn('node', [scriptPath]);
-        
-        activeImport.stdout.on('data', (data) => {
-            const output = data.toString().trim();
-            
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(output);
-                sendProgressToClients(resetClients, {
-                    status: 'running',
-                    ...jsonData
-                });
-            } catch (e) {
-                // If not JSON, send as plain progress
-                sendProgressToClients(resetClients, { 
-                    status: 'running',
-                    progress: output
-                });
-            }
-        });
-
-        activeImport.stderr.on('data', (data) => {
-            const error = data.toString().trim();
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(error);
-                sendProgressToClients(resetClients, { 
-                    status: 'error',
-                    ...jsonData
-                });
-            } catch {
-                sendProgressToClients(resetClients, { 
-                    status: 'error',
-                    error
-                });
-            }
-        });
-
-        await new Promise((resolve, reject) => {
-            activeImport.on('close', (code) => {
-                // Don't treat cancellation (code 143/SIGTERM) as an error
-                if (code === 0 || code === 143) {
-                    sendProgressToClients(resetClients, { 
-                        status: 'complete',
-                        operation: code === 143 ? 'Operation cancelled' : 'Reset complete'
-                    });
-                    resolve();
-                } else {
-                    const errorMsg = `Reset process exited with code ${code}`;
-                    sendProgressToClients(resetClients, { 
-                        status: 'error',
-                        error: errorMsg
-                    });
-                    reject(new Error(errorMsg));
-                }
-                activeImport = null;
-                importProgress = null;
-            });
-        });
-
-        res.json({ success: true });
-    } catch (error) {
-        console.error('Error resetting database:', error);
-        activeImport = null;
-        importProgress = null;
-        sendProgressToClients(resetClients, { 
-            status: 'error',
-            error: error.message
-        });
-        res.status(500).json({ error: 'Failed to reset database', details: error.message });
-    }
-});
-
-// Add reset-metrics endpoint
-router.post('/reset-metrics', async (req, res) => {
-    if (activeImport) {
-        res.status(400).json({ error: 'Operation already in progress' });
-        return;
-    }
-
-    try {
-        // Set active import to prevent concurrent operations
-        activeImport = {
-            type: 'reset-metrics',
-            status: 'running',
-            operation: 'Starting metrics reset'
-        };
-
-        // Send initial response
-        res.status(200).json({ message: 'Reset metrics started' });
-
-        // Send initial progress through SSE
-        sendProgressToClients(resetMetricsClients, {
-            status: 'running',
-            operation: 'Starting metrics reset'
-        });
-
-        // Run the reset metrics script
-        const resetMetrics = require('../../scripts/reset-metrics');
-        await resetMetrics();
-
-        // Send completion through SSE
-        sendProgressToClients(resetMetricsClients, {
-            status: 'complete',
-            operation: 'Metrics reset completed'
-        });
-
-        activeImport = null;
-    } catch (error) {
-        console.error('Error during metrics reset:', error);
-        
-        // Send error through SSE
-        sendProgressToClients(resetMetricsClients, {
-            status: 'error',
-            error: error.message || 'Failed to reset metrics'
-        });
-        
-        activeImport = null;
-        res.status(500).json({ error: error.message || 'Failed to reset metrics' });
-    }
-});
-
-// Add calculate-metrics endpoint
-router.post('/calculate-metrics', async (req, res) => {
-    if (activeImport) {
-        return res.status(409).json({ error: 'Import already in progress' });
-    }
-
-    try {
-        const scriptPath = path.join(__dirname, '..', '..', 'scripts', 'calculate-metrics.js');
-        
-        if (!require('fs').existsSync(scriptPath)) {
-            return res.status(500).json({ error: 'Calculate metrics script not found' });
-        }
-        
-        activeImport = spawn('node', [scriptPath]);
-        let wasCancelled = false;
-        
-        activeImport.stdout.on('data', (data) => {
-            const output = data.toString().trim();
-            
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(output);
-                importProgress = {
-                    status: 'running',
-                    ...jsonData.progress
-                };
-                sendProgressToClients(calculateMetricsClients, importProgress);
-            } catch (e) {
-                // If not JSON, send as plain progress
-                importProgress = { 
-                    status: 'running',
-                    progress: output
-                };
-                sendProgressToClients(calculateMetricsClients, importProgress);
-            }
-        });
-
-        activeImport.stderr.on('data', (data) => {
-            if (wasCancelled) return; // Don't send errors if cancelled
-            
-            const error = data.toString().trim();
-            try {
-                // Try to parse as JSON
-                const jsonData = JSON.parse(error);
-                importProgress = { 
-                    status: 'error',
-                    ...jsonData.progress
-                };
-                sendProgressToClients(calculateMetricsClients, importProgress);
-            } catch {
-                importProgress = { 
-                    status: 'error',
-                    error
-                };
-                sendProgressToClients(calculateMetricsClients, importProgress);
-            }
-        });
-
-        await new Promise((resolve, reject) => {
-            activeImport.on('close', (code, signal) => {
-                wasCancelled = signal === 'SIGTERM' || code === 143;
-                activeImport = null;
-                
-                if (code === 0 || wasCancelled) {
-                    if (wasCancelled) {
-                        importProgress = {
-                            status: 'cancelled',
-                            operation: 'Operation cancelled'
-                        };
-                        sendProgressToClients(calculateMetricsClients, importProgress);
-                    } else {
-                        importProgress = {
-                            status: 'complete',
-                            operation: 'Metrics calculation complete'
-                        };
-                        sendProgressToClients(calculateMetricsClients, importProgress);
-                    }
-                    resolve();
-                } else {
-                    importProgress = null;
-                    reject(new Error(`Metrics calculation process exited with code ${code}`));
-                }
-            });
-        });
-
-        res.json({ success: true });
-    } catch (error) {
-        console.error('Error calculating metrics:', error);
-        activeImport = null;
-        importProgress = null;
-        
-        // Only send error if it wasn't a cancellation
-        if (!error.message?.includes('code 143') && !error.message?.includes('SIGTERM')) {
-            sendProgressToClients(calculateMetricsClients, { 
-                status: 'error',
-                error: error.message
-            });
-            res.status(500).json({ error: 'Failed to calculate metrics', details: error.message });
-        } else {
-            res.json({ success: true });
-        }
-    }
-});
-
-// Route to import from production database
-router.post('/import-from-prod', async (req, res) => {
-    if (activeImport) {
-        return res.status(409).json({ error: 'Import already in progress' });
-    }
-
-    try {
-        const importFromProd = require('../../scripts/import-from-prod');
-        
-        // Set up progress handler
-        const progressHandler = (data) => {
-            importProgress = data;
-            sendProgressToClients(importClients, data);
-        };
-
-        // Start the import process
-        importFromProd.outputProgress = progressHandler;
-        activeImport = importFromProd; // Store the module for cancellation
-        
-        // Run the import in the background
-        importFromProd.main().catch(error => {
-            console.error('Error in import process:', error);
-            activeImport = null;
-            importProgress = {
-                status: error.message === 'Import cancelled' ? 'cancelled' : 'error',
-                operation: 'Import process',
-                error: error.message
-            };
-            sendProgressToClients(importClients, importProgress);
-        }).finally(() => {
-            activeImport = null;
-        });
-
-        res.json({ message: 'Import from production started' });
-    } catch (error) {
-        console.error('Error starting production import:', error);
-        activeImport = null;
-        res.status(500).json({ error: error.message || 'Failed to start production import' });
-    }
-});
-
-// POST /csv/full-update - Run full update script
-router.post('/full-update', async (req, res) => {
-    try {
-        const scriptPath = path.join(__dirname, '../../scripts/full-update.js');
-        runScript(scriptPath, 'update', fullUpdateClients)
-            .catch(error => {
-                console.error('Update failed:', error);
-            });
-        res.status(202).json({ message: 'Update started' });
-    } catch (error) {
-        res.status(500).json({ error: error.message });
-    }
-});
-
-// POST /csv/full-reset - Run full reset script
-router.post('/full-reset', async (req, res) => {
-    try {
-        const scriptPath = path.join(__dirname, '../../scripts/full-reset.js');
-        runScript(scriptPath, 'reset', fullResetClients)
-            .catch(error => {
-                console.error('Reset failed:', error);
-            });
-        res.status(202).json({ message: 'Reset started' });
-    } catch (error) {
-        res.status(500).json({ error: error.message });
-    }
-});
-
-// GET /history/import - Get recent import history
-router.get('/history/import', async (req, res) => {
-    try {
-        const pool = req.app.locals.pool;
-        const { rows } = await pool.query(`
-            SELECT 
-                id,
-                start_time,
-                end_time,
-                status,
-                error_message,
-                rows_processed::integer,
-                files_processed::integer
-            FROM import_history 
-            ORDER BY start_time DESC 
-            LIMIT 20
-        `);
-        res.json(rows || []);
-    } catch (error) {
-        console.error('Error fetching import history:', error);
-        res.status(500).json({ error: error.message });
-    }
-});
-
-// GET /history/calculate - Get recent calculation history
-router.get('/history/calculate', async (req, res) => {
-    try {
-        const pool = req.app.locals.pool;
-        const { rows } = await pool.query(`
-            SELECT 
-                id,
-                start_time,
-                end_time,
-                status,
-                error_message,
-                modules_processed::integer,
-                total_modules::integer
-            FROM calculate_history 
-            ORDER BY start_time DESC 
-            LIMIT 20
-        `);
-        res.json(rows || []);
-    } catch (error) {
-        console.error('Error fetching calculate history:', error);
-        res.status(500).json({ error: error.message });
-    }
-});
-
-// GET /status/modules - Get module calculation status
-router.get('/status/modules', async (req, res) => {
-    try {
-        const pool = req.app.locals.pool;
-        const { rows } = await pool.query(`
-            SELECT 
-                module_name, 
-                last_calculation_timestamp::timestamp 
-            FROM calculate_status 
-            ORDER BY module_name
-        `);
-        res.json(rows || []);
-    } catch (error) {
-        console.error('Error fetching module status:', error);
-        res.status(500).json({ error: error.message });
-    }
-});
-
-// GET /status/tables - Get table sync status
-router.get('/status/tables', async (req, res) => {
-    try {
-        const pool = req.app.locals.pool;
-        const { rows } = await pool.query(`
-            SELECT 
-                table_name, 
-                last_sync_timestamp::timestamp 
-            FROM sync_status 
-            ORDER BY table_name
-        `);
-        res.json(rows || []);
-    } catch (error) {
-        console.error('Error fetching table status:', error);
-        res.status(500).json({ error: error.message });
-    }
-});
-
-module.exports = router; 

inventory-server/src/routes/dashboard.js

@@ -1,933 +1 @@
-const express = require('express');
-const router = express.Router();
-const db = require('../utils/db');
  
-// Import status codes
-const { ReceivingStatus } = require('../types/status-codes');
-
-// Helper function to execute queries using the connection pool
-async function executeQuery(sql, params = []) {
-    const pool = db.getPool();
-    if (!pool) {
-        throw new Error('Database pool not initialized');
-    }
-    return pool.query(sql, params);
-}
-
-// GET /dashboard/stock/metrics
-// Returns brand-level stock metrics
-router.get('/stock/metrics', async (req, res) => {
-    try {
-        // Get stock metrics
-        const { rows: [stockMetrics] } = await executeQuery(`
-            SELECT 
-                COALESCE(COUNT(*), 0)::integer as total_products,
-                COALESCE(COUNT(CASE WHEN stock_quantity > 0 THEN 1 END), 0)::integer as products_in_stock,
-                COALESCE(SUM(CASE WHEN stock_quantity > 0 THEN stock_quantity END), 0)::integer as total_units,
-                ROUND(COALESCE(SUM(CASE WHEN stock_quantity > 0 THEN stock_quantity * cost_price END), 0)::numeric, 3) as total_cost,
-                ROUND(COALESCE(SUM(CASE WHEN stock_quantity > 0 THEN stock_quantity * price END), 0)::numeric, 3) as total_retail
-            FROM products
-        `);
-
-        console.log('Raw stockMetrics from database:', stockMetrics);
-        console.log('stockMetrics.total_products:', stockMetrics.total_products);
-        console.log('stockMetrics.products_in_stock:', stockMetrics.products_in_stock);
-        console.log('stockMetrics.total_units:', stockMetrics.total_units);
-        console.log('stockMetrics.total_cost:', stockMetrics.total_cost);
-        console.log('stockMetrics.total_retail:', stockMetrics.total_retail);
-
-        // Get brand stock values with Other category
-        const { rows: brandValues } = await executeQuery(`
-            WITH brand_totals AS (
-                SELECT 
-                    COALESCE(brand, 'Unbranded') as brand,
-                    COUNT(DISTINCT pid)::integer as variant_count,
-                    COALESCE(SUM(stock_quantity), 0)::integer as stock_units,
-                    ROUND(COALESCE(SUM(stock_quantity * cost_price), 0)::numeric, 3) as stock_cost,
-                    ROUND(COALESCE(SUM(stock_quantity * price), 0)::numeric, 3) as stock_retail
-                FROM products
-                WHERE stock_quantity > 0
-                GROUP BY COALESCE(brand, 'Unbranded')
-                HAVING ROUND(COALESCE(SUM(stock_quantity * cost_price), 0)::numeric, 3) > 0
-            ),
-            other_brands AS (
-                SELECT 
-                    'Other' as brand,
-                    SUM(variant_count)::integer as variant_count,
-                    SUM(stock_units)::integer as stock_units,
-                    ROUND(SUM(stock_cost)::numeric, 3) as stock_cost,
-                    ROUND(SUM(stock_retail)::numeric, 3) as stock_retail
-                FROM brand_totals
-                WHERE stock_cost <= 5000
-            ),
-            main_brands AS (
-                SELECT *
-                FROM brand_totals
-                WHERE stock_cost > 5000
-                ORDER BY stock_cost DESC
-            )
-            SELECT * FROM main_brands
-            UNION ALL
-            SELECT * FROM other_brands
-            WHERE stock_cost > 0
-            ORDER BY CASE WHEN brand = 'Other' THEN 1 ELSE 0 END, stock_cost DESC
-        `);
-
-        // Format the response with explicit type conversion
-        const response = {
-            totalProducts: parseInt(stockMetrics.total_products) || 0,
-            productsInStock: parseInt(stockMetrics.products_in_stock) || 0,
-            totalStockUnits: parseInt(stockMetrics.total_units) || 0,
-            totalStockCost: parseFloat(stockMetrics.total_cost) || 0,
-            totalStockRetail: parseFloat(stockMetrics.total_retail) || 0,
-            brandStock: brandValues.map(v => ({
-                brand: v.brand,
-                variants: parseInt(v.variant_count) || 0,
-                units: parseInt(v.stock_units) || 0,
-                cost: parseFloat(v.stock_cost) || 0,
-                retail: parseFloat(v.stock_retail) || 0
-            }))
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching stock metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch stock metrics' });
-    }
-});
-
-// GET /dashboard/purchase/metrics
-// Returns purchase order metrics by vendor
-router.get('/purchase/metrics', async (req, res) => {
-    try {
-        const { rows: [poMetrics] } = await executeQuery(`
-            SELECT 
-                COALESCE(COUNT(DISTINCT CASE 
-                    WHEN po.receiving_status < $1 
-                    THEN po.po_id 
-                END), 0)::integer as active_pos,
-                COALESCE(COUNT(DISTINCT CASE 
-                    WHEN po.receiving_status < $1 
-                    AND po.expected_date < CURRENT_DATE 
-                    THEN po.po_id 
-                END), 0)::integer as overdue_pos,
-                COALESCE(SUM(CASE 
-                    WHEN po.receiving_status < $1 
-                    THEN po.ordered 
-                    ELSE 0 
-                END), 0)::integer as total_units,
-                ROUND(COALESCE(SUM(CASE 
-                    WHEN po.receiving_status < $1 
-                    THEN po.ordered * po.cost_price 
-                    ELSE 0 
-                END), 0)::numeric, 3) as total_cost,
-                ROUND(COALESCE(SUM(CASE 
-                    WHEN po.receiving_status < $1 
-                    THEN po.ordered * p.price 
-                    ELSE 0 
-                END), 0)::numeric, 3) as total_retail
-            FROM purchase_orders po
-            JOIN products p ON po.pid = p.pid
-        `, [ReceivingStatus.PartialReceived]);
-
-        const { rows: vendorOrders } = await executeQuery(`
-            SELECT 
-                po.vendor,
-                COUNT(DISTINCT po.po_id)::integer as orders,
-                COALESCE(SUM(po.ordered), 0)::integer as units,
-                ROUND(COALESCE(SUM(po.ordered * po.cost_price), 0)::numeric, 3) as cost,
-                ROUND(COALESCE(SUM(po.ordered * p.price), 0)::numeric, 3) as retail
-            FROM purchase_orders po
-            JOIN products p ON po.pid = p.pid
-            WHERE po.receiving_status < $1
-            GROUP BY po.vendor
-            HAVING ROUND(COALESCE(SUM(po.ordered * po.cost_price), 0)::numeric, 3) > 0
-            ORDER BY cost DESC
-        `, [ReceivingStatus.PartialReceived]);
-
-        // Format response to match PurchaseMetricsData interface
-        const response = {
-            activePurchaseOrders: parseInt(poMetrics.active_pos) || 0,
-            overduePurchaseOrders: parseInt(poMetrics.overdue_pos) || 0,
-            onOrderUnits: parseInt(poMetrics.total_units) || 0,
-            onOrderCost: parseFloat(poMetrics.total_cost) || 0,
-            onOrderRetail: parseFloat(poMetrics.total_retail) || 0,
-            vendorOrders: vendorOrders.map(v => ({
-                vendor: v.vendor,
-                orders: parseInt(v.orders) || 0,
-                units: parseInt(v.units) || 0,
-                cost: parseFloat(v.cost) || 0,
-                retail: parseFloat(v.retail) || 0
-            }))
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching purchase metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch purchase metrics' });
-    }
-});
-
-// GET /dashboard/replenishment/metrics
-// Returns replenishment needs by category
-router.get('/replenishment/metrics', async (req, res) => {
-    try {
-        // Get summary metrics
-        const { rows: [metrics] } = await executeQuery(`
-            SELECT 
-                COUNT(DISTINCT p.pid)::integer as products_to_replenish,
-                COALESCE(SUM(CASE 
-                    WHEN p.stock_quantity < 0 THEN ABS(p.stock_quantity) + pm.reorder_qty
-                    ELSE pm.reorder_qty 
-                END), 0)::integer as total_units_needed,
-                ROUND(COALESCE(SUM(CASE 
-                    WHEN p.stock_quantity < 0 THEN (ABS(p.stock_quantity) + pm.reorder_qty) * p.cost_price
-                    ELSE pm.reorder_qty * p.cost_price 
-                END), 0)::numeric, 3) as total_cost,
-                ROUND(COALESCE(SUM(CASE 
-                    WHEN p.stock_quantity < 0 THEN (ABS(p.stock_quantity) + pm.reorder_qty) * p.price
-                    ELSE pm.reorder_qty * p.price 
-                END), 0)::numeric, 3) as total_retail
-            FROM products p
-            JOIN product_metrics pm ON p.pid = pm.pid
-            WHERE p.replenishable = true
-                AND (pm.stock_status IN ('Critical', 'Reorder')
-                    OR p.stock_quantity < 0)
-                AND pm.reorder_qty > 0
-        `);
-
-        // Get top variants to replenish
-        const { rows: variants } = await executeQuery(`
-            SELECT 
-                p.pid,
-                p.title,
-                p.stock_quantity::integer as current_stock,
-                CASE 
-                    WHEN p.stock_quantity < 0 THEN ABS(p.stock_quantity) + pm.reorder_qty
-                    ELSE pm.reorder_qty 
-                END::integer as replenish_qty,
-                ROUND(CASE 
-                    WHEN p.stock_quantity < 0 THEN (ABS(p.stock_quantity) + pm.reorder_qty) * p.cost_price
-                    ELSE pm.reorder_qty * p.cost_price 
-                END::numeric, 3) as replenish_cost,
-                ROUND(CASE 
-                    WHEN p.stock_quantity < 0 THEN (ABS(p.stock_quantity) + pm.reorder_qty) * p.price
-                    ELSE pm.reorder_qty * p.price 
-                END::numeric, 3) as replenish_retail,
-                pm.stock_status
-            FROM products p
-            JOIN product_metrics pm ON p.pid = pm.pid
-            WHERE p.replenishable = true
-                AND (pm.stock_status IN ('Critical', 'Reorder')
-                    OR p.stock_quantity < 0)
-                AND pm.reorder_qty > 0
-            ORDER BY 
-                CASE pm.stock_status 
-                    WHEN 'Critical' THEN 1 
-                    WHEN 'Reorder' THEN 2 
-                END,
-                replenish_cost DESC
-            LIMIT 5
-        `);
-
-        // Format response
-        const response = {
-            productsToReplenish: parseInt(metrics.products_to_replenish) || 0,
-            unitsToReplenish: parseInt(metrics.total_units_needed) || 0,
-            replenishmentCost: parseFloat(metrics.total_cost) || 0,
-            replenishmentRetail: parseFloat(metrics.total_retail) || 0,
-            topVariants: variants.map(v => ({
-                id: v.pid,
-                title: v.title,
-                currentStock: parseInt(v.current_stock) || 0,
-                replenishQty: parseInt(v.replenish_qty) || 0,
-                replenishCost: parseFloat(v.replenish_cost) || 0,
-                replenishRetail: parseFloat(v.replenish_retail) || 0,
-                status: v.stock_status
-            }))
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching replenishment metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch replenishment metrics' });
-    }
-});
-
-// GET /dashboard/forecast/metrics
-// Returns sales forecasts for specified period
-router.get('/forecast/metrics', async (req, res) => {
-    const { startDate, endDate } = req.query;
-    try {
-        // Get summary metrics
-        const [metrics] = await executeQuery(`
-            SELECT 
-                COALESCE(SUM(forecast_units), 0) as total_forecast_units,
-                COALESCE(SUM(forecast_revenue), 0) as total_forecast_revenue,
-                COALESCE(AVG(confidence_level), 0) as overall_confidence
-            FROM sales_forecasts
-            WHERE forecast_date BETWEEN ? AND ?
-        `, [startDate, endDate]);
-
-        // Get daily forecasts
-        const [dailyForecasts] = await executeQuery(`
-            SELECT 
-                DATE(forecast_date) as date,
-                COALESCE(SUM(forecast_revenue), 0) as revenue,
-                COALESCE(AVG(confidence_level), 0) as confidence
-            FROM sales_forecasts
-            WHERE forecast_date BETWEEN ? AND ?
-            GROUP BY DATE(forecast_date)
-            ORDER BY date
-        `, [startDate, endDate]);
-
-        // Get category forecasts
-        const [categoryForecasts] = await executeQuery(`
-            SELECT 
-                c.name as category,
-                COALESCE(SUM(cf.forecast_units), 0) as units,
-                COALESCE(SUM(cf.forecast_revenue), 0) as revenue,
-                COALESCE(AVG(cf.confidence_level), 0) as confidence
-            FROM category_forecasts cf
-            JOIN categories c ON cf.category_id = c.cat_id
-            WHERE cf.forecast_date BETWEEN ? AND ?
-            GROUP BY c.cat_id, c.name
-            ORDER BY revenue DESC
-        `, [startDate, endDate]);
-
-        // Format response
-        const response = {
-            forecastSales: parseInt(metrics[0].total_forecast_units) || 0,
-            forecastRevenue: parseFloat(metrics[0].total_forecast_revenue) || 0,
-            confidenceLevel: parseFloat(metrics[0].overall_confidence) || 0,
-            dailyForecasts: dailyForecasts.map(d => ({
-                date: d.date,
-                revenue: parseFloat(d.revenue) || 0,
-                confidence: parseFloat(d.confidence) || 0
-            })),
-            categoryForecasts: categoryForecasts.map(c => ({
-                category: c.category,
-                units: parseInt(c.units) || 0,
-                revenue: parseFloat(c.revenue) || 0,
-                confidence: parseFloat(c.confidence) || 0
-            }))
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching forecast metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch forecast metrics' });
-    }
-});
-
-// GET /dashboard/overstock/metrics
-// Returns overstock metrics by category
-router.get('/overstock/metrics', async (req, res) => {
-    try {
-        const [rows] = await executeQuery(`
-            WITH category_overstock AS (
-                SELECT 
-                    c.cat_id,
-                    c.name as category_name,
-                    COUNT(DISTINCT CASE 
-                        WHEN pm.stock_status = 'Overstocked' 
-                        THEN p.pid 
-                        END) as overstocked_products,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Overstocked' 
-                        THEN pm.overstocked_amt 
-                        ELSE 0 
-                        END) as total_excess_units,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Overstocked' 
-                        THEN pm.overstocked_amt * p.cost_price 
-                        ELSE 0 
-                        END) as total_excess_cost,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Overstocked' 
-                        THEN pm.overstocked_amt * p.price 
-                        ELSE 0 
-                        END) as total_excess_retail
-                FROM categories c
-                JOIN product_categories pc ON c.cat_id = pc.cat_id
-                JOIN products p ON pc.pid = p.pid
-                JOIN product_metrics pm ON p.pid = pm.pid
-                GROUP BY c.cat_id, c.name
-            )
-            SELECT 
-                SUM(overstocked_products) as total_overstocked,
-                SUM(total_excess_units) as total_excess_units,
-                SUM(total_excess_cost) as total_excess_cost,
-                SUM(total_excess_retail) as total_excess_retail,
-                CONCAT('[', GROUP_CONCAT(
-                    JSON_OBJECT(
-                        'category', category_name,
-                        'products', overstocked_products,
-                        'units', total_excess_units,
-                        'cost', total_excess_cost,
-                        'retail', total_excess_retail
-                    )
-                ), ']') as category_data
-            FROM (
-                SELECT *
-                FROM category_overstock
-                WHERE overstocked_products > 0
-                ORDER BY total_excess_cost DESC
-                LIMIT 8
-            ) filtered_categories
-        `);
-
-        // Format response with explicit type conversion
-        const response = {
-            overstockedProducts: parseInt(rows[0].total_overstocked) || 0,
-            total_excess_units: parseInt(rows[0].total_excess_units) || 0,
-            total_excess_cost: parseFloat(rows[0].total_excess_cost) || 0,
-            total_excess_retail: parseFloat(rows[0].total_excess_retail) || 0,
-            category_data: rows[0].category_data ? 
-                JSON.parse(rows[0].category_data).map(obj => ({
-                    category: obj.category,
-                    products: parseInt(obj.products) || 0,
-                    units: parseInt(obj.units) || 0,
-                    cost: parseFloat(obj.cost) || 0,
-                    retail: parseFloat(obj.retail) || 0
-                })) : []
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching overstock metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch overstock metrics' });
-    }
-});
-
-// GET /dashboard/overstock/products
-// Returns list of most overstocked products
-router.get('/overstock/products', async (req, res) => {
-    const limit = parseInt(req.query.limit) || 50;
-    try {
-        const [rows] = await executeQuery(`
-            SELECT 
-                p.pid,
-                p.SKU,
-                p.title,
-                p.brand,
-                p.vendor,
-                p.stock_quantity,
-                p.cost_price,
-                p.price,
-                pm.daily_sales_avg,
-                pm.days_of_inventory,
-                pm.overstocked_amt,
-                (pm.overstocked_amt * p.cost_price) as excess_cost,
-                (pm.overstocked_amt * p.price) as excess_retail,
-                GROUP_CONCAT(c.name) as categories
-            FROM products p
-            JOIN product_metrics pm ON p.pid = pm.pid
-            LEFT JOIN product_categories pc ON p.pid = pc.pid
-            LEFT JOIN categories c ON pc.cat_id = c.cat_id
-            WHERE pm.stock_status = 'Overstocked'
-            GROUP BY p.pid
-            ORDER BY excess_cost DESC
-            LIMIT ?
-        `, [limit]);
-        res.json(rows);
-    } catch (err) {
-        console.error('Error fetching overstocked products:', err);
-        res.status(500).json({ error: 'Failed to fetch overstocked products' });
-    }
-});
-
-// GET /dashboard/best-sellers
-// Returns best-selling products, vendors, and categories
-router.get('/best-sellers', async (req, res) => {
-    try {
-        const pool = req.app.locals.pool;
-
-        // Common CTE for category paths
-        const categoryPathCTE = `
-            WITH RECURSIVE category_path AS (
-                SELECT 
-                    c.cat_id,
-                    c.name,
-                    c.parent_id,
-                    CAST(c.name AS CHAR(1000)) as path
-                FROM categories c
-                WHERE c.parent_id IS NULL
-                
-                UNION ALL
-                
-                SELECT 
-                    c.cat_id,
-                    c.name,
-                    c.parent_id,
-                    CONCAT(cp.path, ' > ', c.name)
-                FROM categories c
-                JOIN category_path cp ON c.parent_id = cp.cat_id
-            )
-        `;
-
-        // Get best selling products
-        const [products] = await pool.query(`
-            SELECT 
-                p.pid,
-                p.SKU as sku,
-                p.title,
-                SUM(o.quantity) as units_sold,
-                CAST(SUM(o.price * o.quantity) AS DECIMAL(15,3)) as revenue,
-                CAST(SUM(o.price * o.quantity - p.cost_price * o.quantity) AS DECIMAL(15,3)) as profit
-            FROM products p
-            JOIN orders o ON p.pid = o.pid
-            WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-                AND o.canceled = false
-            GROUP BY p.pid
-            ORDER BY units_sold DESC
-            LIMIT 10
-        `);
-
-        // Get best selling brands
-        const [brands] = await pool.query(`
-            SELECT 
-                p.brand,
-                SUM(o.quantity) as units_sold,
-                CAST(SUM(o.price * o.quantity) AS DECIMAL(15,3)) as revenue,
-                CAST(SUM(o.price * o.quantity - p.cost_price * o.quantity) AS DECIMAL(15,3)) as profit,
-                ROUND(
-                    ((SUM(CASE 
-                        WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY) 
-                        THEN o.price * o.quantity 
-                        ELSE 0 
-                    END) / 
-                    NULLIF(SUM(CASE 
-                        WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY) 
-                        AND o.date < DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-                        THEN o.price * o.quantity 
-                        ELSE 0 
-                    END), 0)) - 1) * 100,
-                    1
-                ) as growth_rate
-            FROM products p
-            JOIN orders o ON p.pid = o.pid
-            WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY)
-                AND o.canceled = false
-            GROUP BY p.brand
-            ORDER BY units_sold DESC
-            LIMIT 10
-        `);
-
-        // Get best selling categories with full path
-        const [categories] = await pool.query(`
-            ${categoryPathCTE}
-            SELECT 
-                c.cat_id,
-                c.name,
-                cp.path as categoryPath,
-                SUM(o.quantity) as units_sold,
-                CAST(SUM(o.price * o.quantity) AS DECIMAL(15,3)) as revenue,
-                CAST(SUM(o.price * o.quantity - p.cost_price * o.quantity) AS DECIMAL(15,3)) as profit,
-                ROUND(
-                    ((SUM(CASE 
-                        WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY) 
-                        THEN o.price * o.quantity 
-                        ELSE 0 
-                    END) / 
-                    NULLIF(SUM(CASE 
-                        WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY) 
-                        AND o.date < DATE_SUB(CURDATE(), INTERVAL 30 DAY)
-                        THEN o.price * o.quantity 
-                        ELSE 0 
-                    END), 0)) - 1) * 100,
-                    1
-                ) as growth_rate
-            FROM products p
-            JOIN orders o ON p.pid = o.pid
-            JOIN product_categories pc ON p.pid = pc.pid
-            JOIN categories c ON pc.cat_id = c.cat_id
-            JOIN category_path cp ON c.cat_id = cp.cat_id
-            WHERE o.date >= DATE_SUB(CURDATE(), INTERVAL 60 DAY)
-                AND o.canceled = false
-            GROUP BY c.cat_id, c.name, cp.path
-            ORDER BY units_sold DESC
-            LIMIT 10
-        `);
-
-        res.json({ products, brands, categories });
-    } catch (err) {
-        console.error('Error fetching best sellers:', err);
-        res.status(500).json({ error: 'Failed to fetch best sellers' });
-    }
-});
-
-// GET /dashboard/sales/metrics
-// Returns sales metrics for specified period
-router.get('/sales/metrics', async (req, res) => {
-    const { startDate, endDate } = req.query;
-    try {
-        // Get daily sales data
-        const [dailyRows] = await executeQuery(`
-            SELECT 
-                DATE(o.date) as sale_date,
-                COUNT(DISTINCT o.order_number) as total_orders,
-                SUM(o.quantity) as total_units,
-                SUM(o.price * o.quantity) as total_revenue,
-                SUM(p.cost_price * o.quantity) as total_cogs,
-                SUM((o.price - p.cost_price) * o.quantity) as total_profit
-            FROM orders o
-            JOIN products p ON o.pid = p.pid
-            WHERE o.canceled = false
-                AND o.date BETWEEN ? AND ?
-            GROUP BY DATE(o.date)
-            ORDER BY sale_date
-        `, [startDate, endDate]);
-
-        // Get summary metrics
-        const [metrics] = await executeQuery(`
-            SELECT 
-                COUNT(DISTINCT o.order_number) as total_orders,
-                SUM(o.quantity) as total_units,
-                SUM(o.price * o.quantity) as total_revenue,
-                SUM(p.cost_price * o.quantity) as total_cogs,
-                SUM((o.price - p.cost_price) * o.quantity) as total_profit
-            FROM orders o
-            JOIN products p ON o.pid = p.pid
-            WHERE o.canceled = false
-                AND o.date BETWEEN ? AND ?
-        `, [startDate, endDate]);
-
-        const response = {
-            totalOrders: parseInt(metrics[0]?.total_orders) || 0,
-            totalUnitsSold: parseInt(metrics[0]?.total_units) || 0,
-            totalCogs: parseFloat(metrics[0]?.total_cogs) || 0,
-            totalRevenue: parseFloat(metrics[0]?.total_revenue) || 0,
-            dailySales: dailyRows.map(day => ({
-                date: day.sale_date,
-                units: parseInt(day.total_units) || 0,
-                revenue: parseFloat(day.total_revenue) || 0,
-                cogs: parseFloat(day.total_cogs) || 0
-            }))
-        };
-
-        res.json(response);
-    } catch (err) {
-        console.error('Error fetching sales metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch sales metrics' });
-    }
-});
-
-// GET /dashboard/low-stock/products
-// Returns list of products with critical or low stock levels
-router.get('/low-stock/products', async (req, res) => {
-    const limit = parseInt(req.query.limit) || 50;
-    try {
-        const [rows] = await executeQuery(`
-            SELECT 
-                p.pid,
-                p.SKU,
-                p.title,
-                p.brand,
-                p.vendor,
-                p.stock_quantity,
-                p.cost_price,
-                p.price,
-                pm.daily_sales_avg,
-                pm.days_of_inventory,
-                pm.reorder_qty,
-                (pm.reorder_qty * p.cost_price) as reorder_cost,
-                GROUP_CONCAT(c.name) as categories
-            FROM products p
-            JOIN product_metrics pm ON p.pid = pm.pid
-            LEFT JOIN product_categories pc ON p.pid = pc.pid
-            LEFT JOIN categories c ON pc.cat_id = c.cat_id
-            WHERE pm.stock_status IN ('Critical', 'Reorder')
-                AND p.replenishable = true
-            GROUP BY p.pid
-            ORDER BY 
-                CASE pm.stock_status 
-                    WHEN 'Critical' THEN 1 
-                    WHEN 'Reorder' THEN 2 
-                END,
-                pm.days_of_inventory ASC
-            LIMIT ?
-        `, [limit]);
-        res.json(rows);
-    } catch (err) {
-        console.error('Error fetching low stock products:', err);
-        res.status(500).json({ error: 'Failed to fetch low stock products' });
-    }
-});
-
-// GET /dashboard/trending/products
-// Returns list of trending products based on recent sales velocity
-router.get('/trending/products', async (req, res) => {
-    const days = parseInt(req.query.days) || 30;
-    const limit = parseInt(req.query.limit) || 20;
-    try {
-        const [rows] = await executeQuery(`
-            WITH recent_sales AS (
-                SELECT 
-                    o.pid,
-                    COUNT(DISTINCT o.order_number) as recent_orders,
-                    SUM(o.quantity) as recent_units,
-                    SUM(o.price * o.quantity) as recent_revenue
-                FROM orders o
-                WHERE o.canceled = false
-                    AND o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-                GROUP BY o.pid
-            )
-            SELECT 
-                p.pid,
-                p.SKU,
-                p.title,
-                p.brand,
-                p.vendor,
-                p.stock_quantity,
-                rs.recent_orders,
-                rs.recent_units,
-                rs.recent_revenue,
-                pm.daily_sales_avg,
-                pm.stock_status,
-                (rs.recent_units / ?) as daily_velocity,
-                ((rs.recent_units / ?) - pm.daily_sales_avg) / pm.daily_sales_avg * 100 as velocity_change,
-                GROUP_CONCAT(c.name) as categories
-            FROM recent_sales rs
-            JOIN products p ON rs.pid = p.pid
-            JOIN product_metrics pm ON p.pid = pm.pid
-            LEFT JOIN product_categories pc ON p.pid = pc.pid
-            LEFT JOIN categories c ON pc.cat_id = c.cat_id
-            GROUP BY p.pid
-            HAVING velocity_change > 0
-            ORDER BY velocity_change DESC
-            LIMIT ?
-        `, [days, days, limit]);
-        res.json(rows);
-    } catch (err) {
-        console.error('Error fetching trending products:', err);
-        res.status(500).json({ error: 'Failed to fetch trending products' });
-    }
-});
-
-// GET /dashboard/vendor/performance
-// Returns detailed vendor performance metrics
-router.get('/vendor/performance', async (req, res) => {
-    try {
-        const [rows] = await executeQuery(`
-            WITH vendor_orders AS (
-                SELECT 
-                    po.vendor,
-                    COUNT(DISTINCT po.po_id) as total_orders,
-                    CAST(AVG(DATEDIFF(po.received_date, po.date)) AS DECIMAL(10,2)) as avg_lead_time,
-                    CAST(AVG(CASE 
-                        WHEN po.status = 'completed' 
-                        THEN DATEDIFF(po.received_date, po.expected_date)
-                        END) AS DECIMAL(10,2)) as avg_delay,
-                    CAST(SUM(CASE 
-                        WHEN po.status = 'completed' AND po.received_date <= po.expected_date 
-                        THEN 1 
-                        ELSE 0 
-                        END) * 100.0 / COUNT(*) AS DECIMAL(10,2)) as on_time_delivery_rate,
-                    CAST(AVG(CASE 
-                        WHEN po.status = 'completed' 
-                        THEN po.received / po.ordered * 100 
-                        ELSE NULL 
-                        END) AS DECIMAL(10,2)) as avg_fill_rate
-                FROM purchase_orders po
-                WHERE po.date >= DATE_SUB(CURDATE(), INTERVAL 180 DAY)
-                GROUP BY po.vendor
-            )
-            SELECT 
-                vd.vendor,
-                vd.contact_name,
-                vd.status,
-                CAST(vo.total_orders AS SIGNED) as total_orders,
-                vo.avg_lead_time,
-                vo.avg_delay,
-                vo.on_time_delivery_rate,
-                vo.avg_fill_rate
-            FROM vendor_details vd
-            JOIN vendor_orders vo ON vd.vendor = vo.vendor
-            WHERE vd.status = 'active'
-            ORDER BY vo.on_time_delivery_rate DESC
-        `);
-
-        // Format response with explicit number parsing
-        const formattedRows = rows.map(row => ({
-            vendor: row.vendor,
-            contact_name: row.contact_name,
-            status: row.status,
-            total_orders: parseInt(row.total_orders) || 0,
-            avg_lead_time: parseFloat(row.avg_lead_time) || 0,
-            avg_delay: parseFloat(row.avg_delay) || 0,
-            on_time_delivery_rate: parseFloat(row.on_time_delivery_rate) || 0,
-            avg_fill_rate: parseFloat(row.avg_fill_rate) || 0
-        }));
-
-        res.json(formattedRows);
-    } catch (err) {
-        console.error('Error fetching vendor performance:', err);
-        res.status(500).json({ error: 'Failed to fetch vendor performance' });
-    }
-});
-
-// GET /dashboard/key-metrics
-// Returns key business metrics and KPIs
-router.get('/key-metrics', async (req, res) => {
-    const days = Math.max(1, Math.min(365, parseInt(req.query.days) || 30));
-    try {
-        const [rows] = await executeQuery(`
-            WITH inventory_summary AS (
-                SELECT
-                    COUNT(*) as total_products,
-                    SUM(p.stock_quantity * p.cost_price) as total_inventory_value,
-                    AVG(pm.turnover_rate) as avg_turnover_rate,
-                    COUNT(CASE WHEN pm.stock_status = 'Critical' THEN 1 END) as critical_stock_count,
-                    COUNT(CASE WHEN pm.stock_status = 'Overstocked' THEN 1 END) as overstock_count
-                FROM products p
-                JOIN product_metrics pm ON p.pid = pm.pid
-            ),
-            sales_summary AS (
-                SELECT
-                    COUNT(DISTINCT order_number) as total_orders,
-                    SUM(quantity) as total_units_sold,
-                    SUM(price * quantity) as total_revenue,
-                    AVG(price * quantity) as avg_order_value
-                FROM orders
-                WHERE canceled = false
-                    AND date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-            ),
-            purchase_summary AS (
-                SELECT
-                    COUNT(DISTINCT po_id) as total_pos,
-                    SUM(ordered * cost_price) as total_po_value,
-                    COUNT(CASE WHEN status = 'open' THEN 1 END) as open_pos
-                FROM purchase_orders
-                WHERE order_date >= DATE_SUB(CURDATE(), INTERVAL ? DAY)
-            )
-            SELECT 
-                i.*,
-                s.*,
-                p.*
-            FROM inventory_summary i
-            CROSS JOIN sales_summary s
-            CROSS JOIN purchase_summary p
-        `, [days, days]);
-        res.json(rows[0]);
-    } catch (err) {
-        console.error('Error fetching key metrics:', err);
-        res.status(500).json({ error: 'Failed to fetch key metrics' });
-    }
-});
-
-// GET /dashboard/inventory-health
-// Returns overall inventory health metrics
-router.get('/inventory-health', async (req, res) => {
-    try {
-        const [rows] = await executeQuery(`
-            WITH stock_distribution AS (
-                SELECT
-                    COUNT(*) as total_products,
-                    SUM(CASE WHEN pm.stock_status = 'Healthy' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as healthy_stock_percent,
-                    SUM(CASE WHEN pm.stock_status = 'Critical' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as critical_stock_percent,
-                    SUM(CASE WHEN pm.stock_status = 'Reorder' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as reorder_stock_percent,
-                    SUM(CASE WHEN pm.stock_status = 'Overstocked' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as overstock_percent,
-                    AVG(pm.turnover_rate) as avg_turnover_rate,
-                    AVG(pm.days_of_inventory) as avg_days_inventory
-                FROM products p
-                JOIN product_metrics pm ON p.pid = pm.pid
-                WHERE p.replenishable = true
-            ),
-            value_distribution AS (
-                SELECT
-                    SUM(p.stock_quantity * p.cost_price) as total_inventory_value,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Healthy' 
-                        THEN p.stock_quantity * p.cost_price 
-                        ELSE 0 
-                    END) * 100.0 / SUM(p.stock_quantity * p.cost_price) as healthy_value_percent,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Critical' 
-                        THEN p.stock_quantity * p.cost_price 
-                        ELSE 0 
-                    END) * 100.0 / SUM(p.stock_quantity * p.cost_price) as critical_value_percent,
-                    SUM(CASE 
-                        WHEN pm.stock_status = 'Overstocked' 
-                        THEN p.stock_quantity * p.cost_price 
-                        ELSE 0 
-                    END) * 100.0 / SUM(p.stock_quantity * p.cost_price) as overstock_value_percent
-                FROM products p
-                JOIN product_metrics pm ON p.pid = pm.pid
-            ),
-            category_health AS (
-                SELECT
-                    c.name as category_name,
-                    COUNT(*) as category_products,
-                    SUM(CASE WHEN pm.stock_status = 'Healthy' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as category_healthy_percent,
-                    AVG(pm.turnover_rate) as category_turnover_rate
-                FROM categories c
-                JOIN product_categories pc ON c.cat_id = pc.cat_id
-                JOIN products p ON pc.pid = p.pid
-                JOIN product_metrics pm ON p.pid = pm.pid
-                WHERE p.replenishable = true
-                GROUP BY c.cat_id, c.name
-            )
-            SELECT
-                sd.*,
-                vd.*,
-                JSON_ARRAYAGG(
-                    JSON_OBJECT(
-                        'category', ch.category_name,
-                        'products', ch.category_products,
-                        'healthy_percent', ch.category_healthy_percent,
-                        'turnover_rate', ch.category_turnover_rate
-                    )
-                ) as category_health
-            FROM stock_distribution sd
-            CROSS JOIN value_distribution vd
-            CROSS JOIN category_health ch
-        `);
-        res.json(rows[0]);
-    } catch (err) {
-        console.error('Error fetching inventory health:', err);
-        res.status(500).json({ error: 'Failed to fetch inventory health' });
-    }
-});
-
-// GET /dashboard/replenish/products
-// Returns top products that need replenishment
-router.get('/replenish/products', async (req, res) => {
-    const limit = Math.max(1, Math.min(100, parseInt(req.query.limit) || 50));
-    try {
-        const [products] = await executeQuery(`
-            SELECT 
-                p.pid,
-                p.SKU as sku,
-                p.title,
-                p.stock_quantity,
-                pm.daily_sales_avg,
-                pm.reorder_qty,
-                pm.last_purchase_date
-            FROM products p
-            JOIN product_metrics pm ON p.pid = pm.pid
-            WHERE p.replenishable = true
-                AND pm.stock_status IN ('Critical', 'Reorder')
-                AND pm.reorder_qty > 0
-            ORDER BY 
-                CASE pm.stock_status 
-                    WHEN 'Critical' THEN 1 
-                    WHEN 'Reorder' THEN 2 
-                END,
-                pm.reorder_qty * p.cost_price DESC
-            LIMIT ?
-        `, [limit]);
-
-        res.json(products.map(p => ({
-            ...p,
-            stock_quantity: parseInt(p.stock_quantity) || 0,
-            daily_sales_avg: parseFloat(p.daily_sales_avg) || 0,
-            reorder_qty: parseInt(p.reorder_qty) || 0
-        })));
-    } catch (err) {
-        console.error('Error fetching products to replenish:', err);
-        res.status(500).json({ error: 'Failed to fetch products to replenish' });
-    }
-});
-
-module.exports = router; 

inventory-server/src/routes/import.js

@@ -5,10 +5,14 @@ const mysql = require('mysql2/promise');
 const multer = require('multer');
 const path = require('path');
 const fs = require('fs');
+const fsp = fs.promises;
+const sharp = require('sharp');
 
 // Create uploads directory if it doesn't exist
 const uploadsDir = path.join('/var/www/html/inventory/uploads/products');
+const reusableUploadsDir = path.join('/var/www/html/inventory/uploads/reusable');
 fs.mkdirSync(uploadsDir, { recursive: true });
+fs.mkdirSync(reusableUploadsDir, { recursive: true });
 
 // Create a Map to track image upload times and their scheduled deletion
 const imageUploadMap = new Map();
@@ -33,8 +37,17 @@ const connectionCache = {
   }
 };
 
+const MIN_IMAGE_DIMENSION = 1000;
+const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024;
+
 // Function to schedule image deletion after 24 hours
 const scheduleImageDeletion = (filename, filePath) => {
+  // Only schedule deletion for images in the products folder
+  if (!filePath.includes('/uploads/products/')) {
+    console.log(`Skipping deletion for non-product image: ${filename}`);
+    return;
+  }
+
   // Delete any existing timeout for this file
   if (imageUploadMap.has(filename)) {
     clearTimeout(imageUploadMap.get(filename).timeoutId);
@@ -137,6 +150,255 @@ const cleanupImagesOnStartup = () => {
 // Run cleanup on server start
 cleanupImagesOnStartup();
 
+const bytesToMegabytes = (bytes) => Number((bytes / (1024 * 1024)).toFixed(2));
+
+const processUploadedImage = async (filePath, mimetype) => {
+  const notices = [];
+  const legacyWarnings = [];
+  const metadata = {};
+
+  const originalBuffer = await fsp.readFile(filePath);
+  let baseMetadata = await sharp(originalBuffer, { failOn: 'none' }).metadata();
+
+  metadata.width = baseMetadata.width || 0;
+  metadata.height = baseMetadata.height || 0;
+  metadata.size = originalBuffer.length;
+  metadata.colorSpace = baseMetadata.space || baseMetadata.colourspace || null;
+
+  if (
+    baseMetadata.width &&
+    baseMetadata.height &&
+    (baseMetadata.width < MIN_IMAGE_DIMENSION || baseMetadata.height < MIN_IMAGE_DIMENSION)
+  ) {
+    const message = `Image is ${baseMetadata.width}x${baseMetadata.height}. Recommended minimum is ${MIN_IMAGE_DIMENSION}x${MIN_IMAGE_DIMENSION}.`;
+    notices.push({
+      message,
+      level: 'warning',
+      code: 'dimensions_too_small',
+      source: 'server'
+    });
+    legacyWarnings.push(message);
+  }
+
+  const colorSpace = (baseMetadata.space || baseMetadata.colourspace || '').toLowerCase();
+  let shouldConvertToRgb = colorSpace === 'cmyk';
+
+  if (shouldConvertToRgb) {
+    const message = 'Converted image from CMYK to RGB.';
+    notices.push({
+      message,
+      level: 'info',
+      code: 'converted_to_rgb',
+      source: 'server'
+    });
+    legacyWarnings.push(message);
+  }
+
+  const format = (baseMetadata.format || '').toLowerCase();
+  if (format === 'gif') {
+    if (metadata.size > MAX_IMAGE_SIZE_BYTES) {
+      const message = `GIF optimization is limited; resulting size is ${bytesToMegabytes(metadata.size)}MB (target 5MB).`;
+      notices.push({
+        message,
+        level: 'warning',
+        code: 'gif_size_limit',
+        source: 'server'
+      });
+      legacyWarnings.push(message);
+    }
+    metadata.convertedToRgb = false;
+    metadata.resized = false;
+    return { notices, warnings: legacyWarnings, metadata, finalSize: metadata.size };
+  }
+
+  const supportsQuality = ['jpeg', 'jpg', 'webp'].includes(format);
+  let targetQuality = supportsQuality ? 90 : undefined;
+  let finalQuality = undefined;
+
+  let currentWidth = baseMetadata.width || null;
+  let currentHeight = baseMetadata.height || null;
+
+  let resized = false;
+  let mutated = false;
+  let finalBuffer = originalBuffer;
+  let finalInfo = baseMetadata;
+
+  const encode = async ({ width, height, quality }) => {
+    let pipeline = sharp(originalBuffer, { failOn: 'none' });
+
+    if (shouldConvertToRgb) {
+      pipeline = pipeline.toColorspace('srgb');
+    }
+
+    if (width || height) {
+      pipeline = pipeline.resize({
+        width: width ?? undefined,
+        height: height ?? undefined,
+        fit: 'inside',
+        withoutEnlargement: true,
+      });
+    }
+
+    switch (format) {
+      case 'png':
+        pipeline = pipeline.png({
+          compressionLevel: 9,
+          adaptiveFiltering: true,
+          palette: true,
+        });
+        break;
+      case 'webp':
+        pipeline = pipeline.webp({ quality: quality ?? 90 });
+        break;
+      case 'jpeg':
+      case 'jpg':
+      default:
+        pipeline = pipeline.jpeg({ quality: quality ?? 90, mozjpeg: true });
+        break;
+    }
+
+    return pipeline.toBuffer({ resolveWithObject: true });
+  };
+
+  const canResize =
+    (currentWidth && currentWidth > MIN_IMAGE_DIMENSION) ||
+    (currentHeight && currentHeight > MIN_IMAGE_DIMENSION);
+
+  if (metadata.size > MAX_IMAGE_SIZE_BYTES && (supportsQuality || canResize)) {
+    const maxAttempts = 8;
+
+    for (let attempt = 0; attempt < maxAttempts; attempt++) {
+      let targetWidth = currentWidth;
+      let targetHeight = currentHeight;
+      let resizedThisAttempt = false;
+
+      if (currentWidth && currentWidth > MIN_IMAGE_DIMENSION) {
+        targetWidth = Math.max(MIN_IMAGE_DIMENSION, Math.round(currentWidth * 0.85));
+      }
+
+      if (currentHeight && currentHeight > MIN_IMAGE_DIMENSION) {
+        targetHeight = Math.max(MIN_IMAGE_DIMENSION, Math.round(currentHeight * 0.85));
+      }
+
+      if (
+        (targetWidth && currentWidth && targetWidth < currentWidth) ||
+        (targetHeight && currentHeight && targetHeight < currentHeight)
+      ) {
+        resized = true;
+        resizedThisAttempt = true;
+        currentWidth = targetWidth;
+        currentHeight = targetHeight;
+      } else if (!supportsQuality || (targetQuality && targetQuality <= 70)) {
+        // Cannot resize further and quality cannot be adjusted
+        break;
+      }
+
+      const qualityForAttempt = supportsQuality ? targetQuality : undefined;
+      const { data, info } = await encode({
+        width: currentWidth,
+        height: currentHeight,
+        quality: qualityForAttempt,
+      });
+
+      mutated = true;
+      finalBuffer = data;
+      finalInfo = info;
+      metadata.optimizedSize = data.length;
+      if (info.width) metadata.width = info.width;
+      if (info.height) metadata.height = info.height;
+      if (info.width) currentWidth = info.width;
+      if (info.height) currentHeight = info.height;
+
+      if (supportsQuality && qualityForAttempt) {
+        finalQuality = qualityForAttempt;
+      }
+
+      if (data.length <= MAX_IMAGE_SIZE_BYTES) {
+        break;
+      }
+
+      if (resizedThisAttempt) {
+        continue;
+      }
+
+      if (supportsQuality && targetQuality && targetQuality > 70) {
+        const nextQuality = Math.max(70, targetQuality - 10);
+        if (nextQuality === targetQuality) {
+          break;
+        }
+        targetQuality = nextQuality;
+        continue;
+      }
+
+      break;
+    }
+
+    if (finalBuffer.length > MAX_IMAGE_SIZE_BYTES) {
+      const message = `Optimized image remains ${bytesToMegabytes(finalBuffer.length)}MB (target 5MB).`;
+      notices.push({
+        message,
+        level: 'warning',
+        code: 'size_over_limit',
+        source: 'server'
+      });
+      legacyWarnings.push(message);
+    }
+  } else if (shouldConvertToRgb) {
+    const { data, info } = await encode({ width: currentWidth, height: currentHeight });
+    mutated = true;
+    finalBuffer = data;
+    finalInfo = info;
+    metadata.optimizedSize = data.length;
+    if (info.width) metadata.width = info.width;
+    if (info.height) metadata.height = info.height;
+    if (info.width) currentWidth = info.width;
+    if (info.height) currentHeight = info.height;
+  }
+
+  if (mutated) {
+    await fsp.writeFile(filePath, finalBuffer);
+    metadata.optimizedSize = finalBuffer.length;
+  } else {
+    // No transformation occurred; still need to ensure we report original stats
+    metadata.optimizedSize = metadata.size;
+  }
+
+  metadata.convertedToRgb = shouldConvertToRgb && mutated;
+  metadata.resized = resized;
+  if (finalQuality) {
+    metadata.quality = finalQuality;
+  }
+
+  if (resized && metadata.width && metadata.height) {
+    const message = `Image resized to ${metadata.width}x${metadata.height} during optimization.`;
+    notices.push({
+      message,
+      level: 'info',
+      code: 'resized',
+      source: 'server'
+    });
+    legacyWarnings.push(message);
+  }
+
+  if (finalQuality && finalQuality < 90) {
+    const message = `Image quality adjusted to ${finalQuality} to reduce file size.`;
+    notices.push({
+      message,
+      level: 'info',
+      code: 'quality_adjusted',
+      source: 'server'
+    });
+    legacyWarnings.push(message);
+  }
+
+  return {
+    notices,
+    warnings: legacyWarnings,
+    metadata,
+    finalSize: finalBuffer.length,
+  };
+};
+
 // Configure multer for file uploads
 const storage = multer.diskStorage({
   destination: function (req, file, cb) {
@@ -170,7 +432,7 @@ const storage = multer.diskStorage({
 const upload = multer({ 
   storage: storage,
   limits: {
-    fileSize: 5 * 1024 * 1024, // 5MB max file size
+    fileSize: 15 * 1024 * 1024, // Allow bigger uploads; processing will reduce to 5MB
   },
   fileFilter: function (req, file, cb) {
     // Accept only image files
@@ -337,7 +599,7 @@ async function setupSshTunnel() {
 }
 
 // Image upload endpoint
-router.post('/upload-image', upload.single('image'), (req, res) => {
+router.post('/upload-image', upload.single('image'), async (req, res) => {
   try {
     if (!req.file) {
       return res.status(400).json({ error: 'No image file provided' });
@@ -367,9 +629,13 @@ router.post('/upload-image', upload.single('image'), (req, res) => {
       }
     });
 
+    // Process the image (resize/compress/color-space) before responding
+    const processingResult = await processUploadedImage(filePath, req.file.mimetype);
+    req.file.size = processingResult.finalSize;
+
     // Create URL for the uploaded file - using an absolute URL with domain
-    // This will generate a URL like: https://inventory.acot.site/uploads/products/filename.jpg
-    const baseUrl = 'https://inventory.acot.site';
+    // This will generate a URL like: https://tools.acherryontop.com/uploads/products/filename.jpg
+    const baseUrl = 'https://tools.acherryontop.com';
     const imageUrl = `${baseUrl}/uploads/products/${req.file.filename}`;
     
     // Schedule this image for deletion in 24 hours
@@ -382,11 +648,24 @@ router.post('/upload-image', upload.single('image'), (req, res) => {
       fileName: req.file.filename,
       mimetype: req.file.mimetype,
       fullPath: filePath,
+      notices: processingResult.notices,
+      warnings: processingResult.warnings,
+      metadata: processingResult.metadata,
       message: 'Image uploaded successfully (will auto-delete after 24 hours)'
     });
     
   } catch (error) {
     console.error('Error uploading image:', error);
+    if (req?.file?.filename) {
+      const cleanupPath = path.join(uploadsDir, req.file.filename);
+      if (fs.existsSync(cleanupPath)) {
+        try {
+          fs.unlinkSync(cleanupPath);
+        } catch (cleanupError) {
+          console.error('Failed to remove file after processing error:', cleanupError);
+        }
+      }
+    }
     res.status(500).json({ error: error.message || 'Failed to upload image' });
   }
 });
@@ -407,6 +686,14 @@ router.delete('/delete-image', (req, res) => {
       return res.status(404).json({ error: 'File not found' });
     }
     
+    // Only allow deletion of images in the products folder
+    if (!filePath.includes('/uploads/products/')) {
+      return res.status(403).json({ 
+        error: 'Cannot delete images outside the products folder',
+        message: 'This image is in a protected folder and cannot be deleted through this endpoint'
+      });
+    }
+    
     // Delete the file
     fs.unlinkSync(filePath);
     
@@ -428,6 +715,26 @@ router.delete('/delete-image', (req, res) => {
   }
 });
 
+// Clear all taxonomy caches
+router.post('/clear-taxonomy-cache', (req, res) => {
+  try {
+    // Clear all entries from the query cache
+    const cacheSize = connectionCache.queryCache.size;
+    connectionCache.queryCache.clear();
+
+    console.log(`Cleared ${cacheSize} entries from taxonomy cache`);
+
+    res.json({
+      success: true,
+      message: `Cache cleared (${cacheSize} entries removed)`,
+      clearedEntries: cacheSize
+    });
+  } catch (error) {
+    console.error('Error clearing taxonomy cache:', error);
+    res.status(500).json({ error: 'Failed to clear cache' });
+  }
+});
+
 // Get all options for import fields
 router.get('/field-options', async (req, res) => {
     try {
@@ -641,11 +948,19 @@ router.get('/check-file/:filename', (req, res) => {
     return res.status(400).json({ error: 'Invalid filename' });
   }
   
-  const filePath = path.join(uploadsDir, filename);
+  // First check in products directory
+  let filePath = path.join(uploadsDir, filename);
+  let exists = fs.existsSync(filePath);
+  
+  // If not found in products, check in reusable directory
+  if (!exists) {
+    filePath = path.join(reusableUploadsDir, filename);
+    exists = fs.existsSync(filePath);
+  }
   
   try {
     // Check if file exists
-    if (!fs.existsSync(filePath)) {
+    if (!exists) {
       return res.status(404).json({ 
         error: 'File not found', 
         path: filePath,
@@ -685,13 +1000,23 @@ router.get('/check-file/:filename', (req, res) => {
 // List all files in uploads directory
 router.get('/list-uploads', (req, res) => {
   try {
-    if (!fs.existsSync(uploadsDir)) {
-      return res.status(404).json({ error: 'Uploads directory not found', path: uploadsDir });
+    const { directory = 'products' } = req.query;
+    
+    // Determine which directory to list
+    let targetDir;
+    if (directory === 'reusable') {
+      targetDir = reusableUploadsDir;
+    } else {
+      targetDir = uploadsDir; // default to products
     }
     
-    const files = fs.readdirSync(uploadsDir);
+    if (!fs.existsSync(targetDir)) {
+      return res.status(404).json({ error: 'Uploads directory not found', path: targetDir });
+    }
+    
+    const files = fs.readdirSync(targetDir);
     const fileDetails = files.map(file => {
-      const filePath = path.join(uploadsDir, file);
+      const filePath = path.join(targetDir, file);
       try {
         const stats = fs.statSync(filePath);
         return {
@@ -709,12 +1034,13 @@ router.get('/list-uploads', (req, res) => {
     });
     
     return res.json({
-      directory: uploadsDir,
+      directory: targetDir,
+      type: directory,
       count: files.length,
       files: fileDetails
     });
   } catch (error) {
-    return res.status(500).json({ error: error.message, path: uploadsDir });
+    return res.status(500).json({ error: error.message });
   }
 });
 
@@ -920,6 +1246,155 @@ router.get('/search-products', async (req, res) => {
   }
 });
 
+const UPC_SUPPLIER_PREFIX_LEADING_DIGIT = '4';
+const UPC_MAX_SEQUENCE = 99999;
+const UPC_RESERVATION_TTL = 5 * 60 * 1000; // 5 minutes
+
+function buildSupplierPrefix(supplierId) {
+  const numericId = Number.parseInt(String(supplierId), 10);
+  if (Number.isNaN(numericId) || numericId < 0) {
+    return null;
+  }
+
+  const padded = String(numericId).padStart(5, '0');
+  const prefix = `${UPC_SUPPLIER_PREFIX_LEADING_DIGIT}${padded}`;
+  return prefix.length === 6 ? prefix : null;
+}
+
+function calculateUpcCheckDigit(upcWithoutCheckDigit) {
+  if (!/^\d{11}$/.test(upcWithoutCheckDigit)) {
+    throw new Error('UPC body must be 11 numeric characters');
+  }
+
+  let sum = 0;
+  for (let i = 0; i < upcWithoutCheckDigit.length; i += 1) {
+    const digit = Number.parseInt(upcWithoutCheckDigit[i], 10);
+    sum += (i % 2 === 0) ? digit * 3 : digit;
+  }
+
+  const mod = sum % 10;
+  return mod === 0 ? 0 : 10 - mod;
+}
+
+const upcReservationCache = new Map();
+const upcGenerationLocks = new Map();
+
+function getReservedSequence(prefix) {
+  const entry = upcReservationCache.get(prefix);
+  if (!entry) {
+    return 0;
+  }
+
+  if (Date.now() > entry.expiresAt) {
+    upcReservationCache.delete(prefix);
+    return 0;
+  }
+
+  return entry.lastSequence;
+}
+
+function setReservedSequence(prefix, sequence) {
+  upcReservationCache.set(prefix, {
+    lastSequence: sequence,
+    expiresAt: Date.now() + UPC_RESERVATION_TTL
+  });
+}
+
+async function runWithSupplierLock(prefix, task) {
+  const previous = upcGenerationLocks.get(prefix) || Promise.resolve();
+  const chained = previous.catch(() => {}).then(() => task());
+  upcGenerationLocks.set(prefix, chained);
+
+  try {
+    return await chained;
+  } finally {
+    if (upcGenerationLocks.get(prefix) === chained) {
+      upcGenerationLocks.delete(prefix);
+    }
+  }
+}
+
+router.post('/generate-upc', async (req, res) => {
+  const { supplierId, increment } = req.body || {};
+
+  if (supplierId === undefined || supplierId === null || String(supplierId).trim() === '') {
+    return res.status(400).json({ error: 'Supplier ID is required to generate a UPC' });
+  }
+
+  const supplierPrefix = buildSupplierPrefix(supplierId);
+  if (!supplierPrefix) {
+    return res.status(400).json({ error: 'Supplier ID must be a non-negative number with at most 5 digits' });
+  }
+
+  const step = Number.parseInt(increment, 10);
+  const sequenceIncrement = Number.isNaN(step) || step < 1 ? 1 : step;
+
+  try {
+    const result = await runWithSupplierLock(supplierPrefix, async () => {
+      const { connection } = await getDbConnection();
+
+      const [rows] = await connection.query(
+        `SELECT CAST(SUBSTRING(upc,7,5) AS UNSIGNED) AS num
+           FROM products
+          WHERE LEFT(upc, 6) = ? AND LENGTH(upc) = 12
+       ORDER BY num DESC
+          LIMIT 1`,
+        [supplierPrefix]
+      );
+
+      const lastSequenceFromDb = rows && rows.length > 0 && rows[0].num !== null
+        ? Number.parseInt(rows[0].num, 10) || 0
+        : 0;
+
+      const cachedSequence = getReservedSequence(supplierPrefix);
+      const baselineSequence = Math.max(lastSequenceFromDb, cachedSequence);
+
+      let nextSequence = baselineSequence + sequenceIncrement;
+      let candidateUpc = null;
+      let attempts = 0;
+
+      while (attempts < 10 && nextSequence <= UPC_MAX_SEQUENCE) {
+        const sequencePart = String(nextSequence).padStart(5, '0');
+        const upcBody = `${supplierPrefix}${sequencePart}`;
+        const checkDigit = calculateUpcCheckDigit(upcBody);
+        const fullUpc = `${upcBody}${checkDigit}`;
+
+        const [existing] = await connection.query(
+          'SELECT 1 FROM products WHERE upc = ? LIMIT 1',
+          [fullUpc]
+        );
+
+        if (!existing || existing.length === 0) {
+          candidateUpc = { upc: fullUpc, sequence: nextSequence };
+          break;
+        }
+
+        nextSequence += 1;
+        attempts += 1;
+      }
+
+      if (!candidateUpc) {
+        const reason = nextSequence > UPC_MAX_SEQUENCE
+          ? 'UPC range exhausted for this supplier'
+          : 'Unable to find an available UPC';
+        const error = new Error(reason);
+        error.status = 409;
+        throw error;
+      }
+
+      setReservedSequence(supplierPrefix, candidateUpc.sequence);
+      return candidateUpc.upc;
+    });
+
+    return res.json({ success: true, upc: result });
+  } catch (error) {
+    console.error('Error generating UPC:', error);
+    const status = error.status && Number.isInteger(error.status) ? error.status : 500;
+    const message = status === 500 ? 'Failed to generate UPC' : error.message;
+    return res.status(status).json({ error: message, details: status === 500 ? error.message : undefined });
+  }
+});
+
 // Endpoint to check UPC and generate item number
 router.get('/check-upc-and-generate-sku', async (req, res) => {
   const { upc, supplierId } = req.query;
@@ -939,7 +1414,7 @@ router.get('/check-upc-and-generate-sku', async (req, res) => {
     
     if (upcCheck.length > 0) {
       return res.status(409).json({ 
-        error: 'UPC already exists', 
+        error: 'A product with this UPC already exists', 
         existingProductId: upcCheck[0].pid,
         existingItemNumber: upcCheck[0].itemnumber
       });

inventory-server/src/routes/metrics.js

@@ -1,62 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Get key metrics trends (revenue, inventory value, GMROI)
-router.get('/trends', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { rows } = await pool.query(`
-      WITH MonthlyMetrics AS (
-        SELECT 
-          make_date(pta.year, pta.month, 1) as date,
-          ROUND(COALESCE(SUM(pta.total_revenue), 0)::numeric, 3) as revenue,
-          ROUND(COALESCE(SUM(pta.total_cost), 0)::numeric, 3) as cost,
-          ROUND(COALESCE(SUM(pm.inventory_value), 0)::numeric, 3) as inventory_value,
-          CASE 
-            WHEN SUM(pm.inventory_value) > 0 
-            THEN ROUND((SUM(pta.total_revenue - pta.total_cost) / SUM(pm.inventory_value) * 100)::numeric, 3)
-            ELSE 0 
-          END as gmroi
-        FROM product_time_aggregates pta
-        JOIN product_metrics pm ON pta.pid = pm.pid
-        WHERE (pta.year * 100 + pta.month) >= 
-          EXTRACT(YEAR FROM CURRENT_DATE - INTERVAL '12 months')::integer * 100 + 
-          EXTRACT(MONTH FROM CURRENT_DATE - INTERVAL '12 months')::integer
-        GROUP BY pta.year, pta.month
-        ORDER BY date ASC
-      )
-      SELECT 
-        to_char(date, 'Mon YY') as date,
-        revenue,
-        inventory_value,
-        gmroi
-      FROM MonthlyMetrics
-    `);
-
-    console.log('Raw metrics trends data:', rows);
-
-    // Transform the data into the format expected by the frontend
-    const transformedData = {
-      revenue: rows.map(row => ({
-        date: row.date,
-        value: parseFloat(row.revenue)
-      })),
-      inventory_value: rows.map(row => ({
-        date: row.date,
-        value: parseFloat(row.inventory_value)
-      })),
-      gmroi: rows.map(row => ({
-        date: row.date,
-        value: parseFloat(row.gmroi)
-      }))
-    };
-
-    console.log('Transformed metrics data:', transformedData);
-    res.json(transformedData);
-  } catch (error) {
-    console.error('Error fetching metrics trends:', error);
-    res.status(500).json({ error: 'Failed to fetch metrics trends' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/orders.js

@@ -1,261 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Get all orders with pagination, filtering, and sorting
-router.get('/', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const page = parseInt(req.query.page) || 1;
-    const limit = parseInt(req.query.limit) || 50;
-    const offset = (page - 1) * limit;
-    const search = req.query.search || '';
-    const status = req.query.status || 'all';
-    const fromDate = req.query.fromDate ? new Date(req.query.fromDate) : null;
-    const toDate = req.query.toDate ? new Date(req.query.toDate) : null;
-    const minAmount = parseFloat(req.query.minAmount) || 0;
-    const maxAmount = req.query.maxAmount ? parseFloat(req.query.maxAmount) : null;
-    const sortColumn = req.query.sortColumn || 'date';
-    const sortDirection = req.query.sortDirection === 'desc' ? 'DESC' : 'ASC';
-
-    // Build the WHERE clause
-    const conditions = ['o1.canceled = false'];
-    const params = [];
-    let paramCounter = 1;
-
-    if (search) {
-      conditions.push(`(o1.order_number ILIKE $${paramCounter} OR o1.customer ILIKE $${paramCounter})`);
-      params.push(`%${search}%`);
-      paramCounter++;
-    }
-
-    if (status !== 'all') {
-      conditions.push(`o1.status = $${paramCounter}`);
-      params.push(status);
-      paramCounter++;
-    }
-
-    if (fromDate) {
-      conditions.push(`DATE(o1.date) >= DATE($${paramCounter})`);
-      params.push(fromDate.toISOString());
-      paramCounter++;
-    }
-
-    if (toDate) {
-      conditions.push(`DATE(o1.date) <= DATE($${paramCounter})`);
-      params.push(toDate.toISOString());
-      paramCounter++;
-    }
-
-    if (minAmount > 0) {
-      conditions.push(`total_amount >= $${paramCounter}`);
-      params.push(minAmount);
-      paramCounter++;
-    }
-
-    if (maxAmount) {
-      conditions.push(`total_amount <= $${paramCounter}`);
-      params.push(maxAmount);
-      paramCounter++;
-    }
-
-    // Get total count for pagination
-    const { rows: [countResult] } = await pool.query(`
-      SELECT COUNT(DISTINCT o1.order_number) as total 
-      FROM orders o1
-      LEFT JOIN (
-        SELECT order_number, SUM(price * quantity) as total_amount
-        FROM orders
-        GROUP BY order_number
-      ) totals ON o1.order_number = totals.order_number
-      WHERE ${conditions.join(' AND ')}
-    `, params);
-    
-    const total = countResult.total;
-
-    // Get paginated results
-    const query = `
-      SELECT 
-        o1.order_number,
-        o1.customer,
-        o1.date,
-        o1.status,
-        o1.payment_method,
-        o1.shipping_method,
-        COUNT(o2.pid) as items_count,
-        ROUND(SUM(o2.price * o2.quantity)::numeric, 3) as total_amount
-      FROM orders o1
-      JOIN orders o2 ON o1.order_number = o2.order_number
-      WHERE ${conditions.join(' AND ')}
-      GROUP BY 
-        o1.order_number,
-        o1.customer,
-        o1.date,
-        o1.status,
-        o1.payment_method,
-        o1.shipping_method
-      ORDER BY ${
-        sortColumn === 'items_count' || sortColumn === 'total_amount' 
-          ? `${sortColumn} ${sortDirection}`
-          : `o1.${sortColumn} ${sortDirection}`
-      }
-      LIMIT $${paramCounter} OFFSET $${paramCounter + 1}
-    `;
-
-    params.push(limit, offset);
-    const { rows } = await pool.query(query, params);
-
-    // Get order statistics
-    const { rows: [orderStats] } = await pool.query(`
-      WITH CurrentStats AS (
-        SELECT 
-          COUNT(DISTINCT order_number) as total_orders,
-          ROUND(SUM(price * quantity)::numeric, 3) as total_revenue
-        FROM orders
-        WHERE canceled = false
-          AND DATE(date) >= CURRENT_DATE - INTERVAL '30 days'
-      ),
-      PreviousStats AS (
-        SELECT 
-          COUNT(DISTINCT order_number) as prev_orders,
-          ROUND(SUM(price * quantity)::numeric, 3) as prev_revenue
-        FROM orders
-        WHERE canceled = false
-          AND DATE(date) BETWEEN CURRENT_DATE - INTERVAL '60 days' AND CURRENT_DATE - INTERVAL '30 days'
-      ),
-      OrderValues AS (
-        SELECT 
-          order_number,
-          ROUND(SUM(price * quantity)::numeric, 3) as order_value
-        FROM orders
-        WHERE canceled = false
-          AND DATE(date) >= CURRENT_DATE - INTERVAL '30 days'
-        GROUP BY order_number
-      )
-      SELECT 
-        cs.total_orders,
-        cs.total_revenue,
-        CASE 
-          WHEN ps.prev_orders > 0 
-          THEN ROUND(((cs.total_orders - ps.prev_orders)::numeric / ps.prev_orders * 100), 1)
-          ELSE 0 
-        END as order_growth,
-        CASE 
-          WHEN ps.prev_revenue > 0 
-          THEN ROUND(((cs.total_revenue - ps.prev_revenue)::numeric / ps.prev_revenue * 100), 1)
-          ELSE 0 
-        END as revenue_growth,
-        CASE 
-          WHEN cs.total_orders > 0 
-          THEN ROUND((cs.total_revenue::numeric / cs.total_orders), 3)
-          ELSE 0 
-        END as average_order_value,
-        CASE 
-          WHEN ps.prev_orders > 0 
-          THEN ROUND((ps.prev_revenue::numeric / ps.prev_orders), 3)
-          ELSE 0 
-        END as prev_average_order_value
-      FROM CurrentStats cs
-      CROSS JOIN PreviousStats ps
-    `);
-    
-    res.json({
-      orders: rows.map(row => ({
-        ...row,
-        total_amount: parseFloat(row.total_amount) || 0,
-        items_count: parseInt(row.items_count) || 0,
-        date: row.date
-      })),
-      pagination: {
-        total,
-        pages: Math.ceil(total / limit),
-        currentPage: page,
-        limit
-      },
-      stats: {
-        totalOrders: parseInt(orderStats.total_orders) || 0,
-        totalRevenue: parseFloat(orderStats.total_revenue) || 0,
-        orderGrowth: parseFloat(orderStats.order_growth) || 0,
-        revenueGrowth: parseFloat(orderStats.revenue_growth) || 0,
-        averageOrderValue: parseFloat(orderStats.average_order_value) || 0,
-        aovGrowth: orderStats.prev_average_order_value > 0 
-          ? ((orderStats.average_order_value - orderStats.prev_average_order_value) / orderStats.prev_average_order_value * 100)
-          : 0,
-        conversionRate: 2.5, // Placeholder - would need actual visitor data
-        conversionGrowth: 0.5 // Placeholder - would need actual visitor data
-      }
-    });
-  } catch (error) {
-    console.error('Error fetching orders:', error);
-    res.status(500).json({ error: 'Failed to fetch orders' });
-  }
-});
-
-// Get a single order with its items
-router.get('/:orderNumber', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    // Get order details
-    const { rows: orderRows } = await pool.query(`
-      SELECT DISTINCT
-        o1.order_number,
-        o1.customer,
-        o1.date,
-        o1.status,
-        o1.payment_method,
-        o1.shipping_method,
-        o1.shipping_address,
-        o1.billing_address,
-        COUNT(o2.pid) as items_count,
-        ROUND(SUM(o2.price * o2.quantity)::numeric, 3) as total_amount
-      FROM orders o1
-      JOIN orders o2 ON o1.order_number = o2.order_number
-      WHERE o1.order_number = $1 AND o1.canceled = false
-      GROUP BY 
-        o1.order_number,
-        o1.customer,
-        o1.date,
-        o1.status,
-        o1.payment_method,
-        o1.shipping_method,
-        o1.shipping_address,
-        o1.billing_address
-    `, [req.params.orderNumber]);
-
-    if (orderRows.length === 0) {
-      return res.status(404).json({ error: 'Order not found' });
-    }
-
-    // Get order items
-    const { rows: itemRows } = await pool.query(`
-      SELECT 
-        o.pid,
-        p.title,
-        p.SKU,
-        o.quantity,
-        o.price,
-        ROUND((o.price * o.quantity)::numeric, 3) as total
-      FROM orders o
-      JOIN products p ON o.pid = p.pid
-      WHERE o.order_number = $1 AND o.canceled = false
-    `, [req.params.orderNumber]);
-
-    const order = {
-      ...orderRows[0],
-      total_amount: parseFloat(orderRows[0].total_amount) || 0,
-      items_count: parseInt(orderRows[0].items_count) || 0,
-      items: itemRows.map(item => ({
-        ...item,
-        price: parseFloat(item.price) || 0,
-        total: parseFloat(item.total) || 0,
-        quantity: parseInt(item.quantity) || 0
-      }))
-    };
-
-    res.json(order);
-  } catch (error) {
-    console.error('Error fetching order:', error);
-    res.status(500).json({ error: 'Failed to fetch order' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/products.js

@@ -1,767 +0,0 @@
-const express = require('express');
-const router = express.Router();
-const multer = require('multer');
-const { importProductsFromCSV } = require('../utils/csvImporter');
-const { PurchaseOrderStatus, ReceivingStatus } = require('../types/status-codes');
-
-// Configure multer for file uploads
-const upload = multer({ dest: 'uploads/' });
-
-// Get unique brands
-router.get('/brands', async (req, res) => {
-    console.log('Brands endpoint hit:', {
-        url: req.url,
-        method: req.method,
-        headers: req.headers,
-        path: req.path
-    });
-    
-    try {
-        const pool = req.app.locals.pool;
-        console.log('Fetching brands from database...');
-        
-        const { rows } = await pool.query(`
-            SELECT DISTINCT COALESCE(p.brand, 'Unbranded') as brand
-            FROM products p
-            JOIN purchase_orders po ON p.pid = po.pid
-            WHERE p.visible = true
-            GROUP BY COALESCE(p.brand, 'Unbranded')
-            HAVING SUM(po.cost_price * po.received) >= 500
-            ORDER BY COALESCE(p.brand, 'Unbranded')
-        `);
-        
-        console.log(`Found ${rows.length} brands:`, rows.slice(0, 3));
-        res.json(rows.map(r => r.brand));
-    } catch (error) {
-        console.error('Error fetching brands:', error);
-        res.status(500).json({ error: 'Failed to fetch brands' });
-    }
-});
-
-// Get all products with pagination, filtering, and sorting
-router.get('/', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const page = parseInt(req.query.page) || 1;
-    const limit = parseInt(req.query.limit) || 50;
-    const offset = (page - 1) * limit;
-    const sortColumn = req.query.sort || 'title';
-    const sortDirection = req.query.order === 'desc' ? 'DESC' : 'ASC';
-
-    const conditions = ['p.visible = true'];
-    const params = [];
-    let paramCounter = 1;
-
-    // Add default replenishable filter unless explicitly showing non-replenishable
-    if (req.query.showNonReplenishable !== 'true') {
-      conditions.push('p.replenishable = true');
-    }
-
-    // Handle search filter
-    if (req.query.search) {
-      conditions.push(`(p.title ILIKE $${paramCounter} OR p.SKU ILIKE $${paramCounter} OR p.barcode ILIKE $${paramCounter})`);
-      const searchTerm = `%${req.query.search}%`;
-      params.push(searchTerm);
-      paramCounter++;
-    }
-
-    // Handle numeric filters with operators
-    const numericFields = {
-      stock: 'p.stock_quantity',
-      price: 'p.price',
-      costPrice: 'p.cost_price',
-      landingCost: 'p.landing_cost_price',
-      dailySalesAvg: 'pm.daily_sales_avg',
-      weeklySalesAvg: 'pm.weekly_sales_avg',
-      monthlySalesAvg: 'pm.monthly_sales_avg',
-      margin: 'pm.avg_margin_percent',
-      gmroi: 'pm.gmroi',
-      leadTime: 'pm.current_lead_time',
-      stockCoverage: 'pm.days_of_inventory',
-      daysOfStock: 'pm.days_of_inventory'
-    };
-
-    Object.entries(req.query).forEach(([key, value]) => {
-      const field = numericFields[key];
-      if (field) {
-        const operator = req.query[`${key}_operator`] || '=';
-        if (operator === 'between') {
-          try {
-            const [min, max] = JSON.parse(value);
-            conditions.push(`${field} BETWEEN $${paramCounter} AND $${paramCounter + 1}`);
-            params.push(min, max);
-            paramCounter += 2;
-          } catch (e) {
-            console.error(`Invalid between value for ${key}:`, value);
-          }
-        } else {
-          conditions.push(`${field} ${operator} $${paramCounter}`);
-          params.push(parseFloat(value));
-          paramCounter++;
-        }
-      }
-    });
-
-    // Handle select filters
-    if (req.query.vendor) {
-      conditions.push(`p.vendor = $${paramCounter}`);
-      params.push(req.query.vendor);
-      paramCounter++;
-    }
-
-    if (req.query.brand) {
-      conditions.push(`p.brand = $${paramCounter}`);
-      params.push(req.query.brand);
-      paramCounter++;
-    }
-
-    if (req.query.category) {
-      conditions.push(`p.categories ILIKE $${paramCounter}`);
-      params.push(`%${req.query.category}%`);
-      paramCounter++;
-    }
-
-    if (req.query.stockStatus && req.query.stockStatus !== 'all') {
-      conditions.push(`pm.stock_status = $${paramCounter}`);
-      params.push(req.query.stockStatus);
-      paramCounter++;
-    }
-
-    if (req.query.abcClass) {
-      conditions.push(`pm.abc_class = $${paramCounter}`);
-      params.push(req.query.abcClass);
-      paramCounter++;
-    }
-
-    if (req.query.leadTimeStatus) {
-      conditions.push(`pm.lead_time_status = $${paramCounter}`);
-      params.push(req.query.leadTimeStatus);
-      paramCounter++;
-    }
-
-    if (req.query.replenishable !== undefined) {
-      conditions.push(`p.replenishable = $${paramCounter}`);
-      params.push(req.query.replenishable === 'true');
-      paramCounter++;
-    }
-
-    if (req.query.managingStock !== undefined) {
-      conditions.push(`p.managing_stock = $${paramCounter}`);
-      params.push(req.query.managingStock === 'true');
-      paramCounter++;
-    }
-
-    // Combine all conditions with AND
-    const whereClause = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
-
-    // Get total count for pagination
-    const countQuery = `
-      SELECT COUNT(DISTINCT p.pid) as total 
-      FROM products p
-      LEFT JOIN product_metrics pm ON p.pid = pm.pid
-      ${whereClause}
-    `;
-    const { rows: [countResult] } = await pool.query(countQuery, params);
-    const total = countResult.total;
-
-    // Get available filters
-    const { rows: categories } = await pool.query(
-      'SELECT name FROM categories ORDER BY name'
-    );
-    const { rows: vendors } = await pool.query(
-      'SELECT DISTINCT vendor FROM products WHERE visible = true AND vendor IS NOT NULL AND vendor != \'\' ORDER BY vendor'
-    );
-    const { rows: brands } = await pool.query(
-      'SELECT DISTINCT COALESCE(brand, \'Unbranded\') as brand FROM products WHERE visible = true ORDER BY brand'
-    );
-
-    // Main query with all fields
-    const query = `
-      WITH RECURSIVE 
-        category_path AS (
-          SELECT 
-            c.cat_id,
-            c.name,
-            c.parent_id,
-            CAST(c.name AS text) as path
-          FROM categories c
-          WHERE c.parent_id IS NULL
-          
-          UNION ALL
-          
-          SELECT 
-            c.cat_id,
-            c.name,
-            c.parent_id,
-            cp.path || ' > ' || c.name
-          FROM categories c
-          JOIN category_path cp ON c.parent_id = cp.cat_id
-        ),
-        product_thresholds AS (
-          SELECT 
-            p.pid,
-            COALESCE(
-              (SELECT overstock_days FROM stock_thresholds st 
-               WHERE st.category_id IN (
-                 SELECT pc.cat_id 
-                 FROM product_categories pc 
-                 WHERE pc.pid = p.pid
-               )
-               AND (st.vendor = p.vendor OR st.vendor IS NULL)
-               ORDER BY st.vendor IS NULL
-               LIMIT 1),
-              (SELECT overstock_days FROM stock_thresholds st 
-               WHERE st.category_id IS NULL 
-               AND (st.vendor = p.vendor OR st.vendor IS NULL)
-               ORDER BY st.vendor IS NULL
-               LIMIT 1),
-              90
-            ) as target_days
-          FROM products p
-        ),
-        product_leaf_categories AS (
-          SELECT DISTINCT pc.cat_id
-          FROM product_categories pc
-          WHERE NOT EXISTS (
-            SELECT 1 
-            FROM categories child
-            JOIN product_categories child_pc ON child.cat_id = child_pc.cat_id
-            WHERE child.parent_id = pc.cat_id
-            AND child_pc.pid = pc.pid
-          )
-        )
-      SELECT 
-        p.*,
-        COALESCE(p.brand, 'Unbranded') as brand,
-        string_agg(DISTINCT (c.cat_id || ':' || c.name), ',') as categories,
-        pm.daily_sales_avg,
-        pm.weekly_sales_avg,
-        pm.monthly_sales_avg,
-        pm.avg_quantity_per_order,
-        pm.number_of_orders,
-        pm.first_sale_date,
-        pm.last_sale_date,
-        pm.days_of_inventory,
-        pm.weeks_of_inventory,
-        pm.reorder_point,
-        pm.safety_stock,
-        pm.avg_margin_percent,
-        CAST(pm.total_revenue AS DECIMAL(15,3)) as total_revenue,
-        CAST(pm.inventory_value AS DECIMAL(15,3)) as inventory_value,
-        CAST(pm.cost_of_goods_sold AS DECIMAL(15,3)) as cost_of_goods_sold,
-        CAST(pm.gross_profit AS DECIMAL(15,3)) as gross_profit,
-        pm.gmroi,
-        pm.avg_lead_time_days,
-        pm.last_purchase_date,
-        pm.last_received_date,
-        pm.abc_class,
-        pm.stock_status,
-        pm.turnover_rate
-      FROM products p
-      LEFT JOIN product_metrics pm ON p.pid = pm.pid
-      LEFT JOIN product_categories pc ON p.pid = pc.pid
-      LEFT JOIN categories c ON pc.cat_id = c.cat_id
-      ${whereClause}
-      GROUP BY p.pid, pm.pid
-      ORDER BY ${sortColumn} ${sortDirection}
-      LIMIT $${paramCounter} OFFSET $${paramCounter + 1}
-    `;
-    
-    params.push(limit, offset);
-    const { rows: products } = await pool.query(query, params);
-
-    res.json({
-      products,
-      pagination: {
-        total,
-        pages: Math.ceil(total / limit),
-        currentPage: page,
-        limit
-      },
-      filters: {
-        categories: categories.map(c => c.name),
-        vendors: vendors.map(v => v.vendor),
-        brands: brands.map(b => b.brand)
-      }
-    });
-  } catch (error) {
-    console.error('Error fetching products:', error);
-    res.status(500).json({ error: 'Failed to fetch products' });
-  }
-});
-
-// Get trending products
-router.get('/trending', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    // First check if we have any data
-    const [checkData] = await pool.query(`
-      SELECT COUNT(*) as count, 
-             MAX(total_revenue) as max_revenue,
-             MAX(daily_sales_avg) as max_daily_sales,
-             COUNT(DISTINCT pid) as products_with_metrics
-      FROM product_metrics 
-      WHERE total_revenue > 0 OR daily_sales_avg > 0
-    `);
-    console.log('Product metrics stats:', checkData[0]);
-
-    if (checkData[0].count === 0) {
-      console.log('No products with metrics found');
-      return res.json([]);
-    }
-
-    // Get trending products
-    const [rows] = await pool.query(`
-      SELECT 
-        p.pid,
-        p.sku,
-        p.title,
-        COALESCE(pm.daily_sales_avg, 0) as daily_sales_avg,
-        COALESCE(pm.weekly_sales_avg, 0) as weekly_sales_avg,
-        CASE 
-          WHEN pm.weekly_sales_avg > 0 AND pm.daily_sales_avg > 0
-          THEN ((pm.daily_sales_avg - pm.weekly_sales_avg) / pm.weekly_sales_avg) * 100
-          ELSE 0 
-        END as growth_rate,
-        COALESCE(pm.total_revenue, 0) as total_revenue
-      FROM products p
-      INNER JOIN product_metrics pm ON p.pid = pm.pid
-      WHERE (pm.total_revenue > 0 OR pm.daily_sales_avg > 0)
-        AND p.visible = true
-      ORDER BY growth_rate DESC
-      LIMIT 50
-    `);
-
-    console.log('Trending products:', rows);
-    res.json(rows);
-  } catch (error) {
-    console.error('Error fetching trending products:', error);
-    res.status(500).json({ error: 'Failed to fetch trending products' });
-  }
-});
-
-// Get a single product
-router.get('/:id', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    const id = parseInt(req.params.id);
-
-    // Common CTE for category paths
-    const categoryPathCTE = `
-      WITH RECURSIVE category_path AS (
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CAST(c.name AS CHAR(1000)) as path
-        FROM categories c
-        WHERE c.parent_id IS NULL
-        
-        UNION ALL
-        
-        SELECT 
-          c.cat_id,
-          c.name,
-          c.parent_id,
-          CONCAT(cp.path, ' > ', c.name)
-        FROM categories c
-        JOIN category_path cp ON c.parent_id = cp.cat_id
-      )
-    `;
-
-    // Get product details with category paths
-    const [productRows] = await pool.query(`
-      SELECT 
-        p.*,
-        pm.daily_sales_avg,
-        pm.weekly_sales_avg,
-        pm.monthly_sales_avg,
-        pm.days_of_inventory,
-        pm.reorder_point,
-        pm.safety_stock,
-        pm.stock_status,
-        pm.abc_class,
-        pm.avg_margin_percent,
-        pm.total_revenue,
-        pm.inventory_value,
-        pm.turnover_rate,
-        pm.gmroi,
-        pm.cost_of_goods_sold,
-        pm.gross_profit,
-        pm.avg_lead_time_days,
-        pm.current_lead_time,
-        pm.target_lead_time,
-        pm.lead_time_status,
-        pm.reorder_qty,
-        pm.overstocked_amt
-      FROM products p
-      LEFT JOIN product_metrics pm ON p.pid = pm.pid
-      WHERE p.pid = ?
-    `, [id]);
-
-    if (!productRows.length) {
-      return res.status(404).json({ error: 'Product not found' });
-    }
-
-    // Get categories and their paths separately to avoid GROUP BY issues
-    const [categoryRows] = await pool.query(`
-      WITH RECURSIVE 
-        category_path AS (
-          SELECT 
-            c.cat_id,
-            c.name,
-            c.parent_id,
-            CAST(c.name AS CHAR(1000)) as path
-          FROM categories c
-          WHERE c.parent_id IS NULL
-          
-          UNION ALL
-          
-          SELECT 
-            c.cat_id,
-            c.name,
-            c.parent_id,
-            CONCAT(cp.path, ' > ', c.name)
-          FROM categories c
-          JOIN category_path cp ON c.parent_id = cp.cat_id
-        ),
-        product_leaf_categories AS (
-          -- Find categories assigned to this product that aren't parents
-          -- of other categories assigned to this product
-          SELECT pc.cat_id
-          FROM product_categories pc
-          WHERE pc.pid = ?
-          AND NOT EXISTS (
-            -- Check if there are any child categories also assigned to this product
-            SELECT 1 
-            FROM categories child
-            JOIN product_categories child_pc ON child.cat_id = child_pc.cat_id
-            WHERE child.parent_id = pc.cat_id
-            AND child_pc.pid = pc.pid
-          )
-        )
-      SELECT 
-        c.cat_id,
-        c.name as category_name,
-        cp.path as full_path
-      FROM product_categories pc
-      JOIN categories c ON pc.cat_id = c.cat_id
-      JOIN category_path cp ON c.cat_id = cp.cat_id
-      JOIN product_leaf_categories plc ON c.cat_id = plc.cat_id
-      WHERE pc.pid = ?
-      ORDER BY cp.path
-    `, [id, id]);
-
-    // Transform the results
-    const categoryPathMap = categoryRows.reduce((acc, row) => {
-      // Use cat_id in the key to differentiate categories with the same name
-      acc[`${row.cat_id}:${row.category_name}`] = row.full_path;
-      return acc;
-    }, {});
-
-    const product = {
-      ...productRows[0],
-      // Include cat_id in categories array to match the keys in categoryPathMap
-      categories: categoryRows.map(row => `${row.cat_id}:${row.category_name}`),
-      category_paths: categoryPathMap,
-      price: parseFloat(productRows[0].price),
-      regular_price: parseFloat(productRows[0].regular_price),
-      cost_price: parseFloat(productRows[0].cost_price),
-      landing_cost_price: parseFloat(productRows[0].landing_cost_price),
-      stock_quantity: parseInt(productRows[0].stock_quantity),
-      moq: parseInt(productRows[0].moq),
-      uom: parseInt(productRows[0].uom),
-      managing_stock: Boolean(productRows[0].managing_stock),
-      replenishable: Boolean(productRows[0].replenishable),
-      daily_sales_avg: parseFloat(productRows[0].daily_sales_avg) || 0,
-      weekly_sales_avg: parseFloat(productRows[0].weekly_sales_avg) || 0,
-      monthly_sales_avg: parseFloat(productRows[0].monthly_sales_avg) || 0,
-      avg_quantity_per_order: parseFloat(productRows[0].avg_quantity_per_order) || 0,
-      number_of_orders: parseInt(productRows[0].number_of_orders) || 0,
-      first_sale_date: productRows[0].first_sale_date || null,
-      last_sale_date: productRows[0].last_sale_date || null,
-      days_of_inventory: parseFloat(productRows[0].days_of_inventory) || 0,
-      weeks_of_inventory: parseFloat(productRows[0].weeks_of_inventory) || 0,
-      reorder_point: parseFloat(productRows[0].reorder_point) || 0,
-      safety_stock: parseFloat(productRows[0].safety_stock) || 0,
-      avg_margin_percent: parseFloat(productRows[0].avg_margin_percent) || 0,
-      total_revenue: parseFloat(productRows[0].total_revenue) || 0,
-      inventory_value: parseFloat(productRows[0].inventory_value) || 0,
-      cost_of_goods_sold: parseFloat(productRows[0].cost_of_goods_sold) || 0,
-      gross_profit: parseFloat(productRows[0].gross_profit) || 0,
-      gmroi: parseFloat(productRows[0].gmroi) || 0,
-      avg_lead_time_days: parseFloat(productRows[0].avg_lead_time_days) || 0,
-      current_lead_time: parseFloat(productRows[0].current_lead_time) || 0,
-      target_lead_time: parseFloat(productRows[0].target_lead_time) || 0,
-      lead_time_status: productRows[0].lead_time_status || null,
-      reorder_qty: parseInt(productRows[0].reorder_qty) || 0,
-      overstocked_amt: parseInt(productRows[0].overstocked_amt) || 0
-    };
-
-    res.json(product);
-  } catch (error) {
-    console.error('Error fetching product:', error);
-    res.status(500).json({ error: 'Failed to fetch product' });
-  }
-});
-
-// Import products from CSV
-router.post('/import', upload.single('file'), async (req, res) => {
-  if (!req.file) {
-    return res.status(400).json({ error: 'No file uploaded' });
-  }
-
-  try {
-    const result = await importProductsFromCSV(req.file.path, req.app.locals.pool);
-    // Clean up the uploaded file
-    require('fs').unlinkSync(req.file.path);
-    res.json(result);
-  } catch (error) {
-    console.error('Error importing products:', error);
-    res.status(500).json({ error: 'Failed to import products' });
-  }
-});
-
-// Update a product
-router.put('/:id', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const {
-      title,
-      sku,
-      stock_quantity,
-      price,
-      regular_price,
-      cost_price,
-      vendor,
-      brand,
-      categories,
-      visible,
-      managing_stock
-    } = req.body;
-
-    const [result] = await pool.query(
-      `UPDATE products 
-       SET title = ?,
-           sku = ?,
-           stock_quantity = ?,
-           price = ?,
-           regular_price = ?,
-           cost_price = ?,
-           vendor = ?,
-           brand = ?,
-           categories = ?,
-           visible = ?,
-           managing_stock = ?
-       WHERE pid = ?`,
-      [
-        title,
-        sku,
-        stock_quantity,
-        price,
-        regular_price,
-        cost_price,
-        vendor,
-        brand,
-        categories,
-        visible,
-        managing_stock,
-        req.params.id
-      ]
-    );
-
-    if (result.affectedRows === 0) {
-      return res.status(404).json({ error: 'Product not found' });
-    }
-
-    res.json({ message: 'Product updated successfully' });
-  } catch (error) {
-    console.error('Error updating product:', error);
-    res.status(500).json({ error: 'Failed to update product' });
-  }
-});
-
-// Get product metrics
-router.get('/:id/metrics', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    const { id } = req.params;
-    
-    // Get metrics from product_metrics table with inventory health data
-    const [metrics] = await pool.query(`
-      WITH inventory_status AS (
-        SELECT 
-          p.pid,
-          CASE 
-            WHEN pm.daily_sales_avg = 0 THEN 'New'
-            WHEN p.stock_quantity <= CEIL(pm.daily_sales_avg * 7) THEN 'Critical'
-            WHEN p.stock_quantity <= CEIL(pm.daily_sales_avg * 14) THEN 'Reorder'
-            WHEN p.stock_quantity > (pm.daily_sales_avg * 90) THEN 'Overstocked'
-            ELSE 'Healthy'
-          END as calculated_status
-        FROM products p
-        LEFT JOIN product_metrics pm ON p.pid = pm.pid
-        WHERE p.pid = ?
-      )
-      SELECT 
-        COALESCE(pm.daily_sales_avg, 0) as daily_sales_avg,
-        COALESCE(pm.weekly_sales_avg, 0) as weekly_sales_avg,
-        COALESCE(pm.monthly_sales_avg, 0) as monthly_sales_avg,
-        COALESCE(pm.days_of_inventory, 0) as days_of_inventory,
-        COALESCE(pm.reorder_point, CEIL(COALESCE(pm.daily_sales_avg, 0) * 14)) as reorder_point,
-        COALESCE(pm.safety_stock, CEIL(COALESCE(pm.daily_sales_avg, 0) * 7)) as safety_stock,
-        COALESCE(pm.avg_margin_percent, 
-          ((p.price - COALESCE(p.cost_price, 0)) / NULLIF(p.price, 0)) * 100
-        ) as avg_margin_percent,
-        COALESCE(pm.total_revenue, 0) as total_revenue,
-        COALESCE(pm.inventory_value, p.stock_quantity * COALESCE(p.cost_price, 0)) as inventory_value,
-        COALESCE(pm.turnover_rate, 0) as turnover_rate,
-        COALESCE(pm.abc_class, 'C') as abc_class,
-        COALESCE(pm.stock_status, is.calculated_status) as stock_status,
-        COALESCE(pm.avg_lead_time_days, 0) as avg_lead_time_days,
-        COALESCE(pm.current_lead_time, 0) as current_lead_time,
-        COALESCE(pm.target_lead_time, 14) as target_lead_time,
-        COALESCE(pm.lead_time_status, 'Unknown') as lead_time_status,
-        COALESCE(pm.reorder_qty, 0) as reorder_qty,
-        COALESCE(pm.overstocked_amt, 0) as overstocked_amt
-      FROM products p
-      LEFT JOIN product_metrics pm ON p.pid = pm.pid
-      LEFT JOIN inventory_status is ON p.pid = is.pid
-      WHERE p.pid = ?
-    `, [id]);
-
-    if (!metrics.length) {
-      // Return default metrics structure if no data found
-      res.json({
-        daily_sales_avg: 0,
-        weekly_sales_avg: 0,
-        monthly_sales_avg: 0,
-        days_of_inventory: 0,
-        reorder_point: 0,
-        safety_stock: 0,
-        avg_margin_percent: 0,
-        total_revenue: 0,
-        inventory_value: 0,
-        turnover_rate: 0,
-        abc_class: 'C',
-        stock_status: 'New',
-        avg_lead_time_days: 0,
-        current_lead_time: 0,
-        target_lead_time: 14,
-        lead_time_status: 'Unknown',
-        reorder_qty: 0,
-        overstocked_amt: 0
-      });
-      return;
-    }
-
-    res.json(metrics[0]);
-  } catch (error) {
-    console.error('Error fetching product metrics:', error);
-    res.status(500).json({ error: 'Failed to fetch product metrics' });
-  }
-});
-
-// Get product time series data
-router.get('/:id/time-series', async (req, res) => {
-  const { id } = req.params;
-  try {
-    const pool = req.app.locals.pool;
-
-    // Get monthly sales data
-    const [monthlySales] = await pool.query(`
-      SELECT 
-        DATE_FORMAT(date, '%Y-%m') as month,
-        COUNT(DISTINCT order_number) as order_count,
-        SUM(quantity) as units_sold,
-        CAST(SUM(price * quantity) AS DECIMAL(15,3)) as revenue
-      FROM orders
-      WHERE pid = ?
-        AND canceled = false
-      GROUP BY DATE_FORMAT(date, '%Y-%m')
-      ORDER BY month DESC
-      LIMIT 12
-    `, [id]);
-
-    // Format monthly sales data
-    const formattedMonthlySales = monthlySales.map(month => ({
-      month: month.month,
-      order_count: parseInt(month.order_count),
-      units_sold: parseInt(month.units_sold),
-      revenue: parseFloat(month.revenue),
-      profit: 0 // Set to 0 since we don't have cost data in orders table
-    }));
-
-    // Get recent orders
-    const [recentOrders] = await pool.query(`
-      SELECT 
-        DATE_FORMAT(date, '%Y-%m-%d') as date,
-        order_number,
-        quantity,
-        price,
-        discount,
-        tax,
-        shipping,
-        customer_name as customer,
-        status
-      FROM orders 
-      WHERE pid = ?
-        AND canceled = false
-      ORDER BY date DESC
-      LIMIT 10
-    `, [id]);
-
-    // Get recent purchase orders with detailed status
-    const [recentPurchases] = await pool.query(`
-      SELECT 
-        DATE_FORMAT(date, '%Y-%m-%d') as date,
-        DATE_FORMAT(expected_date, '%Y-%m-%d') as expected_date,
-        DATE_FORMAT(received_date, '%Y-%m-%d') as received_date,
-        po_id,
-        ordered,
-        received,
-        status,
-        receiving_status,
-        cost_price,
-        notes,
-        CASE 
-          WHEN received_date IS NOT NULL THEN 
-            DATEDIFF(received_date, date)
-          WHEN expected_date < CURDATE() AND status < ${PurchaseOrderStatus.ReceivingStarted} THEN 
-            DATEDIFF(CURDATE(), expected_date)
-          ELSE NULL 
-        END as lead_time_days
-      FROM purchase_orders 
-      WHERE pid = ?
-        AND status != ${PurchaseOrderStatus.Canceled}
-      ORDER BY date DESC
-      LIMIT 10
-    `, [id]);
-
-    res.json({
-      monthly_sales: formattedMonthlySales,
-      recent_orders: recentOrders.map(order => ({
-        ...order,
-        price: parseFloat(order.price),
-        discount: parseFloat(order.discount),
-        tax: parseFloat(order.tax),
-        shipping: parseFloat(order.shipping),
-        quantity: parseInt(order.quantity)
-      })),
-      recent_purchases: recentPurchases.map(po => ({
-        ...po,
-        ordered: parseInt(po.ordered),
-        received: parseInt(po.received),
-        status: parseInt(po.status),
-        receiving_status: parseInt(po.receiving_status),
-        cost_price: parseFloat(po.cost_price),
-        lead_time_days: po.lead_time_days ? parseInt(po.lead_time_days) : null
-      }))
-    });
-  } catch (error) {
-    console.error('Error fetching product time series:', error);
-    res.status(500).json({ error: 'Failed to fetch product time series' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/purchase-orders.js

@@ -1,430 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Status code constants
-const STATUS = {
-  CANCELED: 0,
-  CREATED: 1,
-  ELECTRONICALLY_READY_SEND: 10,
-  ORDERED: 11,
-  PREORDERED: 12,
-  ELECTRONICALLY_SENT: 13,
-  RECEIVING_STARTED: 15,
-  DONE: 50
-};
-
-const RECEIVING_STATUS = {
-  CANCELED: 0,
-  CREATED: 1,
-  PARTIAL_RECEIVED: 30,
-  FULL_RECEIVED: 40,
-  PAID: 50
-};
-
-// Get all purchase orders with summary metrics
-router.get('/', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    const { search, status, vendor, startDate, endDate, page = 1, limit = 100, sortColumn = 'date', sortDirection = 'desc' } = req.query;
-    
-    let whereClause = '1=1';
-    const params = [];
-    let paramCounter = 1;
-    
-    if (search) {
-      whereClause += ` AND (po.po_id ILIKE $${paramCounter} OR po.vendor ILIKE $${paramCounter})`;
-      params.push(`%${search}%`);
-      paramCounter++;
-    }
-    
-    if (status && status !== 'all') {
-      whereClause += ` AND po.status = $${paramCounter}`;
-      params.push(Number(status));
-      paramCounter++;
-    }
-    
-    if (vendor && vendor !== 'all') {
-      whereClause += ` AND po.vendor = $${paramCounter}`;
-      params.push(vendor);
-      paramCounter++;
-    }
-    
-    if (startDate) {
-      whereClause += ` AND po.date >= $${paramCounter}`;
-      params.push(startDate);
-      paramCounter++;
-    }
-    
-    if (endDate) {
-      whereClause += ` AND po.date <= $${paramCounter}`;
-      params.push(endDate);
-      paramCounter++;
-    }
-
-    // Get filtered summary metrics
-    const { rows: [summary] } = await pool.query(`
-      WITH po_totals AS (
-        SELECT 
-          po_id,
-          SUM(ordered) as total_ordered,
-          SUM(received) as total_received,
-          ROUND(SUM(ordered * cost_price)::numeric, 3) as total_cost
-        FROM purchase_orders po
-        WHERE ${whereClause}
-        GROUP BY po_id
-      )
-      SELECT 
-        COUNT(DISTINCT po_id) as order_count,
-        SUM(total_ordered) as total_ordered,
-        SUM(total_received) as total_received,
-        ROUND(
-          (SUM(total_received)::numeric / NULLIF(SUM(total_ordered), 0)), 3
-        ) as fulfillment_rate,
-        ROUND(SUM(total_cost)::numeric, 3) as total_value,
-        ROUND(AVG(total_cost)::numeric, 3) as avg_cost
-      FROM po_totals
-    `, params);
-
-    // Get total count for pagination
-    const { rows: [countResult] } = await pool.query(`
-      SELECT COUNT(DISTINCT po_id) as total
-      FROM purchase_orders po
-      WHERE ${whereClause}
-    `, params);
-
-    const total = countResult.total;
-    const offset = (page - 1) * limit;
-    const pages = Math.ceil(total / limit);
-
-    // Get recent purchase orders
-    const { rows: orders } = await pool.query(`
-      WITH po_totals AS (
-        SELECT 
-          po_id,
-          vendor,
-          date,
-          status,
-          receiving_status,
-          COUNT(DISTINCT pid) as total_items,
-          SUM(ordered) as total_quantity,
-          ROUND(SUM(ordered * cost_price)::numeric, 3) as total_cost,
-          SUM(received) as total_received,
-          ROUND(
-            (SUM(received)::numeric / NULLIF(SUM(ordered), 0)), 3
-          ) as fulfillment_rate
-        FROM purchase_orders po
-        WHERE ${whereClause}
-        GROUP BY po_id, vendor, date, status, receiving_status
-      )
-      SELECT 
-        po_id as id,
-        vendor as vendor_name,
-        to_char(date, 'YYYY-MM-DD') as order_date,
-        status,
-        receiving_status,
-        total_items,
-        total_quantity,
-        total_cost,
-        total_received,
-        fulfillment_rate
-      FROM po_totals
-      ORDER BY 
-        CASE 
-          WHEN $${paramCounter} = 'order_date' THEN date
-          WHEN $${paramCounter} = 'vendor_name' THEN vendor
-          WHEN $${paramCounter} = 'total_cost' THEN total_cost
-          WHEN $${paramCounter} = 'total_received' THEN total_received
-          WHEN $${paramCounter} = 'total_items' THEN total_items
-          WHEN $${paramCounter} = 'total_quantity' THEN total_quantity
-          WHEN $${paramCounter} = 'fulfillment_rate' THEN fulfillment_rate
-          WHEN $${paramCounter} = 'status' THEN status
-          ELSE date
-        END ${sortDirection === 'desc' ? 'DESC' : 'ASC'}
-      LIMIT $${paramCounter + 1} OFFSET $${paramCounter + 2}
-    `, [...params, sortColumn, Number(limit), offset]);
-
-    // Get unique vendors for filter options
-    const { rows: vendors } = await pool.query(`
-      SELECT DISTINCT vendor 
-      FROM purchase_orders 
-      WHERE vendor IS NOT NULL AND vendor != ''
-      ORDER BY vendor
-    `);
-
-    // Get unique statuses for filter options
-    const { rows: statuses } = await pool.query(`
-      SELECT DISTINCT status 
-      FROM purchase_orders 
-      WHERE status IS NOT NULL
-      ORDER BY status
-    `);
-
-    // Parse numeric values
-    const parsedOrders = orders.map(order => ({
-      id: order.id,
-      vendor_name: order.vendor_name,
-      order_date: order.order_date,
-      status: Number(order.status),
-      receiving_status: Number(order.receiving_status),
-      total_items: Number(order.total_items) || 0,
-      total_quantity: Number(order.total_quantity) || 0,
-      total_cost: Number(order.total_cost) || 0,
-      total_received: Number(order.total_received) || 0,
-      fulfillment_rate: Number(order.fulfillment_rate) || 0
-    }));
-
-    // Parse summary metrics
-    const parsedSummary = {
-      order_count: Number(summary.order_count) || 0,
-      total_ordered: Number(summary.total_ordered) || 0,
-      total_received: Number(summary.total_received) || 0,
-      fulfillment_rate: Number(summary.fulfillment_rate) || 0,
-      total_value: Number(summary.total_value) || 0,
-      avg_cost: Number(summary.avg_cost) || 0
-    };
-
-    res.json({
-      orders: parsedOrders,
-      summary: parsedSummary,
-      pagination: {
-        total,
-        pages,
-        page: Number(page),
-        limit: Number(limit)
-      },
-      filters: {
-        vendors: vendors.map(v => v.vendor),
-        statuses: statuses.map(s => Number(s.status))
-      }
-    });
-  } catch (error) {
-    console.error('Error fetching purchase orders:', error);
-    res.status(500).json({ error: 'Failed to fetch purchase orders' });
-  }
-});
-
-// Get vendor performance metrics
-router.get('/vendor-metrics', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    const { rows: metrics } = await pool.query(`
-      WITH delivery_metrics AS (
-        SELECT 
-          vendor,
-          po_id,
-          ordered,
-          received,
-          cost_price,
-          CASE 
-            WHEN status >= ${STATUS.RECEIVING_STARTED} AND receiving_status >= ${RECEIVING_STATUS.PARTIAL_RECEIVED}
-            AND received_date IS NOT NULL AND date IS NOT NULL
-            THEN (received_date - date)::integer
-            ELSE NULL
-          END as delivery_days
-        FROM purchase_orders
-        WHERE vendor IS NOT NULL AND vendor != ''
-          AND status != ${STATUS.CANCELED}  -- Exclude canceled orders
-      )
-      SELECT 
-        vendor as vendor_name,
-        COUNT(DISTINCT po_id) as total_orders,
-        SUM(ordered) as total_ordered,
-        SUM(received) as total_received,
-        ROUND(
-          (SUM(received)::numeric / NULLIF(SUM(ordered), 0)), 3
-        ) as fulfillment_rate,
-        ROUND(
-          (SUM(ordered * cost_price)::numeric / NULLIF(SUM(ordered), 0)), 2
-        ) as avg_unit_cost,
-        ROUND(SUM(ordered * cost_price)::numeric, 3) as total_spend,
-        ROUND(
-          AVG(NULLIF(delivery_days, 0))::numeric, 1
-        ) as avg_delivery_days
-      FROM delivery_metrics
-      GROUP BY vendor
-      HAVING COUNT(DISTINCT po_id) > 0
-      ORDER BY total_spend DESC
-    `);
-
-    // Parse numeric values
-    const parsedMetrics = metrics.map(vendor => ({
-      id: vendor.vendor_name,
-      vendor_name: vendor.vendor_name,
-      total_orders: Number(vendor.total_orders) || 0,
-      total_ordered: Number(vendor.total_ordered) || 0,
-      total_received: Number(vendor.total_received) || 0,
-      fulfillment_rate: Number(vendor.fulfillment_rate) || 0,
-      avg_unit_cost: Number(vendor.avg_unit_cost) || 0,
-      total_spend: Number(vendor.total_spend) || 0,
-      avg_delivery_days: Number(vendor.avg_delivery_days) || 0
-    }));
-
-    res.json(parsedMetrics);
-  } catch (error) {
-    console.error('Error fetching vendor metrics:', error);
-    res.status(500).json({ error: 'Failed to fetch vendor metrics' });
-  }
-});
-
-// Get cost analysis
-router.get('/cost-analysis', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    const [analysis] = await pool.query(`
-      WITH category_costs AS (
-        SELECT 
-          c.name as category,
-          po.pid,
-          po.cost_price,
-          po.ordered,
-          po.received,
-          po.status,
-          po.receiving_status
-        FROM purchase_orders po
-        JOIN product_categories pc ON po.pid = pc.pid
-        JOIN categories c ON pc.cat_id = c.cat_id
-        WHERE po.status != ${STATUS.CANCELED}  -- Exclude canceled orders
-      )
-      SELECT 
-        category,
-        COUNT(DISTINCT pid) as unique_products,
-        CAST(AVG(cost_price) AS DECIMAL(15,3)) as avg_cost,
-        CAST(MIN(cost_price) AS DECIMAL(15,3)) as min_cost,
-        CAST(MAX(cost_price) AS DECIMAL(15,3)) as max_cost,
-        CAST(STDDEV(cost_price) AS DECIMAL(15,3)) as cost_variance,
-        CAST(SUM(ordered * cost_price) AS DECIMAL(15,3)) as total_spend
-      FROM category_costs
-      GROUP BY category
-      ORDER BY total_spend DESC
-    `);
-
-    // Parse numeric values
-    const parsedAnalysis = {
-      categories: analysis.map(cat => ({
-        category: cat.category,
-        unique_products: Number(cat.unique_products) || 0,
-        avg_cost: Number(cat.avg_cost) || 0,
-        min_cost: Number(cat.min_cost) || 0,
-        max_cost: Number(cat.max_cost) || 0,
-        cost_variance: Number(cat.cost_variance) || 0,
-        total_spend: Number(cat.total_spend) || 0
-      }))
-    };
-
-    res.json(parsedAnalysis);
-  } catch (error) {
-    console.error('Error fetching cost analysis:', error);
-    res.status(500).json({ error: 'Failed to fetch cost analysis' });
-  }
-});
-
-// Get receiving status metrics
-router.get('/receiving-status', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    const [status] = await pool.query(`
-      WITH po_totals AS (
-        SELECT 
-          po_id,
-          status,
-          receiving_status,
-          SUM(ordered) as total_ordered,
-          SUM(received) as total_received,
-          CAST(SUM(ordered * cost_price) AS DECIMAL(15,3)) as total_cost
-        FROM purchase_orders
-        WHERE status != ${STATUS.CANCELED}
-        GROUP BY po_id, status, receiving_status
-      )
-      SELECT 
-        COUNT(DISTINCT po_id) as order_count,
-        SUM(total_ordered) as total_ordered,
-        SUM(total_received) as total_received,
-        ROUND(
-          SUM(total_received) / NULLIF(SUM(total_ordered), 0), 3
-        ) as fulfillment_rate,
-        CAST(SUM(total_cost) AS DECIMAL(15,3)) as total_value,
-        CAST(AVG(total_cost) AS DECIMAL(15,3)) as avg_cost,
-        COUNT(DISTINCT CASE 
-          WHEN receiving_status = ${RECEIVING_STATUS.CREATED} THEN po_id 
-          END) as pending_count,
-        COUNT(DISTINCT CASE 
-          WHEN receiving_status = ${RECEIVING_STATUS.PARTIAL_RECEIVED} THEN po_id 
-          END) as partial_count,
-        COUNT(DISTINCT CASE 
-          WHEN receiving_status >= ${RECEIVING_STATUS.FULL_RECEIVED} THEN po_id 
-          END) as completed_count,
-        COUNT(DISTINCT CASE 
-          WHEN receiving_status = ${RECEIVING_STATUS.CANCELED} THEN po_id 
-          END) as canceled_count
-      FROM po_totals
-    `);
-
-    // Parse numeric values
-    const parsedStatus = {
-      order_count: Number(status[0].order_count) || 0,
-      total_ordered: Number(status[0].total_ordered) || 0,
-      total_received: Number(status[0].total_received) || 0,
-      fulfillment_rate: Number(status[0].fulfillment_rate) || 0,
-      total_value: Number(status[0].total_value) || 0,
-      avg_cost: Number(status[0].avg_cost) || 0,
-      status_breakdown: {
-        pending: Number(status[0].pending_count) || 0,
-        partial: Number(status[0].partial_count) || 0,
-        completed: Number(status[0].completed_count) || 0,
-        canceled: Number(status[0].canceled_count) || 0
-      }
-    };
-
-    res.json(parsedStatus);
-  } catch (error) {
-    console.error('Error fetching receiving status:', error);
-    res.status(500).json({ error: 'Failed to fetch receiving status' });
-  }
-});
-
-// Get order vs received quantities by product
-router.get('/order-vs-received', async (req, res) => {
-  try {
-    const pool = req.app.locals.pool;
-    
-    const [quantities] = await pool.query(`
-      SELECT 
-        p.product_id,
-        p.title as product,
-        p.SKU as sku,
-        SUM(po.ordered) as ordered_quantity,
-        SUM(po.received) as received_quantity,
-        ROUND(
-          SUM(po.received) / NULLIF(SUM(po.ordered), 0) * 100, 1
-        ) as fulfillment_rate,
-        COUNT(DISTINCT po.po_id) as order_count
-      FROM products p
-      JOIN purchase_orders po ON p.product_id = po.product_id
-      WHERE po.date >= DATE_SUB(CURDATE(), INTERVAL 90 DAY)
-      GROUP BY p.product_id, p.title, p.SKU
-      HAVING order_count > 0
-      ORDER BY ordered_quantity DESC
-      LIMIT 20
-    `);
-
-    // Parse numeric values and add id for React keys
-    const parsedQuantities = quantities.map(q => ({
-      id: q.product_id,
-      ...q,
-      ordered_quantity: Number(q.ordered_quantity),
-      received_quantity: Number(q.received_quantity),
-      fulfillment_rate: Number(q.fulfillment_rate),
-      order_count: Number(q.order_count)
-    }));
-
-    res.json(parsedQuantities);
-  } catch (error) {
-    console.error('Error fetching order vs received quantities:', error);
-    res.status(500).json({ error: 'Failed to fetch order vs received quantities' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/templates.js

@@ -1,283 +0,0 @@
-const express = require('express');
-const { getPool } = require('../utils/db');
-const dotenv = require('dotenv');
-const path = require('path');
-
-dotenv.config({ path: path.join(__dirname, "../../.env") });
-
-const router = express.Router();
-
-// Get all templates
-router.get('/', async (req, res) => {
-  try {
-    const pool = getPool();
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM templates 
-      ORDER BY company ASC, product_type ASC
-    `);
-    res.json(result.rows);
-  } catch (error) {
-    console.error('Error fetching templates:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch templates',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Get template by company and product type
-router.get('/:company/:productType', async (req, res) => {
-  try {
-    const { company, productType } = req.params;
-    const pool = getPool();
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      SELECT * FROM templates 
-      WHERE company = $1 AND product_type = $2
-    `, [company, productType]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'Template not found' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error fetching template:', error);
-    res.status(500).json({ 
-      error: 'Failed to fetch template',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Create new template
-router.post('/', async (req, res) => {
-  try {
-    const {
-      company,
-      product_type,
-      supplier,
-      msrp,
-      cost_each,
-      qty_per_unit,
-      case_qty,
-      hts_code,
-      description,
-      weight,
-      length,
-      width,
-      height,
-      tax_cat,
-      size_cat,
-      categories,
-      ship_restrictions
-    } = req.body;
-
-    // Validate required fields
-    if (!company || !product_type) {
-      return res.status(400).json({ error: 'Company and Product Type are required' });
-    }
-
-    const pool = getPool();
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      INSERT INTO templates (
-        company,
-        product_type,
-        supplier,
-        msrp,
-        cost_each,
-        qty_per_unit,
-        case_qty,
-        hts_code,
-        description,
-        weight,
-        length,
-        width,
-        height,
-        tax_cat,
-        size_cat,
-        categories,
-        ship_restrictions
-      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17)
-      RETURNING *
-    `, [
-      company,
-      product_type,
-      supplier,
-      msrp,
-      cost_each,
-      qty_per_unit,
-      case_qty,
-      hts_code,
-      description,
-      weight,
-      length,
-      width,
-      height,
-      tax_cat,
-      size_cat,
-      categories,
-      ship_restrictions
-    ]);
-
-    res.status(201).json(result.rows[0]);
-  } catch (error) {
-    console.error('Error creating template:', error);
-    // Check for unique constraint violation
-    if (error instanceof Error && error.message.includes('unique constraint')) {
-      return res.status(409).json({ 
-        error: 'Template already exists for this company and product type',
-        details: error.message
-      });
-    }
-    res.status(500).json({ 
-      error: 'Failed to create template',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Update template
-router.put('/:id', async (req, res) => {
-  try {
-    const { id } = req.params;
-    const {
-      company,
-      product_type,
-      supplier,
-      msrp,
-      cost_each,
-      qty_per_unit,
-      case_qty,
-      hts_code,
-      description,
-      weight,
-      length,
-      width,
-      height,
-      tax_cat,
-      size_cat,
-      categories,
-      ship_restrictions
-    } = req.body;
-
-    // Validate required fields
-    if (!company || !product_type) {
-      return res.status(400).json({ error: 'Company and Product Type are required' });
-    }
-
-    const pool = getPool();
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query(`
-      UPDATE templates 
-      SET 
-        company = $1,
-        product_type = $2,
-        supplier = $3,
-        msrp = $4,
-        cost_each = $5,
-        qty_per_unit = $6,
-        case_qty = $7,
-        hts_code = $8,
-        description = $9,
-        weight = $10,
-        length = $11,
-        width = $12,
-        height = $13,
-        tax_cat = $14,
-        size_cat = $15,
-        categories = $16,
-        ship_restrictions = $17
-      WHERE id = $18
-      RETURNING *
-    `, [
-      company,
-      product_type,
-      supplier,
-      msrp,
-      cost_each,
-      qty_per_unit,
-      case_qty,
-      hts_code,
-      description,
-      weight,
-      length,
-      width,
-      height,
-      tax_cat,
-      size_cat,
-      categories,
-      ship_restrictions,
-      id
-    ]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'Template not found' });
-    }
-
-    res.json(result.rows[0]);
-  } catch (error) {
-    console.error('Error updating template:', error);
-    // Check for unique constraint violation
-    if (error instanceof Error && error.message.includes('unique constraint')) {
-      return res.status(409).json({ 
-        error: 'Template already exists for this company and product type',
-        details: error.message
-      });
-    }
-    res.status(500).json({ 
-      error: 'Failed to update template',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Delete template
-router.delete('/:id', async (req, res) => {
-  try {
-    const { id } = req.params;
-    const pool = getPool();
-    if (!pool) {
-      throw new Error('Database pool not initialized');
-    }
-    
-    const result = await pool.query('DELETE FROM templates WHERE id = $1 RETURNING *', [id]);
-
-    if (result.rows.length === 0) {
-      return res.status(404).json({ error: 'Template not found' });
-    }
-
-    res.json({ message: 'Template deleted successfully' });
-  } catch (error) {
-    console.error('Error deleting template:', error);
-    res.status(500).json({ 
-      error: 'Failed to delete template',
-      details: error instanceof Error ? error.message : 'Unknown error'
-    });
-  }
-});
-
-// Error handling middleware
-router.use((err, req, res, next) => {
-  console.error('Template route error:', err);
-  res.status(500).json({ 
-    error: 'Internal server error',
-    details: err.message
-  });
-});
-
-module.exports = router; 

inventory-server/src/routes/vendors.js

@@ -1,108 +0,0 @@
-const express = require('express');
-const router = express.Router();
-
-// Get vendors with pagination, filtering, and sorting
-router.get('/', async (req, res) => {
-  const pool = req.app.locals.pool;
-  try {
-    // Get all vendors with metrics
-    const { rows: vendors } = await pool.query(`
-      SELECT DISTINCT
-        p.vendor as name,
-        COALESCE(vm.active_products, 0) as active_products,
-        COALESCE(vm.total_orders, 0) as total_orders,
-        COALESCE(vm.avg_lead_time_days, 0) as avg_lead_time_days,
-        COALESCE(vm.on_time_delivery_rate, 0) as on_time_delivery_rate,
-        COALESCE(vm.order_fill_rate, 0) as order_fill_rate,
-        CASE 
-          WHEN COALESCE(vm.total_orders, 0) > 0 AND COALESCE(vm.order_fill_rate, 0) >= 75 THEN 'active'
-          WHEN COALESCE(vm.total_orders, 0) > 0 THEN 'inactive'
-          ELSE 'pending'
-        END as status
-      FROM products p
-      LEFT JOIN vendor_metrics vm ON p.vendor = vm.vendor
-      WHERE p.vendor IS NOT NULL AND p.vendor != ''
-    `);
-
-    // Get cost metrics for all vendors
-    const vendorNames = vendors.map(v => v.name);
-    const { rows: costMetrics } = await pool.query(`
-      SELECT 
-        vendor,
-        ROUND((SUM(ordered * cost_price)::numeric / NULLIF(SUM(ordered), 0)), 2) as avg_unit_cost,
-        ROUND(SUM(ordered * cost_price)::numeric, 3) as total_spend
-      FROM purchase_orders
-      WHERE status = 'closed'
-        AND cost_price IS NOT NULL
-        AND ordered > 0
-        AND vendor = ANY($1)
-      GROUP BY vendor
-    `, [vendorNames]);
-
-    // Create a map of cost metrics by vendor
-    const costMetricsMap = costMetrics.reduce((acc, curr) => {
-      acc[curr.vendor] = {
-        avg_unit_cost: curr.avg_unit_cost,
-        total_spend: curr.total_spend
-      };
-      return acc;
-    }, {});
-
-    // Get overall stats
-    const { rows: [stats] } = await pool.query(`
-      SELECT 
-        COUNT(DISTINCT p.vendor) as totalVendors,
-        COUNT(DISTINCT CASE 
-          WHEN COALESCE(vm.total_orders, 0) > 0 AND COALESCE(vm.order_fill_rate, 0) >= 75 
-          THEN p.vendor 
-        END) as activeVendors,
-        COALESCE(ROUND(AVG(NULLIF(vm.avg_lead_time_days, 0))::numeric, 1), 0) as avgLeadTime,
-        COALESCE(ROUND(AVG(NULLIF(vm.order_fill_rate, 0))::numeric, 1), 0) as avgFillRate,
-        COALESCE(ROUND(AVG(NULLIF(vm.on_time_delivery_rate, 0))::numeric, 1), 0) as avgOnTimeDelivery
-      FROM products p
-      LEFT JOIN vendor_metrics vm ON p.vendor = vm.vendor
-      WHERE p.vendor IS NOT NULL AND p.vendor != ''
-    `);
-
-    // Get overall cost metrics
-    const { rows: [overallCostMetrics] } = await pool.query(`
-      SELECT 
-        ROUND((SUM(ordered * cost_price)::numeric / NULLIF(SUM(ordered), 0)), 2) as avg_unit_cost,
-        ROUND(SUM(ordered * cost_price)::numeric, 3) as total_spend
-      FROM purchase_orders
-      WHERE status = 'closed'
-        AND cost_price IS NOT NULL
-        AND ordered > 0
-        AND vendor IS NOT NULL AND vendor != ''
-    `);
-
-    res.json({
-      vendors: vendors.map(vendor => ({
-        vendor_id: vendor.name,
-        name: vendor.name,
-        status: vendor.status,
-        avg_lead_time_days: parseFloat(vendor.avg_lead_time_days),
-        on_time_delivery_rate: parseFloat(vendor.on_time_delivery_rate),
-        order_fill_rate: parseFloat(vendor.order_fill_rate),
-        total_orders: parseInt(vendor.total_orders),
-        active_products: parseInt(vendor.active_products),
-        avg_unit_cost: parseFloat(costMetricsMap[vendor.name]?.avg_unit_cost || 0),
-        total_spend: parseFloat(costMetricsMap[vendor.name]?.total_spend || 0)
-      })),
-      stats: {
-        totalVendors: parseInt(stats.totalvendors),
-        activeVendors: parseInt(stats.activevendors),
-        avgLeadTime: parseFloat(stats.avgleadtime),
-        avgFillRate: parseFloat(stats.avgfillrate),
-        avgOnTimeDelivery: parseFloat(stats.avgontimedelivery),
-        avgUnitCost: parseFloat(overallCostMetrics.avg_unit_cost),
-        totalSpend: parseFloat(overallCostMetrics.total_spend)
-      }
-    });
-  } catch (error) {
-    console.error('Error fetching vendors:', error);
-    res.status(500).json({ error: 'Failed to fetch vendors' });
-  }
-});
-
-module.exports = router; 

inventory-server/src/routes/vendorsAggregate.js

@@ -0,0 +1 @@
+ 

inventory-server/src/server.js

@@ -1,204 +0,0 @@
-const express = require('express');
-const cors = require('cors');
-const { spawn } = require('child_process');
-const path = require('path');
-const fs = require('fs');
-const { corsMiddleware, corsErrorHandler } = require('./middleware/cors');
-const { initPool } = require('./utils/db');
-const productsRouter = require('./routes/products');
-const dashboardRouter = require('./routes/dashboard');
-const ordersRouter = require('./routes/orders');
-const csvRouter = require('./routes/csv');
-const analyticsRouter = require('./routes/analytics');
-const purchaseOrdersRouter = require('./routes/purchase-orders');
-const configRouter = require('./routes/config');
-const metricsRouter = require('./routes/metrics');
-const vendorsRouter = require('./routes/vendors');
-const categoriesRouter = require('./routes/categories');
-const importRouter = require('./routes/import');
-const aiValidationRouter = require('./routes/ai-validation');
-const templatesRouter = require('./routes/templates');
-const aiPromptsRouter = require('./routes/ai-prompts');
-
-// Get the absolute path to the .env file
-const envPath = '/var/www/html/inventory/.env';
-console.log('Looking for .env file at:', envPath);
-console.log('.env file exists:', fs.existsSync(envPath));
-
-try {
-  require('dotenv').config({ path: envPath });
-  console.log('.env file loaded successfully');
-  console.log('Environment check:', {
-    NODE_ENV: process.env.NODE_ENV || 'not set',
-    PORT: process.env.PORT || 'not set',
-    DB_HOST: process.env.DB_HOST || 'not set',
-    DB_USER: process.env.DB_USER || 'not set',
-    DB_NAME: process.env.DB_NAME || 'not set',
-    DB_PASSWORD: process.env.DB_PASSWORD ? '[password set]' : 'not set',
-    DB_PORT: process.env.DB_PORT || 'not set',
-    DB_SSL: process.env.DB_SSL || 'not set'
-  });
-} catch (error) {
-  console.error('Error loading .env file:', error);
-}
-
-// Ensure required directories exist
-['logs', 'uploads'].forEach(dir => {
-  if (!fs.existsSync(dir)) {
-    fs.mkdirSync(dir, { recursive: true });
-  }
-});
-
-const app = express();
-
-// Debug middleware to log request details
-app.use((req, res, next) => {
-  console.log('Request details:', {
-    method: req.method,
-    url: req.url,
-    origin: req.get('Origin'),
-    headers: req.headers
-  });
-  next();
-});
-
-// Apply CORS middleware first, before any other middleware
-app.use(corsMiddleware);
-
-// Body parser middleware
-app.use(express.json({ limit: '10mb' }));
-app.use(express.urlencoded({ extended: true, limit: '10mb' }));
-
-// Initialize database pool and start server
-async function startServer() {
-  try {
-    // Initialize database pool
-    const pool = await initPool({
-      host: process.env.DB_HOST,
-      user: process.env.DB_USER,
-      password: process.env.DB_PASSWORD,
-      database: process.env.DB_NAME,
-      port: process.env.DB_PORT || 5432,
-      max: process.env.NODE_ENV === 'production' ? 20 : 10,
-      idleTimeoutMillis: 30000,
-      connectionTimeoutMillis: 2000,
-      ssl: process.env.DB_SSL === 'true' ? {
-        rejectUnauthorized: false
-      } : false
-    });
-
-    // Make pool available to routes
-    app.locals.pool = pool;
-
-    // Set up routes after pool is initialized
-    app.use('/api/products', productsRouter);
-    app.use('/api/dashboard', dashboardRouter);
-    app.use('/api/orders', ordersRouter);
-    app.use('/api/csv', csvRouter);
-    app.use('/api/analytics', analyticsRouter);
-    app.use('/api/purchase-orders', purchaseOrdersRouter);
-    app.use('/api/config', configRouter);
-    app.use('/api/metrics', metricsRouter);
-    app.use('/api/vendors', vendorsRouter);
-    app.use('/api/categories', categoriesRouter);
-    app.use('/api/import', importRouter);
-    app.use('/api/ai-validation', aiValidationRouter);
-    app.use('/api/templates', templatesRouter);
-    app.use('/api/ai-prompts', aiPromptsRouter);
-
-    // Basic health check route
-    app.get('/health', (req, res) => {
-      res.json({ 
-        status: 'ok',
-        timestamp: new Date().toISOString(),
-        environment: process.env.NODE_ENV
-      });
-    });
-
-    // CORS error handler - must be before other error handlers
-    app.use(corsErrorHandler);
-
-    // Error handling middleware - MUST be after routes and CORS error handler
-    app.use((err, req, res, next) => {
-      console.error(`[${new Date().toISOString()}] Error:`, err);
-      
-      // Send detailed error in development, generic in production
-      const error = process.env.NODE_ENV === 'production'
-        ? 'An internal server error occurred'
-        : err.message || err;
-        
-      res.status(err.status || 500).json({ error });
-    });
-
-    const PORT = process.env.PORT || 3000;
-    app.listen(PORT, () => {
-      console.log(`[Server] Running in ${process.env.NODE_ENV || 'development'} mode on port ${PORT}`);
-    });
-  } catch (error) {
-    console.error('Failed to start server:', error);
-    process.exit(1);
-  }
-}
-
-// Handle uncaught exceptions
-process.on('uncaughtException', (err) => {
-  console.error(`[${new Date().toISOString()}] Uncaught Exception:`, err);
-  process.exit(1);
-});
-
-process.on('unhandledRejection', (reason, promise) => {
-  console.error(`[${new Date().toISOString()}] Unhandled Rejection at:`, promise, 'reason:', reason);
-});
-
-// Initialize client sets for SSE
-const importClients = new Set();
-const updateClients = new Set();
-const resetClients = new Set();
-const resetMetricsClients = new Set();
-
-// Helper function to send progress to SSE clients
-const sendProgressToClients = (clients, data) => {
-  clients.forEach(client => {
-    try {
-      client.write(`data: ${JSON.stringify(data)}\n\n`);
-    } catch (error) {
-      console.error('Error sending SSE update:', error);
-    }
-  });
-};
-
-// Setup SSE connection
-const setupSSE = (req, res) => {
-  const { type } = req.params;
-  
-  // Set headers for SSE
-  res.writeHead(200, {
-    'Content-Type': 'text/event-stream',
-    'Cache-Control': 'no-cache',
-    'Connection': 'keep-alive',
-    'Access-Control-Allow-Origin': req.headers.origin || '*',
-    'Access-Control-Allow-Credentials': 'true'
-  });
-
-  // Send initial message
-  res.write('data: {"status":"connected"}\n\n');
-
-  // Add client to appropriate set
-  const clientSet = type === 'import' ? importClients :
-                   type === 'update' ? updateClients :
-                   type === 'reset' ? resetClients :
-                   type === 'reset-metrics' ? resetMetricsClients :
-                   null;
-
-  if (clientSet) {
-    clientSet.add(res);
-    
-    // Remove client when connection closes
-    req.on('close', () => {
-      clientSet.delete(res);
-    });
-  }
-};
-
-// Start the server
-startServer(); 

inventory-server/src/types/status-codes.js

@@ -1,79 +0,0 @@
-// Purchase Order Status Codes
-const PurchaseOrderStatus = {
-    Canceled: 0,
-    Created: 1,
-    ElectronicallyReadySend: 10,
-    Ordered: 11,
-    Preordered: 12,
-    ElectronicallySent: 13,
-    ReceivingStarted: 15,
-    Done: 50
-};
-
-// Receiving Status Codes
-const ReceivingStatus = {
-    Canceled: 0,
-    Created: 1,
-    PartialReceived: 30,
-    FullReceived: 40,
-    Paid: 50
-};
-
-// Status Code Display Names
-const PurchaseOrderStatusLabels = {
-    [PurchaseOrderStatus.Canceled]: 'Canceled',
-    [PurchaseOrderStatus.Created]: 'Created',
-    [PurchaseOrderStatus.ElectronicallyReadySend]: 'Ready to Send',
-    [PurchaseOrderStatus.Ordered]: 'Ordered',
-    [PurchaseOrderStatus.Preordered]: 'Preordered',
-    [PurchaseOrderStatus.ElectronicallySent]: 'Sent',
-    [PurchaseOrderStatus.ReceivingStarted]: 'Receiving Started',
-    [PurchaseOrderStatus.Done]: 'Done'
-};
-
-const ReceivingStatusLabels = {
-    [ReceivingStatus.Canceled]: 'Canceled',
-    [ReceivingStatus.Created]: 'Created',
-    [ReceivingStatus.PartialReceived]: 'Partially Received',
-    [ReceivingStatus.FullReceived]: 'Fully Received',
-    [ReceivingStatus.Paid]: 'Paid'
-};
-
-// Helper functions
-function getPurchaseOrderStatusLabel(status) {
-    return PurchaseOrderStatusLabels[status] || 'Unknown';
-}
-
-function getReceivingStatusLabel(status) {
-    return ReceivingStatusLabels[status] || 'Unknown';
-}
-
-// Status checks
-function isReceivingComplete(status) {
-    return status >= ReceivingStatus.PartialReceived;
-}
-
-function isPurchaseOrderComplete(status) {
-    return status === PurchaseOrderStatus.Done;
-}
-
-function isPurchaseOrderCanceled(status) {
-    return status === PurchaseOrderStatus.Canceled;
-}
-
-function isReceivingCanceled(status) {
-    return status === ReceivingStatus.Canceled;
-}
-
-module.exports = {
-    PurchaseOrderStatus,
-    ReceivingStatus,
-    PurchaseOrderStatusLabels,
-    ReceivingStatusLabels,
-    getPurchaseOrderStatusLabel,
-    getReceivingStatusLabel,
-    isReceivingComplete,
-    isPurchaseOrderComplete,
-    isPurchaseOrderCanceled,
-    isReceivingCanceled
-}; 

inventory-server/src/utils/csvImporter.js

@@ -1,63 +0,0 @@
-const fs = require('fs');
-const { parse } = require('csv-parse');
-const { v4: uuidv4 } = require('uuid');
-
-async function importProductsFromCSV(filePath, pool) {
-  return new Promise((resolve, reject) => {
-    const products = [];
-    
-    fs.createReadStream(filePath)
-      .pipe(parse({
-        columns: true,
-        skip_empty_lines: true
-      }))
-      .on('data', async (row) => {
-        products.push({
-          id: uuidv4(),
-          sku: row.sku,
-          name: row.name,
-          description: row.description || null,
-          category: row.category || null
-        });
-      })
-      .on('end', async () => {
-        try {
-          const connection = await pool.getConnection();
-          
-          try {
-            await connection.beginTransaction();
-            
-            for (const product of products) {
-              await connection.query(
-                'INSERT INTO products (id, sku, name, description, category) VALUES (?, ?, ?, ?, ?)',
-                [product.id, product.sku, product.name, product.description, product.category]
-              );
-              
-              // Initialize inventory level for the product
-              await connection.query(
-                'INSERT INTO inventory_levels (id, product_id, quantity) VALUES (?, ?, 0)',
-                [uuidv4(), product.id]
-              );
-            }
-            
-            await connection.commit();
-            resolve({ imported: products.length });
-          } catch (error) {
-            await connection.rollback();
-            reject(error);
-          } finally {
-            connection.release();
-          }
-        } catch (error) {
-          reject(error);
-        }
-      })
-      .on('error', (error) => {
-        reject(error);
-      });
-  });
-}
-
-module.exports = {
-  importProductsFromCSV
-}; 

inventory-server/src/utils/db.js

@@ -1,21 +0,0 @@
-const { Pool } = require('pg');
-
-let pool;
-
-function initPool(config) {
-  pool = new Pool(config);
-  return pool;
-}
-
-async function getConnection() {
-  if (!pool) {
-    throw new Error('Database pool not initialized');
-  }
-  return pool.connect();
-}
-
-module.exports = {
-  initPool,
-  getConnection,
-  getPool: () => pool
-}; 

inventory/index.html

@@ -2,9 +2,12 @@
 <html lang="en">
   <head>
     <meta charset="UTF-8" />
-    <link rel="icon" type="image/svg+xml" href="/box.svg" />
+    <link rel="icon" type="image/x-icon" href="/cherrybottom.ico" />
+    <link rel="preconnect" href="https://fonts.googleapis.com">
+    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+    <link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Inventory Manager</title>
+    <title>A Cherry On Bottom</title>
   </head>
   <body>
     <div id="root"></div>

inventory/package.json

@@ -6,8 +6,10 @@
   "scripts": {
     "dev": "vite",
     "build": "tsc -b && vite build",
+    "build:deploy": "tsc -b && COPY_BUILD=true DEPLOY_TARGET=netcup DEPLOY_PATH=/var/www/html/inventory/frontend vite build",
     "lint": "eslint .",
-    "preview": "vite preview"
+    "preview": "vite preview",
+    "mount": "../mountremote.command"
   },
   "dependencies": {
     "@dnd-kit/core": "^6.3.1",
@@ -30,12 +32,12 @@
     "@radix-ui/react-scroll-area": "^1.2.2",
     "@radix-ui/react-select": "^2.1.4",
     "@radix-ui/react-separator": "^1.1.1",
-    "@radix-ui/react-slot": "^1.1.2",
+    "@radix-ui/react-slot": "^1.2.4",
     "@radix-ui/react-switch": "^1.1.2",
     "@radix-ui/react-tabs": "^1.1.2",
     "@radix-ui/react-toast": "^1.2.6",
-    "@radix-ui/react-toggle": "^1.1.1",
-    "@radix-ui/react-toggle-group": "^1.1.1",
+    "@radix-ui/react-toggle": "^1.1.10",
+    "@radix-ui/react-toggle-group": "^1.1.11",
     "@radix-ui/react-tooltip": "^1.1.6",
     "@shadcn/ui": "^0.0.4",
     "@tabler/icons-react": "^3.28.1",
@@ -53,16 +55,21 @@
     "cmdk": "^1.0.0",
     "date-fns": "^3.6.0",
     "diff": "^7.0.0",
+    "embla-carousel-react": "^8.6.0",
     "framer-motion": "^12.4.4",
+    "immer": "^11.1.3",
+    "input-otp": "^1.4.1",
     "js-levenshtein": "^1.1.6",
     "lodash": "^4.17.21",
     "lucide-react": "^0.469.0",
+    "luxon": "^3.5.0",
     "motion": "^11.18.0",
     "next-themes": "^0.4.4",
     "react": "^18.3.1",
     "react-chartjs-2": "^5.3.0",
     "react-data-grid": "^7.0.0-beta.13",
     "react-day-picker": "^8.10.1",
+    "react-debounce-input": "^3.3.0",
     "react-dom": "^18.3.1",
     "react-dropzone": "^14.3.5",
     "react-hook-form": "^7.54.2",
@@ -76,12 +83,14 @@
     "uuid": "^11.0.5",
     "vaul": "^1.1.2",
     "xlsx": "^0.18.5",
-    "zod": "^3.24.2"
+    "zod": "^3.24.2",
+    "zustand": "^5.0.2"
   },
   "devDependencies": {
     "@eslint/js": "^9.17.0",
     "@types/fs-extra": "^11.0.4",
     "@types/lodash": "^4.17.15",
+    "@types/luxon": "^3.7.1",
     "@types/node": "^22.10.5",
     "@types/react": "^18.3.18",
     "@types/react-dom": "^18.3.5",

inventory/public/box.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7l8.7 5l8.7-5M12 22V12"/></g></svg>

inventory/src/App.css

@@ -1,42 +1,3 @@
 #root {
-  max-width: 1800px;
-  margin: 0 auto;
-  padding: 2rem;
-  text-align: center;
-}
-
-.logo {
-  height: 6em;
-  padding: 1.5em;
-  will-change: filter;
-  transition: filter 300ms;
-}
-.logo:hover {
-  filter: drop-shadow(0 0 2em #646cffaa);
-}
-.logo.react:hover {
-  filter: drop-shadow(0 0 2em #61dafbaa);
-}
-
-@keyframes logo-spin {
-  from {
-    transform: rotate(0deg);
-  }
-  to {
-    transform: rotate(360deg);
-  }
-}
-
-@media (prefers-reduced-motion: no-preference) {
-  a:nth-of-type(2) .logo {
-    animation: logo-spin infinite 20s linear;
-  }
-}
-
-.card {
-  padding: 2em;
-}
-
-.read-the-docs {
-  color: #888;
+  font-family: 'Inter', sans-serif;
 }

inventory/src/App.tsx

@@ -1,23 +1,46 @@
 import { Routes, Route, useNavigate, Navigate, useLocation } from 'react-router-dom';
 import { MainLayout } from './components/layout/MainLayout';
 import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
-import { Products } from './pages/Products';
-import { Dashboard } from './pages/Dashboard';
-import { Settings } from './pages/Settings';
-import { Analytics } from './pages/Analytics';
 import { Toaster } from '@/components/ui/sonner';
-import PurchaseOrders from './pages/PurchaseOrders';
-import { Login } from './pages/Login';
-import { useEffect } from 'react';
+import { useEffect, Suspense, lazy } from 'react';
 import config from './config';
 import { RequireAuth } from './components/auth/RequireAuth';
-import Forecasting from "@/pages/Forecasting";
-import { Vendors } from '@/pages/Vendors';
-import { Categories } from '@/pages/Categories';
-import { Import } from '@/pages/Import';
 import { AuthProvider } from './contexts/AuthContext';
 import { Protected } from './components/auth/Protected';
 import { FirstAccessiblePage } from './components/auth/FirstAccessiblePage';
+import { PageLoading } from '@/components/ui/page-loading';
+
+// Always loaded components (Login and Settings stay in main bundle)
+import { Settings } from './pages/Settings';  
+import { Login } from './pages/Login';
+
+// Lazy load the 4 main chunks you wanted:
+
+// 1. Core inventory app - loaded as one chunk when any inventory page is accessed
+const Overview = lazy(() => import('./pages/Overview'));
+const Products = lazy(() => import('./pages/Products').then(module => ({ default: module.Products })));
+const Analytics = lazy(() => import('./pages/Analytics').then(module => ({ default: module.Analytics })));
+const Forecasting = lazy(() => import('./pages/Forecasting'));
+const DiscountSimulator = lazy(() => import('./pages/DiscountSimulator'));
+const HtsLookup = lazy(() => import('./pages/HtsLookup'));
+const Vendors = lazy(() => import('./pages/Vendors'));
+const Categories = lazy(() => import('./pages/Categories'));  
+const Brands = lazy(() => import('./pages/Brands'));
+const PurchaseOrders = lazy(() => import('./pages/PurchaseOrders'));
+const BlackFridayDashboard = lazy(() => import('./pages/BlackFridayDashboard'));
+
+// 2. Dashboard app - separate chunk
+const Dashboard = lazy(() => import('./pages/Dashboard'));
+const SmallDashboard = lazy(() => import('./pages/SmallDashboard'));
+
+// 3. Product import - separate chunk
+const Import = lazy(() => import('./pages/Import').then(module => ({ default: module.Import })));
+
+// Product editor
+const ProductEditor = lazy(() => import('./pages/ProductEditor'));
+
+// 4. Chat archive - separate chunk
+const Chat = lazy(() => import('./pages/Chat').then(module => ({ default: module.Chat })));
 
 const queryClient = new QueryClient();
 
@@ -72,62 +95,142 @@ function App() {
       <AuthProvider>
         <Toaster richColors position="top-center" />
         <Routes>
+          {/* Always loaded routes */}
           <Route path="/login" element={<Login />} />
+          <Route path="/small" element={
+            <Suspense fallback={<PageLoading />}>
+              <SmallDashboard />
+            </Suspense>
+          } />
           <Route element={
             <RequireAuth>
               <MainLayout />
             </RequireAuth>
           }>
-            <Route index element={
-              <Protected page="dashboard" fallback={<FirstAccessiblePage />}>
-                <Dashboard />
-              </Protected>
-            } />
-            <Route path="/" element={
-              <Protected page="dashboard">
-                <Dashboard />
+            {/* Core inventory app routes - will be lazy loaded */}
+            {/* Default route now prioritizes dashboard, then other pages */}
+            <Route index element={<FirstAccessiblePage />} />
+            <Route path="/overview" element={
+              <Protected page="overview">
+                <Suspense fallback={<PageLoading />}>
+                  <Overview />
+                </Suspense>
               </Protected>
             } />
             <Route path="/products" element={
               <Protected page="products">
+                <Suspense fallback={<PageLoading />}>
                   <Products />
-              </Protected>
-            } />
-            <Route path="/import" element={
-              <Protected page="import">
-                <Import />
+                </Suspense>
               </Protected>
             } />
             <Route path="/categories" element={
               <Protected page="categories">
+                <Suspense fallback={<PageLoading />}>
                   <Categories />
+                </Suspense>
               </Protected>
             } />
             <Route path="/vendors" element={
               <Protected page="vendors">
+                <Suspense fallback={<PageLoading />}>
                   <Vendors />
+                </Suspense>
+              </Protected>
+            } />
+            <Route path="/brands" element={
+              <Protected page="brands">
+                <Suspense fallback={<PageLoading />}>
+                  <Brands />
+                </Suspense>
               </Protected>
             } />
             <Route path="/purchase-orders" element={
               <Protected page="purchase_orders">
+                <Suspense fallback={<PageLoading />}>
                   <PurchaseOrders />
+                </Suspense>
               </Protected>
             } />
             <Route path="/analytics" element={
               <Protected page="analytics">
+                <Suspense fallback={<PageLoading />}>
                   <Analytics />
+                </Suspense>
               </Protected>
             } />
+            <Route path="/discount-simulator" element={
+              <Protected page="discount_simulator">
+                <Suspense fallback={<PageLoading />}>
+                  <DiscountSimulator />
+                </Suspense>
+              </Protected>
+            } />
+            <Route path="/hts-lookup" element={
+              <Protected page="hts_lookup">
+                <Suspense fallback={<PageLoading />}>
+                  <HtsLookup />
+                </Suspense>
+              </Protected>
+            } />
+            <Route path="/forecasting" element={
+              <Protected page="forecasting">
+                <Suspense fallback={<PageLoading />}>
+                  <Forecasting />
+                </Suspense>
+              </Protected>
+            } />
+            
+            {/* Always loaded settings */}
             <Route path="/settings" element={
               <Protected page="settings">
                 <Settings />
               </Protected>
             } />
-            <Route path="/forecasting" element={
-              <Protected page="forecasting">
-                <Forecasting />
+            
+            {/* Product editor */}
+            <Route path="/product-editor" element={
+              <Protected page="product_editor">
+                <Suspense fallback={<PageLoading />}>
+                  <ProductEditor />
+                </Suspense>
               </Protected>
             } />
+
+            {/* Product import - separate chunk */}
+            <Route path="/import" element={
+              <Protected page="import">
+                <Suspense fallback={<PageLoading />}>
+                  <Import />
+                </Suspense>
+              </Protected>
+            } />
+            
+            {/* Chat archive - separate chunk */}
+            <Route path="/chat" element={
+              <Protected page="chat">
+                <Suspense fallback={<PageLoading />}>
+                  <Chat />
+                </Suspense>
+              </Protected>
+            } />
+            
+            {/* Dashboard app - separate chunk */}
+            <Route path="/dashboard" element={
+              <Protected page="dashboard">
+                <Suspense fallback={<PageLoading />}>
+                  <Dashboard />
+                </Suspense>
+              </Protected>
+            } />
+            <Route path="/dashboard/black-friday" element={
+              <Protected page="black_friday_dashboard">
+                <Suspense fallback={<PageLoading />}>
+                  <BlackFridayDashboard />
+                </Suspense>
+              </Protected>
+            } />
+            
             <Route path="*" element={<Navigate to="/" replace />} />
           </Route>
         </Routes>
@@ -137,4 +240,3 @@ function App() {
 }
 
 export default App;
-

inventory/src/components/analytics/CategoryPerformance.tsx

@@ -38,21 +38,22 @@ export function CategoryPerformance() {
       const rawData = await response.json();
       return {
         performance: rawData.performance.map((item: any) => ({
-          ...item,
-          categoryPath: item.categoryPath || item.category,
+          category: item.category || '',
+          categoryPath: item.categoryPath || item.categorypath || item.category || '',
           revenue: Number(item.revenue) || 0,
           profit: Number(item.profit) || 0,
           growth: Number(item.growth) || 0,
-          productCount: Number(item.productCount) || 0
+          productCount: Number(item.productCount) || Number(item.productcount) || 0
         })),
         distribution: rawData.distribution.map((item: any) => ({
-          ...item,
-          categoryPath: item.categoryPath || item.category,
+          category: item.category || '',
+          categoryPath: item.categoryPath || item.categorypath || item.category || '',
           value: Number(item.value) || 0
         })),
         trends: rawData.trends.map((item: any) => ({
-          ...item,
-          categoryPath: item.categoryPath || item.category,
+          category: item.category || '',
+          categoryPath: item.categoryPath || item.categorypath || item.category || '',
+          month: item.month || '',
           sales: Number(item.sales) || 0
         }))
       };

inventory/src/components/analytics/PriceAnalysis.tsx

@@ -25,41 +25,91 @@ interface PriceData {
 }
 
 export function PriceAnalysis() {
-  const { data, isLoading } = useQuery<PriceData>({
+  const { data, isLoading, error } = useQuery<PriceData>({
     queryKey: ['price-analysis'],
     queryFn: async () => {
+      try {
         const response = await fetch(`${config.apiUrl}/analytics/pricing`);
         if (!response.ok) {
-        throw new Error('Failed to fetch price analysis');
+          throw new Error(`Failed to fetch: ${response.status}`);
         }
         const rawData = await response.json();
+
+        if (!rawData || !rawData.pricePoints) {
           return {
-        pricePoints: rawData.pricePoints.map((item: any) => ({
-          ...item,
+            pricePoints: [],
+            elasticity: [],
+            recommendations: []
+          };
+        }
+        
+        return {
+          pricePoints: (rawData.pricePoints || []).map((item: any) => ({
             price: Number(item.price) || 0,
-          salesVolume: Number(item.salesVolume) || 0,
-          revenue: Number(item.revenue) || 0
+            salesVolume: Number(item.salesVolume || item.salesvolume) || 0,
+            revenue: Number(item.revenue) || 0,
+            category: item.category || ''
           })),
-        elasticity: rawData.elasticity.map((item: any) => ({
-          ...item,
+          elasticity: (rawData.elasticity || []).map((item: any) => ({
+            date: item.date || '',
             price: Number(item.price) || 0,
             demand: Number(item.demand) || 0
           })),
-        recommendations: rawData.recommendations.map((item: any) => ({
-          ...item,
-          currentPrice: Number(item.currentPrice) || 0,
-          recommendedPrice: Number(item.recommendedPrice) || 0,
-          potentialRevenue: Number(item.potentialRevenue) || 0,
+          recommendations: (rawData.recommendations || []).map((item: any) => ({
+            product: item.product || '',
+            currentPrice: Number(item.currentPrice || item.currentprice) || 0,
+            recommendedPrice: Number(item.recommendedPrice || item.recommendedprice) || 0,
+            potentialRevenue: Number(item.potentialRevenue || item.potentialrevenue) || 0,
             confidence: Number(item.confidence) || 0
           }))
         };
+      } catch (err) {
+        console.error('Error fetching price data:', err);
+        throw err;
+      }
     },
+    retry: 1
   });
 
-  if (isLoading || !data) {
+  if (isLoading) {
     return <div>Loading price analysis...</div>;
   }
 
+  if (error || !data) {
+    return (
+      <Card className="mb-4">
+        <CardHeader>
+          <CardTitle>Price Analysis</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-red-500">
+            Unable to load price analysis. The price metrics may need to be set up in the database.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  // Early return if no data to display
+  if (
+    data.pricePoints.length === 0 && 
+    data.elasticity.length === 0 && 
+    data.recommendations.length === 0
+  ) {
+    return (
+      <Card className="mb-4">
+        <CardHeader>
+          <CardTitle>Price Analysis</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-muted-foreground">
+            No price data available. This may be because the price metrics haven't been calculated yet.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
   return (
     <div className="grid gap-4">
       <div className="grid gap-4 md:grid-cols-2">

inventory/src/components/analytics/ProfitAnalysis.tsx

@@ -38,22 +38,23 @@ export function ProfitAnalysis() {
       const rawData = await response.json();
       return {
         byCategory: rawData.byCategory.map((item: any) => ({
-          ...item,
-          categoryPath: item.categoryPath || item.category,
-          profitMargin: Number(item.profitMargin) || 0,
+          category: item.category || '',
+          categoryPath: item.categorypath || item.category || '',
+          profitMargin: item.profitmargin !== null ? Number(item.profitmargin) : 0,
           revenue: Number(item.revenue) || 0,
           cost: Number(item.cost) || 0
         })),
         overTime: rawData.overTime.map((item: any) => ({
-          ...item,
-          profitMargin: Number(item.profitMargin) || 0,
+          date: item.date || '',
+          profitMargin: item.profitmargin !== null ? Number(item.profitmargin) : 0,
           revenue: Number(item.revenue) || 0,
           cost: Number(item.cost) || 0
         })),
         topProducts: rawData.topProducts.map((item: any) => ({
-          ...item,
-          categoryPath: item.categoryPath || item.category,
-          profitMargin: Number(item.profitMargin) || 0,
+          product: item.product || '',
+          category: item.category || '',
+          categoryPath: item.categorypath || item.category || '',
+          profitMargin: item.profitmargin !== null ? Number(item.profitmargin) : 0,
           revenue: Number(item.revenue) || 0,
           cost: Number(item.cost) || 0
         }))

inventory/src/components/analytics/StockAnalysis.tsx

@@ -28,42 +28,93 @@ interface StockData {
 }
 
 export function StockAnalysis() {
-  const { data, isLoading } = useQuery<StockData>({
+  const { data, isLoading, error } = useQuery<StockData>({
     queryKey: ['stock-analysis'],
     queryFn: async () => {
+      try {
         const response = await fetch(`${config.apiUrl}/analytics/stock`);
         if (!response.ok) {
-        throw new Error('Failed to fetch stock analysis');
+          throw new Error(`Failed to fetch: ${response.status}`);
         }
         const rawData = await response.json();
+        
+        if (!rawData || !rawData.turnoverByCategory) {
           return {
-        turnoverByCategory: rawData.turnoverByCategory.map((item: any) => ({
-          ...item,
-          turnoverRate: Number(item.turnoverRate) || 0,
-          averageStock: Number(item.averageStock) || 0,
-          totalSales: Number(item.totalSales) || 0
+            turnoverByCategory: [],
+            stockLevels: [],
+            criticalItems: []
+          };
+        }
+        
+        return {
+          turnoverByCategory: (rawData.turnoverByCategory || []).map((item: any) => ({
+            category: item.category || '',
+            turnoverRate: Number(item.turnoverRate || item.turnoverrate) || 0,
+            averageStock: Number(item.averageStock || item.averagestock) || 0,
+            totalSales: Number(item.totalSales || item.totalsales) || 0
           })),
-        stockLevels: rawData.stockLevels.map((item: any) => ({
-          ...item,
-          inStock: Number(item.inStock) || 0,
-          lowStock: Number(item.lowStock) || 0,
-          outOfStock: Number(item.outOfStock) || 0
+          stockLevels: (rawData.stockLevels || []).map((item: any) => ({
+            date: item.date || '',
+            inStock: Number(item.inStock || item.instock) || 0,
+            lowStock: Number(item.lowStock || item.lowstock) || 0,
+            outOfStock: Number(item.outOfStock || item.outofstock) || 0
           })),
-        criticalItems: rawData.criticalItems.map((item: any) => ({
-          ...item,
-          stockQuantity: Number(item.stockQuantity) || 0,
-          reorderPoint: Number(item.reorderPoint) || 0,
-          turnoverRate: Number(item.turnoverRate) || 0,
-          daysUntilStockout: Number(item.daysUntilStockout) || 0
+          criticalItems: (rawData.criticalItems || []).map((item: any) => ({
+            product: item.product || '',
+            sku: item.sku || '',
+            stockQuantity: Number(item.stockQuantity || item.stockquantity) || 0,
+            reorderPoint: Number(item.reorderPoint || item.reorderpoint) || 0,
+            turnoverRate: Number(item.turnoverRate || item.turnoverrate) || 0,
+            daysUntilStockout: Number(item.daysUntilStockout || item.daysuntilstockout) || 0
           }))
         };
+      } catch (err) {
+        console.error('Error fetching stock data:', err);
+        throw err;
+      }
     },
+    retry: 1
   });
 
-  if (isLoading || !data) {
+  if (isLoading) {
     return <div>Loading stock analysis...</div>;
   }
 
+  if (error || !data) {
+    return (
+      <Card className="mb-4">
+        <CardHeader>
+          <CardTitle>Stock Analysis</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-red-500">
+            Unable to load stock analysis. The stock metrics may need to be set up in the database.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  // Early return if no data to display
+  if (
+    data.turnoverByCategory.length === 0 && 
+    data.stockLevels.length === 0 && 
+    data.criticalItems.length === 0
+  ) {
+    return (
+      <Card className="mb-4">
+        <CardHeader>
+          <CardTitle>Stock Analysis</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-muted-foreground">
+            No stock data available. This may be because the stock metrics haven't been calculated yet.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
   const getStockStatus = (daysUntilStockout: number) => {
     if (daysUntilStockout <= 7) {
       return <Badge variant="destructive">Critical</Badge>;

inventory/src/components/analytics/VendorPerformance.tsx

@@ -1,7 +1,7 @@
-import { useQuery } from '@tanstack/react-query';
 import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
 import { ResponsiveContainer, BarChart, Bar, XAxis, YAxis, Tooltip, ScatterChart, Scatter, ZAxis } from 'recharts';
 import config from '../../config';
+import { useState, useEffect } from 'react';
 
 interface VendorData {
   performance: {
@@ -10,14 +10,15 @@ interface VendorData {
     profitMargin: number;
     stockTurnover: number;
     productCount: number;
+    growth: number;
   }[];
-  comparison: {
+  comparison?: {
     vendor: string;
     salesPerProduct: number;
     averageMargin: number;
     size: number;
   }[];
-  trends: {
+  trends?: {
     vendor: string;
     month: string;
     sales: number;
@@ -25,40 +26,86 @@ interface VendorData {
 }
 
 export function VendorPerformance() {
-  const { data, isLoading } = useQuery<VendorData>({
-    queryKey: ['vendor-performance'],
-    queryFn: async () => {
-      const response = await fetch(`${config.apiUrl}/analytics/vendors`);
-      if (!response.ok) {
-        throw new Error('Failed to fetch vendor performance');
+  const [vendorData, setVendorData] = useState<VendorData | null>(null);
+  const [isLoading, setIsLoading] = useState(true);
+  const [error, setError] = useState<string | null>(null);
+
+  useEffect(() => {
+    // Use plain fetch to bypass cache issues with React Query
+    const fetchData = async () => {
+      try {
+        setIsLoading(true);
+        
+        // Add cache-busting parameter
+        const response = await fetch(`${config.apiUrl}/analytics/vendors?nocache=${Date.now()}`, {
+          headers: {
+            "Cache-Control": "no-cache, no-store, must-revalidate",
+            "Pragma": "no-cache",
+            "Expires": "0"
           }
-      const rawData = await response.json();
-      return {
-        performance: rawData.performance.map((vendor: any) => ({
-          ...vendor,
-          salesVolume: Number(vendor.salesVolume) || 0,
-          profitMargin: Number(vendor.profitMargin) || 0,
-          stockTurnover: Number(vendor.stockTurnover) || 0,
-          productCount: Number(vendor.productCount) || 0
-        })),
-        comparison: rawData.comparison.map((vendor: any) => ({
-          ...vendor,
-          salesPerProduct: Number(vendor.salesPerProduct) || 0,
-          averageMargin: Number(vendor.averageMargin) || 0,
-          size: Number(vendor.size) || 0
-        })),
-        trends: rawData.trends.map((vendor: any) => ({
-          ...vendor,
-          sales: Number(vendor.sales) || 0
-        }))
-      };
-    },
         });
         
-  if (isLoading || !data) {
+        if (!response.ok) {
+          throw new Error(`Failed to fetch: ${response.status}`);
+        }
+        
+        const rawData = await response.json();
+        
+        if (!rawData || !rawData.performance) {
+          throw new Error('Invalid response format');
+        }
+        
+        // Create a complete structure even if some parts are missing
+        const data: VendorData = {
+          performance: rawData.performance.map((vendor: any) => ({
+            vendor: vendor.vendor || '',
+            salesVolume: vendor.salesVolume !== null ? Number(vendor.salesVolume) : 0,
+            profitMargin: vendor.profitMargin !== null ? Number(vendor.profitMargin) : 0,
+            stockTurnover: vendor.stockTurnover !== null ? Number(vendor.stockTurnover) : 0,
+            productCount: Number(vendor.productCount) || 0,
+            growth: vendor.growth !== null ? Number(vendor.growth) : 0
+          })),
+          comparison: rawData.comparison?.map((vendor: any) => ({
+            vendor: vendor.vendor || '',
+            salesPerProduct: vendor.salesPerProduct !== null ? Number(vendor.salesPerProduct) : 0,
+            averageMargin: vendor.averageMargin !== null ? Number(vendor.averageMargin) : 0,
+            size: Number(vendor.size) || 0
+          })) || [],
+          trends: rawData.trends?.map((vendor: any) => ({
+            vendor: vendor.vendor || '',
+            month: vendor.month || '',
+            sales: Number(vendor.sales) || 0
+          })) || []
+        };
+        
+        setVendorData(data);
+      } catch (err) {
+        console.error('Error fetching vendor data:', err);
+        setError(err instanceof Error ? err.message : 'Unknown error');
+      } finally {
+        setIsLoading(false);
+      }
+    };
+    
+    fetchData();
+  }, []);
+
+  if (isLoading) {
     return <div>Loading vendor performance...</div>;
   }
 
+  if (error || !vendorData) {
+    return <div className="text-red-500">Error loading vendor data: {error}</div>;
+  }
+
+  // Ensure we have at least the performance data
+  const sortedPerformance = vendorData.performance
+    .sort((a, b) => b.salesVolume - a.salesVolume)
+    .slice(0, 10);
+
+  // Use simplified version if comparison data is missing
+  const hasComparisonData = vendorData.comparison && vendorData.comparison.length > 0;
+
   return (
     <div className="grid gap-4">
       <div className="grid gap-4 md:grid-cols-2">
@@ -68,7 +115,7 @@ export function VendorPerformance() {
           </CardHeader>
           <CardContent>
             <ResponsiveContainer width="100%" height={300}>
-              <BarChart data={data.performance}>
+              <BarChart data={sortedPerformance}>
                 <XAxis dataKey="vendor" />
                 <YAxis tickFormatter={(value) => `$${(value / 1000).toFixed(0)}k`} />
                 <Tooltip 
@@ -84,6 +131,7 @@ export function VendorPerformance() {
           </CardContent>
         </Card>
 
+        {hasComparisonData ? (
           <Card>
             <CardHeader>
               <CardTitle>Vendor Performance Matrix</CardTitle>
@@ -114,7 +162,7 @@ export function VendorPerformance() {
                     }}
                   />
                   <Scatter 
-                  data={data.comparison} 
+                    data={vendorData.comparison} 
                     fill="#60a5fa"
                     name="Vendors"
                   />
@@ -122,6 +170,29 @@ export function VendorPerformance() {
               </ResponsiveContainer>
             </CardContent>
           </Card>
+        ) : (
+          <Card>
+            <CardHeader>
+              <CardTitle>Vendor Profit Margins</CardTitle>
+            </CardHeader>
+            <CardContent>
+              <ResponsiveContainer width="100%" height={300}>
+                <BarChart data={sortedPerformance}>
+                  <XAxis dataKey="vendor" />
+                  <YAxis tickFormatter={(value) => `${value}%`} />
+                  <Tooltip 
+                    formatter={(value: number) => [`${value.toFixed(1)}%`, 'Profit Margin']}
+                  />
+                  <Bar 
+                    dataKey="profitMargin" 
+                    fill="#4ade80"
+                    name="Profit Margin"
+                  />
+                </BarChart>
+              </ResponsiveContainer>
+            </CardContent>
+          </Card>
+        )}
       </div>
 
       <Card>
@@ -130,7 +201,7 @@ export function VendorPerformance() {
         </CardHeader>
         <CardContent>
           <div className="space-y-4">
-            {data.performance.map((vendor) => (
+            {sortedPerformance.map((vendor) => (
               <div key={`${vendor.vendor}-${vendor.salesVolume}`} className="flex items-center">
                 <div className="flex-1">
                   <p className="text-sm font-medium">{vendor.vendor}</p>

inventory/src/components/auth/FirstAccessiblePage.tsx

@@ -3,12 +3,18 @@ import { Navigate } from "react-router-dom";
 import { AuthContext } from "@/contexts/AuthContext";
 
 // Define available pages in order of priority
+// Dashboard is first so users with dashboard access default to it
 const PAGES = [
+  { path: "/dashboard", permission: "access:dashboard" },
+  { path: "/dashboard/black-friday", permission: "access:black_friday_dashboard" },
+  { path: "/overview", permission: "access:overview" },
   { path: "/products", permission: "access:products" },
   { path: "/categories", permission: "access:categories" },
   { path: "/vendors", permission: "access:vendors" },
   { path: "/purchase-orders", permission: "access:purchase_orders" },
   { path: "/analytics", permission: "access:analytics" },
+  { path: "/discount-simulator", permission: "access:discount_simulator" },
+  { path: "/hts-lookup", permission: "access:hts_lookup" },
   { path: "/forecasting", permission: "access:forecasting" },
   { path: "/import", permission: "access:import" },
   { path: "/settings", permission: "access:settings" },
@@ -23,14 +29,12 @@ export function FirstAccessiblePage() {
     return null;
   }
   
-  // Admin users have access to all pages, so this component 
-  // shouldn't be rendering for them (handled by App.tsx)
+  // Admins can access every page, otherwise check explicit permissions
+  const firstAccessiblePage = PAGES.find(page => {
     if (user.is_admin) {
-    return null;
+      return true;
     }
 
-  // Find the first page the user has access to
-  const firstAccessiblePage = PAGES.find(page => {
     return user.permissions?.includes(page.permission);
   });
   

inventory/src/components/auth/PERMISSIONS.md

@@ -1,67 +1,181 @@
 # Permission System Documentation
 
-This document outlines the simplified permission system implemented in the Inventory Manager application.
+This document outlines the permission system implemented in the Inventory Manager application.
 
 ## Permission Structure
 
 Permissions follow this naming convention:
 
 - Page access: `access:{page_name}`
-- Actions: `{action}:{resource}`
+- Settings sections: `settings:{section_name}`
+- Admin features: `admin:{feature}`
 
 Examples:
 - `access:products` - Can access the Products page
-- `create:products` - Can create new products
-- `edit:users` - Can edit user accounts
+- `settings:user_management` - Can access User Management settings
+- `admin:debug` - Can see debug information
 
-## Permission Component
+## Permission Components
 
-### Protected
+### PermissionGuard
 
 The core component that conditionally renders content based on permissions.
 
 ```tsx
-<Protected 
-  permission="create:products"
+<PermissionGuard 
+  permission="settings:user_management"
   fallback={<p>No permission</p>}
 >
-  <button>Create Product</button>
-</Protected>
+  <button>Manage Users</button>
+</PermissionGuard>
 ```
 
 Options:
-- `permission`: Single permission code (e.g., "create:products")
-- `page`: Page name (checks `access:{page}` permission)
-- `resource` + `action`: Resource and action (checks `{action}:{resource}` permission)
+- `permission`: Single permission code
+- `anyPermissions`: Array of permissions (ANY match grants access)
+- `allPermissions`: Array of permissions (ALL required)
 - `adminOnly`: For admin-only sections
+- `page`: Page name (checks `access:{page}` permission)
 - `fallback`: Content to show if permission check fails
 
-### RequireAuth
+### PermissionProtectedRoute
 
-Used for basic authentication checks (is user logged in?).
+Protects entire pages based on page access permissions.
 
 ```tsx
-<Route element={
-  <RequireAuth>
-    <MainLayout />
-  </RequireAuth>
-}>
-  {/* Protected routes */}
-</Route>
+<Route path="/products" element={
+  <PermissionProtectedRoute page="products">
+    <Products />
+  </PermissionProtectedRoute>
+} />
 ```
 
-## Common Permission Codes
+### ProtectedSection
 
+Protects sections within a page based on action permissions.
+
+```tsx
+<ProtectedSection page="products" action="create">
+  <button>Add Product</button>
+</ProtectedSection>
+```
+
+### PermissionButton
+
+Button that automatically handles permissions.
+
+```tsx
+<PermissionButton 
+  page="products" 
+  action="create"
+  onClick={handleCreateProduct}
+>
+  Add Product
+</PermissionButton>
+```
+
+### SettingsSection
+
+Specific component for settings with built-in permission checks.
+
+```tsx
+<SettingsSection
+  title="System Settings"
+  description="Configure global settings"
+  permission="settings:global"
+>
+  {/* Settings content */}
+</SettingsSection>
+```
+
+## Permission Hooks
+
+### usePermissions
+
+Core hook for checking any permission.
+
+```tsx
+const { hasPermission, hasPageAccess, isAdmin } = usePermissions();
+if (hasPermission('settings:user_management')) {
+  // Can access user management
+}
+```
+
+### usePagePermission
+
+Specialized hook for page-level permissions.
+
+```tsx
+const { canView, canCreate, canEdit, canDelete } = usePagePermission('products');
+if (canView()) {
+  // Can view products
+}
+```
+
+## Database Schema
+
+Permissions are stored in the database:
+- `permissions` table: Stores all available permissions
+- `user_permissions` junction table: Maps permissions to users
+
+Admin users automatically have all permissions.
+
+## Implemented Permission Codes
+
+### Page Access Permissions
 | Code | Description |
 |------|-------------|
 | `access:dashboard` | Access to Dashboard page |
+| `access:overview` | Access to Overview page |
 | `access:products` | Access to Products page |
-| `create:products` | Create new products |
-| `edit:products` | Edit existing products |
-| `delete:products` | Delete products |
-| `view:users` | View user accounts |
-| `edit:users` | Edit user accounts |
-| `manage:permissions` | Assign permissions to users |
+| `access:categories` | Access to Categories page |
+| `access:brands` | Access to Brands page |
+| `access:vendors` | Access to Vendors page |
+| `access:purchase_orders` | Access to Purchase Orders page |
+| `access:analytics` | Access to Analytics page |
+| `access:discount_simulator` | Access to Discount Simulator page |
+| `access:hts_lookup` | Access to HTS Lookup page |
+| `access:forecasting` | Access to Forecasting page |
+| `access:import` | Access to Import page |
+| `access:settings` | Access to Settings page |
+| `access:chat` | Access to Chat Archive page |
+
+### Settings Permissions
+| Code | Description |
+|------|-------------|
+| `settings:global` | Access to Global Settings section |
+| `settings:products` | Access to Product Settings section |
+| `settings:vendors` | Access to Vendor Settings section |
+| `settings:data_management` | Access to Data Management settings |
+| `settings:calculation_settings` | Access to Calculation Settings |
+| `settings:library_management` | Access to Image Library Management |
+| `settings:performance_metrics` | Access to Performance Metrics |
+| `settings:prompt_management` | Access to AI Prompt Management |
+| `settings:stock_management` | Access to Stock Management |
+| `settings:templates` | Access to Template Management |
+| `settings:user_management` | Access to User Management |
+
+### Dashboard Component Permissions
+| Code | Description |
+|------|-------------|
+| `dashboard:stats` | Can view statistics cards on the Dashboard |
+| `dashboard:realtime` | Can view realtime analytics on the Dashboard |
+| `dashboard:financial` | Can view financial overview on the Dashboard |
+| `dashboard:feed` | Can view event feed on the Dashboard |
+| `dashboard:sales` | Can view sales chart on the Dashboard |
+| `dashboard:products` | Can view product grid on the Dashboard |
+| `dashboard:campaigns` | Can view Klaviyo campaigns on the Dashboard |
+| `dashboard:analytics` | Can view analytics overview on the Dashboard |
+| `dashboard:user_behavior` | Can view user behavior insights on the Dashboard |
+| `dashboard:meta_campaigns` | Can view Meta campaigns on the Dashboard |
+| `dashboard:typeform` | Can view Typeform metrics on the Dashboard |
+| `dashboard:gorgias` | Can view Gorgias overview on the Dashboard |
+| `dashboard:calls` | Can view Aircall metrics on the Dashboard |
+
+### Admin Permissions
+| Code | Description |
+|------|-------------|
+| `admin:debug` | Can see debug information and features |
 
 ## Implementation Examples
 
@@ -70,35 +184,40 @@ Used for basic authentication checks (is user logged in?).
 In `App.tsx`:
 ```tsx
 <Route path="/products" element={
-  <Protected page="products" fallback={<Navigate to="/" />}>
+  <PermissionProtectedRoute page="products">
     <Products />
-  </Protected>
+  </PermissionProtectedRoute>
 } />
 ```
 
 ### Component Level Protection
 
 ```tsx
-<Protected permission="edit:products">
-  <form>
-    {/* Form fields */}
-    <button type="submit">Save Changes</button>
-  </form>
-</Protected>
+const { hasPermission } = usePermissions();
+
+function handleAction() {
+  if (!hasPermission('settings:user_management')) {
+    toast.error("You don't have permission");
+    return;
+  }
+  // Action logic
+}
 ```
 
-### Button Protection
+### UI Element Protection
 
 ```tsx
-<Button 
-  onClick={handleDelete}
-  disabled={!hasPermission('delete:products')}
->
-  Delete
-</Button>
-
-// With Protected component
-<Protected permission="delete:products" fallback={null}>
-  <Button onClick={handleDelete}>Delete</Button>
-</Protected>
+<PermissionGuard permission="settings:user_management">
+  <button onClick={handleManageUsers}>
+    Manage Users
+  </button>
+</PermissionGuard>
 ```
+
+## Notes
+
+- **Page Access**: These permissions control which pages a user can navigate to
+- **Settings Access**: These permissions control access to different sections within the Settings page
+- **Admin Features**: Special permissions for administrative functions
+- **CRUD Operations**: The application currently focuses on viewing and managing data rather than creating/editing/deleting individual records
+- **User Management**: User CRUD operations are handled through the settings interface rather than dedicated user management pages 

inventory/src/components/chat/ChatRoom.tsx

@@ -0,0 +1,557 @@
+import { useState, useEffect, useRef } from 'react';
+import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
+import { Button } from '@/components/ui/button';
+import { Avatar, AvatarFallback, AvatarImage } from '@/components/ui/avatar';
+import { Loader2, Hash, MessageSquare, ChevronUp, Search, ExternalLink, FileText, Image, Download, MessageCircle, Users2 } from 'lucide-react';
+import { Input } from '@/components/ui/input';
+import config from '@/config';
+import { convertEmojiShortcodes } from '@/utils/emojiUtils';
+
+interface Message {
+  id: number;
+  msg: string;
+  ts: string;
+  u: {
+    _id: string;
+    username: string;
+    name?: string;
+  };
+  _updatedat: string;
+  urls?: any[];
+  mentions?: any[];
+  md?: any[];
+  attachments?: {
+    id: number;
+    mongo_id: string;
+    name: string;
+    size: number;
+    type: string;
+    url: string;
+    path: string;
+    typegroup: string;
+    identify?: {
+      size?: { width: number; height: number };
+      format?: string;
+    };
+  }[];
+}
+
+interface Room {
+  id: number;
+  name: string;
+  fname: string;
+  type: string;
+  msgs: number;
+  last_message_date: string;
+  display_name: string;
+  description?: string;
+  teamid?: string;
+  participants?: { username: string; name: string }[];
+}
+
+interface ChatRoomProps {
+  roomId: string;
+  selectedUserId: string;
+}
+
+export function ChatRoom({ roomId, selectedUserId }: ChatRoomProps) {
+  const [room, setRoom] = useState<Room | null>(null);
+  const [messages, setMessages] = useState<Message[]>([]);
+  const [loading, setLoading] = useState(false);
+  const [loadingMore, setLoadingMore] = useState(false);
+  const [error, setError] = useState<string | null>(null);
+  const [hasMore, setHasMore] = useState(true);
+  const [searchQuery, setSearchQuery] = useState('');
+  const [showSearch, setShowSearch] = useState(false);
+  
+  const messagesEndRef = useRef<HTMLDivElement>(null);
+  const messagesContainerRef = useRef<HTMLDivElement>(null);
+
+  const scrollToBottom = () => {
+    messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
+  };
+
+  const fetchRoom = async () => {
+    try {
+      const response = await fetch(`${config.chatUrl}/rooms/${roomId}?userId=${selectedUserId}`);
+      const data = await response.json();
+      
+      if (data.status === 'success') {
+        setRoom(data.room);
+      } else {
+        throw new Error(data.error || 'Failed to fetch room');
+      }
+    } catch (err) {
+      console.error('Error fetching room:', err);
+      setError(err instanceof Error ? err.message : 'An error occurred');
+    }
+  };
+
+  const fetchMessages = async (before?: string, append = false) => {
+    if (!append) setLoading(true);
+    else setLoadingMore(true);
+    
+    try {
+      const params = new URLSearchParams({
+        limit: '50',
+        offset: append ? messages.length.toString() : '0'
+      });
+      
+      if (before) {
+        params.set('before', before);
+      }
+      
+      const response = await fetch(`${config.chatUrl}/rooms/${roomId}/messages?${params}`);
+      const data = await response.json();
+      
+      if (data.status === 'success') {
+        const newMessages = data.messages;
+        
+        if (append) {
+          // Prepend older messages
+          setMessages(prev => [...newMessages, ...prev]);
+          setHasMore(newMessages.length === 50);
+        } else {
+          setMessages(newMessages);
+          setHasMore(newMessages.length === 50);
+          // Scroll to bottom on initial load
+          setTimeout(scrollToBottom, 100);
+        }
+        
+        // Load attachments for these messages in the background (non-blocking)
+        if (newMessages.length > 0) {
+          loadAttachments(newMessages.map((m: Message) => m.id));
+        }
+      } else {
+        throw new Error(data.error || 'Failed to fetch messages');
+      }
+    } catch (err) {
+      console.error('Error fetching messages:', err);
+      setError(err instanceof Error ? err.message : 'An error occurred');
+    } finally {
+      setLoading(false);
+      setLoadingMore(false);
+    }
+  };
+
+  const loadAttachments = async (messageIds: number[]) => {
+    try {
+      const response = await fetch(`${config.chatUrl}/messages/attachments`, {
+        method: 'POST',
+        headers: {
+          'Content-Type': 'application/json',
+        },
+        body: JSON.stringify({ messageIds }),
+      });
+      
+      const data = await response.json();
+      
+      if (data.status === 'success' && data.attachments) {
+        // Update messages with their attachments
+        setMessages(prevMessages => 
+          prevMessages.map(msg => ({
+            ...msg,
+            attachments: data.attachments[msg.id] || []
+          }))
+        );
+      }
+    } catch (err) {
+      console.error('Error loading attachments:', err);
+      // Don't show error to user for attachments - messages are already displayed
+    }
+  };
+
+  const loadMoreMessages = () => {
+    if (messages.length > 0 && hasMore && !loadingMore) {
+      const oldestMessage = messages[0];
+      fetchMessages(oldestMessage.ts, true);
+    }
+  };
+
+  const searchMessages = async () => {
+    if (!searchQuery || searchQuery.length < 2) return;
+    
+    try {
+      const response = await fetch(
+        `${config.chatUrl}/users/${selectedUserId}/search?q=${encodeURIComponent(searchQuery)}&limit=20`
+      );
+      const data = await response.json();
+      
+      if (data.status === 'success') {
+        // Handle search results
+      }
+    } catch (err) {
+      console.error('Error searching messages:', err);
+    }
+  };
+
+  useEffect(() => {
+    if (roomId && selectedUserId) {
+      setMessages([]);
+      setError(null);
+      setHasMore(true);
+      fetchRoom();
+      fetchMessages();
+    }
+  }, [roomId, selectedUserId]);
+
+  const getRoomIcon = (room: Room) => {
+    switch (room.type) {
+      case 'c':
+        return <Hash className="h-4 w-4 text-blue-500" />;
+      case 'p':
+        // Distinguish between teams and discussions based on teamid
+        if (room.teamid) {
+          return <Users2 className="h-4 w-4 text-purple-500" />; // Teams
+        } else {
+          return <MessageCircle className="h-4 w-4 text-orange-500" />; // Discussions
+        }
+      case 'd':
+        return <MessageSquare className="h-4 w-4 text-green-500" />;
+      default:
+        return <Hash className="h-4 w-4 text-gray-500" />;
+    }
+  };
+
+  const formatTime = (timestamp: string) => {
+    // The stored timestamps are actually in Eastern Time but labeled as UTC
+    // We need to compensate by treating them as UTC and then converting back to Eastern
+    const originalDate = new Date(timestamp);
+    
+    // Subtract 4 hours to compensate for the EDT offset (adjust to 5 hours for EST months)
+    // This assumes the original data was incorrectly stored as UTC when it was actually EDT/EST
+    const isDST = (date: Date) => {
+      const jan = new Date(date.getFullYear(), 0, 1);
+      const jul = new Date(date.getFullYear(), 6, 1);
+      return date.getTimezoneOffset() < Math.max(jan.getTimezoneOffset(), jul.getTimezoneOffset());
+    };
+    
+    // Determine if the timestamp falls in DST period for Eastern Time
+    const offsetHours = isDST(originalDate) ? 4 : 5; // EDT = UTC-4, EST = UTC-5
+    const correctedDate = new Date(originalDate.getTime() - (offsetHours * 60 * 60 * 1000));
+    
+    const now = new Date();
+    const timeZone = 'America/New_York';
+    
+    // Compare dates in Eastern Time to ensure correct "today" detection
+    const dateInET = correctedDate.toLocaleDateString([], { timeZone });
+    const nowInET = now.toLocaleDateString([], { timeZone });
+    const isToday = dateInET === nowInET;
+    
+    if (isToday) {
+      return correctedDate.toLocaleTimeString([], { 
+        hour: '2-digit', 
+        minute: '2-digit',
+        timeZone 
+      });
+    } else {
+      return correctedDate.toLocaleDateString([], { timeZone }) + ' ' + correctedDate.toLocaleTimeString([], { 
+        hour: '2-digit', 
+        minute: '2-digit',
+        timeZone 
+      });
+    }
+  };
+
+  const renderMessageText = (text: string, urls?: any[]) => {
+    if (!text) return '';
+    
+    // First, convert emoji shortcodes to actual emoji
+    let processedText = convertEmojiShortcodes(text);
+    
+    // Then, handle markdown links [text](url) and convert them to HTML
+    processedText = processedText.replace(
+      /\[([^\]]+)\]\((https?:\/\/[^\s\)]+)\)/g,
+      '<a href="$2" target="_blank" rel="noopener noreferrer" class="text-blue-600 hover:underline">$1</a>'
+    );
+    
+    // If we have URL previews, replace standalone URLs (that aren't already in markdown) with just the preview
+    if (urls && urls.length > 0) {
+      urls.forEach((urlData) => {
+        // Only replace standalone URLs that aren't part of markdown links
+        const standaloneUrlRegex = new RegExp(`(?<!\\]\\()${urlData.url.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?!\\))`, 'g');
+        processedText = processedText.replace(standaloneUrlRegex, '');
+      });
+    }
+    
+    return <span dangerouslySetInnerHTML={{ __html: processedText }} />;
+  };
+
+  const renderURLPreviews = (urls: any[]) => {
+    if (!urls || urls.length === 0) return null;
+    
+    return (
+      <div className="mt-2 space-y-2">
+        {urls.map((urlData, index) => (
+          <div key={index} className="border rounded-lg p-3 bg-gray-50">
+            <div className="flex items-start gap-2">
+              <ExternalLink className="h-4 w-4 mt-1 text-blue-500 flex-shrink-0" />
+              <div className="min-w-0 flex-1">
+                <a 
+                  href={urlData.url} 
+                  target="_blank" 
+                  rel="noopener noreferrer"
+                  className="text-blue-600 hover:underline text-sm break-all"
+                >
+                  {urlData.meta?.pageTitle || urlData.url}
+                </a>
+                {urlData.meta?.ogDescription && (
+                  <div className="text-xs text-muted-foreground mt-1">{urlData.meta.ogDescription}</div>
+                )}
+              </div>
+            </div>
+          </div>
+        ))}
+      </div>
+    );
+  };
+
+    const renderAttachments = (attachments: any[]) => {
+    if (!attachments || attachments.length === 0) return null;
+    
+    // Filter out thumbnail attachments (they're usually lower quality versions)
+    const filteredAttachments = attachments.filter(attachment => 
+      !attachment.name?.toLowerCase().startsWith('thumb-')
+    );
+    
+    if (filteredAttachments.length === 0) return null;
+    
+    return (
+      <div className="mt-2 space-y-2">
+        {filteredAttachments.map((attachment, index) => {
+          const isImage = attachment.typegroup === 'image';
+          const filePath = `${config.chatUrl}/files/by-id/${attachment.mongo_id}`;
+          
+          const handleDownload = () => {
+            // Create a temporary anchor element to trigger download
+            const link = document.createElement('a');
+            link.href = filePath;
+            link.download = attachment.name || 'download';
+            link.target = '_blank';
+            document.body.appendChild(link);
+            link.click();
+            document.body.removeChild(link);
+          };
+          
+          return (
+            <div key={index} className="border rounded-lg p-3 bg-gray-50">
+              <div className="flex items-center gap-2">
+                {isImage ? (
+                  <Image className="h-4 w-4 text-green-500" />
+                ) : (
+                  <FileText className="h-4 w-4 text-blue-500" />
+                )}
+                <div className="flex-1 min-w-0">
+                  <div className="font-medium text-sm truncate">{attachment.name}</div>
+                  <div className="text-xs text-muted-foreground">
+                    {(attachment.size / 1024).toFixed(1)} KB
+                    {attachment.identify?.size && (
+                      <span> • {attachment.identify.size.width}×{attachment.identify.size.height}</span>
+                    )}
+                  </div>
+                </div>
+                <Button variant="ghost" size="sm" onClick={handleDownload}>
+                  <Download className="h-4 w-4" />
+                </Button>
+              </div>
+              {isImage && attachment.identify?.size && (
+                <div className="mt-2">
+                  <img 
+                    src={filePath}
+                    alt={attachment.name}
+                    className="max-w-xs max-h-48 rounded border cursor-pointer"
+                    loading="lazy"
+                    onClick={() => window.open(filePath, '_blank')}
+                  />
+                </div>
+              )}
+            </div>
+          );
+        })}
+      </div>
+    );
+  };
+
+  const renderMentions = (text: string, mentions: any[]) => {
+    if (!mentions || mentions.length === 0) return text;
+    
+    // First, convert emoji shortcodes to actual emoji
+    let renderedText = convertEmojiShortcodes(text);
+    
+    // Then process mentions
+    mentions.forEach((mention) => {
+      if (mention.username) {
+        const mentionPattern = new RegExp(`@${mention.username}`, 'g');
+        renderedText = renderedText.replace(
+          mentionPattern, 
+          `<span class="bg-blue-100 text-blue-800 px-1 rounded">@${mention.username}</span>`
+        );
+      }
+    });
+    
+    return <span dangerouslySetInnerHTML={{ __html: renderedText }} />;
+  };
+
+  const renderMessage = (message: Message, index: number) => {
+    const prevMessage = index > 0 ? messages[index - 1] : null;
+    const isConsecutive = prevMessage && 
+      prevMessage.u.username === message.u.username &&
+      new Date(message.ts).getTime() - new Date(prevMessage.ts).getTime() < 300000; // 5 minutes
+
+    return (
+      <div key={message.id} className={`${isConsecutive ? 'mt-1' : 'mt-4'}`}>
+        {!isConsecutive && (
+          <div className="flex items-center gap-2 mb-1">
+            <Avatar className="h-8 w-8">
+              <AvatarImage 
+                src={`${config.chatUrl}/avatar/${message.u._id}`}
+                alt={message.u.name || message.u.username}
+              />
+              <AvatarFallback className="text-sm font-medium">
+                {(message.u.name || message.u.username).charAt(0).toUpperCase()}
+              </AvatarFallback>
+            </Avatar>
+            <span className="font-medium text-sm">
+              {message.u.name || message.u.username}
+            </span>
+            <span className="text-xs text-muted-foreground">
+              {formatTime(message.ts)}
+            </span>
+          </div>
+        )}
+        <div className={`${isConsecutive ? 'ml-10' : 'ml-10'} text-sm`}>
+          <div className="break-words">
+            {message.mentions && message.mentions.length > 0 
+              ? renderMentions(message.msg, message.mentions)
+              : renderMessageText(message.msg, message.urls)
+            }
+          </div>
+          {message.urls && renderURLPreviews(message.urls)}
+          {message.attachments && renderAttachments(message.attachments)}
+        </div>
+      </div>
+    );
+  };
+
+  if (!roomId) {
+    return (
+      <Card className="h-full">
+        <CardContent className="flex items-center justify-center h-full">
+          <p className="text-muted-foreground">Select a room to view messages</p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (loading && messages.length === 0) {
+    return (
+      <Card className="h-full">
+        <CardContent className="flex items-center justify-center h-full">
+          <div className="flex items-center gap-2">
+            <Loader2 className="h-4 w-4 animate-spin" />
+            <span>Loading messages...</span>
+          </div>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (error) {
+    return (
+      <Card className="h-full border-red-200 bg-red-50">
+        <CardContent className="flex items-center justify-center h-full">
+          <p className="text-red-700">{error}</p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  return (
+    <Card className="h-full flex flex-col">
+      <CardHeader className="border-b p-4">
+        <div className="flex items-center justify-between">
+          <div className="flex items-center gap-3">
+            {room && getRoomIcon(room)}
+            <div>
+              <CardTitle className="text-lg">
+                {room?.type === 'd' 
+                  ? `Direct message with ${room?.display_name || 'Unknown User'}`
+                  : room?.display_name || room?.fname || room?.name || 'Unnamed Room'
+                }
+              </CardTitle>
+              {room?.description && room?.type !== 'd' && (
+                <p className="text-sm text-muted-foreground">{room.description}</p>
+              )}
+              {/* Only show participants for non-direct messages since DM names are already in the title */}
+              {room?.participants && room.participants.length > 0 && room?.type !== 'd' && (
+                <p className="text-xs text-muted-foreground">
+                  {room.participants.map(p => p.name || p.username).join(', ')}
+                </p>
+              )}
+            </div>
+          </div>
+          
+          <div className="flex items-center gap-2">
+            <Button
+              variant="outline"
+              size="sm"
+              onClick={() => setShowSearch(!showSearch)}
+            >
+              <Search className="h-4 w-4" />
+            </Button>
+           
+          </div>
+        </div>
+        
+        {showSearch && (
+          <div className="flex gap-2 mt-2">
+            <Input
+              placeholder="Search messages..."
+              value={searchQuery}
+              onChange={(e) => setSearchQuery(e.target.value)}
+              onKeyPress={(e) => e.key === 'Enter' && searchMessages()}
+            />
+            <Button onClick={searchMessages} size="sm">Search</Button>
+          </div>
+        )}
+      </CardHeader>
+      
+      <CardContent className="flex-1 p-0 overflow-hidden">
+        <div 
+          ref={messagesContainerRef}
+          className="h-full overflow-y-auto p-4"
+        >
+          {hasMore && messages.length > 0 && (
+            <div className="text-center mb-4">
+              <Button
+                variant="outline"
+                size="sm"
+                onClick={loadMoreMessages}
+                disabled={loadingMore}
+              >
+                {loadingMore ? (
+                  <Loader2 className="h-4 w-4 animate-spin mr-2" />
+                ) : (
+                  <ChevronUp className="h-4 w-4 mr-2" />
+                )}
+                Load older messages
+              </Button>
+            </div>
+          )}
+          
+          {messages.length === 0 ? (
+            <div className="text-center text-muted-foreground">
+              No messages in this room
+            </div>
+          ) : (
+            <div className="space-y-1">
+              {messages.map((message, index) => renderMessage(message, index))}
+            </div>
+          )}
+          
+          <div ref={messagesEndRef} />
+        </div>
+      </CardContent>
+    </Card>
+  );
+} 

inventory/src/components/chat/ChatTest.tsx

@@ -0,0 +1,177 @@
+import { useState, useEffect } from 'react';
+import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
+import { Badge } from '@/components/ui/badge';
+import { Loader2, Hash, Lock, Users, MessageSquare } from 'lucide-react';
+import config from '@/config';
+
+interface Room {
+  id: number;
+  name: string;
+  fname: string;
+  type: string;
+  msgs: number;
+  last_message_date: string;
+}
+
+interface ChatTestProps {
+  selectedUserId: string;
+}
+
+export function ChatTest({ selectedUserId }: ChatTestProps) {
+  const [rooms, setRooms] = useState<Room[]>([]);
+  const [loading, setLoading] = useState(false);
+  const [error, setError] = useState<string | null>(null);
+
+  useEffect(() => {
+    if (!selectedUserId) {
+      setRooms([]);
+      return;
+    }
+
+    const fetchUserRooms = async () => {
+      setLoading(true);
+      setError(null);
+      
+      try {
+        const response = await fetch(`${config.chatUrl}/users/${selectedUserId}/rooms`);
+        const data = await response.json();
+        
+        if (data.status === 'success') {
+          setRooms(data.rooms);
+        } else {
+          throw new Error(data.error || 'Failed to fetch rooms');
+        }
+      } catch (err) {
+        console.error('Error fetching user rooms:', err);
+        setError(err instanceof Error ? err.message : 'An error occurred');
+      } finally {
+        setLoading(false);
+      }
+    };
+
+    fetchUserRooms();
+  }, [selectedUserId]);
+
+  const getRoomIcon = (roomType: string) => {
+    switch (roomType) {
+      case 'c':
+        return <Hash className="h-4 w-4 text-blue-500" />;
+      case 'p':
+        return <Lock className="h-4 w-4 text-orange-500" />;
+      case 'd':
+        return <MessageSquare className="h-4 w-4 text-green-500" />;
+      default:
+        return <Users className="h-4 w-4 text-gray-500" />;
+    }
+  };
+
+  const getRoomTypeLabel = (roomType: string) => {
+    switch (roomType) {
+      case 'c':
+        return 'Channel';
+      case 'p':
+        return 'Private';
+      case 'd':
+        return 'Direct';
+      default:
+        return 'Unknown';
+    }
+  };
+
+  if (!selectedUserId) {
+    return (
+      <Card>
+        <CardHeader>
+          <CardTitle>Database Connection Test</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-muted-foreground">
+            Select a user from the dropdown above to view their rooms and test the database connection.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (loading) {
+    return (
+      <Card>
+        <CardHeader>
+          <CardTitle>Loading User Rooms...</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <div className="flex items-center gap-2">
+            <Loader2 className="h-4 w-4 animate-spin" />
+            <span>Fetching rooms for selected user...</span>
+          </div>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (error) {
+    return (
+      <Card className="border-red-200 bg-red-50">
+        <CardHeader>
+          <CardTitle className="text-red-800">Error Loading Rooms</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-red-700">{error}</p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  return (
+    <Card>
+      <CardHeader>
+        <CardTitle>User Rooms ({rooms.length})</CardTitle>
+        <p className="text-sm text-muted-foreground">
+          Rooms accessible to the selected user
+        </p>
+      </CardHeader>
+      <CardContent>
+        {rooms.length === 0 ? (
+          <p className="text-muted-foreground">No rooms found for this user.</p>
+        ) : (
+          <div className="space-y-2">
+            {rooms.map((room) => (
+              <div
+                key={room.id}
+                className="flex items-center justify-between p-3 border rounded-lg hover:bg-gray-50"
+              >
+                <div className="flex items-center gap-3">
+                  {getRoomIcon(room.type)}
+                  <div>
+                    <div className="font-medium">
+                      {room.fname || room.name || 'Unnamed Room'}
+                    </div>
+                    <div className="text-sm text-muted-foreground">
+                      {room.name && room.fname !== room.name && (
+                        <span className="font-mono">#{room.name}</span>
+                      )}
+                    </div>
+                  </div>
+                </div>
+                
+                <div className="flex items-center gap-2">
+                  <Badge variant="secondary">
+                    {getRoomTypeLabel(room.type)}
+                  </Badge>
+                  <Badge variant="outline">
+                    {room.msgs} messages
+                  </Badge>
+                  {room.last_message_date && (
+                    <Badge variant="outline" className="text-xs">
+                      {new Date(room.last_message_date).toLocaleDateString()}
+                    </Badge>
+                  )}
+                </div>
+              </div>
+            ))}
+          </div>
+        )}
+      </CardContent>
+    </Card>
+  );
+} 

inventory/src/components/chat/RoomList.tsx

@@ -0,0 +1,330 @@
+import React, { useState, useEffect } from 'react';
+import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
+import { Avatar, AvatarFallback, AvatarImage } from '@/components/ui/avatar';
+import { Loader2, Hash, Users, MessageSquare, MessageCircle, Users2 } from 'lucide-react';
+import config from '@/config';
+
+interface Room {
+  id: number;
+  name: string;
+  fname: string;
+  type: string;
+  msgs: number;
+  last_message_date: string;
+  display_name: string;
+  userscount?: number;
+  description?: string;
+  teamid?: string;
+  archived?: boolean;
+  open?: boolean;
+  participants?: {
+    username: string;
+    name: string;
+    mongo_id: string;
+    avataretag?: string;
+  }[];
+}
+
+interface RoomListProps {
+  selectedUserId: string;
+  selectedRoomId: string | null;
+  onRoomSelect: (roomId: string) => void;
+}
+
+export function RoomList({ selectedUserId, selectedRoomId, onRoomSelect }: RoomListProps) {
+  const [rooms, setRooms] = useState<Room[]>([]);
+  const [filteredRooms, setFilteredRooms] = useState<Room[]>([]);
+  const [loading, setLoading] = useState(false);
+  const [error, setError] = useState<string | null>(null);
+  const [searchFilter] = useState('');
+
+  useEffect(() => {
+    if (!selectedUserId) {
+      setRooms([]);
+      setFilteredRooms([]);
+      return;
+    }
+
+    const fetchUserRooms = async () => {
+      setLoading(true);
+      setError(null);
+      
+      try {
+        const response = await fetch(`${config.chatUrl}/users/${selectedUserId}/rooms`);
+        const data = await response.json();
+        
+        if (data.status === 'success') {
+          setRooms(data.rooms);
+          setFilteredRooms(data.rooms);
+        } else {
+          throw new Error(data.error || 'Failed to fetch rooms');
+        }
+      } catch (err) {
+        console.error('Error fetching user rooms:', err);
+        setError(err instanceof Error ? err.message : 'An error occurred');
+      } finally {
+        setLoading(false);
+      }
+    };
+
+    fetchUserRooms();
+  }, [selectedUserId]);
+
+  useEffect(() => {
+    if (!searchFilter) {
+      setFilteredRooms(rooms);
+    } else {
+      const filtered = rooms.filter(room => 
+        (room.display_name?.toLowerCase() || '').includes(searchFilter.toLowerCase()) ||
+        (room.name?.toLowerCase() || '').includes(searchFilter.toLowerCase()) ||
+        (room.fname?.toLowerCase() || '').includes(searchFilter.toLowerCase())
+      );
+      setFilteredRooms(filtered);
+    }
+  }, [searchFilter, rooms]);
+
+  const groupRoomsByType = (rooms: Room[]) => {
+    const teams: Room[] = [];
+    const discussions: Room[] = [];
+    const channels: Room[] = [];
+    const directMessages: Room[] = [];
+
+    rooms.forEach(room => {
+      switch (room.type) {
+        case 'p':
+          if (room.teamid) {
+            teams.push(room);
+          } else {
+            discussions.push(room);
+          }
+          break;
+        case 'c':
+          channels.push(room);
+          break;
+        case 'd':
+          directMessages.push(room);
+          break;
+        default:
+          channels.push(room); // fallback for unknown types
+      }
+    });
+
+    // Sort each group by message count descending
+    const sortByMessages = (a: Room, b: Room) => (b.msgs || 0) - (a.msgs || 0);
+    
+    teams.sort(sortByMessages);
+    discussions.sort(sortByMessages);
+    channels.sort(sortByMessages);
+    directMessages.sort(sortByMessages);
+
+    return { teams, discussions, channels, directMessages };
+  };
+
+  const renderRoomIcon = (room: Room) => {
+    // For direct messages, show participant avatars
+    if (room.type === 'd' && room.participants && room.participants.length > 0) {
+      if (room.participants.length === 1) {
+        // Single participant - show their avatar
+        const participant = room.participants[0];
+        return (
+          <Avatar className="h-10 w-10">
+            <AvatarImage 
+              src={`${config.chatUrl}/avatar/${participant.mongo_id}`}
+              alt={participant.name || participant.username}
+            />
+            <AvatarFallback className="text-lg">
+              {(participant.name || participant.username).charAt(0).toUpperCase()}
+            </AvatarFallback>
+          </Avatar>
+        );
+      } else {
+        // Multiple participants - show overlapping avatars
+        return (
+          <div className="relative flex items-center h-10 w-10">
+            {room.participants.slice(0, 3).map((participant, index) => (  
+              <Avatar 
+                key={participant.mongo_id} 
+                className={`h-8 w-8 border-2 border-white ${index > 0 ? '-ml-4' : ''}`}
+                style={{ zIndex: 30 - index }}
+              >
+                <AvatarImage 
+                  src={`${config.chatUrl}/avatar/${participant.mongo_id}`}
+                  alt={participant.name || participant.username}
+                />
+                <AvatarFallback className="text-lg">
+                  {(participant.name || participant.username).charAt(0).toUpperCase()}
+                </AvatarFallback>
+              </Avatar>
+            ))}
+          </div>
+        );
+      }
+    }
+
+    // For other room types, use icons
+    switch (room.type) {
+      case 'c':
+        return (
+          <div className="h-10 w-10 bg-blue-50 rounded-full flex items-center justify-center">
+            <Hash className="h-6 w-6 text-blue-500" />
+          </div>
+        );
+      case 'p':
+        // Distinguish between teams and discussions based on teamid
+        if (room.teamid) {
+          return (
+            <div className="h-10 w-10 bg-purple-50 rounded-full flex items-center justify-center">
+              <Users2 className="h-6 w-6 text-purple-500" />
+            </div>
+          ); // Teams
+        } else {
+          return (
+            <div className="h-10 w-10 bg-orange-50 rounded-full flex items-center justify-center">
+              <MessageCircle className="h-6 w-6 text-orange-500" />
+            </div>
+          ); // Discussions
+        }
+      case 'd':
+        return (
+          <div className="h-12 w-12 bg-green-50 rounded-full flex items-center justify-center">
+            <MessageSquare className="h-6 w-6 text-green-500" />
+          </div>
+        );
+      default:
+        return (
+          <div className="h-12 w-12 bg-gray-50 rounded-full flex items-center justify-center">
+            <Users className="h-6 w-6 text-gray-500" />
+          </div>
+        );
+    }
+  };
+
+  if (!selectedUserId) {
+    return (
+      <Card className="h-full">
+        <CardHeader>
+          <CardTitle>Rooms</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-muted-foreground text-sm">
+            Select a user from the dropdown above to view their rooms.
+          </p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (loading) {
+    return (
+      <Card className="h-full">
+        <CardHeader>
+          <CardTitle>Loading Rooms...</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <div className="flex items-center gap-2">
+            <Loader2 className="h-4 w-4 animate-spin" />
+            <span className="text-sm">Fetching rooms for selected user...</span>
+          </div>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  if (error) {
+    return (
+      <Card className="h-full border-red-200 bg-red-50">
+        <CardHeader>
+          <CardTitle className="text-red-800">Error Loading Rooms</CardTitle>
+        </CardHeader>
+        <CardContent>
+          <p className="text-red-700 text-sm">{error}</p>
+        </CardContent>
+      </Card>
+    );
+  }
+
+  return (
+    <Card className="h-full flex flex-col">
+      <CardHeader className="pb-3">
+        <CardTitle className="text-lg">Rooms</CardTitle>
+      </CardHeader>
+      
+      <CardContent className="flex-1 p-0 overflow-hidden">
+        <div className="h-full overflow-y-auto">
+          {filteredRooms.length === 0 ? (
+            <div className="p-4">
+              <p className="text-muted-foreground text-sm">
+                {searchFilter ? 'No rooms match your search.' : 'No rooms found for this user.'}
+              </p>
+            </div>
+          ) : (
+            <div className="space-y-3 px-2 pb-4">
+              {(() => {
+                const { teams, discussions, channels, directMessages } = groupRoomsByType(filteredRooms);
+                
+                const renderRoomGroup = (title: string, rooms: Room[], icon: React.ReactNode) => {
+                  if (rooms.length === 0) return null;
+                  
+                  return (
+                    <div key={title} className="space-y-0.5">
+                      <div className="flex items-center gap-2 px-2 py-1 border-b border-gray-100">
+                        {icon}
+                        <h3 className="text-xs font-semibold text-muted-foreground uppercase tracking-wide">
+                          {title} ({rooms.length})
+                        </h3>
+                      </div>
+                      <div className="space-y-0.5">
+                        {rooms.map((room) => (
+                          <div
+                            key={room.id}
+                            onClick={() => onRoomSelect(room.id.toString())}
+                            className={`
+                              flex items-center justify-between py-0.5 px-3 rounded-lg cursor-pointer transition-colors
+                              hover:bg-gray-100 
+                              ${selectedRoomId === room.id.toString() ? 'bg-blue-50 border-l-4 border-blue-500' : ''}
+                              ${(room.open === false || room.archived === true) ? 'opacity-60' : ''}
+                            `}
+                          >
+                                                        <div className="grid grid-cols-4 items-center gap-2 min-w-0 flex-1">
+                              {renderRoomIcon(room)}
+                              <div className="min-w-0 flex-1 col-span-3">
+                                <div className="flex items-center justify-between">
+                                  <div className={`font-medium text-sm truncate ${(room.open === false || room.archived === true) ? 'text-muted-foreground' : ''}`}>
+                                    {room.display_name || room.fname || room.name || 'Unnamed Room'}
+                                  </div>
+                                  {room.msgs > 0 && (
+                                    <span className="text-xs text-muted-foreground ml-2 flex-shrink-0">
+                                      {room.msgs} msgs
+                                    </span>
+                                  )}
+                                </div>
+                                {room.description && (
+                                  <div className="text-xs text-muted-foreground truncate">
+                                    {room.description}
+                                  </div>
+                                )}
+                              </div>
+                            </div>
+                          </div>
+                        ))}
+                      </div>
+                    </div>
+                  );
+                };
+
+                return (
+                  <>
+                    {renderRoomGroup('Teams', teams, <Users2 className="h-3 w-3 text-purple-500" />)}
+                    {renderRoomGroup('Discussions', discussions, <MessageCircle className="h-3 w-3 text-orange-500" />)}
+                    {renderRoomGroup('Channels', channels, <Hash className="h-3 w-3 text-blue-500" />)}
+                    {renderRoomGroup('Direct Messages', directMessages, <MessageSquare className="h-3 w-3 text-green-500" />)}
+                  </>
+                );
+              })()}
+            </div>
+          )}
+        </div>
+      </CardContent>
+    </Card>
+  );
+} 

inventory/src/components/chat/SearchResults.tsx

@@ -0,0 +1,116 @@
+import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
+import { Badge } from '@/components/ui/badge';
+import { Button } from '@/components/ui/button';
+import { Hash, Lock, MessageSquare, X } from 'lucide-react';
+import { convertEmojiShortcodes } from '@/utils/emojiUtils';
+
+interface SearchResult {
+  id: number;
+  msg: string;
+  ts: string;
+  u: {
+    username: string;
+    name?: string;
+  };
+  room_id: number;
+  room_name: string;
+  room_fname: string;
+  room_type: string;
+}
+
+interface SearchResultsProps {
+  results: SearchResult[];
+  query: string;
+  onClose: () => void;
+  onRoomSelect: (roomId: string) => void;
+}
+
+export function SearchResults({ results, query, onClose, onRoomSelect }: SearchResultsProps) {
+  const getRoomIcon = (roomType: string) => {
+    switch (roomType) {
+      case 'c':
+        return <Hash className="h-3 w-3 text-blue-500" />;
+      case 'p':
+        return <Lock className="h-3 w-3 text-orange-500" />;
+      case 'd':
+        return <MessageSquare className="h-3 w-3 text-green-500" />;
+      default:
+        return <Hash className="h-3 w-3 text-gray-500" />;
+    }
+  };
+
+  const highlightText = (text: string, query: string) => {
+    if (!query) return convertEmojiShortcodes(text);
+    
+    // First convert emoji shortcodes
+    const textWithEmoji = convertEmojiShortcodes(text);
+    
+    const regex = new RegExp(`(${query})`, 'gi');
+    const parts = textWithEmoji.split(regex);
+    
+    return parts.map((part, index) => 
+      regex.test(part) ? (
+        <span key={index} className="bg-yellow-200 font-medium">
+          {part}
+        </span>
+      ) : (
+        part
+      )
+    );
+  };
+
+  const formatTime = (timestamp: string) => {
+    const date = new Date(timestamp);
+    return date.toLocaleDateString() + ' ' + date.toLocaleTimeString([], { 
+      hour: '2-digit', 
+      minute: '2-digit' 
+    });
+  };
+
+  return (
+    <Card className="absolute top-full left-0 right-0 z-10 mt-2 max-h-96 overflow-y-auto">
+      <CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
+        <CardTitle className="text-sm">
+          Search Results for "{query}" ({results.length})
+        </CardTitle>
+        <Button variant="ghost" size="sm" onClick={onClose}>
+          <X className="h-4 w-4" />
+        </Button>
+      </CardHeader>
+      <CardContent className="pt-0">
+        {results.length === 0 ? (
+          <p className="text-sm text-muted-foreground">No messages found matching your search.</p>
+        ) : (
+          <div className="space-y-3">
+            {results.map((result) => (
+              <div 
+                key={result.id}
+                className="border rounded-lg p-3 hover:bg-gray-50 cursor-pointer"
+                onClick={() => {
+                  onRoomSelect(result.room_id.toString());
+                  onClose();
+                }}
+              >
+                <div className="flex items-center gap-2 mb-2">
+                  {getRoomIcon(result.room_type)}
+                  <span className="text-sm font-medium">
+                    {result.room_fname || result.room_name || 'Unnamed Room'}
+                  </span>
+                  <Badge variant="outline" className="text-xs">
+                    {result.u.name || result.u.username}
+                  </Badge>
+                  <span className="text-xs text-muted-foreground ml-auto">
+                    {formatTime(result.ts)}
+                  </span>
+                </div>
+                <div className="text-sm">
+                  {highlightText(result.msg, query)}
+                </div>
+              </div>
+            ))}
+          </div>
+        )}
+      </CardContent>
+    </Card>
+  );
+} 

inventory/src/components/config.ts

@@ -0,0 +1,7 @@
+const config = {
+  // API base URL - update based on your actual API endpoint
+  apiUrl: '/api',
+  // Add other config values as needed
+};
+
+export default config; 

inventory/src/components/dashboard/AcotTest.jsx

@@ -0,0 +1,133 @@
+import React, { useState, useEffect } from "react";
+import axios from "axios";
+import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
+import { Button } from "@/components/ui/button";
+import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
+import { Loader2, AlertCircle, CheckCircle, RefreshCw } from "lucide-react";
+
+const AcotTest = () => {
+  const [loading, setLoading] = useState(false);
+  const [error, setError] = useState(null);
+  const [data, setData] = useState(null);
+  const [connectionStatus, setConnectionStatus] = useState(null);
+
+  const testConnection = async () => {
+    setLoading(true);
+    setError(null);
+    try {
+      const response = await axios.get("/api/acot/test/test-connection");
+      setConnectionStatus(response.data);
+    } catch (err) {
+      setError(err.response?.data?.error || err.message);
+      setConnectionStatus(null);
+    } finally {
+      setLoading(false);
+    }
+  };
+
+  const fetchOrderCount = async () => {
+    setLoading(true);
+    setError(null);
+    try {
+      const response = await axios.get("/api/acot/test/order-count");
+      setData(response.data.data);
+    } catch (err) {
+      setError(err.response?.data?.error || err.message);
+      setData(null);
+    } finally {
+      setLoading(false);
+    }
+  };
+
+  useEffect(() => {
+    testConnection();
+  }, []);
+
+  return (
+    <Card className="w-full max-w-md">
+      <CardHeader>
+        <CardTitle className="flex items-center justify-between">
+          ACOT Server Test
+          <Button
+            size="icon"
+            variant="outline"
+            onClick={() => {
+              testConnection();
+              if (connectionStatus?.success) {
+                fetchOrderCount();
+              }
+            }}
+            disabled={loading}
+          >
+            {loading ? (
+              <Loader2 className="h-4 w-4 animate-spin" />
+            ) : (
+              <RefreshCw className="h-4 w-4" />
+            )}
+          </Button>
+        </CardTitle>
+      </CardHeader>
+      <CardContent className="space-y-4">
+        {/* Connection Status */}
+        <div className="space-y-2">
+          <h3 className="text-sm font-medium">Connection Status</h3>
+          {connectionStatus?.success ? (
+            <Alert className="bg-green-50 border-green-200">
+              <CheckCircle className="h-4 w-4 text-green-600" />
+              <AlertTitle className="text-green-800">Connected</AlertTitle>
+              <AlertDescription className="text-green-700">
+                {connectionStatus.message}
+              </AlertDescription>
+            </Alert>
+          ) : error ? (
+            <Alert variant="destructive">
+              <AlertCircle className="h-4 w-4" />
+              <AlertTitle>Connection Failed</AlertTitle>
+              <AlertDescription>{error}</AlertDescription>
+            </Alert>
+          ) : (
+            <div className="text-sm text-muted-foreground">
+              Testing connection...
+            </div>
+          )}
+        </div>
+
+        {/* Order Count */}
+        {connectionStatus?.success && (
+          <div className="space-y-2">
+            <Button
+              onClick={fetchOrderCount}
+              disabled={loading}
+              className="w-full"
+            >
+              {loading ? (
+                <>
+                  <Loader2 className="mr-2 h-4 w-4 animate-spin" />
+                  Loading...
+                </>
+              ) : (
+                "Fetch Order Count"
+              )}
+            </Button>
+
+            {data && (
+              <div className="p-4 bg-muted rounded-lg">
+                <div className="text-sm text-muted-foreground">
+                  Total Orders in Database
+                </div>
+                <div className="text-2xl font-bold">
+                  {data.orderCount?.toLocaleString()}
+                </div>
+                <div className="text-xs text-muted-foreground mt-1">
+                  Last updated: {new Date(data.timestamp).toLocaleTimeString()}
+                </div>
+              </div>
+            )}
+          </div>
+        )}
+      </CardContent>
+    </Card>
+  );
+};
+
+export default AcotTest; 

inventory/src/components/dashboard/AircallDashboard.jsx

@@ -0,0 +1,405 @@
+// components/AircallDashboard.jsx
+import React, { useState, useEffect } from "react";
+import { Card, CardContent } from "@/components/ui/card";
+import {
+  Select,
+  SelectContent,
+  SelectItem,
+  SelectTrigger,
+  SelectValue,
+} from "@/components/ui/select";
+import {
+  BarChart,
+  Bar,
+  XAxis,
+  YAxis,
+  CartesianGrid,
+  Tooltip as RechartsTooltip,
+  Legend,
+  ResponsiveContainer,
+} from "recharts";
+
+// Import shared components and tokens
+import {
+  DashboardChartTooltip,
+  DashboardStatCard,
+  DashboardStatCardSkeleton,
+  DashboardSectionHeader,
+  DashboardErrorState,
+  DashboardTable,
+  ChartSkeleton,
+  CARD_STYLES,
+  METRIC_COLORS,
+} from "@/components/dashboard/shared";
+import { Phone, Clock, Zap, Timer } from "lucide-react";
+
+// Aircall-specific colors using the standardized palette
+const CHART_COLORS = {
+  inbound: METRIC_COLORS.aov,        // Purple for inbound
+  outbound: METRIC_COLORS.revenue,   // Green for outbound
+  missed: METRIC_COLORS.comparison,  // Amber for missed
+  answered: METRIC_COLORS.revenue,   // Green for answered
+  duration: METRIC_COLORS.orders,    // Blue for duration
+  hourly: METRIC_COLORS.tertiary,    // Pink for hourly
+};
+
+const TIME_RANGES = [
+  { label: "Today", value: "today" },
+  { label: "Yesterday", value: "yesterday" },
+  { label: "Last 7 Days", value: "last7days" },
+  { label: "Last 30 Days", value: "last30days" },
+  { label: "Last 90 Days", value: "last90days" },
+];
+
+const REFRESH_INTERVAL = 5 * 60 * 1000;
+
+const formatDuration = (seconds) => {
+  const hours = Math.floor(seconds / 3600);
+  const minutes = Math.floor((seconds % 3600) / 60);
+  const remainingSeconds = seconds % 60;
+
+  if (hours > 0) {
+    return `${hours}h ${minutes}m`;
+  }
+  return `${minutes}m ${remainingSeconds}s`;
+};
+
+const AircallDashboard = () => {
+  const [timeRange, setTimeRange] = useState("last7days");
+  const [metrics, setMetrics] = useState(null);
+  const [lastUpdated, setLastUpdated] = useState(null);
+  const [isLoading, setIsLoading] = useState(true);
+  const [error, setError] = useState(null);
+  const [agentSort, setAgentSort] = useState({
+    key: "total",
+    direction: "desc",
+  });
+
+  const safeArray = (arr) => (Array.isArray(arr) ? arr : []);
+
+  const sortedAgents = metrics?.by_users
+    ? Object.values(metrics.by_users).sort((a, b) => {
+        const multiplier = agentSort.direction === "desc" ? -1 : 1;
+        return multiplier * (a[agentSort.key] - b[agentSort.key]);
+      })
+    : [];
+
+  const chartData = {
+    hourly: metrics?.by_hour
+      ? metrics.by_hour.map((count, hour) => ({
+          hour: new Date(2000, 0, 1, hour).toLocaleString('en-US', {
+            hour: 'numeric',
+            hour12: true
+          }).toUpperCase(),
+          calls: count || 0,
+        }))
+      : [],
+
+    missedReasons: metrics?.by_missed_reason
+      ? Object.entries(metrics.by_missed_reason).map(([reason, count]) => ({
+          reason: (reason || "").replace(/_/g, " "),
+          count: count || 0,
+        }))
+      : [],
+
+    daily: safeArray(metrics?.daily_data).map((day) => ({
+      ...day,
+      inbound: day.inbound || 0,
+      outbound: day.outbound || 0,
+      date: new Date(day.date).toLocaleString('en-US', {
+        month: 'short',
+        day: 'numeric'
+      }),
+    })),
+  };
+
+  // Column definitions for Agent Performance table
+  const agentColumns = [
+    {
+      key: "name",
+      header: "Agent",
+      render: (value) => <span className="font-medium text-foreground">{value}</span>,
+    },
+    {
+      key: "total",
+      header: "Total Calls",
+      align: "right",
+      sortable: true,
+      render: (value) => <span className="text-muted-foreground">{value}</span>,
+    },
+    {
+      key: "answered",
+      header: "Answered",
+      align: "right",
+      sortable: true,
+      render: (value) => <span className="text-trend-positive">{value}</span>,
+    },
+    {
+      key: "missed",
+      header: "Missed",
+      align: "right",
+      sortable: true,
+      render: (value) => <span className="text-trend-negative">{value}</span>,
+    },
+    {
+      key: "average_duration",
+      header: "Avg Duration",
+      align: "right",
+      sortable: true,
+      render: (value) => <span className="text-muted-foreground">{formatDuration(value)}</span>,
+    },
+  ];
+
+  // Column definitions for Missed Reasons table
+  const missedReasonsColumns = [
+    {
+      key: "reason",
+      header: "Reason",
+      render: (value) => <span className="font-medium text-foreground">{value}</span>,
+    },
+    {
+      key: "count",
+      header: "Count",
+      align: "right",
+      render: (value) => <span className="text-trend-negative">{value}</span>,
+    },
+  ];
+
+  const fetchData = async () => {
+    try {
+      setIsLoading(true);
+      const response = await fetch(`/api/aircall/metrics/${timeRange}`);
+      if (!response.ok) throw new Error("Failed to fetch metrics");
+      const data = await response.json();
+      setMetrics(data);
+      setLastUpdated(data._meta?.generatedAt);
+      setError(null);
+    } catch (err) {
+      setError(err.message);
+    } finally {
+      setIsLoading(false);
+    }
+  };
+
+  useEffect(() => {
+    fetchData();
+    const interval = setInterval(fetchData, REFRESH_INTERVAL);
+    return () => clearInterval(interval);
+  }, [timeRange]);
+
+  if (error) {
+    return (
+      <Card className={CARD_STYLES.base}>
+        <CardContent className="p-4">
+          <DashboardErrorState
+            title="Failed to load call data"
+            message={error}
+          />
+        </CardContent>
+      </Card>
+    );
+  }
+
+  return (
+    <div className="space-y-6">
+      <Card className={CARD_STYLES.base}>
+        <DashboardSectionHeader
+          title="Calls"
+          timeSelector={
+            <Select value={timeRange} onValueChange={setTimeRange}>
+              <SelectTrigger className="w-[130px] h-9">
+                <SelectValue placeholder="Select range" />
+              </SelectTrigger>
+              <SelectContent>
+                {TIME_RANGES.map((range) => (
+                  <SelectItem key={range.value} value={range.value}>
+                    {range.label}
+                  </SelectItem>
+                ))}
+              </SelectContent>
+            </Select>
+          }
+        />
+
+        <CardContent className="p-6 pt-0 space-y-4">
+          {/* Metric Cards */}
+          <div className="grid grid-cols-2 lg:grid-cols-4 gap-4 dashboard-stagger">
+            {isLoading ? (
+              [...Array(4)].map((_, i) => (
+                <DashboardStatCardSkeleton key={i} hasSubtitle />
+              ))
+            ) : metrics ? (
+              <>
+                <DashboardStatCard
+                  title="Total Calls"
+                  value={metrics.total}
+                  subtitle={
+                    <span className="flex gap-3">
+                      <span><span className="text-chart-orders">↑ {metrics.by_direction.inbound}</span> in</span>
+                      <span><span className="text-chart-revenue">↓ {metrics.by_direction.outbound}</span> out</span>
+                    </span>
+                  }
+                  icon={Phone}
+                  iconColor="blue"
+                />
+
+                <DashboardStatCard
+                  title="Answer Rate"
+                  value={`${((metrics.by_status.answered / metrics.total) * 100).toFixed(1)}%`}
+                  subtitle={
+                    <span className="flex gap-3">
+                      <span><span className="text-trend-positive">{metrics.by_status.answered}</span> answered</span>
+                      <span><span className="text-trend-negative">{metrics.by_status.missed}</span> missed</span>
+                    </span>
+                  }
+                  icon={Zap}
+                  iconColor="green"
+                />
+
+                <DashboardStatCard
+                  title="Peak Hour"
+                  value={
+                    metrics?.by_hour
+                      ? new Date(2000, 0, 1, metrics.by_hour.indexOf(Math.max(...metrics.by_hour)))
+                          .toLocaleString('en-US', { hour: 'numeric', hour12: true }).toUpperCase()
+                      : 'N/A'
+                  }
+                  subtitle={`Busiest Agent: ${sortedAgents[0]?.name || "N/A"}`}
+                  icon={Clock}
+                  iconColor="purple"
+                />
+
+                <DashboardStatCard
+                  title="Avg Duration"
+                  value={formatDuration(metrics.average_duration)}
+                  subtitle={
+                    metrics?.daily_data?.length > 0
+                      ? `${Math.round(metrics.total / metrics.daily_data.length)} calls/day`
+                      : "N/A"
+                  }
+                  tooltip={
+                    metrics?.duration_distribution
+                      ? `Duration Distribution: ${metrics.duration_distribution.map(d => `${d.range}: ${d.count}`).join(', ')}`
+                      : undefined
+                  }
+                  icon={Timer}
+                  iconColor="teal"
+                />
+              </>
+            ) : null}
+          </div>
+
+          {/* Charts and Tables Section */}
+          <div className="space-y-4">
+            {/* Charts Row */}
+            <div className="grid grid-cols-1 lg:grid-cols-2 gap-4">
+              {/* Daily Call Volume */}
+              <Card className={CARD_STYLES.base}>
+                <DashboardSectionHeader title="Daily Call Volume" compact />
+                <CardContent className="h-[300px]">
+                  {isLoading ? (
+                    <ChartSkeleton type="bar" height="md" withCard={false} />
+                  ) : (
+                    <ResponsiveContainer width="100%" height="100%">
+                      <BarChart data={chartData.daily} margin={{ top: 0, right: 5, left: -35, bottom: 0 }}>
+                        <CartesianGrid strokeDasharray="3 3" className="stroke-border/40" />
+                        <XAxis
+                          dataKey="date"
+                          tick={{ fontSize: 12 }}
+                          className="text-muted-foreground"
+                          tickLine={false}
+                          axisLine={false}
+                        />
+                        <YAxis
+                          tick={{ fontSize: 12 }}
+                          className="text-muted-foreground"
+                          tickLine={false}
+                          axisLine={false}
+                        />
+                        <RechartsTooltip content={<DashboardChartTooltip />} />
+                        <Legend />
+                        <Bar dataKey="inbound" fill={CHART_COLORS.inbound} name="Inbound" radius={[4, 4, 0, 0]} />
+                        <Bar dataKey="outbound" fill={CHART_COLORS.outbound} name="Outbound" radius={[4, 4, 0, 0]} />
+                      </BarChart>
+                    </ResponsiveContainer>
+                  )}
+                </CardContent>
+              </Card>
+
+              {/* Hourly Distribution */}
+              <Card className={CARD_STYLES.base}>
+                <DashboardSectionHeader title="Hourly Distribution" compact />
+                <CardContent className="h-[300px]">
+                  {isLoading ? (
+                    <ChartSkeleton type="bar" height="md" withCard={false} />
+                  ) : (
+                    <ResponsiveContainer width="100%" height="100%">
+                      <BarChart data={chartData.hourly} margin={{ top: 0, right: 5, left: -35, bottom: 0 }}>
+                        <CartesianGrid strokeDasharray="3 3" className="stroke-border/40" />
+                        <XAxis
+                          dataKey="hour"
+                          tick={{ fontSize: 12 }}
+                          interval={2}
+                          className="text-muted-foreground"
+                          tickLine={false}
+                          axisLine={false}
+                        />
+                        <YAxis
+                          tick={{ fontSize: 12 }}
+                          className="text-muted-foreground"
+                          tickLine={false}
+                          axisLine={false}
+                        />
+                        <RechartsTooltip content={<DashboardChartTooltip />} />
+                        <Bar dataKey="calls" fill={CHART_COLORS.hourly} name="Calls" radius={[4, 4, 0, 0]} />
+                      </BarChart>
+                    </ResponsiveContainer>
+                  )}
+                </CardContent>
+              </Card>
+            </div>
+
+            {/* Tables Row */}
+            <div className="grid grid-cols-1 lg:grid-cols-2 gap-4">
+              {/* Agent Performance */}
+              <Card className={CARD_STYLES.base}>
+                <DashboardSectionHeader title="Agent Performance" compact />
+                <CardContent>
+                  <DashboardTable
+                    columns={agentColumns}
+                    data={sortedAgents}
+                    loading={isLoading}
+                    skeletonRows={5}
+                    getRowKey={(agent) => agent.id}
+                    sortConfig={agentSort}
+                    onSort={(key, direction) => setAgentSort({ key, direction })}
+                    maxHeight="md"
+                    compact
+                  />
+                </CardContent>
+              </Card>
+
+              {/* Missed Call Reasons Table */}
+              <Card className={CARD_STYLES.base}>
+                <DashboardSectionHeader title="Missed Call Reasons" compact />
+                <CardContent>
+                  <DashboardTable
+                    columns={missedReasonsColumns}
+                    data={chartData.missedReasons}
+                    loading={isLoading}
+                    skeletonRows={5}
+                    getRowKey={(reason, index) => `${reason.reason}-${index}`}
+                    maxHeight="md"
+                    compact
+                  />
+                </CardContent>
+              </Card>
+            </div>
+          </div>
+        </CardContent>
+      </Card>
+    </div>
+  );
+};
+
+export default AircallDashboard;

inventory/src/components/dashboard/AnalyticsDashboard.jsx

@@ -0,0 +1,480 @@
+import React, { useState, useEffect } from "react";
+import { Card, CardContent } from "@/components/ui/card";
+import {
+  Select,
+  SelectContent,
+  SelectItem,
+  SelectTrigger,
+  SelectValue,
+} from "@/components/ui/select";
+import { Button } from "@/components/ui/button";
+import {
+  LineChart,
+  Line,
+  XAxis,
+  YAxis,
+  CartesianGrid,
+  Tooltip,
+  Legend,
+  ResponsiveContainer,
+} from "recharts";
+import { TrendingUp } from "lucide-react";
+import { Skeleton } from "@/components/ui/skeleton";
+import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogTrigger } from "@/components/ui/dialog";
+import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table";
+import { CARD_STYLES } from "@/lib/dashboard/designTokens";
+import {
+  DashboardSectionHeader,
+  DashboardStatCard,
+  DashboardStatCardSkeleton,
+  DashboardChartTooltip,
+  ChartSkeleton,
+  DashboardEmptyState,
+} from "@/components/dashboard/shared";
+
+// Note: Using ChartSkeleton from @/components/dashboard/shared
+
+const SkeletonStats = () => (
+  <div className="grid grid-cols-2 lg:grid-cols-4 gap-4 py-4">
+    {[...Array(4)].map((_, i) => (
+      <DashboardStatCardSkeleton key={i} size="compact" hasIcon={false} hasSubtitle />
+    ))}
+  </div>
+);
+
+// Note: Using shared DashboardStatCard from @/components/dashboard/shared
+
+// Add color constants
+const METRIC_COLORS = {
+  activeUsers: {
+    color: "#8b5cf6",
+    className: "text-purple-600 dark:text-purple-400",
+  },
+  newUsers: {
+    color: "#10b981",
+    className: "text-emerald-600 dark:text-emerald-400",
+  },
+  pageViews: {
+    color: "#f59e0b",
+    className: "text-amber-600 dark:text-amber-400",
+  },
+  conversions: {
+    color: "#3b82f6",
+    className: "text-blue-600 dark:text-blue-400",
+  },
+};
+
+export const AnalyticsDashboard = () => {
+  const [data, setData] = useState([]);
+  const [loading, setLoading] = useState(true);
+  const [timeRange, setTimeRange] = useState("30");
+  const [metrics, setMetrics] = useState({
+    activeUsers: true,
+    newUsers: true,
+    pageViews: true,
+    conversions: true,
+  });
+
+  useEffect(() => {
+    const fetchData = async () => {
+      setLoading(true);
+      try {
+        const response = await fetch(
+          `/api/dashboard-analytics/metrics?startDate=${timeRange}daysAgo`,
+          {
+            credentials: "include",
+          }
+        );
+
+        if (!response.ok) {
+          throw new Error("Failed to fetch metrics");
+        }
+
+        const result = await response.json();
+
+        if (!result?.data?.rows) {
+          return;
+        }
+
+        const processedData = result.data.rows.map((row) => ({
+          date: formatGADate(row.dimensionValues[0].value),
+          activeUsers: parseInt(row.metricValues[0].value),
+          newUsers: parseInt(row.metricValues[1].value),
+          avgSessionDuration: parseFloat(row.metricValues[2].value),
+          pageViews: parseInt(row.metricValues[3].value),
+          bounceRate: parseFloat(row.metricValues[4].value) * 100,
+          conversions: parseInt(row.metricValues[5].value),
+        }));
+
+        const sortedData = processedData.sort((a, b) => a.date - b.date);
+        setData(sortedData);
+      } catch (error) {
+        console.error("Failed to fetch analytics:", error);
+      } finally {
+        setLoading(false);
+      }
+    };
+    fetchData();
+  }, [timeRange]);
+
+  const formatGADate = (gaDate) => {
+    const year = gaDate.substring(0, 4);
+    const month = gaDate.substring(4, 6);
+    const day = gaDate.substring(6, 8);
+    return new Date(year, month - 1, day);
+  };
+
+  const formatXAxis = (date) => {
+    if (!date) return "";
+    return date.toLocaleDateString([], { month: "short", day: "numeric" });
+  };
+
+  const calculateSummaryStats = () => {
+    if (!data.length) return null;
+
+    const totals = data.reduce(
+      (acc, day) => ({
+        activeUsers: acc.activeUsers + day.activeUsers,
+        newUsers: acc.newUsers + day.newUsers,
+        pageViews: acc.pageViews + day.pageViews,
+        conversions: acc.conversions + day.conversions,
+      }),
+      {
+        activeUsers: 0,
+        newUsers: 0,
+        pageViews: 0,
+        conversions: 0,
+      }
+    );
+
+    const averages = {
+      activeUsers: totals.activeUsers / data.length,
+      newUsers: totals.newUsers / data.length,
+      pageViews: totals.pageViews / data.length,
+      conversions: totals.conversions / data.length,
+    };
+
+    return { totals, averages };
+  };
+
+  const summaryStats = calculateSummaryStats();
+
+  // Time selector for DashboardSectionHeader
+  const timeSelector = (
+    <Select value={timeRange} onValueChange={setTimeRange}>
+      <SelectTrigger className="w-[130px] h-9">
+        <SelectValue placeholder="Select range" />
+      </SelectTrigger>
+      <SelectContent>
+        <SelectItem value="7">Last 7 days</SelectItem>
+        <SelectItem value="14">Last 14 days</SelectItem>
+        <SelectItem value="30">Last 30 days</SelectItem>
+        <SelectItem value="90">Last 90 days</SelectItem>
+      </SelectContent>
+    </Select>
+  );
+
+  // Header actions: Details dialog
+  const headerActions = !loading ? (
+    <Dialog>
+      <DialogTrigger asChild>
+        <Button variant="outline" className="h-9">
+          Details
+        </Button>
+      </DialogTrigger>
+      <DialogContent className={`max-w-[95vw] w-fit max-h-[85vh] overflow-hidden flex flex-col ${CARD_STYLES.base}`}>
+        <DialogHeader className="flex-none">
+          <DialogTitle className="text-foreground">Daily Details</DialogTitle>
+          <div className="flex items-center justify-center gap-2 pt-4">
+            <div className="flex flex-wrap gap-1">
+              {Object.entries(metrics).map(([key, value]) => (
+                <Button
+                  key={key}
+                  variant={value ? "default" : "outline"}
+                  size="sm"
+                  onClick={() =>
+                    setMetrics((prev) => ({
+                      ...prev,
+                      [key]: !prev[key],
+                    }))
+                  }
+                >
+                  {key === "activeUsers" ? "Active Users" :
+                   key === "newUsers" ? "New Users" :
+                   key === "pageViews" ? "Page Views" :
+                   "Conversions"}
+                </Button>
+              ))}
+            </div>
+          </div>
+        </DialogHeader>
+        <div className="flex-1 overflow-y-auto mt-6">
+          <div className={`rounded-lg border ${CARD_STYLES.base} w-full`}>
+            <Table className="w-full">
+              <TableHeader>
+                <TableRow>
+                  <TableHead className="text-center whitespace-nowrap px-6 w-[120px]">Date</TableHead>
+                  {metrics.activeUsers && (
+                    <TableHead className="text-center whitespace-nowrap px-6 min-w-[100px]">Active Users</TableHead>
+                  )}
+                  {metrics.newUsers && (
+                    <TableHead className="text-center whitespace-nowrap px-6 min-w-[100px]">New Users</TableHead>
+                  )}
+                  {metrics.pageViews && (
+                    <TableHead className="text-center whitespace-nowrap px-6 min-w-[140px]">Page Views</TableHead>
+                  )}
+                  {metrics.conversions && (
+                    <TableHead className="text-center whitespace-nowrap px-6 min-w-[120px]">Conversions</TableHead>
+                  )}
+                </TableRow>
+              </TableHeader>
+              <TableBody>
+                {data.map((day) => (
+                  <TableRow key={day.date}>
+                    <TableCell className="text-center whitespace-nowrap px-6">{formatXAxis(day.date)}</TableCell>
+                    {metrics.activeUsers && (
+                      <TableCell className="text-center whitespace-nowrap px-6">
+                        {day.activeUsers.toLocaleString()}
+                      </TableCell>
+                    )}
+                    {metrics.newUsers && (
+                      <TableCell className="text-center whitespace-nowrap px-6">
+                        {day.newUsers.toLocaleString()}
+                      </TableCell>
+                    )}
+                    {metrics.pageViews && (
+                      <TableCell className="text-center whitespace-nowrap px-6">
+                        {day.pageViews.toLocaleString()}
+                      </TableCell>
+                    )}
+                    {metrics.conversions && (
+                      <TableCell className="text-center whitespace-nowrap px-6">
+                        {day.conversions.toLocaleString()}
+                      </TableCell>
+                    )}
+                  </TableRow>
+                ))}
+              </TableBody>
+            </Table>
+          </div>
+        </div>
+      </DialogContent>
+    </Dialog>
+  ) : <Skeleton className="h-9 w-20 bg-muted rounded-sm" />;
+
+  // Label formatter for chart tooltip
+  const analyticsLabelFormatter = (label) => {
+    const date = label instanceof Date ? label : new Date(label);
+    return date.toLocaleDateString("en-US", { month: "short", day: "numeric" });
+  };
+
+  return (
+    <Card className={`w-full ${CARD_STYLES.base}`}>
+      <DashboardSectionHeader
+        title="Analytics Overview"
+        loading={loading}
+        timeSelector={timeSelector}
+        actions={headerActions}
+      />
+
+      <CardContent className="p-6 pt-0 space-y-4">
+        {/* Stats cards */}
+        {loading ? (
+          <SkeletonStats />
+        ) : summaryStats ? (
+          <div className="grid grid-cols-2 lg:grid-cols-4 gap-4 dashboard-stagger">
+            <DashboardStatCard
+              title="Active Users"
+              value={summaryStats.totals.activeUsers.toLocaleString()}
+              subtitle={`Avg: ${Math.round(
+                summaryStats.averages.activeUsers
+              ).toLocaleString()} per day`}
+              size="compact"
+            />
+            <DashboardStatCard
+              title="New Users"
+              value={summaryStats.totals.newUsers.toLocaleString()}
+              subtitle={`Avg: ${Math.round(
+                summaryStats.averages.newUsers
+              ).toLocaleString()} per day`}
+              size="compact"
+            />
+            <DashboardStatCard
+              title="Page Views"
+              value={summaryStats.totals.pageViews.toLocaleString()}
+              subtitle={`Avg: ${Math.round(
+                summaryStats.averages.pageViews
+              ).toLocaleString()} per day`}
+              size="compact"
+            />
+            <DashboardStatCard
+              title="Conversions"
+              value={summaryStats.totals.conversions.toLocaleString()}
+              subtitle={`Avg: ${Math.round(
+                summaryStats.averages.conversions
+              ).toLocaleString()} per day`}
+              size="compact"
+            />
+          </div>
+        ) : null}
+
+        {/* Metric toggles */}
+        <div className="flex items-center flex-col sm:flex-row gap-0 sm:gap-4">
+          <div className="flex flex-wrap gap-1">
+            <Button
+              variant={metrics.activeUsers ? "default" : "outline"}
+              size="sm"
+              className="font-medium"
+              onClick={() =>
+                setMetrics((prev) => ({
+                  ...prev,
+                  activeUsers: !prev.activeUsers,
+                }))
+              }
+            >
+              <span className="hidden sm:inline">Active Users</span>
+              <span className="sm:hidden">Active</span>
+            </Button>
+            <Button
+              variant={metrics.newUsers ? "default" : "outline"}
+              size="sm"
+              className="font-medium"
+              onClick={() =>
+                setMetrics((prev) => ({
+                  ...prev,
+                  newUsers: !prev.newUsers,
+                }))
+              }
+            >
+              <span className="hidden sm:inline">New Users</span>
+              <span className="sm:hidden">New</span>
+            </Button>
+            <Button
+              variant={metrics.pageViews ? "default" : "outline"}
+              size="sm"
+              className="font-medium"
+              onClick={() =>
+                setMetrics((prev) => ({
+                  ...prev,
+                  pageViews: !prev.pageViews,
+                }))
+              }
+            >
+              <span className="hidden sm:inline">Page Views</span>
+              <span className="sm:hidden">Views</span>
+            </Button>
+            <Button
+              variant={metrics.conversions ? "default" : "outline"}
+              size="sm"
+              className="font-medium"
+              onClick={() =>
+                setMetrics((prev) => ({
+                  ...prev,
+                  conversions: !prev.conversions,
+                }))
+              }
+            >
+              <span className="hidden sm:inline">Conversions</span>
+              <span className="sm:hidden">Conv.</span>
+            </Button>
+          </div>
+        </div>
+        {loading ? (
+          <ChartSkeleton height="default" withCard={false} />
+        ) : !data.length ? (
+          <DashboardEmptyState
+            icon={TrendingUp}
+            title="No analytics data available"
+            description="Try selecting a different time range"
+          />
+        ) : (
+          <div className={`h-[400px] mt-4 ${CARD_STYLES.base} rounded-lg p-0 relative`}>
+            <ResponsiveContainer width="100%" height="100%">
+              <LineChart
+                data={data}
+                margin={{ top: 5, right: -30, left: -5, bottom: 5 }}
+              >
+                <CartesianGrid
+                  strokeDasharray="3 3"
+                  className="stroke-muted"
+                />
+                <XAxis
+                  dataKey="date"
+                  tickFormatter={formatXAxis}
+                  className="text-xs text-muted-foreground"
+                  tick={{ fill: "currentColor" }}
+                />
+                <YAxis
+                  yAxisId="left"
+                  className="text-xs text-muted-foreground"
+                  tick={{ fill: "currentColor" }}
+                />
+                <YAxis
+                  yAxisId="right"
+                  orientation="right"
+                  className="text-xs text-muted-foreground"
+                  tick={{ fill: "currentColor" }}
+                />
+                <Tooltip
+                  content={
+                    <DashboardChartTooltip
+                      labelFormatter={analyticsLabelFormatter}
+                      valueFormatter={(value) => value.toLocaleString()}
+                    />
+                  }
+                />
+                <Legend />
+                {metrics.activeUsers && (
+                  <Line
+                    yAxisId="left"
+                    type="monotone"
+                    dataKey="activeUsers"
+                    name="Active Users"
+                    stroke={METRIC_COLORS.activeUsers.color}
+                    strokeWidth={2}
+                    dot={false}
+                  />
+                )}
+                {metrics.newUsers && (
+                  <Line
+                    yAxisId="left"
+                    type="monotone"
+                    dataKey="newUsers"
+                    name="New Users"
+                    stroke={METRIC_COLORS.newUsers.color}
+                    strokeWidth={2}
+                    dot={false}
+                  />
+                )}
+                {metrics.pageViews && (
+                  <Line
+                    yAxisId="right"
+                    type="monotone"
+                    dataKey="pageViews"
+                    name="Page Views"
+                    stroke={METRIC_COLORS.pageViews.color}
+                    strokeWidth={2}
+                    dot={false}
+                  />
+                )}
+                {metrics.conversions && (
+                  <Line
+                    yAxisId="right"
+                    type="monotone"
+                    dataKey="conversions"
+                    name="Conversions"
+                    stroke={METRIC_COLORS.conversions.color}
+                    strokeWidth={2}
+                    dot={false}
+                  />
+                )}
+              </LineChart>
+            </ResponsiveContainer>
+          </div>
+        )}
+      </CardContent>
+    </Card>
+  );
+};
+
+export default AnalyticsDashboard; 

inventory/src/components/dashboard/DateTime.jsx

@@ -0,0 +1,456 @@
+import React, { useState, useEffect } from 'react';
+import { Card, CardContent } from '@/components/ui/card';
+import { Calendar as CalendarComponent } from '@/components/ui/calendaredit';
+import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
+import { Alert, AlertDescription } from '@/components/ui/alert';
+import {
+  Sun,
+  Cloud,
+  CloudRain,
+  CloudDrizzle,
+  CloudSnow,
+  CloudLightning,
+  CloudFog,
+  CloudSun,
+  CircleAlert,
+  Tornado,
+  Haze,
+  Moon,
+  Wind,
+  Droplets,
+  ThermometerSun,
+  ThermometerSnowflake,
+  Sunrise,
+  Sunset,
+  AlertTriangle,
+  Umbrella,
+  ChevronLeft,
+  ChevronRight
+} from 'lucide-react';
+import { cn } from "@/lib/utils";
+
+const DateTimeWeatherDisplay = ({ scaleFactor = 1 }) => {
+  const [datetime, setDatetime] = useState(new Date());
+  const [prevTime, setPrevTime] = useState(getTimeComponents(new Date()));
+  const [isTimeChanging, setIsTimeChanging] = useState(false);
+  const [mounted, setMounted] = useState(false);
+  const [weather, setWeather] = useState(null);
+  const [forecast, setForecast] = useState(null);
+
+  useEffect(() => {
+    setTimeout(() => setMounted(true), 150);
+
+    const timer = setInterval(() => {
+      const newDate = new Date();
+      const newTime = getTimeComponents(newDate);
+      
+      if (newTime.minutes !== prevTime.minutes) {
+        setIsTimeChanging(true);
+        setTimeout(() => setIsTimeChanging(false), 200);
+      }
+      
+      setPrevTime(newTime);
+      setDatetime(newDate);
+    }, 1000);
+    return () => clearInterval(timer);
+  }, [prevTime]);
+
+  useEffect(() => {
+    const fetchWeatherData = async () => {
+      try {
+        const API_KEY = import.meta.env.VITE_OPENWEATHER_API_KEY;
+        const [weatherResponse, forecastResponse] = await Promise.all([
+          fetch(
+            `https://api.openweathermap.org/data/2.5/weather?lat=43.63507&lon=-84.18995&appid=${API_KEY}&units=imperial`
+          ),
+          fetch(
+            `https://api.openweathermap.org/data/2.5/forecast?lat=43.63507&lon=-84.18995&appid=${API_KEY}&units=imperial`
+          )
+        ]);
+        
+        const weatherData = await weatherResponse.json();
+        const forecastData = await forecastResponse.json();
+        
+        setWeather(weatherData);
+        
+        // Process forecast data to get daily forecasts with precipitation
+        const dailyForecasts = forecastData.list.reduce((acc, item) => {
+          const date = new Date(item.dt * 1000).toLocaleDateString();
+          if (!acc[date]) {
+            acc[date] = {
+              ...item,
+              precipitation: item.rain?.['3h'] || item.snow?.['3h'] || 0,
+              pop: item.pop * 100 // Probability of precipitation as percentage
+            };
+          }
+          return acc;
+        }, {});
+        
+        setForecast(Object.values(dailyForecasts).slice(0, 5));
+      } catch (error) {
+        console.error("Error fetching weather:", error);
+      }
+    };
+
+    fetchWeatherData();
+    const weatherTimer = setInterval(fetchWeatherData, 300000);
+    return () => clearInterval(weatherTimer);
+  }, []);
+
+  function getTimeComponents(date) {
+    let hours = date.getHours();
+    const minutes = date.getMinutes();
+    const ampm = hours >= 12 ? 'PM' : 'AM';
+    hours = hours % 12;
+    hours = hours ? hours : 12;
+    return {
+      hours: hours.toString(),
+      minutes: minutes.toString().padStart(2, '0'),
+      ampm
+    };
+  }
+
+  const formatDate = (date) => {
+    return {
+      weekday: date.toLocaleDateString('en-US', { weekday: 'long' }),
+      month: date.toLocaleDateString('en-US', { month: 'long' }),
+      day: date.getDate()
+    };
+  };
+
+  const getWeatherIcon = (weatherCode, currentTime, small = false) => {
+    if (!weatherCode) return <CircleAlert className="w-12 h-12 text-red-500" />;
+    const code = parseInt(weatherCode, 10);
+    const iconProps = small ? "w-8 h-8" : "w-12 h-12";
+    const isNight = currentTime.getHours() >= 18 || currentTime.getHours() < 6;
+    
+    switch (true) {
+      case code >= 200 && code < 300:
+        return <CloudLightning className={cn(iconProps, "text-yellow-300")} />;
+      case code >= 300 && code < 500:
+        return <CloudDrizzle className={cn(iconProps, "text-blue-300")} />;
+      case code >= 500 && code < 600:
+        return <CloudRain className={cn(iconProps, "text-blue-300")} />;
+      case code >= 600 && code < 700:
+        return <CloudSnow className={cn(iconProps, "text-blue-200")} />;
+      case code >= 700 && code < 721:
+        return <CloudFog className={cn(iconProps, "text-gray-300")} />;
+      case code === 721:
+        return <Haze className={cn(iconProps, "text-gray-300")} />;
+      case code >= 722 && code < 781:
+        return <CloudFog className={cn(iconProps, "text-gray-300")} />;
+      case code === 781:
+        return <Tornado className={cn(iconProps, "text-gray-300")} />;
+      case code === 800:
+        return currentTime.getHours() >= 6 && currentTime.getHours() < 18 ? (
+          <Sun className={cn(iconProps, "text-yellow-300")} />
+        ) : (
+          <Moon className={cn(iconProps, "text-gray-300")} />
+        );
+      case code >= 800 && code < 803:
+        return <CloudSun className={cn(iconProps, isNight ? "text-gray-300" : "text-gray-200")} />;
+      case code >= 803:
+        return <Cloud className={cn(iconProps, "text-gray-300")} />;
+      default:
+        return <CircleAlert className={cn(iconProps, "text-red-500")} />;
+    }
+  };
+
+  const getWeatherBackground = (weatherCode, isNight) => {
+    const code = parseInt(weatherCode, 10);
+    
+    // Thunderstorm (200-299)
+    if (code >= 200 && code < 300) {
+      return "bg-gradient-to-br from-slate-900 to-purple-800";
+    }
+    
+    // Drizzle (300-399)
+    if (code >= 300 && code < 400) {
+      return "bg-gradient-to-br from-slate-800 to-blue-800";
+    }
+    
+    // Rain (500-599)
+    if (code >= 500 && code < 600) {
+      return "bg-gradient-to-br from-slate-800 to-blue-800";
+    }
+    
+    // Snow (600-699)
+    if (code >= 600 && code < 700) {
+      return "bg-gradient-to-br from-slate-700 to-blue-800";
+    }
+    
+    // Atmosphere (700-799: mist, smoke, haze, fog, etc.)
+    if (code >= 700 && code < 800) {
+      return "bg-gradient-to-br from-slate-700 to-slate-500";
+    }
+    
+    // Clear (800)
+    if (code === 800) {
+      if (isNight) {
+        return "bg-gradient-to-br from-slate-900 to-blue-900";
+      }
+      return "bg-gradient-to-br from-blue-600 to-sky-400";
+    }
+    
+    // Clouds (801-804)
+    if (code > 800) {
+      if (isNight) {
+        return "bg-gradient-to-br from-slate-800 to-slate-600";
+      }
+      return "bg-gradient-to-br from-slate-600 to-slate-400";
+    }
+    
+    // Default fallback
+    return "bg-gradient-to-br from-slate-700 to-slate-500";
+  };
+  
+  const getTemperatureColor = (weatherCode, isNight) => {
+    const code = parseInt(weatherCode, 10);
+    
+    // Snow - dark background, light text
+    if (code >= 600 && code < 700) {
+      return "text-white";
+    }
+    
+    // Clear day - light background, dark text
+    if (code === 800 && !isNight) {
+      return "text-white";
+    }
+    
+    // Cloudy day - medium background, ensure contrast
+    if (code > 800 && !isNight) {
+      return "text-white";
+    }
+    
+    // All other cases (darker backgrounds)
+    return "text-white";
+  };
+
+  const { hours, minutes, ampm } = getTimeComponents(datetime);
+  const dateInfo = formatDate(datetime);
+
+  const formatTime = (timestamp) => {
+    if (!timestamp) return '--:--';
+    const date = new Date(timestamp * 1000);
+    return date.toLocaleTimeString('en-US', { 
+      hour: 'numeric',
+      minute: '2-digit',
+      hour12: true 
+    });
+  };
+
+  const WeatherDetails = () => (
+    <div className="space-y-4 p-3 bg-gradient-to-br from-slate-800 to-slate-700">
+      <div className="grid grid-cols-3 gap-2">
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <ThermometerSun className="w-5 h-5 text-yellow-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">High</span>
+              <span className="text-sm font-bold text-white">{Math.round(weather.main.temp_max)}°F</span>
+            </div>
+          </div>
+        </Card>
+        
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <ThermometerSnowflake className="w-5 h-5 text-blue-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">Low</span>
+              <span className="text-sm font-bold text-white">{Math.round(weather.main.temp_min)}°F</span>
+            </div>
+          </div>
+        </Card>
+
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <Droplets className="w-5 h-5 text-blue-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">Humidity</span>
+              <span className="text-sm font-bold text-white">{weather.main.humidity}%</span>
+            </div>
+          </div>
+        </Card>
+
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <Wind className="w-5 h-5 text-slate-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">Wind</span>
+              <span className="text-sm font-bold text-white">{Math.round(weather.wind.speed)} mph</span>
+            </div>
+          </div>
+        </Card>
+
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <Sunrise className="w-5 h-5 text-yellow-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">Sunrise</span>
+              <span className="text-sm font-bold text-white">{formatTime(weather.sys?.sunrise)}</span>
+            </div>
+          </div>
+        </Card>
+
+        <Card className="bg-gradient-to-br from-slate-700 to-slate-600 backdrop-blur-sm p-2">
+          <div className="flex items-center gap-1">
+            <Sunset className="w-5 h-5 text-orange-300" />
+            <div className="flex flex-col">
+              <span className="text-xs text-slate-300">Sunset</span>
+              <span className="text-sm font-bold text-white">{formatTime(weather.sys?.sunset)}</span>
+            </div>
+          </div>
+        </Card>
+      </div>
+
+      {forecast && (
+        <div>
+          <div className="grid grid-cols-5 gap-2">
+            {forecast.map((day, index) => {
+              const forecastTime = new Date(day.dt * 1000);
+              const isNight = forecastTime.getHours() >= 18 || forecastTime.getHours() < 6;
+              return (
+                <Card 
+                  key={index} 
+                  className={cn(
+                    getWeatherBackground(day.weather[0].id, isNight),
+                    "p-2"
+                  )}
+                >
+                  <div className="flex flex-col items-center gap-1">
+                    <span className="text-sm font-medium text-white">
+                      {forecastTime.toLocaleDateString('en-US', { weekday: 'short' })}
+                    </span>
+                    {getWeatherIcon(day.weather[0].id, forecastTime, true)}
+                    <div className="flex justify-center gap-1 items-baseline w-full">
+                      <span className="text-sm font-medium text-white">
+                        {Math.round(day.main.temp_max)}°
+                      </span>
+                      <span className="text-xs text-slate-300">
+                        {Math.round(day.main.temp_min)}°
+                      </span>
+                    </div>
+                    <div className="flex flex-col items-center gap-1 w-full pt-1">
+                      {day.rain?.['3h'] > 0 && (
+                        <div className="flex items-center gap-1">
+                          <CloudRain className="w-3 h-3 text-blue-300" />
+                          <span className="text-xs text-white">{day.rain['3h'].toFixed(2)}"</span>
+                        </div>
+                      )}
+                      {day.snow?.['3h'] > 0 && (
+                        <div className="flex items-center gap-1">
+                          <CloudSnow className="w-3 h-3 text-blue-300" />
+                          <span className="text-xs text-white">{day.snow['3h'].toFixed(2)}"</span>
+                        </div>
+                      )}
+                      {!day.rain?.['3h'] && !day.snow?.['3h'] && (
+                        <div className="flex items-center gap-1">
+                          <Umbrella className="w-3 h-3 text-slate-300" />
+                          <span className="text-xs text-white">0"</span>
+                        </div>
+                      )}
+                    </div>
+                  </div>
+                </Card>
+              );
+            })}
+          </div>
+        </div>
+      )}
+    </div>
+  );
+
+  return (
+    <div className="flex flex-col items-center w-full transition-opacity duration-300 ${mounted ? 'opacity-100' : 'opacity-0'}">
+      {/* Time Display */}
+      <Card className="bg-gradient-to-br mb-[7px] from-indigo-900 to-blue-800 backdrop-blur-sm dark:bg-slate-800 px-1 py-2 w-full hover:scale-[1.02] transition-transform duration-300">
+        <CardContent className="p-3 h-[106px] flex items-center">
+          <div className="flex justify-center items-baseline w-full">
+            <div className={`transition-opacity duration-200 ${isTimeChanging ? 'opacity-60' : 'opacity-100'}`}>
+              <span className="text-6xl font-bold text-white">{hours}</span>
+              <span className="text-6xl font-bold text-white">:</span>
+              <span className="text-6xl font-bold text-white">{minutes}</span>
+              <span className="text-lg font-medium text-white/90 ml-1">{ampm}</span>
+            </div>
+          </div>
+        </CardContent>
+      </Card>
+
+      {/* Date and Weather Display */}
+      <div className="h-[125px] mb-[6px] grid grid-cols-2 gap-2 w-full">
+        <Card className="h-full bg-gradient-to-br from-violet-900 to-purple-800 backdrop-blur-sm flex items-center justify-center">
+          <CardContent className="h-full p-0">
+            <div className="flex flex-col items-center justify-center h-full">
+              <span className="text-6xl font-bold text-white">
+                {dateInfo.day}
+              </span>
+              <span className="text-sm font-bold text-white mt-2">
+                {dateInfo.weekday}
+              </span>
+            </div>
+          </CardContent>
+        </Card>
+
+        {weather?.main && (
+          <Popover>
+            <PopoverTrigger asChild>
+              <Card className={cn(
+                getWeatherBackground(
+                  weather.weather[0]?.id,
+                  datetime.getHours() >= 18 || datetime.getHours() < 6
+                ),
+                "flex items-center justify-center cursor-pointer hover:brightness-110 transition-all relative backdrop-blur-sm"
+              )}>
+                <CardContent className="h-full p-3">
+                  <div className="flex flex-col items-center">
+                    {getWeatherIcon(weather.weather[0]?.id, datetime)}
+                    <span className="text-3xl font-bold ml-1 mt-2 text-white">
+                      {Math.round(weather.main.temp)}°
+                    </span>
+                  </div>
+                </CardContent>
+                {weather.alerts && (
+                  <div className="absolute top-1 right-1">
+                    <AlertTriangle className="w-5 h-5 text-red-500" />
+                  </div>
+                )}
+              </Card>
+            </PopoverTrigger>
+            <PopoverContent 
+              className="w-[450px] bg-gradient-to-br from-slate-800 to-slate-700 border-slate-600" 
+              align="start"
+              side="right"
+              sideOffset={10}
+              style={{ 
+                transform: `scale(${scaleFactor})`,
+                transformOrigin: 'left top'
+              }}
+            >
+              {weather.alerts && (
+                <Alert variant="warning" className="mb-3 bg-amber-900/50 border-amber-700">
+                  <AlertTriangle className="h-3 w-3 text-amber-500" />
+                  <AlertDescription className="text-xs text-amber-200">
+                    {weather.alerts[0].event}
+                  </AlertDescription>
+                </Alert>
+              )}
+              <WeatherDetails />
+            </PopoverContent>
+          </Popover>
+        )}
+      </div>
+
+      {/* Calendar Display */}
+      <Card className="w-full bg-white dark:bg-slate-800">
+        <CardContent className="p-0">
+          <CalendarComponent
+            selected={datetime}
+            className="w-full"
+          />
+        </CardContent>
+      </Card>
+    </div>
+  );
+};
+
+export default DateTimeWeatherDisplay;