Fix more import script bugs/missing data

docs/import-from-prod-data-mapping.md

@@ -0,0 +1,342 @@
+# MySQL to PostgreSQL Import Process Documentation
+
+This document outlines the data import process from the production MySQL database to the local PostgreSQL database, focusing on column mappings, data transformations, and the overall import architecture.
+
+## Table of Contents
+1. [Overview](#overview)
+2. [Import Architecture](#import-architecture)
+3. [Column Mappings](#column-mappings)
+    - [Categories](#categories)
+    - [Products](#products)
+    - [Product Categories (Relationship)](#product-categories-relationship)
+    - [Orders](#orders)
+    - [Purchase Orders](#purchase-orders)
+    - [Metadata Tables](#metadata-tables)
+4. [Special Calculations](#special-calculations)
+5. [Implementation Notes](#implementation-notes)
+
+## Overview
+
+The import process extracts data from a MySQL 5.7 production database and imports it into a PostgreSQL database. It can operate in two modes:
+
+- **Full Import**: Imports all data regardless of last sync time
+- **Incremental Import**: Only imports data that has changed since the last import
+
+The process handles four main data types:
+- Categories (product categorization hierarchy)
+- Products (inventory items)
+- Orders (sales records)
+- Purchase Orders (vendor orders)
+
+## Import Architecture
+
+The import process follows these steps:
+
+1. **Establish Connection**: Creates a SSH tunnel to the production server and establishes database connections
+2. **Setup Import History**: Creates a record of the current import operation
+3. **Import Categories**: Processes product categories in hierarchical order
+4. **Import Products**: Processes products with their attributes and category relationships
+5. **Import Orders**: Processes customer orders with line items, taxes, and discounts
+6. **Import Purchase Orders**: Processes vendor purchase orders with line items
+7. **Record Results**: Updates the import history with results
+8. **Close Connections**: Cleans up connections and resources
+
+Each import step uses temporary tables for processing and wraps operations in transactions to ensure data consistency.
+
+## Column Mappings
+
+### Categories
+| PostgreSQL Column | MySQL Source                    | Transformation                               |
+|-------------------|---------------------------------|----------------------------------------------|
+| cat_id            | product_categories.cat_id       | Direct mapping                               |
+| name              | product_categories.name         | Direct mapping                               |
+| type              | product_categories.type         | Direct mapping                               |
+| parent_id         | product_categories.master_cat_id| NULL for top-level categories (types 10, 20) |
+| description       | product_categories.combined_name| Direct mapping                               |
+| status            | N/A                             | Hard-coded 'active'                          |
+| created_at        | N/A                             | Current timestamp                            |
+| updated_at        | N/A                             | Current timestamp                            |
+
+**Notes:**
+- Categories are processed in hierarchical order by type: [10, 20, 11, 21, 12, 13]
+- Type 10/20 are top-level categories with no parent
+- Types 11/21/12/13 are child categories that reference parent categories
+
+### Products
+| PostgreSQL Column    | MySQL Source                     | Transformation                                                |
+|----------------------|----------------------------------|---------------------------------------------------------------|
+| pid                  | products.pid                     | Direct mapping                                                |
+| title                | products.description             | Direct mapping                                                |
+| description          | products.notes                   | Direct mapping                                                |
+| sku                  | products.itemnumber              | Fallback to 'NO-SKU' if empty                                 |
+| stock_quantity       | shop_inventory.available_local   | Capped at 5000, minimum 0                                     |
+| preorder_count       | current_inventory.onpreorder     | Default 0                                                     |
+| notions_inv_count    | product_notions_b2b.inventory    | Default 0                                                     |
+| price                | product_current_prices.price_each| Default 0, filtered on active=1                              |
+| regular_price        | products.sellingprice            | Default 0                                                     |
+| cost_price           | product_inventory                | Weighted average: SUM(costeach * count) / SUM(count) when count > 0, or latest costeach |
+| vendor               | suppliers.companyname            | Via supplier_item_data.supplier_id                            |
+| vendor_reference     | supplier_item_data               | supplier_itemnumber or notions_itemnumber based on vendor     |
+| notions_reference    | supplier_item_data.notions_itemnumber | Direct mapping                                           |
+| brand                | product_categories.name          | Linked via products.company                                   |
+| line                 | product_categories.name          | Linked via products.line                                      |
+| subline              | product_categories.name          | Linked via products.subline                                   |
+| artist               | product_categories.name          | Linked via products.artist                                    |
+| categories           | product_category_index           | Comma-separated list of category IDs                          |
+| created_at           | products.date_created            | Validated date, NULL if invalid                               |
+| first_received       | products.datein                  | Validated date, NULL if invalid                               |
+| landing_cost_price   | NULL                             | Not set                                                       |
+| barcode              | products.upc                     | Direct mapping                                                |
+| harmonized_tariff_code| products.harmonized_tariff_code | Direct mapping                                                |
+| updated_at           | products.stamp                   | Validated date, NULL if invalid                               |
+| visible              | shop_inventory                   | Calculated from show + buyable > 0                            |
+| managing_stock       | N/A                              | Hard-coded true                                               |
+| replenishable        | Multiple fields                  | Complex calculation based on reorder, dates, etc.             |
+| permalink            | N/A                              | Constructed URL with product ID                               |
+| moq                  | supplier_item_data               | notions_qty_per_unit or supplier_qty_per_unit, minimum 1      |
+| uom                  | N/A                              | Hard-coded 1                                                  |
+| rating               | products.rating                  | Direct mapping                                                |
+| reviews              | products.rating_votes            | Direct mapping                                                |
+| weight               | products.weight                  | Direct mapping                                                |
+| length               | products.length                  | Direct mapping                                                |
+| width                | products.width                   | Direct mapping                                                |
+| height               | products.height                  | Direct mapping                                                |
+| country_of_origin    | products.country_of_origin       | Direct mapping                                                |
+| location             | products.location                | Direct mapping                                                |
+| total_sold           | order_items                      | SUM(qty_ordered) for all order_items where prod_pid = pid     |
+| baskets              | mybasket                         | COUNT of records where mb.item = pid and qty > 0              |
+| notifies             | product_notify                   | COUNT of records where pn.pid = pid                           |
+| date_last_sold       | product_last_sold.date_sold      | Validated date, NULL if invalid                               |
+| image                | N/A                              | Constructed from pid and image URL pattern                    |
+| image_175            | N/A                              | Constructed from pid and image URL pattern                    |
+| image_full           | N/A                              | Constructed from pid and image URL pattern                    |
+| options              | NULL                             | Not set                                                       |
+| tags                 | NULL                             | Not set                                                       |
+
+**Notes:**
+- Replenishable calculation: 
+  ```javascript
+  CASE 
+    WHEN p.reorder < 0 THEN 0
+    WHEN (
+      (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
+      AND (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+      AND (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+    ) THEN 0
+    ELSE 1
+  END
+  ```
+
+In business terms, a product is considered NOT replenishable only if:
+- It was manually flagged as not replenishable (negative reorder value)
+- OR it shows no activity across ALL metrics (no sales AND no receipts AND no refills in the past 5 years)
+- Image URLs are constructed using this pattern:
+  ```javascript
+  const paddedPid = pid.toString().padStart(6, '0');
+  const prefix = paddedPid.slice(0, 3);
+  const basePath = `${imageUrlBase}${prefix}/${pid}`;
+  return {
+    image: `${basePath}-t-${iid}.jpg`,
+    image_175: `${basePath}-175x175-${iid}.jpg`,
+    image_full: `${basePath}-o-${iid}.jpg`
+  };
+  ```
+
+### Product Categories (Relationship)
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| pid               | products.pid                      | Direct mapping                                                |
+| cat_id            | product_category_index.cat_id     | Direct mapping, filtered by category types                    |
+
+**Notes:**
+- Only categories of types 10, 20, 11, 21, 12, 13 are imported
+- Categories 16 and 17 are explicitly excluded
+
+### Orders
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| order_number      | order_items.order_id              | Direct mapping                                                |
+| pid               | order_items.prod_pid              | Direct mapping                                                |
+| sku               | order_items.prod_itemnumber       | Fallback to 'NO-SKU' if empty                                 |
+| date              | _order.date_placed_onlydate       | Via join to _order table                                      |
+| price             | order_items.prod_price            | Direct mapping                                                |
+| quantity          | order_items.qty_ordered           | Direct mapping                                                |
+| discount          | Multiple sources                  | Complex calculation (see notes)                               |
+| tax               | order_tax_info_products.item_taxes_to_collect | Via latest order_tax_info record                  |
+| tax_included      | N/A                               | Hard-coded false                                              |
+| shipping          | N/A                               | Hard-coded 0                                                  |
+| customer          | _order.order_cid                  | Direct mapping                                                |
+| customer_name     | users                             | CONCAT(users.firstname, ' ', users.lastname)                  |
+| status            | _order.order_status               | Direct mapping                                                |
+| canceled          | _order.date_cancelled             | Boolean: true if date_cancelled is not '0000-00-00 00:00:00'  |
+| costeach          | order_costs                       | From latest record or fallback to price * 0.5                 |
+
+**Notes:**
+- Only orders with order_status >= 15 and with a valid date_placed are processed
+- For incremental imports, only orders modified since last sync are processed
+- Discount calculation combines three sources:
+  1. Base discount: order_items.prod_price_reg - order_items.prod_price
+  2. Promo discount: SUM of order_discount_items.amount
+  3. Proportional order discount: Calculation based on order subtotal proportion
+  ```javascript
+  (oi.base_discount + 
+   COALESCE(ot.promo_discount, 0) + 
+   CASE 
+    WHEN om.summary_discount > 0 AND om.summary_subtotal > 0 THEN 
+      ROUND((om.summary_discount * (oi.price * oi.quantity)) / NULLIF(om.summary_subtotal, 0), 2)
+    ELSE 0 
+   END)::DECIMAL(10,2)
+  ```
+- Taxes are taken from the latest tax record for an order
+- Cost data is taken from the latest non-pending cost record
+
+### Purchase Orders
+
+| PostgreSQL Column | MySQL Source                      | Transformation                                                |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| po_id             | po.po_id                          | Default 0 if NULL                                             |
+| pid               | po_products.pid                   | Direct mapping                                                |
+| sku               | products.itemnumber               | Fallback to 'NO-SKU' if empty                                 |
+| name              | products.description              | Fallback to 'Unknown Product'                                 |
+| cost_price        | po_products.cost_each             | Direct mapping                                                |
+| po_cost_price     | po_products.cost_each             | Duplicate of cost_price                                       |
+| vendor            | suppliers.companyname             | Fallback to 'Unknown Vendor' if empty                         |
+| date              | po.date_ordered                   | Fallback to po.date_created if NULL                           |
+| expected_date     | po.date_estin                     | Direct mapping                                                |
+| status            | po.status                         | Default 1 if NULL                                             |
+| notes             | po.short_note                     | Fallback to po.notes if NULL                                  |
+| ordered           | po_products.qty_each              | Direct mapping                                                |
+| received          | N/A                               | Hard-coded 0                                                  |
+| receiving_status  | N/A                               | Hard-coded 1                                                  |
+
+**Notes:**
+- Only POs created within last 1 year (incremental) or 5 years (full) are processed
+- For incremental imports, only POs modified since last sync are processed
+
+### Metadata Tables
+
+#### import_history
+
+| PostgreSQL Column | Source                            | Notes                                                         |
+|-------------------|-----------------------------------|---------------------------------------------------------------|
+| id                | Auto-increment                    | Primary key                                                   |
+| table_name        | Code                              | 'all_tables' for overall import                               |
+| start_time        | NOW()                             | Import start time                                             |
+| end_time          | NOW()                             | Import completion time                                        |
+| duration_seconds  | Calculation                       | Elapsed seconds                                               |
+| is_incremental    | INCREMENTAL_UPDATE                | Flag from config                                              |
+| records_added     | Calculation                       | Sum from all imports                                          |
+| records_updated   | Calculation                       | Sum from all imports                                          |
+| status            | Code                              | 'running', 'completed', 'failed', or 'cancelled'              |
+| error_message     | Exception                         | Error message if failed                                       |
+| additional_info   | JSON                              | Configuration and results                                     |
+
+#### sync_status
+
+| PostgreSQL Column    | Source                         | Notes                                                         |
+|----------------------|--------------------------------|---------------------------------------------------------------|
+| table_name           | Code                           | Name of imported table                                        |
+| last_sync_timestamp  | NOW()                          | Timestamp of successful sync                                  |
+| last_sync_id         | NULL                           | Not used currently                                            |
+
+## Special Calculations
+
+### Date Validation
+
+MySQL dates are validated before insertion into PostgreSQL:
+
+```javascript
+function validateDate(mysqlDate) {
+  if (!mysqlDate || mysqlDate === '0000-00-00' || mysqlDate === '0000-00-00 00:00:00') {
+    return null;
+  }
+  // Check if the date is valid
+  const date = new Date(mysqlDate);
+  return isNaN(date.getTime()) ? null : mysqlDate;
+}
+```
+
+### Retry Mechanism
+
+Operations that might fail temporarily are retried with exponential backoff:
+
+```javascript
+async function withRetry(operation, errorMessage) {
+  let lastError;
+  for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error;
+      console.error(`${errorMessage} (Attempt ${attempt}/${MAX_RETRIES}):`, error);
+      if (attempt < MAX_RETRIES) {
+        const backoffTime = RETRY_DELAY * Math.pow(2, attempt - 1);
+        await new Promise(resolve => setTimeout(resolve, backoffTime));
+      }
+    }
+  }
+  throw lastError;
+}
+```
+
+### Progress Tracking
+
+Progress is tracked with estimated time remaining:
+
+```javascript
+function estimateRemaining(startTime, current, total) {
+  if (current === 0) return "Calculating...";
+  const elapsedSeconds = (Date.now() - startTime) / 1000;
+  const itemsPerSecond = current / elapsedSeconds;
+  const remainingItems = total - current;
+  const remainingSeconds = remainingItems / itemsPerSecond;
+  return formatElapsedTime(remainingSeconds);
+}
+```
+
+## Implementation Notes
+
+### Transaction Management
+
+All imports use transactions to ensure data consistency:
+
+- **Categories**: Uses savepoints for each category type
+- **Products**: Uses a single transaction for the entire import
+- **Orders**: Uses a single transaction with temporary tables
+- **Purchase Orders**: Uses a single transaction with temporary tables
+
+### Memory Usage Optimization
+
+To minimize memory usage when processing large datasets:
+
+1. Data is processed in batches (100-5000 records per batch)
+2. Temporary tables are used for intermediate data
+3. Some queries use cursors to avoid loading all results at once
+
+### MySQL vs PostgreSQL Compatibility
+
+The scripts handle differences between MySQL and PostgreSQL:
+
+1. MySQL-specific syntax like `USE INDEX` is removed for PostgreSQL
+2. `GROUP_CONCAT` in MySQL becomes string operations in PostgreSQL
+3. Transaction syntax differences are abstracted in the connection wrapper
+4. PostgreSQL's `ON CONFLICT` replaces MySQL's `ON DUPLICATE KEY UPDATE`
+
+### SSH Tunnel
+
+Database connections go through an SSH tunnel for security:
+
+```javascript
+ssh.forwardOut(
+  "127.0.0.1",
+  0,
+  sshConfig.prodDbConfig.host,
+  sshConfig.prodDbConfig.port,
+  async (err, stream) => {
+    if (err) reject(err);
+    resolve({ ssh, stream });
+  }
+);
+``` 

inventory-server/db/schema.sql

@@ -4,7 +4,12 @@ SET session_replication_role = 'replica';  -- Disable foreign key checks tempora
 -- Create function for updating timestamps
 CREATE OR REPLACE FUNCTION update_updated_column() RETURNS TRIGGER AS $func$
 BEGIN
-    NEW.updated = CURRENT_TIMESTAMP;
+    -- Check which table is being updated and use the appropriate column
+    IF TG_TABLE_NAME = 'categories' THEN
+        NEW.updated_at = CURRENT_TIMESTAMP;
+    ELSE
+        NEW.updated = CURRENT_TIMESTAMP;
+    END IF;
     RETURN NEW;
 END;
 $func$ language plpgsql;
@@ -160,7 +165,7 @@ CREATE TABLE purchase_orders (
     expected_date DATE,
     pid BIGINT NOT NULL,
     sku VARCHAR(50) NOT NULL,
-    name VARCHAR(100) NOT NULL,
+    name VARCHAR(255) NOT NULL,
     cost_price DECIMAL(10, 3) NOT NULL,
     po_cost_price DECIMAL(10, 3) NOT NULL,
     status SMALLINT DEFAULT 1,
@@ -171,7 +176,7 @@ CREATE TABLE purchase_orders (
     received INTEGER DEFAULT 0,
     received_date DATE,
     last_received_date DATE,
-    received_by VARCHAR(100),
+    received_by VARCHAR,
     receiving_history JSONB,
     updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
     FOREIGN KEY (pid) REFERENCES products(pid),

inventory-server/db/setup-schema.sql

@@ -23,6 +23,56 @@ CREATE TABLE IF NOT EXISTS templates (
     UNIQUE(company, product_type)
 );
 
+-- AI Prompts table for storing validation prompts
+CREATE TABLE IF NOT EXISTS ai_prompts (
+    id SERIAL PRIMARY KEY,
+    prompt_text TEXT NOT NULL,
+    prompt_type TEXT NOT NULL CHECK (prompt_type IN ('general', 'company_specific', 'system')),
+    company TEXT,
+    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
+    CONSTRAINT unique_company_prompt UNIQUE (company),
+    CONSTRAINT company_required_for_specific CHECK (
+        (prompt_type = 'general' AND company IS NULL) OR 
+        (prompt_type = 'system' AND company IS NULL) OR 
+        (prompt_type = 'company_specific' AND company IS NOT NULL)
+    )
+);
+
+-- Create a unique partial index to ensure only one general prompt
+CREATE UNIQUE INDEX IF NOT EXISTS idx_unique_general_prompt 
+ON ai_prompts (prompt_type) 
+WHERE prompt_type = 'general';
+
+-- Create a unique partial index to ensure only one system prompt
+CREATE UNIQUE INDEX IF NOT EXISTS idx_unique_system_prompt 
+ON ai_prompts (prompt_type) 
+WHERE prompt_type = 'system';
+
+-- Reusable Images table for storing persistent images
+CREATE TABLE IF NOT EXISTS reusable_images (
+    id SERIAL PRIMARY KEY,
+    name TEXT NOT NULL,
+    filename TEXT NOT NULL,
+    file_path TEXT NOT NULL,
+    image_url TEXT NOT NULL,
+    is_global BOOLEAN NOT NULL DEFAULT false,
+    company TEXT,
+    mime_type TEXT,
+    file_size INTEGER,
+    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
+    CONSTRAINT company_required_for_non_global CHECK (
+        (is_global = true AND company IS NULL) OR
+        (is_global = false AND company IS NOT NULL)
+    )
+);
+
+-- Create index on company for efficient querying
+CREATE INDEX IF NOT EXISTS idx_reusable_images_company ON reusable_images(company);
+-- Create index on is_global for efficient querying
+CREATE INDEX IF NOT EXISTS idx_reusable_images_is_global ON reusable_images(is_global);
+
 -- AI Validation Performance Tracking
 CREATE TABLE IF NOT EXISTS ai_validation_performance (
     id SERIAL PRIMARY KEY,
@@ -50,4 +100,16 @@ $$ language 'plpgsql';
 CREATE TRIGGER update_templates_updated_at
     BEFORE UPDATE ON templates
     FOR EACH ROW
+    EXECUTE FUNCTION update_updated_at_column();
+
+-- Trigger to automatically update the updated_at column for ai_prompts
+CREATE TRIGGER update_ai_prompts_updated_at
+    BEFORE UPDATE ON ai_prompts
+    FOR EACH ROW
+    EXECUTE FUNCTION update_updated_at_column();
+    
+-- Trigger to automatically update the updated_at column for reusable_images
+CREATE TRIGGER update_reusable_images_updated_at
+    BEFORE UPDATE ON reusable_images
+    FOR EACH ROW
     EXECUTE FUNCTION update_updated_at_column(); 

inventory-server/scripts/import-from-prod.js

@@ -10,9 +10,9 @@ const importPurchaseOrders = require('./import/purchase-orders');
 dotenv.config({ path: path.join(__dirname, "../.env") });
 
 // Constants to control which imports run
-const IMPORT_CATEGORIES = true;
-const IMPORT_PRODUCTS = true;
-const IMPORT_ORDERS = true;
+const IMPORT_CATEGORIES = false;
+const IMPORT_PRODUCTS = false;
+const IMPORT_ORDERS = false;
 const IMPORT_PURCHASE_ORDERS = true;
 
 // Add flag for incremental updates
@@ -120,27 +120,38 @@ async function main() {
     `);
 
     // Create import history record for the overall session
-    const [historyResult] = await localConnection.query(`
-      INSERT INTO import_history (
-        table_name, 
-        start_time, 
-        is_incremental,
-        status,
-        additional_info
-      ) VALUES (
-        'all_tables',
-        NOW(),
-        $1::boolean,
-        'running',
-        jsonb_build_object(
-          'categories_enabled', $2::boolean,
-          'products_enabled', $3::boolean,
-          'orders_enabled', $4::boolean,
-          'purchase_orders_enabled', $5::boolean
-        )
-      ) RETURNING id
-    `, [INCREMENTAL_UPDATE, IMPORT_CATEGORIES, IMPORT_PRODUCTS, IMPORT_ORDERS, IMPORT_PURCHASE_ORDERS]);
-    importHistoryId = historyResult.rows[0].id;
+    try {
+      const [historyResult] = await localConnection.query(`
+        INSERT INTO import_history (
+          table_name, 
+          start_time, 
+          is_incremental,
+          status,
+          additional_info
+        ) VALUES (
+          'all_tables',
+          NOW(),
+          $1::boolean,
+          'running',
+          jsonb_build_object(
+            'categories_enabled', $2::boolean,
+            'products_enabled', $3::boolean,
+            'orders_enabled', $4::boolean,
+            'purchase_orders_enabled', $5::boolean
+          )
+        ) RETURNING id
+      `, [INCREMENTAL_UPDATE, IMPORT_CATEGORIES, IMPORT_PRODUCTS, IMPORT_ORDERS, IMPORT_PURCHASE_ORDERS]);
+      importHistoryId = historyResult.rows[0].id;
+    } catch (error) {
+      console.error("Error creating import history record:", error);
+      outputProgress({
+        status: "error",
+        operation: "Import process",
+        message: "Failed to create import history record",
+        error: error.message
+      });
+      throw error;
+    }
 
     const results = {
       categories: null,
@@ -158,8 +169,8 @@ async function main() {
       if (isImportCancelled) throw new Error("Import cancelled");
       completedSteps++;
       console.log('Categories import result:', results.categories);
-      totalRecordsAdded += parseInt(results.categories?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.categories?.recordsUpdated || 0);
+      totalRecordsAdded += parseInt(results.categories?.recordsAdded || 0) || 0;
+      totalRecordsUpdated += parseInt(results.categories?.recordsUpdated || 0) || 0;
     }
 
     if (IMPORT_PRODUCTS) {
@@ -167,8 +178,8 @@ async function main() {
       if (isImportCancelled) throw new Error("Import cancelled");
       completedSteps++;
       console.log('Products import result:', results.products);
-      totalRecordsAdded += parseInt(results.products?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.products?.recordsUpdated || 0);
+      totalRecordsAdded += parseInt(results.products?.recordsAdded || 0) || 0;
+      totalRecordsUpdated += parseInt(results.products?.recordsUpdated || 0) || 0;
     }
 
     if (IMPORT_ORDERS) {
@@ -176,17 +187,34 @@ async function main() {
       if (isImportCancelled) throw new Error("Import cancelled");
       completedSteps++;
       console.log('Orders import result:', results.orders);
-      totalRecordsAdded += parseInt(results.orders?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.orders?.recordsUpdated || 0);
+      totalRecordsAdded += parseInt(results.orders?.recordsAdded || 0) || 0;
+      totalRecordsUpdated += parseInt(results.orders?.recordsUpdated || 0) || 0;
     }
 
     if (IMPORT_PURCHASE_ORDERS) {
-      results.purchaseOrders = await importPurchaseOrders(prodConnection, localConnection, INCREMENTAL_UPDATE);
-      if (isImportCancelled) throw new Error("Import cancelled");
-      completedSteps++;
-      console.log('Purchase orders import result:', results.purchaseOrders);
-      totalRecordsAdded += parseInt(results.purchaseOrders?.recordsAdded || 0);
-      totalRecordsUpdated += parseInt(results.purchaseOrders?.recordsUpdated || 0);
+      try {
+        results.purchaseOrders = await importPurchaseOrders(prodConnection, localConnection, INCREMENTAL_UPDATE);
+        if (isImportCancelled) throw new Error("Import cancelled");
+        completedSteps++;
+        console.log('Purchase orders import result:', results.purchaseOrders);
+        
+        // Handle potential error status
+        if (results.purchaseOrders?.status === 'error') {
+          console.error('Purchase orders import had an error:', results.purchaseOrders.error);
+        } else {
+          totalRecordsAdded += parseInt(results.purchaseOrders?.recordsAdded || 0) || 0;
+          totalRecordsUpdated += parseInt(results.purchaseOrders?.recordsUpdated || 0) || 0;
+        }
+      } catch (error) {
+        console.error('Error during purchase orders import:', error);
+        // Continue with other imports, don't fail the whole process
+        results.purchaseOrders = {
+          status: 'error',
+          error: error.message,
+          recordsAdded: 0,
+          recordsUpdated: 0
+        };
+      }
     }
 
     const endTime = Date.now();
@@ -214,8 +242,8 @@ async function main() {
       WHERE id = $12
     `, [
       totalElapsedSeconds,
-      totalRecordsAdded,
-      totalRecordsUpdated,
+      parseInt(totalRecordsAdded) || 0,
+      parseInt(totalRecordsUpdated) || 0,
       IMPORT_CATEGORIES,
       IMPORT_PRODUCTS,
       IMPORT_ORDERS,

inventory-server/scripts/import/categories.js

@@ -47,42 +47,18 @@ async function importCategories(prodConnection, localConnection) {
           continue;
         }
 
-        console.log(`\nProcessing ${categories.length} type ${type} categories`);
-        if (type === 10) {
-          console.log("Type 10 categories:", JSON.stringify(categories, null, 2));
-        }
+        console.log(`Processing ${categories.length} type ${type} categories`);
 
-        // For types that can have parents (11, 21, 12, 13), verify parent existence
+        // For types that can have parents (11, 21, 12, 13), we'll proceed directly
+        // No need to check for parent existence since we process in hierarchical order
         let categoriesToInsert = categories;
-        if (![10, 20].includes(type)) {
-          // Get all parent IDs
-          const parentIds = [
-            ...new Set(
-              categories
-                .filter(c => c && c.parent_id !== null)
-                .map(c => c.parent_id)
-            ),
-          ];
-
-          console.log(`Processing ${categories.length} type ${type} categories with ${parentIds.length} unique parent IDs`);
-          console.log('Parent IDs:', parentIds);
-
-          // No need to check for parent existence - we trust they exist since they were just inserted
-          categoriesToInsert = categories;
-        }
 
         if (categoriesToInsert.length === 0) {
-          console.log(
-            `No valid categories of type ${type} to insert`
-          );
+          console.log(`No valid categories of type ${type} to insert`);
           await localConnection.query(`RELEASE SAVEPOINT category_type_${type}`);
           continue;
         }
 
-        console.log(
-          `Inserting ${categoriesToInsert.length} type ${type} categories`
-        );
-
         // PostgreSQL upsert query with parameterized values
         const values = categoriesToInsert.flatMap((cat) => [
           cat.cat_id,
@@ -95,14 +71,10 @@ async function importCategories(prodConnection, localConnection) {
           new Date()
         ]);
 
-        console.log('Attempting to insert/update with values:', JSON.stringify(values, null, 2));
-
         const placeholders = categoriesToInsert
           .map((_, i) => `($${i * 8 + 1}, $${i * 8 + 2}, $${i * 8 + 3}, $${i * 8 + 4}, $${i * 8 + 5}, $${i * 8 + 6}, $${i * 8 + 7}, $${i * 8 + 8})`)
           .join(',');
 
-        console.log('Using placeholders:', placeholders);
-
         // Insert categories with ON CONFLICT clause for PostgreSQL
         const query = `
           WITH inserted_categories AS (
@@ -129,17 +101,14 @@ async function importCategories(prodConnection, localConnection) {
             COUNT(*) FILTER (WHERE is_insert) as inserted,
             COUNT(*) FILTER (WHERE NOT is_insert) as updated
           FROM inserted_categories`;
-
-        console.log('Executing query:', query);
         
         const result = await localConnection.query(query, values);
-        console.log('Query result:', result);
 
         // Get the first result since query returns an array
         const queryResult = Array.isArray(result) ? result[0] : result;
 
         if (!queryResult || !queryResult.rows || !queryResult.rows[0]) {
-          console.error('Query failed to return results. Result:', queryResult);
+          console.error('Query failed to return results');
           throw new Error('Query did not return expected results');
         }
 
@@ -173,6 +142,14 @@ async function importCategories(prodConnection, localConnection) {
     // Commit the entire transaction - we'll do this even if we have skipped categories
     await localConnection.query('COMMIT');
 
+    // Update sync status
+    await localConnection.query(`
+      INSERT INTO sync_status (table_name, last_sync_timestamp)
+      VALUES ('categories', NOW())
+      ON CONFLICT (table_name) DO UPDATE SET
+        last_sync_timestamp = NOW()
+    `);
+
     outputProgress({
       status: "complete",
       operation: "Categories import completed",

inventory-server/scripts/import/orders.js

@@ -26,6 +26,9 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
   let cumulativeProcessedOrders = 0;
 
   try {
+    // Begin transaction
+    await localConnection.beginTransaction();
+    
     // Get last sync info
     const [syncInfo] = await localConnection.query(
       "SELECT last_sync_timestamp FROM sync_status WHERE table_name = 'orders'"
@@ -38,7 +41,6 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
     const [[{ total }]] = await prodConnection.query(`
       SELECT COUNT(*) as total
       FROM order_items oi
-      USE INDEX (PRIMARY)
       JOIN _order o ON oi.order_id = o.order_id
       WHERE o.order_status >= 15
         AND o.date_placed_onlydate >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
@@ -78,7 +80,6 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         COALESCE(oi.prod_price_reg - oi.prod_price, 0) as base_discount,
         oi.stamp as last_modified
       FROM order_items oi
-      USE INDEX (PRIMARY)
       JOIN _order o ON oi.order_id = o.order_id
       WHERE o.order_status >= 15
         AND o.date_placed_onlydate >= DATE_SUB(CURRENT_DATE, INTERVAL ${incrementalUpdate ? '1' : '5'} YEAR)
@@ -105,15 +106,15 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
 
     console.log('Orders: Found', orderItems.length, 'order items to process');
 
-    // Create tables in PostgreSQL for debugging
+    // Create tables in PostgreSQL for data processing
     await localConnection.query(`
-      DROP TABLE IF EXISTS debug_order_items;
-      DROP TABLE IF EXISTS debug_order_meta;
-      DROP TABLE IF EXISTS debug_order_discounts;
-      DROP TABLE IF EXISTS debug_order_taxes;
-      DROP TABLE IF EXISTS debug_order_costs;
+      DROP TABLE IF EXISTS temp_order_items;
+      DROP TABLE IF EXISTS temp_order_meta;
+      DROP TABLE IF EXISTS temp_order_discounts;
+      DROP TABLE IF EXISTS temp_order_taxes;
+      DROP TABLE IF EXISTS temp_order_costs;
 
-      CREATE TABLE debug_order_items (
+      CREATE TEMP TABLE temp_order_items (
         order_id INTEGER NOT NULL,
         pid INTEGER NOT NULL,
         SKU VARCHAR(50) NOT NULL,
@@ -123,7 +124,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         PRIMARY KEY (order_id, pid)
       );
 
-      CREATE TABLE debug_order_meta (
+      CREATE TEMP TABLE temp_order_meta (
         order_id INTEGER NOT NULL,
         date DATE NOT NULL,
         customer VARCHAR(100) NOT NULL,
@@ -135,26 +136,29 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         PRIMARY KEY (order_id)
       );
 
-      CREATE TABLE debug_order_discounts (
+      CREATE TEMP TABLE temp_order_discounts (
         order_id INTEGER NOT NULL,
         pid INTEGER NOT NULL,
         discount DECIMAL(10,2) NOT NULL,
         PRIMARY KEY (order_id, pid)
       );
 
-      CREATE TABLE debug_order_taxes (
+      CREATE TEMP TABLE temp_order_taxes (
         order_id INTEGER NOT NULL,
         pid INTEGER NOT NULL,
         tax DECIMAL(10,2) NOT NULL,
         PRIMARY KEY (order_id, pid)
       );
 
-      CREATE TABLE debug_order_costs (
+      CREATE TEMP TABLE temp_order_costs (
         order_id INTEGER NOT NULL,
         pid INTEGER NOT NULL,
         costeach DECIMAL(10,3) DEFAULT 0.000,
         PRIMARY KEY (order_id, pid)
       );
+
+      CREATE INDEX idx_temp_order_items_pid ON temp_order_items(pid);
+      CREATE INDEX idx_temp_order_meta_order_id ON temp_order_meta(order_id);
     `);
 
     // Insert order items in batches
@@ -168,7 +172,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
       ]);
 
       await localConnection.query(`
-        INSERT INTO debug_order_items (order_id, pid, SKU, price, quantity, base_discount)
+        INSERT INTO temp_order_items (order_id, pid, SKU, price, quantity, base_discount)
         VALUES ${placeholders}
         ON CONFLICT (order_id, pid) DO UPDATE SET
           SKU = EXCLUDED.SKU,
@@ -202,6 +206,14 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
     const METADATA_BATCH_SIZE = 2000;
     const PG_BATCH_SIZE = 200;
     
+    // Add a helper function for title case conversion
+    function toTitleCase(str) {
+      if (!str) return '';
+      return str.toLowerCase().split(' ').map(word => {
+        return word.charAt(0).toUpperCase() + word.slice(1);
+      }).join(' ');
+    }
+
     const processMetadataBatch = async (batchIds) => {
       const [orders] = await prodConnection.query(`
         SELECT 
@@ -231,7 +243,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
           order.order_id,
           order.date,
           order.customer,
-          order.customer_name || '',
+          toTitleCase(order.customer_name) || '',
           order.status,
           order.canceled,
           order.summary_discount || 0,
@@ -239,7 +251,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         ]);
 
         await localConnection.query(`
-          INSERT INTO debug_order_meta (
+          INSERT INTO temp_order_meta (
             order_id, date, customer, customer_name, status, canceled,
             summary_discount, summary_subtotal
           )
@@ -281,7 +293,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         ]);
 
         await localConnection.query(`
-          INSERT INTO debug_order_discounts (order_id, pid, discount)
+          INSERT INTO temp_order_discounts (order_id, pid, discount)
           VALUES ${placeholders}
           ON CONFLICT (order_id, pid) DO UPDATE SET
             discount = EXCLUDED.discount
@@ -321,7 +333,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         ]);
 
         await localConnection.query(`
-          INSERT INTO debug_order_taxes (order_id, pid, tax)
+          INSERT INTO temp_order_taxes (order_id, pid, tax)
           VALUES ${placeholders}
           ON CONFLICT (order_id, pid) DO UPDATE SET
             tax = EXCLUDED.tax
@@ -330,14 +342,23 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
     };
 
     const processCostsBatch = async (batchIds) => {
+      // Modified query to ensure one row per order_id/pid by using a subquery
       const [costs] = await prodConnection.query(`
         SELECT 
           oc.orderid as order_id,
           oc.pid,
           oc.costeach
         FROM order_costs oc
-        WHERE oc.orderid IN (?)
-          AND oc.pending = 0
+        INNER JOIN (
+          SELECT 
+            orderid,
+            pid, 
+            MAX(id) as max_id
+          FROM order_costs
+          WHERE orderid IN (?) 
+            AND pending = 0
+          GROUP BY orderid, pid
+        ) latest ON oc.orderid = latest.orderid AND oc.pid = latest.pid AND oc.id = latest.max_id
       `, [batchIds]);
 
       if (costs.length === 0) return;
@@ -357,7 +378,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
         ]);
 
         await localConnection.query(`
-          INSERT INTO debug_order_costs (order_id, pid, costeach)
+          INSERT INTO temp_order_costs (order_id, pid, costeach)
           VALUES ${placeholders}
           ON CONFLICT (order_id, pid) DO UPDATE SET
             costeach = EXCLUDED.costeach
@@ -416,11 +437,12 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
               oi.pid,
               SUM(COALESCE(od.discount, 0)) as promo_discount,
               COALESCE(ot.tax, 0) as total_tax,
-              COALESCE(oi.price * 0.5, 0) as costeach
-            FROM debug_order_items oi
-            LEFT JOIN debug_order_discounts od ON oi.order_id = od.order_id AND oi.pid = od.pid
-            LEFT JOIN debug_order_taxes ot ON oi.order_id = ot.order_id AND oi.pid = ot.pid
-            GROUP BY oi.order_id, oi.pid, ot.tax
+              COALESCE(oc.costeach, oi.price * 0.5) as costeach
+            FROM temp_order_items oi
+            LEFT JOIN temp_order_discounts od ON oi.order_id = od.order_id AND oi.pid = od.pid
+            LEFT JOIN temp_order_taxes ot ON oi.order_id = ot.order_id AND oi.pid = ot.pid
+            LEFT JOIN temp_order_costs oc ON oi.order_id = oc.order_id AND oi.pid = oc.pid
+            GROUP BY oi.order_id, oi.pid, ot.tax, oc.costeach
           )
           SELECT 
             oi.order_id as order_number,
@@ -447,11 +469,11 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
           FROM (
             SELECT DISTINCT ON (order_id, pid)
               order_id, pid, SKU, price, quantity, base_discount
-            FROM debug_order_items
+            FROM temp_order_items
             WHERE order_id = ANY($1)
             ORDER BY order_id, pid
           ) oi
-          JOIN debug_order_meta om ON oi.order_id = om.order_id
+          JOIN temp_order_meta om ON oi.order_id = om.order_id
           LEFT JOIN order_totals ot ON oi.order_id = ot.order_id AND oi.pid = ot.pid
           ORDER BY oi.order_id, oi.pid
         `, [subBatchIds]);
@@ -478,8 +500,8 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
           const subBatch = validOrders.slice(k, k + FINAL_BATCH_SIZE);
           
           const placeholders = subBatch.map((_, idx) => {
-            const base = idx * 14;  // 14 columns (removed updated)
-            return `($${base + 1}, $${base + 2}, $${base + 3}, $${base + 4}, $${base + 5}, $${base + 6}, $${base + 7}, $${base + 8}, $${base + 9}, $${base + 10}, $${base + 11}, $${base + 12}, $${base + 13}, $${base + 14})`;
+            const base = idx * 15;  // 15 columns including costeach
+            return `($${base + 1}, $${base + 2}, $${base + 3}, $${base + 4}, $${base + 5}, $${base + 6}, $${base + 7}, $${base + 8}, $${base + 9}, $${base + 10}, $${base + 11}, $${base + 12}, $${base + 13}, $${base + 14}, $${base + 15})`;
           }).join(',');
 
           const batchValues = subBatch.flatMap(o => [
@@ -496,7 +518,8 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
             o.customer,
             o.customer_name,
             o.status,
-            o.canceled
+            o.canceled,
+            o.costeach
           ]);
 
           const [result] = await localConnection.query(`
@@ -504,7 +527,7 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
               INSERT INTO orders (
                 order_number, pid, sku, date, price, quantity, discount,
                 tax, tax_included, shipping, customer, customer_name,
-                status, canceled
+                status, canceled, costeach
               )
               VALUES ${placeholders}
               ON CONFLICT (order_number, pid) DO UPDATE SET
@@ -519,7 +542,8 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
                 customer = EXCLUDED.customer,
                 customer_name = EXCLUDED.customer_name,
                 status = EXCLUDED.status,
-                canceled = EXCLUDED.canceled
+                canceled = EXCLUDED.canceled,
+                costeach = EXCLUDED.costeach
               RETURNING xmax = 0 as inserted
             )
             SELECT 
@@ -529,8 +553,8 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
           `, batchValues);
           
           const { inserted, updated } = result.rows[0];
-          recordsAdded += inserted;
-          recordsUpdated += updated;
+          recordsAdded += parseInt(inserted) || 0;
+          recordsUpdated += parseInt(updated) || 0;
           importedCount += subBatch.length;
         }
 
@@ -555,19 +579,39 @@ async function importOrders(prodConnection, localConnection, incrementalUpdate =
       ON CONFLICT (table_name) DO UPDATE SET
         last_sync_timestamp = NOW()
     `);
+    
+    // Cleanup temporary tables
+    await localConnection.query(`
+      DROP TABLE IF EXISTS temp_order_items;
+      DROP TABLE IF EXISTS temp_order_meta;
+      DROP TABLE IF EXISTS temp_order_discounts;
+      DROP TABLE IF EXISTS temp_order_taxes;
+      DROP TABLE IF EXISTS temp_order_costs;
+    `);
+    
+    // Commit transaction
+    await localConnection.commit();
 
     return {
       status: "complete",
-      totalImported: Math.floor(importedCount),
-      recordsAdded: recordsAdded || 0,
-      recordsUpdated: Math.floor(recordsUpdated),
-      totalSkipped: skippedOrders.size,
-      missingProducts: missingProducts.size,
+      totalImported: Math.floor(importedCount) || 0,
+      recordsAdded: parseInt(recordsAdded) || 0,
+      recordsUpdated: parseInt(recordsUpdated) || 0,
+      totalSkipped: skippedOrders.size || 0,
+      missingProducts: missingProducts.size || 0,
       incrementalUpdate,
       lastSyncTime
     };
   } catch (error) {
     console.error("Error during orders import:", error);
+    
+    // Rollback transaction
+    try {
+      await localConnection.rollback();
+    } catch (rollbackError) {
+      console.error("Error during rollback:", rollbackError);
+    }
+    
     throw error;
   }
 }

inventory-server/scripts/import/products.js

@@ -1,10 +1,13 @@
 const { outputProgress, formatElapsedTime, estimateRemaining, calculateRate } = require('../metrics/utils/progress');
-const BATCH_SIZE = 100; // Smaller batch size for better progress tracking
+const BATCH_SIZE = 1000; // Smaller batch size for better progress tracking
 const MAX_RETRIES = 3;
 const RETRY_DELAY = 5000; // 5 seconds
+const dotenv = require("dotenv");
+const path = require("path");
+dotenv.config({ path: path.join(__dirname, "../../.env") });
 
 // Utility functions
-const imageUrlBase = 'https://sbing.com/i/products/0000/';
+const imageUrlBase = process.env.PRODUCT_IMAGE_URL_BASE || 'https://sbing.com/i/products/0000/';
 const getImageUrls = (pid, iid = 1) => {
   const paddedPid = pid.toString().padStart(6, '0');
   // Use padded PID only for the first 3 digits
@@ -18,7 +21,7 @@ const getImageUrls = (pid, iid = 1) => {
   };
 };
 
-// Add helper function for retrying operations
+// Add helper function for retrying operations with exponential backoff
 async function withRetry(operation, errorMessage) {
   let lastError;
   for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
@@ -28,7 +31,8 @@ async function withRetry(operation, errorMessage) {
       lastError = error;
       console.error(`${errorMessage} (Attempt ${attempt}/${MAX_RETRIES}):`, error);
       if (attempt < MAX_RETRIES) {
-        await new Promise(resolve => setTimeout(resolve, RETRY_DELAY));
+        const backoffTime = RETRY_DELAY * Math.pow(2, attempt - 1);
+        await new Promise(resolve => setTimeout(resolve, backoffTime));
       }
     }
   }
@@ -140,10 +144,12 @@ async function importMissingProducts(prodConnection, localConnection, missingPid
         CASE WHEN si.show + si.buyable > 0 THEN 1 ELSE 0 END AS visible,
         CASE 
           WHEN p.reorder < 0 THEN 0
+          WHEN p.date_created >= DATE_SUB(CURRENT_DATE, INTERVAL 1 YEAR) THEN 1
+          WHEN COALESCE(pnb.inventory, 0) > 0 THEN 1
           WHEN (
             (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
-            OR (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-            OR (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+            AND (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+            AND (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
           ) THEN 0
           ELSE 1
         END AS replenishable,
@@ -155,7 +161,11 @@ async function importMissingProducts(prodConnection, localConnection, missingPid
         COALESCE(p.sellingprice, 0) AS regular_price,
         CASE 
           WHEN EXISTS (SELECT 1 FROM product_inventory WHERE pid = p.pid AND count > 0) 
-          THEN (SELECT ROUND(AVG(costeach), 5) FROM product_inventory WHERE pid = p.pid AND count > 0)
+          THEN (
+            SELECT ROUND(SUM(costeach * count) / SUM(count), 5) 
+            FROM product_inventory 
+            WHERE pid = p.pid AND count > 0
+          )
           ELSE (SELECT costeach FROM product_inventory WHERE pid = p.pid ORDER BY daterec DESC LIMIT 1)
         END AS cost_price,
         NULL as landing_cost_price,
@@ -183,7 +193,7 @@ async function importMissingProducts(prodConnection, localConnection, missingPid
         p.country_of_origin,
         (SELECT COUNT(*) FROM mybasket mb WHERE mb.item = p.pid AND mb.qty > 0) AS baskets,
         (SELECT COUNT(*) FROM product_notify pn WHERE pn.pid = p.pid) AS notifies,
-        p.totalsold AS total_sold,
+        (SELECT COALESCE(SUM(oi.qty_ordered), 0) FROM order_items oi WHERE oi.prod_pid = p.pid) AS total_sold,
         pls.date_sold as date_last_sold,
         GROUP_CONCAT(DISTINCT CASE 
           WHEN pc.cat_id IS NOT NULL 
@@ -233,7 +243,7 @@ async function importMissingProducts(prodConnection, localConnection, missingPid
           row.pid,
           row.title,
           row.description,
-          row.itemnumber || '',
+          row.sku || '',
           row.stock_quantity > 5000 ? 0 : Math.max(0, row.stock_quantity),
           row.preorder_count,
           row.notions_inv_count,
@@ -335,10 +345,12 @@ async function materializeCalculations(prodConnection, localConnection, incremen
       CASE WHEN si.show + si.buyable > 0 THEN 1 ELSE 0 END AS visible,
       CASE 
         WHEN p.reorder < 0 THEN 0
+        WHEN p.date_created >= DATE_SUB(CURRENT_DATE, INTERVAL 1 YEAR) THEN 1
+        WHEN COALESCE(pnb.inventory, 0) > 0 THEN 1
         WHEN (
           (COALESCE(pls.date_sold, '0000-00-00') = '0000-00-00' OR pls.date_sold <= DATE_SUB(CURRENT_DATE, INTERVAL 5 YEAR))
-          OR (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
-          OR (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+          AND (p.datein = '0000-00-00 00:00:00' OR p.datein <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
+          AND (p.date_refill = '0000-00-00 00:00:00' OR p.date_refill <= DATE_SUB(CURRENT_TIMESTAMP, INTERVAL 5 YEAR))
         ) THEN 0
         ELSE 1
       END AS replenishable,
@@ -350,7 +362,11 @@ async function materializeCalculations(prodConnection, localConnection, incremen
       COALESCE(p.sellingprice, 0) AS regular_price,
       CASE 
         WHEN EXISTS (SELECT 1 FROM product_inventory WHERE pid = p.pid AND count > 0) 
-        THEN (SELECT ROUND(AVG(costeach), 5) FROM product_inventory WHERE pid = p.pid AND count > 0)
+        THEN (
+          SELECT ROUND(SUM(costeach * count) / SUM(count), 5) 
+          FROM product_inventory 
+          WHERE pid = p.pid AND count > 0
+        )
         ELSE (SELECT costeach FROM product_inventory WHERE pid = p.pid ORDER BY daterec DESC LIMIT 1)
       END AS cost_price,
       NULL as landing_cost_price,
@@ -378,7 +394,7 @@ async function materializeCalculations(prodConnection, localConnection, incremen
       p.country_of_origin,
       (SELECT COUNT(*) FROM mybasket mb WHERE mb.item = p.pid AND mb.qty > 0) AS baskets,
       (SELECT COUNT(*) FROM product_notify pn WHERE pn.pid = p.pid) AS notifies,
-      p.totalsold AS total_sold,
+      (SELECT COALESCE(SUM(oi.qty_ordered), 0) FROM order_items oi WHERE oi.prod_pid = p.pid) AS total_sold,
       pls.date_sold as date_last_sold,
       GROUP_CONCAT(DISTINCT CASE 
         WHEN pc.cat_id IS NOT NULL 
@@ -432,7 +448,7 @@ async function materializeCalculations(prodConnection, localConnection, incremen
           row.pid,
           row.title,
           row.description,
-          row.itemnumber || '',
+          row.sku || '',
           row.stock_quantity > 5000 ? 0 : Math.max(0, row.stock_quantity),
           row.preorder_count,
           row.notions_inv_count,
@@ -772,32 +788,44 @@ async function importProducts(prodConnection, localConnection, incrementalUpdate
         recordsAdded += parseInt(result.rows[0].inserted, 10) || 0;
         recordsUpdated += parseInt(result.rows[0].updated, 10) || 0;
 
-        // Process category relationships for each product in the batch
+        // Process category relationships in batches
+        const allCategories = [];
         for (const row of batch) {
           if (row.categories) {
             const categoryIds = row.categories.split(',').filter(id => id && id.trim());
             if (categoryIds.length > 0) {
-              const catPlaceholders = categoryIds.map((_, idx) => 
-                `($${idx * 2 + 1}, $${idx * 2 + 2})`
-              ).join(',');
-              const catValues = categoryIds.flatMap(catId => [row.pid, parseInt(catId.trim(), 10)]);
-              
-              // First delete existing relationships for this product
-              await localConnection.query(
-                'DELETE FROM product_categories WHERE pid = $1',
-                [row.pid]
-              );
-              
-              // Then insert the new relationships
-              await localConnection.query(`
-                INSERT INTO product_categories (pid, cat_id)
-                VALUES ${catPlaceholders}
-                ON CONFLICT (pid, cat_id) DO NOTHING
-              `, catValues);
+              categoryIds.forEach(catId => {
+                allCategories.push([row.pid, parseInt(catId.trim(), 10)]);
+              });
             }
           }
         }
 
+        // If we have categories to process
+        if (allCategories.length > 0) {
+          // First get all products in this batch
+          const productIds = batch.map(p => p.pid);
+          
+          // Delete all existing relationships for products in this batch
+          await localConnection.query(
+            'DELETE FROM product_categories WHERE pid = ANY($1)',
+            [productIds]
+          );
+          
+          // Insert all new relationships in one batch
+          const catPlaceholders = allCategories.map((_, idx) => 
+            `($${idx * 2 + 1}, $${idx * 2 + 2})`
+          ).join(',');
+          
+          const catValues = allCategories.flat();
+          
+          await localConnection.query(`
+            INSERT INTO product_categories (pid, cat_id)
+            VALUES ${catPlaceholders}
+            ON CONFLICT (pid, cat_id) DO NOTHING
+          `, catValues);
+        }
+
         outputProgress({
           status: "running",
           operation: "Products import",
@@ -816,6 +844,14 @@ async function importProducts(prodConnection, localConnection, incrementalUpdate
       // Commit the transaction
       await localConnection.commit();
 
+      // Update sync status
+      await localConnection.query(`
+        INSERT INTO sync_status (table_name, last_sync_timestamp)
+        VALUES ('products', NOW())
+        ON CONFLICT (table_name) DO UPDATE SET
+          last_sync_timestamp = NOW()
+      `);
+
       return {
         status: 'complete',
         recordsAdded,

inventory-server/scripts/reset-db.js

@@ -184,7 +184,7 @@ async function resetDatabase() {
             SELECT string_agg(tablename, ', ') as tables
             FROM pg_tables
             WHERE schemaname = 'public'
-            AND tablename NOT IN ('users', 'permissions', 'user_permissions', 'calculate_history', 'import_history');
+            AND tablename NOT IN ('users', 'permissions', 'user_permissions', 'calculate_history', 'import_history', 'ai_prompts', 'ai_validation_performance', 'templates');
         `);
 
         if (!tablesResult.rows[0].tables) {

inventory-server/scripts/update-order-costs.js

@@ -0,0 +1,337 @@
+/**
+ * This script updates the costeach values for existing orders from the original MySQL database
+ * without needing to run the full import process.
+ */
+const dotenv = require("dotenv");
+const path = require("path");
+const fs = require("fs");
+const { setupConnections, closeConnections } = require('./import/utils');
+const { outputProgress, formatElapsedTime } = require('./metrics/utils/progress');
+
+dotenv.config({ path: path.join(__dirname, "../.env") });
+
+// SSH configuration
+const sshConfig = {
+  ssh: {
+    host: process.env.PROD_SSH_HOST,
+    port: process.env.PROD_SSH_PORT || 22,
+    username: process.env.PROD_SSH_USER,
+    privateKey: process.env.PROD_SSH_KEY_PATH
+      ? fs.readFileSync(process.env.PROD_SSH_KEY_PATH)
+      : undefined,
+    compress: true, // Enable SSH compression
+  },
+  prodDbConfig: {
+    // MySQL config for production
+    host: process.env.PROD_DB_HOST || "localhost",
+    user: process.env.PROD_DB_USER,
+    password: process.env.PROD_DB_PASSWORD,
+    database: process.env.PROD_DB_NAME,
+    port: process.env.PROD_DB_PORT || 3306,
+    timezone: 'Z',
+  },
+  localDbConfig: {
+    // PostgreSQL config for local
+    host: process.env.DB_HOST,
+    user: process.env.DB_USER,
+    password: process.env.DB_PASSWORD,
+    database: process.env.DB_NAME,
+    port: process.env.DB_PORT || 5432,
+    ssl: process.env.DB_SSL === 'true',
+    connectionTimeoutMillis: 60000,
+    idleTimeoutMillis: 30000,
+    max: 10 // connection pool max size
+  }
+};
+
+async function updateOrderCosts() {
+  const startTime = Date.now();
+  let connections;
+  let updatedCount = 0;
+  let errorCount = 0;
+
+  try {
+    outputProgress({
+      status: "running",
+      operation: "Order costs update",
+      message: "Initializing SSH tunnel..."
+    });
+
+    connections = await setupConnections(sshConfig);
+    const { prodConnection, localConnection } = connections;
+
+    // 1. Get all orders from local database that need cost updates
+    outputProgress({
+      status: "running",
+      operation: "Order costs update",
+      message: "Getting orders from local database..."
+    });
+
+    const [orders] = await localConnection.query(`
+      SELECT DISTINCT order_number, pid 
+      FROM orders 
+      WHERE costeach = 0 OR costeach IS NULL
+      ORDER BY order_number
+    `);
+
+    if (!orders || !orders.rows || orders.rows.length === 0) {
+      console.log("No orders found that need cost updates");
+      return { updatedCount: 0, errorCount: 0 };
+    }
+
+    const totalOrders = orders.rows.length;
+    console.log(`Found ${totalOrders} orders that need cost updates`);
+
+    // Process in batches of 1000 orders
+    const BATCH_SIZE = 500;
+    for (let i = 0; i < orders.rows.length; i += BATCH_SIZE) {
+      try {
+        // Start transaction for this batch
+        await localConnection.beginTransaction();
+        
+        const batch = orders.rows.slice(i, i + BATCH_SIZE);
+        
+        const orderNumbers = [...new Set(batch.map(o => o.order_number))];
+        
+        // 2. Fetch costs from production database for these orders
+        outputProgress({
+          status: "running",
+          operation: "Order costs update",
+          message: `Fetching costs for orders ${i + 1} to ${Math.min(i + BATCH_SIZE, totalOrders)} of ${totalOrders}`,
+          current: i,
+          total: totalOrders,
+          elapsed: formatElapsedTime((Date.now() - startTime) / 1000)
+        });
+
+        const [costs] = await prodConnection.query(`
+          SELECT 
+            oc.orderid as order_number,
+            oc.pid,
+            oc.costeach
+          FROM order_costs oc
+          INNER JOIN (
+            SELECT 
+              orderid,
+              pid, 
+              MAX(id) as max_id
+            FROM order_costs
+            WHERE orderid IN (?) 
+              AND pending = 0
+            GROUP BY orderid, pid
+          ) latest ON oc.orderid = latest.orderid AND oc.pid = latest.pid AND oc.id = latest.max_id
+        `, [orderNumbers]);
+
+        // Create a map of costs for easy lookup
+        const costMap = {};
+        if (costs && costs.length) {
+          costs.forEach(c => {
+            costMap[`${c.order_number}-${c.pid}`] = c.costeach || 0;
+          });
+        }
+
+        // 3. Update costs in local database by batches
+        // Using a more efficient update approach with a temporary table
+        
+        // Create a temporary table for each batch
+        await localConnection.query(`
+          DROP TABLE IF EXISTS temp_order_costs;
+          CREATE TEMP TABLE temp_order_costs (
+            order_number VARCHAR(50) NOT NULL,
+            pid BIGINT NOT NULL,
+            costeach DECIMAL(10,3) NOT NULL,
+            PRIMARY KEY (order_number, pid)
+          );
+        `);
+        
+        // Insert cost data into the temporary table
+        const costEntries = [];
+        for (const order of batch) {
+          const key = `${order.order_number}-${order.pid}`;
+          if (key in costMap) {
+            costEntries.push({
+              order_number: order.order_number,
+              pid: order.pid,
+              costeach: costMap[key]
+            });
+          }
+        }
+        
+        // Insert in sub-batches of 100
+        const DB_BATCH_SIZE = 50;
+        for (let j = 0; j < costEntries.length; j += DB_BATCH_SIZE) {
+          const subBatch = costEntries.slice(j, j + DB_BATCH_SIZE);
+          if (subBatch.length === 0) continue;
+          
+          const placeholders = subBatch.map((_, idx) => 
+            `($${idx * 3 + 1}, $${idx * 3 + 2}, $${idx * 3 + 3})`
+          ).join(',');
+          
+          const values = subBatch.flatMap(item => [
+            item.order_number,
+            item.pid,
+            item.costeach
+          ]);
+          
+          await localConnection.query(`
+            INSERT INTO temp_order_costs (order_number, pid, costeach)
+            VALUES ${placeholders}
+          `, values);
+        }
+        
+        // Perform bulk update from the temporary table
+        const [updateResult] = await localConnection.query(`
+          UPDATE orders o
+          SET costeach = t.costeach
+          FROM temp_order_costs t
+          WHERE o.order_number = t.order_number AND o.pid = t.pid
+          RETURNING o.id
+        `);
+        
+        const batchUpdated = updateResult.rowCount || 0;
+        updatedCount += batchUpdated;
+
+        // Commit transaction for this batch
+        await localConnection.commit();
+
+        outputProgress({
+          status: "running",
+          operation: "Order costs update",
+          message: `Updated ${updatedCount} orders with costs from production (batch: ${batchUpdated})`,
+          current: i + batch.length,
+          total: totalOrders,
+          elapsed: formatElapsedTime((Date.now() - startTime) / 1000)
+        });
+      } catch (error) {
+        // If a batch fails, roll back that batch's transaction and continue
+        try {
+          await localConnection.rollback();
+        } catch (rollbackError) {
+          console.error("Error during batch rollback:", rollbackError);
+        }
+        
+        console.error(`Error processing batch ${i}-${i + BATCH_SIZE}:`, error);
+        errorCount++;
+      }
+    }
+
+    // 4. For orders with no matching costs, set a default based on price
+    outputProgress({
+      status: "running",
+      operation: "Order costs update",
+      message: "Setting default costs for remaining orders..."
+    });
+
+    // Process remaining updates in smaller batches
+    const DEFAULT_BATCH_SIZE = 10000;
+    let totalDefaultUpdated = 0;
+    
+    try {
+      // Start with a count query to determine how many records need the default update
+      const [countResult] = await localConnection.query(`
+        SELECT COUNT(*) as count FROM orders
+        WHERE (costeach = 0 OR costeach IS NULL)
+      `);
+      
+      const totalToUpdate = parseInt(countResult.rows[0]?.count || 0);
+      
+      if (totalToUpdate > 0) {
+        console.log(`Applying default cost to ${totalToUpdate} orders`);
+        
+        // Apply the default in batches with separate transactions
+        for (let i = 0; i < totalToUpdate; i += DEFAULT_BATCH_SIZE) {
+          try {
+            await localConnection.beginTransaction();
+            
+            const [defaultUpdates] = await localConnection.query(`
+              WITH orders_to_update AS (
+                SELECT id FROM orders
+                WHERE (costeach = 0 OR costeach IS NULL)
+                LIMIT ${DEFAULT_BATCH_SIZE}
+              )
+              UPDATE orders o
+              SET costeach = price * 0.5
+              FROM orders_to_update otu
+              WHERE o.id = otu.id
+              RETURNING o.id
+            `);
+            
+            const batchDefaultUpdated = defaultUpdates.rowCount || 0;
+            totalDefaultUpdated += batchDefaultUpdated;
+            
+            await localConnection.commit();
+            
+            outputProgress({
+              status: "running",
+              operation: "Order costs update",
+              message: `Applied default costs to ${totalDefaultUpdated} of ${totalToUpdate} orders`,
+              current: totalDefaultUpdated,
+              total: totalToUpdate,
+              elapsed: formatElapsedTime((Date.now() - startTime) / 1000)
+            });
+          } catch (error) {
+            try {
+              await localConnection.rollback();
+            } catch (rollbackError) {
+              console.error("Error during default update rollback:", rollbackError);
+            }
+            
+            console.error(`Error applying default costs batch ${i}-${i + DEFAULT_BATCH_SIZE}:`, error);
+            errorCount++;
+          }
+        }
+      }
+    } catch (error) {
+      console.error("Error counting or updating remaining orders:", error);
+      errorCount++;
+    }
+
+    updatedCount += totalDefaultUpdated;
+    
+    const endTime = Date.now();
+    const totalSeconds = (endTime - startTime) / 1000;
+
+    outputProgress({
+      status: "complete",
+      operation: "Order costs update",
+      message: `Updated ${updatedCount} orders (${totalDefaultUpdated} with default values) in ${formatElapsedTime(totalSeconds)}`,
+      elapsed: formatElapsedTime(totalSeconds)
+    });
+
+    return {
+      status: "complete",
+      updatedCount,
+      errorCount
+    };
+  } catch (error) {
+    console.error("Error during order costs update:", error);
+    
+    return {
+      status: "error",
+      error: error.message,
+      updatedCount,
+      errorCount
+    };
+  } finally {
+    if (connections) {
+      await closeConnections(connections).catch(err => {
+        console.error("Error closing connections:", err);
+      });
+    }
+  }
+}
+
+// Run the script only if this is the main module
+if (require.main === module) {
+  updateOrderCosts().then((results) => {
+    console.log('Cost update completed:', results);
+    // Force exit after a small delay to ensure all logs are written
+    setTimeout(() => process.exit(0), 500);
+  }).catch((error) => {
+    console.error("Unhandled error:", error);
+    // Force exit with error code after a small delay
+    setTimeout(() => process.exit(1), 500);
+  });
+}
+
+// Export the function for use in other scripts
+module.exports = updateOrderCosts; 

inventory-server/src/prompts/product-validation.txt

@@ -1,226 +0,0 @@
-I will provide a JSON array with product data. Process the array by combining all products from validData and invalidData arrays into a single array, excluding any fields starting with “__”, such as “__index” or “__errors”. Process each product according to the reference guidelines below. If a field is not included in the data, do not include it in your response (e.g. do not include its key or any value) unless the specific field guidelines below say otherwise. If a product appears to be from an empty or entirely invalid line, do not include it in your response.
-
-Your response should be a JSON object with the following structure:
-{
-  "correctedData": [], // Array of corrected products
-  "changes": [], // Array of strings describing each change made
-  "warnings": [] // Array of strings with warnings or suggestions for manual review (see below for details)
-}
-
-IMPORTANT: For all fields that use IDs (categories, supplier, company, line, subline, ship_restrictions, tax_cat, artist, themes, etc.), you MUST return the ID values, not the display names. The system will handle converting IDs to display names.
-
-Using the provided guidelines, focus on:
-1. Correcting typos and any incorrect spelling or grammar
-2. Standardizing product names
-3. Correcting and enhancing descriptions by adding details, keywords, and SEO-friendly language
-4. Fixing any obvious errors or inconsistencies between similar products in measurements, prices, or quantities
-5. Adding correct categories, themes, and colors
-
-Use only the provided data and your own knowledge to make changes. Do not make assumptions or make up information that you're not sure about. If you're unable to make a change you're confident about, leave the field as is. All data passed in should be validated, corrected, and returned. All values returned should be strings, not numbers. Do not leave out any fields that were present in the original data.
-
-Possible reasons for including a warning in the warnings array:
-- If you're unable to make a change you're confident about but you believe one needs to be made
-- If there are inconsistencies in the data that could be valid but need to be reviewed
-- If not enough information is provided to make a change that you believe is needed
-- If you infer a value for a required field based on context
-
-
-----------PRODUCT FIELD GUIDELINES----------
-
-Fields: supplier, private_notes, company, line, subline, artist
-Changes: Not allowed
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, return these fields exactly as provided with no changes
-
-Fields: upc, supplier_no, notions_no, item_number
-Changes: Formatting only
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, trim outside white space and return these fields exactly as provided with no other changes
-
-Fields: hts_code
-Changes: Minimal, you can correct formatting, obvious errors or inconsistencies
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, trim white space and any non-numeric characters, then return as a string. Do not validate in any other way.
-
-Fields: image_url
-Changes: Formatting only
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, convert all comma-separated values to valid https:// URLs and return
-
-Fields: msrp, cost_each
-Changes: Minimal, you can correct formatting, obvious errors or inconsistencies
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, strip any currency symbols and return as a string with exactly two decimal places, even if the last place is a 0.
-
-Fields: qty_per_unit, case_qty
-Changes: Minimal, you can correct formatting, obvious errors or inconsistencies
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, strip non-numeric characters and return
-
-Fields: ship_restrictions
-Changes: Only add a value if it's not already present
-Required: You must always return a value for this field, even if it's not provided in the original data. If no value is provided, return 0.
-Instructions: Always return a value exactly as provided, or return 0 if no value is provided.
-
-Fields: eta
-Changes: Minimal, you can correct formatting, obvious errors or inconsistencies
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, return a full month name, day is optional, no year ever (e.g. “January” or “March 3”). This value is not required if not provided.
-
-Fields: name
-Changes: Allowed to conform to guidelines, to fix typos or formatting
-Required: You must always return a value for this field, even if it's not provided in the original data. If no value is provided, return the most reasonable value possible based on the naming guidelines and the other information you have.
-Instructions: Always return a value that is corrected and enhanced per additional guidelines below
-
-Fields: description
-Changes: Full creative control allowed within guidelines
-Required: You must always return a value for this field, even if it's not provided in the original data. If no value is provided, return the most accurate description possible based on the description guidelines and the other information you have.
-Instructions: Always return a value that is corrected and enhanced per additional guidelines below
-
-Fields: weight, length, width, height
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: You must always return a value for this field, even if it's not provided in the original data. If no value is provided, return your best guess based on the other information you have or the dimensions for similar products.
-Instructions: Always return a reasonable value (weights in ounces and dimensions in inches) that is validated against similar provided products and your knowledge of general object measurements (e.g. a sheet of paper is not going to be 3 inches thick, a pack of stickers is not going to be 250 ounces, this sheet of paper is very likely going to be the same size as that other sheet of paper from the same line). If a value is unusual or unreasonable, even wildly so, change it to match similar products or to be more reasonable. When correcting unreasonable weights or dimensions, prioritize comparisons to products from the same company and product line first, then broader category matches or common knowledge if necessary.Do not return 0 or null for any of these fields.
-
-Fields: coo
-Changes: Formatting only
-Required: Return if present in the original data. Do not return if not present.
-Instructions: If present, convert all country names and abbreviations to the official ISO 3166-1 alpha-2 two-character country code. Convert any value with more than two characters to two characters only (e.g. "United States" or "USA" should both return "US").
-
-Fields: tax_cat
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: You must always return a value for this field, even if it's not provided in the original data. If no value is provided, return 0.
-Instructions: Always return a valid numerical tax code ID from the Available Tax Codes array below. Give preference to the value provided, but correct it if another value is more accurate. You must return a value for this field. 0 should be the default value in most cases.
-
-Fields: size_cat
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: Return if present in the original data or if not present and applicable. Do not return if not applicable (e.g. if no size categories apply based on what you know about the product).
-Instructions: If present or if applicable, return one valid numerical size category ID from the Available Size Categories array below. Give preference to the value provided, but correct it if another value is more accurate. If the product name contains a match for one of the size categories (such as 12x12, 6x6, 2oz, etc) you MUST return that size category with the results. A value is not required if none of the size categories apply.
-
-Fields: themes
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: Return if present in the original data or if not present and applicable. Do not return any value if not applicable (e.g. if no themes apply based on what you know about the product).
-Instructions: If present, confirm that each provided theme matches what you understand to be a theme of the product. Remove any themes that do not match and add any themes that are missing. Most products will have zero or one theme. Return a comma-separated list of numerical theme IDs from the Available Themes array below. If you choose a sub-theme, you do not need to include its parent theme in the list.
-
-Fields: colors
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: Return if present in the original data or if not present and applicable. Do not return any value if not applicable (e.g. if no colors apply based on what you know about the product).
-Instructions: If present or if applicable, return a comma-separated list of numerical color IDs from the Available Colors array below, using the product name as the primary guide (e.g. if the name contains Blue or a blue variant, you should return the blue color ID). A value is not required if none of the colors apply. Most products will have zero colors.
-
-Fields: categories
-Changes: Allowed to correct obvious errors or inconsistencies or to add missing values
-Required: You must always return at least one value for this field, even if it's not provided in the original data. If no value is provided, return the most appropriate category or categories based on the other information you have.
-Instructions: Always return a comma-separated list of one or more valid numerical category IDs from the Available Categories array below. Give preference to the values provided, particularly if the other information isn't enough to determine a category, but correct them or add new categories if another value is more accurate. Do not return categories in the Deals or Black Friday categories, and strip these from the list if present. If you choose a subcategory at any level, you do not need to include its parent categories in the list. You must return at least one category and you can return multiple categories if applicable. All categories have equal value so their order is not important. Always try to return the most specific categories possible (e.g. one in the third level of the category hierarchy is better than one in the second level).
-
-----------PRODUCT NAMING GUIDELINES----------
-If there's only one of this type of product in a line: [Line Name] [Product Name] - [Company]
-Example: "Cosmos Infinity Chipboard - Stamperia"
-Example: "Serene Petals 6x6 Paper Pad - Prima"
-
-Multiple similar products in a line: [Differentiator] [Product Type] - [Line Name] - [Company]
-Example: "Ice & Shells Stencil - Arctic Antarctic - Stamperia"
-Example: "Astronomy Paper - Cosmos Infinity - Stamperia"
-
-Standalone products: [Product Name] - [Company]
-Example: "Hedwig Puffy Stickers - Paper House Productions"
-Example: "Heart Tree Dies - Lawn Fawn"
-
-Color-based products: [Color] [Product Name] - [Company]
-Example: "Green Valley Enamel Dots - Altenew"
-Example: "Magenta Aqua Pigment - Brutus Monroe"
-
-Complex products: [Differentiator] [Line] [Product Type] - [Company]
-Example: "Size 6 Round Black Velvet Watercolor Brush - Silver Brush Limited" (Size 6 Round is the differentiator, Black Velvet is the line, Watercolor Brush is the product type)
-
-These should not be included in the name, unless there are multiple products that are otherwise identical:
-- Product size
-- Product weight
-- Number of pages
-- How many are in the package
-
-Naming Conventions:
-- Paper sizes: Use "12x12", "8x8", "6x6" (no spaces or units of measure)
-- Company names must match backend exactly
-- Always capitalize every word in the name, including short articles like "The" and "An"
-- Use "Idea-ology" (not "idea-ology" or "Ideaology")
-- All stamps are "Stamp Set" (not "Clear Stamps" or "Rubber Stamps")
-- All dies are "Dies" or "Die" (not "Die Set")
-- Brands with their own naming conventions should be respected, such as "Doodle Cuts" for dies from Doodlebug
-
-Special Brand Rules - Ranger: 
-Format: [Product Name] - [Designer Line] - Ranger
-Possible Designers: Dylusions, Dina Wakley MEdia, Simon Hurley create., Wendy Vecchi
-Example: "Stacked Stencil - Dina Wakley MEdia - Ranger"
-
-Special Brand Rules - Tim Holtz products from Ranger:
-Format: [Color] [Product Name/Type] - Tim Holtz Distress - Ranger
-Example: "Mermaid Lagoon Tim Holtz Distress Oxide Ink Pad - Ranger"
-
-Special Brand Rules - Tim Holtz products from Sizzix or Stampers Anonymous:
-Format: [Product Name] [Product Type] by Tim Holtz - [Company]
-Example: "Leaf Fragments Thinlits Dies by Tim Holtz - Sizzix"
-
-Special Brand Rules - Tim Holtz products from Advantus/Idea-ology:
-Format: [Product Name] - Tim Holtz Idea-ology
-Example: "Tiny Vials - Tim Holtz Idea-ology"
-
-Special Brand Rules - Dies from Sizzix:
-Include die type plus "Dies" or "Die"
-Examples:
-"Art Nouveau 3-D Textured Impressions Embossing Folder - Sizzix"
-"Pocket Pals Thinlits Dies - Sizzix"
-"Butterfly Wishes Framelits Dies & Stamps - Sizzix"
-
-Important Notes
-- Ensure that product names are consistent across all products of the same type
-- Use the minimum amount of information needed to uniquely identify the product
-- Put detailed specifications in the product description, not its name
-
-Edge Cases
-- If the product is missing a company name, infer one from the other products included in the data
-- If the product is missing a clear differentiator and needs one to be unique, infer and add one from the other data provided (e.g. the description, existing size categories, etc.)
-
-Incorrect example: MVP Rugby - Collection Pack - Photoplay
-Notes: there should be no dash between the line and the product
-
-Incorrect Example: A2 Easel Cards - Black - Photoplay
-Notes: the differentiating factor should come first: “Black A2 Easel Cards - Photoplay”. Size is ok to include here because this is the name printed on the package.
-
-Incorrect Example: 6” - Scriber Needle Modeling Tool
-Notes: this product only comes in one size, so 6” isn’t needed. The company name should also be included.
-
-Incorrect Example: Slick - White - Tulip Dimensional Fabric Paint 4oz
-Notes: color should be first, then type, then product, then company, so “White Slick Dimensional Fabric Paint - Tulip”. It appears there’s only one size available so no need to differentiate in the name.
-
-Incorrect Example: Silhouette Adhesive Cork Sheets 5”X7” 8/Pkg
-Notes: should be “Adhesive Cork Sheets - Silhouette”
-
-Incorrect Example: Galaxy - Opaque - American Crafts Color Pour Resin Dyes
-Notes: “Galaxy Opaque Dye Set - Color Pour Resin - American Crafts”
-
-Incorrect Example: Slate - Lion Brand Truboo Yarn
-Notes: [Differentiator] [Line] [Product Type] - [Company] : “Slate Truboo Yarn - Lion Brand”
-
-Incorrect Example: Rose Quartz Dylusions Shimmer Paint
-Notes: “Rose Quartz Shimmer Paint - Dylusions - Ranger”
-
-
-----------PRODUCT DESCRIPTION GUIDELINES----------
-Product descriptions are an extremely important part of the listing and are the most important part of your response. Care should be taken to ensure they are correct, helpful, and SEO-friendly.
-
-If a description is provided in the data, use it as a starting point. Correct any spelling errors, typos, poor grammar, or awkward phrasing. If necessary and you have the information, add more details, describe how the customer could use it, etc. Use complete sentences and keep SEO in mind.
-
-If no description is provided, make one up using the product name, the information you have, and the other provided guidelines. At minimum, a description should be one complete sentence that starts with a capital letter and ends with a period. Unless the product is extremely complex, 2-4 sentences is usually sufficient if you have enough information.
-
-Important Notes:
-- Every description should state exactly what's included in the product (e.g. "Includes one 12x12 sheet of patterned cardstock." or "Includes one 6x12 sheet with 27 unique stickers." or "Includes 55 pieces." or "Package includes machine, power cord, 12 sheets of cardstock, 3 dies, and project instructions.")
-- Do not use the word "our" in the description (this usually shows up when we copy a description from the manufacturer). Instead use "these" or "[Company name] [product]" or similar. (e.g. don't use "Our journals are hand-made in the USA", instead use "These journals are hand made..." or "Archer & Olive journals are handmade...")
-- Don't include statements that add no value like “this is perfect for all your paper crafts”. If the product helps to solve a unique problem or has a unique feature, by all means describe it, but if it’s just a normal sheet of paper or pack of stickers, you don’t have to pretend like it’s the best thing ever. At the same time, ensure that you add enough copy to ensure good SEO.
-- State as many facts as you can about the product, considering the viewpoint of the customer and what they would want to know when looking at it. They probably want to know dimensions, what products it’s compatible with, how thick the paper is, how many sheets are included, whether the sheets are double-sided or not, which items are in the kit, etc. Say as much as you possibly can with the information that you have.
-- !!DO NOT make up information if you aren't sure about it. A minimal correct description is better than a long incorrect one!!
-
-Avoid/remove:
-- The word "Imported"
-- Any warnings about Prop 65, choking hazards, etc
-- The manufacturer's name if it's included as the very first thing in the description
-- Any statement similar to "comes in a variety of colors, each sold separately"

inventory-server/src/routes/ai-prompts.js

@@ -0,0 +1,335 @@
+const express = require('express');
+const router = express.Router();
+
+// Get all AI prompts
+router.get('/', async (req, res) => {
+  try {
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM ai_prompts 
+      ORDER BY prompt_type ASC, company ASC
+    `);
+    res.json(result.rows);
+  } catch (error) {
+    console.error('Error fetching AI prompts:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch AI prompts',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get prompt by ID
+router.get('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE id = $1
+    `, [id]);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'AI prompt not found' });
+    }
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error fetching AI prompt:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get prompt by company
+router.get('/company/:companyId', async (req, res) => {
+  try {
+    const { companyId } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE company = $1
+    `, [companyId]);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'AI prompt not found for this company' });
+    }
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error fetching AI prompt by company:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch AI prompt by company',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get general prompt
+router.get('/type/general', async (req, res) => {
+  try {
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE prompt_type = 'general'
+    `);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'General AI prompt not found' });
+    }
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error fetching general AI prompt:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch general AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get system prompt
+router.get('/type/system', async (req, res) => {
+  try {
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE prompt_type = 'system'
+    `);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'System AI prompt not found' });
+    }
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error fetching system AI prompt:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch system AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Create new AI prompt
+router.post('/', async (req, res) => {
+  try {
+    const {
+      prompt_text,
+      prompt_type,
+      company
+    } = req.body;
+
+    // Validate required fields
+    if (!prompt_text || !prompt_type) {
+      return res.status(400).json({ error: 'Prompt text and type are required' });
+    }
+    
+    // Validate prompt type
+    if (!['general', 'company_specific', 'system'].includes(prompt_type)) {
+      return res.status(400).json({ error: 'Prompt type must be either "general", "company_specific", or "system"' });
+    }
+    
+    // Validate company is provided for company-specific prompts
+    if (prompt_type === 'company_specific' && !company) {
+      return res.status(400).json({ error: 'Company is required for company-specific prompts' });
+    }
+
+    // Validate company is not provided for general or system prompts
+    if ((prompt_type === 'general' || prompt_type === 'system') && company) {
+      return res.status(400).json({ error: 'Company should not be provided for general or system prompts' });
+    }
+
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      INSERT INTO ai_prompts (
+        prompt_text,
+        prompt_type,
+        company
+      ) VALUES ($1, $2, $3)
+      RETURNING *
+    `, [
+      prompt_text,
+      prompt_type,
+      company
+    ]);
+
+    res.status(201).json(result.rows[0]);
+  } catch (error) {
+    console.error('Error creating AI prompt:', error);
+    
+    // Check for unique constraint violations
+    if (error instanceof Error && error.message.includes('unique constraint')) {
+      if (error.message.includes('unique_company_prompt')) {
+        return res.status(409).json({ 
+          error: 'A prompt already exists for this company',
+          details: error.message
+        });
+      } else if (error.message.includes('idx_unique_general_prompt')) {
+        return res.status(409).json({ 
+          error: 'A general prompt already exists',
+          details: error.message
+        });
+      } else if (error.message.includes('idx_unique_system_prompt')) {
+        return res.status(409).json({ 
+          error: 'A system prompt already exists',
+          details: error.message
+        });
+      }
+    }
+    
+    res.status(500).json({ 
+      error: 'Failed to create AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Update AI prompt
+router.put('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const {
+      prompt_text,
+      prompt_type,
+      company
+    } = req.body;
+
+    // Validate required fields
+    if (!prompt_text || !prompt_type) {
+      return res.status(400).json({ error: 'Prompt text and type are required' });
+    }
+    
+    // Validate prompt type
+    if (!['general', 'company_specific', 'system'].includes(prompt_type)) {
+      return res.status(400).json({ error: 'Prompt type must be either "general", "company_specific", or "system"' });
+    }
+    
+    // Validate company is provided for company-specific prompts
+    if (prompt_type === 'company_specific' && !company) {
+      return res.status(400).json({ error: 'Company is required for company-specific prompts' });
+    }
+    
+    // Validate company is not provided for general or system prompts
+    if ((prompt_type === 'general' || prompt_type === 'system') && company) {
+      return res.status(400).json({ error: 'Company should not be provided for general or system prompts' });
+    }
+
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    // Check if the prompt exists
+    const checkResult = await pool.query('SELECT * FROM ai_prompts WHERE id = $1', [id]);
+    if (checkResult.rows.length === 0) {
+      return res.status(404).json({ error: 'AI prompt not found' });
+    }
+    
+    const result = await pool.query(`
+      UPDATE ai_prompts 
+      SET 
+        prompt_text = $1,
+        prompt_type = $2,
+        company = $3
+      WHERE id = $4
+      RETURNING *
+    `, [
+      prompt_text,
+      prompt_type,
+      company,
+      id
+    ]);
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error updating AI prompt:', error);
+    
+    // Check for unique constraint violations
+    if (error instanceof Error && error.message.includes('unique constraint')) {
+      if (error.message.includes('unique_company_prompt')) {
+        return res.status(409).json({ 
+          error: 'A prompt already exists for this company',
+          details: error.message
+        });
+      } else if (error.message.includes('idx_unique_general_prompt')) {
+        return res.status(409).json({ 
+          error: 'A general prompt already exists',
+          details: error.message
+        });
+      } else if (error.message.includes('idx_unique_system_prompt')) {
+        return res.status(409).json({ 
+          error: 'A system prompt already exists',
+          details: error.message
+        });
+      }
+    }
+    
+    res.status(500).json({ 
+      error: 'Failed to update AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Delete AI prompt
+router.delete('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query('DELETE FROM ai_prompts WHERE id = $1 RETURNING *', [id]);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'AI prompt not found' });
+    }
+
+    res.json({ message: 'AI prompt deleted successfully' });
+  } catch (error) {
+    console.error('Error deleting AI prompt:', error);
+    res.status(500).json({ 
+      error: 'Failed to delete AI prompt',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Error handling middleware
+router.use((err, req, res, next) => {
+  console.error('AI prompts route error:', err);
+  res.status(500).json({ 
+    error: 'Internal server error',
+    details: err.message
+  });
+});
+
+module.exports = router; 

inventory-server/src/routes/ai-validation.js

@@ -289,8 +289,108 @@ async function generateDebugResponse(productsToUse, res) {
     });
     
     try {
-      const prompt = await loadPrompt(promptConnection, productsToUse);
-      const fullPrompt = prompt + "\n" + JSON.stringify(productsToUse);
+      // Get the local PostgreSQL pool to fetch prompts
+      const pool = res.app.locals.pool;
+      if (!pool) {
+        console.warn("⚠️ Local database pool not available for prompts");
+        throw new Error("Database connection not available");
+      }
+
+      // First, fetch the system prompt
+      const systemPromptResult = await pool.query(`
+        SELECT * FROM ai_prompts 
+        WHERE prompt_type = 'system'
+      `);
+      
+      // Get system prompt or use default
+      let systemPrompt = null;
+      if (systemPromptResult.rows.length > 0) {
+        systemPrompt = systemPromptResult.rows[0];
+        console.log("📝 Loaded system prompt from database, ID:", systemPrompt.id);
+      } else {
+        console.warn("⚠️ No system prompt found in database, will use default");
+      }
+
+      // Then, fetch the general prompt
+      const generalPromptResult = await pool.query(`
+        SELECT * FROM ai_prompts 
+        WHERE prompt_type = 'general'
+      `);
+      
+      if (generalPromptResult.rows.length === 0) {
+        console.warn("⚠️ No general prompt found in database");
+        throw new Error("No general prompt found in database");
+      }
+
+      // Get the general prompt text and info
+      const generalPrompt = generalPromptResult.rows[0];
+      console.log("📝 Loaded general prompt from database, ID:", generalPrompt.id);
+
+      // Fetch company-specific prompts if we have products to validate
+      let companyPrompts = [];
+      if (productsToUse && Array.isArray(productsToUse)) {
+        // Extract unique company IDs from products
+        const companyIds = new Set();
+        productsToUse.forEach(product => {
+          if (product.company) {
+            companyIds.add(String(product.company));
+          }
+        });
+
+        if (companyIds.size > 0) {
+          console.log(`🔍 Found ${companyIds.size} unique companies in products:`, Array.from(companyIds));
+          
+          // Fetch company-specific prompts
+          const companyPromptsResult = await pool.query(`
+            SELECT * FROM ai_prompts 
+            WHERE prompt_type = 'company_specific' 
+            AND company = ANY($1)
+          `, [Array.from(companyIds)]);
+          
+          companyPrompts = companyPromptsResult.rows;
+          console.log(`📝 Loaded ${companyPrompts.length} company-specific prompts`);
+        }
+      }
+
+      // Find company names from taxonomy for the validation endpoint
+      const companyPromptsWithNames = companyPrompts.map(prompt => {
+        let companyName = "Unknown Company";
+        if (taxonomy.companies && Array.isArray(taxonomy.companies)) {
+          const companyData = taxonomy.companies.find(company => 
+            String(company[0]) === String(prompt.company)
+          );
+          if (companyData && companyData[1]) {
+            companyName = companyData[1];
+          }
+        }
+        
+        return {
+          id: prompt.id,
+          company: prompt.company,
+          companyName: companyName,
+          prompt_text: prompt.prompt_text
+        };
+      });
+
+      // Now use loadPrompt to get the actual combined prompt
+      const promptData = await loadPrompt(promptConnection, productsToUse, res.app.locals.pool);
+      const fullUserPrompt = promptData.userContent + "\n" + JSON.stringify(productsToUse);
+      const promptLength = promptData.systemInstructions.length + fullUserPrompt.length; // Store prompt length for performance metrics
+      console.log("📝 Generated prompt length:", promptLength);
+      console.log("📝 System instructions length:", promptData.systemInstructions.length);
+      console.log("📝 User content length:", fullUserPrompt.length);
+
+      // Format the messages as they would be sent to the API
+      const apiMessages = [
+        {
+          role: "system",
+          content: promptData.systemInstructions
+        },
+        {
+          role: "user",
+          content: fullUserPrompt
+        }
+      ];
 
       // Create the response with taxonomy stats
       let categoriesCount = 0;
@@ -330,9 +430,28 @@ async function generateDebugResponse(productsToUse, res) {
                 : null,
             }
           : null,
-        basePrompt: prompt,
-        sampleFullPrompt: fullPrompt,
-        promptLength: fullPrompt.length,
+        basePrompt: systemPrompt ? systemPrompt.prompt_text + "\n\n" + generalPrompt.prompt_text : generalPrompt.prompt_text,
+        sampleFullPrompt: fullUserPrompt,
+        promptLength: promptLength,
+        apiFormat: apiMessages,
+        promptSources: {
+          ...(systemPrompt ? {
+            systemPrompt: {
+              id: systemPrompt.id,
+              prompt_text: systemPrompt.prompt_text
+            }
+          } : {
+            systemPrompt: {
+              id: 0,
+              prompt_text: `You are a specialized e-commerce product data processor for a crafting supplies website tasked with providing complete, correct, appealing, and SEO-friendly product listings. You should write professionally, but in a friendly and engaging tone. You have meticulous attention to detail and are a master at your craft.`
+            }
+          }),
+          generalPrompt: {
+            id: generalPrompt.id,
+            prompt_text: generalPrompt.prompt_text
+          },
+          companyPrompts: companyPromptsWithNames
+        }
       };
 
       console.log("Sending response with taxonomy stats:", response.taxonomyStats);
@@ -513,22 +632,101 @@ SELECT t.cat_id,t.name,null as master_cat_id,1 AS level_order FROM product_categ
   }
 }
 
-// Load the prompt from file and inject taxonomy data
-async function loadPrompt(connection, productsToValidate = null) {
+// Load prompts from database and inject taxonomy data
+async function loadPrompt(connection, productsToValidate = null, appPool = null) {
   try {
-    const promptPath = path.join(
-      __dirname,
-      "..",
-      "prompts",
-      "product-validation.txt"
-    );
-    const basePrompt = await fs.readFile(promptPath, "utf8");
-
     // Get taxonomy data using the provided MySQL connection
     const taxonomy = await getTaxonomyData(connection);
 
-    // Add system instructions to the prompt
-    const systemInstructions = `You are a specialized e-commerce product data processor for a crafting supplies website tasked with providing complete, correct, appealing, and SEO-friendly product listings. You should write professionally, but in a friendly and engaging tone. You have meticulous attention to detail and are a master at your craft.`;
+    // Use the provided pool parameter instead of global.app
+    const pool = appPool;
+    if (!pool) {
+      console.warn("⚠️ Local database pool not available for prompts");
+      throw new Error("Database connection not available");
+    }
+
+    // Fetch the system prompt
+    const systemPromptResult = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE prompt_type = 'system'
+    `);
+    
+    // Default system instructions in case the system prompt is not found
+    let systemInstructions = `You are a specialized e-commerce product data processor for a crafting supplies website tasked with providing complete, correct, appealing, and SEO-friendly product listings. You should write professionally, but in a friendly and engaging tone. You have meticulous attention to detail and are a master at your craft.`;
+    
+    // If system prompt exists in the database, use it
+    if (systemPromptResult.rows.length > 0) {
+      systemInstructions = systemPromptResult.rows[0].prompt_text;
+      console.log("📝 Loaded system prompt from database");
+    } else {
+      console.warn("⚠️ No system prompt found in database, using default");
+    }
+
+    // Fetch the general prompt
+    const generalPromptResult = await pool.query(`
+      SELECT * FROM ai_prompts 
+      WHERE prompt_type = 'general'
+    `);
+    
+    if (generalPromptResult.rows.length === 0) {
+      console.warn("⚠️ No general prompt found in database");
+      throw new Error("No general prompt found in database");
+    }
+
+    // Get the general prompt text
+    const basePrompt = generalPromptResult.rows[0].prompt_text;
+    console.log("📝 Loaded general prompt from database");
+
+    // Fetch company-specific prompts if we have products to validate
+    let companyPrompts = [];
+    if (productsToValidate && Array.isArray(productsToValidate)) {
+      // Extract unique company IDs from products
+      const companyIds = new Set();
+      productsToValidate.forEach(product => {
+        if (product.company) {
+          companyIds.add(String(product.company));
+        }
+      });
+
+      if (companyIds.size > 0) {
+        console.log(`🔍 Found ${companyIds.size} unique companies in products:`, Array.from(companyIds));
+        
+        // Fetch company-specific prompts
+        const companyPromptsResult = await pool.query(`
+          SELECT * FROM ai_prompts 
+          WHERE prompt_type = 'company_specific' 
+          AND company = ANY($1)
+        `, [Array.from(companyIds)]);
+        
+        companyPrompts = companyPromptsResult.rows;
+        console.log(`📝 Loaded ${companyPrompts.length} company-specific prompts`);
+      }
+    }
+
+    // Combine prompts - start with the general prompt
+    let combinedPrompt = basePrompt;
+    
+    // Add any company-specific prompts with annotations
+    if (companyPrompts.length > 0) {
+      combinedPrompt += "\n\n--- COMPANY-SPECIFIC INSTRUCTIONS ---\n";
+      
+      for (const prompt of companyPrompts) {
+        // Find company name from taxonomy
+        let companyName = "Unknown Company";
+        if (taxonomy.companies && Array.isArray(taxonomy.companies)) {
+          const companyData = taxonomy.companies.find(company => 
+            String(company[0]) === String(prompt.company)
+          );
+          if (companyData && companyData[1]) {
+            companyName = companyData[1];
+          }
+        }
+        
+        combinedPrompt += `\n[SPECIFIC TO COMPANY: ${companyName} (ID: ${prompt.company})]:\n${prompt.prompt_text}\n`;
+      }
+      
+      combinedPrompt += "\n--- END COMPANY-SPECIFIC INSTRUCTIONS ---\n";
+    }
 
     // If we have products to validate, create a filtered prompt
     if (productsToValidate) {
@@ -655,11 +853,14 @@ ${JSON.stringify(mixedTaxonomy.sizeCategories)}${
 
 ----------Here is the product data to validate----------`;
 
-      // Return the filtered prompt
-      return systemInstructions + basePrompt + "\n" + taxonomySection;
+      // Return both system instructions and user content separately
+      return {
+        systemInstructions,
+        userContent: combinedPrompt + "\n" + taxonomySection
+      };
     }
 
-    // Generate the full unfiltered prompt
+    // Generate the full unfiltered prompt for taxonomy section
     const taxonomySection = `
 Available Categories:
 ${JSON.stringify(taxonomy.categories)}
@@ -687,7 +888,11 @@ ${JSON.stringify(taxonomy.artists)}
 
 Here is the product data to validate:`;
 
-    return systemInstructions + basePrompt + "\n" + taxonomySection;
+    // Return both system instructions and user content separately
+    return {
+      systemInstructions,
+      userContent: combinedPrompt + "\n" + taxonomySection
+    };
   } catch (error) {
     console.error("Error loading prompt:", error);
     throw error; // Re-throw to be handled by the calling function
@@ -735,18 +940,24 @@ router.post("/validate", async (req, res) => {
 
       // Load the prompt with the products data to filter taxonomy
       console.log("🔄 Loading prompt with filtered taxonomy...");
-      const prompt = await loadPrompt(connection, products);
-      const fullPrompt = prompt + "\n" + JSON.stringify(products);
-      promptLength = fullPrompt.length; // Store prompt length for performance metrics
+      const promptData = await loadPrompt(connection, products, req.app.locals.pool);
+      const fullUserPrompt = promptData.userContent + "\n" + JSON.stringify(products);
+      const promptLength = promptData.systemInstructions.length + fullUserPrompt.length; // Store prompt length for performance metrics
       console.log("📝 Generated prompt length:", promptLength);
+      console.log("📝 System instructions length:", promptData.systemInstructions.length);
+      console.log("📝 User content length:", fullUserPrompt.length);
 
       console.log("🤖 Sending request to OpenAI...");
       const completion = await openai.chat.completions.create({
-        model: "o3-mini",
+        model: "gpt-4o",
         messages: [
+          {
+            role: "system",
+            content: promptData.systemInstructions,
+          },
           {
             role: "user",
-            content: fullPrompt,
+            content: fullUserPrompt,
           },
         ],
         temperature: 0.2,
@@ -884,7 +1095,94 @@ router.post("/validate", async (req, res) => {
           console.error("⚠️ Failed to record performance metrics:", metricError);
         }
         
-        // Include performance metrics in the response
+        // Get sources of the prompts for tracking
+        let promptSources = null;
+
+        try {
+          // Get system prompt
+          const systemPromptResult = await pool.query(`
+            SELECT * FROM ai_prompts WHERE prompt_type = 'system'
+          `);
+          
+          // Get general prompt
+          const generalPromptResult = await pool.query(`
+            SELECT * FROM ai_prompts WHERE prompt_type = 'general'
+          `);
+          
+          // Extract unique company IDs from products
+          const companyIds = new Set();
+          products.forEach(product => {
+            if (product.company) {
+              companyIds.add(String(product.company));
+            }
+          });
+          
+          let companyPrompts = [];
+          if (companyIds.size > 0) {
+            // Fetch company-specific prompts
+            const companyPromptsResult = await pool.query(`
+              SELECT * FROM ai_prompts 
+              WHERE prompt_type = 'company_specific' 
+              AND company = ANY($1)
+            `, [Array.from(companyIds)]);
+            
+            companyPrompts = companyPromptsResult.rows;
+          }
+          
+          // Find company names from taxonomy for the validation endpoint
+          const companyPromptsWithNames = companyPrompts.map(prompt => {
+            let companyName = "Unknown Company";
+            if (taxonomy.companies && Array.isArray(taxonomy.companies)) {
+              const companyData = taxonomy.companies.find(company => 
+                String(company[0]) === String(prompt.company)
+              );
+              if (companyData && companyData[1]) {
+                companyName = companyData[1];
+              }
+            }
+            
+            return {
+              id: prompt.id,
+              company: prompt.company,
+              companyName: companyName,
+              prompt_text: prompt.prompt_text
+            };
+          });
+          
+          // Set prompt sources
+          if (generalPromptResult.rows.length > 0) {
+            const generalPrompt = generalPromptResult.rows[0];
+            let systemPrompt = null;
+            
+            if (systemPromptResult.rows.length > 0) {
+              systemPrompt = systemPromptResult.rows[0];
+            }
+            
+            promptSources = {
+              ...(systemPrompt ? {
+                systemPrompt: {
+                  id: systemPrompt.id,
+                  prompt_text: systemPrompt.prompt_text
+                }
+              } : {
+                systemPrompt: {
+                  id: 0,
+                  prompt_text: `You are a specialized e-commerce product data processor for a crafting supplies website tasked with providing complete, correct, appealing, and SEO-friendly product listings. You should write professionally, but in a friendly and engaging tone. You have meticulous attention to detail and are a master at your craft.`
+                }
+              }),
+              generalPrompt: {
+                id: generalPrompt.id,
+                prompt_text: generalPrompt.prompt_text
+              },
+              companyPrompts: companyPromptsWithNames
+            };
+          }
+        } catch (promptSourceError) {
+          console.error("⚠️ Error getting prompt sources:", promptSourceError);
+          // Don't fail the entire validation if just prompt sources retrieval fails
+        }
+
+        // Include prompt sources in the response
         res.json({
           success: true,
           changeDetails: changeDetails,
@@ -895,6 +1193,7 @@ router.post("/validate", async (req, res) => {
             isEstimate: true,
             productCount: products.length
           },
+          promptSources: promptSources,
           ...aiResponse,
         });
       } catch (parseError) {

inventory-server/src/routes/csv.js

@@ -757,8 +757,8 @@ router.get('/history/import', async (req, res) => {
                 end_time,
                 status,
                 error_message,
-                rows_processed::integer,
-                files_processed::integer
+                records_added::integer,
+                records_updated::integer
             FROM import_history 
             ORDER BY start_time DESC 
             LIMIT 20

inventory-server/src/routes/reusable-images.js

@@ -0,0 +1,396 @@
+const express = require('express');
+const router = express.Router();
+const multer = require('multer');
+const path = require('path');
+const fs = require('fs');
+
+// Create reusable uploads directory if it doesn't exist
+const uploadsDir = path.join('/var/www/html/inventory/uploads/reusable');
+fs.mkdirSync(uploadsDir, { recursive: true });
+
+// Configure multer for file uploads
+const storage = multer.diskStorage({
+  destination: function (req, file, cb) {
+    console.log(`Saving reusable image to: ${uploadsDir}`);
+    cb(null, uploadsDir);
+  },
+  filename: function (req, file, cb) {
+    // Create unique filename with original extension
+    const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9);
+    
+    // Make sure we preserve the original file extension
+    let fileExt = path.extname(file.originalname).toLowerCase();
+    
+    // Ensure there is a proper extension based on mimetype if none exists
+    if (!fileExt) {
+      switch (file.mimetype) {
+        case 'image/jpeg': fileExt = '.jpg'; break;
+        case 'image/png': fileExt = '.png'; break;
+        case 'image/gif': fileExt = '.gif'; break;
+        case 'image/webp': fileExt = '.webp'; break;
+        default: fileExt = '.jpg'; // Default to jpg
+      }
+    }
+    
+    const fileName = `reusable-${uniqueSuffix}${fileExt}`;
+    console.log(`Generated filename: ${fileName} with mimetype: ${file.mimetype}`);
+    cb(null, fileName);
+  }
+});
+
+const upload = multer({ 
+  storage: storage,
+  limits: {
+    fileSize: 5 * 1024 * 1024, // 5MB max file size
+  },
+  fileFilter: function (req, file, cb) {
+    // Accept only image files
+    const filetypes = /jpeg|jpg|png|gif|webp/;
+    const mimetype = filetypes.test(file.mimetype);
+    const extname = filetypes.test(path.extname(file.originalname).toLowerCase());
+    
+    if (mimetype && extname) {
+      return cb(null, true);
+    }
+    cb(new Error('Only image files are allowed'));
+  }
+});
+
+// Get all reusable images
+router.get('/', async (req, res) => {
+  try {
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM reusable_images 
+      ORDER BY created_at DESC
+    `);
+    res.json(result.rows);
+  } catch (error) {
+    console.error('Error fetching reusable images:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch reusable images',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get images by company or global images
+router.get('/by-company/:companyId', async (req, res) => {
+  try {
+    const { companyId } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    // Get images that are either global or belong to this company
+    const result = await pool.query(`
+      SELECT * FROM reusable_images 
+      WHERE is_global = true OR company = $1
+      ORDER BY created_at DESC
+    `, [companyId]);
+    
+    res.json(result.rows);
+  } catch (error) {
+    console.error('Error fetching reusable images by company:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch reusable images by company',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get global images only
+router.get('/global', async (req, res) => {
+  try {
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM reusable_images 
+      WHERE is_global = true
+      ORDER BY created_at DESC
+    `);
+    
+    res.json(result.rows);
+  } catch (error) {
+    console.error('Error fetching global reusable images:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch global reusable images',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Get a single image by ID
+router.get('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    const result = await pool.query(`
+      SELECT * FROM reusable_images 
+      WHERE id = $1
+    `, [id]);
+
+    if (result.rows.length === 0) {
+      return res.status(404).json({ error: 'Reusable image not found' });
+    }
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error fetching reusable image:', error);
+    res.status(500).json({ 
+      error: 'Failed to fetch reusable image',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Upload a new reusable image
+router.post('/upload', upload.single('image'), async (req, res) => {
+  try {
+    if (!req.file) {
+      return res.status(400).json({ error: 'No image file provided' });
+    }
+
+    const { name, is_global, company } = req.body;
+    
+    // Validate required fields
+    if (!name) {
+      return res.status(400).json({ error: 'Image name is required' });
+    }
+    
+    // Convert is_global from string to boolean
+    const isGlobal = is_global === 'true' || is_global === true;
+    
+    // Validate company is provided for non-global images
+    if (!isGlobal && !company) {
+      return res.status(400).json({ error: 'Company is required for non-global images' });
+    }
+
+    // Log file information
+    console.log('Reusable image uploaded:', {
+      filename: req.file.filename,
+      originalname: req.file.originalname,
+      mimetype: req.file.mimetype,
+      size: req.file.size,
+      path: req.file.path
+    });
+
+    // Ensure the file exists
+    const filePath = path.join(uploadsDir, req.file.filename);
+    if (!fs.existsSync(filePath)) {
+      return res.status(500).json({ error: 'File was not saved correctly' });
+    }
+
+    // Create URL for the uploaded file
+    const baseUrl = 'https://inventory.acot.site';
+    const imageUrl = `${baseUrl}/uploads/reusable/${req.file.filename}`;
+    
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    // Insert record into database
+    const result = await pool.query(`
+      INSERT INTO reusable_images (
+        name,
+        filename,
+        file_path,
+        image_url,
+        is_global,
+        company,
+        mime_type,
+        file_size
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+      RETURNING *
+    `, [
+      name,
+      req.file.filename,
+      filePath,
+      imageUrl,
+      isGlobal,
+      isGlobal ? null : company,
+      req.file.mimetype,
+      req.file.size
+    ]);
+
+    // Return success response with image data
+    res.status(201).json({
+      success: true,
+      image: result.rows[0],
+      message: 'Image uploaded successfully'
+    });
+    
+  } catch (error) {
+    console.error('Error uploading reusable image:', error);
+    res.status(500).json({ error: error.message || 'Failed to upload image' });
+  }
+});
+
+// Update image details (name, is_global, company)
+router.put('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const { name, is_global, company } = req.body;
+    
+    // Validate required fields
+    if (!name) {
+      return res.status(400).json({ error: 'Image name is required' });
+    }
+    
+    // Convert is_global from string to boolean if necessary
+    const isGlobal = typeof is_global === 'string' ? is_global === 'true' : !!is_global;
+    
+    // Validate company is provided for non-global images
+    if (!isGlobal && !company) {
+      return res.status(400).json({ error: 'Company is required for non-global images' });
+    }
+    
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    // Check if the image exists
+    const checkResult = await pool.query('SELECT * FROM reusable_images WHERE id = $1', [id]);
+    if (checkResult.rows.length === 0) {
+      return res.status(404).json({ error: 'Reusable image not found' });
+    }
+    
+    const result = await pool.query(`
+      UPDATE reusable_images 
+      SET 
+        name = $1,
+        is_global = $2,
+        company = $3
+      WHERE id = $4
+      RETURNING *
+    `, [
+      name,
+      isGlobal,
+      isGlobal ? null : company,
+      id
+    ]);
+
+    res.json(result.rows[0]);
+  } catch (error) {
+    console.error('Error updating reusable image:', error);
+    res.status(500).json({ 
+      error: 'Failed to update reusable image',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Delete a reusable image
+router.delete('/:id', async (req, res) => {
+  try {
+    const { id } = req.params;
+    const pool = req.app.locals.pool;
+    if (!pool) {
+      throw new Error('Database pool not initialized');
+    }
+    
+    // Get the image data first to get the filename
+    const imageResult = await pool.query('SELECT * FROM reusable_images WHERE id = $1', [id]);
+    
+    if (imageResult.rows.length === 0) {
+      return res.status(404).json({ error: 'Reusable image not found' });
+    }
+    
+    const image = imageResult.rows[0];
+    
+    // Delete from database
+    await pool.query('DELETE FROM reusable_images WHERE id = $1', [id]);
+    
+    // Delete the file from filesystem
+    const filePath = path.join(uploadsDir, image.filename);
+    if (fs.existsSync(filePath)) {
+      fs.unlinkSync(filePath);
+    }
+
+    res.json({ 
+      message: 'Reusable image deleted successfully',
+      image
+    });
+  } catch (error) {
+    console.error('Error deleting reusable image:', error);
+    res.status(500).json({ 
+      error: 'Failed to delete reusable image',
+      details: error instanceof Error ? error.message : 'Unknown error'
+    });
+  }
+});
+
+// Check if file exists and permissions
+router.get('/check-file/:filename', (req, res) => {
+  const { filename } = req.params;
+  
+  // Prevent directory traversal
+  if (filename.includes('..') || filename.includes('/')) {
+    return res.status(400).json({ error: 'Invalid filename' });
+  }
+  
+  const filePath = path.join(uploadsDir, filename);
+  
+  try {
+    // Check if file exists
+    if (!fs.existsSync(filePath)) {
+      return res.status(404).json({ 
+        error: 'File not found', 
+        path: filePath,
+        exists: false,
+        readable: false 
+      });
+    }
+    
+    // Check if file is readable
+    fs.accessSync(filePath, fs.constants.R_OK);
+    
+    // Get file stats
+    const stats = fs.statSync(filePath);
+    
+    return res.json({
+      filename,
+      path: filePath,
+      exists: true,
+      readable: true,
+      isFile: stats.isFile(),
+      isDirectory: stats.isDirectory(),
+      size: stats.size,
+      created: stats.birthtime,
+      modified: stats.mtime,
+      permissions: stats.mode.toString(8)
+    });
+  } catch (error) {
+    return res.status(500).json({ 
+      error: error.message,
+      path: filePath,
+      exists: fs.existsSync(filePath),
+      readable: false
+    });
+  }
+});
+
+// Error handling middleware
+router.use((err, req, res, next) => {
+  console.error('Reusable images route error:', err);
+  res.status(500).json({ 
+    error: 'Internal server error',
+    details: err.message
+  });
+});
+
+module.exports = router; 

inventory-server/src/server.js

@@ -18,6 +18,8 @@ const categoriesRouter = require('./routes/categories');
 const importRouter = require('./routes/import');
 const aiValidationRouter = require('./routes/ai-validation');
 const templatesRouter = require('./routes/templates');
+const aiPromptsRouter = require('./routes/ai-prompts');
+const reusableImagesRouter = require('./routes/reusable-images');
 
 // Get the absolute path to the .env file
 const envPath = '/var/www/html/inventory/.env';
@@ -103,6 +105,8 @@ async function startServer() {
     app.use('/api/import', importRouter);
     app.use('/api/ai-validation', aiValidationRouter);
     app.use('/api/templates', templatesRouter);
+    app.use('/api/ai-prompts', aiPromptsRouter);
+    app.use('/api/reusable-images', reusableImagesRouter);
 
     // Basic health check route
     app.get('/health', (req, res) => {

inventory-server/src/utils/db.js

@@ -1,9 +1,9 @@
-const mysql = require('mysql2/promise');
+const { Pool } = require('pg');
 
 let pool;
 
 function initPool(config) {
-  pool = mysql.createPool(config);
+  pool = new Pool(config);
   return pool;
 }
 

inventory/src/components/product-import/steps/ImageUploadStep/ImageUploadStep.tsx

@@ -253,6 +253,7 @@ export const ImageUploadStep = ({
                     }
                     getProductContainerClasses={() => getProductContainerClasses(index)}
                     findContainer={findContainer}
+                    handleAddImageFromUrl={handleAddImageFromUrl}
                   />
                 ))}
               </div>

inventory/src/components/product-import/steps/ImageUploadStep/components/ProductCard/ProductCard.tsx

@@ -1,7 +1,7 @@
 import { Button } from "@/components/ui/button";
 import { Input } from "@/components/ui/input";
 import { Card, CardContent } from "@/components/ui/card";
-import { Loader2, Link as LinkIcon } from "lucide-react";
+import { Loader2, Link as LinkIcon, Image as ImageIcon } from "lucide-react";
 import { cn } from "@/lib/utils";
 import { ImageDropzone } from "./ImageDropzone";
 import { SortableImage } from "./SortableImage";
@@ -9,6 +9,25 @@ import { CopyButton } from "./CopyButton";
 import { ProductImageSortable, Product } from "../../types";
 import { DroppableContainer } from "../DroppableContainer";
 import { SortableContext, horizontalListSortingStrategy } from '@dnd-kit/sortable';
+import { useQuery } from "@tanstack/react-query";
+import config from "@/config";
+import {
+  Dialog,
+  DialogContent,
+  DialogDescription,
+  DialogHeader,
+  DialogTitle,
+} from "@/components/ui/dialog";
+import { ScrollArea } from "@/components/ui/scroll-area";
+import { useState, useMemo } from "react";
+
+interface ReusableImage {
+  id: number;
+  name: string;
+  image_url: string;
+  is_global: boolean;
+  company: string | null;
+}
 
 interface ProductCardProps {
   product: Product;
@@ -26,6 +45,7 @@ interface ProductCardProps {
   onRemoveImage: (id: string) => void;
   getProductContainerClasses: () => string;
   findContainer: (id: string) => string | null;
+  handleAddImageFromUrl: (productIndex: number, url: string) => void;
 }
 
 export const ProductCard = ({
@@ -43,8 +63,11 @@ export const ProductCard = ({
   onDragOver,
   onRemoveImage,
   getProductContainerClasses,
-  findContainer
+  findContainer,
+  handleAddImageFromUrl
 }: ProductCardProps) => {
+  const [isReusableDialogOpen, setIsReusableDialogOpen] = useState(false);
+
   // Function to get images for this product
   const getProductImages = () => {
     return productImages.filter(img => img.productIndex === index);
@@ -56,6 +79,32 @@ export const ProductCard = ({
     return result !== null ? parseInt(result) : null;
   };
 
+  // Fetch reusable images
+  const { data: reusableImages, isLoading: isLoadingReusable } = useQuery<ReusableImage[]>({
+    queryKey: ["reusable-images"],
+    queryFn: async () => {
+      const response = await fetch(`${config.apiUrl}/reusable-images`);
+      if (!response.ok) {
+        throw new Error("Failed to fetch reusable images");
+      }
+      return response.json();
+    },
+  });
+
+  // Filter reusable images based on product's company
+  const availableReusableImages = useMemo(() => {
+    if (!reusableImages) return [];
+    return reusableImages.filter(img => 
+      img.is_global || img.company === product.company
+    );
+  }, [reusableImages, product.company]);
+
+  // Handle adding a reusable image
+  const handleAddReusableImage = (imageUrl: string) => {
+    handleAddImageFromUrl(index, imageUrl);
+    setIsReusableDialogOpen(false);
+  };
+
   return (
     <Card 
       className={cn(
@@ -83,6 +132,18 @@ export const ProductCard = ({
                 className="flex items-center gap-2"
                 onSubmit={onUrlSubmit}
               >
+                {getProductImages().length === 0 && (
+                  <Button
+                    type="button"
+                    variant="outline"
+                    size="sm"
+                    className="h-8 whitespace-nowrap flex gap-1 items-center text-xs"
+                    onClick={() => setIsReusableDialogOpen(true)}
+                  >
+                    <ImageIcon className="h-3.5 w-3.5" />
+                    Select from Library
+                  </Button>
+                )}
                 <Input
                   placeholder="Add image from URL"
                   value={urlInput}
@@ -105,7 +166,7 @@ export const ProductCard = ({
           </div>
         
           <div className="flex flex-col sm:flex-row gap-2">
-            <div className="flex flex-row gap-2 items-start">
+            <div className="flex flex-row gap-2 items-center gap-4">
               <ImageDropzone 
                 productIndex={index} 
                 onDrop={onImageUpload}
@@ -158,6 +219,50 @@ export const ProductCard = ({
           />
         </div>
       </CardContent>
+
+      {/* Reusable Images Dialog */}
+      <Dialog open={isReusableDialogOpen} onOpenChange={setIsReusableDialogOpen}>
+        <DialogContent className="max-w-3xl">
+          <DialogHeader>
+            <DialogTitle>Select from Image Library</DialogTitle>
+            <DialogDescription>
+              Choose a global or company-specific image to add to this product.
+            </DialogDescription>
+          </DialogHeader>
+          <ScrollArea className="h-[400px] pr-4">
+            {isLoadingReusable ? (
+              <div className="flex items-center justify-center h-full">
+                <Loader2 className="h-8 w-8 animate-spin" />
+              </div>
+            ) : availableReusableImages.length === 0 ? (
+              <div className="flex flex-col items-center justify-center h-full text-muted-foreground">
+                <ImageIcon className="h-8 w-8 mb-2" />
+                <p>No reusable images available</p>
+              </div>
+            ) : (
+              <div className="grid grid-cols-2 sm:grid-cols-3 md:grid-cols-4 gap-4">
+                {availableReusableImages.map((image) => (
+                  <div
+                    key={image.id}
+                    className="group relative aspect-square border rounded-lg overflow-hidden cursor-pointer hover:ring-2 hover:ring-primary"
+                    onClick={() => handleAddReusableImage(image.image_url)}
+                  >
+                    <img
+                      src={image.image_url}
+                      alt={image.name}
+                      className="w-full h-full object-cover"
+                    />
+                    <div className="absolute inset-0 bg-black/0 group-hover:bg-black/20 transition-colors" />
+                    <div className="absolute bottom-0 left-0 right-0 p-2 bg-gradient-to-t from-black/60 to-transparent">
+                      <p className="text-xs text-white truncate">{image.name}</p>
+                    </div>
+                  </div>
+                ))}
+              </div>
+            )}
+          </ScrollArea>
+        </DialogContent>
+      </Dialog>
     </Card>
   );
 };

inventory/src/components/product-import/steps/ImageUploadStep/types.ts

@@ -31,5 +31,6 @@ export interface Product {
   supplier_no?: string;
   sku?: string;
   model?: string;
+  company?: string;
   product_images?: string | string[];
 } 

inventory/src/components/product-import/steps/ValidationStepNew/ValidationStepNew.tsx

@@ -1,108 +1,61 @@
 import React, { useState } from 'react';
-import { AiValidationDialogs } from '../../../components/AiValidationDialogs';
-import { Product } from '../../../types/product';
-import { config } from '../../../config';
-
-interface CurrentPrompt {
-  isOpen: boolean;
-  prompt: string;
-  isLoading: boolean;
-  debugData?: {
-    taxonomyStats: {
-      categories: number;
-      themes: number;
-      colors: number;
-      taxCodes: number;
-      sizeCategories: number;
-      suppliers: number;
-      companies: number;
-      artists: number;
-    } | null;
-    basePrompt: string;
-    sampleFullPrompt: string;
-    promptLength: number;
-    estimatedProcessingTime?: {
-      seconds: number | null;
-      sampleCount: number;
-    };
-  };
-}
+import { AiValidationDialogs } from './components/AiValidationDialogs';
+import { Product } from '../../../../types/products';
+import { 
+  AiValidationProgress, 
+  AiValidationDetails, 
+  CurrentPrompt as AiValidationCurrentPrompt 
+} from './hooks/useAiValidation';
 
 const ValidationStepNew: React.FC = () => {
-  const [aiValidationProgress, setAiValidationProgress] = useState(0);
-  const [aiValidationDetails, setAiValidationDetails] = useState('');
-  const [currentPrompt, setCurrentPrompt] = useState<CurrentPrompt>({
+  const [aiValidationProgress, setAiValidationProgress] = useState<AiValidationProgress>({
+    isOpen: false,
+    status: 'idle',
+    step: 0
+  });
+  
+  const [aiValidationDetails, setAiValidationDetails] = useState<AiValidationDetails>({
+    changes: [],
+    warnings: [],
+    changeDetails: [],
+    isOpen: false
+  });
+  
+  const [currentPrompt, setCurrentPrompt] = useState<AiValidationCurrentPrompt>({
     isOpen: false,
     prompt: '',
     isLoading: true,
   });
-  const [isChangeReverted, setIsChangeReverted] = useState(false);
-  const [fieldData, setFieldData] = useState<Product[]>([]);
+  
+  // Track reversion state (for internal use)
+  const [reversionState, setReversionState] = useState<Record<string, boolean>>({});
+  
+  const [fieldData] = useState<Product[]>([]);
 
-  const showCurrentPrompt = async (products: Product[]) => {
-    setCurrentPrompt((prev) => ({ ...prev, isOpen: true, isLoading: true }));
-    
-    try {
-      // Get the prompt
-      const promptResponse = await fetch(`${config.apiUrl}/ai-validation/prompt`, {
-        method: 'POST',
-        headers: { 'Content-Type': 'application/json' },
-        body: JSON.stringify({ products })
-      });
-      
-      if (!promptResponse.ok) {
-        throw new Error('Failed to fetch AI prompt');
-      }
-      
-      const promptData = await promptResponse.json();
-      
-      // Get the debug data in the same request or as a separate request
-      const debugResponse = await fetch(`${config.apiUrl}/ai-validation/debug-info`, {
-        method: 'POST',
-        headers: { 'Content-Type': 'application/json' },
-        body: JSON.stringify({ prompt: promptData.prompt })
-      });
-      
-      let debugData;
-      if (debugResponse.ok) {
-        debugData = await debugResponse.json();
-      } else {
-        // If debug-info fails, use a fallback to get taxonomy stats
-        const fallbackResponse = await fetch(`${config.apiUrl}/ai-validation/debug`, {
-          method: 'POST',
-          headers: { 'Content-Type': 'application/json' },
-          body: JSON.stringify({ products: [products[0]] }) // Use first product for stats
-        });
-        
-        if (fallbackResponse.ok) {
-          debugData = await fallbackResponse.json();
-          // Set promptLength correctly from the actual prompt
-          debugData.promptLength = promptData.prompt.length;
-        }
-      }
-      
-      setCurrentPrompt((prev) => ({
-        ...prev,
-        prompt: promptData.prompt,
-        isLoading: false,
-        debugData: debugData
-      }));
-    } catch (error) {
-      console.error('Error fetching prompt:', error);
-      setCurrentPrompt((prev) => ({
-        ...prev,
-        prompt: 'Error loading prompt',
-        isLoading: false
-      }));
-    }
+  // eslint-disable-next-line @typescript-eslint/no-unused-vars
+
+  const revertAiChange = (productIndex: number, fieldKey: string) => {
+    const key = `${productIndex}-${fieldKey}`;
+    setReversionState(prev => ({
+      ...prev,
+      [key]: true
+    }));
   };
 
-  const revertAiChange = () => {
-    setIsChangeReverted(true);
+  const isChangeReverted = (productIndex: number, fieldKey: string): boolean => {
+    const key = `${productIndex}-${fieldKey}`;
+    return !!reversionState[key];
   };
 
-  const getFieldDisplayValueWithHighlight = (value: string, highlight: string) => {
-    // Implementation of getFieldDisplayValueWithHighlight
+  const getFieldDisplayValueWithHighlight = (
+    _fieldKey: string,
+    originalValue: any,
+    correctedValue: any
+  ) => {
+    return {
+      originalHtml: String(originalValue),
+      correctedHtml: String(correctedValue)
+    };
   };
 
   return (

inventory/src/components/product-import/steps/ValidationStepNew/components/AiValidationDialogs.tsx

@@ -8,7 +8,7 @@ import {
 } from "@/components/ui/dialog";
 import { ScrollArea } from "@/components/ui/scroll-area";
 import { Button } from "@/components/ui/button";
-import { Loader2, CheckIcon } from "lucide-react";
+import { Loader2, CheckIcon, XIcon } from "lucide-react";
 import { Code } from "@/components/ui/code";
 import {
   Table,
@@ -24,6 +24,7 @@ import {
   CurrentPrompt,
 } from "../hooks/useAiValidation";
 import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
+import { Badge } from "@/components/ui/badge";
 
 interface TaxonomyStats {
   categories: number;
@@ -41,6 +42,20 @@ interface DebugData {
   basePrompt: string;
   sampleFullPrompt: string;
   promptLength: number;
+  apiFormat?: Array<{
+    role: string;
+    content: string;
+  }>;
+  promptSources?: {
+    systemPrompt?: { id: number; prompt_text: string };
+    generalPrompt?: { id: number; prompt_text: string };
+    companyPrompts?: Array<{
+      id: number;
+      company: string;
+      companyName?: string;
+      prompt_text: string;
+    }>;
+  };
   estimatedProcessingTime?: {
     seconds: number | null;
     sampleCount: number;
@@ -83,6 +98,75 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
   debugData,
 }) => {
   const [costPerMillionTokens, setCostPerMillionTokens] = useState(2.5); // Default cost
+  const hasCompanyPrompts =
+    currentPrompt.debugData?.promptSources?.companyPrompts &&
+    currentPrompt.debugData.promptSources.companyPrompts.length > 0;
+
+  // Create our own state to track changes
+  const [localReversionState, setLocalReversionState] = useState<
+    Record<string, boolean>
+  >({});
+
+  // Initialize local state from the isChangeReverted function when component mounts
+  // or when aiValidationDetails changes
+  React.useEffect(() => {
+    if (
+      aiValidationDetails.changeDetails &&
+      aiValidationDetails.changeDetails.length > 0
+    ) {
+      const initialState: Record<string, boolean> = {};
+
+      aiValidationDetails.changeDetails.forEach((product) => {
+        product.changes.forEach((change) => {
+          const key = `${product.productIndex}-${change.field}`;
+          initialState[key] = isChangeReverted(
+            product.productIndex,
+            change.field
+          );
+        });
+      });
+
+      setLocalReversionState(initialState);
+    }
+  }, [aiValidationDetails.changeDetails, isChangeReverted]);
+
+  // This function will toggle the local state for a given change
+  const toggleChangeAcceptance = (productIndex: number, fieldKey: string) => {
+    const key = `${productIndex}-${fieldKey}`;
+    const currentlyRejected = !!localReversionState[key];
+
+    // Toggle the local state
+    setLocalReversionState((prev) => ({
+      ...prev,
+      [key]: !prev[key],
+    }));
+
+    // Only call revertAiChange when toggling to rejected state
+    // Since revertAiChange is specifically for rejecting changes
+    if (!currentlyRejected) {
+      revertAiChange(productIndex, fieldKey);
+    }
+  };
+
+  // Function to check local reversion state
+  const isChangeLocallyReverted = (
+    productIndex: number,
+    fieldKey: string
+  ): boolean => {
+    const key = `${productIndex}-${fieldKey}`;
+    return !!localReversionState[key];
+  };
+
+  // Use "full" as the default tab
+  const defaultTab = "full";
+  const [activeTab, setActiveTab] = useState(defaultTab);
+
+  // Update activeTab when the dialog is opened with new data
+  React.useEffect(() => {
+    if (currentPrompt.isOpen) {
+      setActiveTab("full");
+    }
+  }, [currentPrompt.isOpen]);
 
   // Format time helper
   const formatTime = (seconds: number): string => {
@@ -123,136 +207,433 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
           </DialogHeader>
 
           <div className="flex flex-col h-[calc(90vh-120px)] overflow-hidden">
-            {/* Debug Information Section */}
-            <div className="mb-4 flex-shrink-0">
+            {/* Debug Information Section - Fixed at the top */}
+            <div className="flex-shrink-0">
               {currentPrompt.isLoading ? (
                 <div className="flex justify-center items-center h-[100px]"></div>
               ) : (
-                <div className="grid grid-cols-3 gap-4">
-                  <Card className="py-2">
-                    <CardHeader className="py-2">
-                      <CardTitle className="text-base">Prompt Length</CardTitle>
-                    </CardHeader>
-                    <CardContent className="py-2">
-                      <div className="flex flex-col space-y-2">
-                        <div className="text-sm">
-                          <span className="text-muted-foreground">
-                            Characters:
-                          </span>{" "}
-                          <span className="font-semibold">{promptLength}</span>
+                <>
+                  <div className="grid grid-cols-3 gap-4 mb-4">
+                    <Card className="py-2">
+                      <CardHeader className="py-2">
+                        <CardTitle className="text-base">
+                          Prompt Length
+                        </CardTitle>
+                      </CardHeader>
+                      <CardContent className="py-2">
+                        <div className="flex flex-col space-y-2">
+                          <div className="text-sm">
+                            <span className="text-muted-foreground">
+                              Characters:
+                            </span>{" "}
+                            <span className="font-semibold">
+                              {promptLength}
+                            </span>
+                          </div>
+                          <div className="text-sm">
+                            <span className="text-muted-foreground">
+                              Tokens:
+                            </span>{" "}
+                            <span className="font-semibold">
+                              ~{Math.round(promptLength / 4)}
+                            </span>
+                          </div>
                         </div>
-                        <div className="text-sm">
-                          <span className="text-muted-foreground">Tokens:</span>{" "}
-                          <span className="font-semibold">
-                            ~{Math.round(promptLength / 4)}
-                          </span>
-                        </div>
-                      </div>
-                    </CardContent>
-                  </Card>
+                      </CardContent>
+                    </Card>
 
-                  <Card className="py-2">
-                    <CardHeader className="py-2">
-                      <CardTitle className="text-base">Cost Estimate</CardTitle>
-                    </CardHeader>
-                    <CardContent className="py-2">
-                      <div className="flex flex-col space-y-2">
-                        <div className="flex flex-row items-center">
-                          <label
-                            htmlFor="costPerMillion"
-                            className="text-sm text-muted-foreground"
-                          >
-                            $
-                          </label>
-                          <input
-                            id="costPerMillion"
-                            className="w-[40px] px-1 border rounded-md text-sm"
-                            defaultValue={costPerMillionTokens.toFixed(2)}
-                            onChange={(e) => {
-                              const value = parseFloat(e.target.value);
-                              if (!isNaN(value)) {
-                                setCostPerMillionTokens(value);
-                              }
-                            }}
-                          />
-                          <label
-                            htmlFor="costPerMillion"
-                            className="text-sm text-muted-foreground ml-1"
-                          >
-                            per million input tokens
-                          </label>
+                    <Card className="py-2">
+                      <CardHeader className="py-2">
+                        <CardTitle className="text-base">
+                          Cost Estimate
+                        </CardTitle>
+                      </CardHeader>
+                      <CardContent className="py-2">
+                        <div className="flex flex-col space-y-2">
+                          <div className="flex flex-row items-center">
+                            <label
+                              htmlFor="costPerMillion"
+                              className="text-sm text-muted-foreground"
+                            >
+                              $
+                            </label>
+                            <input
+                              id="costPerMillion"
+                              className="w-[40px] px-1 border rounded-md text-sm"
+                              defaultValue={costPerMillionTokens.toFixed(2)}
+                              onChange={(e) => {
+                                const value = parseFloat(e.target.value);
+                                if (!isNaN(value)) {
+                                  setCostPerMillionTokens(value);
+                                }
+                              }}
+                            />
+                            <label
+                              htmlFor="costPerMillion"
+                              className="text-sm text-muted-foreground ml-1"
+                            >
+                              per million input tokens
+                            </label>
+                          </div>
+                          <div className="text-sm">
+                            <span className="text-muted-foreground">Cost:</span>{" "}
+                            <span className="font-semibold">
+                              {calculateTokenCost(promptLength).toFixed(1)}¢
+                            </span>
+                          </div>
                         </div>
-                        <div className="text-sm">
-                          <span className="text-muted-foreground">Cost:</span>{" "}
-                          <span className="font-semibold">
-                            {calculateTokenCost(promptLength).toFixed(1)}¢
-                          </span>
-                        </div>
-                      </div>
-                    </CardContent>
-                  </Card>
+                      </CardContent>
+                    </Card>
 
-                  <Card className="py-2">
-                    <CardHeader className="py-2">
-                      <CardTitle className="text-base">
-                        Processing Time
-                      </CardTitle>
-                    </CardHeader>
-                    <CardContent className="py-2">
-                      <div className="flex flex-col space-y-2">
-                        {debugData?.estimatedProcessingTime ? (
-                          debugData.estimatedProcessingTime.seconds ? (
-                            <>
-                              <div className="text-sm">
-                                <span className="text-muted-foreground">
-                                  Estimated time:
-                                </span>{" "}
-                                <span className="font-semibold">
-                                  {formatTime(
-                                    debugData.estimatedProcessingTime.seconds
-                                  )}
-                                </span>
+                    <Card className="py-2">
+                      <CardHeader className="py-2">
+                        <CardTitle className="text-base">
+                          Processing Time
+                        </CardTitle>
+                      </CardHeader>
+                      <CardContent className="py-2">
+                        <div className="flex flex-col space-y-2">
+                          {currentPrompt.debugData?.estimatedProcessingTime ? (
+                            currentPrompt.debugData.estimatedProcessingTime
+                              .seconds ? (
+                              <>
+                                <div className="text-sm">
+                                  <span className="text-muted-foreground">
+                                    Estimated time:
+                                  </span>{" "}
+                                  <span className="font-semibold">
+                                    {formatTime(
+                                      currentPrompt.debugData
+                                        .estimatedProcessingTime.seconds
+                                    )}
+                                  </span>
+                                </div>
+                                <div className="text-xs text-muted-foreground">
+                                  Based on{" "}
+                                  {
+                                    currentPrompt.debugData
+                                      .estimatedProcessingTime.sampleCount
+                                  }{" "}
+                                  similar validation
+                                  {currentPrompt.debugData
+                                    .estimatedProcessingTime.sampleCount !== 1
+                                    ? "s"
+                                    : ""}
+                                </div>
+                              </>
+                            ) : (
+                              <div className="text-sm text-muted-foreground">
+                                No historical data available for this prompt
+                                size
                               </div>
-                              <div className="text-xs text-muted-foreground">
-                                Based on{" "}
-                                {debugData.estimatedProcessingTime.sampleCount}{" "}
-                                similar validation
-                                {debugData.estimatedProcessingTime
-                                  .sampleCount !== 1
-                                  ? "s"
-                                  : ""}
-                              </div>
-                            </>
+                            )
                           ) : (
                             <div className="text-sm text-muted-foreground">
-                              No historical data available for this prompt size
+                              No processing time data available
                             </div>
-                          )
-                        ) : (
-                          <div className="text-sm text-muted-foreground">
-                            No processing time data available
-                          </div>
-                        )}
-                      </div>
-                    </CardContent>
-                  </Card>
-                </div>
+                          )}
+                        </div>
+                      </CardContent>
+                    </Card>
+                  </div>
+                </>
               )}
             </div>
 
-            {/* Prompt Section */}
+            {/* Prompt Section - Scrollable content */}
             <div className="flex-1 min-h-0">
-              <ScrollArea className="h-full w-full">
-                {currentPrompt.isLoading ? (
-                  <div className="flex items-center justify-center h-full">
-                    <Loader2 className="h-8 w-8 animate-spin" />
-                  </div>
-                ) : (
-                  <Code className="whitespace-pre-wrap p-4 break-normal max-w-full">
-                    {currentPrompt.prompt}
-                  </Code>
-                )}
-              </ScrollArea>
+              {currentPrompt.isLoading ? (
+                <div className="flex items-center justify-center h-full">
+                  <Loader2 className="h-8 w-8 animate-spin" />
+                </div>
+              ) : (
+                <>
+                  {currentPrompt.debugData?.apiFormat ? (
+                    <div className="flex flex-col h-full">
+                      {/* Prompt Sources Card - Fixed at the top of the content area */}
+                      <Card className="py-2 mb-4 flex-shrink-0">
+                        <CardHeader className="py-2">
+                          <CardTitle className="text-base">
+                            Prompt Sources
+                          </CardTitle>
+                        </CardHeader>
+                        <CardContent className="py-2">
+                          <div className="flex flex-wrap gap-2">
+                            <Badge
+                              variant="outline"
+                              className="bg-purple-100 hover:bg-purple-200 cursor-pointer"
+                              onClick={() =>
+                                document
+                                  .getElementById("system-message")
+                                  ?.scrollIntoView({ behavior: "smooth" })
+                              }
+                            >
+                              System
+                            </Badge>
+                            <Badge
+                              variant="outline"
+                              className="bg-green-100 hover:bg-green-200 cursor-pointer"
+                              onClick={() =>
+                                document
+                                  .getElementById("general-section")
+                                  ?.scrollIntoView({ behavior: "smooth" })
+                              }
+                            >
+                              General
+                            </Badge>
+
+                            {currentPrompt.debugData.promptSources?.companyPrompts?.map(
+                              (company, idx) => (
+                                <Badge
+                                  key={idx}
+                                  variant="outline"
+                                  className="bg-blue-100 hover:bg-blue-200 cursor-pointer"
+                                  onClick={() =>
+                                    document
+                                      .getElementById("company-section")
+                                      ?.scrollIntoView({ behavior: "smooth" })
+                                  }
+                                >
+                                  {company.companyName ||
+                                    `Company ${company.company}`}
+                                </Badge>
+                              )
+                            )}
+
+                            <Badge
+                              variant="outline"
+                              className="bg-amber-100 hover:bg-amber-200 cursor-pointer"
+                              onClick={() =>
+                                document
+                                  .getElementById("taxonomy-section")
+                                  ?.scrollIntoView({ behavior: "smooth" })
+                              }
+                            >
+                              Taxonomy
+                            </Badge>
+                            <Badge
+                              variant="outline"
+                              className="bg-pink-100 hover:bg-pink-200 cursor-pointer"
+                              onClick={() =>
+                                document
+                                  .getElementById("product-section")
+                                  ?.scrollIntoView({ behavior: "smooth" })
+                              }
+                            >
+                              Products
+                            </Badge>
+                          </div>
+                        </CardContent>
+                      </Card>
+
+                      <ScrollArea className="flex-1 w-full overflow-y-auto">
+                        {currentPrompt.debugData.apiFormat.map(
+                          (message, idx: number) => (
+                            <div
+                              key={idx}
+                              className="border rounded-md p-2 mb-4"
+                            >
+                              <div
+                                id={
+                                  message.role === "system"
+                                    ? "system-message"
+                                    : ""
+                                }
+                                className={`p-2 mb-2 rounded-sm font-medium ${
+                                  message.role === "system"
+                                    ? "bg-purple-50 text-purple-800"
+                                    : "bg-green-50 text-green-800"
+                                }`}
+                              >
+                                Role: {message.role}
+                              </div>
+
+                              <Code
+                                className={`whitespace-pre-wrap p-4 break-normal max-w-full ${
+                                  message.role === "system"
+                                    ? "bg-purple-50/30"
+                                    : "bg-green-50/30"
+                                }`}
+                              >
+                                {message.role === "user" ? (
+                                  <div className="text-wrapper">
+                                    {(() => {
+                                      const content = message.content;
+
+                                      // Find section boundaries by looking for specific markers
+                                      const companySpecificStartIndex =
+                                        content.indexOf(
+                                          "--- COMPANY-SPECIFIC INSTRUCTIONS ---"
+                                        );
+                                      const companySpecificEndIndex =
+                                        content.indexOf(
+                                          "--- END COMPANY-SPECIFIC INSTRUCTIONS ---"
+                                        );
+
+                                      const taxonomyStartIndex =
+                                        content.indexOf(
+                                          "All Available Categories:"
+                                        );
+                                      const taxonomyFallbackStartIndex =
+                                        content.indexOf(
+                                          "Available Categories:"
+                                        );
+                                      const actualTaxonomyStartIndex =
+                                        taxonomyStartIndex >= 0
+                                          ? taxonomyStartIndex
+                                          : taxonomyFallbackStartIndex;
+
+                                      const productDataStartIndex =
+                                        content.indexOf(
+                                          "----------Here is the product data to validate----------"
+                                        );
+
+                                      // If we can't find any markers, just return the content as-is
+                                      if (
+                                        actualTaxonomyStartIndex < 0 &&
+                                        productDataStartIndex < 0 &&
+                                        companySpecificStartIndex < 0
+                                      ) {
+                                        return content;
+                                      }
+
+                                      // Determine section indices
+                                      let generalEndIndex = content.length;
+
+                                      if (companySpecificStartIndex >= 0) {
+                                        generalEndIndex =
+                                          companySpecificStartIndex;
+                                      } else if (
+                                        actualTaxonomyStartIndex >= 0
+                                      ) {
+                                        generalEndIndex =
+                                          actualTaxonomyStartIndex;
+                                      } else if (productDataStartIndex >= 0) {
+                                        generalEndIndex = productDataStartIndex;
+                                      }
+
+                                      // Determine where taxonomy starts
+                                      let taxonomyEndIndex = content.length;
+                                      if (productDataStartIndex >= 0) {
+                                        taxonomyEndIndex =
+                                          productDataStartIndex;
+                                      }
+
+                                      // Segments to render with appropriate styling
+                                      const segments = [];
+
+                                      // General section (beginning to company/taxonomy/product)
+                                      if (generalEndIndex > 0) {
+                                        segments.push(
+                                          <div
+                                            id="general-section"
+                                            key="general"
+                                            className="border-l-4 border-green-500 pl-4 py-0 my-1"
+                                          >
+                                            <div className="text-xs font-semibold text-green-700 mb-2">
+                                              General Prompt
+                                            </div>
+                                            <pre className="whitespace-pre-wrap">
+                                              {content.substring(
+                                                0,
+                                                generalEndIndex
+                                              )}
+                                            </pre>
+                                          </div>
+                                        );
+                                      }
+
+                                      // Company-specific section if present
+                                      if (
+                                        companySpecificStartIndex >= 0 &&
+                                        companySpecificEndIndex >= 0
+                                      ) {
+                                        segments.push(
+                                          <div
+                                            id="company-section"
+                                            key="company"
+                                            className="border-l-4 border-blue-500 pl-4 py-0 my-1"
+                                          >
+                                            <div className="text-xs font-semibold text-blue-700 mb-2">
+                                              Company-Specific Instructions
+                                            </div>
+                                            <pre className="whitespace-pre-wrap">
+                                              {content.substring(
+                                                companySpecificStartIndex,
+                                                companySpecificEndIndex +
+                                                  "--- END COMPANY-SPECIFIC INSTRUCTIONS ---"
+                                                    .length
+                                              )}
+                                            </pre>
+                                          </div>
+                                        );
+                                      }
+
+                                      // Taxonomy section
+                                      if (actualTaxonomyStartIndex >= 0) {
+                                        const taxEnd = taxonomyEndIndex;
+                                        segments.push(
+                                          <div
+                                            id="taxonomy-section"
+                                            key="taxonomy"
+                                            className="border-l-4 border-amber-500 pl-4 py-0 my-1"
+                                          >
+                                            <div className="text-xs font-semibold text-amber-700 mb-2">
+                                              Taxonomy Data
+                                            </div>
+                                            <pre className="whitespace-pre-wrap">
+                                              {content.substring(
+                                                actualTaxonomyStartIndex,
+                                                taxEnd
+                                              )}
+                                            </pre>
+                                          </div>
+                                        );
+                                      }
+
+                                      // Product data section
+                                      if (productDataStartIndex >= 0) {
+                                        segments.push(
+                                          <div
+                                            id="product-section"
+                                            key="product"
+                                            className="border-l-4 border-pink-500 pl-4 py-0 my-1"
+                                          >
+                                            <div className="text-xs font-semibold text-pink-700 mb-2">
+                                              Product Data
+                                            </div>
+                                            <pre className="whitespace-pre-wrap">
+                                              {content.substring(
+                                                productDataStartIndex
+                                              )}
+                                            </pre>
+                                          </div>
+                                        );
+                                      }
+
+                                      return <>{segments}</>;
+                                    })()}
+                                  </div>
+                                ) : (
+                                  <pre className="whitespace-pre-wrap">
+                                    {message.content}
+                                  </pre>
+                                )}
+                              </Code>
+                            </div>
+                          )
+                        )}
+                      </ScrollArea>
+                    </div>
+                  ) : (
+                    <ScrollArea className="h-full w-full">
+                      <Code className="whitespace-pre-wrap p-4 break-normal max-w-full">
+                        {currentPrompt.prompt}
+                      </Code>
+                    </ScrollArea>
+                  )}
+                </>
+              )}
             </div>
           </div>
         </DialogContent>
@@ -280,8 +661,9 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                     className="h-full bg-primary transition-all duration-500"
                     style={{
                       width: `${
-                        aiValidationProgress.progressPercent ??
-                        Math.round((aiValidationProgress.step / 5) * 100)
+                        aiValidationProgress.progressPercent !== undefined
+                          ? Math.round(aiValidationProgress.progressPercent)
+                          : Math.round((aiValidationProgress.step / 5) * 100)
                       }%`,
                       backgroundColor:
                         aiValidationProgress.step === -1
@@ -295,8 +677,9 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                 {aiValidationProgress.step === -1
                   ? "❌"
                   : `${
-                      aiValidationProgress.progressPercent ??
-                      Math.round((aiValidationProgress.step / 5) * 100)
+                      aiValidationProgress.progressPercent !== undefined
+                        ? Math.round(aiValidationProgress.progressPercent)
+                        : Math.round((aiValidationProgress.step / 5) * 100)
                     }%`}
               </div>
             </div>
@@ -355,14 +738,14 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
           setAiValidationDetails((prev) => ({ ...prev, isOpen: open }))
         }
       >
-        <DialogContent className="max-w-4xl">
+        <DialogContent className="max-w-6xl w-[90vw]">
           <DialogHeader>
             <DialogTitle>AI Validation Results</DialogTitle>
             <DialogDescription>
               Review the changes and warnings suggested by the AI
             </DialogDescription>
           </DialogHeader>
-          <ScrollArea className="max-h-[60vh]">
+          <ScrollArea className="max-h-[70vh]">
             {aiValidationDetails.changeDetails &&
             aiValidationDetails.changeDetails.length > 0 ? (
               <div className="mb-6 space-y-6">
@@ -384,10 +767,16 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                       <Table>
                         <TableHeader>
                           <TableRow>
-                            <TableHead className="w-[180px]">Field</TableHead>
-                            <TableHead>Original Value</TableHead>
-                            <TableHead>Corrected Value</TableHead>
-                            <TableHead className="text-right">Action</TableHead>
+                            <TableHead className="">Field</TableHead>
+                            <TableHead className="w-[35%]">
+                              Original Value
+                            </TableHead>
+                            <TableHead className="w-[35%]">
+                              Corrected Value
+                            </TableHead>
+                            <TableHead className="text-right">
+                              Accept Changes?
+                            </TableHead>
                           </TableRow>
                         </TableHeader>
                         <TableBody>
@@ -398,7 +787,7 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                             const fieldLabel = field
                               ? field.label
                               : change.field;
-                            const isReverted = isChangeReverted(
+                            const isReverted = isChangeLocallyReverted(
                               product.productIndex,
                               change.field
                             );
@@ -421,7 +810,6 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                                     dangerouslySetInnerHTML={{
                                       __html: originalHtml,
                                     }}
-                                    className={isReverted ? "font-medium" : ""}
                                   />
                                 </TableCell>
                                 <TableCell>
@@ -429,36 +817,46 @@ export const AiValidationDialogs: React.FC<AiValidationDialogsProps> = ({
                                     dangerouslySetInnerHTML={{
                                       __html: correctedHtml,
                                     }}
-                                    className={!isReverted ? "font-medium" : ""}
                                   />
                                 </TableCell>
-                                <TableCell className="text-right">
-                                  <div className="mt-2">
-                                    {isReverted ? (
-                                      <Button
-                                        variant="ghost"
-                                        size="sm"
-                                        className="text-green-600 bg-green-50 hover:bg-green-100 hover:text-green-700"
-                                        disabled
-                                      >
-                                        <CheckIcon className="w-4 h-4 mr-1" />
-                                        Reverted
-                                      </Button>
-                                    ) : (
-                                      <Button
-                                        variant="outline"
-                                        size="sm"
-                                        onClick={() => {
-                                          // Call the revert function directly
-                                          revertAiChange(
-                                            product.productIndex,
-                                            change.field
-                                          );
-                                        }}
-                                      >
-                                        Revert Change
-                                      </Button>
-                                    )}
+                                <TableCell className="text-right align-top">
+                                  <div className="flex justify-end gap-2">
+                                    <Button
+                                      variant="outline"
+                                      size="sm"
+                                      onClick={() => {
+                                        // Toggle to Accepted state if currently rejected
+                                        toggleChangeAcceptance(
+                                          product.productIndex,
+                                          change.field
+                                        );
+                                      }}
+                                      className={
+                                        !isReverted
+                                          ? "bg-green-100 text-green-600 border-green-300 flex items-center"
+                                          : "border-gray-200 text-gray-600 hover:bg-green-50 hover:text-green-600 hover:border-green-200 flex items-center"
+                                      }
+                                    >
+                                      <CheckIcon className="h-4 w-4" />
+                                    </Button>
+                                    <Button
+                                      variant="outline"
+                                      size="sm"
+                                      onClick={() => {
+                                        // Toggle to Rejected state if currently accepted
+                                        toggleChangeAcceptance(
+                                          product.productIndex,
+                                          change.field
+                                        );
+                                      }}
+                                      className={
+                                        isReverted
+                                          ? "bg-red-100 text-red-600 border-red-300 flex items-center"
+                                          : "border-gray-200 text-gray-600 hover:bg-red-50 hover:text-red-600 hover:border-red-200 flex items-center"
+                                      }
+                                    >
+                                      <XIcon className="h-4 w-4" />
+                                    </Button>
                                   </div>
                                 </TableCell>
                               </TableRow>

inventory/src/components/product-import/steps/ValidationStepNew/hooks/useAiValidation.tsx

@@ -56,6 +56,20 @@ export interface CurrentPrompt {
     basePrompt: string;
     sampleFullPrompt: string;
     promptLength: number;
+    apiFormat?: Array<{
+      role: string;
+      content: string;
+    }>;
+    promptSources?: {
+      systemPrompt?: { id: number; prompt_text: string };
+      generalPrompt?: { id: number; prompt_text: string };
+      companyPrompts?: Array<{ 
+        id: number; 
+        company: string; 
+        companyName: string; 
+        prompt_text: string 
+      }>;
+    };
     estimatedProcessingTime?: {
       seconds: number | null;
       sampleCount: number;
@@ -323,7 +337,9 @@ export const useAiValidation = <T extends string>(
             basePrompt: result.basePrompt || '',
             sampleFullPrompt: result.sampleFullPrompt || '',
             promptLength: result.promptLength || (promptContent ? promptContent.length : 0),
-            estimatedProcessingTime: result.estimatedProcessingTime
+            promptSources: result.promptSources,
+            estimatedProcessingTime: result.estimatedProcessingTime,
+            apiFormat: result.apiFormat
           }
         }));
       } else {
@@ -490,6 +506,27 @@ export const useAiValidation = <T extends string>(
         throw new Error(result.error || 'AI validation failed');
       }
 
+      // Store the prompt sources if they exist
+      if (result.promptSources) {
+        setCurrentPrompt(prev => {
+          // Create debugData if it doesn't exist
+          const prevDebugData = prev.debugData || {
+            taxonomyStats: null,
+            basePrompt: '',
+            sampleFullPrompt: '',
+            promptLength: 0
+          };
+          
+          return { 
+            ...prev, 
+            debugData: {
+              ...prevDebugData,
+              promptSources: result.promptSources
+            }
+          };
+        });
+      }
+
       // Update progress with actual processing time if available
       if (result.performanceMetrics) {
         console.log('Performance metrics:', result.performanceMetrics);

inventory/src/components/settings/PromptManagement.tsx

@@ -0,0 +1,584 @@
+import { useState, useMemo } from "react";
+import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
+import { Button } from "@/components/ui/button";
+import {
+  Table,
+  TableBody,
+  TableCell,
+  TableHead,
+  TableHeader,
+  TableRow,
+} from "@/components/ui/table";
+import { Input } from "@/components/ui/input";
+import { Textarea } from "@/components/ui/textarea";
+import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
+import { Label } from "@/components/ui/label";
+import { ArrowUpDown, Pencil, Trash2, PlusCircle } from "lucide-react";
+import config from "@/config";
+import {
+  useReactTable,
+  getCoreRowModel,
+  getSortedRowModel,
+  SortingState,
+  flexRender,
+  type ColumnDef,
+} from "@tanstack/react-table";
+import {
+  Dialog,
+  DialogContent,
+  DialogDescription,
+  DialogFooter,
+  DialogHeader,
+  DialogTitle,
+} from "@/components/ui/dialog";
+import {
+  AlertDialog,
+  AlertDialogAction,
+  AlertDialogCancel,
+  AlertDialogContent,
+  AlertDialogDescription,
+  AlertDialogFooter,
+  AlertDialogHeader,
+  AlertDialogTitle,
+} from "@/components/ui/alert-dialog";
+import { toast } from "sonner";
+
+interface FieldOption {
+  label: string;
+  value: string;
+}
+
+interface PromptFormData {
+  id?: number;
+  prompt_text: string;
+  prompt_type: 'general' | 'company_specific' | 'system';
+  company: string | null;
+}
+
+interface AiPrompt {
+  id: number;
+  prompt_text: string;
+  prompt_type: 'general' | 'company_specific' | 'system';
+  company: string | null;
+  created_at: string;
+  updated_at: string;
+}
+
+interface FieldOptions {
+  companies: FieldOption[];
+}
+
+export function PromptManagement() {
+  const [isFormOpen, setIsFormOpen] = useState(false);
+  const [isDeleteOpen, setIsDeleteOpen] = useState(false);
+  const [promptToDelete, setPromptToDelete] = useState<AiPrompt | null>(null);
+  const [editingPrompt, setEditingPrompt] = useState<AiPrompt | null>(null);
+  const [sorting, setSorting] = useState<SortingState>([
+    { id: "prompt_type", desc: true },
+    { id: "company", desc: false }
+  ]);
+  const [searchQuery, setSearchQuery] = useState("");
+  const [formData, setFormData] = useState<PromptFormData>({
+    prompt_text: "",
+    prompt_type: "general",
+    company: null,
+  });
+
+  const queryClient = useQueryClient();
+
+  const { data: prompts, isLoading } = useQuery<AiPrompt[]>({
+    queryKey: ["ai-prompts"],
+    queryFn: async () => {
+      const response = await fetch(`${config.apiUrl}/ai-prompts`);
+      if (!response.ok) {
+        throw new Error("Failed to fetch AI prompts");
+      }
+      return response.json();
+    },
+  });
+
+  const { data: fieldOptions } = useQuery<FieldOptions>({
+    queryKey: ["fieldOptions"],
+    queryFn: async () => {
+      const response = await fetch(`${config.apiUrl}/import/field-options`);
+      if (!response.ok) {
+        throw new Error("Failed to fetch field options");
+      }
+      return response.json();
+    },
+  });
+
+  // Check if general and system prompts already exist
+  const generalPromptExists = useMemo(() => {
+    return prompts?.some(prompt => prompt.prompt_type === 'general');
+  }, [prompts]);
+  
+  const systemPromptExists = useMemo(() => {
+    return prompts?.some(prompt => prompt.prompt_type === 'system');
+  }, [prompts]);
+
+  const createMutation = useMutation({
+    mutationFn: async (data: PromptFormData) => {
+      const response = await fetch(`${config.apiUrl}/ai-prompts`, {
+        method: "POST",
+        headers: {
+          "Content-Type": "application/json",
+        },
+        body: JSON.stringify(data),
+      });
+      
+      if (!response.ok) {
+        const error = await response.json();
+        throw new Error(error.message || error.error || "Failed to create prompt");
+      }
+      
+      return response.json();
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["ai-prompts"] });
+      toast.success("Prompt created successfully");
+      resetForm();
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to create prompt");
+    },
+  });
+
+  const updateMutation = useMutation({
+    mutationFn: async (data: PromptFormData) => {
+      if (!data.id) throw new Error("Prompt ID is required for update");
+      
+      const response = await fetch(`${config.apiUrl}/ai-prompts/${data.id}`, {
+        method: "PUT",
+        headers: {
+          "Content-Type": "application/json",
+        },
+        body: JSON.stringify(data),
+      });
+      
+      if (!response.ok) {
+        const error = await response.json();
+        throw new Error(error.message || error.error || "Failed to update prompt");
+      }
+      
+      return response.json();
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["ai-prompts"] });
+      toast.success("Prompt updated successfully");
+      resetForm();
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to update prompt");
+    },
+  });
+
+  const deleteMutation = useMutation({
+    mutationFn: async (id: number) => {
+      const response = await fetch(`${config.apiUrl}/ai-prompts/${id}`, {
+        method: "DELETE",
+      });
+      if (!response.ok) {
+        throw new Error("Failed to delete prompt");
+      }
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["ai-prompts"] });
+      toast.success("Prompt deleted successfully");
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to delete prompt");
+    },
+  });
+
+  const handleEdit = (prompt: AiPrompt) => {
+    setEditingPrompt(prompt);
+    setFormData({
+      id: prompt.id,
+      prompt_text: prompt.prompt_text,
+      prompt_type: prompt.prompt_type,
+      company: prompt.company,
+    });
+    setIsFormOpen(true);
+  };
+
+  const handleDeleteClick = (prompt: AiPrompt) => {
+    setPromptToDelete(prompt);
+    setIsDeleteOpen(true);
+  };
+
+  const handleDeleteConfirm = () => {
+    if (promptToDelete) {
+      deleteMutation.mutate(promptToDelete.id);
+      setIsDeleteOpen(false);
+      setPromptToDelete(null);
+    }
+  };
+
+  const handleSubmit = (e: React.FormEvent) => {
+    e.preventDefault();
+    
+    // If prompt_type is general or system, ensure company is null
+    const submitData = {
+      ...formData,
+      company: formData.prompt_type === 'company_specific' ? formData.company : null,
+    };
+    
+    if (editingPrompt) {
+      updateMutation.mutate(submitData);
+    } else {
+      createMutation.mutate(submitData);
+    }
+  };
+
+  const resetForm = () => {
+    setFormData({
+      prompt_text: "",
+      prompt_type: "general",
+      company: null,
+    });
+    setEditingPrompt(null);
+    setIsFormOpen(false);
+  };
+
+  const handleCreateClick = () => {
+    resetForm();
+    
+    // If general prompt and system prompt exist, default to company-specific
+    if (generalPromptExists && systemPromptExists) {
+      setFormData(prev => ({
+        ...prev,
+        prompt_type: 'company_specific'
+      }));
+    } else if (generalPromptExists && !systemPromptExists) {
+      // If general exists but system doesn't, suggest system prompt
+      setFormData(prev => ({
+        ...prev,
+        prompt_type: 'system'
+      }));
+    } else if (!generalPromptExists) {
+      // If no general prompt, suggest that first
+      setFormData(prev => ({
+        ...prev,
+        prompt_type: 'general'
+      }));
+    }
+    
+    setIsFormOpen(true);
+  };
+
+  const columns = useMemo<ColumnDef<AiPrompt>[]>(() => [
+    {
+      accessorKey: "prompt_type",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Type
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => {
+        const type = row.getValue("prompt_type") as string;
+        if (type === 'general') return 'General';
+        if (type === 'system') return 'System';
+        return 'Company Specific';
+      },
+    },
+    {
+      accessorFn: (row) => row.prompt_text.length,
+      id: "length",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Length
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ getValue }) => {
+        const length = getValue() as number;
+        return <span>{length.toLocaleString()}</span>;
+      },
+    },
+    {
+      accessorKey: "company",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Company
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => {
+        const companyId = row.getValue("company");
+        if (!companyId) return 'N/A';
+        return fieldOptions?.companies.find(c => c.value === companyId)?.label || companyId;
+      },
+    },
+    {
+      accessorKey: "updated_at",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Last Updated
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => new Date(row.getValue("updated_at")).toLocaleDateString(),
+    },
+    {
+      id: "actions",
+      cell: ({ row }) => (
+        <div className="flex gap-2 justify-end pr-4">
+          <Button
+            variant="ghost"
+            onClick={() => handleEdit(row.original)}
+          >
+            <Pencil className="h-4 w-4" />
+            Edit
+          </Button>
+          <Button
+            variant="ghost"
+            className="text-destructive hover:text-destructive"
+            onClick={() => handleDeleteClick(row.original)}
+          >
+            <Trash2 className="h-4 w-4" />
+            Delete
+          </Button>
+        </div>
+      ),
+    },
+  ], [fieldOptions]);
+
+  const filteredData = useMemo(() => {
+    if (!prompts) return [];
+    return prompts.filter((prompt) => {
+      const searchString = searchQuery.toLowerCase();
+      return (
+        prompt.prompt_type.toLowerCase().includes(searchString) ||
+        (prompt.company && prompt.company.toLowerCase().includes(searchString))
+      );
+    });
+  }, [prompts, searchQuery]);
+
+  const table = useReactTable({
+    data: filteredData,
+    columns,
+    state: {
+      sorting,
+    },
+    onSortingChange: setSorting,
+    getSortedRowModel: getSortedRowModel(),
+    getCoreRowModel: getCoreRowModel(),
+  });
+
+  return (
+    <div className="space-y-6">
+      <div className="flex items-center justify-between">
+        <h2 className="text-2xl font-bold">AI Validation Prompts</h2>
+        <Button onClick={handleCreateClick}>
+          <PlusCircle className="mr-2 h-4 w-4" />
+          Create New Prompt
+        </Button>
+      </div>
+
+      <div className="flex items-center gap-4">
+        <Input
+          placeholder="Search prompts..."
+          value={searchQuery}
+          onChange={(e) => setSearchQuery(e.target.value)}
+          className="max-w-sm"
+        />
+      </div>
+
+      {isLoading ? (
+        <div>Loading prompts...</div>
+      ) : (
+        <div className="border rounded-lg">
+          <Table>
+            <TableHeader className="bg-muted">
+              {table.getHeaderGroups().map((headerGroup) => (
+                <TableRow key={headerGroup.id}>
+                  {headerGroup.headers.map((header) => (
+                    <TableHead key={header.id}>
+                      {header.isPlaceholder
+                        ? null
+                        : flexRender(
+                            header.column.columnDef.header,
+                            header.getContext()
+                          )}
+                    </TableHead>
+                  ))}
+                </TableRow>
+              ))}
+            </TableHeader>
+            <TableBody>
+              {table.getRowModel().rows?.length ? (
+                table.getRowModel().rows.map((row) => (
+                  <TableRow key={row.id} className="hover:bg-gray-100">
+                    {row.getVisibleCells().map((cell) => (
+                      <TableCell key={cell.id} className="pl-6">
+                        {flexRender(cell.column.columnDef.cell, cell.getContext())}
+                      </TableCell>
+                    ))}
+                  </TableRow>
+                ))
+              ) : (
+                <TableRow>
+                  <TableCell colSpan={columns.length} className="text-center">
+                    No prompts found
+                  </TableCell>
+                </TableRow>
+              )}
+            </TableBody>
+          </Table>
+        </div>
+      )}
+
+      {/* Prompt Form Dialog */}
+      <Dialog open={isFormOpen} onOpenChange={setIsFormOpen}>
+        <DialogContent className="max-w-3xl">
+          <DialogHeader>
+            <DialogTitle>{editingPrompt ? "Edit Prompt" : "Create New Prompt"}</DialogTitle>
+            <DialogDescription>
+              {editingPrompt 
+                ? "Update this AI validation prompt."
+                : "Create a new AI validation prompt that will be used during product validation."}
+            </DialogDescription>
+          </DialogHeader>
+          
+          <form onSubmit={handleSubmit}>
+            <div className="grid gap-4 py-4">
+              <div className="grid gap-2">
+                <Label htmlFor="prompt_type">Prompt Type</Label>
+                <Select
+                  value={formData.prompt_type}
+                  onValueChange={(value: 'general' | 'company_specific' | 'system') => 
+                    setFormData({ ...formData, prompt_type: value })
+                  }
+                  disabled={(generalPromptExists && formData.prompt_type !== 'general' && !editingPrompt?.id) || 
+                           (systemPromptExists && formData.prompt_type !== 'system' && !editingPrompt?.id)}
+                >
+                  <SelectTrigger>
+                    <SelectValue placeholder="Select prompt type" />
+                  </SelectTrigger>
+                  <SelectContent>
+                    <SelectItem 
+                      value="general" 
+                      disabled={generalPromptExists && !editingPrompt?.prompt_type?.includes('general')}
+                    >
+                      General
+                    </SelectItem>
+                    <SelectItem 
+                      value="system" 
+                      disabled={systemPromptExists && !editingPrompt?.prompt_type?.includes('system')}
+                    >
+                      System
+                    </SelectItem>
+                    <SelectItem value="company_specific">Company Specific</SelectItem>
+                  </SelectContent>
+                </Select>
+                {generalPromptExists && formData.prompt_type !== 'general' && !editingPrompt?.id && systemPromptExists && formData.prompt_type !== 'system' && (
+                  <p className="text-xs text-muted-foreground">
+                    General and system prompts already exist. You can only create company-specific prompts.
+                  </p>
+                )}
+                {generalPromptExists && !systemPromptExists && formData.prompt_type !== 'general' && !editingPrompt?.id && (
+                  <p className="text-xs text-muted-foreground">
+                    A general prompt already exists. You can create a system prompt or company-specific prompts.
+                  </p>
+                )}
+                {systemPromptExists && !generalPromptExists && formData.prompt_type !== 'system' && !editingPrompt?.id && (
+                  <p className="text-xs text-muted-foreground">
+                    A system prompt already exists. You can create a general prompt or company-specific prompts.
+                  </p>
+                )}
+              </div>
+              
+              {formData.prompt_type === 'company_specific' && (
+                <div className="grid gap-2">
+                  <Label htmlFor="company">Company</Label>
+                  <Select
+                    value={formData.company || ''}
+                    onValueChange={(value) => setFormData({ ...formData, company: value })}
+                    required={formData.prompt_type === 'company_specific'}
+                  >
+                    <SelectTrigger>
+                      <SelectValue placeholder="Select company" />
+                    </SelectTrigger>
+                    <SelectContent>
+                      {fieldOptions?.companies.map((company) => (
+                        <SelectItem key={company.value} value={company.value}>
+                          {company.label}
+                        </SelectItem>
+                      ))}
+                    </SelectContent>
+                  </Select>
+                </div>
+              )}
+              
+              <div className="grid gap-2">
+                <Label htmlFor="prompt_text">Prompt Text</Label>
+                <Textarea
+                  id="prompt_text"
+                  value={formData.prompt_text}
+                  onChange={(e) => setFormData({ ...formData, prompt_text: e.target.value })}
+                  placeholder={`Enter your ${formData.prompt_type === 'system' ? 'system instructions' : 'validation prompt'} text...`}
+                  className="h-80 font-mono text-sm"
+                  required
+                />
+                {formData.prompt_type === 'system' && (
+                  <p className="text-xs text-muted-foreground mt-1">
+                    System prompts provide the initial instructions to the AI. This sets the tone and approach for all validations.
+                  </p>
+                )}
+              </div>
+            </div>
+            
+            <DialogFooter>
+              <Button type="button" variant="outline" onClick={() => {
+                resetForm();
+                setIsFormOpen(false);
+              }}>
+                Cancel
+              </Button>
+              <Button type="submit">
+                {editingPrompt ? "Update" : "Create"} Prompt
+              </Button>
+            </DialogFooter>
+          </form>
+        </DialogContent>
+      </Dialog>
+
+      {/* Delete Confirmation Dialog */}
+      <AlertDialog open={isDeleteOpen} onOpenChange={setIsDeleteOpen}>
+        <AlertDialogContent>
+          <AlertDialogHeader>
+            <AlertDialogTitle>Delete Prompt</AlertDialogTitle>
+            <AlertDialogDescription>
+              Are you sure you want to delete this prompt? This action cannot be undone.
+            </AlertDialogDescription>
+          </AlertDialogHeader>
+          <AlertDialogFooter>
+            <AlertDialogCancel onClick={() => {
+              setIsDeleteOpen(false);
+              setPromptToDelete(null);
+            }}>
+              Cancel
+            </AlertDialogCancel>
+            <AlertDialogAction onClick={handleDeleteConfirm}>
+              Delete
+            </AlertDialogAction>
+          </AlertDialogFooter>
+        </AlertDialogContent>
+      </AlertDialog>
+    </div>
+  );
+} 

inventory/src/components/settings/ReusableImageManagement.tsx

@@ -0,0 +1,773 @@
+import { useState, useMemo, useCallback, useRef, useEffect } from "react";
+import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
+import { Button } from "@/components/ui/button";
+import {
+  Table,
+  TableBody,
+  TableCell,
+  TableHead,
+  TableHeader,
+  TableRow,
+} from "@/components/ui/table";
+import { Input } from "@/components/ui/input";
+import { Label } from "@/components/ui/label";
+import { Checkbox } from "@/components/ui/checkbox";
+import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
+import { ArrowUpDown, Pencil, Trash2, PlusCircle, Image, Eye } from "lucide-react";
+import config from "@/config";
+import {
+  useReactTable,
+  getCoreRowModel,
+  getSortedRowModel,
+  SortingState,
+  flexRender,
+  type ColumnDef,
+} from "@tanstack/react-table";
+import {
+  Dialog,
+  DialogContent,
+  DialogDescription,
+  DialogFooter,
+  DialogHeader,
+  DialogTitle,
+  DialogClose
+} from "@/components/ui/dialog";
+import {
+  AlertDialog,
+  AlertDialogAction,
+  AlertDialogCancel,
+  AlertDialogContent,
+  AlertDialogDescription,
+  AlertDialogFooter,
+  AlertDialogHeader,
+  AlertDialogTitle,
+} from "@/components/ui/alert-dialog";
+import { toast } from "sonner";
+import { useDropzone } from "react-dropzone";
+import { cn } from "@/lib/utils";
+
+interface FieldOption {
+  label: string;
+  value: string;
+}
+
+interface ImageFormData {
+  id?: number;
+  name: string;
+  is_global: boolean;
+  company: string | null;
+  file?: File;
+}
+
+interface ReusableImage {
+  id: number;
+  name: string;
+  filename: string;
+  file_path: string;
+  image_url: string;
+  is_global: boolean;
+  company: string | null;
+  mime_type: string;
+  file_size: number;
+  created_at: string;
+  updated_at: string;
+}
+
+interface FieldOptions {
+  companies: FieldOption[];
+}
+
+const ImageForm = ({ 
+  editingImage, 
+  formData, 
+  setFormData, 
+  onSubmit, 
+  onCancel, 
+  fieldOptions,
+  getRootProps,
+  getInputProps,
+  isDragActive 
+}: {
+  editingImage: ReusableImage | null;
+  formData: ImageFormData;
+  setFormData: (data: ImageFormData) => void;
+  onSubmit: (e: React.FormEvent) => void;
+  onCancel: () => void;
+  fieldOptions: FieldOptions | undefined;
+  getRootProps: any;
+  getInputProps: any;
+  isDragActive: boolean;
+}) => {
+  const handleNameChange = useCallback((e: React.ChangeEvent<HTMLInputElement>) => {
+    setFormData(prev => ({ ...prev, name: e.target.value }));
+  }, [setFormData]);
+
+  const handleGlobalChange = useCallback((checked: boolean) => {
+    setFormData(prev => ({ 
+      ...prev, 
+      is_global: checked,
+      company: checked ? null : prev.company
+    }));
+  }, [setFormData]);
+
+  const handleCompanyChange = useCallback((value: string) => {
+    setFormData(prev => ({ ...prev, company: value }));
+  }, [setFormData]);
+
+  return (
+    <form onSubmit={onSubmit}>
+      <div className="grid gap-4 py-4">
+        <div className="grid gap-2">
+          <Label htmlFor="image_name">Image Name</Label>
+          <Input
+            id="image_name"
+            name="image_name"
+            value={formData.name}
+            onChange={handleNameChange}
+            placeholder="Enter image name"
+            required
+          />
+        </div>
+        
+        {!editingImage && (
+          <div className="grid gap-2">
+            <Label htmlFor="image">Upload Image</Label>
+            <div 
+              {...getRootProps()}
+              className={cn(
+                "border-2 border-dashed border-secondary-foreground/30 bg-muted/90 rounded-md w-full py-6 flex flex-col items-center justify-center cursor-pointer hover:bg-muted/70 transition-colors",
+                isDragActive && "border-primary bg-muted"
+              )}
+            >
+              <input {...getInputProps()} />
+              <div className="flex flex-col items-center justify-center py-2">
+                {formData.file ? (
+                  <>
+                    <div className="mb-4">
+                      <ImagePreview file={formData.file} />
+                    </div>
+                    <div className="flex items-center gap-2 mb-2">
+                      <Image className="h-4 w-4 text-primary" />
+                      <span className="text-sm">{formData.file.name}</span>
+                    </div>
+                    <p className="text-xs text-muted-foreground">Click or drag to replace</p>
+                  </>
+                ) : isDragActive ? (
+                  <>
+                    <Image className="h-8 w-8 mb-2 text-primary" />
+                    <p className="text-base text-muted-foreground">Drop image here</p>
+                  </>
+                ) : (
+                  <>
+                    <Image className="h-8 w-8 mb-2 text-muted-foreground" />
+                    <p className="text-base text-muted-foreground">Click or drag to upload</p>
+                  </>
+                )}
+              </div>
+            </div>
+          </div>
+        )}
+        
+        <div className="flex items-center space-x-2">
+          <Checkbox 
+            id="is_global" 
+            checked={formData.is_global}
+            onCheckedChange={handleGlobalChange}
+          />
+          <Label htmlFor="is_global">Available for all companies</Label>
+        </div>
+        
+        {!formData.is_global && (
+          <div className="grid gap-2">
+            <Label htmlFor="company">Company</Label>
+            <Select
+              value={formData.company || ''}
+              onValueChange={handleCompanyChange}
+              required={!formData.is_global}
+            >
+              <SelectTrigger>
+                <SelectValue placeholder="Select company" />
+              </SelectTrigger>
+              <SelectContent>
+                {fieldOptions?.companies.map((company) => (
+                  <SelectItem key={company.value} value={company.value}>
+                    {company.label}
+                  </SelectItem>
+                ))}
+              </SelectContent>
+            </Select>
+          </div>
+        )}
+      </div>
+      
+      <DialogFooter>
+        <Button type="button" variant="outline" onClick={onCancel}>
+          Cancel
+        </Button>
+        <Button type="submit">
+          {editingImage ? "Update" : "Upload"} Image
+        </Button>
+      </DialogFooter>
+    </form>
+  );
+};
+
+export function ReusableImageManagement() {
+  const [isFormOpen, setIsFormOpen] = useState(false);
+  const [isDeleteOpen, setIsDeleteOpen] = useState(false);
+  const [isPreviewOpen, setIsPreviewOpen] = useState(false);
+  const [imageToDelete, setImageToDelete] = useState<ReusableImage | null>(null);
+  const [previewImage, setPreviewImage] = useState<ReusableImage | null>(null);
+  const [editingImage, setEditingImage] = useState<ReusableImage | null>(null);
+  const [sorting, setSorting] = useState<SortingState>([
+    { id: "created_at", desc: true }
+  ]);
+  const [searchQuery, setSearchQuery] = useState("");
+  const [formData, setFormData] = useState<ImageFormData>({
+    name: "",
+    is_global: false,
+    company: null,
+    file: undefined
+  });
+
+  const queryClient = useQueryClient();
+
+  const { data: images, isLoading } = useQuery<ReusableImage[]>({
+    queryKey: ["reusable-images"],
+    queryFn: async () => {
+      const response = await fetch(`${config.apiUrl}/reusable-images`);
+      if (!response.ok) {
+        throw new Error("Failed to fetch reusable images");
+      }
+      return response.json();
+    },
+  });
+
+  const { data: fieldOptions } = useQuery<FieldOptions>({
+    queryKey: ["fieldOptions"],
+    queryFn: async () => {
+      const response = await fetch(`${config.apiUrl}/import/field-options`);
+      if (!response.ok) {
+        throw new Error("Failed to fetch field options");
+      }
+      return response.json();
+    },
+  });
+
+  const createMutation = useMutation({
+    mutationFn: async (data: ImageFormData) => {
+      // Create FormData for file upload
+      const formData = new FormData();
+      formData.append('name', data.name);
+      formData.append('is_global', String(data.is_global));
+      
+      if (!data.is_global && data.company) {
+        formData.append('company', data.company);
+      }
+      
+      if (data.file) {
+        formData.append('image', data.file);
+      } else {
+        throw new Error("Image file is required");
+      }
+      
+      const response = await fetch(`${config.apiUrl}/reusable-images/upload`, {
+        method: "POST",
+        body: formData,
+      });
+      
+      if (!response.ok) {
+        const error = await response.json();
+        throw new Error(error.message || error.error || "Failed to upload image");
+      }
+      
+      return response.json();
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["reusable-images"] });
+      toast.success("Image uploaded successfully");
+      resetForm();
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to upload image");
+    },
+  });
+
+  const updateMutation = useMutation({
+    mutationFn: async (data: ImageFormData) => {
+      if (!data.id) throw new Error("Image ID is required for update");
+      
+      const response = await fetch(`${config.apiUrl}/reusable-images/${data.id}`, {
+        method: "PUT",
+        headers: {
+          "Content-Type": "application/json",
+        },
+        body: JSON.stringify({
+          name: data.name,
+          is_global: data.is_global,
+          company: data.is_global ? null : data.company
+        }),
+      });
+      
+      if (!response.ok) {
+        const error = await response.json();
+        throw new Error(error.message || error.error || "Failed to update image");
+      }
+      
+      return response.json();
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["reusable-images"] });
+      toast.success("Image updated successfully");
+      resetForm();
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to update image");
+    },
+  });
+
+  const deleteMutation = useMutation({
+    mutationFn: async (id: number) => {
+      const response = await fetch(`${config.apiUrl}/reusable-images/${id}`, {
+        method: "DELETE",
+      });
+      if (!response.ok) {
+        throw new Error("Failed to delete image");
+      }
+    },
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: ["reusable-images"] });
+      toast.success("Image deleted successfully");
+    },
+    onError: (error) => {
+      toast.error(error instanceof Error ? error.message : "Failed to delete image");
+    },
+  });
+
+  const handleEdit = (image: ReusableImage) => {
+    setEditingImage(image);
+    setFormData({
+      id: image.id,
+      name: image.name,
+      is_global: image.is_global,
+      company: image.company,
+    });
+    setIsFormOpen(true);
+  };
+
+  const handleDeleteClick = (image: ReusableImage) => {
+    setImageToDelete(image);
+    setIsDeleteOpen(true);
+  };
+
+  const handlePreview = (image: ReusableImage) => {
+    setPreviewImage(image);
+    setIsPreviewOpen(true);
+  };
+
+  const handleDeleteConfirm = () => {
+    if (imageToDelete) {
+      deleteMutation.mutate(imageToDelete.id);
+      setIsDeleteOpen(false);
+      setImageToDelete(null);
+    }
+  };
+
+  const handleSubmit = (e: React.FormEvent) => {
+    e.preventDefault();
+    
+    // If is_global is true, ensure company is null
+    const submitData = {
+      ...formData,
+      company: formData.is_global ? null : formData.company,
+    };
+    
+    if (editingImage) {
+      updateMutation.mutate(submitData);
+    } else {
+      if (!submitData.file) {
+        toast.error("Please select an image file");
+        return;
+      }
+      createMutation.mutate(submitData);
+    }
+  };
+
+  const resetForm = () => {
+    setFormData({
+      name: "",
+      is_global: false,
+      company: null,
+      file: undefined
+    });
+    setEditingImage(null);
+    setIsFormOpen(false);
+  };
+
+  const handleCreateClick = () => {
+    resetForm();
+    setIsFormOpen(true);
+  };
+
+  // Configure dropzone for image uploads
+  const onDrop = useCallback((acceptedFiles: File[]) => {
+    if (acceptedFiles.length > 0) {
+      const file = acceptedFiles[0]; // Take only the first file
+      setFormData(prev => ({
+        ...prev,
+        file
+      }));
+    }
+  }, []);
+
+  const { getRootProps, getInputProps, isDragActive } = useDropzone({
+    accept: {
+      'image/*': ['.jpeg', '.jpg', '.png', '.gif', '.webp']
+    },
+    onDrop,
+    multiple: false // Only accept single files
+  });
+
+  const columns = useMemo<ColumnDef<ReusableImage>[]>(() => [
+    {
+      accessorKey: "name",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Name
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+    },
+    {
+      accessorKey: "is_global",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Type
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => {
+        const isGlobal = row.getValue("is_global") as boolean;
+        return isGlobal ? "Global" : "Company Specific";
+      },
+    },
+    {
+      accessorKey: "company",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Company
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => {
+        const isGlobal = row.getValue("is_global") as boolean;
+        if (isGlobal) return 'N/A';
+        
+        const companyId = row.getValue("company");
+        if (!companyId) return 'None';
+        return fieldOptions?.companies.find(c => c.value === companyId)?.label || companyId;
+      },
+    },
+    {
+      accessorKey: "file_size",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Size
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => {
+        const size = row.getValue("file_size") as number;
+        return `${(size / 1024).toFixed(1)} KB`;
+      },
+    },
+    {
+      accessorKey: "created_at",
+      header: ({ column }) => (
+        <Button
+          variant="ghost"
+          onClick={() => column.toggleSorting(column.getIsSorted() === "asc")}
+        >
+          Created
+          <ArrowUpDown className="ml-2 h-4 w-4" />
+        </Button>
+      ),
+      cell: ({ row }) => new Date(row.getValue("created_at")).toLocaleDateString(),
+    },
+    {
+      accessorKey: "image_url",
+      header: "Thumbnail",
+      cell: ({ row }) => (
+        <div className="flex items-center justify-center">
+          <img 
+            src={row.getValue("image_url") as string} 
+            alt={row.getValue("name") as string}
+            className="w-10 h-10 object-contain border rounded"
+          />
+        </div>
+      ),
+    },
+    {
+      id: "actions",
+      cell: ({ row }) => (
+        <div className="flex gap-2 justify-end">
+          <Button
+            variant="ghost"
+            size="icon"
+            onClick={() => handlePreview(row.original)}
+            title="Preview Image"
+          >
+            <Eye className="h-4 w-4" />
+          </Button>
+          <Button
+            variant="ghost"
+            size="icon"
+            onClick={() => handleEdit(row.original)}
+            title="Edit Image"
+          >
+            <Pencil className="h-4 w-4" />
+          </Button>
+          <Button
+            variant="ghost"
+            size="icon"
+            className="text-destructive hover:text-destructive"
+            onClick={() => handleDeleteClick(row.original)}
+            title="Delete Image"
+          >
+            <Trash2 className="h-4 w-4" />
+          </Button>
+        </div>
+      ),
+    },
+  ], [fieldOptions]);
+
+  const filteredData = useMemo(() => {
+    if (!images) return [];
+    return images.filter((image) => {
+      const searchString = searchQuery.toLowerCase();
+      return (
+        image.name.toLowerCase().includes(searchString) ||
+        (image.is_global ? "global" : "company").includes(searchString) ||
+        (image.company && image.company.toLowerCase().includes(searchString))
+      );
+    });
+  }, [images, searchQuery]);
+
+  const table = useReactTable({
+    data: filteredData,
+    columns,
+    state: {
+      sorting,
+    },
+    onSortingChange: setSorting,
+    getSortedRowModel: getSortedRowModel(),
+    getCoreRowModel: getCoreRowModel(),
+  });
+
+  return (
+    <div className="space-y-6">
+      <div className="flex items-center justify-between">
+        <h2 className="text-2xl font-bold">Reusable Images</h2>
+        <Button onClick={handleCreateClick}>
+          <PlusCircle className="mr-2 h-4 w-4" />
+          Upload New Image
+        </Button>
+      </div>
+
+      <div className="flex items-center gap-4">
+        <Input
+          placeholder="Search images..."
+          value={searchQuery}
+          onChange={(e) => setSearchQuery(e.target.value)}
+          className="max-w-sm"
+        />
+      </div>
+
+      {isLoading ? (
+        <div>Loading images...</div>
+      ) : (
+        <div className="border rounded-lg">
+          <Table>
+            <TableHeader className="bg-muted">
+              {table.getHeaderGroups().map((headerGroup) => (
+                <TableRow key={headerGroup.id}>
+                  {headerGroup.headers.map((header) => (
+                    <TableHead key={header.id}>
+                      {header.isPlaceholder
+                        ? null
+                        : flexRender(
+                            header.column.columnDef.header,
+                            header.getContext()
+                          )}
+                    </TableHead>
+                  ))}
+                </TableRow>
+              ))}
+            </TableHeader>
+            <TableBody>
+              {table.getRowModel().rows?.length ? (
+                table.getRowModel().rows.map((row) => (
+                  <TableRow key={row.id} className="hover:bg-gray-100">
+                    {row.getVisibleCells().map((cell) => (
+                      <TableCell key={cell.id} className="pl-6">
+                        {flexRender(cell.column.columnDef.cell, cell.getContext())}
+                      </TableCell>
+                    ))}
+                  </TableRow>
+                ))
+              ) : (
+                <TableRow>
+                  <TableCell colSpan={columns.length} className="text-center">
+                    No images found
+                  </TableCell>
+                </TableRow>
+              )}
+            </TableBody>
+          </Table>
+        </div>
+      )}
+
+      {/* Image Form Dialog */}
+      <Dialog open={isFormOpen} onOpenChange={setIsFormOpen}>
+        <DialogContent className="max-w-md">
+          <DialogHeader>
+            <DialogTitle>{editingImage ? "Edit Image" : "Upload New Image"}</DialogTitle>
+            <DialogDescription>
+              {editingImage 
+                ? "Update this reusable image's details."
+                : "Upload a new reusable image that can be used across products."}
+            </DialogDescription>
+          </DialogHeader>
+          
+          <ImageForm
+            editingImage={editingImage}
+            formData={formData}
+            setFormData={setFormData}
+            onSubmit={handleSubmit}
+            onCancel={() => {
+              resetForm();
+              setIsFormOpen(false);
+            }}
+            fieldOptions={fieldOptions}
+            getRootProps={getRootProps}
+            getInputProps={getInputProps}
+            isDragActive={isDragActive}
+          />
+        </DialogContent>
+      </Dialog>
+
+      {/* Delete Confirmation Dialog */}
+      <AlertDialog open={isDeleteOpen} onOpenChange={setIsDeleteOpen}>
+        <AlertDialogContent>
+          <AlertDialogHeader>
+            <AlertDialogTitle>Delete Image</AlertDialogTitle>
+            <AlertDialogDescription>
+              Are you sure you want to delete this image? This action cannot be undone.
+            </AlertDialogDescription>
+          </AlertDialogHeader>
+          <AlertDialogFooter>
+            <AlertDialogCancel onClick={() => {
+              setIsDeleteOpen(false);
+              setImageToDelete(null);
+            }}>
+              Cancel
+            </AlertDialogCancel>
+            <AlertDialogAction onClick={handleDeleteConfirm}>
+              Delete
+            </AlertDialogAction>
+          </AlertDialogFooter>
+        </AlertDialogContent>
+      </AlertDialog>
+
+      {/* Preview Dialog */}
+      <Dialog open={isPreviewOpen} onOpenChange={setIsPreviewOpen}>
+        <DialogContent className="max-w-3xl">
+          <DialogHeader>
+            <DialogTitle>{previewImage?.name}</DialogTitle>
+            <DialogDescription>
+              {previewImage?.is_global 
+                ? "Global image" 
+                : `Company specific image for ${fieldOptions?.companies.find(c => c.value === previewImage?.company)?.label}`}
+            </DialogDescription>
+          </DialogHeader>
+          
+          <div className="flex justify-center p-4">
+            {previewImage && (
+              <div className="bg-checkerboard rounded-md overflow-hidden">
+                <img 
+                  src={previewImage.image_url} 
+                  alt={previewImage.name}
+                  className="max-h-[500px] max-w-full object-contain"
+                />
+              </div>
+            )}
+          </div>
+          
+          <div className="grid grid-cols-2 gap-4 text-sm">
+            <div>
+              <span className="font-medium">Filename:</span> {previewImage?.filename}
+            </div>
+            <div>
+              <span className="font-medium">Size:</span> {previewImage && `${(previewImage.file_size / 1024).toFixed(1)} KB`}
+            </div>
+            <div>
+              <span className="font-medium">Type:</span> {previewImage?.mime_type}
+            </div>
+            <div>
+              <span className="font-medium">Uploaded:</span> {previewImage && new Date(previewImage.created_at).toLocaleString()}
+            </div>
+          </div>
+          
+          <DialogFooter>
+            <DialogClose asChild>
+              <Button>Close</Button>
+            </DialogClose>
+          </DialogFooter>
+        </DialogContent>
+      </Dialog>
+
+      <style jsx global>{`
+        .bg-checkerboard {
+          background-image: linear-gradient(45deg, #f0f0f0 25%, transparent 25%),
+                            linear-gradient(-45deg, #f0f0f0 25%, transparent 25%),
+                            linear-gradient(45deg, transparent 75%, #f0f0f0 75%),
+                            linear-gradient(-45deg, transparent 75%, #f0f0f0 75%);
+          background-size: 20px 20px;
+          background-position: 0 0, 0 10px, 10px -10px, -10px 0px;
+        }
+      `}</style>
+    </div>
+  );
+}
+
+const ImagePreview = ({ file }: { file: File }) => {
+  const [previewUrl, setPreviewUrl] = useState<string>('');
+
+  useEffect(() => {
+    const url = URL.createObjectURL(file);
+    setPreviewUrl(url);
+    return () => {
+      URL.revokeObjectURL(url);
+    };
+  }, [file]);
+
+  return (
+    <img 
+      src={previewUrl}
+      alt="Preview"
+      className="max-h-32 max-w-full object-contain rounded-md"
+    />
+  );
+}; 

inventory/src/pages/Settings.tsx

@@ -5,6 +5,8 @@ import { PerformanceMetrics } from "@/components/settings/PerformanceMetrics";
 import { CalculationSettings } from "@/components/settings/CalculationSettings";
 import { TemplateManagement } from "@/components/settings/TemplateManagement";
 import { UserManagement } from "@/components/settings/UserManagement";
+import { PromptManagement } from "@/components/settings/PromptManagement";
+import { ReusableImageManagement } from "@/components/settings/ReusableImageManagement";
 import { motion } from 'framer-motion';
 import { Alert, AlertDescription } from "@/components/ui/alert";
 import { Protected } from "@/components/auth/Protected";
@@ -41,6 +43,8 @@ const SETTINGS_GROUPS: SettingsGroup[] = [
     label: "Content Management", 
     tabs: [
       { id: "templates", permission: "settings:templates", label: "Template Management" },
+      { id: "ai-prompts", permission: "settings:prompt_management", label: "AI Prompts" },
+      { id: "reusable-images", permission: "settings:library_management", label: "Reusable Images" },
     ]
   },
   {
@@ -216,6 +220,36 @@ export function Settings() {
             </Protected>
           </TabsContent>
           
+          <TabsContent value="ai-prompts" className="mt-0 focus-visible:outline-none focus-visible:ring-0">
+            <Protected 
+              permission="settings:prompt_management" 
+              fallback={
+                <Alert>
+                  <AlertDescription>
+                    You don't have permission to access AI Prompts.
+                  </AlertDescription>
+                </Alert>
+              }
+            >
+              <PromptManagement />
+            </Protected>
+          </TabsContent>
+          
+          <TabsContent value="reusable-images" className="mt-0 focus-visible:outline-none focus-visible:ring-0">
+            <Protected 
+              permission="settings:library_management" 
+              fallback={
+                <Alert>
+                  <AlertDescription>
+                    You don't have permission to access Reusable Images.
+                  </AlertDescription>
+                </Alert>
+              }
+            >
+              <ReusableImageManagement />
+            </Protected>
+          </TabsContent>
+          
           <TabsContent value="user-management" className="mt-0 focus-visible:outline-none focus-visible:ring-0">
             <Protected 
               permission="settings:user_management" 

inventory/vite.config.js

@@ -166,6 +166,12 @@ export default defineConfig(function (_a) {
                         });
                     },
                 },
+                "/uploads": {
+                    target: "https://inventory.kent.pw",
+                    changeOrigin: true,
+                    secure: false,
+                    rewrite: function (path) { return path; },
+                },
             },
         },
         build: {