Fix o3 issues on time-aggregates script

docs/calculate-issues.md

@@ -0,0 +1,185 @@
+1. **Missing Updates for Reorder Point and Safety Stock** [RESOLVED - product-metrics.js]
+- **Problem:** In the **product_metrics** table (used by the inventory health view), the fields **reorder_point** and **safety_stock** are never updated in the product metrics calculations. Although a helper function (`calculateReorderQuantities`) exists and computes these values, the update query in the `calculateProductMetrics` function does not assign any values to these columns.
+- **Effect:** The inventory health view relies on these fields (using COALESCE to default them to 0), which means that stock might never be classified as "Reorder" or "Healthy" based on the proper reorder point or safety stock calculations.
+- **Example:** Even if a product's base metrics would require a reorder (for example, if its days of inventory are low), the view always shows a value of 0 for reorder_point and safety_stock.
+- **Fix:** Update the product metrics query (or add a subsequent update) so that **pm.reorder_point** and **pm.safety_stock** are calculated (for instance, by integrating the logic from `calculateReorderQuantities`) and stored in the table.
+
+2. **Overwritten Module Exports When Combining Scripts** [RESOLVED - calculate-metrics.js]
+- **Problem:** The code provided shows two distinct exports. The main metrics calculation module exports `calculateMetrics` (along with cancel and getProgress helpers), but later in the same concatenated file the module exports are overwritten.
+- **Effect:** If these two code sections end up in a single module file, the export for the main calculation will be lost. This would break any code that calls the overall metrics calculation.
+- **Example:** An external caller expecting to run `calculateMetrics` would instead receive the `calculateProductMetrics` function.
+- **Fix:** Make sure each script resides in its own module file. Verify that the module boundaries and exports are not accidentally merged or overwritten when deployed.
+
+3. **Potential Formula Issue in EOQ Calculation (Reorder Qty)** [RESOLVED - product-metrics.js]
+- **Problem:** The helper function `calculateReorderQuantities` uses an EOQ formula with a holding cost expressed as a percentage (0.25) rather than a per‐unit cost.
+- **Effect:** If the intent was to use the traditional EOQ formula (which expects a holding cost per unit rather than a percentage), this could lead to an incorrect reorder quantity.
+- **Example:** For a given annual demand and fixed order cost, the computed reorder quantity might be higher or lower than expected.
+- **Fix:** Double-check the EOQ formula. If the intention is to compute based on a percentage, then document that clearly; otherwise, adjust the formula to use the proper holding cost value.
+
+4. **Potential Overlap or Redundancy in GMROI Calculation** [RESOLVED - time-aggregates.js]
+- **Problem:** In the time aggregates function, GMROI is calculated in two steps. The initial INSERT query computes GMROI as
+
+	`CASE WHEN s.inventory_value > 0 THEN (s.total_revenue - s.total_cost) / s.inventory_value ELSE 0 END`
+
+	and then a subsequent UPDATE query recalculates it as an annualized value using gross profit and active days.
+
+
+- **Effect:** Overwriting a computed value may be intentional to refine the metric, but if not coordinated it can cause confusion or unexpected output in the `product_time_aggregates` table.
+- **Example:** A product's GMROI might first appear as a simple ratio but then be updated to a scaled value based on the number of active days, which could lead to inconsistent reporting if not documented.
+- **Fix:** Consolidated the GMROI calculation into a single step in the initial INSERT query, properly handling annualization and NULL values.
+
+5. **Handling of Products Without Orders or Purchase Data** [RESOLVED - time-aggregates.js]
+- **Problem:** In the INSERT query of the time aggregates function, the UNION covers two cases: one for products with order data (from `monthly_sales`) and one for products that have entries in `monthly_stock` but no matching order data.
+- **Effect:** If a product has neither orders nor purchase orders, it won't get an entry in `product_time_aggregates`. Depending on business rules, this might be acceptable or might mean missing data.
+- **Example:** A product that's new or rarely ordered might not appear in the time aggregates view, potentially affecting downstream calculations.
+- **Fix:** Added an `all_products` CTE and modified the JOIN structure to ensure every product gets an entry with appropriate default values, even if it has no orders or purchase orders.
+
+6. **Redundant Recalculation of Vendor Metrics**
+- **Problem:** Similar concepts from prior scripts where cumulative metrics (like **total_revenue** and **total_cost**) are calculated in multiple query steps without necessary validation or optimization. In the vendor metrics script, calculations for total revenue and margin are performed within a `WITH` clause, which is then used in other parts of the process, making it more complex than needed.
+- **Effect:** There's unnecessary duplication in querying the same data multiple times across subqueries. It could result in decreased performance and may even lead to excess computation if the subqueries are not optimized or correctly indexed.
+- **Example:** Vendor sales and vendor purchase orders (PO) metrics are calculated in separate `WITH` clauses, leading to repeated calculations.
+- **Fix:** Synthesize the required metrics into fewer queries or reuse the results within the `WITH` clause itself. Avoid redundant calculations of **revenue** and **cost** unless truly necessary.
+
+7. **Handling Products Without Orders or Purchase Orders**
+- **Problem:** In your `calculateVendorMetrics` script, the initial insert for vendor sales doesn't fully address the products that might not have matching orders or purchase orders. If a vendor has products without any sales within the last 12 months, the results may not be fully accurate unless handled explicitly.
+- **Effect:** If no orders exist for a product associated with a particular vendor, that product will not contribute to the vendor's metrics, potentially omitting important data when calculating **total_orders** or **total_revenue**.
+- **Example:** The scripted statistics fill gaps, but products with no recent purchase or sales orders might not be counted accurately.
+- **Fix:** Include logic to handle scenarios where these products still need to be part of the vendor calculation. Use a `LEFT JOIN` wherever possible to account for cases without sales or purchase orders.
+
+8. **Redundant `ON DUPLICATE KEY UPDATE`**
+- **Problem:** Multiple queries in the `calculateVendorMetrics` script use `ON DUPLICATE KEY UPDATE` clauses to handle repeated metrics updates. This is useful for ensuring the most up-to-date calculations but can cause inconsistencies if multiple calculations happen for the same product or vendor simultaneously.
+- **Effect:** This approach can lead to an inaccurate update of brand-specific data when insertion and update overlap. Each time you add a new batch, an existing entry could be overwritten if not handled correctly.
+- **Example:** Vendor country, category, or sales-related metrics could unintentionally update during processing.
+- **Fix:** Match on current status more robustly in case of existing rows to avoid unnecessary updates. Ensure that the key used for `ON DUPLICATE KEY` aligns with any foreign key relationships that might indicate an already processed entry.
+
+9. **SQL Query Performance with Multiple Nested `WITH` Clauses**
+- **Problem:** Heavily nested queries (especially **WITH** clauses) may lead to slow performance depending on the size of the dataset.
+- **Effect:** Computational burden could be high when the database is large, e.g., querying **purchase orders**, **vendor sales**, and **product info** simultaneously. Even with proper indexes, the deployment might struggle in production environments.
+- **Example:** Multiple `WITH` clauses in the vendor and brand metrics calculation scripts might work fine in small datasets but degrade performance in production.
+- **Fix:** Combine some subqueries and reduce the layer of computations needed for calculating final metrics. Test performance on a production-sized dataset to see how nested queries are handled.
+
+10. **Missing Updates for Reorder Metrics (Vendor/Brand)**
+- **Previously Identified Issue:** Inconsistent updates for **reorder_point** and **safety_stock** across earlier scripts.
+- **Current Impact on This Script:** The vendor and brand metrics do not have explicit updates for reorder point or safety stock, which are essential for inventory evaluation.
+- **Effect:** The correct thresholds and reorder logic for vendor product inventory aren't fully accounted for in these scripts.
+- **Fix:** Integrate relevant logic to update **reorder_point** or **safety_stock** within the vendor and brand metrics calculations. Ensure that it's consistently computed and stored.
+
+11. **Data Integrity and Consistency** 
+
+**w**hen tracking sales growth or performance
+
+
+- **Problem:** Brand metrics include a sales growth clause where negative results can sometimes be skewed severely if period data varies considerably.
+- **Effect:** If period boundaries are incorrect or records are missing, this can create drastic growth rate calculations.
+- **Example:** If the "previous" period has no sales but "current" has a substantial increase, the growth rate will show as **100%**.
+- **Fix:** Implement checks that ensure both periods are valid and that the system calculates growth accurately, avoiding growth rates based solely on potential outliers. Replace consistent gaps with a no-growth rate or a meaningful zero.
+
+12. **Exclusion of Vendors With No Sales**
+
+	The vendor metrics query is driven by the `vendor_sales` CTE, which aggregates data only for vendors that have orders in the past 12 months.
+
+
+	- **Impact:** Vendors that have purchase activity (or simply exist in vendor_details) but no recent sales won't show up in vendor_metrics. This could cause the frontend to miss metrics for vendors that might still be important.
+	- **Fix:** Consider adding a UNION or changing the driving set so that all vendors (for example, from vendor_details) are included—even if they have zero sales.
+13. **Identical Formulas for On-Time Delivery and Order Fill Rates**
+
+	Both metrics are calculated as `(received_orders / total_orders) * 100`.
+
+
+	- **Impact:** If the business expects these to be distinct (for example, one might factor in on-time receipt versus mere receipt), then showing identical values on the frontend could be misleading.
+	- **Fix:** Verify and adjust the formulas if on-time delivery and order fill rates should be computed differently.
+14. **Handling Nulls and Defaults in Aggregations**
+
+	The query uses COALESCE in most places, but be sure that every aggregated value (like average lead time) correctly defaults when no data is present.
+
+
+	- **Impact:** Incorrect defaults might cause odd or missing numbers on the production interface.
+	- **Fix:** Double-check that all numeric aggregates reliably default to 0 where needed.
+
+15. **Inconsistent Stock Filtering Conditions**
+
+	In the main brand metrics query the CTE filters products with the condition
+
+	`p.stock_quantity <= 5000 AND p.stock_quantity >= 0`
+
+	whereas in the brand time-based metrics query the condition is only `p.stock_quantity <= 5000`.
+
+
+	- **Impact:** This discrepancy may lead to inconsistent numbers (for example, if any products have negative stock, which might be due to data issues) between overall brand metrics and time-based metrics on the frontend.
+	- **Fix:** Standardize the filtering criteria so that both queries treat out-of-range stock values in the same way.
+16. **Growth Rate Calculation Periods**
+
+	The growth rate is computed by comparing revenue from the last 3 months ("current") against a period from 15–12 months ago ("previous").
+
+
+	- **Impact:** This narrow window may not reflect typical year-over-year performance and could lead to volatile or unexpected growth percentages on the frontend.
+	- **Fix:** Revisit the business logic for growth—if a longer or different comparison period is preferred, adjust the date intervals accordingly.
+17. **Potential NULLs in Aggregated Time-Based Metrics**
+
+	In the brand time-based metrics query, aggregate expressions such as `SUM(o.quantity * o.price)` aren't wrapped with COALESCE.
+
+
+	- **Impact:** If there are no orders for a given brand/month, these sums might return NULL rather than 0, which could propagate into the frontend display.
+	- **Fix:** Wrap such aggregates in COALESCE (e.g. `COALESCE(SUM(o.quantity * o.price), 0)`) to ensure a default numeric value.
+
+18. **Grouping by Category Status in Base Metrics Insert**
+	- **Problem:** The INSERT for base category metrics groups by both `c.cat_id` and `c.status` even though the table's primary key is just `category_id`.
+	- **Effect:** If a category's status changes over time, the grouping may produce unexpected updates (or even multiple groups before the duplicate key update kicks in), possibly causing the wrong status or aggregated figures to be stored.
+	- **Example:** A category that toggles between "active" and "inactive" might have its metrics calculated differently on different runs.
+	- **Fix:** Ensure that the grouping keys match the primary key (or that the status update logic is exactly as intended) so that a single row per category is maintained.
+19. **Potential Null Handling in Margin Calculations**
+	- **Problem:** In the query for category time metrics, the calculation of average margin uses expressions such as `SUM(o.quantity * (o.price - GREATEST(p.cost_price, 0)))` without using `COALESCE` on `p.cost_price`.
+	- **Effect:** If any product's `cost_price` is `NULL`, then `GREATEST(p.cost_price, 0)` returns `NULL` and the resulting sum (and thus the margin) could become `NULL` rather than defaulting to 0. This might lead to missing or misleading margin figures on the frontend.
+	- **Example:** A product with a missing cost price would make the entire margin expression evaluate to `NULL` even when sales exist.
+	- **Fix:** Replace `GREATEST(p.cost_price, 0)` with `GREATEST(COALESCE(p.cost_price, 0), 0)` (or simply use `COALESCE(p.cost_price, 0)`) to ensure that missing values are handled.
+20. **Data Coverage in Growth Rate Calculation**
+	- **Problem:** The growth rate update depends on multiple CTEs (current period, previous period, and trend analysis) that require a minimum amount of data (for instance, `HAVING COUNT(*) >= 6` in the trend_stats CTE).
+	- **Effect:** Categories with insufficient historical data will fall into the "ELSE" branch (or may even be skipped if no revenue is present), which might result in a growth rate of 0.0 or an unexpected value.
+	- **Example:** A newly created category that has only two months of data won't have trend analysis, so its growth rate will be calculated solely by the simple difference, which might not reflect true performance.
+	- **Fix:** Confirm that this fallback behavior is acceptable for production; if not, adjust the logic so that every category receives a consistent growth rate even with sparse data.
+21. **Omission of Forecasts for Zero–Sales Categories**
+	- **Observation:** The category–sales metrics query uses a `HAVING AVG(cs.daily_quantity) > 0` clause.
+	- **Effect:** Categories without any average daily sales will not receive a forecast record in `category_sales_metrics`. If the frontend expects a row (even with zeros) for every category, this will lead to missing data.
+	- **Fix:** Verify that it's acceptable for categories with no sales to have no forecast entry. If not, adjust the query so that a default forecast (with zeros) is inserted.
+
+22. **Randomness in Category-Level Forecast Revenue Calculation**
+	- **Problem:** In the category-level forecasts query, the forecast revenue is multiplied by a factor of `(0.95 + (RAND() * 0.1))`.
+	- **Effect:** This introduces randomness into the forecast figures so that repeated runs could yield slightly different values. If deterministic forecasts are expected on the production frontend, this could lead to inconsistent displays.
+	- **Example:** The same category might show a 5% higher forecast on one run and 3% on another because of the random multiplier.
+	- **Fix:** Confirm that this randomness is intentional for your forecasting model; if forecasts are meant to be reproducible, remove or replace the `RAND()` factor with a fixed multiplier.
+23. **Multi-Statement Cleanup of Temporary Tables**
+	- **Problem:** The cleanup query drops multiple temporary tables in one call (separated by semicolons).
+	- **Effect:** If your Node.js MySQL driver isn't configured to allow multi-statement execution, this query may fail, leaving temporary tables behind. Leftover temporary tables might eventually cause conflicts or resource issues.
+	- **Example:** Running the cleanup query could produce an error like "multi-statement queries not enabled," preventing proper cleanup.
+	- **Fix:** Either configure your database connection to allow multi-statements or issue separate queries for each temporary table drop to ensure that the cleanup runs successfully.
+24. **Handling Products with No Sales Data**
+	- **Problem:** In the product-level forecast calculation, the CTE `daily_stats` includes a `HAVING AVG(ds.daily_quantity) > 0` clause.
+	- **Effect:** Products that have no sales (or a zero average daily quantity) will be excluded from the forecasts. This means the frontend won't show forecasts for non–selling products, which might be acceptable but could also be a completeness issue.
+	- **Example:** A product that has never sold will not appear in the `sales_forecasts` table.
+	- **Fix:** Confirm that it is intended for forecasts to be generated only for products with some sales activity. If forecasts are required for all products, adjust the query to insert default forecast records for products with zero sales.
+25. **Complexity of the Forecast Formula Involving the Seasonality Factor**
+- **Issue:**
+
+	The sales forecast calculations incorporate an adjustment factor using `COALESCE(sf.seasonality_factor, 0)` to modify forecast units and revenue. This means that if the seasonality data is missing (or not populated), the factor defaults to 0.
+
+
+- **Potential Problem:**
+
+	A default value of 0 will drastically alter the forecast calculations—often leading to a forecast of 0 or an overly dampened forecast—when in reality the intended behavior might be to use a neutral multiplier (typically 1.0). This could result in forecasts that are not reflective of the actual seasonal impact, thereby skewing the figures that reach the frontend.
+
+
+- **Fix:**
+
+	Review your data source for seasonality (the `sales_seasonality` table) and ensure it's consistently populated. Alternatively, if missing seasonality data is possible, consider using a more neutral default (such as 1.0) in your COALESCE. This change would prevent the forecast formulas from over-simplifying (or even nullifying) the forecast output due to missing seasonality factors.
+
+
+26. **Group By with Seasonality Factor Variability**
+	- **Observation:** In the forecast insertion query, the GROUP BY clause includes `sf.seasonality_factor` along with other fields.
+	- **Effect:** If the seasonality factor differs (or is `NULL` versus a value) for different forecast dates, this might result in multiple rows for the same product and forecast date. However, the `ON DUPLICATE KEY UPDATE` clause will merge them—but only if the primary key (pid, forecast_date) is truly unique.
+	- **Fix:** Verify that the grouping produces exactly one row per product per forecast date. If there's potential for multiple rows due to seasonality variability, consider applying a COALESCE or an aggregation on the seasonality factor so that it does not affect grouping.
+
+27. **Memory Management for Temporary Tables** [RESOLVED - calculate-metrics.js]
+- **Problem:** In metrics calculations, temporary tables aren't always properly cleaned up if the process fails between creation and the DROP statement.
+- **Effect:** If a process fails after creating temporary tables but before dropping them, these tables remain in memory until the connection is closed. In a production environment with multiple calculation runs, this could lead to memory leaks or table name conflicts.
+- **Example:** The `temp_revenue_ranks` table creation in ABC classification could remain if the process fails before reaching the DROP statement.
+- **Fix:** Implement proper cleanup in a finally block or use transaction management that ensures temporary tables are always cleaned up, even in failure scenarios.

inventory-server/db/metrics-schema.sql

@@ -102,19 +102,17 @@ CREATE TABLE IF NOT EXISTS product_time_aggregates (
     INDEX idx_date (year, month)
 );
 
--- Create vendor details table
-CREATE TABLE IF NOT EXISTS vendor_details (
-    vendor VARCHAR(100) NOT NULL,
+-- Create vendor_details table
+CREATE TABLE vendor_details (
+    vendor VARCHAR(100) PRIMARY KEY,
     contact_name VARCHAR(100),
-    email VARCHAR(100),
-    phone VARCHAR(20),
+    email VARCHAR(255),
+    phone VARCHAR(50),
     status VARCHAR(20) DEFAULT 'active',
-    notes TEXT,
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
-    PRIMARY KEY (vendor),
-    INDEX idx_vendor_status (status)
-);
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+    INDEX idx_status (status)
+) ENGINE=InnoDB;
 
 -- New table for vendor metrics
 CREATE TABLE IF NOT EXISTS vendor_metrics (
@@ -410,21 +408,4 @@ LEFT JOIN
     category_metrics cm ON c.cat_id = cm.category_id;
 
 -- Re-enable foreign key checks
-SET FOREIGN_KEY_CHECKS = 1;
-
--- Create table for sales seasonality factors
-CREATE TABLE IF NOT EXISTS sales_seasonality (
-    month INT NOT NULL,
-    seasonality_factor DECIMAL(5,3) DEFAULT 0,
-    last_updated TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    PRIMARY KEY (month),
-    CHECK (month BETWEEN 1 AND 12),
-    CHECK (seasonality_factor BETWEEN -1.0 AND 1.0)
-);
-
--- Insert default seasonality factors (neutral)
-INSERT INTO sales_seasonality (month, seasonality_factor)
-VALUES 
-    (1, 0), (2, 0), (3, 0), (4, 0), (5, 0), (6, 0),
-    (7, 0), (8, 0), (9, 0), (10, 0), (11, 0), (12, 0)
-ON DUPLICATE KEY UPDATE last_updated = CURRENT_TIMESTAMP; 
+SET FOREIGN_KEY_CHECKS = 1;

inventory-server/db/schema.sql

@@ -79,18 +79,6 @@ CREATE TABLE categories (
     INDEX idx_name_type (name, type)
 ) ENGINE=InnoDB;
 
--- Create vendor_details table
-CREATE TABLE vendor_details (
-    vendor VARCHAR(100) PRIMARY KEY,
-    contact_name VARCHAR(100),
-    email VARCHAR(255),
-    phone VARCHAR(50),
-    status VARCHAR(20) DEFAULT 'active',
-    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
-    INDEX idx_status (status)
-) ENGINE=InnoDB;
-
 -- Create product_categories junction table
 CREATE TABLE product_categories (
     cat_id BIGINT NOT NULL,

inventory-server/scripts/calculate-metrics.js

@@ -44,6 +44,34 @@ global.clearProgress = progress.clearProgress;
 global.getProgress = progress.getProgress;
 global.logError = progress.logError;
 
+// List of temporary tables used in the calculation process
+const TEMP_TABLES = [
+    'temp_revenue_ranks',
+    'temp_sales_metrics',
+    'temp_purchase_metrics',
+    'temp_product_metrics',
+    'temp_vendor_metrics',
+    'temp_category_metrics',
+    'temp_brand_metrics',
+    'temp_forecast_dates',
+    'temp_daily_sales',
+    'temp_product_stats',
+    'temp_category_sales',
+    'temp_category_stats'
+];
+
+// Add cleanup function for temporary tables
+async function cleanupTemporaryTables(connection) {
+    try {
+        for (const table of TEMP_TABLES) {
+            await connection.query(`DROP TEMPORARY TABLE IF EXISTS ${table}`);
+        }
+    } catch (error) {
+        logError(error, 'Error cleaning up temporary tables');
+        throw error; // Re-throw to be handled by the caller
+    }
+}
+
 const { getConnection, closePool } = require('./metrics/utils/db');
 const calculateProductMetrics = require('./metrics/product-metrics');
 const calculateTimeAggregates = require('./metrics/time-aggregates');
@@ -650,19 +678,26 @@ async function calculateMetrics() {
             throw error;
         } finally {
             if (connection) {
+                // Ensure temporary tables are cleaned up
+                await cleanupTemporaryTables(connection);
                 connection.release();
             }
+            // Close the connection pool when we're done
+            await closePool();
         }
-    } finally {
-        // Close the connection pool when we're done
-        await closePool();
+    } catch (error) {
+        success = false;
+        logError(error, 'Error in metrics calculation');
+        throw error;
     }
 }
 
-// Export both functions and progress checker
-module.exports = calculateMetrics;
-module.exports.cancelCalculation = cancelCalculation;
-module.exports.getProgress = global.getProgress;
+// Export as a module with all necessary functions
+module.exports = {
+    calculateMetrics,
+    cancelCalculation,
+    getProgress: global.getProgress
+};
 
 // Run directly if called from command line
 if (require.main === module) {

inventory-server/scripts/full-reset.js

@@ -0,0 +1,107 @@
+const path = require('path');
+const { spawn } = require('child_process');
+
+function outputProgress(data) {
+    if (!data.status) {
+        data = {
+            status: 'running',
+            ...data
+        };
+    }
+    console.log(JSON.stringify(data));
+}
+
+function runScript(scriptPath) {
+    return new Promise((resolve, reject) => {
+        const child = spawn('node', [scriptPath], {
+            stdio: ['inherit', 'pipe', 'pipe']
+        });
+
+        let output = '';
+
+        child.stdout.on('data', (data) => {
+            const lines = data.toString().split('\n');
+            lines.filter(line => line.trim()).forEach(line => {
+                try {
+                    console.log(line); // Pass through the JSON output
+                    output += line + '\n';
+                } catch (e) {
+                    console.log(line); // If not JSON, just log it directly
+                }
+            });
+        });
+
+        child.stderr.on('data', (data) => {
+            console.error(data.toString());
+        });
+
+        child.on('close', (code) => {
+            if (code !== 0) {
+                reject(new Error(`Script ${scriptPath} exited with code ${code}`));
+            } else {
+                resolve(output);
+            }
+        });
+
+        child.on('error', (err) => {
+            reject(err);
+        });
+    });
+}
+
+async function fullReset() {
+    try {
+        // Step 1: Reset Database
+        outputProgress({
+            operation: 'Starting full reset',
+            message: 'Step 1/3: Resetting database...'
+        });
+        await runScript(path.join(__dirname, 'reset-db.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Database reset step complete',
+            message: 'Database reset finished, moving to import...'
+        });
+
+        // Step 2: Import from Production
+        outputProgress({
+            operation: 'Starting import',
+            message: 'Step 2/3: Importing from production...'
+        });
+        await runScript(path.join(__dirname, 'import-from-prod.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Import step complete',
+            message: 'Import finished, moving to metrics calculation...'
+        });
+
+        // Step 3: Calculate Metrics
+        outputProgress({
+            operation: 'Starting metrics calculation',
+            message: 'Step 3/3: Calculating metrics...'
+        });
+        await runScript(path.join(__dirname, 'calculate-metrics.js'));
+        
+        // Final completion message
+        outputProgress({
+            status: 'complete',
+            operation: 'Full reset complete',
+            message: 'Successfully completed all steps: database reset, import, and metrics calculation'
+        });
+    } catch (error) {
+        outputProgress({
+            status: 'error',
+            operation: 'Full reset failed',
+            error: error.message,
+            stack: error.stack
+        });
+        process.exit(1);
+    }
+}
+
+// Run if called directly
+if (require.main === module) {
+    fullReset();
+}
+
+module.exports = fullReset; 

inventory-server/scripts/full-update.js

@@ -0,0 +1,100 @@
+const path = require('path');
+const { spawn } = require('child_process');
+
+function outputProgress(data) {
+    if (!data.status) {
+        data = {
+            status: 'running',
+            ...data
+        };
+    }
+    console.log(JSON.stringify(data));
+}
+
+function runScript(scriptPath) {
+    return new Promise((resolve, reject) => {
+        const child = spawn('node', [scriptPath], {
+            stdio: ['inherit', 'pipe', 'pipe']
+        });
+
+        let output = '';
+
+        child.stdout.on('data', (data) => {
+            const lines = data.toString().split('\n');
+            lines.filter(line => line.trim()).forEach(line => {
+                try {
+                    console.log(line); // Pass through the JSON output
+                    output += line + '\n';
+                } catch (e) {
+                    console.log(line); // If not JSON, just log it directly
+                }
+            });
+        });
+
+        child.stderr.on('data', (data) => {
+            console.error(data.toString());
+        });
+
+        child.on('close', (code) => {
+            if (code !== 0) {
+                reject(new Error(`Script ${scriptPath} exited with code ${code}`));
+            } else {
+                resolve(output);
+            }
+        });
+
+        child.on('error', (err) => {
+            reject(err);
+        });
+    });
+}
+
+async function fullUpdate() {
+    try {
+        // Step 1: Import from Production
+        outputProgress({
+            operation: 'Starting full update',
+            message: 'Step 1/2: Importing from production...'
+        });
+        await runScript(path.join(__dirname, 'import-from-prod.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Import step complete',
+            message: 'Import finished, moving to metrics calculation...'
+        });
+
+        // Step 2: Calculate Metrics
+        outputProgress({
+            operation: 'Starting metrics calculation',
+            message: 'Step 2/2: Calculating metrics...'
+        });
+        await runScript(path.join(__dirname, 'calculate-metrics.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Metrics step complete',
+            message: 'Metrics calculation finished'
+        });
+
+        // Final completion message
+        outputProgress({
+            status: 'complete',
+            operation: 'Full update complete',
+            message: 'Successfully completed all steps: import and metrics calculation'
+        });
+    } catch (error) {
+        outputProgress({
+            status: 'error',
+            operation: 'Full update failed',
+            error: error.message,
+            stack: error.stack
+        });
+        process.exit(1);
+    }
+}
+
+// Run if called directly
+if (require.main === module) {
+    fullUpdate();
+}
+
+module.exports = fullUpdate; 

inventory-server/scripts/metrics/product-metrics.js

@@ -153,7 +153,7 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                     LEFT JOIN temp_sales_metrics sm ON pm.pid = sm.pid
                     LEFT JOIN temp_purchase_metrics lm ON pm.pid = lm.pid
                     SET 
-                        pm.inventory_value = p.stock_quantity * p.cost_price,
+                        pm.inventory_value = p.stock_quantity * NULLIF(p.cost_price, 0),
                         pm.daily_sales_avg = COALESCE(sm.daily_sales_avg, 0),
                         pm.weekly_sales_avg = COALESCE(sm.weekly_sales_avg, 0),
                         pm.monthly_sales_avg = COALESCE(sm.monthly_sales_avg, 0),
@@ -164,12 +164,12 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                         pm.avg_lead_time_days = COALESCE(lm.avg_lead_time_days, 30),
                         pm.days_of_inventory = CASE 
                             WHEN COALESCE(sm.daily_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / sm.daily_sales_avg)
+                            THEN FLOOR(p.stock_quantity / NULLIF(sm.daily_sales_avg, 0))
                             ELSE NULL 
                         END,
                         pm.weeks_of_inventory = CASE 
                             WHEN COALESCE(sm.weekly_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / sm.weekly_sales_avg)
+                            THEN FLOOR(p.stock_quantity / NULLIF(sm.weekly_sales_avg, 0))
                             ELSE NULL 
                         END,
                         pm.stock_status = CASE
@@ -181,10 +181,21 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                             WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) > ? THEN 'Overstocked'
                             ELSE 'Healthy'
                         END,
-                        pm.reorder_qty = CASE
+                        pm.safety_stock = CASE
                             WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
+                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
+                            ELSE ?
+                        END,
+                        pm.reorder_point = CASE
+                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
+                                CEIL(sm.daily_sales_avg * COALESCE(lm.avg_lead_time_days, 30)) + 
+                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
+                            ELSE ?
+                        END,
+                        pm.reorder_qty = CASE
+                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 AND NULLIF(p.cost_price, 0) IS NOT NULL THEN
                                 GREATEST(
-                                    CEIL(sm.daily_sales_avg * COALESCE(lm.avg_lead_time_days, 30) * 1.96),
+                                    CEIL(SQRT((2 * (sm.daily_sales_avg * 365) * 25) / (NULLIF(p.cost_price, 0) * 0.25))),
                                     ?
                                 )
                             ELSE ?
@@ -195,18 +206,22 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                             ELSE 0
                         END,
                         pm.last_calculated_at = NOW()
-                    WHERE p.pid IN (?)
-                `, [
+                    WHERE p.pid IN (${batch.map(() => '?').join(',')})
+                `,
+                [
                     defaultThresholds.low_stock_threshold,
                     defaultThresholds.critical_days,
                     defaultThresholds.reorder_days,
                     defaultThresholds.overstock_days,
                     defaultThresholds.low_stock_threshold,
                     defaultThresholds.low_stock_threshold,
+                    defaultThresholds.low_stock_threshold,
+                    defaultThresholds.low_stock_threshold,
                     defaultThresholds.overstock_days,
                     defaultThresholds.overstock_days,
-                    batch.map(row => row.pid)
-                ]);
+                    ...batch.map(row => row.pid)
+                ]
+                );
 
                 lastPid = batch[batch.length - 1].pid;
                 processedCount += batch.length;
@@ -603,9 +618,9 @@ function calculateReorderQuantities(stock, stock_status, daily_sales_avg, avg_le
     if (daily_sales_avg > 0) {
         const annual_demand = daily_sales_avg * 365;
         const order_cost = 25; // Fixed cost per order
-        const holding_cost_percent = 0.25; // 25% annual holding cost
+        const holding_cost = config.cost_price * 0.25; // 25% of unit cost as annual holding cost
         
-        reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost_percent));
+        reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost));
     } else {
         // If no sales data, use a basic calculation
         reorder_qty = Math.max(safety_stock, config.low_stock_threshold);

inventory-server/scripts/metrics/time-aggregates.js

@@ -104,51 +104,99 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                     SUM(ordered) as stock_ordered
                 FROM purchase_orders
                 GROUP BY pid, YEAR(date), MONTH(date)
+            ),
+            base_products AS (
+                SELECT 
+                    p.pid,
+                    p.cost_price * p.stock_quantity as inventory_value
+                FROM products p
             )
             SELECT 
-                s.pid,
-                s.year,
-                s.month,
-                s.total_quantity_sold,
-                s.total_revenue,
-                s.total_cost,
-                s.order_count,
+                COALESCE(s.pid, ms.pid) as pid,
+                COALESCE(s.year, ms.year) as year,
+                COALESCE(s.month, ms.month) as month,
+                COALESCE(s.total_quantity_sold, 0) as total_quantity_sold,
+                COALESCE(s.total_revenue, 0) as total_revenue,
+                COALESCE(s.total_cost, 0) as total_cost,
+                COALESCE(s.order_count, 0) as order_count,
                 COALESCE(ms.stock_received, 0) as stock_received,
                 COALESCE(ms.stock_ordered, 0) as stock_ordered,
-                s.avg_price,
-                s.profit_margin,
-                s.inventory_value,
+                COALESCE(s.avg_price, 0) as avg_price,
+                COALESCE(s.profit_margin, 0) as profit_margin,
+                COALESCE(s.inventory_value, bp.inventory_value, 0) as inventory_value,
                 CASE 
-                    WHEN s.inventory_value > 0 THEN 
-                        (s.total_revenue - s.total_cost) / s.inventory_value
+                    WHEN COALESCE(s.inventory_value, bp.inventory_value, 0) > 0 
+                        AND COALESCE(s.active_days, 0) > 0 
+                    THEN (COALESCE(s.total_revenue - s.total_cost, 0) * (365.0 / s.active_days)) 
+                        / COALESCE(s.inventory_value, bp.inventory_value)
                     ELSE 0 
                 END as gmroi
-            FROM monthly_sales s
+            FROM (
+                SELECT * FROM monthly_sales s
+                UNION ALL
+                SELECT 
+                    ms.pid,
+                    ms.year,
+                    ms.month,
+                    0 as total_quantity_sold,
+                    0 as total_revenue,
+                    0 as total_cost,
+                    0 as order_count,
+                    NULL as avg_price,
+                    0 as profit_margin,
+                    NULL as inventory_value,
+                    0 as active_days
+                FROM monthly_stock ms
+                WHERE NOT EXISTS (
+                    SELECT 1 FROM monthly_sales s2
+                    WHERE s2.pid = ms.pid
+                    AND s2.year = ms.year
+                    AND s2.month = ms.month
+                )
+            ) s
             LEFT JOIN monthly_stock ms 
                 ON s.pid = ms.pid 
                 AND s.year = ms.year 
                 AND s.month = ms.month
+            JOIN base_products bp ON COALESCE(s.pid, ms.pid) = bp.pid
             UNION
             SELECT 
-                p.pid,
-                p.year,
-                p.month,
+                ms.pid,
+                ms.year,
+                ms.month,
                 0 as total_quantity_sold,
                 0 as total_revenue,
                 0 as total_cost,
                 0 as order_count,
-                p.stock_received,
-                p.stock_ordered,
+                ms.stock_received,
+                ms.stock_ordered,
                 0 as avg_price,
                 0 as profit_margin,
-                (SELECT cost_price * stock_quantity FROM products WHERE pid = p.pid) as inventory_value,
+                bp.inventory_value,
                 0 as gmroi
-            FROM monthly_stock p
-            LEFT JOIN monthly_sales s
-                ON p.pid = s.pid 
-                AND p.year = s.year 
-                AND p.month = s.month
-            WHERE s.pid IS NULL
+            FROM monthly_stock ms
+            JOIN base_products bp ON ms.pid = bp.pid
+            WHERE NOT EXISTS (
+                SELECT 1 FROM (
+                    SELECT * FROM monthly_sales
+                    UNION ALL
+                    SELECT 
+                        ms2.pid,
+                        ms2.year,
+                        ms2.month,
+                        0, 0, 0, 0, NULL, 0, NULL, 0
+                    FROM monthly_stock ms2
+                    WHERE NOT EXISTS (
+                        SELECT 1 FROM monthly_sales s2
+                        WHERE s2.pid = ms2.pid
+                        AND s2.year = ms2.year
+                        AND s2.month = ms2.month
+                    )
+                ) s
+                WHERE s.pid = ms.pid
+                AND s.year = ms.year
+                AND s.month = ms.month
+            )
             ON DUPLICATE KEY UPDATE
                 total_quantity_sold = VALUES(total_quantity_sold),
                 total_revenue = VALUES(total_revenue),
@@ -196,7 +244,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                     MONTH(o.date) as month,
                     p.cost_price * p.stock_quantity as inventory_value,
                     SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
-                    COUNT(DISTINCT DATE(o.date)) as days_in_period
+                    COUNT(DISTINCT DATE(o.date)) as active_days
                 FROM products p
                 LEFT JOIN orders o ON p.pid = o.pid
                 WHERE o.canceled = false
@@ -205,12 +253,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                 AND pta.year = fin.year 
                 AND pta.month = fin.month
             SET 
-                pta.inventory_value = COALESCE(fin.inventory_value, 0),
-                pta.gmroi = CASE 
-                    WHEN COALESCE(fin.inventory_value, 0) > 0 AND fin.days_in_period > 0 THEN 
-                        (COALESCE(fin.gross_profit, 0) * (365.0 / fin.days_in_period)) / COALESCE(fin.inventory_value, 0)
-                    ELSE 0 
-                END
+                pta.inventory_value = COALESCE(fin.inventory_value, 0)
         `);
 
         processedCount = Math.floor(totalProducts * 0.65);

inventory-server/scripts/reset-db.js

@@ -156,7 +156,7 @@ async function resetDatabase() {
             SELECT GROUP_CONCAT(table_name) as tables 
             FROM information_schema.tables 
             WHERE table_schema = DATABASE()
-            AND table_name NOT IN ('users', 'import_history')
+            AND table_name NOT IN ('users', 'import_history', 'calculate_history')
         `);
 
         if (!tables[0].tables) {
@@ -175,7 +175,7 @@ async function resetDatabase() {
                 DROP TABLE IF EXISTS 
                 ${tables[0].tables
                     .split(',')
-                    .filter(table => table !== 'users')
+                    .filter(table => !['users', 'calculate_history'].includes(table))
                     .map(table => '`' + table + '`')
                     .join(', ')}
             `;
@@ -543,5 +543,15 @@ async function resetDatabase() {
     }
 }
 
-// Run the reset
-resetDatabase(); 
+// Export if required as a module
+if (typeof module !== 'undefined' && module.exports) {
+    module.exports = resetDatabase;
+}
+
+// Run if called directly
+if (require.main === module) {
+    resetDatabase().catch(error => {
+        console.error('Error:', error);
+        process.exit(1);
+    });
+} 

inventory-server/src/routes/csv.js

@@ -2,6 +2,7 @@ const express = require('express');
 const router = express.Router();
 const { spawn } = require('child_process');
 const path = require('path');
+const db = require('../utils/db');
 
 // Debug middleware MUST be first
 router.use((req, res, next) => {
@@ -9,9 +10,11 @@ router.use((req, res, next) => {
     next();
 });
 
-// Store active import process and its progress
+// Store active processes and their progress
 let activeImport = null;
 let importProgress = null;
+let activeFullUpdate = null;
+let activeFullReset = null;
 
 // SSE clients for progress updates
 const updateClients = new Set();
@@ -19,17 +22,16 @@ const importClients = new Set();
 const resetClients = new Set();
 const resetMetricsClients = new Set();
 const calculateMetricsClients = new Set();
+const fullUpdateClients = new Set();
+const fullResetClients = new Set();
 
 // Helper to send progress to specific clients
-function sendProgressToClients(clients, progress) {
-    const data = typeof progress === 'string' ? { progress } : progress;
-    
-    // Ensure we have a status field
-    if (!data.status) {
-        data.status = 'running';
-    }
-    
-    const message = `data: ${JSON.stringify(data)}\n\n`;
+function sendProgressToClients(clients, data) {
+    // If data is a string, send it directly
+    // If it's an object, convert it to JSON
+    const message = typeof data === 'string' 
+        ? `data: ${data}\n\n`
+        : `data: ${JSON.stringify(data)}\n\n`;
     
     clients.forEach(client => {
         try {
@@ -45,115 +47,149 @@ function sendProgressToClients(clients, progress) {
     });
 }
 
+// Helper to run a script and stream progress
+function runScript(scriptPath, type, clients) {
+    return new Promise((resolve, reject) => {
+        // Kill any existing process of this type
+        let activeProcess;
+        switch (type) {
+            case 'update':
+                if (activeFullUpdate) {
+                    try { activeFullUpdate.kill(); } catch (e) { }
+                }
+                activeProcess = activeFullUpdate;
+                break;
+            case 'reset':
+                if (activeFullReset) {
+                    try { activeFullReset.kill(); } catch (e) { }
+                }
+                activeProcess = activeFullReset;
+                break;
+        }
+
+        const child = spawn('node', [scriptPath], {
+            stdio: ['inherit', 'pipe', 'pipe']
+        });
+
+        switch (type) {
+            case 'update':
+                activeFullUpdate = child;
+                break;
+            case 'reset':
+                activeFullReset = child;
+                break;
+        }
+
+        let output = '';
+
+        child.stdout.on('data', (data) => {
+            const text = data.toString();
+            output += text;
+            
+            // Split by lines to handle multiple JSON outputs
+            const lines = text.split('\n');
+            lines.filter(line => line.trim()).forEach(line => {
+                try {
+                    // Try to parse as JSON but don't let it affect the display
+                    const jsonData = JSON.parse(line);
+                    // Only end the process if we get a final status
+                    if (jsonData.status === 'complete' || jsonData.status === 'error' || jsonData.status === 'cancelled') {
+                        if (jsonData.status === 'complete' && !jsonData.operation?.includes('complete')) {
+                            // Don't close for intermediate completion messages
+                            sendProgressToClients(clients, line);
+                            return;
+                        }
+                        // Close only on final completion/error/cancellation
+                        switch (type) {
+                            case 'update':
+                                activeFullUpdate = null;
+                                break;
+                            case 'reset':
+                                activeFullReset = null;
+                                break;
+                        }
+                        if (jsonData.status === 'error') {
+                            reject(new Error(jsonData.error || 'Unknown error'));
+                        } else {
+                            resolve({ output });
+                        }
+                    }
+                } catch (e) {
+                    // Not JSON, just display as is
+                }
+                // Always send the raw line
+                sendProgressToClients(clients, line);
+            });
+        });
+
+        child.stderr.on('data', (data) => {
+            const text = data.toString();
+            console.error(text);
+            // Send stderr output directly too
+            sendProgressToClients(clients, text);
+        });
+
+        child.on('close', (code) => {
+            switch (type) {
+                case 'update':
+                    activeFullUpdate = null;
+                    break;
+                case 'reset':
+                    activeFullReset = null;
+                    break;
+            }
+
+            if (code !== 0) {
+                const error = `Script ${scriptPath} exited with code ${code}`;
+                sendProgressToClients(clients, error);
+                reject(new Error(error));
+            }
+            // Don't resolve here - let the completion message from the script trigger the resolve
+        });
+
+        child.on('error', (err) => {
+            switch (type) {
+                case 'update':
+                    activeFullUpdate = null;
+                    break;
+                case 'reset':
+                    activeFullReset = null;
+                    break;
+            }
+            sendProgressToClients(clients, err.message);
+            reject(err);
+        });
+    });
+}
+
 // Progress endpoints
-router.get('/update/progress', (req, res) => {
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Send an initial message to test the connection
-    res.write('data: {"status":"running","operation":"Initializing connection..."}\n\n');
-
-    // Add this client to the update set
-    updateClients.add(res);
-
-    // Remove client when connection closes
-    req.on('close', () => {
-        updateClients.delete(res);
-    });
-});
-
-router.get('/import/progress', (req, res) => {
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Send an initial message to test the connection
-    res.write('data: {"status":"running","operation":"Initializing connection..."}\n\n');
-
-    // Add this client to the import set
-    importClients.add(res);
-
-    // Remove client when connection closes
-    req.on('close', () => {
-        importClients.delete(res);
-    });
-});
-
-router.get('/reset/progress', (req, res) => {
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Send an initial message to test the connection
-    res.write('data: {"status":"running","operation":"Initializing connection..."}\n\n');
-
-    // Add this client to the reset set
-    resetClients.add(res);
-
-    // Remove client when connection closes
-    req.on('close', () => {
-        resetClients.delete(res);
-    });
-});
-
-// Add reset-metrics progress endpoint
-router.get('/reset-metrics/progress', (req, res) => {
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Send an initial message to test the connection
-    res.write('data: {"status":"running","operation":"Initializing connection..."}\n\n');
-
-    // Add this client to the reset-metrics set
-    resetMetricsClients.add(res);
-
-    // Remove client when connection closes
-    req.on('close', () => {
-        resetMetricsClients.delete(res);
-    });
-});
-
-// Add calculate-metrics progress endpoint
-router.get('/calculate-metrics/progress', (req, res) => {
-    res.writeHead(200, {
-        'Content-Type': 'text/event-stream',
-        'Cache-Control': 'no-cache',
-        'Connection': 'keep-alive',
-        'Access-Control-Allow-Origin': req.headers.origin || '*',
-        'Access-Control-Allow-Credentials': 'true'
-    });
-
-    // Send current progress if it exists
-    if (importProgress) {
-        res.write(`data: ${JSON.stringify(importProgress)}\n\n`);
-    } else {
-        res.write('data: {"status":"running","operation":"Initializing connection..."}\n\n');
+router.get('/:type/progress', (req, res) => {
+    const { type } = req.params;
+    if (!['update', 'reset'].includes(type)) {
+        return res.status(400).json({ error: 'Invalid operation type' });
     }
 
-    // Add this client to the calculate-metrics set
-    calculateMetricsClients.add(res);
+    res.writeHead(200, {
+        'Content-Type': 'text/event-stream',
+        'Cache-Control': 'no-cache',
+        'Connection': 'keep-alive',
+        'Access-Control-Allow-Origin': req.headers.origin || '*',
+        'Access-Control-Allow-Credentials': 'true'
+    });
 
-    // Remove client when connection closes
+    // Add this client to the correct set
+    const clients = type === 'update' ? fullUpdateClients : fullResetClients;
+    clients.add(res);
+
+    // Send initial connection message
+    sendProgressToClients(new Set([res]), JSON.stringify({
+        status: 'running',
+        operation: 'Initializing connection...'
+    }));
+
+    // Handle client disconnect
     req.on('close', () => {
-        calculateMetricsClients.delete(res);
+        clients.delete(res);
     });
 });
 
@@ -174,7 +210,6 @@ router.get('/status', (req, res) => {
 
 // Add calculate-metrics status endpoint
 router.get('/calculate-metrics/status', (req, res) => {
-    console.log('Calculate metrics status endpoint hit');
     const calculateMetrics = require('../../scripts/calculate-metrics');
     const progress = calculateMetrics.getProgress();
     
@@ -371,49 +406,35 @@ router.post('/import', async (req, res) => {
 
 // Route to cancel active process
 router.post('/cancel', (req, res) => {
-    if (!activeImport) {
-        return res.status(404).json({ error: 'No active process to cancel' });
+    let killed = false;
+
+    // Get the operation type from the request
+    const { type } = req.query;
+    const clients = type === 'update' ? fullUpdateClients : fullResetClients;
+    const activeProcess = type === 'update' ? activeFullUpdate : activeFullReset;
+
+    if (activeProcess) {
+        try {
+            activeProcess.kill('SIGTERM');
+            if (type === 'update') {
+                activeFullUpdate = null;
+            } else {
+                activeFullReset = null;
+            }
+            killed = true;
+            sendProgressToClients(clients, JSON.stringify({
+                status: 'cancelled',
+                operation: 'Operation cancelled'
+            }));
+        } catch (err) {
+            console.error(`Error killing ${type} process:`, err);
+        }
     }
 
-    try {
-        // If it's the prod import module, call its cancel function
-        if (typeof activeImport.cancelImport === 'function') {
-            activeImport.cancelImport();
-        } else {
-            // Otherwise it's a child process
-            activeImport.kill('SIGTERM');
-        }
-        
-        // Get the operation type from the request
-        const { operation } = req.query;
-        
-        // Send cancel message only to the appropriate client set
-        const cancelMessage = { 
-            status: 'cancelled',
-            operation: 'Operation cancelled'
-        };
-
-        switch (operation) {
-            case 'update':
-                sendProgressToClients(updateClients, cancelMessage);
-                break;
-            case 'import':
-                sendProgressToClients(importClients, cancelMessage);
-                break;
-            case 'reset':
-                sendProgressToClients(resetClients, cancelMessage);
-                break;
-            case 'calculate-metrics':
-                sendProgressToClients(calculateMetricsClients, cancelMessage);
-                break;
-        }
-        
+    if (killed) {
         res.json({ success: true });
-    } catch (error) {
-        // Even if there's an error, try to clean up
-        activeImport = null;
-        importProgress = null;
-        res.status(500).json({ error: 'Failed to cancel process' });
+    } else {
+        res.status(404).json({ error: 'No active process to cancel' });
     }
 });
 
@@ -552,20 +573,6 @@ router.post('/reset-metrics', async (req, res) => {
     }
 });
 
-// Add calculate-metrics status endpoint
-router.get('/calculate-metrics/status', (req, res) => {
-    const calculateMetrics = require('../../scripts/calculate-metrics');
-    const progress = calculateMetrics.getProgress();
-    
-    // Only consider it active if both the process is running and we have progress
-    const isActive = !!activeImport && !!progress;
-    
-    res.json({
-        active: isActive,
-        progress: isActive ? progress : null
-    });
-});
-
 // Add calculate-metrics endpoint
 router.post('/calculate-metrics', async (req, res) => {
     if (activeImport) {
@@ -711,4 +718,96 @@ router.post('/import-from-prod', async (req, res) => {
     }
 });
 
+// POST /csv/full-update - Run full update script
+router.post('/full-update', async (req, res) => {
+    try {
+        const scriptPath = path.join(__dirname, '../../scripts/full-update.js');
+        runScript(scriptPath, 'update', fullUpdateClients)
+            .catch(error => {
+                console.error('Update failed:', error);
+            });
+        res.status(202).json({ message: 'Update started' });
+    } catch (error) {
+        res.status(500).json({ error: error.message });
+    }
+});
+
+// POST /csv/full-reset - Run full reset script
+router.post('/full-reset', async (req, res) => {
+    try {
+        const scriptPath = path.join(__dirname, '../../scripts/full-reset.js');
+        runScript(scriptPath, 'reset', fullResetClients)
+            .catch(error => {
+                console.error('Reset failed:', error);
+            });
+        res.status(202).json({ message: 'Reset started' });
+    } catch (error) {
+        res.status(500).json({ error: error.message });
+    }
+});
+
+// GET /history/import - Get recent import history
+router.get('/history/import', async (req, res) => {
+    try {
+        const pool = req.app.locals.pool;
+        const [rows] = await pool.query(`
+            SELECT * FROM import_history 
+            ORDER BY start_time DESC 
+            LIMIT 20
+        `);
+        res.json(rows || []);
+    } catch (error) {
+        console.error('Error fetching import history:', error);
+        res.status(500).json({ error: error.message });
+    }
+});
+
+// GET /history/calculate - Get recent calculation history
+router.get('/history/calculate', async (req, res) => {
+    try {
+        const pool = req.app.locals.pool;
+        const [rows] = await pool.query(`
+            SELECT * FROM calculate_history 
+            ORDER BY start_time DESC 
+            LIMIT 20
+        `);
+        res.json(rows || []);
+    } catch (error) {
+        console.error('Error fetching calculate history:', error);
+        res.status(500).json({ error: error.message });
+    }
+});
+
+// GET /status/modules - Get module calculation status
+router.get('/status/modules', async (req, res) => {
+    try {
+        const pool = req.app.locals.pool;
+        const [rows] = await pool.query(`
+            SELECT module_name, last_calculation_timestamp 
+            FROM calculate_status 
+            ORDER BY module_name
+        `);
+        res.json(rows || []);
+    } catch (error) {
+        console.error('Error fetching module status:', error);
+        res.status(500).json({ error: error.message });
+    }
+});
+
+// GET /status/tables - Get table sync status
+router.get('/status/tables', async (req, res) => {
+    try {
+        const pool = req.app.locals.pool;
+        const [rows] = await pool.query(`
+            SELECT table_name, last_sync_timestamp 
+            FROM sync_status 
+            ORDER BY table_name
+        `);
+        res.json(rows || []);
+    } catch (error) {
+        console.error('Error fetching table status:', error);
+        res.status(500).json({ error: error.message });
+    }
+});
+
 module.exports = router; 

inventory/src/components/settings/PerformanceMetrics.tsx

@@ -133,6 +133,10 @@ export function PerformanceMetrics() {
     }
   };
 
+  function getCategoryName(_cat_id: number): import("react").ReactNode {
+    throw new Error('Function not implemented.');
+  }
+
   return (
     <div className="max-w-[700px] space-y-4">
       {/* Lead Time Thresholds Card */}
@@ -205,11 +209,11 @@ export function PerformanceMetrics() {
             <Table>
               <TableHeader>
                 <TableRow>
-                  <TableHead>Category</TableHead>
-                  <TableHead>Vendor</TableHead>
-                  <TableHead className="text-right">A Threshold</TableHead>
-                  <TableHead className="text-right">B Threshold</TableHead>
-                  <TableHead className="text-right">Period Days</TableHead>
+                  <TableCell>Category</TableCell>
+                  <TableCell>Vendor</TableCell>
+                  <TableCell className="text-right">A Threshold</TableCell>
+                  <TableCell className="text-right">B Threshold</TableCell>
+                  <TableCell className="text-right">Period Days</TableCell>
                 </TableRow>
               </TableHeader>
               <TableBody>
@@ -242,10 +246,10 @@ export function PerformanceMetrics() {
             <Table>
               <TableHeader>
                 <TableRow>
-                  <TableHead>Category</TableHead>
-                  <TableHead>Vendor</TableHead>
-                  <TableHead className="text-right">Period Days</TableHead>
-                  <TableHead className="text-right">Target Rate</TableHead>
+                  <TableCell>Category</TableCell>
+                  <TableCell>Vendor</TableCell>
+                  <TableCell className="text-right">Period Days</TableCell>
+                  <TableCell className="text-right">Target Rate</TableCell>
                 </TableRow>
               </TableHeader>
               <TableBody>

inventory/src/components/settings/StockManagement.tsx

@@ -5,7 +5,6 @@ import { Input } from "@/components/ui/input";
 import { Label } from "@/components/ui/label";
 import { toast } from "sonner";
 import config from '../../config';
-import { Table, TableBody, TableCell, TableHeader, TableRow } from "@/components/ui/table";
 
 interface StockThreshold {
   id: number;
@@ -244,54 +243,6 @@ export function StockManagement() {
           </div>
         </CardContent>
       </Card>
-
-      <Table>
-        <TableHeader>
-          <TableRow>
-            <TableHead>Category</TableHead>
-            <TableHead>Vendor</TableHead>
-            <TableHead className="text-right">Critical Days</TableHead>
-            <TableHead className="text-right">Reorder Days</TableHead>
-            <TableHead className="text-right">Overstock Days</TableHead>
-            <TableHead className="text-right">Low Stock</TableHead>
-            <TableHead className="text-right">Min Reorder</TableHead>
-          </TableRow>
-        </TableHeader>
-        <TableBody>
-          {stockThresholds.map((threshold) => (
-            <TableRow key={`${threshold.cat_id}-${threshold.vendor}`}>
-              <TableCell>{threshold.cat_id ? getCategoryName(threshold.cat_id) : 'Global'}</TableCell>
-              <TableCell>{threshold.vendor || 'All Vendors'}</TableCell>
-              <TableCell className="text-right">{threshold.critical_days}</TableCell>
-              <TableCell className="text-right">{threshold.reorder_days}</TableCell>
-              <TableCell className="text-right">{threshold.overstock_days}</TableCell>
-              <TableCell className="text-right">{threshold.low_stock_threshold}</TableCell>
-              <TableCell className="text-right">{threshold.min_reorder_quantity}</TableCell>
-            </TableRow>
-          ))}
-        </TableBody>
-      </Table>
-
-      <Table>
-        <TableHeader>
-          <TableRow>
-            <TableHead>Category</TableHead>
-            <TableHead>Vendor</TableHead>
-            <TableHead className="text-right">Coverage Days</TableHead>
-            <TableHead className="text-right">Service Level</TableHead>
-          </TableRow>
-        </TableHeader>
-        <TableBody>
-          {safetyStockConfigs.map((config) => (
-            <TableRow key={`${config.cat_id}-${config.vendor}`}>
-              <TableCell>{config.cat_id ? getCategoryName(config.cat_id) : 'Global'}</TableCell>
-              <TableCell>{config.vendor || 'All Vendors'}</TableCell>
-              <TableCell className="text-right">{config.coverage_days}</TableCell>
-              <TableCell className="text-right">{config.service_level}%</TableCell>
-            </TableRow>
-          ))}
-        </TableBody>
-      </Table>
     </div>
   );
 } 

mountremote.command

@@ -1,7 +1,7 @@
 #!/bin/zsh
 
 #Clear previous mount in case it’s still there
-umount ~/Dev/inventory/inventory-server
+umount /Users/matt/Library/Mobile Documents/com~apple~CloudDocs/Dev/inventory/inventory-server
 
 #Mount
-sshfs matt@dashboard.kent.pw:/var/www/html/inventory -p 22122 ~/Dev/inventory/inventory-server/
+sshfs matt@dashboard.kent.pw:/var/www/html/inventory -p 22122 /Users/matt/Library/Mobile Documents/com~apple~CloudDocs/Dev/inventory/inventory-server/