Fix o3 issues on time-aggregates script

docs/calculate-issues.md

@@ -1,4 +1,4 @@
-1. **Missing Updates for Reorder Point and Safety Stock**
+1. **Missing Updates for Reorder Point and Safety Stock** [RESOLVED - product-metrics.js]
 - **Problem:** In the **product_metrics** table (used by the inventory health view), the fields **reorder_point** and **safety_stock** are never updated in the product metrics calculations. Although a helper function (`calculateReorderQuantities`) exists and computes these values, the update query in the `calculateProductMetrics` function does not assign any values to these columns.
 - **Effect:** The inventory health view relies on these fields (using COALESCE to default them to 0), which means that stock might never be classified as "Reorder" or "Healthy" based on the proper reorder point or safety stock calculations.
 - **Example:** Even if a product's base metrics would require a reorder (for example, if its days of inventory are low), the view always shows a value of 0 for reorder_point and safety_stock.
@@ -10,13 +10,13 @@
 - **Example:** An external caller expecting to run `calculateMetrics` would instead receive the `calculateProductMetrics` function.
 - **Fix:** Make sure each script resides in its own module file. Verify that the module boundaries and exports are not accidentally merged or overwritten when deployed.
 
-3. **Potential Formula Issue in EOQ Calculation (Reorder Qty)**
+3. **Potential Formula Issue in EOQ Calculation (Reorder Qty)** [RESOLVED - product-metrics.js]
 - **Problem:** The helper function `calculateReorderQuantities` uses an EOQ formula with a holding cost expressed as a percentage (0.25) rather than a per‐unit cost.
 - **Effect:** If the intent was to use the traditional EOQ formula (which expects a holding cost per unit rather than a percentage), this could lead to an incorrect reorder quantity.
 - **Example:** For a given annual demand and fixed order cost, the computed reorder quantity might be higher or lower than expected.
 - **Fix:** Double-check the EOQ formula. If the intention is to compute based on a percentage, then document that clearly; otherwise, adjust the formula to use the proper holding cost value.
 
-4. **Potential Overlap or Redundancy in GMROI Calculation**
+4. **Potential Overlap or Redundancy in GMROI Calculation** [RESOLVED - time-aggregates.js]
 - **Problem:** In the time aggregates function, GMROI is calculated in two steps. The initial INSERT query computes GMROI as
 
 	`CASE WHEN s.inventory_value > 0 THEN (s.total_revenue - s.total_cost) / s.inventory_value ELSE 0 END`
@@ -26,15 +26,13 @@
 
 - **Effect:** Overwriting a computed value may be intentional to refine the metric, but if not coordinated it can cause confusion or unexpected output in the `product_time_aggregates` table.
 - **Example:** A product's GMROI might first appear as a simple ratio but then be updated to a scaled value based on the number of active days, which could lead to inconsistent reporting if not documented.
-- **Fix:** Confirm that the two-step process is intended. If only the annualized GMROI is desired, consolidate the calculation into one query or clearly document why both steps are needed.
+- **Fix:** Consolidated the GMROI calculation into a single step in the initial INSERT query, properly handling annualization and NULL values.
 
-*This observation complements the earlier note about duplicate or overwritten calculations in the previous script. In both cases, it's important to verify that updates (or recalculations) are intentional rather than an oversight.*
-
-5. **Handling of Products Without Orders or Purchase Data**
-- ******Problem:** In the INSERT query of the time aggregates function, the UNION covers two cases: one for products with order data (from `monthly_sales`) and one for products that have entries in `monthly_stock` but no matching order data.
+5. **Handling of Products Without Orders or Purchase Data** [RESOLVED - time-aggregates.js]
+- **Problem:** In the INSERT query of the time aggregates function, the UNION covers two cases: one for products with order data (from `monthly_sales`) and one for products that have entries in `monthly_stock` but no matching order data.
 - **Effect:** If a product has neither orders nor purchase orders, it won't get an entry in `product_time_aggregates`. Depending on business rules, this might be acceptable or might mean missing data.
 - **Example:** A product that's new or rarely ordered might not appear in the time aggregates view, potentially affecting downstream calculations.
-- **Fix:** If you need every product to have an aggregate record (even with zeros), add an additional query or logic to ensure that products without any matching records in both CTEs are inserted with default values.
+- **Fix:** Added an `all_products` CTE and modified the JOIN structure to ensure every product gets an entry with appropriate default values, even if it has no orders or purchase orders.
 
 6. **Redundant Recalculation of Vendor Metrics**
 - **Problem:** Similar concepts from prior scripts where cumulative metrics (like **total_revenue** and **total_cost**) are calculated in multiple query steps without necessary validation or optimization. In the vendor metrics script, calculations for total revenue and margin are performed within a `WITH` clause, which is then used in other parts of the process, making it more complex than needed.

inventory-server/scripts/full-reset.js

@@ -57,25 +57,36 @@ async function fullReset() {
             message: 'Step 1/3: Resetting database...'
         });
         await runScript(path.join(__dirname, 'reset-db.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Database reset step complete',
+            message: 'Database reset finished, moving to import...'
+        });
 
         // Step 2: Import from Production
         outputProgress({
-            operation: 'Database reset complete',
+            operation: 'Starting import',
             message: 'Step 2/3: Importing from production...'
         });
         await runScript(path.join(__dirname, 'import-from-prod.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Import step complete',
+            message: 'Import finished, moving to metrics calculation...'
+        });
 
         // Step 3: Calculate Metrics
         outputProgress({
-            operation: 'Import complete',
+            operation: 'Starting metrics calculation',
             message: 'Step 3/3: Calculating metrics...'
         });
         await runScript(path.join(__dirname, 'calculate-metrics.js'));
-
+        
+        // Final completion message
         outputProgress({
             status: 'complete',
             operation: 'Full reset complete',
-            message: 'Successfully completed database reset, import, and metrics calculation'
+            message: 'Successfully completed all steps: database reset, import, and metrics calculation'
         });
     } catch (error) {
         outputProgress({

inventory-server/scripts/full-update.js

@@ -57,18 +57,29 @@ async function fullUpdate() {
             message: 'Step 1/2: Importing from production...'
         });
         await runScript(path.join(__dirname, 'import-from-prod.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Import step complete',
+            message: 'Import finished, moving to metrics calculation...'
+        });
 
         // Step 2: Calculate Metrics
         outputProgress({
-            operation: 'Import complete',
+            operation: 'Starting metrics calculation',
             message: 'Step 2/2: Calculating metrics...'
         });
         await runScript(path.join(__dirname, 'calculate-metrics.js'));
+        outputProgress({
+            status: 'complete',
+            operation: 'Metrics step complete',
+            message: 'Metrics calculation finished'
+        });
 
+        // Final completion message
         outputProgress({
             status: 'complete',
             operation: 'Full update complete',
-            message: 'Successfully completed import and metrics calculation'
+            message: 'Successfully completed all steps: import and metrics calculation'
         });
     } catch (error) {
         outputProgress({

inventory-server/scripts/metrics/product-metrics.js

@@ -153,7 +153,7 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                     LEFT JOIN temp_sales_metrics sm ON pm.pid = sm.pid
                     LEFT JOIN temp_purchase_metrics lm ON pm.pid = lm.pid
                     SET 
-                        pm.inventory_value = p.stock_quantity * p.cost_price,
+                        pm.inventory_value = p.stock_quantity * NULLIF(p.cost_price, 0),
                         pm.daily_sales_avg = COALESCE(sm.daily_sales_avg, 0),
                         pm.weekly_sales_avg = COALESCE(sm.weekly_sales_avg, 0),
                         pm.monthly_sales_avg = COALESCE(sm.monthly_sales_avg, 0),
@@ -164,12 +164,12 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                         pm.avg_lead_time_days = COALESCE(lm.avg_lead_time_days, 30),
                         pm.days_of_inventory = CASE 
                             WHEN COALESCE(sm.daily_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / sm.daily_sales_avg)
+                            THEN FLOOR(p.stock_quantity / NULLIF(sm.daily_sales_avg, 0))
                             ELSE NULL 
                         END,
                         pm.weeks_of_inventory = CASE 
                             WHEN COALESCE(sm.weekly_sales_avg, 0) > 0 
-                            THEN FLOOR(p.stock_quantity / sm.weekly_sales_avg)
+                            THEN FLOOR(p.stock_quantity / NULLIF(sm.weekly_sales_avg, 0))
                             ELSE NULL 
                         END,
                         pm.stock_status = CASE
@@ -181,10 +181,21 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                             WHEN p.stock_quantity / NULLIF(sm.daily_sales_avg, 0) > ? THEN 'Overstocked'
                             ELSE 'Healthy'
                         END,
-                        pm.reorder_qty = CASE
+                        pm.safety_stock = CASE
                             WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
+                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
+                            ELSE ?
+                        END,
+                        pm.reorder_point = CASE
+                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 THEN
+                                CEIL(sm.daily_sales_avg * COALESCE(lm.avg_lead_time_days, 30)) + 
+                                CEIL(sm.daily_sales_avg * SQRT(COALESCE(lm.avg_lead_time_days, 30)) * 1.96)
+                            ELSE ?
+                        END,
+                        pm.reorder_qty = CASE
+                            WHEN COALESCE(sm.daily_sales_avg, 0) > 0 AND NULLIF(p.cost_price, 0) IS NOT NULL THEN
                                 GREATEST(
-                                    CEIL(sm.daily_sales_avg * COALESCE(lm.avg_lead_time_days, 30) * 1.96),
+                                    CEIL(SQRT((2 * (sm.daily_sales_avg * 365) * 25) / (NULLIF(p.cost_price, 0) * 0.25))),
                                     ?
                                 )
                             ELSE ?
@@ -195,18 +206,22 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
                             ELSE 0
                         END,
                         pm.last_calculated_at = NOW()
-                    WHERE p.pid IN (?)
-                `, [
+                    WHERE p.pid IN (${batch.map(() => '?').join(',')})
+                `,
+                [
                     defaultThresholds.low_stock_threshold,
                     defaultThresholds.critical_days,
                     defaultThresholds.reorder_days,
                     defaultThresholds.overstock_days,
                     defaultThresholds.low_stock_threshold,
                     defaultThresholds.low_stock_threshold,
+                    defaultThresholds.low_stock_threshold,
+                    defaultThresholds.low_stock_threshold,
                     defaultThresholds.overstock_days,
                     defaultThresholds.overstock_days,
-                    batch.map(row => row.pid)
-                ]);
+                    ...batch.map(row => row.pid)
+                ]
+                );
 
                 lastPid = batch[batch.length - 1].pid;
                 processedCount += batch.length;
@@ -603,9 +618,9 @@ function calculateReorderQuantities(stock, stock_status, daily_sales_avg, avg_le
     if (daily_sales_avg > 0) {
         const annual_demand = daily_sales_avg * 365;
         const order_cost = 25; // Fixed cost per order
-        const holding_cost_percent = 0.25; // 25% annual holding cost
+        const holding_cost = config.cost_price * 0.25; // 25% of unit cost as annual holding cost
         
-        reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost_percent));
+        reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost));
     } else {
         // If no sales data, use a basic calculation
         reorder_qty = Math.max(safety_stock, config.low_stock_threshold);

inventory-server/scripts/metrics/time-aggregates.js

@@ -104,51 +104,99 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                     SUM(ordered) as stock_ordered
                 FROM purchase_orders
                 GROUP BY pid, YEAR(date), MONTH(date)
+            ),
+            base_products AS (
+                SELECT 
+                    p.pid,
+                    p.cost_price * p.stock_quantity as inventory_value
+                FROM products p
             )
             SELECT 
-                s.pid,
-                s.year,
-                s.month,
-                s.total_quantity_sold,
-                s.total_revenue,
-                s.total_cost,
-                s.order_count,
+                COALESCE(s.pid, ms.pid) as pid,
+                COALESCE(s.year, ms.year) as year,
+                COALESCE(s.month, ms.month) as month,
+                COALESCE(s.total_quantity_sold, 0) as total_quantity_sold,
+                COALESCE(s.total_revenue, 0) as total_revenue,
+                COALESCE(s.total_cost, 0) as total_cost,
+                COALESCE(s.order_count, 0) as order_count,
                 COALESCE(ms.stock_received, 0) as stock_received,
                 COALESCE(ms.stock_ordered, 0) as stock_ordered,
-                s.avg_price,
-                s.profit_margin,
-                s.inventory_value,
+                COALESCE(s.avg_price, 0) as avg_price,
+                COALESCE(s.profit_margin, 0) as profit_margin,
+                COALESCE(s.inventory_value, bp.inventory_value, 0) as inventory_value,
                 CASE 
-                    WHEN s.inventory_value > 0 THEN 
-                        (s.total_revenue - s.total_cost) / s.inventory_value
+                    WHEN COALESCE(s.inventory_value, bp.inventory_value, 0) > 0 
+                        AND COALESCE(s.active_days, 0) > 0 
+                    THEN (COALESCE(s.total_revenue - s.total_cost, 0) * (365.0 / s.active_days)) 
+                        / COALESCE(s.inventory_value, bp.inventory_value)
                     ELSE 0 
                 END as gmroi
-            FROM monthly_sales s
+            FROM (
+                SELECT * FROM monthly_sales s
+                UNION ALL
+                SELECT 
+                    ms.pid,
+                    ms.year,
+                    ms.month,
+                    0 as total_quantity_sold,
+                    0 as total_revenue,
+                    0 as total_cost,
+                    0 as order_count,
+                    NULL as avg_price,
+                    0 as profit_margin,
+                    NULL as inventory_value,
+                    0 as active_days
+                FROM monthly_stock ms
+                WHERE NOT EXISTS (
+                    SELECT 1 FROM monthly_sales s2
+                    WHERE s2.pid = ms.pid
+                    AND s2.year = ms.year
+                    AND s2.month = ms.month
+                )
+            ) s
             LEFT JOIN monthly_stock ms 
                 ON s.pid = ms.pid 
                 AND s.year = ms.year 
                 AND s.month = ms.month
+            JOIN base_products bp ON COALESCE(s.pid, ms.pid) = bp.pid
             UNION
             SELECT 
-                p.pid,
-                p.year,
-                p.month,
+                ms.pid,
+                ms.year,
+                ms.month,
                 0 as total_quantity_sold,
                 0 as total_revenue,
                 0 as total_cost,
                 0 as order_count,
-                p.stock_received,
-                p.stock_ordered,
+                ms.stock_received,
+                ms.stock_ordered,
                 0 as avg_price,
                 0 as profit_margin,
-                (SELECT cost_price * stock_quantity FROM products WHERE pid = p.pid) as inventory_value,
+                bp.inventory_value,
                 0 as gmroi
-            FROM monthly_stock p
-            LEFT JOIN monthly_sales s
-                ON p.pid = s.pid 
-                AND p.year = s.year 
-                AND p.month = s.month
-            WHERE s.pid IS NULL
+            FROM monthly_stock ms
+            JOIN base_products bp ON ms.pid = bp.pid
+            WHERE NOT EXISTS (
+                SELECT 1 FROM (
+                    SELECT * FROM monthly_sales
+                    UNION ALL
+                    SELECT 
+                        ms2.pid,
+                        ms2.year,
+                        ms2.month,
+                        0, 0, 0, 0, NULL, 0, NULL, 0
+                    FROM monthly_stock ms2
+                    WHERE NOT EXISTS (
+                        SELECT 1 FROM monthly_sales s2
+                        WHERE s2.pid = ms2.pid
+                        AND s2.year = ms2.year
+                        AND s2.month = ms2.month
+                    )
+                ) s
+                WHERE s.pid = ms.pid
+                AND s.year = ms.year
+                AND s.month = ms.month
+            )
             ON DUPLICATE KEY UPDATE
                 total_quantity_sold = VALUES(total_quantity_sold),
                 total_revenue = VALUES(total_revenue),
@@ -196,7 +244,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                     MONTH(o.date) as month,
                     p.cost_price * p.stock_quantity as inventory_value,
                     SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
-                    COUNT(DISTINCT DATE(o.date)) as days_in_period
+                    COUNT(DISTINCT DATE(o.date)) as active_days
                 FROM products p
                 LEFT JOIN orders o ON p.pid = o.pid
                 WHERE o.canceled = false
@@ -205,12 +253,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount
                 AND pta.year = fin.year 
                 AND pta.month = fin.month
             SET 
-                pta.inventory_value = COALESCE(fin.inventory_value, 0),
-                pta.gmroi = CASE 
-                    WHEN COALESCE(fin.inventory_value, 0) > 0 AND fin.days_in_period > 0 THEN 
-                        (COALESCE(fin.gross_profit, 0) * (365.0 / fin.days_in_period)) / COALESCE(fin.inventory_value, 0)
-                    ELSE 0 
-                END
+                pta.inventory_value = COALESCE(fin.inventory_value, 0)
         `);
 
         processedCount = Math.floor(totalProducts * 0.65);

inventory-server/src/routes/csv.js

@@ -85,8 +85,41 @@ function runScript(scriptPath, type, clients) {
         child.stdout.on('data', (data) => {
             const text = data.toString();
             output += text;
-            // Send raw output directly
-            sendProgressToClients(clients, text);
+            
+            // Split by lines to handle multiple JSON outputs
+            const lines = text.split('\n');
+            lines.filter(line => line.trim()).forEach(line => {
+                try {
+                    // Try to parse as JSON but don't let it affect the display
+                    const jsonData = JSON.parse(line);
+                    // Only end the process if we get a final status
+                    if (jsonData.status === 'complete' || jsonData.status === 'error' || jsonData.status === 'cancelled') {
+                        if (jsonData.status === 'complete' && !jsonData.operation?.includes('complete')) {
+                            // Don't close for intermediate completion messages
+                            sendProgressToClients(clients, line);
+                            return;
+                        }
+                        // Close only on final completion/error/cancellation
+                        switch (type) {
+                            case 'update':
+                                activeFullUpdate = null;
+                                break;
+                            case 'reset':
+                                activeFullReset = null;
+                                break;
+                        }
+                        if (jsonData.status === 'error') {
+                            reject(new Error(jsonData.error || 'Unknown error'));
+                        } else {
+                            resolve({ output });
+                        }
+                    }
+                } catch (e) {
+                    // Not JSON, just display as is
+                }
+                // Always send the raw line
+                sendProgressToClients(clients, line);
+            });
         });
 
         child.stderr.on('data', (data) => {
@@ -110,10 +143,8 @@ function runScript(scriptPath, type, clients) {
                 const error = `Script ${scriptPath} exited with code ${code}`;
                 sendProgressToClients(clients, error);
                 reject(new Error(error));
-            } else {
-                sendProgressToClients(clients, `${type} completed successfully`);
-                resolve({ output });
             }
+            // Don't resolve here - let the completion message from the script trigger the resolve
         });
 
         child.on('error', (err) => {

inventory/src/components/settings/DataManagement.tsx

@@ -24,7 +24,7 @@ import {
   AlertDialogTitle,
   AlertDialogTrigger,
 } from "@/components/ui/alert-dialog";
-import { Loader2, X, RefreshCw, AlertTriangle } from "lucide-react";
+import { Loader2, X, RefreshCw, AlertTriangle, RefreshCcw, Hourglass } from "lucide-react";
 import config from "../../config";
 import { toast } from "sonner";
 import { Table, TableBody, TableCell, TableRow, TableHeader, TableHead } from "@/components/ui/table";
@@ -101,7 +101,26 @@ export function DataManagement() {
   };
 
   // Helper to format duration with seconds
-  const formatDurationWithSeconds = (minutes: number) => {
+  const formatDurationWithSeconds = (minutes: number, isRunning: boolean = false, startTime?: string) => {
+    if (isRunning && startTime) {
+      const elapsedMinutes = (new Date().getTime() - new Date(startTime).getTime()) / (1000 * 60);
+      const hours = Math.floor(elapsedMinutes / 60);
+      const remainingMinutes = Math.floor(elapsedMinutes % 60);
+      const seconds = Math.round((elapsedMinutes % 1) * 60);
+      
+      const parts = [];
+      if (hours > 0) parts.push(`${hours}h`);
+      if (remainingMinutes > 0) parts.push(`${remainingMinutes}m`);
+      if (seconds > 0) parts.push(`${seconds}s`);
+
+      return (
+        <span className="flex items-center gap-1">
+          {parts.join(" ")}
+          <Hourglass className="h-3 w-3 animate-pulse" />
+        </span>
+      );
+    }
+
     if (minutes < 1 / 60) return "Less than a second";
 
     const hours = Math.floor(minutes / 60);
@@ -181,25 +200,30 @@ export function DataManagement() {
         // Try to parse for status updates, but don't affect display
         try {
           const data = JSON.parse(event.data);
-          if (
-            data.status === "complete" ||
-            data.status === "error" ||
-            data.status === "cancelled"
-          ) {
-            source.close();
-            setEventSource(null);
-            setIsUpdating(false);
+          if (data.status === 'complete' || data.status === 'error' || data.status === 'cancelled') {
+            // Only close and reset state if this is the final completion message
+            if (data.operation === 'Full update complete' || 
+                data.status === 'error' || 
+                data.status === 'cancelled') {
+              source.close();
+              setEventSource(null);
+              setIsUpdating(false);
 
-            if (data.status === "complete") {
-              toast.success("Update completed successfully");
-              fetchHistory();
-            } else if (data.status === "error") {
-              toast.error(`Update failed: ${data.error || "Unknown error"}`);
-            } else {
-              toast.warning("Update cancelled");
+              if (data.status === 'complete') {
+                toast.success("Update completed successfully");
+                fetchHistory();
+              } else if (data.status === 'error') {
+                toast.error(`Update failed: ${data.error || 'Unknown error'}`);
+              } else {
+                toast.warning("Update cancelled");
+              }
+            }
+            // For intermediate completions, just show a toast
+            else if (data.status === 'complete') {
+              toast.success(data.message || "Step completed");
             }
           }
-          } catch (error) {
+        } catch (error) {
           // Not JSON, just display as is
         }
       };
@@ -246,25 +270,30 @@ export function DataManagement() {
         // Try to parse for status updates, but don't affect display
         try {
           const data = JSON.parse(event.data);
-          if (
-            data.status === "complete" ||
-            data.status === "error" ||
-            data.status === "cancelled"
-          ) {
-            source.close();
-            setEventSource(null);
-            setIsResetting(false);
+          if (data.status === 'complete' || data.status === 'error' || data.status === 'cancelled') {
+            // Only close and reset state if this is the final completion message
+            if (data.operation === 'Full reset complete' || 
+                data.status === 'error' || 
+                data.status === 'cancelled') {
+              source.close();
+              setEventSource(null);
+              setIsResetting(false);
 
-            if (data.status === "complete") {
-              toast.success("Reset completed successfully");
-              fetchHistory();
-            } else if (data.status === "error") {
-              toast.error(`Reset failed: ${data.error || "Unknown error"}`);
-            } else {
-              toast.warning("Reset cancelled");
-        }
-      }
-    } catch (error) {
+              if (data.status === 'complete') {
+                toast.success("Reset completed successfully");
+                fetchHistory();
+              } else if (data.status === 'error') {
+                toast.error(`Reset failed: ${data.error || 'Unknown error'}`);
+              } else {
+                toast.warning("Reset cancelled");
+              }
+            }
+            // For intermediate completions, just show a toast
+            else if (data.status === 'complete') {
+              toast.success(data.message || "Step completed");
+            }
+          }
+        } catch (error) {
           // Not JSON, just display as is
         }
       };
@@ -355,6 +384,60 @@ export function DataManagement() {
     }
   };
 
+  const refreshTableStatus = async () => {
+    try {
+      const response = await fetch(`${config.apiUrl}/csv/status/tables`);
+      const data = await response.json();
+      setTableStatus(data);
+    } catch (error) {
+      toast.error("Failed to refresh table status");
+    }
+  };
+
+  const refreshModuleStatus = async () => {
+    try {
+      const response = await fetch(`${config.apiUrl}/csv/status/modules`);
+      const data = await response.json();
+      setModuleStatus(data);
+    } catch (error) {
+      toast.error("Failed to refresh module status");
+    }
+  };
+
+  const refreshImportHistory = async () => {
+    try {
+      const response = await fetch(`${config.apiUrl}/csv/history/import`);
+      const data = await response.json();
+      setImportHistory(data);
+    } catch (error) {
+      toast.error("Failed to refresh import history");
+    }
+  };
+
+  const refreshCalculateHistory = async () => {
+    try {
+      const response = await fetch(`${config.apiUrl}/csv/history/calculate`);
+      const data = await response.json();
+      setCalculateHistory(data);
+    } catch (error) {
+      toast.error("Failed to refresh calculate history");
+    }
+  };
+
+  const refreshAllData = async () => {
+    try {
+      await Promise.all([
+        refreshTableStatus(),
+        refreshModuleStatus(),
+        refreshImportHistory(),
+        refreshCalculateHistory()
+      ]);
+      toast.success("All data refreshed");
+    } catch (error) {
+      toast.error("Failed to refresh some data");
+    }
+  };
+
   useEffect(() => {
     fetchHistory();
   }, []);
@@ -501,7 +584,17 @@ export function DataManagement() {
 
       {/* History Section */}
       <div className="space-y-6">
-        <h2 className="text-2xl font-bold">History & Status</h2>
+        <div className="flex justify-between items-center">
+          <h2 className="text-2xl font-bold">History & Status</h2>
+          <Button
+            variant="ghost"
+            size="icon"
+            onClick={refreshAllData}
+            className="h-8 w-8"
+          >
+            <RefreshCcw className="h-4 w-4" />
+          </Button>
+        </div>
 
         <div className="grid gap-4 md:grid-cols-2">
           {/* Table Status */}
@@ -511,17 +604,23 @@ export function DataManagement() {
             </CardHeader>
             <CardContent>
               <div className="">
-                {tableStatus.map((table) => (
-                  <div
-                    key={table.table_name}
-                    className="flex justify-between text-sm items-center py-2 border-b last:border-0"
-                  >
-                    <span className="font-medium">{table.table_name}</span>
-                    <span className="text-sm text-gray-600">
-                      {formatStatusTime(table.last_sync_timestamp)}
-                    </span>
+                {tableStatus.length > 0 ? (
+                  tableStatus.map((table) => (
+                    <div
+                      key={table.table_name}
+                      className="flex justify-between text-sm items-center py-2 border-b last:border-0"
+                    >
+                      <span className="font-medium">{table.table_name}</span>
+                      <span className="text-sm text-gray-600">
+                        {formatStatusTime(table.last_sync_timestamp)}
+                      </span>
+                    </div>
+                  ))
+                ) : (
+                  <div className="text-sm text-muted-foreground py-4 text-center">
+                    No imports have been performed yet.<br/>Run a full update or reset to import data.
                   </div>
-                ))}
+                )}
               </div>
             </CardContent>
           </Card>
@@ -532,17 +631,23 @@ export function DataManagement() {
             </CardHeader>
             <CardContent>
               <div className="">
-                {moduleStatus.map((module) => (
-                  <div
-                    key={module.module_name}
-                    className="flex justify-between text-sm items-center py-2 border-b last:border-0"
-                  >
-                    <span className="font-medium">{module.module_name}</span>
-                    <span className="text-sm text-gray-600">
-                      {formatStatusTime(module.last_calculation_timestamp)}
-                    </span>
+                {moduleStatus.length > 0 ? (
+                  moduleStatus.map((module) => (
+                    <div
+                      key={module.module_name}
+                      className="flex justify-between text-sm items-center py-2 border-b last:border-0"
+                    >
+                      <span className="font-medium">{module.module_name}</span>
+                      <span className="text-sm text-gray-600">
+                        {formatStatusTime(module.last_calculation_timestamp)}
+                      </span>
+                    </div>
+                  ))
+                ) : (
+                  <div className="text-sm text-muted-foreground py-4 text-center">
+                    No metrics have been calculated yet.<br/>Run a full update or reset to calculate metrics.
                   </div>
-                ))}
+                )}
               </div>
             </CardContent>
           </Card>
@@ -573,7 +678,9 @@ export function DataManagement() {
                               </span>
                               <span className="text-sm min-w-[100px]">
                                 {formatDurationWithSeconds(
-                                  record.duration_minutes
+                                  record.duration_minutes,
+                                  record.status === "running",
+                                  record.start_time
                                 )}
                               </span>
                               <span
@@ -653,7 +760,9 @@ export function DataManagement() {
                               </span>
                               <span className="text-sm min-w-[100px]">
                                 {formatDurationWithSeconds(
-                                  record.duration_minutes
+                                  record.duration_minutes,
+                                  record.status === "running",
+                                  record.start_time
                                 )}
                               </span>