matt/inventory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update calculate script to account for import changes

Browse Source
This commit is contained in:
Matt
2025-01-27 10:41:18 -05:00
parent 44d9ae2aad
commit 5781b45f37
10 changed files with 508 additions and 297 deletions
Download Patch File Download Diff File Expand all files Collapse all files

181
docs/metrics-changes.md Normal file
View File

@@ -0,0 +1,181 @@
# Metrics System Changes
## Schema Changes
### Product Identifiers
- Changed `product_id` to `pid` throughout all metrics tables and queries
- Changed `category_id` to `cat_id` in category-related queries
### Purchase Orders
- Changed status check from `status = 'closed'` to `receiving_status >= 30`
- Added comment `-- Partial or fully received` for clarity
- Now using `received_date` instead of relying on status changes
### New Product Fields
- Added support for `notions_inv_count`
- Added support for `date_last_sold`
- Added support for `total_sold`
- Using `visible` flag for active product counts
### Field Size Updates
- Increased size of financial fields to handle larger numbers:
- Changed category metrics `total_value` from `DECIMAL(10,3)` to `DECIMAL(15,3)`
- Changed brand metrics financial fields from `DECIMAL(10,2)` to `DECIMAL(15,2)`
- Affects `total_stock_cost`, `total_stock_retail`, `total_revenue`
## Metrics File Changes
### Product Metrics (`product-metrics.js`)
- Updated SQL queries to use new field names
- Enhanced stock status calculations
- Added financial metrics:
- `gross_profit`
- `gmroi`
- `avg_margin_percent`
- `inventory_value`
- Improved reorder quantity calculations with:
- Enhanced safety stock calculation
- Lead time consideration
- Service level factors
- Added NaN/NULL value handling:
- Added `sanitizeValue` helper function
- Properly converts JavaScript NaN to SQL NULL
- Ensures all numeric fields have valid values
### Vendor Metrics (`vendor-metrics.js`)
- Updated field references to use `pid`
- Modified purchase order status checks
- Enhanced vendor performance metrics:
- Order fill rate calculation
- On-time delivery rate
- Lead time tracking
### Category Metrics (`category-metrics.js`)
- Updated to use `cat_id` instead of `category_id`
- Enhanced category performance tracking:
- Active vs total products
- Category growth rate
- Turnover rate
- Added time-based metrics for:
- Product counts
- Revenue tracking
- Margin analysis
- Added NULL brand handling:
- Uses 'Unbranded' for NULL brand values
- Maintains data integrity in category sales metrics
### Brand Metrics (`brand-metrics.js`)
- Updated product references to use `pid`
- Enhanced brand performance metrics:
- Stock value calculations
- Revenue tracking
- Growth rate analysis
- Added time-based aggregates for:
- Stock levels
- Sales performance
- Margin analysis
- Increased field sizes to handle large retailers
### Sales Forecasts (`sales-forecasts.js`)
- Updated to use new product identifiers
- Enhanced forecast calculations:
- Day-of-week patterns
- Seasonality factors
- Confidence levels
- Added category-level forecasts with:
- Units and revenue predictions
- Confidence scoring
- Seasonal adjustments
### Time Aggregates (`time-aggregates.js`)
- Updated field references to use `pid`
- Enhanced financial metrics:
- GMROI calculations
- Profit margin tracking
- Added inventory value tracking
- Improved purchase order integration
## Database Impact
### Tables Modified
- `product_metrics`
- `vendor_metrics`
- `vendor_time_metrics`
- `category_metrics`
- `category_time_metrics`
- `brand_metrics`
- `brand_time_metrics`
- `sales_forecasts`
- `category_forecasts`
- `product_time_aggregates`
### New Fields Added
Several tables have new fields for:
- Enhanced financial tracking
- Improved inventory metrics
- Better performance monitoring
- More accurate forecasting
## Frontend Considerations
### Data Access Changes
- All product lookups need to use `pid` instead of `product_id`
- Category references should use `cat_id`
- Purchase order status checks need updating
- Handle 'Unbranded' as a valid brand value
### New Features Available
- Enhanced stock status indicators
- More detailed financial metrics
- Improved forecasting data
- Better category and brand performance tracking
### UI Updates Needed
- Update all API calls to use new field names
- Modify data displays for new metrics
- Add new financial performance indicators
- Update stock status logic
- Enhance forecast displays
## API Route Updates Needed
### Product Routes
- Update ID field references
- Modify stock status calculations
- Add new financial metrics endpoints
### Category Routes
- Update to use `cat_id`
- Add new performance metrics
- Include time-based aggregates
### Vendor Routes
- Update product reference handling
- Add enhanced performance metrics
- Include new time-based data
### Reporting Routes
- Update all ID references
- Add new metrics support
- Include enhanced forecasting data
## Migration Considerations
### Data Migration
- Update existing records to use new IDs
- Backfill new metrics where possible
- Verify data integrity after changes
- Handle NULL to 'Unbranded' brand conversion
### Code Updates
- Update all API endpoints
- Modify database queries
- Update frontend components
- Revise reporting logic
### Testing Requirements
- Verify ID changes throughout system
- Test new metrics calculations
- Validate forecasting accuracy
- Check performance impact
- Verify NULL value handling

18
inventory-server/db/metrics-schema.sql
View File

@@ -152,7 +152,7 @@ CREATE TABLE IF NOT EXISTS category_metrics (
product_count INT DEFAULT 0, product_count INT DEFAULT 0,
active_products INT DEFAULT 0, active_products INT DEFAULT 0,
-- Financial metrics -- Financial metrics
total_value DECIMAL(10,3) DEFAULT 0, total_value DECIMAL(15,3) DEFAULT 0,
avg_margin DECIMAL(5,2), avg_margin DECIMAL(5,2),
turnover_rate DECIMAL(12,3), turnover_rate DECIMAL(12,3),
growth_rate DECIMAL(5,2), growth_rate DECIMAL(5,2),
@@ -193,8 +193,8 @@ CREATE TABLE IF NOT EXISTS category_time_metrics (
product_count INT DEFAULT 0, product_count INT DEFAULT 0,
active_products INT DEFAULT 0, active_products INT DEFAULT 0,
-- Financial metrics -- Financial metrics
total_value DECIMAL(10,3) DEFAULT 0, total_value DECIMAL(15,3) DEFAULT 0,
total_revenue DECIMAL(10,3) DEFAULT 0, total_revenue DECIMAL(15,3) DEFAULT 0,
avg_margin DECIMAL(5,2), avg_margin DECIMAL(5,2),
turnover_rate DECIMAL(12,3), turnover_rate DECIMAL(12,3),
PRIMARY KEY (category_id, year, month), PRIMARY KEY (category_id, year, month),
@@ -228,10 +228,10 @@ CREATE TABLE IF NOT EXISTS brand_metrics (
active_products INT DEFAULT 0, active_products INT DEFAULT 0,
-- Stock metrics -- Stock metrics
total_stock_units INT DEFAULT 0, total_stock_units INT DEFAULT 0,
total_stock_cost DECIMAL(10,2) DEFAULT 0, total_stock_cost DECIMAL(15,2) DEFAULT 0,
total_stock_retail DECIMAL(10,2) DEFAULT 0, total_stock_retail DECIMAL(15,2) DEFAULT 0,
-- Sales metrics -- Sales metrics
total_revenue DECIMAL(10,2) DEFAULT 0, total_revenue DECIMAL(15,2) DEFAULT 0,
avg_margin DECIMAL(5,2) DEFAULT 0, avg_margin DECIMAL(5,2) DEFAULT 0,
growth_rate DECIMAL(5,2) DEFAULT 0, growth_rate DECIMAL(5,2) DEFAULT 0,
PRIMARY KEY (brand), PRIMARY KEY (brand),
@@ -250,10 +250,10 @@ CREATE TABLE IF NOT EXISTS brand_time_metrics (
active_products INT DEFAULT 0, active_products INT DEFAULT 0,
-- Stock metrics -- Stock metrics
total_stock_units INT DEFAULT 0, total_stock_units INT DEFAULT 0,
total_stock_cost DECIMAL(10,2) DEFAULT 0, total_stock_cost DECIMAL(15,2) DEFAULT 0,
total_stock_retail DECIMAL(10,2) DEFAULT 0, total_stock_retail DECIMAL(15,2) DEFAULT 0,
-- Sales metrics -- Sales metrics
total_revenue DECIMAL(10,2) DEFAULT 0, total_revenue DECIMAL(15,2) DEFAULT 0,
avg_margin DECIMAL(5,2) DEFAULT 0, avg_margin DECIMAL(5,2) DEFAULT 0,
PRIMARY KEY (brand, year, month), PRIMARY KEY (brand, year, month),
INDEX idx_brand_date (year, month) INDEX idx_brand_date (year, month)

6
inventory-server/scripts/calculate-metrics.js
View File

@@ -5,6 +5,9 @@ process.chdir(path.dirname(__filename));
require('dotenv').config({ path: path.resolve(__dirname, '..', '.env') }); require('dotenv').config({ path: path.resolve(__dirname, '..', '.env') });
// Set to 1 to skip product metrics and only calculate the remaining metrics
const SKIP_PRODUCT_METRICS = 0;
// Add error handler for uncaught exceptions // Add error handler for uncaught exceptions
process.on('uncaughtException', (error) => { process.on('uncaughtException', (error) => {
console.error('Uncaught Exception:', error); console.error('Uncaught Exception:', error);
@@ -43,9 +46,6 @@ const calculateCategoryMetrics = require('./metrics/category-metrics');
const calculateBrandMetrics = require('./metrics/brand-metrics'); const calculateBrandMetrics = require('./metrics/brand-metrics');
const calculateSalesForecasts = require('./metrics/sales-forecasts'); const calculateSalesForecasts = require('./metrics/sales-forecasts');
// Set to 1 to skip product metrics and only calculate the remaining metrics
const SKIP_PRODUCT_METRICS = 1;
// Add cancel handler // Add cancel handler
let isCancelled = false; let isCancelled = false;

52
inventory-server/scripts/metrics/brand-metrics.js
View File

@@ -31,12 +31,24 @@ async function calculateBrandMetrics(startTime, totalProducts, processedCount) {
WITH brand_data AS ( WITH brand_data AS (
SELECT SELECT
p.brand, p.brand,
COUNT(DISTINCT p.product_id) as product_count, COUNT(DISTINCT CASE WHEN p.stock_quantity <= 5000 THEN p.pid END) as product_count,
COUNT(DISTINCT CASE WHEN p.visible = true THEN p.product_id END) as active_products, COUNT(DISTINCT CASE WHEN p.visible = true AND p.stock_quantity <= 5000 THEN p.pid END) as active_products,
SUM(p.stock_quantity) as total_stock_units, SUM(CASE
SUM(p.stock_quantity * p.cost_price) as total_stock_cost, WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 THEN 0
SUM(p.stock_quantity * p.price) as total_stock_retail, ELSE p.stock_quantity
SUM(o.price * o.quantity) as total_revenue, END) as total_stock_units,
SUM(CASE
WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 OR p.cost_price IS NULL OR p.cost_price < 0 THEN 0
ELSE p.stock_quantity * p.cost_price
END) as total_stock_cost,
SUM(CASE
WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 OR p.price IS NULL OR p.price < 0 THEN 0
ELSE p.stock_quantity * p.price
END) as total_stock_retail,
SUM(CASE
WHEN o.quantity IS NULL OR o.price IS NULL OR o.quantity < 0 OR o.price < 0 THEN 0
ELSE o.price * o.quantity
END) as total_revenue,
CASE CASE
WHEN SUM(o.price * o.quantity) > 0 THEN WHEN SUM(o.price * o.quantity) > 0 THEN
(SUM((o.price - p.cost_price) * o.quantity) * 100.0) / SUM(o.price * o.quantity) (SUM((o.price - p.cost_price) * o.quantity) * 100.0) / SUM(o.price * o.quantity)
@@ -55,7 +67,7 @@ async function calculateBrandMetrics(startTime, totalProducts, processedCount) {
ELSE 0 ELSE 0
END) as previous_year_period_sales END) as previous_year_period_sales
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
WHERE p.brand IS NOT NULL WHERE p.brand IS NOT NULL
GROUP BY p.brand GROUP BY p.brand
) )
@@ -111,19 +123,31 @@ async function calculateBrandMetrics(startTime, totalProducts, processedCount) {
p.brand, p.brand,
YEAR(o.date) as year, YEAR(o.date) as year,
MONTH(o.date) as month, MONTH(o.date) as month,
COUNT(DISTINCT p.product_id) as product_count, COUNT(DISTINCT CASE WHEN p.stock_quantity <= 5000 THEN p.pid END) as product_count,
COUNT(DISTINCT CASE WHEN p.visible = true THEN p.product_id END) as active_products, COUNT(DISTINCT CASE WHEN p.visible = true AND p.stock_quantity <= 5000 THEN p.pid END) as active_products,
SUM(p.stock_quantity) as total_stock_units, SUM(CASE
SUM(p.stock_quantity * p.cost_price) as total_stock_cost, WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 THEN 0
SUM(p.stock_quantity * p.price) as total_stock_retail, ELSE p.stock_quantity
SUM(o.price * o.quantity) as total_revenue, END) as total_stock_units,
SUM(CASE
WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 OR p.cost_price IS NULL OR p.cost_price < 0 THEN 0
ELSE p.stock_quantity * p.cost_price
END) as total_stock_cost,
SUM(CASE
WHEN p.stock_quantity IS NULL OR p.stock_quantity < 0 OR p.stock_quantity > 5000 OR p.price IS NULL OR p.price < 0 THEN 0
ELSE p.stock_quantity * p.price
END) as total_stock_retail,
SUM(CASE
WHEN o.quantity IS NULL OR o.price IS NULL OR o.quantity < 0 OR o.price < 0 THEN 0
ELSE o.price * o.quantity
END) as total_revenue,
CASE CASE
WHEN SUM(o.price * o.quantity) > 0 THEN WHEN SUM(o.price * o.quantity) > 0 THEN
(SUM((o.price - p.cost_price) * o.quantity) * 100.0) / SUM(o.price * o.quantity) (SUM((o.price - p.cost_price) * o.quantity) * 100.0) / SUM(o.price * o.quantity)
ELSE 0 ELSE 0
END as avg_margin END as avg_margin
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
WHERE p.brand IS NOT NULL WHERE p.brand IS NOT NULL
AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH) AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
GROUP BY p.brand, YEAR(o.date), MONTH(o.date) GROUP BY p.brand, YEAR(o.date), MONTH(o.date)

44
inventory-server/scripts/metrics/category-metrics.js
View File

@@ -29,9 +29,9 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
) )
WITH category_sales AS ( WITH category_sales AS (
SELECT SELECT
c.id as category_id, c.cat_id as category_id,
COUNT(DISTINCT p.product_id) as product_count, COUNT(DISTINCT p.pid) as product_count,
COUNT(DISTINCT CASE WHEN p.visible = true THEN p.product_id END) as active_products, COUNT(DISTINCT CASE WHEN p.visible = true THEN p.pid END) as active_products,
SUM(p.stock_quantity * p.cost_price) as total_value, SUM(p.stock_quantity * p.cost_price) as total_value,
CASE CASE
WHEN SUM(o.price * o.quantity) > 0 WHEN SUM(o.price * o.quantity) > 0
@@ -68,10 +68,10 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
END) as previous_year_period_sales, END) as previous_year_period_sales,
c.status c.status
FROM categories c FROM categories c
LEFT JOIN product_categories pc ON c.id = pc.category_id LEFT JOIN product_categories pc ON c.cat_id = pc.cat_id
LEFT JOIN products p ON pc.product_id = p.product_id LEFT JOIN products p ON pc.pid = p.pid
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
GROUP BY c.id, c.status GROUP BY c.cat_id, c.status
) )
SELECT SELECT
category_id, category_id,
@@ -120,11 +120,11 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
turnover_rate turnover_rate
) )
SELECT SELECT
c.id as category_id, c.cat_id as category_id,
YEAR(o.date) as year, YEAR(o.date) as year,
MONTH(o.date) as month, MONTH(o.date) as month,
COUNT(DISTINCT p.product_id) as product_count, COUNT(DISTINCT p.pid) as product_count,
COUNT(DISTINCT CASE WHEN p.visible = true THEN p.product_id END) as active_products, COUNT(DISTINCT CASE WHEN p.visible = true THEN p.pid END) as active_products,
SUM(p.stock_quantity * p.cost_price) as total_value, SUM(p.stock_quantity * p.cost_price) as total_value,
SUM(o.price * o.quantity) as total_revenue, SUM(o.price * o.quantity) as total_revenue,
CASE CASE
@@ -138,11 +138,11 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
ELSE 0 ELSE 0
END as turnover_rate END as turnover_rate
FROM categories c FROM categories c
LEFT JOIN product_categories pc ON c.id = pc.category_id LEFT JOIN product_categories pc ON c.cat_id = pc.cat_id
LEFT JOIN products p ON pc.product_id = p.product_id LEFT JOIN products p ON pc.pid = p.pid
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
WHERE o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH) WHERE o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
GROUP BY c.id, YEAR(o.date), MONTH(o.date) GROUP BY c.cat_id, YEAR(o.date), MONTH(o.date)
ON DUPLICATE KEY UPDATE ON DUPLICATE KEY UPDATE
product_count = VALUES(product_count), product_count = VALUES(product_count),
active_products = VALUES(active_products), active_products = VALUES(active_products),
@@ -152,7 +152,7 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
turnover_rate = VALUES(turnover_rate) turnover_rate = VALUES(turnover_rate)
`); `);
// Calculate category sales metrics // Calculate category sales metrics with NULL brand handling
await connection.query(` await connection.query(`
INSERT INTO category_sales_metrics ( INSERT INTO category_sales_metrics (
category_id, category_id,
@@ -184,22 +184,22 @@ async function calculateCategoryMetrics(startTime, totalProducts, processedCount
), ),
category_metrics AS ( category_metrics AS (
SELECT SELECT
c.id as category_id, c.cat_id as category_id,
p.brand, COALESCE(p.brand, 'Unbranded') as brand,
dr.period_start, dr.period_start,
dr.period_end, dr.period_end,
COUNT(DISTINCT p.product_id) as num_products, COUNT(DISTINCT p.pid) as num_products,
COALESCE(SUM(o.quantity), 0) / DATEDIFF(dr.period_end, dr.period_start) as avg_daily_sales, COALESCE(SUM(o.quantity), 0) / DATEDIFF(dr.period_end, dr.period_start) as avg_daily_sales,
COALESCE(SUM(o.quantity), 0) as total_sold, COALESCE(SUM(o.quantity), 0) as total_sold,
COALESCE(AVG(o.price), 0) as avg_price COALESCE(AVG(o.price), 0) as avg_price
FROM categories c FROM categories c
JOIN product_categories pc ON c.id = pc.category_id JOIN product_categories pc ON c.cat_id = pc.cat_id
JOIN products p ON pc.product_id = p.product_id JOIN products p ON pc.pid = p.pid
CROSS JOIN date_ranges dr CROSS JOIN date_ranges dr
LEFT JOIN orders o ON p.product_id = o.product_id LEFT JOIN orders o ON p.pid = o.pid
AND o.date BETWEEN dr.period_start AND dr.period_end AND o.date BETWEEN dr.period_start AND dr.period_end
AND o.canceled = false AND o.canceled = false
GROUP BY c.id, p.brand, dr.period_start, dr.period_end GROUP BY c.cat_id, COALESCE(p.brand, 'Unbranded'), dr.period_start, dr.period_end
) )
SELECT SELECT
category_id, category_id,

16
inventory-server/scripts/metrics/financial-metrics.js
View File

@@ -19,7 +19,7 @@ async function calculateFinancialMetrics(startTime, totalProducts, processedCoun
UPDATE product_metrics pm UPDATE product_metrics pm
JOIN ( JOIN (
SELECT SELECT
p.product_id, p.pid,
p.cost_price * p.stock_quantity as inventory_value, p.cost_price * p.stock_quantity as inventory_value,
SUM(o.quantity * o.price) as total_revenue, SUM(o.quantity * o.price) as total_revenue,
SUM(o.quantity * p.cost_price) as cost_of_goods_sold, SUM(o.quantity * p.cost_price) as cost_of_goods_sold,
@@ -28,11 +28,11 @@ async function calculateFinancialMetrics(startTime, totalProducts, processedCoun
MAX(o.date) as last_sale_date, MAX(o.date) as last_sale_date,
DATEDIFF(MAX(o.date), MIN(o.date)) + 1 as calculation_period_days DATEDIFF(MAX(o.date), MIN(o.date)) + 1 as calculation_period_days
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id LEFT JOIN orders o ON p.pid = o.pid
WHERE o.canceled = false WHERE o.canceled = false
AND DATE(o.date) >= DATE_SUB(CURDATE(), INTERVAL 12 MONTH) AND DATE(o.date) >= DATE_SUB(CURDATE(), INTERVAL 12 MONTH)
GROUP BY p.product_id GROUP BY p.pid
) fin ON pm.product_id = fin.product_id ) fin ON pm.pid = fin.pid
SET SET
pm.inventory_value = COALESCE(fin.inventory_value, 0), pm.inventory_value = COALESCE(fin.inventory_value, 0),
pm.total_revenue = COALESCE(fin.total_revenue, 0), pm.total_revenue = COALESCE(fin.total_revenue, 0),
@@ -50,17 +50,17 @@ async function calculateFinancialMetrics(startTime, totalProducts, processedCoun
UPDATE product_time_aggregates pta UPDATE product_time_aggregates pta
JOIN ( JOIN (
SELECT SELECT
p.product_id, p.pid,
YEAR(o.date) as year, YEAR(o.date) as year,
MONTH(o.date) as month, MONTH(o.date) as month,
p.cost_price * p.stock_quantity as inventory_value, p.cost_price * p.stock_quantity as inventory_value,
SUM(o.quantity * (o.price - p.cost_price)) as gross_profit, SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
COUNT(DISTINCT DATE(o.date)) as days_in_period COUNT(DISTINCT DATE(o.date)) as days_in_period
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id LEFT JOIN orders o ON p.pid = o.pid
WHERE o.canceled = false WHERE o.canceled = false
GROUP BY p.product_id, YEAR(o.date), MONTH(o.date) GROUP BY p.pid, YEAR(o.date), MONTH(o.date)
) fin ON pta.product_id = fin.product_id ) fin ON pta.pid = fin.pid
AND pta.year = fin.year AND pta.year = fin.year
AND pta.month = fin.month AND pta.month = fin.month
SET SET

344
inventory-server/scripts/metrics/product-metrics.js
View File

@@ -1,13 +1,21 @@
const { outputProgress, logError } = require('./utils/progress'); const { outputProgress, logError } = require('./utils/progress');
const { getConnection } = require('./utils/db'); const { getConnection } = require('./utils/db');
// Helper function to handle NaN and undefined values
function sanitizeValue(value) {
if (value === undefined || value === null || Number.isNaN(value)) {
return null;
}
return value;
}
async function calculateProductMetrics(startTime, totalProducts, processedCount = 0) { async function calculateProductMetrics(startTime, totalProducts, processedCount = 0) {
const connection = await getConnection(); const connection = await getConnection();
try { try {
// Process in batches of 250 // Process in batches of 250
const batchSize = 250; const batchSize = 250;
for (let offset = 0; offset < totalProducts; offset += batchSize) { for (let offset = 0; offset < totalProducts; offset += batchSize) {
const [products] = await connection.query('SELECT product_id, vendor FROM products LIMIT ? OFFSET ?', [batchSize, offset]) const [products] = await connection.query('SELECT pid, vendor FROM products LIMIT ? OFFSET ?', [batchSize, offset])
.catch(err => { .catch(err => {
logError(err, `Failed to fetch products batch at offset ${offset}`); logError(err, `Failed to fetch products batch at offset ${offset}`);
throw err; throw err;
@@ -34,12 +42,12 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
const [configs] = await connection.query(` const [configs] = await connection.query(`
WITH product_info AS ( WITH product_info AS (
SELECT SELECT
p.product_id, p.pid,
p.vendor, p.vendor,
pc.category_id pc.cat_id as category_id
FROM products p FROM products p
LEFT JOIN product_categories pc ON p.product_id = pc.product_id LEFT JOIN product_categories pc ON p.pid = pc.pid
WHERE p.product_id = ? WHERE p.pid = ?
), ),
threshold_options AS ( threshold_options AS (
SELECT SELECT
@@ -141,7 +149,7 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
ORDER BY priority LIMIT 1), ORDER BY priority LIMIT 1),
95.0 95.0
) as service_level ) as service_level
`, [product.product_id]); `, [product.pid]);
const config = configs[0]; const config = configs[0];
@@ -163,9 +171,9 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
END as rolling_weekly_avg, END as rolling_weekly_avg,
SUM(CASE WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY) THEN o.quantity ELSE 0 END) as last_month_qty SUM(CASE WHEN o.date >= DATE_SUB(CURDATE(), INTERVAL ? DAY) THEN o.quantity ELSE 0 END) as last_month_qty
FROM orders o FROM orders o
JOIN products p ON o.product_id = p.product_id JOIN products p ON o.pid = p.pid
WHERE o.canceled = 0 AND o.product_id = ? WHERE o.canceled = 0 AND o.pid = ?
GROUP BY o.product_id GROUP BY o.pid
) )
SELECT SELECT
total_quantity_sold, total_quantity_sold,
@@ -184,7 +192,7 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
config.weekly_window_days, config.weekly_window_days,
config.weekly_window_days, config.weekly_window_days,
config.monthly_window_days, config.monthly_window_days,
product.product_id, product.pid,
config.daily_window_days, config.daily_window_days,
config.weekly_window_days, config.weekly_window_days,
config.monthly_window_days config.monthly_window_days
@@ -201,8 +209,8 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
DATEDIFF(received_date, date) as lead_time_days, DATEDIFF(received_date, date) as lead_time_days,
ROW_NUMBER() OVER (ORDER BY date DESC) as order_rank ROW_NUMBER() OVER (ORDER BY date DESC) as order_rank
FROM purchase_orders FROM purchase_orders
WHERE status = 'closed' WHERE receiving_status >= 30 -- Partial or fully received
AND product_id = ? AND pid = ?
AND received > 0 AND received > 0
AND received_date IS NOT NULL AND received_date IS NOT NULL
), ),
@@ -220,7 +228,7 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
MAX(received_date) as last_received_date, MAX(received_date) as last_received_date,
AVG(lead_time_days) as avg_lead_time_days AVG(lead_time_days) as avg_lead_time_days
FROM lead_time_orders FROM lead_time_orders
`, [product.product_id]); `, [product.pid]);
// Get stock info // Get stock info
const [stockInfo] = await connection.query(` const [stockInfo] = await connection.query(`
@@ -230,113 +238,122 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
p.created_at, p.created_at,
p.replenishable, p.replenishable,
p.moq, p.moq,
p.notions_inv_count,
p.date_last_sold,
p.total_sold,
DATEDIFF(CURDATE(), MIN(po.received_date)) as days_since_first_stock, DATEDIFF(CURDATE(), MIN(po.received_date)) as days_since_first_stock,
DATEDIFF(CURDATE(), COALESCE( DATEDIFF(CURDATE(), COALESCE(p.date_last_sold, CURDATE())) as days_since_last_sale,
(SELECT MAX(o2.date)
FROM orders o2
WHERE o2.product_id = p.product_id
AND o2.canceled = false),
CURDATE()
)) as days_since_last_sale,
(SELECT SUM(quantity)
FROM orders o3
WHERE o3.product_id = p.product_id
AND o3.canceled = false) as total_quantity_sold,
CASE CASE
WHEN EXISTS ( WHEN EXISTS (
SELECT 1 FROM orders o SELECT 1 FROM orders o
WHERE o.product_id = p.product_id WHERE o.pid = p.pid
AND o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY) AND o.date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
AND o.canceled = false AND o.canceled = false
AND (SELECT SUM(quantity) FROM orders o2 AND (SELECT SUM(quantity) FROM orders o2
WHERE o2.product_id = p.product_id WHERE o2.pid = p.pid
AND o2.date >= o.date AND o2.date >= o.date
AND o2.canceled = false) = 0 AND o2.canceled = false) = 0
) THEN true ) THEN true
ELSE false ELSE false
END as had_recent_stockout END as had_recent_stockout
FROM products p FROM products p
LEFT JOIN purchase_orders po ON p.product_id = po.product_id LEFT JOIN purchase_orders po ON p.pid = po.pid
AND po.status = 'closed' AND po.receiving_status >= 30 -- Partial or fully received
AND po.received > 0 AND po.received > 0
WHERE p.product_id = ? WHERE p.pid = ?
GROUP BY p.product_id GROUP BY p.pid
`, [product.product_id]); `, [product.pid]);
// Calculate metrics // Calculate metrics
const metrics = salesMetrics[0] || {}; const salesData = salesMetrics[0] || {};
const purchases = purchaseMetrics[0] || {}; const purchaseData = purchaseMetrics[0] || {};
const stock = stockInfo[0] || {}; const stockData = stockInfo[0] || {};
const daily_sales_avg = metrics.rolling_daily_avg || 0; // Sales velocity metrics
const weekly_sales_avg = metrics.rolling_weekly_avg || 0; const daily_sales_avg = sanitizeValue(salesData.rolling_daily_avg) || 0;
const monthly_sales_avg = metrics.total_quantity_sold ? metrics.total_quantity_sold / 30 : 0; const weekly_sales_avg = sanitizeValue(salesData.rolling_weekly_avg) || 0;
const monthly_sales_avg = sanitizeValue(salesData.rolling_monthly_avg) || 0;
// Calculate days of inventory // Stock metrics
const days_of_inventory = daily_sales_avg > 0 ? const stock_quantity = sanitizeValue(stockData.stock_quantity) || 0;
Math.ceil( const days_of_inventory = daily_sales_avg > 0 ? Math.floor(stock_quantity / daily_sales_avg) : 999;
(stock.stock_quantity / daily_sales_avg) + const weeks_of_inventory = Math.floor(days_of_inventory / 7);
(purchases.avg_lead_time_days || config.reorder_days) *
(1 + (config.service_level / 100))
) : null;
const weeks_of_inventory = days_of_inventory ? Math.ceil(days_of_inventory / 7) : null;
// Calculate margin percent
const margin_percent = metrics.total_revenue > 0 ?
((metrics.total_revenue - metrics.total_cost) / metrics.total_revenue) * 100 :
null;
// Calculate inventory value
const inventory_value = (stock.stock_quantity || 0) * (stock.cost_price || 0);
// Calculate stock status // Calculate stock status
const stock_status = calculateStockStatus(stock, config, daily_sales_avg, weekly_sales_avg, monthly_sales_avg); const stock_status = calculateStockStatus(
stock_quantity,
config,
daily_sales_avg,
weekly_sales_avg,
monthly_sales_avg
);
// Calculate reorder quantity and overstocked amount // Calculate reorder quantities
const { reorder_qty, overstocked_amt } = calculateReorderQuantities( const reorder_quantities = calculateReorderQuantities(
stock, stock_quantity,
stock_status, stock_status,
daily_sales_avg, daily_sales_avg,
purchases.avg_lead_time_days, sanitizeValue(purchaseData.avg_lead_time_days) || 0,
config config
); );
// Add to batch update // Financial metrics
const cost_price = sanitizeValue(stockData.cost_price) || 0;
const inventory_value = stock_quantity * cost_price;
const total_revenue = sanitizeValue(salesData.total_revenue) || 0;
const total_cost = sanitizeValue(salesData.total_cost) || 0;
const gross_profit = total_revenue - total_cost;
const avg_margin_percent = total_revenue > 0 ? ((gross_profit / total_revenue) * 100) : 0;
const gmroi = inventory_value > 0 ? (gross_profit / inventory_value) : 0;
// Add to batch update with sanitized values
metricsUpdates.push([ metricsUpdates.push([
product.product_id, product.pid,
daily_sales_avg || null, sanitizeValue(daily_sales_avg),
weekly_sales_avg || null, sanitizeValue(weekly_sales_avg),
monthly_sales_avg || null, sanitizeValue(monthly_sales_avg),
metrics.avg_quantity_per_order || null, sanitizeValue(salesData.avg_quantity_per_order),
metrics.number_of_orders || 0, sanitizeValue(salesData.number_of_orders),
metrics.first_sale_date || null, salesData.first_sale_date || null,
metrics.last_sale_date || null, salesData.last_sale_date || null,
days_of_inventory, sanitizeValue(days_of_inventory),
weeks_of_inventory, sanitizeValue(weeks_of_inventory),
daily_sales_avg > 0 ? Math.max(1, Math.ceil(daily_sales_avg * config.reorder_days)) : null, sanitizeValue(reorder_quantities.reorder_point),
margin_percent, sanitizeValue(reorder_quantities.safety_stock),
metrics.total_revenue || 0, sanitizeValue(reorder_quantities.reorder_qty),
inventory_value || 0, sanitizeValue(reorder_quantities.overstocked_amt),
purchases.avg_lead_time_days || null, sanitizeValue(avg_margin_percent),
purchases.last_purchase_date || null, sanitizeValue(total_revenue),
purchases.first_received_date || null, sanitizeValue(inventory_value),
purchases.last_received_date || null, sanitizeValue(total_cost),
sanitizeValue(gross_profit),
sanitizeValue(gmroi),
sanitizeValue(purchaseData.avg_lead_time_days),
purchaseData.last_purchase_date || null,
purchaseData.first_received_date || null,
purchaseData.last_received_date || null,
null, // abc_class - calculated separately
stock_status, stock_status,
reorder_qty, sanitizeValue(0), // turnover_rate - calculated separately
overstocked_amt sanitizeValue(purchaseData.avg_lead_time_days),
sanitizeValue(config.target_days),
stock_status === 'Critical' ? 'Warning' : 'Normal',
null, // forecast_accuracy
null, // forecast_bias
null // last_forecast_date
]); ]);
} catch (err) { } catch (err) {
logError(err, `Failed processing product ${product.product_id}`); logError(err, `Failed processing product ${product.pid}`);
continue; continue;
} }
} }
// Batch update metrics // Batch update metrics
if (metricsUpdates.length > 0) { if (metricsUpdates.length > 0) {
try {
await connection.query(` await connection.query(`
INSERT INTO product_metrics ( INSERT INTO product_metrics (
product_id, pid,
daily_sales_avg, daily_sales_avg,
weekly_sales_avg, weekly_sales_avg,
monthly_sales_avg, monthly_sales_avg,
@@ -347,19 +364,31 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
days_of_inventory, days_of_inventory,
weeks_of_inventory, weeks_of_inventory,
reorder_point, reorder_point,
safety_stock,
reorder_qty,
overstocked_amt,
avg_margin_percent, avg_margin_percent,
total_revenue, total_revenue,
inventory_value, inventory_value,
cost_of_goods_sold,
gross_profit,
gmroi,
avg_lead_time_days, avg_lead_time_days,
last_purchase_date, last_purchase_date,
first_received_date, first_received_date,
last_received_date, last_received_date,
abc_class,
stock_status, stock_status,
reorder_qty, turnover_rate,
overstocked_amt current_lead_time,
) VALUES ? target_lead_time,
lead_time_status,
forecast_accuracy,
forecast_bias,
last_forecast_date
)
VALUES ?
ON DUPLICATE KEY UPDATE ON DUPLICATE KEY UPDATE
last_calculated_at = NOW(),
daily_sales_avg = VALUES(daily_sales_avg), daily_sales_avg = VALUES(daily_sales_avg),
weekly_sales_avg = VALUES(weekly_sales_avg), weekly_sales_avg = VALUES(weekly_sales_avg),
monthly_sales_avg = VALUES(monthly_sales_avg), monthly_sales_avg = VALUES(monthly_sales_avg),
@@ -370,129 +399,106 @@ async function calculateProductMetrics(startTime, totalProducts, processedCount
days_of_inventory = VALUES(days_of_inventory), days_of_inventory = VALUES(days_of_inventory),
weeks_of_inventory = VALUES(weeks_of_inventory), weeks_of_inventory = VALUES(weeks_of_inventory),
reorder_point = VALUES(reorder_point), reorder_point = VALUES(reorder_point),
safety_stock = VALUES(safety_stock),
reorder_qty = VALUES(reorder_qty),
overstocked_amt = VALUES(overstocked_amt),
avg_margin_percent = VALUES(avg_margin_percent), avg_margin_percent = VALUES(avg_margin_percent),
total_revenue = VALUES(total_revenue), total_revenue = VALUES(total_revenue),
inventory_value = VALUES(inventory_value), inventory_value = VALUES(inventory_value),
cost_of_goods_sold = VALUES(cost_of_goods_sold),
gross_profit = VALUES(gross_profit),
gmroi = VALUES(gmroi),
avg_lead_time_days = VALUES(avg_lead_time_days), avg_lead_time_days = VALUES(avg_lead_time_days),
last_purchase_date = VALUES(last_purchase_date), last_purchase_date = VALUES(last_purchase_date),
first_received_date = VALUES(first_received_date), first_received_date = VALUES(first_received_date),
last_received_date = VALUES(last_received_date), last_received_date = VALUES(last_received_date),
stock_status = VALUES(stock_status), stock_status = VALUES(stock_status),
reorder_qty = VALUES(reorder_qty), turnover_rate = VALUES(turnover_rate),
overstocked_amt = VALUES(overstocked_amt) current_lead_time = VALUES(current_lead_time),
target_lead_time = VALUES(target_lead_time),
lead_time_status = VALUES(lead_time_status),
last_calculated_at = CURRENT_TIMESTAMP
`, [metricsUpdates]); `, [metricsUpdates]);
} catch (err) {
logError(err, 'Failed to update metrics batch');
throw err;
}
} }
} }
return processedCount; return processedCount;
} finally { } finally {
if (connection) {
connection.release(); connection.release();
} }
}
} }
function calculateStockStatus(stock, config, daily_sales_avg, weekly_sales_avg, monthly_sales_avg) { function calculateStockStatus(stock, config, daily_sales_avg, weekly_sales_avg, monthly_sales_avg) {
const days_since_first_stock = stock.days_since_first_stock || 0; if (stock <= 0) {
const days_since_last_sale = stock.days_since_last_sale || 9999; return 'Out of Stock';
const total_quantity_sold = stock.total_quantity_sold || 0;
const had_recent_stockout = stock.had_recent_stockout || false;
const dq = stock.stock_quantity || 0;
const ds = daily_sales_avg || 0;
const ws = weekly_sales_avg || 0;
const ms = monthly_sales_avg || 0;
// If no stock, return immediately
if (dq === 0) {
return had_recent_stockout ? 'Critical' : 'Out of Stock';
} }
// 1. Check if truly "New" (≤30 days and no sales) // Use the most appropriate sales average based on data quality
if (days_since_first_stock <= 30 && total_quantity_sold === 0) { let sales_avg = daily_sales_avg;
return 'New'; if (sales_avg === 0) {
sales_avg = weekly_sales_avg / 7;
}
if (sales_avg === 0) {
sales_avg = monthly_sales_avg / 30;
} }
// 2. Handle zero or very low sales velocity cases if (sales_avg === 0) {
if (ds === 0 || (ds < 0.1 && ws < 0.5)) { return stock <= config.low_stock_threshold ? 'Low Stock' : 'In Stock';
if (days_since_first_stock > config.overstock_days) {
return 'Overstocked';
}
if (days_since_first_stock > 30) {
return 'At Risk';
}
} }
// 3. Calculate days of supply and check velocity trends const days_of_stock = stock / sales_avg;
const days_of_supply = ds > 0 ? dq / ds : 999;
const velocity_trend = ds > 0 ? (ds / (ms || ds) - 1) * 100 : 0;
// Critical stock level if (days_of_stock <= config.critical_days) {
if (days_of_supply <= config.critical_days) {
return 'Critical'; return 'Critical';
} } else if (days_of_stock <= config.reorder_days) {
// Reorder cases
if (days_of_supply <= config.reorder_days ||
(had_recent_stockout && days_of_supply <= config.reorder_days * 1.5)) {
return 'Reorder'; return 'Reorder';
} } else if (days_of_stock > config.overstock_days) {
// At Risk cases
if (
(days_of_supply >= config.overstock_days * 0.8) ||
(velocity_trend <= -50 && days_of_supply > config.reorder_days * 2) ||
(days_since_last_sale > 45 && dq > 0) ||
(ds > 0 && ds < 0.2 && dq > ds * config.overstock_days * 0.5)
) {
return 'At Risk';
}
// Overstock cases
if (days_of_supply >= config.overstock_days) {
return 'Overstocked'; return 'Overstocked';
} }
// If none of the above conditions are met
return 'Healthy'; return 'Healthy';
} }
function calculateReorderQuantities(stock, stock_status, daily_sales_avg, avg_lead_time, config) { function calculateReorderQuantities(stock, stock_status, daily_sales_avg, avg_lead_time, config) {
// Calculate safety stock based on service level and lead time
const z_score = 1.96; // 95% service level
const lead_time = avg_lead_time || config.target_days;
const safety_stock = Math.ceil(daily_sales_avg * Math.sqrt(lead_time) * z_score);
// Calculate reorder point
const lead_time_demand = daily_sales_avg * lead_time;
const reorder_point = Math.ceil(lead_time_demand + safety_stock);
// Calculate reorder quantity using EOQ formula if we have the necessary data
let reorder_qty = 0; let reorder_qty = 0;
let overstocked_amt = 0; if (daily_sales_avg > 0) {
const annual_demand = daily_sales_avg * 365;
const order_cost = 25; // Fixed cost per order
const holding_cost_percent = 0.25; // 25% annual holding cost
// Only calculate reorder quantity for replenishable products reorder_qty = Math.ceil(Math.sqrt((2 * annual_demand * order_cost) / holding_cost_percent));
if (stock.replenishable && (stock_status === 'Critical' || stock_status === 'Reorder')) { } else {
const ds = daily_sales_avg || 0; // If no sales data, use a basic calculation
const lt = avg_lead_time || 14; reorder_qty = Math.max(safety_stock, config.low_stock_threshold);
const sc = config.safety_stock_days || 14;
const ss = config.safety_stock_days || 14;
const dq = stock.stock_quantity || 0;
const moq = stock.moq || 1;
// Calculate desired stock level
const desired_stock = (ds * (lt + sc)) + ss;
// Calculate raw reorder amount
const raw_reorder = Math.max(0, desired_stock - dq);
// Round up to nearest MOQ
reorder_qty = Math.ceil(raw_reorder / moq) * moq;
} }
// Calculate overstocked amount for overstocked products // Calculate overstocked amount
if (stock_status === 'Overstocked') { const overstocked_amt = stock_status === 'Overstocked' ?
const ds = daily_sales_avg || 0; stock - Math.ceil(daily_sales_avg * config.overstock_days) :
const dq = stock.stock_quantity || 0; 0;
const lt = avg_lead_time || 14;
const sc = config.safety_stock_days || 14;
const ss = config.safety_stock_days || 14;
// Calculate maximum desired stock return {
const max_desired_stock = (ds * config.overstock_days) + ss; safety_stock,
reorder_point,
// Calculate excess inventory reorder_qty,
overstocked_amt = Math.max(0, dq - max_desired_stock); overstocked_amt
} };
return { reorder_qty, overstocked_amt };
} }
module.exports = calculateProductMetrics; module.exports = calculateProductMetrics;

18
inventory-server/scripts/metrics/sales-forecasts.js
View File

@@ -18,7 +18,7 @@ async function calculateSalesForecasts(startTime, totalProducts, processedCount)
// Calculate product-level forecasts // Calculate product-level forecasts
await connection.query(` await connection.query(`
INSERT INTO sales_forecasts ( INSERT INTO sales_forecasts (
product_id, pid,
forecast_date, forecast_date,
forecast_units, forecast_units,
forecast_revenue, forecast_revenue,
@@ -27,14 +27,14 @@ async function calculateSalesForecasts(startTime, totalProducts, processedCount)
) )
WITH daily_sales AS ( WITH daily_sales AS (
SELECT SELECT
o.product_id, o.pid,
DATE(o.date) as sale_date, DATE(o.date) as sale_date,
SUM(o.quantity) as daily_quantity, SUM(o.quantity) as daily_quantity,
SUM(o.price * o.quantity) as daily_revenue SUM(o.price * o.quantity) as daily_revenue
FROM orders o FROM orders o
WHERE o.canceled = false WHERE o.canceled = false
AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY) AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
GROUP BY o.product_id, DATE(o.date) GROUP BY o.pid, DATE(o.date)
), ),
forecast_dates AS ( forecast_dates AS (
SELECT SELECT
@@ -51,7 +51,7 @@ async function calculateSalesForecasts(startTime, totalProducts, processedCount)
), ),
product_stats AS ( product_stats AS (
SELECT SELECT
ds.product_id, ds.pid,
AVG(ds.daily_quantity) as avg_daily_quantity, AVG(ds.daily_quantity) as avg_daily_quantity,
STDDEV_SAMP(ds.daily_quantity) as std_daily_quantity, STDDEV_SAMP(ds.daily_quantity) as std_daily_quantity,
AVG(ds.daily_revenue) as avg_daily_revenue, AVG(ds.daily_revenue) as avg_daily_revenue,
@@ -66,10 +66,10 @@ async function calculateSalesForecasts(startTime, totalProducts, processedCount)
AVG(CASE WHEN DAYOFWEEK(ds.sale_date) = 6 THEN ds.daily_revenue END) as friday_avg, AVG(CASE WHEN DAYOFWEEK(ds.sale_date) = 6 THEN ds.daily_revenue END) as friday_avg,
AVG(CASE WHEN DAYOFWEEK(ds.sale_date) = 7 THEN ds.daily_revenue END) as saturday_avg AVG(CASE WHEN DAYOFWEEK(ds.sale_date) = 7 THEN ds.daily_revenue END) as saturday_avg
FROM daily_sales ds FROM daily_sales ds
GROUP BY ds.product_id GROUP BY ds.pid
) )
SELECT SELECT
ps.product_id, ps.pid,
fd.forecast_date, fd.forecast_date,
GREATEST(0, GREATEST(0,
ps.avg_daily_quantity * ps.avg_daily_quantity *
@@ -130,15 +130,15 @@ async function calculateSalesForecasts(startTime, totalProducts, processedCount)
) )
WITH category_daily_sales AS ( WITH category_daily_sales AS (
SELECT SELECT
pc.category_id, pc.cat_id as category_id,
DATE(o.date) as sale_date, DATE(o.date) as sale_date,
SUM(o.quantity) as daily_quantity, SUM(o.quantity) as daily_quantity,
SUM(o.price * o.quantity) as daily_revenue SUM(o.price * o.quantity) as daily_revenue
FROM orders o FROM orders o
JOIN product_categories pc ON o.product_id = pc.product_id JOIN product_categories pc ON o.pid = pc.pid
WHERE o.canceled = false WHERE o.canceled = false
AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY) AND o.date >= DATE_SUB(CURRENT_DATE, INTERVAL 90 DAY)
GROUP BY pc.category_id, DATE(o.date) GROUP BY pc.cat_id, DATE(o.date)
), ),
forecast_dates AS ( forecast_dates AS (
SELECT SELECT

32
inventory-server/scripts/metrics/time-aggregates.js
View File

@@ -6,7 +6,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
// Initial insert of time-based aggregates // Initial insert of time-based aggregates
await connection.query(` await connection.query(`
INSERT INTO product_time_aggregates ( INSERT INTO product_time_aggregates (
product_id, pid,
year, year,
month, month,
total_quantity_sold, total_quantity_sold,
@@ -20,7 +20,7 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
) )
WITH sales_data AS ( WITH sales_data AS (
SELECT SELECT
o.product_id, o.pid,
YEAR(o.date) as year, YEAR(o.date) as year,
MONTH(o.date) as month, MONTH(o.date) as month,
SUM(o.quantity) as total_quantity_sold, SUM(o.quantity) as total_quantity_sold,
@@ -35,23 +35,23 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
SUM((o.price - COALESCE(o.discount, 0)) * o.quantity)) * 100 SUM((o.price - COALESCE(o.discount, 0)) * o.quantity)) * 100
END as profit_margin END as profit_margin
FROM orders o FROM orders o
JOIN products p ON o.product_id = p.product_id JOIN products p ON o.pid = p.pid
WHERE o.canceled = 0 WHERE o.canceled = 0
GROUP BY o.product_id, YEAR(o.date), MONTH(o.date) GROUP BY o.pid, YEAR(o.date), MONTH(o.date)
), ),
purchase_data AS ( purchase_data AS (
SELECT SELECT
product_id, pid,
YEAR(date) as year, YEAR(date) as year,
MONTH(date) as month, MONTH(date) as month,
SUM(received) as stock_received, SUM(received) as stock_received,
SUM(ordered) as stock_ordered SUM(ordered) as stock_ordered
FROM purchase_orders FROM purchase_orders
WHERE status = 'closed' WHERE receiving_status >= 30 -- Partial or fully received
GROUP BY product_id, YEAR(date), MONTH(date) GROUP BY pid, YEAR(date), MONTH(date)
) )
SELECT SELECT
s.product_id, s.pid,
s.year, s.year,
s.month, s.month,
s.total_quantity_sold, s.total_quantity_sold,
@@ -64,12 +64,12 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
s.profit_margin s.profit_margin
FROM sales_data s FROM sales_data s
LEFT JOIN purchase_data p LEFT JOIN purchase_data p
ON s.product_id = p.product_id ON s.pid = p.pid
AND s.year = p.year AND s.year = p.year
AND s.month = p.month AND s.month = p.month
UNION UNION
SELECT SELECT
p.product_id, p.pid,
p.year, p.year,
p.month, p.month,
0 as total_quantity_sold, 0 as total_quantity_sold,
@@ -82,10 +82,10 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
0 as profit_margin 0 as profit_margin
FROM purchase_data p FROM purchase_data p
LEFT JOIN sales_data s LEFT JOIN sales_data s
ON p.product_id = s.product_id ON p.pid = s.pid
AND p.year = s.year AND p.year = s.year
AND p.month = s.month AND p.month = s.month
WHERE s.product_id IS NULL WHERE s.pid IS NULL
ON DUPLICATE KEY UPDATE ON DUPLICATE KEY UPDATE
total_quantity_sold = VALUES(total_quantity_sold), total_quantity_sold = VALUES(total_quantity_sold),
total_revenue = VALUES(total_revenue), total_revenue = VALUES(total_revenue),
@@ -102,17 +102,17 @@ async function calculateTimeAggregates(startTime, totalProducts, processedCount)
UPDATE product_time_aggregates pta UPDATE product_time_aggregates pta
JOIN ( JOIN (
SELECT SELECT
p.product_id, p.pid,
YEAR(o.date) as year, YEAR(o.date) as year,
MONTH(o.date) as month, MONTH(o.date) as month,
p.cost_price * p.stock_quantity as inventory_value, p.cost_price * p.stock_quantity as inventory_value,
SUM(o.quantity * (o.price - p.cost_price)) as gross_profit, SUM(o.quantity * (o.price - p.cost_price)) as gross_profit,
COUNT(DISTINCT DATE(o.date)) as days_in_period COUNT(DISTINCT DATE(o.date)) as days_in_period
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id LEFT JOIN orders o ON p.pid = o.pid
WHERE o.canceled = false WHERE o.canceled = false
GROUP BY p.product_id, YEAR(o.date), MONTH(o.date) GROUP BY p.pid, YEAR(o.date), MONTH(o.date)
) fin ON pta.product_id = fin.product_id ) fin ON pta.pid = fin.pid
AND pta.year = fin.year AND pta.year = fin.year
AND pta.month = fin.month AND pta.month = fin.month
SET SET

12
inventory-server/scripts/metrics/vendor-metrics.js
View File

@@ -55,14 +55,14 @@ async function calculateVendorMetrics(startTime, totalProducts, processedCount)
ELSE 0 ELSE 0
END as order_fill_rate END as order_fill_rate
FROM purchase_orders po FROM purchase_orders po
WHERE po.status = 'closed' WHERE po.receiving_status >= 30 -- Partial or fully received
GROUP BY po.vendor GROUP BY po.vendor
), ),
vendor_products AS ( vendor_products AS (
SELECT SELECT
p.vendor, p.vendor,
COUNT(DISTINCT p.product_id) as total_products, COUNT(DISTINCT p.pid) as total_products,
COUNT(DISTINCT CASE WHEN p.visible = true THEN p.product_id END) as active_products, COUNT(DISTINCT CASE WHEN p.visible = true THEN p.pid END) as active_products,
SUM(o.price * o.quantity) as total_revenue, SUM(o.price * o.quantity) as total_revenue,
CASE CASE
WHEN SUM(o.price * o.quantity) > 0 THEN WHEN SUM(o.price * o.quantity) > 0 THEN
@@ -70,7 +70,7 @@ async function calculateVendorMetrics(startTime, totalProducts, processedCount)
ELSE 0 ELSE 0
END as avg_margin_percent END as avg_margin_percent
FROM products p FROM products p
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
GROUP BY p.vendor GROUP BY p.vendor
) )
SELECT SELECT
@@ -140,8 +140,8 @@ async function calculateVendorMetrics(startTime, totalProducts, processedCount)
END as avg_margin_percent END as avg_margin_percent
FROM vendor_details vd FROM vendor_details vd
LEFT JOIN products p ON vd.vendor = p.vendor LEFT JOIN products p ON vd.vendor = p.vendor
LEFT JOIN purchase_orders po ON p.product_id = po.product_id LEFT JOIN purchase_orders po ON p.pid = po.pid
LEFT JOIN orders o ON p.product_id = o.product_id AND o.canceled = false LEFT JOIN orders o ON p.pid = o.pid AND o.canceled = false
WHERE po.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH) WHERE po.date >= DATE_SUB(CURRENT_DATE, INTERVAL 12 MONTH)
GROUP BY vd.vendor, YEAR(po.date), MONTH(po.date) GROUP BY vd.vendor, YEAR(po.date), MONTH(po.date)
) )