matt/inventory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add skeleton loading state to template field, remove duplicated or unused code in validate step hooks

Browse Source
This commit is contained in:
Matt
2025-03-19 14:30:39 -04:00
parent 1496aa57b1
commit 7d46ebd6ba
6 changed files with 1195 additions and 1642 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
docs/validation-hook-refactor.md Normal file
View File

@@ -0,0 +1,131 @@
# Refactoring Plan for Validation Code
## Current Structure Analysis
- **useValidationState.tsx**: ~1650 lines - Core validation state management
- **useValidation.tsx**: ~425 lines - Field/data validation utility
- **useUpcValidation.tsx**: ~410 lines - UPC-specific validation
## Proposed New Structure
### 1. Core Types & Utilities (150-200 lines)
**File: `validation/types.ts`**
- All interfaces and types (RowData, ValidationError, FilterState, Template, etc.)
- Shared utility functions (isEmpty, getCellKey, etc.)
**File: `validation/utils.ts`**
- Generic validation utility functions
- Caching mechanism and cache clearing helpers
- API URL helpers
### 2. Field Validation (300-350 lines)
**File: `validation/hooks/useFieldValidation.ts`**
- `validateField` function
- Field-level validation logic
- Required, regex, and other field validations
### 3. Uniqueness Validation (250-300 lines)
**File: `validation/hooks/useUniquenessValidation.ts`**
- `validateUniqueField` function
- `validateUniqueItemNumbers` function
- All uniqueness checking logic
### 4. UPC Validation (300-350 lines)
**File: `validation/hooks/useUpcValidation.ts`**
- `fetchProductByUpc` function
- `validateUpc` function
- `applyItemNumbersToData` function
- UPC validation state management
### 5. Validation Status Management (300-350 lines)
**File: `validation/hooks/useValidationStatus.ts`**
- Error state management
- Row validation status tracking
- Validation indicators and refs
- Batch validation processing
### 6. Data Management (300-350 lines)
**File: `validation/hooks/useValidationData.ts`**
- Data state management
- Row updates
- Data filtering
- Initial data processing
### 7. Template Management (250-300 lines)
**File: `validation/hooks/useTemplateManagement.ts`**
- Template saving
- Template application
- Template loading
- Template display helpers
### 8. Main Validation Hook (300-350 lines)
**File: `validation/hooks/useValidation.ts`**
- Main hook that composes all other hooks
- Public API export
- Initialization logic
- Core validation flow
## Function Distribution
### Core Types & Utilities
- All interfaces (InfoWithSource, ValidationState, etc.)
- `isEmpty` utility
- `getApiUrl` helper
### Field Validation
- `validateField`
- `validateRow`
- `validateData` (partial)
- All validation result caching
### Uniqueness Validation
- `validateUniqueField`
- `validateUniqueItemNumbers`
- Uniqueness caching mechanisms
### UPC Validation
- `fetchProductByUpc`
- `validateUpc`
- `validateAllUPCs`
- `applyItemNumbersToData`
- UPC validation state tracking (cells, rows)
### Validation Status Management
- `startValidatingCell`/`stopValidatingCell`
- `startValidatingRow`/`stopValidatingRow`
- `isValidatingCell`/`isRowValidatingUpc`
- Error state management
- `revalidateRows`
### Data Management
- Initial data cleaning/processing
- `updateRow`
- `copyDown`
- Search/filter functionality
- `filteredData` calculation
### Template Management
- `saveTemplate`
- `applyTemplate`
- `applyTemplateToSelected`
- `getTemplateDisplayText`
- `loadTemplates`/`refreshTemplates`
### Main Validation Hook
- Composition of all other hooks
- Initialization logic
- Button/navigation handling
- Field options management
## Implementation Approach
1. **Start with Types**: Create the types file first, as all other files will depend on it
2. **Create Utility Functions**: Move shared utilities next
3. **Build Core Validation**: Extract the field validation and uniqueness validation
4. **Separate UPC Logic**: Move all UPC-specific code to its own module
5. **Extract State Management**: Move data and status management to separate files
6. **Move Template Logic**: Extract template functionality
7. **Create Composition Hook**: Build the main hook that uses all other hooks
This approach will give you more maintainable code with clearer separation of concerns, making it easier to understand, test, and modify each component independently.

9
inventory/src/components/product-import/steps/ValidationStepNew/components/ValidationTable.tsx
View File

@@ -15,7 +15,7 @@ import { Table, TableBody, TableRow, TableCell } from '@/components/ui/table'
import { Checkbox } from '@/components/ui/checkbox' import { Checkbox } from '@/components/ui/checkbox'
import { cn } from '@/lib/utils' import { cn } from '@/lib/utils'
import { Button } from '@/components/ui/button' import { Button } from '@/components/ui/button'
import { Loader2 } from 'lucide-react' import { Skeleton } from '@/components/ui/skeleton'
// Define a simple Error type locally to avoid import issues // Define a simple Error type locally to avoid import issues
type ErrorType = { type ErrorType = {
@@ -67,10 +67,9 @@ const MemoizedTemplateSelect = React.memo(({
}) => { }) => {
if (isLoading) { if (isLoading) {
return ( return (
<Button variant="outline" className="w-full justify-between overflow-hidden" disabled> <div className="flex items-center justify-center gap-2 border border-input rounded-md px-2 py-2">
<Loader2 className="h-4 w-4 mr-2 animate-spin flex-none" /> <Skeleton className="h-4 w-full" />
<span className="truncate overflow-hidden">Loading...</span> </div>
</Button>
); );
} }

263
inventory/src/components/product-import/steps/ValidationStepNew/hooks/useTemplates.tsx
View File

@@ -1,263 +0,0 @@
import { useState, useEffect, useCallback } from 'react'
import { RowSelectionState } from '@tanstack/react-table'
import { Template, RowData, getApiUrl } from './useValidationState'
interface TemplateState {
selectedTemplateId: string | null
showSaveTemplateDialog: boolean
newTemplateName: string
newTemplateType: string
}
export const useTemplates = <T extends string>(
data: RowData<T>[],
setData: React.Dispatch<React.SetStateAction<RowData<T>[]>>,
toast: any,
rowSelection: RowSelectionState
) => {
const [templates, setTemplates] = useState<Template[]>([])
const [templateState, setTemplateState] = useState<TemplateState>({
selectedTemplateId: null,
showSaveTemplateDialog: false,
newTemplateName: '',
newTemplateType: '',
})
// Load templates from API
const loadTemplates = useCallback(async () => {
try {
console.log('Fetching templates...');
const response = await fetch(`${getApiUrl()}/templates`)
console.log('Templates response status:', response.status);
if (!response.ok) throw new Error('Failed to fetch templates')
const templateData = await response.json()
console.log('Templates fetched successfully:', templateData);
// Validate template data
const validTemplates = templateData.filter((t: any) =>
t && typeof t === 'object' && t.id && t.company && t.product_type
);
if (validTemplates.length !== templateData.length) {
console.warn('Some templates were filtered out due to invalid data', {
original: templateData.length,
valid: validTemplates.length
});
}
setTemplates(validTemplates)
} catch (error) {
console.error('Error loading templates:', error)
toast({
title: 'Error',
description: 'Failed to load templates',
})
}
}, [toast])
// Save a new template based on selected rows
const saveTemplate = useCallback(async (name: string, type: string) => {
try {
// Get selected rows
const selectedRows = Object.keys(rowSelection)
.map(index => data[parseInt(index)])
.filter(Boolean)
if (selectedRows.length === 0) {
toast({
title: 'Error',
description: 'Please select at least one row to create a template',
})
return
}
// Create template based on selected rows
const template: Template = {
id: Date.now(), // Temporary ID, will be replaced by server
company: selectedRows[0].company as string || '',
product_type: type,
...selectedRows[0], // Copy all fields from the first selected row
}
// Remove metadata fields
delete (template as any).__meta
delete (template as any).__template
delete (template as any).__original
delete (template as any).__corrected
delete (template as any).__changes
// Send to API
const response = await fetch(`${getApiUrl()}/templates`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
company: template.company,
product_type: type,
...Object.fromEntries(
Object.entries(template).filter(([key]) =>
!['company', 'product_type'].includes(key)
)
)
}),
})
if (!response.ok) {
throw new Error('Failed to save template')
}
// Reload templates to get the server-generated ID
await loadTemplates()
toast({
title: 'Success',
description: `Template "${name}" saved successfully`,
})
// Reset dialog state
setTemplateState(prev => ({
...prev,
showSaveTemplateDialog: false,
newTemplateName: '',
newTemplateType: '',
}))
} catch (error) {
console.error('Error saving template:', error)
toast({
title: 'Error',
description: 'Failed to save template',
})
}
}, [data, rowSelection, toast, loadTemplates])
// Apply a template to selected rows
const applyTemplate = useCallback((templateId: string, rowIndexes: number[]) => {
const template = templates.find(t => t.id.toString() === templateId)
if (!template) {
toast({
title: 'Error',
description: 'Template not found',
})
return
}
setData(prevData => {
const newData = [...prevData]
rowIndexes.forEach(index => {
if (index >= 0 && index < newData.length) {
// Create a new row with template values
const updatedRow = { ...newData[index] }
// Apply template fields (excluding metadata and ID fields)
Object.entries(template).forEach(([key, value]) => {
if (!['id', 'company', 'product_type', 'created_at', 'updated_at'].includes(key)) {
// Handle numeric values that might be stored as strings
if (typeof value === 'string' && /^\d+(\.\d+)?$/.test(value)) {
// If it's a price field, add the dollar sign
if (['msrp', 'cost_each'].includes(key)) {
updatedRow[key as keyof typeof updatedRow] = `$${value}` as any;
} else {
updatedRow[key as keyof typeof updatedRow] = value as any;
}
}
// Special handling for array fields like categories and ship_restrictions
else if (key === 'categories' || key === 'ship_restrictions') {
if (Array.isArray(value)) {
updatedRow[key as keyof typeof updatedRow] = value as any;
} else if (typeof value === 'string') {
try {
// Try to parse as JSON if it's a JSON string
if (value.startsWith('[') && value.endsWith(']')) {
const parsed = JSON.parse(value);
updatedRow[key as keyof typeof updatedRow] = parsed as any;
}
// Otherwise, it might be a PostgreSQL array format like {val1,val2}
else if (value.startsWith('{') && value.endsWith('}')) {
const parsed = value.slice(1, -1).split(',');
updatedRow[key as keyof typeof updatedRow] = parsed as any;
}
// If it's a single value, wrap it in an array
else {
updatedRow[key as keyof typeof updatedRow] = [value] as any;
}
} catch (error) {
console.error(`Error parsing ${key}:`, error);
// If parsing fails, use as-is
updatedRow[key as keyof typeof updatedRow] = value as any;
}
} else {
updatedRow[key as keyof typeof updatedRow] = value as any;
}
} else {
updatedRow[key as keyof typeof updatedRow] = value as any;
}
}
})
// Mark the row as using this template
updatedRow.__template = templateId
// Update the row in the data array
newData[index] = updatedRow
}
})
return newData
})
toast({
title: 'Success',
description: `Template applied to ${rowIndexes.length} row(s)`,
})
}, [templates, setData, toast])
// Get display text for a template
const getTemplateDisplayText = useCallback((templateId: string | null) => {
if (!templateId) return 'Select a template'
const template = templates.find(t => t.id.toString() === templateId)
return template
? `${template.company} - ${template.product_type}`
: 'Unknown template'
}, [templates])
// Load templates on component mount and set up refresh event listener
useEffect(() => {
loadTemplates()
// Add event listener for template refresh
const handleRefreshTemplates = () => {
loadTemplates()
}
window.addEventListener('refresh-templates', handleRefreshTemplates)
// Clean up event listener
return () => {
window.removeEventListener('refresh-templates', handleRefreshTemplates)
}
}, [loadTemplates])
return {
templates,
selectedTemplateId: templateState.selectedTemplateId,
showSaveTemplateDialog: templateState.showSaveTemplateDialog,
newTemplateName: templateState.newTemplateName,
newTemplateType: templateState.newTemplateType,
setTemplateState,
loadTemplates,
saveTemplate,
applyTemplate,
getTemplateDisplayText,
// Helper method to apply to selected rows
applyTemplateToSelected: (templateId: string) => {
const selectedIndexes = Object.keys(rowSelection).map(i => parseInt(i))
applyTemplate(templateId, selectedIndexes)
}
}
}

106
inventory/src/components/product-import/steps/ValidationStepNew/hooks/useUpcValidation.tsx
View File

@@ -30,13 +30,6 @@ export const useUpcValidation = (
const processedUpcMapRef = useRef(new Map<string, string>()); const processedUpcMapRef = useRef(new Map<string, string>());
const initialUpcValidationDoneRef = useRef(false); const initialUpcValidationDoneRef = useRef(false);
// For batch validation
const validationQueueRef = useRef<Array<{rowIndex: number, supplierId: string, upcValue: string}>>([]);
const isProcessingBatchRef = useRef(false);
// For validation results
const [upcValidationResults] = useState<Map<number, { itemNumber: string }>>(new Map());
// Helper to create cell key // Helper to create cell key
const getCellKey = (rowIndex: number, fieldKey: string) => `${rowIndex}-${fieldKey}`; const getCellKey = (rowIndex: number, fieldKey: string) => `${rowIndex}-${fieldKey}`;
@@ -249,102 +242,6 @@ export const useUpcValidation = (
} }
}, [setData]); }, [setData]);
// Process validation queue in batches - faster processing with smaller batches
const processBatchValidation = useCallback(async () => {
if (isProcessingBatchRef.current) return;
if (validationQueueRef.current.length === 0) return;
console.log(`Processing validation batch with ${validationQueueRef.current.length} items`);
isProcessingBatchRef.current = true;
// Process in smaller batches for better UI responsiveness
const BATCH_SIZE = 5;
const queue = [...validationQueueRef.current];
validationQueueRef.current = [];
// Track if any updates were made
let updatesApplied = false;
// Track updated row indices
const updatedRows: number[] = [];
try {
// Process in small batches
for (let i = 0; i < queue.length; i += BATCH_SIZE) {
const batch = queue.slice(i, i + BATCH_SIZE);
// Process batch in parallel
const results = await Promise.all(batch.map(async ({ rowIndex, supplierId, upcValue }) => {
try {
// Skip if already validated
const cacheKey = `${supplierId}-${upcValue}`;
if (processedUpcMapRef.current.has(cacheKey)) {
const cachedItemNumber = processedUpcMapRef.current.get(cacheKey);
if (cachedItemNumber) {
console.log(`Using cached item number for row ${rowIndex}: ${cachedItemNumber}`);
updateItemNumber(rowIndex, cachedItemNumber);
updatesApplied = true;
updatedRows.push(rowIndex);
return true;
}
return false;
}
// Fetch from API
const result = await fetchProductByUpc(supplierId, upcValue);
if (!result.error && result.data?.itemNumber) {
const itemNumber = result.data.itemNumber;
// Store in cache
processedUpcMapRef.current.set(cacheKey, itemNumber);
// Update item number
updateItemNumber(rowIndex, itemNumber);
updatesApplied = true;
updatedRows.push(rowIndex);
console.log(`Set item number for row ${rowIndex} to ${itemNumber}`);
return true;
}
return false;
} catch (error) {
console.error(`Error processing row ${rowIndex}:`, error);
return false;
} finally {
// Clear validation state
stopValidatingRow(rowIndex);
}
}));
// If any updates were applied in this batch, update the data
if (results.some(Boolean) && updatesApplied) {
applyItemNumbersToData(updatedRowIds => {
console.log(`Processed batch UPC validation for rows: ${updatedRowIds.join(', ')}`);
});
updatesApplied = false;
updatedRows.length = 0; // Clear the array
}
// Small delay between batches to allow UI to update
if (i + BATCH_SIZE < queue.length) {
await new Promise(resolve => setTimeout(resolve, 10));
}
}
} catch (error) {
console.error('Error in batch processing:', error);
} finally {
isProcessingBatchRef.current = false;
// Process any new items
if (validationQueueRef.current.length > 0) {
setTimeout(processBatchValidation, 0);
}
}
}, [fetchProductByUpc, updateItemNumber, stopValidatingRow, applyItemNumbersToData]);
// For immediate processing
// Batch validate all UPCs in the data // Batch validate all UPCs in the data
const validateAllUPCs = useCallback(async () => { const validateAllUPCs = useCallback(async () => {
// Skip if we've already done the initial validation // Skip if we've already done the initial validation
@@ -508,9 +405,6 @@ export const useUpcValidation = (
getItemNumber, getItemNumber,
applyItemNumbersToData, applyItemNumbersToData,
// Results
upcValidationResults,
// Initialization state // Initialization state
initialValidationDone: initialUpcValidationDoneRef.current initialValidationDone: initialUpcValidationDoneRef.current
}; };

102
inventory/src/components/product-import/steps/ValidationStepNew/hooks/useValidation.tsx
View File

@@ -23,13 +23,15 @@ const isEmpty = (value: any): boolean =>
// Create a cache for validation results to avoid repeated validation of the same data // Create a cache for validation results to avoid repeated validation of the same data
const validationResultCache = new Map(); const validationResultCache = new Map();
const validationCache: Record<string, any> = {};
// Add a function to clear cache for a specific field value // Add a function to clear cache for a specific field value
export const clearValidationCacheForField = (fieldKey: string) => { export const clearValidationCacheForField = (fieldKey: string) => {
// Clear cache // Look for entries that match this field key
const cacheKey = `field_${fieldKey}`; validationResultCache.forEach((_, key) => {
delete validationCache[cacheKey]; if (key.startsWith(`${fieldKey}-`)) {
validationResultCache.delete(key);
}
});
}; };
// Add a special function to clear all uniqueness validation caches // Add a special function to clear all uniqueness validation caches
@@ -170,96 +172,6 @@ export const useValidation = <T extends string>(
} }
}, [fields, validateField, rowHook]) }, [fields, validateField, rowHook])
// Validate all data at the table level
const validateTable = useCallback(async (data: RowData<T>[]): Promise<Meta[]> => {
if (!tableHook) {
return data.map((row, index) => ({
__index: row.__index || String(index)
}))
}
try {
const tableResults = await tableHook(data)
// Process table validation results
return tableResults.map((result, index) => {
return {
__index: result.__index || data[index].__index || String(index)
}
})
} catch (error) {
console.error('Error in table hook:', error)
return data.map((row, index) => ({
__index: row.__index || String(index)
}))
}
}, [tableHook])
// Validate unique fields across the table
const validateUnique = useCallback((data: RowData<T>[]) => {
// Create a map to store errors for each row
const uniqueErrors = new Map<number, Record<string, InfoWithSource>>();
// Find fields with unique validation
const uniqueFields = fields.filter(field =>
field.validations?.some(v => v.rule === 'unique')
);
if (uniqueFields.length === 0) {
return uniqueErrors;
}
// Check each unique field
uniqueFields.forEach(field => {
const { key } = field;
const validation = field.validations?.find(v => v.rule === 'unique');
const allowEmpty = validation?.allowEmpty ?? false;
const errorMessage = validation?.errorMessage || `${field.label} must be unique`;
const level = validation?.level || 'error';
// Track values for uniqueness check
const valueMap = new Map<string, number[]>();
// Build value map
data.forEach((row, rowIndex) => {
const value = String(row[String(key) as keyof typeof row] || '');
// Skip empty values if allowed
if (allowEmpty && isEmpty(value)) {
return;
}
if (!valueMap.has(value)) {
valueMap.set(value, [rowIndex]);
} else {
valueMap.get(value)?.push(rowIndex);
}
});
// Add errors for duplicate values
valueMap.forEach((rowIndexes) => {
if (rowIndexes.length > 1) {
// Add error to all duplicate rows
rowIndexes.forEach(rowIndex => {
// Get existing errors for this row or create a new object
const rowErrors = uniqueErrors.get(rowIndex) || {};
rowErrors[String(key)] = {
message: errorMessage,
level,
source: ErrorSources.Table,
type: ErrorType.Unique
};
uniqueErrors.set(rowIndex, rowErrors);
});
}
});
});
return uniqueErrors;
}, [fields]);
// Additional function to explicitly validate uniqueness for specified fields // Additional function to explicitly validate uniqueness for specified fields
const validateUniqueField = useCallback((data: RowData<T>[], fieldKey: string) => { const validateUniqueField = useCallback((data: RowData<T>[], fieldKey: string) => {
// Field keys that need special handling for uniqueness // Field keys that need special handling for uniqueness
@@ -506,8 +418,6 @@ export const useValidation = <T extends string>(
validateData, validateData,
validateField, validateField,
validateRow, validateRow,
validateTable,
validateUnique,
validateUniqueField, validateUniqueField,
clearValidationCacheForField, clearValidationCacheForField,
clearAllUniquenessCaches clearAllUniquenessCaches

1900
inventory/src/components/product-import/steps/ValidationStepNew/hooks/useValidationState.tsx
View File

File diff suppressed because it is too large Load Diff