Add standardized error handling with new enums and interfaces for validation errors

docs/validation-process-issues.md

@@ -0,0 +1,236 @@
+## 1. ✅ Error Filtering Logic Inconsistency (RESOLVED)
+
+> **Note: This issue has been resolved by implementing a type-based error system.**
+
+The filtering logic in `ValidationCell.tsx` previously relied on string matching, which was fragile:
+
+```typescript
+// Old implementation (string-based matching)
+const filteredErrors = React.useMemo(() => {
+  return !isEmpty(value) 
+    ? errors.filter(error => !error.message?.toLowerCase().includes('required'))
+    : errors;
+}, [value, errors]);
+
+// New implementation (type-based filtering)
+const filteredErrors = React.useMemo(() => {
+  return !isEmpty(value) 
+    ? errors.filter(error => error.type !== ErrorType.Required)
+    : errors;
+}, [value, errors]);
+```
+
+The solution implemented:
+- Added an `ErrorType` enum in `types.ts` to standardize error categorization
+- Updated all error creation to include the appropriate error type
+- Modified error filtering to use the type property instead of string matching
+- Ensured consistent error handling across the application
+
+**Guidelines for future development:**
+- Always use the `ErrorType` enum when creating errors
+- Never rely on string matching for error filtering
+- Ensure all error objects include the `type` property
+- Use the appropriate error type for each validation rule:
+  - `ErrorType.Required` for required field validations
+  - `ErrorType.Regex` for regex validations
+  - `ErrorType.Unique` for uniqueness validations
+  - `ErrorType.Custom` for custom validations
+  - `ErrorType.Api` for API-based validations
+
+## 2. Redundant Error Processing
+
+The system processes errors in multiple places:
+- In `ValidationCell.tsx`, errors are filtered and processed again
+- In `useValidation.tsx`, errors are already filtered once
+- In `ValidationContainer.tsx`, errors are manipulated directly
+
+This redundancy could lead to inconsistent behavior and makes the code harder to maintain.
+
+## 3. Race Conditions in Async Validation
+
+The UPC validation and other async validations could create race conditions:
+- If a user types quickly, multiple validation requests might be in flight
+- Later responses could overwrite more recent ones if they complete out of order
+- The debouncing helps but doesn't fully solve this issue
+
+## 4. Memory Leaks in Timeout Management
+
+The validation timeouts are stored in refs:
+```typescript
+const validationTimeoutsRef = useRef<Record<number, NodeJS.Timeout>>({});
+```
+
+While there is cleanup on unmount, if rows are added/removed dynamically, timeouts for deleted rows might not be properly cleared.
+
+## 5. Inefficient Error Storage
+
+Errors are stored in multiple places:
+- In the `validationErrors` Map
+- In the row data itself as `__errors`
+- In the UPC validation results
+
+This duplication makes it harder to maintain a single source of truth and could lead to inconsistencies.
+
+## 6. Excessive Re-rendering
+
+Despite optimization attempts, the system might still cause excessive re-renders:
+- Each cell validation can trigger updates to the entire data structure
+- The batch update system helps but still has limitations
+- The memoization in `ValidationCell` might not catch all cases where re-rendering is unnecessary
+
+## 7. Complex Error Merging Logic
+
+When merging errors from different sources, the logic is complex and potentially error-prone:
+```typescript
+// Merge field errors and row hook errors
+const mergedErrors: Record<string, InfoWithSource> = {}
+
+// Convert field errors to InfoWithSource
+Object.entries(fieldErrors).forEach(([key, errors]) => {
+  if (errors.length > 0) {
+    mergedErrors[key] = {
+      message: errors[0].message,
+      level: errors[0].level,
+      source: ErrorSources.Row,
+      type: errors[0].type || ErrorType.Custom
+    }
+  }
+})
+```
+
+This only takes the first error for each field, potentially hiding important validation issues.
+
+## 8. ✅ Inconsistent Error Handling for Empty Values (PARTIALLY RESOLVED)
+
+> **Note: This issue has been partially resolved by standardizing the isEmpty function and error type system.**
+
+The system previously had different approaches to handling empty values:
+- Some validations skipped empty values unless they're required
+- Others processed empty values differently
+- The `isEmpty` function was defined multiple times with slight variations
+
+The solution implemented:
+- Standardized the `isEmpty` function implementation
+- Ensured consistent error type usage for required field validations
+- Made error filtering consistent across the application
+
+**Guidelines for future development:**
+- Always use the shared `isEmpty` function for checking empty values
+- Ensure consistent handling of empty values across all validation rules
+- Use the `ErrorType.Required` type for all required field validations
+
+## 9. Tight Coupling Between Components
+
+The validation system is tightly coupled across components:
+- `ValidationCell` needs to understand the structure of errors
+- `ValidationTable` needs to extract and pass the right errors
+- `ValidationContainer` directly manipulates the error structure
+
+This makes it harder to refactor or reuse components independently.
+
+## 10. Limited Error Prioritization
+
+There's no clear prioritization of errors:
+- When multiple errors exist for a field, which one should be shown first?
+- Are some errors more important than others?
+- The current system mostly shows the first error it finds
+
+A more robust approach would be to have a consistent error source identification system and a clear prioritization strategy for displaying errors.
+
+------------
+
+Let me explain how these hooks fit together to create the validation errors that eventually get filtered in the `ValidationCell` component:
+
+## The Validation Flow
+
+1. **useValidationState Hook**:
+   This is the main state management hook used by the `ValidationContainer` component. It:
+   - Manages the core data state (`data`)
+   - Tracks validation errors in a Map (`validationErrors`)
+   - Provides functions to update and validate rows
+
+2. **useValidation Hook**:
+   This is a utility hook that provides the core validation logic:
+   - `validateField`: Validates a single field against its validation rules
+   - `validateRow`: Validates an entire row, field by field
+   - `validateTable`: Runs table-level validations
+   - `validateUnique`: Checks for uniqueness constraints
+   - `validateData`: Orchestrates the complete validation process
+
+## How Errors Are Generated
+
+Validation errors come from multiple sources:
+
+1. **Field-Level Validations**:
+   In `useValidation.tsx`, the `validateField` function checks individual fields against rules like:
+   - `required`: Field must have a value
+   - `regex`: Value must match a pattern
+   - `min`/`max`: Numeric constraints
+
+2. **Row-Level Validations**:
+   The `validateRow` function in `useValidation.tsx` runs:
+   - Field validations for each field in the row
+   - Special validations for required fields like supplier and company
+   - Custom row hooks provided by the application
+
+3. **Table-Level Validations**:
+   - `validateUnique` checks for duplicate values in fields marked as unique
+   - `validateTable` runs custom table hooks for cross-row validations
+
+4. **API-Based Validations**:
+   In `useValidationState.tsx` and `ValidationContainer.tsx`:
+   - UPC validation via API calls
+   - Item number uniqueness checks
+
+## The Error Flow
+
+1. Errors are collected in the `validationErrors` Map in `useValidationState`
+2. This Map is passed to `ValidationTable` as a prop
+3. `ValidationTable` extracts the relevant errors for each cell and passes them to `ValidationCell`
+4. In `ValidationCell`, the errors are filtered based on whether the cell has a value:
+   ```typescript
+   // Updated implementation using type-based filtering
+   const filteredErrors = React.useMemo(() => {
+     return !isEmpty(value) 
+       ? errors.filter(error => error.type !== ErrorType.Required)
+       : errors;
+   }, [value, errors]);
+   ```
+
+## Key Insights
+
+1. **Error Structure**:
+   Errors now have a consistent structure with type information:
+   ```typescript
+   type ErrorObject = {
+     message: string;
+     level: string;  // 'error', 'warning', etc.
+     source?: ErrorSources; // Where the error came from
+     type: ErrorType; // The type of error (Required, Regex, Unique, etc.)
+   }
+   ```
+
+2. **Error Sources**:
+   Errors can come from:
+   - Field validations (required, regex, etc.)
+   - Row validations (custom business logic)
+   - Table validations (uniqueness checks)
+   - API validations (UPC checks)
+
+3. **Error Types**:
+   Errors are now categorized by type:
+   - `ErrorType.Required`: Field is required but empty
+   - `ErrorType.Regex`: Value doesn't match the regex pattern
+   - `ErrorType.Unique`: Value must be unique across rows
+   - `ErrorType.Custom`: Custom validation errors
+   - `ErrorType.Api`: Errors from API calls
+
+4. **Error Filtering**:
+   The filtering in `ValidationCell` is now more robust:
+   - When a field has a value, errors of type `ErrorType.Required` are filtered out
+   - When a field is empty, all errors are shown
+
+5. **Performance Optimizations**:
+   - Batch processing of validations
+   - Debounced updates to avoid excessive re-renders
+   - Memoization of computed values

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/ValidationCell.tsx

@@ -1,5 +1,5 @@
 import React from 'react'
-import { Field } from '../../../types'
+import { Field, ErrorType } from '../../../types'
 import { Loader2, AlertCircle, ArrowDown, Check, X } from 'lucide-react'
 import {
   Tooltip,
@@ -40,6 +40,7 @@ type ErrorObject = {
   message: string;
   level: string;
   source?: string;
+  type?: ErrorType;
 }
 
 // Helper function to check if a value is empty - utility function shared by all components
@@ -195,12 +196,8 @@ function processErrors(value: any, errors: ErrorObject[]): {
     };
   }
   
-  // Check if value is empty - using local function for speed
+  // Use the shared isEmpty function instead of defining a local one
-  const valueIsEmpty = value === undefined || 
+  const valueIsEmpty = isEmpty(value);
-    value === null || 
-    value === '' || 
-    (Array.isArray(value) && value.length === 0) ||
-    (typeof value === 'object' && !Array.isArray(value) && Object.keys(value).length === 0);
   
   // If not empty, filter out required errors
   // Create a new array only if we need to filter (avoid unnecessary allocations)
@@ -210,13 +207,13 @@ function processErrors(value: any, errors: ErrorObject[]): {
   if (valueIsEmpty) {
     // For empty values, check if there are required errors
     hasRequiredError = errors.some(error => 
-      error.message?.toLowerCase().includes('required')
+      error.type === ErrorType.Required
     );
     filteredErrors = errors;
   } else {
     // For non-empty values, filter out required errors
     filteredErrors = errors.filter(error => 
-      !error.message?.toLowerCase().includes('required')
+      error.type !== ErrorType.Required
     );
   }
   
@@ -351,7 +348,8 @@ const ItemNumberCell = React.memo(({
           <div className="absolute right-1.5 top-1/2 -translate-y-1/2 z-20">
             <ValidationIcon error={{ 
               message: errorMessages, 
-              level: 'error' 
+              level: 'error',
+              type: ErrorType.Custom
             }} />
           </div>
         )}
@@ -474,7 +472,7 @@ const ValidationCell = ({
   // Memoize filtered errors to avoid recalculation on every render
   const filteredErrors = React.useMemo(() => {
     return !isEmpty(value) 
-      ? errors.filter(error => !error.message?.toLowerCase().includes('required'))
+      ? errors.filter(error => error.type !== ErrorType.Required)
       : errors;
   }, [value, errors]);
   
@@ -485,7 +483,7 @@ const ValidationCell = ({
     
     // Determine if the field is required but empty
     const isRequiredButEmpty = isEmpty(value) && 
-      errors.some(error => error.message?.toLowerCase().includes('required'));
+      errors.some(error => error.type === ErrorType.Required);
     
     // Only show error icons for non-empty fields with actual errors (not just required errors)
     const shouldShowErrorIcon = hasError && !isEmpty(value);
@@ -555,7 +553,8 @@ const ValidationCell = ({
           <div className="absolute right-1.5 top-1/2 -translate-y-1/2 z-20">
             <ValidationIcon error={{ 
               message: errorMessages, 
-              level: 'error' 
+              level: 'error',
+              type: ErrorType.Custom
             }} />
           </div>
         )}

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/ValidationContainer.tsx

@@ -10,7 +10,7 @@ import SearchableTemplateSelect from './SearchableTemplateSelect'
 import { useAiValidation } from '../hooks/useAiValidation'
 import { AiValidationDialogs } from './AiValidationDialogs'
 import config from '@/config'
-import { Fields } from '../../../types'
+import { Fields, ErrorSources, ErrorType } from '../../../types'
 import { SearchProductTemplateDialog } from '@/components/templates/SearchProductTemplateDialog'
 import { TemplateForm } from '@/components/templates/TemplateForm'
 import axios from 'axios'
@@ -229,7 +229,9 @@ const ValidationContainer = <T extends string>({
                 ...(rowToUpdate.__errors || {}),
                 [fieldKey]: {
                   level: 'error',
-                  message: `UPC already exists (${errorData.existingItemNumber})`
+                  message: `UPC already exists (${errorData.existingItemNumber})`,
+                  source: ErrorSources.Upc,
+                  type: ErrorType.Unique
                 }
               }
             };

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/hooks/useAiValidation.tsx

@@ -1,7 +1,7 @@
 import { useState, useCallback } from 'react';
 import { toast } from 'sonner';
 import { getApiUrl, RowData } from './useValidationState';
-import { Fields, InfoWithSource, ErrorSources } from '../../../types';
+import { Fields, InfoWithSource, ErrorSources, ErrorType } from '../../../types';
 import { Meta } from '../types';
 import { addErrorsAndRunHooks } from '../utils/dataMutations';
 import * as Diff from 'diff';
@@ -99,7 +99,8 @@ export const useAiValidation = <T extends string>(
             return [key, {
               message: errorArray[0].message,
               level: errorArray[0].level,
-              source: ErrorSources.Row
+              source: ErrorSources.Row,
+              type: ErrorType.Custom
             } as InfoWithSource]
           })
         ) : null
@@ -120,7 +121,8 @@ export const useAiValidation = <T extends string>(
             return [key, {
               message: errorArray[0].message,
               level: errorArray[0].level,
-              source: ErrorSources.Table
+              source: ErrorSources.Table,
+              type: ErrorType.Custom
             } as InfoWithSource]
           })
         ) : null

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/hooks/useValidation.tsx

@@ -1,18 +1,16 @@
 import { useCallback } from 'react'
 import type { Field, Fields, RowHook, TableHook } from '../../../types'
 import type { Meta } from '../types'
-import { ErrorSources } from '../../../types'
+import { ErrorSources, ErrorType, ValidationError } from '../../../types'
 import { RowData } from './useValidationState'
 
-interface ValidationError {
+// Define InfoWithSource to match the expected structure
-  message: string
+// Make sure source is required (not optional)
-  level: 'info' | 'warning' | 'error'
-}
-
 interface InfoWithSource {
-  message: string
+  message: string;
-  level: 'info' | 'warning' | 'error'
+  level: 'info' | 'warning' | 'error';
-  source: ErrorSources
+  source: ErrorSources;
+  type: ErrorType;
 }
 
 // Shared utility function for checking empty values - defined once to avoid duplication
@@ -44,7 +42,8 @@ export const useValidation = <T extends string>(
           if (isEmpty(value)) {
             errors.push({
               message: validation.errorMessage || 'This field is required',
-              level: validation.level || 'error'
+              level: validation.level || 'error',
+              type: ErrorType.Required
             })
           }
           break
@@ -60,7 +59,8 @@ export const useValidation = <T extends string>(
               if (!regex.test(String(value))) {
                 errors.push({
                   message: validation.errorMessage,
-                  level: validation.level || 'error'
+                  level: validation.level || 'error',
+                  type: ErrorType.Regex
                 })
               }
             } catch (error) {
@@ -98,14 +98,16 @@ export const useValidation = <T extends string>(
     if (fields.some(field => String(field.key) === 'supplier') && isEmpty(row.supplier)) {
       fieldErrors['supplier'] = [{
         message: 'Supplier is required',
-        level: 'error'
+        level: 'error',
+        type: ErrorType.Required
       }]
     }
     
     if (fields.some(field => String(field.key) === 'company') && isEmpty(row.company)) {
       fieldErrors['company'] = [{
         message: 'Company is required',
-        level: 'error'
+        level: 'error',
+        type: ErrorType.Required
       }]
     }
     
@@ -131,7 +133,8 @@ export const useValidation = <T extends string>(
         mergedErrors[key] = {
           message: errors[0].message,
           level: errors[0].level,
-          source: ErrorSources.Row
+          source: ErrorSources.Row,
+          type: errors[0].type || ErrorType.Custom
         }
       }
     })
@@ -140,7 +143,12 @@ export const useValidation = <T extends string>(
     if (rowHookResult.__errors) {
       Object.entries(rowHookResult.__errors).forEach(([key, error]) => {
         if (error) {
-          mergedErrors[key] = error
+          // Add type if not present
+          const errorWithType = {
+            ...error,
+            type: error.type || ErrorType.Custom
+          }
+          mergedErrors[key] = errorWithType as InfoWithSource
         }
       })
     }
@@ -173,8 +181,9 @@ export const useValidation = <T extends string>(
             if (error) {
               formattedErrors[key] = {
                 ...error,
-                source: ErrorSources.Table
+                source: ErrorSources.Table,
-              }
+                type: error.type || ErrorType.Custom
+              } as InfoWithSource
             }
           })
         }
@@ -246,7 +255,8 @@ export const useValidation = <T extends string>(
             rowErrors[String(key)] = { 
               message: errorMessage, 
               level,
-              source: ErrorSources.Table
+              source: ErrorSources.Table,
+              type: ErrorType.Unique
             }
             
             uniqueErrors[rowIndex].__errors = rowErrors
@@ -295,12 +305,12 @@ export const useValidation = <T extends string>(
       Object.entries(combinedErrors).forEach(([key, error]) => {
         const fieldValue = row[key as keyof typeof row]
         
-        // If the field has a value and the only error is "required", skip it
+        // If the field has a value and the error is of type Required, skip it
         if (!isEmpty(fieldValue) && 
             error && 
             typeof error === 'object' && 
-            'message' in error && 
+            'type' in error && 
-            error.message?.toLowerCase().includes('required')) {
+            error.type === ErrorType.Required) {
           return
         }
         

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/hooks/useValidationState.tsx

@@ -1,18 +1,26 @@
 import { useState, useEffect, useCallback, useMemo, useRef } from 'react'
 import { useRsi } from '../../../hooks/useRsi'
 import type { Data, Field } from '../../../types'
+import { ErrorSources, ErrorType, ValidationError } from '../../../types'
 import { RowSelectionState } from '@tanstack/react-table'
 import { toast } from 'sonner'
 import { useQuery } from "@tanstack/react-query";
 import config from "@/config";
 
-// Define ErrorType directly in file to avoid import issues
+// Helper function to check if a value is empty
-type ErrorType = {
+const isEmpty = (val: any): boolean => 
-  message: string;
+  val === undefined || 
-  level: string;
+  val === null || 
-  source?: string;
+  val === '' || 
-}
+  (Array.isArray(val) && val.length === 0) ||
+  (typeof val === 'object' && !Array.isArray(val) && Object.keys(val).length === 0);
 
+// Use the ValidationError type from types.ts instead of defining ErrorType here
+// type ErrorType = {
+//   message: string;
+//   level: string;
+//   source?: string;
+// }
 
 // Define the Props interface for ValidationStepNew
 export interface Props<T extends string> {
@@ -26,7 +34,7 @@ export interface Props<T extends string> {
 // Extended Data type with meta information
 export type RowData<T extends string> = Data<T> & {
   __index?: string;
-  __errors?: Record<string, ErrorType[] | ErrorType>;
+  __errors?: Record<string, ValidationError[] | ValidationError>;
   __template?: string;
   __original?: Record<string, any>;
   __corrected?: Record<string, any>;
@@ -55,6 +63,8 @@ export interface ValidationResult {
   error?: boolean;
   message?: string;
   data?: Record<string, any>;
+  type?: ErrorType;
+  source?: ErrorSources;
 }
 
 // Filter state interface
@@ -178,7 +188,7 @@ export const useValidationState = <T extends string>({
   
   // Validation state
   const [isValidating] = useState(false)
-  const [validationErrors, setValidationErrors] = useState<Map<number, Record<string, ErrorType[]>>>(new Map())
+  const [validationErrors, setValidationErrors] = useState<Map<number, Record<string, ValidationError[]>>>(new Map())
   const [rowValidationStatus, setRowValidationStatus] = useState<Map<number, 'pending' | 'validating' | 'validated' | 'error'>>(new Map())
   const [, setValidatingCells] = useState<Set<string>>(new Set())
   
@@ -212,7 +222,7 @@ export const useValidationState = <T extends string>({
   
   // Add batch update state
   const pendingUpdatesRef = useRef<{
-    errors: Map<number, Record<string, ErrorType[]>>,
+    errors: Map<number, Record<string, ValidationError[]>>,
     statuses: Map<number, 'pending' | 'validating' | 'validated' | 'error'>,
     data: Array<RowData<T>>
   }>({
@@ -357,7 +367,7 @@ export const useValidationState = <T extends string>({
 
   // Queue updates instead of immediate setState calls
   const queueUpdate = useCallback((rowIndex: number, updates: {
-    errors?: Record<string, ErrorType[]>,
+    errors?: Record<string, ValidationError[]>,
     status?: 'pending' | 'validating' | 'validated' | 'error',
     data?: RowData<T>
   }) => {
@@ -384,7 +394,7 @@ export const useValidationState = <T extends string>({
     const itemNumberMap = new Map<string, number[]>();
     
     // Initialize batch updates
-    const errors = new Map<number, Record<string, ErrorType[]>>();
+    const errors = new Map<number, Record<string, ValidationError[]>>();
     
     // Single pass through data to identify all item numbers
     data.forEach((row, index) => {
@@ -403,8 +413,9 @@ export const useValidationState = <T extends string>({
       if (indices.length > 1) {
         const errorObj = {
           message: `Duplicate item number: ${itemNumber}`,
-          level: 'error',
+          level: 'error' as 'error',
-          source: 'validation'
+          source: ErrorSources.Table,
+          type: ErrorType.Unique
         };
         
         // Add error to each row with this item number
@@ -470,10 +481,12 @@ export const useValidationState = <T extends string>({
           const errorData = await response.json();
           return {
             error: true,
-            message: 'UPC already exists',
+            message: `UPC already exists (${errorData.existingItemNumber})`,
             data: {
               itemNumber: errorData.existingItemNumber || '',
-            }
+            },
+            type: ErrorType.Unique,
+            source: ErrorSources.Api
           };
         } else if (response.status === 429) {
           return {
@@ -691,65 +704,38 @@ export const useValidationState = <T extends string>({
   }, [])
   
   // Helper function to validate a field value
-  const validateField = useCallback((value: any, field: Field<T>): ErrorType[] => {
+  const validateField = useCallback((value: any, field: Field<T>): ValidationError[] => {
-    const errors: ErrorType[] = [];
+    const errors: ValidationError[] = [];
     
     // Required field validation - improved to better handle various value types
     if (field.validations?.some(v => v.rule === 'required')) {
-      // Handle different value types more carefully
+      if (value === undefined || value === null || value === '' || 
-      const isEmpty = 
+          (Array.isArray(value) && value.length === 0) ||
-        value === undefined || 
+          (typeof value === 'object' && !Array.isArray(value) && Object.keys(value).length === 0)) {
-        value === null || 
-        (typeof value === 'string' && value.trim() === '') ||
-        (Array.isArray(value) && value.length === 0);
-      
-      if (isEmpty) {
         errors.push({
-          message: 'Required field',
+          message: field.validations.find(v => v.rule === 'required')?.errorMessage || `${field.label} is required`,
           level: 'error',
-          source: 'required'
+          source: ErrorSources.Row,
+          type: ErrorType.Required
         });
-        // Return early since other validations may not be relevant for empty values
-        return errors;
       }
     }
     
-    // Continue with other validations if the required check passed
+    // Regex validation
-    if (field.validations) {
+    const regexValidation = field.validations?.find(v => v.rule === 'regex');
-      for (const validation of field.validations) {
+    if (regexValidation && value !== undefined && value !== null && value !== '') {
-        // Skip required validation as we've already handled it
+      try {
-        if (validation.rule === 'required') continue;
+        const regex = new RegExp(regexValidation.value, regexValidation.flags);
-        
+        if (!regex.test(String(value))) {
-        if (validation.rule === 'regex' && typeof value === 'string') {
+          errors.push({
-          // Implement regex validation
+            message: regexValidation.errorMessage,
-          const regex = new RegExp(validation.value!);
+            level: regexValidation.level || 'error',
-          if (!regex.test(value)) {
+            source: ErrorSources.Row,
-            errors.push({
+            type: ErrorType.Regex
-              message: validation.errorMessage || 'Invalid format',
+          });
-              level: validation.level || 'error',
-              source: 'validation'
-            });
-          }
-        } else if (validation.rule === 'min' && typeof value === 'number') {
-          // Implement min validation
-          if (value < validation.value) {
-            errors.push({
-              message: validation.errorMessage || `Value must be at least ${validation.value}`,
-              level: validation.level || 'error',
-              source: 'validation'
-            });
-          }
-        } else if (validation.rule === 'max' && typeof value === 'number') {
-          // Implement max validation
-          if (value > validation.value) {
-            errors.push({
-              message: validation.errorMessage || `Value must be at most ${validation.value}`,
-              level: validation.level || 'error',
-              source: 'validation'
-            });
-          }
         }
-        // Add other validation types as needed
+      } catch (error) {
+        console.error('Invalid regex in validation:', error);
       }
     }
     
@@ -770,7 +756,7 @@ export const useValidationState = <T extends string>({
     
     // Get the row data
     const row = data[rowIndex];
-    const fieldErrors: Record<string, ErrorType[]> = {};
+    const fieldErrors: Record<string, ValidationError[]> = {};
     let hasErrors = false;
     
     // Get current errors for comparison
@@ -823,7 +809,8 @@ export const useValidationState = <T extends string>({
       fieldErrors['supplier'] = [{
         message: 'Supplier is required',
         level: 'error',
-        source: 'required'
+        source: ErrorSources.Row,
+        type: ErrorType.Required
       }];
       hasErrors = true;
     }
@@ -831,7 +818,8 @@ export const useValidationState = <T extends string>({
       fieldErrors['company'] = [{
         message: 'Company is required',
         level: 'error',
-        source: 'required'
+        source: ErrorSources.Row,
+        type: ErrorType.Required
       }];
       hasErrors = true;
     }
@@ -1095,7 +1083,7 @@ export const useValidationState = <T extends string>({
     
     // Create a copy of data and process all rows at once to minimize state updates
     const newData = [...data];
-    const batchErrors = new Map<number, Record<string, ErrorType[]>>();
+    const batchErrors = new Map<number, Record<string, ValidationError[]>>();
     const batchStatuses = new Map<number, 'pending' | 'validating' | 'validated' | 'error'>();
     
     // Extract template fields once outside the loop
@@ -1337,7 +1325,7 @@ export const useValidationState = <T extends string>({
       // When proceeding to the next screen, check for unvalidated rows first
       const hasErrors = [...validationErrors.entries()].some(([_, errors]) => {
         return Object.values(errors).some(errorSet => 
-          errorSet.some(error => error.source !== 'required')
+          errorSet.some(error => error.type !== ErrorType.Required)
         );
       });
       
@@ -1390,7 +1378,7 @@ export const useValidationState = <T extends string>({
       // Create a copy for data modifications
       const newData = [...data];
       // Use Maps for better performance with large datasets
-      const batchErrors = new Map<number, Record<string, ErrorType[]>>();
+      const batchErrors = new Map<number, Record<string, ValidationError[]>>();
       const batchStatuses = new Map<number, 'pending' | 'validating' | 'validated' | 'error'>();
 
       console.log(`Validating ${data.length} rows`);
@@ -1423,7 +1411,7 @@ export const useValidationState = <T extends string>({
           batchPromises.push(
             new Promise<void>(resolve => {
               const row = data[rowIndex];
-              const fieldErrors: Record<string, ErrorType[]> = {};
+              const fieldErrors: Record<string, ValidationError[]> = {};
               let hasErrors = false;
 
               // Set default values for tax_cat and ship_restrictions if not already set
@@ -1479,7 +1467,8 @@ export const useValidationState = <T extends string>({
                   fieldErrors[key] = [{
                     message: field.validations?.find(v => v.rule === 'required')?.errorMessage || 'This field is required',
                     level: 'error',
-                    source: 'required'
+                    source: ErrorSources.Row,
+                    type: ErrorType.Required
                   }];
                   hasErrors = true;
                 }
@@ -1490,7 +1479,8 @@ export const useValidationState = <T extends string>({
                 fieldErrors['supplier'] = [{
                   message: 'Supplier is required',
                   level: 'error',
-                  source: 'required'
+                  source: ErrorSources.Row,
+                  type: ErrorType.Required
                 }];
                 hasErrors = true;
               }
@@ -1499,7 +1489,8 @@ export const useValidationState = <T extends string>({
                 fieldErrors['company'] = [{
                   message: 'Company is required',
                   level: 'error',
-                  source: 'required'
+                  source: ErrorSources.Row,
+                  type: ErrorType.Required
                 }];
                 hasErrors = true;
               }

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/types/index.ts

@@ -1,16 +1,17 @@
-import { InfoWithSource, ErrorLevel } from "../../../types"
+import { InfoWithSource, ErrorLevel, ErrorSources, ErrorType as ValidationErrorType } from "../../../types"
 
 // Define our own Error type that's compatible with the original
 export interface ErrorType {
   message: string;
   level: ErrorLevel;
-  source?: string;
+  source?: ErrorSources;
+  type: ValidationErrorType;
 }
 
 // Export a namespace to make it accessible at runtime
 export const ErrorTypes = {
-  createError: (message: string, level: ErrorLevel = 'error', source: string = 'row'): ErrorType => {
+  createError: (message: string, level: ErrorLevel = 'error', source: ErrorSources = ErrorSources.Row, type: ValidationErrorType = ValidationErrorType.Custom): ErrorType => {
-    return { message, level, source };
+    return { message, level, source, type };
   }
 };
 

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/utils/dataMutations.ts

@@ -1,7 +1,7 @@
 import type { Data, Fields, Info, RowHook, TableHook } from "../../../types"
 import type { Meta, Error, Errors } from "../types"
 import { v4 } from "uuid"
-import { ErrorSources } from "../../../types"
+import { ErrorSources, ErrorType } from "../../../types"
 
 
 type DataWithMeta<T extends string> = Data<T> & Meta & {
@@ -18,10 +18,10 @@ export const addErrorsAndRunHooks = async <T extends string>(
 ): Promise<DataWithMeta<T>[]> => {
   const errors: Errors = {}
 
-  const addError = (source: ErrorSources, rowIndex: number, fieldKey: string, error: Info) => {
+  const addError = (source: ErrorSources, rowIndex: number, fieldKey: string, error: Info, type: ErrorType = ErrorType.Custom) => {
     errors[rowIndex] = {
       ...errors[rowIndex],
-      [fieldKey]: { ...error, source },
+      [fieldKey]: { ...error, source, type },
     }
   }
 
@@ -89,7 +89,7 @@ export const addErrorsAndRunHooks = async <T extends string>(
               addError(ErrorSources.Table, index, fieldKey, {
                 level: validation.level || "error",
                 message: validation.errorMessage || "Field must be unique",
-              })
+              }, ErrorType.Unique)
             }
           })
           break
@@ -103,7 +103,7 @@ export const addErrorsAndRunHooks = async <T extends string>(
               addError(ErrorSources.Row, realIndex, fieldKey, {
                 level: validation.level || "error",
                 message: validation.errorMessage || "Field is required",
-              })
+              }, ErrorType.Required)
             }
           })
           break
@@ -120,7 +120,7 @@ export const addErrorsAndRunHooks = async <T extends string>(
                 level: validation.level || "error",
                 message:
                   validation.errorMessage || `Field did not match the regex /${validation.value}/${validation.flags} `,
-              })
+              }, ErrorType.Regex)
             }
           })
           break

inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/utils/validationUtils.ts

@@ -1,5 +1,5 @@
-import { Field, Data, ErrorSources } from '../../../types'
+import { Field, Data, ErrorSources, ErrorType as ValidationErrorType } from '../../../types'
-import { ErrorType } from '../types/index'
+import { ErrorType } from '../types'
 
 /**
  * Formats a price value to a consistent format
@@ -80,17 +80,20 @@ export const validateRegex = (value: any, regex: string, flags?: string): boolea
  * @param message Error message
  * @param level Error level
  * @param source Error source
+ * @param type Error type
  * @returns Error object
  */
 export const createError = (
   message: string, 
   level: 'info' | 'warning' | 'error' = 'error',
-  source: ErrorSources = ErrorSources.Row
+  source: ErrorSources = ErrorSources.Row,
+  type: ValidationErrorType = ValidationErrorType.Custom
 ): ErrorType => {
   return {
     message,
     level,
-    source
+    source,
+    type
   } as ErrorType
 }
 
@@ -136,7 +139,8 @@ export const validateSpecialFields = <T extends string>(row: Data<T>): Record<st
     errors['supplier'] = [{
       message: 'Supplier is required',
       level: 'error',
-      source: ErrorSources.Row
+      source: ErrorSources.Row,
+      type: ValidationErrorType.Required
     }]
   }
   
@@ -145,7 +149,8 @@ export const validateSpecialFields = <T extends string>(row: Data<T>): Record<st
     errors['company'] = [{
       message: 'Company is required',
       level: 'error',
-      source: ErrorSources.Row
+      source: ErrorSources.Row,
+      type: ValidationErrorType.Required
     }]
   }
   

inventory/src/lib/react-spreadsheet-import/src/types.ts

@@ -163,8 +163,27 @@ export type Info = {
 }
 
 export enum ErrorSources {
-  Table = "table",
+  Row = 'row',
-  Row = "row",
+  Table = 'table',
+  Api = 'api',
+  Upc = 'upc'
+}
+
+// Add a standardized error type enum
+export enum ErrorType {
+  Required = 'required',
+  Regex = 'regex',
+  Unique = 'unique',
+  Custom = 'custom',
+  Api = 'api'
+}
+
+// Add a standardized error interface
+export interface ValidationError {
+  message: string;
+  level: 'info' | 'warning' | 'error';
+  source?: ErrorSources;
+  type: ErrorType;
 }
 
 /*
@@ -174,7 +193,8 @@ export enum ErrorSources {
    it is used to determine if row.__errors should be updated or not depending on different validations
 */
 export type InfoWithSource = Info & {
-  source: ErrorSources
+  source: ErrorSources;
+  type: ErrorType;
 }
 
 export type Result<T extends string> = {