matt/inventory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rearrange docs

Browse Source
This commit is contained in:
Matt
2025-03-09 22:07:14 -04:00
parent de1408bd58
commit 1c8709f520
4 changed files with 227 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

396
docs/ValidationStep-Refactoring-Plan.md Normal file
View File

@@ -0,0 +1,396 @@
# ValidationStep Component Refactoring Plan
## Overview
This document outlines a comprehensive plan to refactor the current ValidationStep component (4000+ lines) into a more maintainable, modular structure. The new implementation will be developed alongside the existing component without modifying the original code. Once completed, the previous step in the workflow will offer the option to continue to either the original ValidationStep or the new implementation.
## Table of Contents
1. [Current Component Analysis](#current-component-analysis)
2. [New Architecture Design](#new-architecture-design)
3. [Component Structure](#component-structure)
4. [State Management](#state-management)
5. [Key Features Implementation](#key-features-implementation)
6. [Integration Plan](#integration-plan)
7. [Testing Strategy](#testing-strategy)
8. [Project Timeline](#project-timeline)
9. [Design Principles](#design-principles)
10. [Appendix: Function Reference](#appendix-function-reference)
## Current Component Analysis
The current ValidationStep component has several issues:
- **Size**: At over 4000 lines, it's difficult to maintain and understand
- **Multiple responsibilities**: Handles validation, UI rendering, template management, and more
- **Special cases**: Contains numerous special case handlers and exceptions
- **Complex state management**: State is distributed across multiple useState calls
- **Tightly coupled concerns**: UI, validation logic, and business rules are intertwined
### Key Features to Preserve
1. **Data Validation**
- Field-level validation (required, regex, unique)
- Row-level validation (supplier, company fields)
- UPC validation with API integration
- AI-assisted validation
2. **Template Management**
- Saving, loading, and applying templates
- Template-based validation
3. **UI Components**
- Editable table with specialized cell renderers
- Error display and management
- Filtering and sorting capabilities
- Status indicators and progress tracking
4. **Special Field Handling**
- Input fields with price formatting
- Multi-input fields with separator configuration
- Select fields with dropdown options
- Checkbox fields with boolean value mapping
- UPC fields with specialized validation
5. **User Interaction Flows**
- Tab and keyboard navigation
- Bulk operations (select all, apply template)
- Row validation on value change
- Error reporting and display
## New Architecture Design
The new architecture will follow these principles:
1. **Separation of Concerns**
- UI rendering separate from business logic
- Validation logic isolated from state management
- Clear interfaces between components
2. **Composable Components**
- Small, focused components with single responsibilities
- Reusable pattern for different field types
3. **Centralized State Management**
- Custom hooks for state management
- Clear data flow patterns
- Reduced prop drilling
4. **Consistent Error Handling**
- Standardized error structure
- Predictable error propagation
- User-friendly error display
5. **Performance Optimization**
- Virtualized table rendering
- Memoization of expensive computations
- Deferred validation for better user experience
## Component Structure
The new ValidationStepNew folder has the following structure:
```
ValidationStepNew/
├── index.tsx # Main entry point that composes all pieces
├── components/ # UI Components
│ ├── ValidationContainer.tsx # Main wrapper component
│ ├── ValidationTable.tsx # Table implementation
│ ├── ValidationCell.tsx # Cell component
│ ├── ValidationSidebar.tsx # Sidebar with controls
│ ├── ValidationToolbar.tsx # Top toolbar (removed as unnecessary)
│ ├── TemplateManager.tsx # Template management
│ ├── FilterPanel.tsx # Filtering interface (integrated into Container)
│ └── cells/ # Specialized cell renderers
│ ├── InputCell.tsx
│ ├── SelectCell.tsx
│ ├── MultiInputCell.tsx
│ └── CheckboxCell.tsx
├── hooks/ # Custom hooks
│ ├── useValidationState.tsx # Main state management
│ ├── useTemplates.tsx # Template-related logic (integrated into ValidationState)
│ ├── useFilters.tsx # Filtering logic (integrated into ValidationState)
│ └── useUpcValidation.tsx # UPC-specific validation
└── utils/ # Utility functions
├── validationUtils.ts # Validation helper functions
├── formatters.ts # Value formatting utilities
└── constants.ts # Constant values and configuration
```
### Component Responsibilities
#### ValidationContainer
- Main container component
- Coordinates between subcomponents
- Manages global state
- Handles navigation events (next, back)
- Contains filter controls
#### ValidationTable
- Displays the data in tabular form
- Manages selection state
- Handles keyboard navigation
- Integrates with TanStack Table
- Displays properly styled rows and cells
#### ValidationCell
- Factory component that renders appropriate cell type
- Manages cell-level state
- Handles validation errors display
- Manages edit mode
- Shows consistent error indicators
#### TemplateManager
- Handles template selection UI
- Provides template save/load functionality
- Manages template application to rows
#### Cell Components
- **InputCell**: Handles text input with multiline and price support
- **MultiInputCell**: Handles multiple values with separator configuration
- **SelectCell**: Command/popover component for single selection
- **CheckboxCell**: Boolean value selection with mapping support
## State Management
### Core State Interface
```typescript
interface ValidationState<T extends string> {
// Core data
data: RowData<T>[];
filteredData: RowData<T>[];
// Validation state
isValidating: boolean;
validationErrors: Map<number, Record<string, Error[]>>;
rowValidationStatus: Map<number, 'pending' | 'validating' | 'validated' | 'error'>;
// Selection state
rowSelection: RowSelectionState;
// Template state
templates: Template[];
selectedTemplateId: string | null;
// Filter state
filters: FilterState;
// Methods
updateRow: (rowIndex: number, key: T, value: any) => void;
validateRow: (rowIndex: number) => Promise<void>;
validateUpc: (rowIndex: number, upcValue: string) => Promise<void>;
applyTemplate: (templateId: string, rowIndexes: number[]) => void;
saveTemplate: (name: string, type: string) => void;
setFilters: (newFilters: Partial<FilterState>) => void;
// Additional methods...
}
```
### useValidationState Hook
The main state management hook handles:
- Data manipulation (update, sort, filter)
- Selection management
- Validation coordination
- Integration with validation utilities
- Template management
- Filtering and sorting
## Key Features Implementation
### 1. Field Type Handling
Implemented a strategy pattern for different field types:
```typescript
// In ValidationCell
const renderCellContent = () => {
const fieldType = field.fieldType.type
switch (fieldType) {
case 'input':
return <InputCell<T> field={field} value={value} onChange={onChange} ... />
case 'multi-input':
return <MultiInputCell<T> field={field} value={value} onChange={onChange} ... />
case 'select':
return <SelectCell<T> field={field} value={value} onChange={onChange} ... />
// etc.
}
}
```
### 2. Validation Logic
Validation is broken down into clear steps:
1. **Field Validation**: Apply field-level validations (required, regex, etc.)
2. **Row Validation**: Apply row-level validations and rowHook
3. **Table Validation**: Apply table-level validations (unique) and tableHook
Validation now happens automatically without explicit buttons, with immediate feedback on field blur.
### 3. UI Components
UI components follow these principles:
1. **Consistent Styling**: All components use shadcn UI for consistent look and feel
2. **Visual Feedback**: Errors are clearly indicated with icons and border styling
3. **Intuitive Editing**: Fields show outlines even when not in focus, and edit on click
4. **Proper Command Pattern**: Select and multi-select fields use command/popover pattern for better UX
5. **Focus Management**: Fields close when clicking away and perform validation on blur
## Design Principles
Based on user preferences and best practices, the following design principles guide this refactoring:
1. **Automatic Validation**
- Validation should happen automatically without explicit buttons
- All validation should run on initial data load
- Fields should validate on blur (when user clicks away)
2. **Modern UI Patterns**
- Command/popover components for all selects and multi-selects
- Consistent field outlines and borders even when not in focus
- Badge patterns for multi-select items
- Clear visual indicators for errors
3. **Reduced Complexity**
- Remove unnecessary UI elements like "validate all" buttons
- Eliminate redundant state and toast notifications
- Simplify component hierarchy where possible
- Find root causes rather than adding special cases
4. **Consistent Component Behavior**
- Fields should close when clicking away
- All inputs should follow the same editing pattern
- Error handling should be consistent across all field types
- Multi-select fields should allow selecting multiple items with clear visual feedback
## Integration Plan
### 1. Creating the New Component Structure
Folder structure has been created without modifying the existing code:
```bash
mkdir -p inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/{components,hooks,utils}
mkdir -p inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/cells
```
### 2. Implementing Basic Components
Core components have been implemented:
1. Created index.tsx as the main entry point
2. Implemented ValidationContainer with basic state management
3. Created ValidationTable for data display
4. Implemented basic cell rendering with specialized cell types
### 3. Implementing State Management
State management has been implemented:
1. Created useValidationState hook
2. Implemented data transformation utilities
3. Added validation logic
### 4. Integrating with Previous Step
The previous step component allows choosing between validation implementations, enabling gradual testing and adoption.
## Testing Strategy
1. **Unit Tests**
- Test individual utility functions
- Test hooks in isolation
- Test individual UI components
2. **Integration Tests**
- Test component interactions
- Test state management flow
- Test validation logic integration
3. **Comparison Tests**
- Compare output of new component with original
- Verify that all functionality works the same
4. **Performance Tests**
- Measure render times
- Evaluate memory usage
- Compare against original component
## Project Timeline
1. **Phase 1: Initial Structure (Completed)**
- Set up folder structure
- Implement basic components
- Create core state management
2. **Phase 2: Core Functionality (In Progress)**
- Implement validation logic (completed)
- Create cell renderers (completed)
- Add template management (in progress)
3. **Phase 3: Special Features (Upcoming)**
- Implement UPC validation
- Add AI validation
- Handle special cases
4. **Phase 4: UI Refinement (Ongoing)**
- Improve error display (completed)
- Enhance user interactions (completed)
- Optimize performance (in progress)
5. **Phase 5: Testing and Integration (Upcoming)**
- Write tests
- Fix bugs
- Integrate with previous step
## Appendix: Function Reference
This section documents the core functions from the original ValidationStep that need to be preserved in the new implementation.
### Validation Functions
1. **validateRegex** - Validates values against regex patterns
2. **getValidationError** - Determines field-level validation errors
3. **validateAndCommit** - Validates and commits new values
4. **validateData** - Validates all data rows
5. **validateUpcAndGenerateItemNumbers** - Validates UPC codes and generates item numbers
### Formatting Functions
1. **formatPrice** - Formats price values
2. **getDisplayValue** - Gets formatted display value based on field type
3. **isMultiInputType** - Checks if field is multi-input type
4. **getMultiInputSeparator** - Gets separator for multi-input fields
5. **isPriceField** - Checks if field should be formatted as price
### Template Functions
1. **loadTemplates** - Loads templates from storage
2. **saveTemplate** - Saves a new template
3. **applyTemplate** - Applies a template to selected rows
4. **getTemplateDisplayText** - Gets display text for a template
### AI Validation Functions
1. **handleAiValidation** - Triggers AI validation
2. **showCurrentPrompt** - Shows current AI prompt
3. **getFieldDisplayValue** - Gets display value for a field
4. **highlightDifferences** - Highlights differences between original and corrected values
5. **getFieldDisplayValueWithHighlight** - Gets display value with highlighted differences
6. **revertAiChange** - Reverts an AI-suggested change
7. **isChangeReverted** - Checks if an AI change has been reverted
### Event Handlers
1. **handleUpcValueUpdate** - Handles UPC value updates
2. **handleBlur** - Handles input blur events
3. **handleWheel** - Handles wheel events for navigation
4. **copyValueDown** - Copies a value to cells below
5. **handleSkuGeneration** - Generates SKUs
By following this refactoring plan, we continue to transform the monolithic ValidationStep component into a modular, maintainable set of components while preserving all existing functionality and aligning with user preferences for design and behavior.

137
docs/ValidationStepNew-Implementation-Status.md Normal file
View File

@@ -0,0 +1,137 @@
# ValidationStepNew Implementation Status
## Overview
This document outlines the current status of the ValidationStepNew implementation, a refactored version of the original ValidationStep component. The goal is to create a more maintainable, modular component that preserves all functionality of the original while eliminating technical debt and implementing modern UI patterns.
## Design Principles
Based on the user's preferences, we're following these core design principles:
1. **Automatic Validation**
- ✅ Validation runs automatically on data load
- ✅ No explicit "validate all" button needed
- ✅ Fields validate on blur when user clicks away
- ✅ Immediate visual feedback for validation errors
2. **Modern UI Patterns**
- ✅ Command/popover components for selects and multi-selects
- ✅ Consistent field outlines and borders even when not in focus
- ✅ Badge pattern for multi-select field items
- ✅ Visual indicators for errors with appropriate styling
3. **Reduced Complexity**
- ✅ Removed unnecessary UI elements like "validate all" button
- ✅ Eliminated redundant toast notifications
- ✅ Simplified component hierarchy
- ✅ Fixed root causes rather than adding special cases
4. **Consistent Behavior**
- ✅ Fields close when clicking away
- ✅ All inputs follow the same editing pattern
- ✅ Error handling is consistent across field types
- ✅ Multi-select fields allow selecting multiple items
## Completed Components
### Core Structure
- ✅ Main component structure
- ✅ Directory organization
- ✅ TypeScript interfaces
- ✅ Props definition and passing
### State Management
- ✅ `useValidationState` hook for centralized state
- ✅ Data validation logic
- ✅ Integration with rowHook and tableHook
- ✅ Error tracking and management
- ✅ Row selection
- ✅ Automatic validation on data load
### UI Components
- ✅ ValidationContainer with appropriate layout
- ✅ ValidationTable with shadcn UI components
- ✅ ValidationCell factory component
- ✅ Row select/deselect functionality
- ✅ Error display and indicators
- ✅ Selection action bar
### Cell Components
- ✅ InputCell with price and multiline support
- ✅ MultiInputCell with separator configuration
- ✅ SelectCell using command/popover pattern
- ✅ CheckboxCell with boolean mapping
- ✅ Consistent styling across all field types
- ✅ Proper edit/view state management
- ✅ Outlined borders in both edit and view modes
### Utility Functions
- ✅ Value formatting for display
- ✅ Field type detection
- ✅ Error creation and management
- ✅ Price formatting
### UI Improvements
- ✅ Consistent borders and field outlines
- ✅ Fields that properly close when clicking away
- ✅ Multi-select with badge UI pattern
- ✅ Command pattern for searchable select menus
- ✅ Better visual error indication
## Pending Tasks
### Enhanced Validation
- ⏳ AI validation system
- ⏳ Custom validation hooks
- ⏳ Enhanced UPC validation with API integration
- ⏳ Validation visualizations
### Advanced UI Features
- ⏳ Table virtualization for performance
- ⏳ Drag-and-drop reordering
- ⏳ Bulk operations (copy down, fill all, etc.)
- ⏳ Keyboard navigation improvements
- ⏳ Template dialogs and management UI
### Special Features
- ⏳ Image preview integration
- ⏳ SKU generation system
- ⏳ Item number generation
- ⏳ Dependent dropdown values
### Testing
- ⏳ Unit tests for utility functions
- ⏳ Component tests
- ⏳ Integration tests
- ⏳ Performance benchmarks
## Known Issues
1. TypeScript error for `validationDisabled` property in ValidationCell.tsx
2. Some type casting is needed due to complex generic types
3. Need to address edge cases for multi-select fields validation
4. Proper error handling for API calls needs implementation
## Next Steps
1. Fix TypeScript errors in ValidationCell and related components
2. Complete template management functionality
3. Implement UPC validation with API integration
4. Make multi-select field validation more robust
5. Add comprehensive tests
## Performance Improvements
We've already implemented several performance optimizations:
1. ✅ More efficient state updates by removing unnecessary re-renders
2. ✅ Better error handling to prevent cascading validations
3. ✅ Improved component isolation to prevent unnecessary re-renders
4. ✅ Automatic validation that doesn't block the UI
Additional planned improvements:
1. Virtualized table rendering for large datasets
2. Memoization of expensive calculations
3. Optimized state updates to minimize re-renders
4. Batched API calls for validation

72
docs/fix-multi-select.md Normal file
View File

@@ -0,0 +1,72 @@
# Solution: Keeping Dropdowns Open During Multiple Selections
## The Problem
When implementing a multi-select dropdown in React, a common issue occurs:
1. You select an item in the dropdown
2. The `onChange` handler is called, updating the data
3. This triggers a re-render of the parent component (in this case, the entire table)
4. During the re-render, the dropdown is unmounted and remounted
5. This causes the dropdown to close before you can make multiple selections
## The Solution: Deferred State Updates
The key insight is to **separate local state management from parent state updates**:
```typescript
// Step 1: Add local state to track selections
const [internalValue, setInternalValue] = useState<string[]>(value)
// Step 2: Handle popover open state changes
const handleOpenChange = useCallback((newOpen: boolean) => {
if (open && !newOpen) {
// Only update parent state when dropdown closes
if (JSON.stringify(internalValue) !== JSON.stringify(value)) {
onChange(internalValue);
}
}
setOpen(newOpen);
if (newOpen) {
// Sync internal state with external state when opening
setInternalValue(value);
}
}, [open, internalValue, value, onChange]);
// Step 3: Toggle selection only updates internal state
const toggleSelection = useCallback((selectedValue: string) => {
setInternalValue(prev => {
if (prev.includes(selectedValue)) {
return prev.filter(v => v !== selectedValue);
} else {
return [...prev, selectedValue];
}
});
}, []);
```
## Why This Works
1. **No parent re-renders during selection**: Since we're only updating local state, the parent component doesn't re-render during selection.
2. **Consistent UI**: The dropdown shows accurate selected states using the internal value.
3. **Data integrity**: The final selections are properly synchronized back to the parent when done.
4. **Resilient to external changes**: Initial state is synchronized when opening the dropdown.
## Implementation Steps
1. Create a local state variable to track selections inside the component
2. Only make selections against this local state while the dropdown is open
3. Defer updating the parent until the dropdown is explicitly closed
4. When opening, synchronize the internal state with the external value
## Benefits
This pattern:
- Avoids re-render cycles that would unmount the dropdown
- Maintains UI consistency during multi-selection
- Simplifies the component's interaction with parent components
- Works with existing component lifecycles rather than fighting against them
This solution is much simpler than trying to prevent event propagation or manipulating DOM events, and addresses the root cause of the issue: premature re-rendering.

227
docs/validation-table-scroll-issue.md Normal file
View File

@@ -0,0 +1,227 @@
# ValidationTable Scroll Position Issue
## Problem Description
The `ValidationTable` component in the inventory application suffers from a persistent scroll position issue. When the table content updates or re-renders, the scroll position resets to the top left corner. This creates a poor user experience, especially when users are working with large datasets and need to maintain their position while making edits or filtering data.
Specific behaviors:
- Scroll position resets to the top left corner during re-renders
- User loses their place in the table when data is updated
- The table does not preserve vertical or horizontal scroll position
## Relevant Files
- **`inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/ValidationTable.tsx`**
- Main component that renders the validation table
- Handles scroll position management
- **`inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/ValidationContainer.tsx`**
- Parent component that wraps ValidationTable
- Creates an EnhancedValidationTable wrapper component
- Manages data and state for the validation table
- **`inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/hooks/useValidationState.tsx`**
- Provides state management and data manipulation functions
- Contains scroll-related code in the `updateRow` function
- **`inventory/src/lib/react-spreadsheet-import/src/steps/ValidationStepNew/components/ValidationCell.tsx`**
- Renders individual cells in the table
- May influence re-renders that affect scroll position
## Failed Attempts
We've tried multiple approaches to fix the scroll position issue, none of which have been successful:
### 1. Using Refs for Scroll Position
```typescript
const scrollPosition = useRef({
left: 0,
top: 0
});
// Capture position on scroll
const handleScroll = useCallback(() => {
if (tableContainerRef.current) {
scrollPosition.current = {
left: tableContainerRef.current.scrollLeft,
top: tableContainerRef.current.scrollTop
};
}
}, []);
// Restore in useLayoutEffect
useLayoutEffect(() => {
const container = tableContainerRef.current;
if (container) {
const { left, top } = scrollPosition.current;
if (left || top) {
container.scrollLeft = left;
container.scrollTop = top;
}
}
});
```
Result: Scroll position was still lost during updates.
### 2. Multiple Restoration Attempts with Timeouts
```typescript
// Multiple timeouts at different intervals
setTimeout(() => {
if (tableContainerRef.current) {
tableContainerRef.current.scrollTop = savedPosition.top;
tableContainerRef.current.scrollLeft = savedPosition.left;
}
}, 0);
setTimeout(() => {
if (tableContainerRef.current) {
tableContainerRef.current.scrollTop = savedPosition.top;
tableContainerRef.current.scrollLeft = savedPosition.left;
}
}, 50);
// Additional timeouts at 100ms, 300ms
```
Result: Still not reliable, scroll position would reset between timeouts or after all timeouts completed.
### 3. Using MutationObserver and ResizeObserver
```typescript
// Create a mutation observer to detect DOM changes
const mutationObserver = new MutationObserver(() => {
if (shouldPreserveScroll) {
restoreScrollPosition();
}
});
// Start observing the table for DOM changes
mutationObserver.observe(scrollableContainer, {
childList: true,
subtree: true,
attributes: true,
attributeFilter: ['style', 'class']
});
// Create a resize observer
const resizeObserver = new ResizeObserver(() => {
if (shouldPreserveScroll) {
restoreScrollPosition();
}
});
// Observe the table container
resizeObserver.observe(scrollableContainer);
```
Result: Did not reliably maintain scroll position, and sometimes caused other rendering issues.
### 4. Recursive Restoration Approach
```typescript
let attempts = 0;
const maxAttempts = 5;
const restore = () => {
if (tableContainerRef.current) {
tableContainerRef.current.scrollTop = y;
tableContainerRef.current.scrollLeft = x;
attempts++;
if (attempts < maxAttempts) {
setTimeout(restore, 50 * attempts);
}
}
};
restore();
```
Result: No improvement, scroll position still reset.
### 5. Using React State for Scroll Position
```typescript
const [scrollPos, setScrollPos] = useState<{top: number; left: number}>({top: 0, left: 0});
// Track the scroll event
useEffect(() => {
const handleScroll = () => {
if (scrollContainerRef.current) {
setScrollPos({
top: scrollContainerRef.current.scrollTop,
left: scrollContainerRef.current.scrollLeft
});
}
};
// Add scroll listener...
}, []);
// Restore scroll position
useLayoutEffect(() => {
const container = scrollContainerRef.current;
const { top, left } = scrollPos;
if (top > 0 || left > 0) {
requestAnimationFrame(() => {
if (container) {
container.scrollTop = top;
container.scrollLeft = left;
}
});
}
}, [scrollPos, data]);
```
Result: Caused the screen to shake violently when scrolling and did not preserve position.
### 6. Using Key Attribute for Stability
```typescript
return (
<div
key="validation-table-container"
ref={scrollContainerRef}
className="overflow-auto max-h-[calc(100vh-300px)]"
>
{/* Table content */}
</div>
);
```
Result: Did not resolve the issue and may have contributed to rendering instability.
### 7. Removing Scroll Management from Other Components
We removed scroll position management code from:
- `useValidationState.tsx` (in the updateRow function)
- `ValidationContainer.tsx` (in the enhancedUpdateRow function)
Result: This did not fix the issue either.
## Current Understanding
The scroll position issue appears to be complex and likely stems from multiple factors:
1. React's virtual DOM reconciliation may be replacing the scroll container element during updates
2. The table uses complex memo patterns with custom equality checks that may not be working as expected
3. The data structure may be changing in ways that cause complete re-renders
4. The component hierarchy (with EnhancedValidationTable wrapper) may be affecting DOM stability
## Next Steps to Consider
Potential approaches that haven't been tried yet:
1. Implement a completely separate scroll container that exists outside of React's rendering cycle
2. Use a third-party virtualized table library that handles scroll position natively
3. Restructure the component hierarchy to minimize re-renders
4. Use the React DevTools profiler to identify which components are causing re-renders
5. Consider simplifying the data structure to reduce the complexity of renders
## Conclusion
This issue has proven particularly challenging to resolve. The current ValidationTable implementation struggles with scroll position preservation despite multiple different approaches. A more fundamental restructuring of the component or its rendering approach may be necessary.