inventory/PRODUCT_IMPORT_ENHANCEMENTS.md

# Product Import Module - Enhancement & Issues Outline

This document outlines the investigation and implementation requirements for each requested enhancement to the product import module.

---

## 1. UPC Import - Strip Quotes and Spaces ✅ IMPLEMENTED

**Issue:** When importing UPCs, strip `'`, `"` characters and any spaces, leaving only numbers.

**Implementation (Completed):**
- Modified `normalizeUpcValue()` in [Import.tsx:661-667](inventory/src/pages/Import.tsx#L661-L667)
- Strips single quotes, double quotes, smart quotes (`'"`), and whitespace before processing
- Then handles scientific notation and extracts only digits

**Files Modified:**
- `inventory/src/pages/Import.tsx` - `normalizeUpcValue()` function

---

## 2. AI Context Columns in Validation Payloads ✅ IMPLEMENTED

**Issue:** The match columns step has a setting to use a field only for AI context (`isAiSupplemental`). Update AI description validation to include any columns selected with this option in the payload. Also include in sanity check payload. Not needed for names.

**Current Implementation:**
- AI Supplemental toggle: [MatchColumnsStep.tsx:102-118](inventory/src/components/product-import/steps/MatchColumnsStep/MatchColumnsStep.tsx#L102-L118)
- AI supplemental data stored in `__aiSupplemental` field on each row
- Description payload builder: [inlineAiPayload.ts:183-195](inventory/src/components/product-import/steps/ValidationStep/utils/inlineAiPayload.ts#L183-L195)

**Implementation:**
1. **Update `buildDescriptionValidationPayload()` in `inlineAiPayload.ts`** to include AI supplemental data:
   ```typescript
   export const buildDescriptionValidationPayload = (
     row: Data<string>,
     fieldOptions: FieldOptionsMap,
     productLinesCache: Map<string, SelectOption[]>,
     sublinesCache: Map<string, SelectOption[]>
   ) => {
     const payload: Record<string, unknown> = {
       name: row.name,
       description: row.description,
       company_name: getFieldOptionLabel(row.company, fieldOptions, 'company'),
       company_id: row.company,
       categories: getFieldOptionLabel(row.category, fieldOptions, 'category'),
     };

     // Add AI supplemental context if present
     if (row.__aiSupplemental && typeof row.__aiSupplemental === 'object') {
       payload.additional_context = row.__aiSupplemental;
     }

     return payload;
   };
   ```

2. **Update sanity check payload** - Locate sanity check submission logic and include `__aiSupplemental` data

3. **Verify `__aiSupplemental` is properly populated** from MatchColumnsStep when columns are marked as AI context only

**Files to Modify:**
- `inventory/src/components/product-import/steps/ValidationStep/utils/inlineAiPayload.ts`
- Backend sanity check endpoint (if separate from description validation)
- Verify data flow in `MatchColumnsStep.tsx` → `ValidationStep`

---

## 3. Fresh Taxonomy Data Per Session ✅ IMPLEMENTED

**Issue:** Ensure taxonomy data is brought in fresh with each session - cache should be invalidated if we exit the import flow and start again.

**Current Implementation:**
- Field options cached 5 minutes: [ValidationStep/index.tsx:128-133](inventory/src/components/product-import/steps/ValidationStep/index.tsx#L128-L133)
- Product lines cache: `productLinesCache` in Zustand store
- Sublines cache: `sublinesCache` in Zustand store
- Caches set to 10-minute stale time

**Implementation:**
1. **Add cache invalidation on import flow mount/unmount** in `UploadFlow.tsx`:
   ```typescript
   useEffect(() => {
     // On mount - invalidate import-related query cache
     queryClient.invalidateQueries({ queryKey: ['import-field-options'] });

     return () => {
       // On unmount - clear caches
       queryClient.removeQueries({ queryKey: ['import-field-options'] });
       queryClient.removeQueries({ queryKey: ['product-lines'] });
       queryClient.removeQueries({ queryKey: ['sublines'] });
     };
   }, []);
   ```

2. **Clear Zustand store caches** when exiting import flow:
   - Add action to `validationStore.ts` to clear `productLinesCache` and `sublinesCache`
   - Call this action on unmount of `UploadFlow` or when navigating away

3. **Consider adding a `sessionId`** that changes on each import flow start, used as part of cache keys

**Files to Modify:**
- `inventory/src/components/product-import/steps/UploadFlow.tsx` - Add cleanup effect
- `inventory/src/components/product-import/steps/ValidationStep/store/validationStore.ts` - Add cache clear action
- Potentially `inventory/src/components/product-import/steps/ValidationStep/index.tsx` - Query key updates

---

## 4. Save Template from Confirmation Page ✅ IMPLEMENTED

**Issue:** Add option to save rows of submitted data as a new template on the confirmation page after completing the import flow. Verify this works with new validation step changes.

**Current Implementation:**
- **Import Results section already exists** inline in [Import.tsx:968-1150](inventory/src/pages/Import.tsx#L968-L1150)
  - Shows created products (lines 1021-1097) with image, name, UPC, item number
  - Shows errored products (lines 1100-1138) with error details
  - "Fix products with errors" button resumes validation flow for failed items
- Template saving logic in ValidationStep: [useTemplateManagement.ts:204-266](inventory/src/components/product-import/steps/ValidationStep/hooks/useTemplateManagement.ts#L204-L266)
- Saves via `POST /api/templates`
- `importOutcome.submittedProducts` contains the full product data for each row

**Implementation:**
1. **Add "Save as Template" button** to each created product row in the results section (around line 1087-1092 in Import.tsx):
   ```typescript
   // Add button after the item number display
   <Button
     variant="ghost"
     size="sm"
     onClick={() => handleSaveAsTemplate(index)}
   >
     <BookmarkPlus className="h-4 w-4" />
   </Button>
   ```

2. **Add state and dialog** for template saving in Import.tsx:
   ```typescript
   const [templateSaveDialogOpen, setTemplateSaveDialogOpen] = useState(false);
   const [selectedProductForTemplate, setSelectedProductForTemplate] = useState<NormalizedProduct | null>(null);
   ```

3. **Extract/reuse template save logic** from `useTemplateManagement.ts`:
   - The `saveNewTemplate()` function (lines 204-266) can be extracted into a shared utility
   - Or create a `SaveTemplateDialog` component that can be used in both places
   - Key fields needed: `company` (for template name), `product_type`, and all product field values

4. **Data mapping consideration:**
   - `importOutcome.submittedProducts` uses `NormalizedProduct` type
   - Templates expect raw field values - may need to map back from normalized format
   - Exclude metadata fields: `['id', '__index', '__meta', '__template', '__original', '__corrected', '__changes', '__aiSupplemental']`

**Files to Modify:**
- `inventory/src/pages/Import.tsx` - Add save template button, state, and dialog
- Consider creating `inventory/src/components/product-import/SaveTemplateDialog.tsx` for reusability
- Potentially extract core save logic from `useTemplateManagement.ts` into shared utility

---

## 5. Sheet Preview on Select Sheet Step ✅ IMPLEMENTED

**Issue:** On the select sheet step, show a preview of the first 10 lines or so of each sheet underneath the options.

**Implementation (Completed):**
- Added `workbook` prop to `SelectSheetStep` component
- Added `sheetPreviews` memoized computation using `XLSXLib.utils.sheet_to_json()`
- Shows first 10 rows, 8 columns max per sheet
- Added `truncateCell()` helper to limit cell content to 30 characters with ellipsis
- Each sheet option is now a clickable card with:
  - Radio button and sheet name
  - Row count indicator
  - Scrollable preview table with horizontal scroll
  - Selected state highlighted with primary border
- Updated `UploadFlow.tsx` to pass workbook prop

**Files Modified:**
- `inventory/src/components/product-import/steps/SelectSheetStep/SelectSheetStep.tsx`
- `inventory/src/components/product-import/steps/UploadFlow.tsx`

---

## 6. Empty Row Removal ✅ IMPLEMENTED

**Issue:** When importing a sheet, automatically remove completely empty rows.

**Current Implementation:**
- Empty columns are filtered: [MatchColumnsStep.tsx:616-634](inventory/src/components/product-import/steps/MatchColumnsStep/MatchColumnsStep.tsx#L616-L634)
- A "Remove empty/duplicates" button exists that removes empty rows, single-value rows, AND duplicates
- The automatic removal should ONLY remove completely empty rows, not duplicates or single-value rows

**Implementation (Completed):**
- Added `isRowCompletelyEmpty()` helper function to [SelectHeaderStep.tsx](inventory/src/components/product-import/steps/SelectHeaderStep/SelectHeaderStep.tsx)
- Added `useMemo` to filter empty rows on initial data load
- Uses `Object.values(row)` to check all cell values (matches existing button logic)
- Only removes rows where ALL values are undefined, null, or whitespace-only strings
- Manual "Remove Empty/Duplicates" button still available for additional cleanup (duplicates, single-value rows)

**Files Modified:**
- `inventory/src/components/product-import/steps/SelectHeaderStep/SelectHeaderStep.tsx`

---

## 7. Unit Conversion for Weight/Dimensions ✅ IMPLEMENTED

**Issue:** Add unit conversion feature for weight and dimensions columns - similar to calculator button on cost/msrp, add button that opens popover with options to convert grams → oz, lbs → oz for the whole column at once.

**Current Implementation:**
- Calculator button on price columns: [ValidationTable.tsx:1491-1627](inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx#L1491-L1627)
- `PriceColumnHeader` component shows calculator icon on hover
- Weight field defined in config with validation

**Implementation:**
1. **Create `UnitConversionColumnHeader` component** (similar to `PriceColumnHeader`):
   ```typescript
   const UnitConversionColumnHeader = ({ field, table }) => {
     const [showPopover, setShowPopover] = useState(false);

     const conversions = {
       weight: [
         { label: 'Grams → Ounces', factor: 0.035274 },
         { label: 'Pounds → Ounces', factor: 16 },
         { label: 'Kilograms → Ounces', factor: 35.274 },
       ],
       dimensions: [
         { label: 'Centimeters → Inches', factor: 0.393701 },
         { label: 'Millimeters → Inches', factor: 0.0393701 },
       ]
     };

     const applyConversion = (factor: number) => {
       // Batch update all cells in column
       table.rows.forEach((row, index) => {
         const currentValue = parseFloat(row[field.key]);
         if (!isNaN(currentValue)) {
           updateCell(index, field.key, (currentValue * factor).toFixed(2));
         }
       });
     };

     return (
       <Popover open={showPopover} onOpenChange={setShowPopover}>
         <PopoverTrigger>
           <Scale className="h-4 w-4" /> {/* or similar icon */}
         </PopoverTrigger>
         <PopoverContent>
           {conversions[fieldType].map(conv => (
             <Button key={conv.label} onClick={() => applyConversion(conv.factor)}>
               {conv.label}
             </Button>
           ))}
         </PopoverContent>
       </Popover>
     );
   };
   ```

2. **Identify weight/dimension fields** in config:
   - `weight_oz`, `length_in`, `width_in`, `height_in` (check actual field keys)

3. **Add to column header render logic** in ValidationTable

**Files to Modify:**
- `inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx`
- Potentially create new component file for `UnitConversionColumnHeader`
- Update column header rendering to use new component for weight/dimension fields

---

## 8. Expanded MSRP Auto-Fill from Cost ✅ IMPLEMENTED

**Issue:** Expand auto-fill functionality for MSRP from cost - open small popover with options for 2x, 2.1x, 2.2x, 2.3x, 2.4x, 2.5x multipliers, plus checkbox to round up to nearest 9.

**Current Implementation:**
- Calculator on MSRP column: [ValidationTable.tsx:1540-1584](inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx#L1540-L1584)
- Currently only does `Cost × 2` then subtracts 0.01 if whole number

**Implementation:**
1. **Replace simple click with popover** in `PriceColumnHeader`:
   ```typescript
   const [selectedMultiplier, setSelectedMultiplier] = useState(2.0);
   const [roundToNine, setRoundToNine] = useState(false);
   const multipliers = [2.0, 2.1, 2.2, 2.3, 2.4, 2.5];

   const roundUpToNine = (value: number): number => {
     // 1.41 → 1.49, 2.78 → 2.79, 12.32 → 12.39
     const wholePart = Math.floor(value);
     const decimal = value - wholePart;
     if (decimal <= 0.09) return wholePart + 0.09;
     if (decimal <= 0.19) return wholePart + 0.19;
     // ... continue pattern, or:
     const lastDigit = Math.floor(decimal * 10);
     return wholePart + (lastDigit / 10) + 0.09;
   };

   const calculateMsrp = (cost: number): number => {
     let result = cost * selectedMultiplier;
     if (roundToNine) {
       result = roundUpToNine(result);
     }
     return result;
   };
   ```

2. **Create popover UI**:
   ```tsx
   <Popover>
     <PopoverTrigger><Calculator className="h-4 w-4" /></PopoverTrigger>
     <PopoverContent className="w-48">
       <div className="space-y-2">
         <Label>Multiplier</Label>
         <div className="grid grid-cols-3 gap-1">
           {multipliers.map(m => (
             <Button
               key={m}
               variant={selectedMultiplier === m ? 'default' : 'outline'}
               size="sm"
               onClick={() => setSelectedMultiplier(m)}
             >
               {m}x
             </Button>
           ))}
         </div>
         <div className="flex items-center gap-2">
           <Checkbox checked={roundToNine} onCheckedChange={setRoundToNine} />
           <Label>Round to .X9</Label>
         </div>
         <Button onClick={applyCalculation} className="w-full">
           Apply
         </Button>
       </div>
     </PopoverContent>
   </Popover>
   ```

**Files to Modify:**
- `inventory/src/components/product-import/steps/ValidationStep/components/ValidationTable.tsx` - `PriceColumnHeader` component

---

## 9. Debug Mode - Skip API Submission ✅ IMPLEMENTED

**Issue:** Add a third switch in the footer of image upload step (visible only to users with `admin:debug` permission) that will not submit data to any API, only complete the process and show results page as if it had worked.

**Implementation (Completed):**
- Added `skipApiSubmission` state to `ImageUploadStep.tsx`
- Added amber-colored "Skip API (Debug)" switch (visible only with `admin:debug` permission)
- When skip is active, "Use Test API" and "Use Test Database" switches are hidden
- Added `skipApiSubmission?: boolean` to `SubmitOptions` type in `types.ts`
- In `Import.tsx`, when `skipApiSubmission` is true:
  - Skips the actual API call entirely
  - Generates mock success response with mock PIDs
  - Shows `[DEBUG]` prefix in toast and result message
  - Displays results page as if submission succeeded

**Files Modified:**
- `inventory/src/components/product-import/types.ts` - Added `skipApiSubmission` to `SubmitOptions`
- `inventory/src/components/product-import/steps/ImageUploadStep/ImageUploadStep.tsx` - Added switch UI
- `inventory/src/pages/Import.tsx` - Added skip logic in `handleData()`

---

## Summary

| # | Enhancement | Complexity | Status |
|---|-------------|------------|--------|
| 1 | Strip UPC quotes/spaces | Low | ✅ Implemented |
| 2 | AI context in validation | Medium | ✅ Implemented |
| 3 | Fresh taxonomy per session | Medium | ✅ Implemented |
| 4 | Save template from confirmation | Medium-High | ✅ Implemented |
| 5 | Sheet preview | Low-Medium | ✅ Implemented |
| 6 | Remove empty rows | Low | ✅ Implemented |
| 7 | Unit conversion | Medium | ✅ Implemented |
| 8 | MSRP multiplier options | Medium | ✅ Implemented |
| 9 | Debug skip API | Low-Medium | ✅ Implemented |

**Implemented:** 9 of 9 items - All enhancements complete!

---

*Document generated: 2026-01-25*