diff --git a/docs/reference/api.md b/docs/reference/api.md index d90f41129..9b4b6bb75 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -524,15 +524,17 @@ Router: `main/routes/print-templates.ts`. Full path: `/api/print-templates`. Router: `main/routes/menu-csv.ts`. Full path: `/api/menu-csv`. +Category, product, and add-on exports include `flo_csv_encoding=quote-v1` for reversible formula protection; see [product invariants](product-invariants.md#catalog-csv-encoding-and-legacy-apostrophe-ambiguity) for marked, unmarked legacy, and unsupported encoding rules. + | Method | Path | Authorization | Parameters | Response | | --- | --- | --- | --- | --- | -| `GET` | `/template/:type` | `ROLE_ACCESS.ownerManager` | path: `type` | CSV header template for `categories`, `products`, or `addons`. | -| `GET` | `/export/categories` | `ROLE_ACCESS.ownerManager` | none | `text/csv` attachment `categories-export.csv`. | -| `GET` | `/export/products` | `ROLE_ACCESS.ownerManager` | none | `text/csv` attachment `products-export.csv`. | -| `GET` | `/export/addons` | `ROLE_ACCESS.ownerManager` | none | `text/csv` attachment `addons-export.csv`. | -| `POST` | `/import/categories` | `ROLE_ACCESS.ownerManager` | body: `csv` | Body `csv` is the raw file text. Per-row validation errors are reported in the response. | -| `POST` | `/import/products` | `ROLE_ACCESS.ownerManager` | body: `csv` | Body `csv` is the raw file text. A supplied `id` matching a non-deleted product updates it; a missing or unrecognized `id` creates a product with a generated id unless the name and category already exist, in which case the row is skipped. Returns `created`, `updated`, `reactivated`, `skipped`, and `failed` counts plus per-row `errors`. | -| `POST` | `/import/addons` | `ROLE_ACCESS.ownerManager` | body: `csv` | - | +| `GET` | `/template/:type` | `catalog.import-export` | path: `type` | CSV header template for `categories`, `products`, or `addons`. | +| `GET` | `/export/categories` | `catalog.import-export` | none | `text/csv` attachment `categories-export.csv` with `flo_csv_encoding=quote-v1`. | +| `GET` | `/export/products` | `catalog.import-export` | none | `text/csv` attachment `products-export.csv` with `flo_csv_encoding=quote-v1`. | +| `GET` | `/export/addons` | `catalog.import-export` | none | `text/csv` attachment `addons-export.csv` with `flo_csv_encoding=quote-v1`. | +| `POST` | `/import/categories` | `catalog.import-export` | body: `csv` | Body `csv` is the raw file text. Decodes `quote-v1` rows, preserves unmarked category names, and reports per-row validation errors in the response. | +| `POST` | `/import/products` | `catalog.import-export` | body: `csv` | Body `csv` is the raw file text. A supplied `id` matching a non-deleted product updates it; a missing or unrecognized `id` creates a product with a generated id unless the name and category already exist, in which case the row is skipped. Decodes `quote-v1` rows and ID-proven legacy product fields. Returns `created`, `updated`, `reactivated`, `skipped`, and `failed` counts plus per-row `errors`. | +| `POST` | `/import/addons` | `catalog.import-export` | body: `csv` | Body `csv` is the raw file text. Decodes `quote-v1` rows and ID-proven legacy add-on fields, and reports counts plus per-row `errors`. | ### Settings diff --git a/docs/reference/daily-sales-export.md b/docs/reference/daily-sales-export.md index 97399bc93..193213153 100644 --- a/docs/reference/daily-sales-export.md +++ b/docs/reference/daily-sales-export.md @@ -15,7 +15,7 @@ day as a single XLSX workbook or a two-file CSV pair. |-------|------|----------------| | Dataset builder | `main/services/daily-sales-export.ts` | Single source of truth for all accounting: `buildDailySalesExportDataset`, `dailySalesSummaryMetricRows`, `DAILY_SALES_ITEM_COLUMNS`. | | Serializers | `main/services/daily-sales-export-files.ts` | Encode only - no accounting math. `serializeDailySalesExportXlsx`, `serializeDailySalesExportCsv`, `dailySalesExportFilename`. | -| Shared CSV helper | `main/lib/csv.ts` | `toCsvRow` with CWE-1236 formula neutralization (leading quote on `= + - @` non-numerics). Also used by `main/routes/menu-csv.ts`. | +| Shared CSV helper | `main/lib/csv.ts` | `toCsvRow` with CWE-1236 formula neutralization (leading quote on `= + - @` non-numerics); `main/routes/menu-csv.ts` wraps it via `toQuoteV1CsvRow` (see [product invariants](product-invariants.md#catalog-csv-encoding-and-legacy-apostrophe-ambiguity)). | | Endpoint | `main/routes/reports.ts` | Validation, serialization, headers, error mapping (400 / 401 / 403 / 409 / 500). | | Frontend | `frontend/src/app/(dashboard)/dashboard/page.tsx` | Blob download only; two buttons shown when `periodMode === 'day'`. | diff --git a/docs/reference/product-invariants.md b/docs/reference/product-invariants.md index 11fb7b039..0b4692975 100644 --- a/docs/reference/product-invariants.md +++ b/docs/reference/product-invariants.md @@ -418,3 +418,47 @@ address that looks complete is worse than one holding a marked one. product migration because store-local overrides live in sparse tables. It would also have to address `customers.view`, which is the wider exposure. Raise it as its own issue rather than narrowing one print route. + +--- + +## Catalog CSV encoding and legacy apostrophe ambiguity + +**Rule:** Catalog CSV exports (`categories`, `products`, and `addons`) carry a `flo_csv_encoding` +column set to `quote-v1`. Under `quote-v1`, an original leading apostrophe (`'`) on a non-numeric +field is doubled before `toCsvRow` applies spreadsheet formula protection (prefixing `'` onto +non-numeric values starting with `=`, `+`, `-`, `@`, or leading whitespace before those +characters). + +1. **Marked imports (`flo_csv_encoding=quote-v1`).** The importer removes the single apostrophe + added by `quote-v1` encoding (either from doubling an original leading `'` or from formula + protection), making repeated export and import cycles lossless across formula-leading values, + leading whitespace before formula triggers, and genuine leading apostrophes. +2. **Unmarked legacy imports (missing or empty `flo_csv_encoding`).** The `quote-v1` decoding rule + is never applied to unmarked rows. On product or add-on rows carrying a stable ID that matches an + existing record, a leading apostrophe is removed only when comparison with that exact existing + record's stored value proves the apostrophe was added by formula protection. All other unmarked + values - including category-only rows without proof of identity, rows without a matching ID, and + existing records whose stored values already begin with `'` - are preserved as-is. +3. **Unsupported markers.** Any non-empty `flo_csv_encoding` value other than `quote-v1` is rejected + as a per-row import validation error. + +**Reason:** Pre-`quote-v1` exports prefixed formula-triggering strings with `'` without escaping +genuine leading apostrophes, making `'=foo` in an unmarked file ambiguous between an escaped formula +and a literal apostrophe. Stripping leading apostrophes unconditionally on unmarked files would +corrupt genuine apostrophes, while leaving new exports unmarked would mutate formula-leading names +and spawn duplicate categories and add-ons on round-trip imports. + +**Enforced by:** `toQuoteV1CsvRow()`, `encodeQuoteV1Field()`, `decodeQuoteV1Field()`, and +`decodeLegacyProvenField()` in [`main/lib/csv.ts`](../../main/lib/csv.ts), and the export and import +handlers in [`main/routes/menu-csv.ts`](../../main/routes/menu-csv.ts). Other CSV exporters continue +using `toCsvRow()` directly. + +**How to verify:** + +```sh +npm run test:issue-248-csv +``` + +**Change policy:** changing the `quote-v1` encoding rules or stripping unmarked apostrophes without +existing-record proof requires an explicit product decision, because either change can silently +rename catalog items or create duplicate categories and add-ons during CSV imports. diff --git a/main/lib/csv.ts b/main/lib/csv.ts index 077d991ce..38c54c253 100644 --- a/main/lib/csv.ts +++ b/main/lib/csv.ts @@ -1,3 +1,9 @@ +export const FLO_CSV_ENCODING_COLUMN = 'flo_csv_encoding'; +export const FLO_CSV_ENCODING_QUOTE_V1 = 'quote-v1'; + +const FORMULA_TRIGGER_PREFIX = /^\s*[=+\-@]/; +const QUOTE_V1_ADDED_APOSTROPHE = /^'(?:'|\s*[=+\-@])/; + /** CSV field serializer with spreadsheet formula neutralization (CWE-1236). */ export function toCsvRow(fields: (string | number | null | undefined)[]): string { return fields @@ -7,7 +13,7 @@ export function toCsvRow(fields: (string | number | null | undefined)[]): string // with a leading quote. Leading whitespace counts as part of the prefix: // importers strip it before deciding the cell is a formula, so "\t=1+1" // evaluates just like "=1+1" (CWE-1236). - if (typeof f !== 'number' && /^\s*[=+\-@]/.test(s)) { + if (typeof f !== 'number' && FORMULA_TRIGGER_PREFIX.test(s)) { s = "'" + s; } return s.includes(',') || s.includes('"') || s.includes('\n') || s.includes('\r') @@ -16,3 +22,41 @@ export function toCsvRow(fields: (string | number | null | undefined)[]): string }) .join(','); } + +/** Doubles an original leading apostrophe before `toCsvRow` formula neutralization. */ +export function encodeQuoteV1Field( + field: string | number | null | undefined, +): string | number | null | undefined { + if (typeof field === 'number' || field === null || field === undefined) { + return field; + } + const s = String(field); + return s.startsWith("'") ? `'${s}` : s; +} + +/** Serializes a CSV row using reversible `quote-v1` escaping plus spreadsheet formula neutralization. */ +export function toQuoteV1CsvRow(fields: (string | number | null | undefined)[]): string { + return toCsvRow(fields.map(encodeQuoteV1Field)); +} + +/** Removes the single apostrophe added by `quote-v1` encoding. */ +export function decodeQuoteV1Field(value: string): string { + return QUOTE_V1_ADDED_APOSTROPHE.test(value) ? value.slice(1) : value; +} + +/** + * Decodes an unmarked legacy CSV field only when comparison with an existing + * record's stored value proves the leading apostrophe was added by formula neutralization. + */ +export function decodeLegacyProvenField( + csvValue: string, + storedValue: string | null | undefined, +): string { + if (!csvValue || typeof storedValue !== 'string' || !storedValue) { + return csvValue; + } + if (FORMULA_TRIGGER_PREFIX.test(storedValue) && csvValue === `'${storedValue}`.trim()) { + return storedValue; + } + return csvValue; +} diff --git a/main/routes/menu-csv.ts b/main/routes/menu-csv.ts index 9d8a2c5a4..78ee75c0e 100644 --- a/main/routes/menu-csv.ts +++ b/main/routes/menu-csv.ts @@ -13,10 +13,56 @@ import { resolveRegionalSnapshot, type RegionalSnapshot, } from '../countries'; -import { toCsvRow } from '../lib/csv'; +import { + FLO_CSV_ENCODING_COLUMN, + FLO_CSV_ENCODING_QUOTE_V1, + decodeLegacyProvenField, + decodeQuoteV1Field, + toQuoteV1CsvRow, +} from '../lib/csv'; const router = Router(); +const TEXT_HEADER_FIELDS = new Set([ + 'name', + 'description', + 'color', + 'icon', + 'sku', + 'category', + 'tags', + 'group_name', + 'addon_name', +]); + +type CsvRowEncodingResult = + | { ok: true; mode: 'quote-v1' | 'unmarked' } + | { ok: false; error: string }; + +function resolveCsvRowEncoding(row: Record): CsvRowEncodingResult { + const marker = row[FLO_CSV_ENCODING_COLUMN] ?? ''; + if (marker === '') { + return { ok: true, mode: 'unmarked' }; + } + if (marker === FLO_CSV_ENCODING_QUOTE_V1) { + return { ok: true, mode: 'quote-v1' }; + } + return { + ok: false, + error: `unsupported ${FLO_CSV_ENCODING_COLUMN} "${marker}"`, + }; +} + +function formatProductTagsForCsv(rawTags: string | null | undefined): string { + if (!rawTags) return ''; + try { + const parsed = JSON.parse(rawTags); + return Array.isArray(parsed) ? parsed.join(',') : String(rawTags); + } catch { + return String(rawTags); + } +} + function captureInvalidMenuCsvInput(): void { const failure = serializeOperationFailure({ capture_point: 'fc.backend.menu_csv.import.invalid_input', @@ -195,7 +241,11 @@ function toObjects(rows: string[][]): Record[] { } const obj: Record = {}; headers.forEach((h, i) => { - obj[h] = (row[i] ?? '').trim(); + const rawCell = row[i] ?? ''; + obj[h] = + TEXT_HEADER_FIELDS.has(h) && /^\s+[=+\-@]/.test(rawCell) + ? rawCell.trimEnd() + : rawCell.trim(); }); return obj; }); @@ -358,9 +408,18 @@ router.get( const rows = db .prepare('SELECT * FROM categories WHERE deleted_at IS NULL ORDER BY sort_order, name') .all() as any[]; - const lines = ['name,description,color,icon,sort_order']; + const lines = [`name,description,color,icon,sort_order,${FLO_CSV_ENCODING_COLUMN}`]; for (const c of rows) - lines.push(toCsvRow([c.name, c.description, c.color, c.icon, c.sort_order])); + lines.push( + toQuoteV1CsvRow([ + c.name, + c.description, + c.color, + c.icon, + c.sort_order, + FLO_CSV_ENCODING_QUOTE_V1, + ]), + ); res.setHeader('Content-Type', 'text/csv'); res.setHeader('Content-Disposition', 'attachment; filename="categories-export.csv"'); res.send(lines.join('\n')); @@ -388,20 +447,12 @@ router.get( ) .all() as any[]; const lines = [ - 'id,sku,name,category,price,description,cost,tax_category,tax_behavior,cashback_percent,tags,is_active', + `id,sku,name,category,price,description,cost,tax_category,tax_behavior,cashback_percent,tags,is_active,${FLO_CSV_ENCODING_COLUMN}`, ]; for (const p of rows) { - let tags = ''; - if (p.tags) { - try { - const t = JSON.parse(p.tags); - tags = Array.isArray(t) ? t.join(',') : p.tags; - } catch { - tags = p.tags; - } - } + const tags = formatProductTagsForCsv(p.tags); lines.push( - toCsvRow([ + toQuoteV1CsvRow([ p.id, p.sku, p.name, @@ -414,6 +465,7 @@ router.get( p.cb_percent !== null ? p.cb_percent : '', tags, p.is_active ? 'yes' : 'no', + FLO_CSV_ENCODING_QUOTE_V1, ]), ); } @@ -440,7 +492,7 @@ router.get( .prepare('SELECT * FROM addon_groups WHERE is_active = 1 ORDER BY sort_order, name') .all() as any[]; const lines = [ - 'group_name,addon_name,price,group_required,group_min_select,group_max_select,track_inventory,stock_quantity,low_stock_threshold', + `group_name,addon_name,price,group_required,group_min_select,group_max_select,track_inventory,stock_quantity,low_stock_threshold,${FLO_CSV_ENCODING_COLUMN}`, ]; for (const g of groups) { const addons = db @@ -450,7 +502,7 @@ router.get( .all(g.id) as any[]; for (const a of addons) lines.push( - toCsvRow([ + toQuoteV1CsvRow([ g.name, a.name, formatAmountForCsv(a.price, regionalSnapshot), @@ -460,6 +512,7 @@ router.get( a.track_inventory ? 'yes' : 'no', Number(a.stock_quantity ?? 0), Number(a.low_stock_threshold ?? 0), + FLO_CSV_ENCODING_QUOTE_V1, ]), ); } @@ -505,7 +558,23 @@ router.post( db.transaction(() => { for (let i = 0; i < rows.length; i++) { - const r = rows[i]; + const rawRow = rows[i]; + const encoding = resolveCsvRowEncoding(rawRow); + if (!encoding.ok) { + failed++; + errors.push(`Row ${i + 2}: ${encoding.error}`); + continue; + } + const r: Record = + encoding.mode === 'quote-v1' + ? { + ...rawRow, + name: decodeQuoteV1Field(rawRow.name ?? ''), + description: decodeQuoteV1Field(rawRow.description ?? ''), + color: decodeQuoteV1Field(rawRow.color ?? ''), + icon: decodeQuoteV1Field(rawRow.icon ?? ''), + } + : rawRow; if (!r.name) { failed++; errors.push(`Row ${i + 2}: missing name`); @@ -606,9 +675,79 @@ router.post( changedRows = 0; const errors: string[] = []; + const findExistingProduct = db.prepare( + `SELECT p.id, p.name, p.sku, p.description, p.tags, p.is_active, c.name AS category_name + FROM products p + LEFT JOIN categories c ON p.category_id = c.id + WHERE p.id = ? AND p.deleted_at IS NULL`, + ); + db.transaction(() => { for (let i = 0; i < rows.length; i++) { - const r = rows[i]; + const rawRow = rows[i]; + const encoding = resolveCsvRowEncoding(rawRow); + if (!encoding.ok) { + failed++; + errors.push(`Row ${i + 2}: ${encoding.error}`); + continue; + } + + let r: Record; + let existingProduct: + | { + id: string; + name: string; + sku: string | null; + description: string | null; + tags: string | null; + is_active: number; + category_name: string | null; + } + | undefined; + + if (encoding.mode === 'quote-v1') { + r = { + ...rawRow, + id: decodeQuoteV1Field(rawRow.id ?? ''), + sku: decodeQuoteV1Field(rawRow.sku ?? ''), + name: decodeQuoteV1Field(rawRow.name ?? ''), + category: decodeQuoteV1Field(rawRow.category ?? ''), + description: decodeQuoteV1Field(rawRow.description ?? ''), + tax_category: decodeQuoteV1Field(rawRow.tax_category ?? ''), + tax_behavior: decodeQuoteV1Field(rawRow.tax_behavior ?? ''), + tags: decodeQuoteV1Field(rawRow.tags ?? ''), + }; + if (r.id) { + existingProduct = findExistingProduct.get(r.id) as typeof existingProduct; + } + } else { + if (rawRow.id) { + existingProduct = findExistingProduct.get(rawRow.id) as typeof existingProduct; + } + if (existingProduct) { + r = { + ...rawRow, + id: existingProduct.id, + sku: decodeLegacyProvenField(rawRow.sku ?? '', existingProduct.sku), + name: decodeLegacyProvenField(rawRow.name ?? '', existingProduct.name), + category: decodeLegacyProvenField( + rawRow.category ?? '', + existingProduct.category_name, + ), + description: decodeLegacyProvenField( + rawRow.description ?? '', + existingProduct.description, + ), + tags: decodeLegacyProvenField( + rawRow.tags ?? '', + formatProductTagsForCsv(existingProduct.tags), + ), + }; + } else { + r = rawRow; + } + } + if (!r.name) { failed++; errors.push(`Row ${i + 2}: missing name`); @@ -655,7 +794,9 @@ router.post( if (r.tags) { const arr = r.tags .split(',') - .map((t: string) => t.trim()) + // quote-v1 round-trips stored tags verbatim; trimming would strip + // the restored formula-safety whitespace and break reversibility. + .map((t: string) => (encoding.mode === 'quote-v1' ? t : t.trim())) .filter(Boolean); if (arr.length) tagsJson = JSON.stringify(arr); } @@ -711,42 +852,37 @@ router.post( } // If an id is provided and matches an existing product, update it. - if (r.id) { - const existing = db - .prepare('SELECT id, is_active FROM products WHERE id = ? AND deleted_at IS NULL') - .get(r.id) as { id: string; is_active: number } | undefined; - if (existing) { - db.prepare( - `UPDATE products SET name=?, category_id=?, price=?, description=?, cost=?, + if (existingProduct) { + db.prepare( + `UPDATE products SET name=?, category_id=?, price=?, description=?, cost=?, tax_type=?, tax_rate=?, tax_category_id=CASE WHEN ? = 1 THEN ? ELSE tax_category_id END, tax_behavior=CASE WHEN ? = 1 THEN ? ELSE tax_behavior END, cb_percent=?, tags=?, is_active=?, sku=?, updated_at=? WHERE id=?`, - ).run( - r.name, - categoryId, - price, - r.description || null, - cost, - 'none', - 0, - hasTaxCategoryColumn ? 1 : 0, - taxCategoryId, - hasTaxBehaviorColumn ? 1 : 0, - taxBehavior, - cbPercent, - tagsJson, - isActive, - sku, - now(), - r.id, - ); - if (existing.is_active === 0 && isActive === 1) reactivated++; - else updated++; - changedRows++; - continue; - } + ).run( + r.name, + categoryId, + price, + r.description || null, + cost, + 'none', + 0, + hasTaxCategoryColumn ? 1 : 0, + taxCategoryId, + hasTaxBehaviorColumn ? 1 : 0, + taxBehavior, + cbPercent, + tagsJson, + isActive, + sku, + now(), + existingProduct.id, + ); + if (existingProduct.is_active === 0 && isActive === 1) reactivated++; + else updated++; + changedRows++; + continue; } // No id or unrecognized id — insert as new, skip if name+category duplicate. @@ -814,8 +950,8 @@ router.post( const hasTrackInventoryColumn = headers.has('track_inventory'); const hasStockQuantityColumn = headers.has('stock_quantity'); const hasLowStockThresholdColumn = headers.has('low_stock_threshold'); - const rows = toObjects(parsedCsv); - if (!rows.length) { + const rawRows = toObjects(parsedCsv); + if (!rawRows.length) { captureInvalidMenuCsvInput(); return res.status(400).json({ error: 'CSV has no data rows' }); } @@ -835,6 +971,63 @@ router.post( changedRows = 0; const errors: string[] = []; const groupCache: Record = {}; + + const findExistingAddon = db.prepare( + `SELECT a.id, a.name AS addon_name, a.addon_group_id, g.name AS group_name + FROM addons a + JOIN addon_groups g ON g.id = a.addon_group_id + WHERE a.id = ?`, + ); + const findExistingGroupById = db.prepare('SELECT id, name FROM addon_groups WHERE id = ?'); + + const preparedRows = rawRows.map( + (rawRow): { encodingError: string | null; row: Record } => { + const encoding = resolveCsvRowEncoding(rawRow); + if (!encoding.ok) { + return { encodingError: encoding.error, row: rawRow }; + } + if (encoding.mode === 'quote-v1') { + return { + encodingError: null, + row: { + ...rawRow, + group_name: decodeQuoteV1Field(rawRow.group_name ?? ''), + addon_name: decodeQuoteV1Field(rawRow.addon_name ?? ''), + }, + }; + } + let groupName = rawRow.group_name ?? ''; + let addonName = rawRow.addon_name ?? ''; + const addonId = rawRow.id || rawRow.addon_id || ''; + const groupId = rawRow.group_id || rawRow.addon_group_id || ''; + if (addonId) { + const existingAddon = findExistingAddon.get(addonId) as + | { id: string; addon_name: string; addon_group_id: string; group_name: string } + | undefined; + if (existingAddon) { + groupName = decodeLegacyProvenField(groupName, existingAddon.group_name); + addonName = decodeLegacyProvenField(addonName, existingAddon.addon_name); + } + } + if (groupId) { + const existingGroup = findExistingGroupById.get(groupId) as + | { id: string; name: string } + | undefined; + if (existingGroup) { + groupName = decodeLegacyProvenField(groupName, existingGroup.name); + } + } + return { + encodingError: null, + row: { + ...rawRow, + group_name: groupName, + addon_name: addonName, + }, + }; + }, + ); + type AddonGroupImportPlan = { existing?: { id: string; @@ -852,7 +1045,8 @@ router.post( }; const groupPlans = new Map(); - for (const row of rows) { + for (const { encodingError, row } of preparedRows) { + if (encodingError) continue; if (!row.group_name || !row.addon_name) continue; const priceResult = parseNumericField(row.price, 'price', { @@ -940,8 +1134,13 @@ router.post( } db.transaction(() => { - for (let i = 0; i < rows.length; i++) { - const r = rows[i]; + for (let i = 0; i < preparedRows.length; i++) { + const { encodingError, row: r } = preparedRows[i]; + if (encodingError) { + failed++; + errors.push(`Row ${i + 2}: ${encodingError}`); + continue; + } let rowChanged = false; if (!r.group_name || !r.addon_name) { failed++; diff --git a/tests/issue-248-menu-csv.test.ts b/tests/issue-248-menu-csv.test.ts index 7d456fb44..ad40c97fd 100644 --- a/tests/issue-248-menu-csv.test.ts +++ b/tests/issue-248-menu-csv.test.ts @@ -817,6 +817,727 @@ async function main() { tooLongCell.data.error.includes('length'), 'value-bound error identifies the resource limit', ); + + console.log('\n─── Multi-cycle quote-v1 export/import round trips ───'); + const roundTripCategories = [ + { + id: 'cat-rt-eq', + name: '=Equals Category', + desc: '-Minus description', + color: 'blue', + icon: '☕', + sort: 10, + }, + { + id: 'cat-rt-plus', + name: '+Plus, Category', + desc: '@At description,\nwith newline', + color: 'green', + icon: '🍔', + sort: 11, + }, + { + id: 'cat-rt-minus', + name: '-Minus Category', + desc: '=Formula desc', + color: 'pink', + icon: '🍰', + sort: 12, + }, + { + id: 'cat-rt-at', + name: '@At Category', + desc: '+Plus desc', + color: 'amber', + icon: '🎁', + sort: 13, + }, + { + id: 'cat-rt-ws', + name: ' =Space Formula Category', + desc: '\t+Tab Formula Desc', + color: 'blue', + icon: '☕', + sort: 14, + }, + { + id: 'cat-rt-quote', + name: "'Chef's Category", + desc: '\'Genuine quote, "quoted"\r\ndesc', + color: 'green', + icon: '🍔', + sort: 15, + }, + { + id: 'cat-rt-qformula', + name: "'+Quoted Formula Category", + desc: "''Double Quoted Desc", + color: 'pink', + icon: '🍰', + sort: 16, + }, + ]; + for (const c of roundTripCategories) { + db.prepare( + `INSERT INTO categories (id, name, slug, description, color, icon, sort_order, is_active, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)`, + ).run(c.id, c.name, c.id, c.desc, c.color, c.icon, c.sort, now(), now()); + } + + const roundTripProducts = [ + { + id: 'prod-rt-eq', + catId: 'cat-rt-eq', + sku: '=SKU-EQ', + name: '=Combo Deal', + price: 11, + desc: '+Rich, "bold"\nflavor', + cost: 4, + tags: ['=tag-eq', 'featured'], + }, + { + id: 'prod-rt-plus', + catId: 'cat-rt-plus', + sku: '+SKU-PLUS', + name: '+Special Curry, Large', + price: 14.5, + desc: '-Spiced\r\ncurry', + cost: 5, + tags: ['+tag-plus'], + }, + { + id: 'prod-rt-minus', + catId: 'cat-rt-minus', + sku: '-SKU-MINUS', + name: '-Discount Tea', + price: 6, + desc: '@Tea note', + cost: 2, + tags: ['-tag-minus'], + }, + { + id: 'prod-rt-at', + catId: 'cat-rt-at', + sku: '@SKU-AT', + name: '@House Special', + price: 18, + desc: '=House note', + cost: 7, + tags: ['@tag-at'], + }, + { + id: 'prod-rt-ws-space', + catId: 'cat-rt-ws', + sku: ' +SKU-SPACE', + name: ' =Space Formula Roast', + price: 9, + desc: ' -Space desc', + cost: 3, + tags: ['veg'], + }, + { + id: 'prod-rt-ws-tab', + catId: 'cat-rt-ws', + sku: '\t@SKU-TAB', + name: '\t+Tab Formula Espresso', + price: 9.5, + desc: '\n=Newline Formula Desc', + cost: 3, + tags: ['veg'], + }, + { + id: 'prod-rt-quote', + catId: 'cat-rt-quote', + sku: "'SKU-QUOTE", + name: '\'Chef\'s Signature, "Hot"\nBurger', + price: 21, + desc: "'Authentic recipe", + cost: 8, + tags: ["'chef-tag"], + }, + { + id: 'prod-rt-qformula', + catId: 'cat-rt-qformula', + sku: "'+SKU-QF", + name: "'+Literal Escaped Plus", + price: 16, + desc: "''Double Apostrophe Note", + cost: 6, + tags: ["'+literal-tag"], + }, + { + id: 'prod-rt-qws', + catId: 'cat-rt-qformula', + sku: "' =SKU-QWS", + name: "' =Literal Space Formula", + price: 17, + desc: "''@Double Quote At", + cost: 6, + tags: ['veg'], + }, + ]; + for (const p of roundTripProducts) { + db.prepare( + `INSERT INTO products (id, name, category_id, price, description, cost, tax_type, tax_rate, + cb_percent, tags, is_active, sku, sort_order, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, 'none', 0, NULL, ?, 1, ?, 0, ?, ?)`, + ).run( + p.id, + p.name, + p.catId, + p.price, + p.desc, + p.cost, + JSON.stringify(p.tags), + p.sku, + now(), + now(), + ); + } + + const roundTripAddons = [ + { + groupId: 'ag-rt-eq', + groupName: '=Formula Group', + addonId: 'ad-rt-plus', + addonName: '+Extra Spice, Hot', + price: 2, + }, + { + groupId: 'ag-rt-minus', + groupName: '-Minus Group', + addonId: 'ad-rt-at', + addonName: '@House Drizzle', + price: 3, + }, + { + groupId: 'ag-rt-ws', + groupName: ' =Space Group', + addonId: 'ad-rt-ws', + addonName: '\t+Tab Addon', + price: 1.5, + }, + { + groupId: 'ag-rt-quote', + groupName: "'Chef's Addons", + addonId: 'ad-rt-quote', + addonName: "'Artisan Cheese", + price: 4, + }, + { + groupId: 'ag-rt-qf', + groupName: "'+Quoted Formula Group", + addonId: 'ad-rt-qf', + addonName: "''Double Apostrophe Addon", + price: 2.5, + }, + ]; + for (const a of roundTripAddons) { + db.prepare( + `INSERT INTO addon_groups (id, name, is_required, min_selection, max_selection, is_active, sort_order, created_at, updated_at) + VALUES (?, ?, 0, 0, 1, 1, 0, ?, ?)`, + ).run(a.groupId, a.groupName, now(), now()); + db.prepare( + `INSERT INTO addons (id, addon_group_id, name, price, is_active, sort_order, created_at, updated_at) + VALUES (?, ?, ?, ?, 1, 0, ?, ?)`, + ).run(a.addonId, a.groupId, a.addonName, a.price, now(), now()); + } + db.prepare( + `INSERT INTO addon_group_product (product_id, addon_group_id) VALUES (?, ?), (?, ?)`, + ).run('prod-rt-plus', 'ag-rt-eq', 'prod-rt-quote', 'ag-rt-quote'); + + const initialCatCount = ( + db.prepare('SELECT COUNT(*) AS count FROM categories WHERE deleted_at IS NULL').get() as any + ).count; + const initialProdCount = ( + db.prepare('SELECT COUNT(*) AS count FROM products WHERE deleted_at IS NULL').get() as any + ).count; + const initialGroupCount = ( + db.prepare('SELECT COUNT(*) AS count FROM addon_groups').get() as any + ).count; + const initialAddonCount = (db.prepare('SELECT COUNT(*) AS count FROM addons').get() as any) + .count; + + for (let cycle = 1; cycle <= 3; cycle++) { + const catExportCsv = await fetchText(baseUrl, '/api/menu/csv/export/categories', authHeader); + const prodExportCsv = await fetchText(baseUrl, '/api/menu/csv/export/products', authHeader); + const addonExportCsv = await fetchText(baseUrl, '/api/menu/csv/export/addons', authHeader); + + assertIncludesOrThrow( + catExportCsv.split('\n')[0], + 'flo_csv_encoding', + `cycle ${cycle}: categories export header includes flo_csv_encoding`, + ); + assertIncludesOrThrow( + prodExportCsv.split('\n')[0], + 'flo_csv_encoding', + `cycle ${cycle}: products export header includes flo_csv_encoding`, + ); + assertIncludesOrThrow( + addonExportCsv.split('\n')[0], + 'flo_csv_encoding', + `cycle ${cycle}: addons export header includes flo_csv_encoding`, + ); + + const catImportRes = await api(baseUrl, '/api/menu/csv/import/categories', { + method: 'POST', + body: { csv: catExportCsv }, + headers: authHeader, + }); + assertEqualOrThrow(catImportRes.status, 200, `cycle ${cycle}: categories re-import succeeds`); + assertEqualOrThrow( + catImportRes.data.created, + 0, + `cycle ${cycle}: categories re-import creates no duplicates`, + ); + assertEqualOrThrow( + catImportRes.data.failed, + 0, + `cycle ${cycle}: categories re-import has no failures`, + ); + + const prodImportRes = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { csv: prodExportCsv }, + headers: authHeader, + }); + assertEqualOrThrow(prodImportRes.status, 200, `cycle ${cycle}: products re-import succeeds`); + assertEqualOrThrow( + prodImportRes.data.created, + 0, + `cycle ${cycle}: products re-import creates no duplicates`, + ); + assertEqualOrThrow( + prodImportRes.data.failed, + 0, + `cycle ${cycle}: products re-import has no failures`, + ); + + const addonImportRes = await api(baseUrl, '/api/menu/csv/import/addons', { + method: 'POST', + body: { csv: addonExportCsv }, + headers: authHeader, + }); + assertEqualOrThrow(addonImportRes.status, 200, `cycle ${cycle}: addons re-import succeeds`); + assertEqualOrThrow( + addonImportRes.data.groups_created, + 0, + `cycle ${cycle}: addons re-import creates no duplicate groups`, + ); + assertEqualOrThrow( + addonImportRes.data.addons_created, + 0, + `cycle ${cycle}: addons re-import creates no duplicate addons`, + ); + assertEqualOrThrow( + addonImportRes.data.failed, + 0, + `cycle ${cycle}: addons re-import has no failures`, + ); + } + + assertEqualOrThrow( + (db.prepare('SELECT COUNT(*) AS count FROM categories WHERE deleted_at IS NULL').get() as any) + .count, + initialCatCount, + 'category count is unchanged after 3 export/import cycles', + ); + assertEqualOrThrow( + (db.prepare('SELECT COUNT(*) AS count FROM products WHERE deleted_at IS NULL').get() as any) + .count, + initialProdCount, + 'product count is unchanged after 3 export/import cycles', + ); + assertEqualOrThrow( + (db.prepare('SELECT COUNT(*) AS count FROM addon_groups').get() as any).count, + initialGroupCount, + 'addon_group count is unchanged after 3 export/import cycles', + ); + assertEqualOrThrow( + (db.prepare('SELECT COUNT(*) AS count FROM addons').get() as any).count, + initialAddonCount, + 'addon count is unchanged after 3 export/import cycles', + ); + + for (const c of roundTripCategories) { + const row = db + .prepare('SELECT name, description FROM categories WHERE id = ?') + .get(c.id) as any; + assertEqualOrThrow(row.name, c.name, `category ${c.id} name is preserved across round trips`); + assertEqualOrThrow( + row.description, + c.desc, + `category ${c.id} description is preserved across round trips`, + ); + } + + for (const p of roundTripProducts) { + const row = db + .prepare('SELECT name, sku, description, category_id, tags FROM products WHERE id = ?') + .get(p.id) as any; + assertEqualOrThrow(row.name, p.name, `product ${p.id} name is preserved across round trips`); + assertEqualOrThrow(row.sku, p.sku, `product ${p.id} sku is preserved across round trips`); + assertEqualOrThrow( + row.description, + p.desc, + `product ${p.id} description is preserved across round trips`, + ); + assertEqualOrThrow( + row.category_id, + p.catId, + `product ${p.id} category_id association is preserved`, + ); + assertEqualOrThrow( + row.tags, + JSON.stringify(p.tags), + `product ${p.id} tags are preserved across round trips`, + ); + } + + for (const a of roundTripAddons) { + const groupRow = db + .prepare('SELECT name FROM addon_groups WHERE id = ?') + .get(a.groupId) as any; + const addonRow = db + .prepare('SELECT name, addon_group_id FROM addons WHERE id = ?') + .get(a.addonId) as any; + assertEqualOrThrow(groupRow.name, a.groupName, `addon group ${a.groupId} name is preserved`); + assertEqualOrThrow(addonRow.name, a.addonName, `addon ${a.addonId} name is preserved`); + assertEqualOrThrow( + addonRow.addon_group_id, + a.groupId, + `addon ${a.addonId} group association is preserved`, + ); + } + + const preservedLinks = db + .prepare( + 'SELECT COUNT(*) AS count FROM addon_group_product WHERE (product_id = ? AND addon_group_id = ?) OR (product_id = ? AND addon_group_id = ?)', + ) + .get('prod-rt-plus', 'ag-rt-eq', 'prod-rt-quote', 'ag-rt-quote') as any; + assertEqualOrThrow( + preservedLinks.count, + 2, + 'product-to-addon-group associations remain stable', + ); + + console.log('\n─── Unmarked legacy imports: decode only when proven by stable ID ───'); + // 1. Existing product with formula-leading fields: unmarked legacy import with matching ID decodes proven fields. + const legacyProvenProductImport = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { + csv: productCsv( + `prod-rt-plus,'+SKU-PLUS,"'+Special Curry, Large","'+Plus, Category",19,"'-Spiced\r\ncurry",5,,,,'+tag-plus,yes`, + `prod-rt-ws-space,' +SKU-SPACE,' =Space Formula Roast,' =Space Formula Category,11,' -Space desc,3,,,,veg,yes`, + ), + }, + headers: authHeader, + }); + assertEqualOrThrow( + legacyProvenProductImport.status, + 200, + 'unmarked legacy product import with proven IDs succeeds', + ); + assertEqualOrThrow( + legacyProvenProductImport.data.updated, + 2, + 'proven legacy product rows update existing records', + ); + assertEqualOrThrow( + legacyProvenProductImport.data.failed, + 0, + 'proven legacy product rows do not fail', + ); + const provenPlusProd = db + .prepare('SELECT name, sku, description, price, category_id, tags FROM products WHERE id = ?') + .get('prod-rt-plus') as any; + assertEqualOrThrow( + provenPlusProd.name, + '+Special Curry, Large', + 'proven legacy product name is decoded', + ); + assertEqualOrThrow(provenPlusProd.sku, '+SKU-PLUS', 'proven legacy product sku is decoded'); + assertEqualOrThrow( + provenPlusProd.description, + '-Spiced\r\ncurry', + 'proven legacy product description is decoded', + ); + assertEqualOrThrow(provenPlusProd.price, 19, 'proven legacy product price is updated'); + assertEqualOrThrow( + provenPlusProd.category_id, + 'cat-rt-plus', + 'proven legacy product category association is preserved', + ); + assertEqualOrThrow( + provenPlusProd.tags, + JSON.stringify(['+tag-plus']), + 'proven legacy product tags are decoded', + ); + + const provenSpaceProd = db + .prepare('SELECT name, sku, description FROM products WHERE id = ?') + .get('prod-rt-ws-space') as any; + assertEqualOrThrow( + provenSpaceProd.name, + ' =Space Formula Roast', + 'proven legacy whitespace-formula product name is decoded', + ); + assertEqualOrThrow( + provenSpaceProd.sku, + ' +SKU-SPACE', + 'proven legacy whitespace-formula product sku is decoded', + ); + assertEqualOrThrow( + provenSpaceProd.description, + ' -Space desc', + 'proven legacy whitespace-formula product description is decoded', + ); + + // 2. Existing product whose stored name already begins with an apostrophe (ambiguous record): must NOT be renamed! + const legacyAmbiguousExistingProd = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { + csv: productCsv( + `prod-rt-qformula,'+SKU-QF,'+Literal Escaped Plus,'+Quoted Formula Category,22,''Double Apostrophe Note,6,,,,'+literal-tag,yes`, + ), + }, + headers: authHeader, + }); + assertEqualOrThrow( + legacyAmbiguousExistingProd.status, + 200, + 'unmarked import of existing ambiguous product succeeds', + ); + assertEqualOrThrow( + legacyAmbiguousExistingProd.data.updated, + 1, + 'existing ambiguous product is updated', + ); + const ambiguousProdRow = db + .prepare('SELECT name, sku, price FROM products WHERE id = ?') + .get('prod-rt-qformula') as any; + assertEqualOrThrow( + ambiguousProdRow.name, + "'+Literal Escaped Plus", + 'existing ambiguous product name is not stripped or renamed', + ); + assertEqualOrThrow( + ambiguousProdRow.sku, + "'+SKU-QF", + 'existing ambiguous product sku is not stripped', + ); + assertEqualOrThrow( + ambiguousProdRow.price, + 22, + 'existing ambiguous product price updates normally', + ); + + // 3. Unmarked product rows without ID or with unrecognized ID: preserve ambiguous leading apostrophes. + const legacyUnprovenProd = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { + csv: productCsv( + `,,'+Ambiguous Unmarked New,CSV Category,10,'-Ambiguous Desc,2,,,,,yes`, + `unknown-legacy-id,,'+Ambiguous Unmarked Fallback,CSV Category,12,'=Ambiguous Desc 2,3,,,,,yes`, + ), + }, + headers: authHeader, + }); + assertEqualOrThrow(legacyUnprovenProd.status, 200, 'unmarked unproven product rows import'); + assertEqualOrThrow( + legacyUnprovenProd.data.created, + 2, + 'unmarked unproven products are created', + ); + assert( + db.prepare('SELECT id FROM products WHERE name = ?').get("'+Ambiguous Unmarked New") !== + undefined, + 'unmarked product without ID preserves its leading apostrophe', + ); + assert( + db.prepare('SELECT id FROM products WHERE name = ?').get("'+Ambiguous Unmarked Fallback") !== + undefined, + 'unmarked product with unrecognized ID preserves its leading apostrophe', + ); + + // 4. Unmarked category import: category-only rows have no proof of identity, so ambiguous names are preserved. + const legacyCatImport = await api(baseUrl, '/api/menu/csv/import/categories', { + method: 'POST', + body: { + csv: [ + 'name,description,color,icon,sort_order', + "'=Equals Category,'-Minus description,blue,☕,20", + ].join('\n'), + }, + headers: authHeader, + }); + assertEqualOrThrow(legacyCatImport.status, 200, 'unmarked legacy category import succeeds'); + assertEqualOrThrow( + legacyCatImport.data.created, + 1, + 'unmarked category row without proof of identity preserves its literal leading apostrophe', + ); + assert( + db + .prepare('SELECT id FROM categories WHERE name = ? AND deleted_at IS NULL') + .get("'=Equals Category") !== undefined, + 'ambiguous unmarked category name is preserved as-is', + ); + assert( + db.prepare('SELECT id FROM categories WHERE id = ?').get('cat-rt-eq') !== undefined, + 'original category record remains intact', + ); + + // 5. Unmarked add-on import: proven when stable ID matches existing add-on, preserved when unproven or ambiguous. + const legacyProvenAddonImport = await api(baseUrl, '/api/menu/csv/import/addons', { + method: 'POST', + body: { + csv: addonCsvWithHeader( + 'id,group_name,addon_name,price,group_required,group_min_select,group_max_select', + "ad-rt-at,'-Minus Group,'@House Drizzle,3,yes,0,1", + ",'+Unproven Legacy Group,'@Unproven Legacy Addon,4,no,0,1", + ), + }, + headers: authHeader, + }); + assertEqualOrThrow( + legacyProvenAddonImport.status, + 200, + 'unmarked add-on import with mixed proven/unproven rows succeeds', + ); + assertEqualOrThrow( + legacyProvenAddonImport.data.groups_updated, + 1, + 'ID-proven legacy add-on row updates its existing group', + ); + assertEqualOrThrow( + legacyProvenAddonImport.data.skipped, + 1, + 'ID-proven legacy add-on row matches existing active add-on', + ); + assertEqualOrThrow( + legacyProvenAddonImport.data.groups_created, + 1, + 'unproven legacy add-on row creates its literal group', + ); + assertEqualOrThrow( + legacyProvenAddonImport.data.addons_created, + 1, + 'unproven legacy add-on row creates its literal add-on', + ); + assert( + db.prepare('SELECT id FROM addon_groups WHERE name = ?').get("'+Unproven Legacy Group") !== + undefined, + 'unproven legacy add-on group preserves its leading apostrophe', + ); + assert( + db.prepare('SELECT id FROM addons WHERE name = ?').get("'@Unproven Legacy Addon") !== + undefined, + 'unproven legacy add-on preserves its leading apostrophe', + ); + + console.log('\n─── Unsupported non-empty flo_csv_encoding markers are rejected ───'); + const badCatEncoding = await api(baseUrl, '/api/menu/csv/import/categories', { + method: 'POST', + body: { + csv: [ + 'name,description,color,icon,sort_order,flo_csv_encoding', + 'Rejected Encoding Category,Desc,blue,☕,1,quote-v2', + 'Blank Marker Category,Desc,green,🍔,2,', + ].join('\n'), + }, + headers: authHeader, + }); + assertEqualOrThrow( + badCatEncoding.status, + 200, + 'category import with unsupported encoding returns row validation results', + ); + assertEqualOrThrow( + badCatEncoding.data.failed, + 1, + 'unsupported category encoding marker fails validation', + ); + assertEqualOrThrow( + badCatEncoding.data.created, + 1, + 'row with empty flo_csv_encoding beside unsupported row still imports', + ); + assert( + badCatEncoding.data.errors.some((e: string) => + e.includes('unsupported flo_csv_encoding "quote-v2"'), + ), + 'category validation error reports unsupported flo_csv_encoding', + ); + assertEqualOrThrow( + db.prepare('SELECT id FROM categories WHERE name = ?').get('Rejected Encoding Category'), + undefined, + 'category row with unsupported encoding marker is not persisted', + ); + + const badProdEncoding = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { + csv: [ + `${PRODUCT_HEADER},flo_csv_encoding`, + ',,Rejected Encoding Product,CSV Category,10,Desc,2,,,,,yes,unsupported-v9', + ].join('\n'), + }, + headers: authHeader, + }); + assertEqualOrThrow( + badProdEncoding.status, + 200, + 'product import with unsupported encoding returns row validation results', + ); + assertEqualOrThrow( + badProdEncoding.data.failed, + 1, + 'unsupported product encoding marker fails validation', + ); + assertEqualOrThrow( + badProdEncoding.data.created, + 0, + 'unsupported product encoding marker creates no product', + ); + assert( + badProdEncoding.data.errors.some((e: string) => + e.includes('unsupported flo_csv_encoding "unsupported-v9"'), + ), + 'product validation error reports unsupported flo_csv_encoding', + ); + + const badAddonEncoding = await api(baseUrl, '/api/menu/csv/import/addons', { + method: 'POST', + body: { + csv: addonCsvWithHeader( + `${ADDON_HEADER},flo_csv_encoding`, + 'Rejected Encoding Group,Rejected Encoding Addon,5,no,0,1,legacy-v0', + ), + }, + headers: authHeader, + }); + assertEqualOrThrow( + badAddonEncoding.status, + 200, + 'addon import with unsupported encoding returns row validation results', + ); + assertEqualOrThrow( + badAddonEncoding.data.failed, + 1, + 'unsupported addon encoding marker fails validation', + ); + assertEqualOrThrow( + badAddonEncoding.data.addons_created, + 0, + 'unsupported addon encoding marker creates no addon', + ); + assert( + badAddonEncoding.data.errors.some((e: string) => + e.includes('unsupported flo_csv_encoding "legacy-v0"'), + ), + 'addon validation error reports unsupported flo_csv_encoding', + ); } finally { server.close(); closeDatabase(); diff --git a/tests/menu-csv-formula-export.test.ts b/tests/menu-csv-formula-export.test.ts index cf34390cd..07b72ec96 100644 --- a/tests/menu-csv-formula-export.test.ts +++ b/tests/menu-csv-formula-export.test.ts @@ -23,6 +23,7 @@ const { seedOwnerUser, seedCategory, seedProduct, + api, assert, assertIncludes, assertEqual, @@ -32,7 +33,11 @@ const { } = require('./helpers/test-setup'); const { menuCsvRoutes } = require('../main/routes/menu-csv'); -async function getText(baseUrl: string, urlPath: string, headers: Record): Promise { +async function getText( + baseUrl: string, + urlPath: string, + headers: Record, +): Promise { const response = await (globalThis as any).fetch(baseUrl + urlPath, { headers }); return await response.text(); } @@ -62,7 +67,9 @@ async function main() { seedProduct(db, 'prod-formula', 'cat-formula', '+SUM(1,2)', 12.5); db.prepare(`UPDATE products SET description = ? WHERE id = ?`).run('-1+2', 'prod-formula'); db.prepare(`INSERT INTO addon_groups (id, name) VALUES ('ag-formula', 'Extras')`).run(); - db.prepare(`INSERT INTO addons (id, addon_group_id, name, price, is_active) VALUES ('addon-formula', 'ag-formula', '@SUM(A1:A2)', 5, 1)`).run(); + db.prepare( + `INSERT INTO addons (id, addon_group_id, name, price, is_active) VALUES ('addon-formula', 'ag-formula', '@SUM(A1:A2)', 5, 1)`, + ).run(); const app = createApp({ '/api/menu/csv': menuCsvRoutes }); const { baseUrl, server } = await startServer(app); @@ -71,13 +78,28 @@ async function main() { console.log('\n─── Categories export neutralizes a leading "=" ───'); { const csv = await getText(baseUrl, '/api/menu/csv/export/categories', authHeader); + assertIncludes( + csv.split('\n')[0], + 'flo_csv_encoding', + 'categories export header includes flo_csv_encoding', + ); + assertIncludes(csv, ',quote-v1', 'categories export rows carry quote-v1 marker'); assertIncludes(csv, "'=HYPERLINK", 'leading "=" is prefixed with a single quote'); - assert(!/,"?=HYPERLINK/.test(csv) && !/^=HYPERLINK/.test(csv), 'no bare "=" formula cell remains in the categories export'); + assert( + !/,"?=HYPERLINK/.test(csv) && !/^=HYPERLINK/.test(csv), + 'no bare "=" formula cell remains in the categories export', + ); } console.log('\n─── Products export neutralizes "+" and "-" ───'); { const csv = await getText(baseUrl, '/api/menu/csv/export/products', authHeader); + assertIncludes( + csv.split('\n')[0], + 'flo_csv_encoding', + 'products export header includes flo_csv_encoding', + ); + assertIncludes(csv, ',quote-v1', 'products export rows carry quote-v1 marker'); assertIncludes(csv, "'+SUM(1,2)", 'leading "+" is prefixed with a single quote'); assertIncludes(csv, "'-1+2", 'leading "-" is prefixed with a single quote'); assertIncludes(csv, '12.5', 'numeric fields (price) are preserved as numbers'); @@ -104,30 +126,85 @@ async function main() { const csv = await getText(baseUrl, '/api/menu/csv/export/products', authHeader); for (const [label, tag] of whitespaceTriggers) { - assertIncludes(csv, `'${tag}`, `leading-${label} formula tag is prefixed with a single quote`); + assertIncludes( + csv, + `'${tag}`, + `leading-${label} formula tag is prefixed with a single quote`, + ); const escaped = tag.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - assert(!new RegExp(`(^|,)${escaped}(,|$)`, 'm').test(csv), - `no unneutralized leading-${label} formula cell remains in the export`); + assert( + !new RegExp(`(^|,)${escaped}(,|$)`, 'm').test(csv), + `no unneutralized leading-${label} formula cell remains in the export`, + ); } for (const [label, tag] of whitespaceTriggers) { const id = `prod-ws-${whitespaceTriggers.findIndex((t) => t[0] === label)}`; const row = csvRecordStartingAt(csv, id); assert(row !== null, `a record carrying the leading-${label} tag is present`); - // 12 columns means 11 separators; quoting the cell must not add one. - assertEqual((row!.match(/,/g) || []).length, 11, - `the leading-${label} tag did not shift the surrounding columns`); + // 13 columns means 12 separators; quoting the cell must not add one. + assertEqual( + (row!.match(/,/g) || []).length, + 12, + `the leading-${label} tag did not shift the surrounding columns`, + ); + } + + // Re-importing the quote-v1 export must restore each tag byte-for-byte: + // decodeQuoteV1Field puts the safety whitespace back, and the importer + // must not trim it away again (regression: "\t=1+1" stored as "=1+1"). + const reimport = await api(baseUrl, '/api/menu/csv/import/products', { + method: 'POST', + body: { csv }, + headers: authHeader, + }); + assertEqual(reimport.status, 200, 'quote-v1 product export re-imports'); + assertEqual(reimport.data.failed, 0, 'whitespace-trigger rows re-import without failures'); + for (const [label, tag] of whitespaceTriggers) { + const id = `prod-ws-${whitespaceTriggers.findIndex((t) => t[0] === label)}`; + const stored = db.prepare('SELECT tags FROM products WHERE id = ?').get(id) as any; + assertEqual( + stored.tags, + JSON.stringify([tag]), + `the leading-${label} formula tag survives the quote-v1 round trip unchanged`, + ); } } console.log('\n─── Add-ons export neutralizes a leading "@" ───'); { const csv = await getText(baseUrl, '/api/menu/csv/export/addons', authHeader); + assertIncludes( + csv.split('\n')[0], + 'flo_csv_encoding', + 'addons export header includes flo_csv_encoding', + ); + assertIncludes(csv, ',quote-v1', 'addons export rows carry quote-v1 marker'); assertIncludes(csv, "'@SUM(A1:A2)", 'leading "@" is prefixed with a single quote'); } + + console.log('\n─── Quote-v1 doubles genuine leading apostrophes on export ───'); + { + seedCategory(db, 'cat-quote', "'Chef's Category"); + seedProduct(db, 'prod-quote', 'cat-quote', "'+Quoted Formula", 9); + const catCsv = await getText(baseUrl, '/api/menu/csv/export/categories', authHeader); + const prodCsv = await getText(baseUrl, '/api/menu/csv/export/products', authHeader); + assertIncludes( + catCsv, + "''Chef's Category", + 'genuine leading apostrophe is doubled on category export', + ); + assertIncludes( + prodCsv, + "''+Quoted Formula", + 'genuine leading apostrophe before formula char is doubled once on product export', + ); + } } finally { server.close(); closeDatabase(); - try { fs.rmSync(testDir, { recursive: true, force: true }); } catch {} + try { + fs.rmSync(testDir, { recursive: true, force: true }); + } catch {} } const { passed, failed, total } = getResults();