diff --git a/practical-projects/07-fictional-reconciliation-workflow/README.es.md b/practical-projects/07-fictional-reconciliation-workflow/README.es.md new file mode 100644 index 0000000..b2cfe0a --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/README.es.md @@ -0,0 +1,387 @@ +# Flujo Ficticio de Conciliación + +[🇺🇸 English](README.md) · [🇧🇷 Português](README.pt-BR.md) · [🇪🇸 Español](README.es.md) + +[← Volver a Proyectos Prácticos](../README.es.md) + +Este es el **Proyecto 07 de la Fase 10: Proyectos Prácticos**. Convierte dos colecciones ficticias de registros en un informe de conciliación explícito y determinista. + +El ejemplo es original y ficticio. No reproduce ninguna empresa real, cliente, sistema contable ni flujo privado. + +## Qué vas a practicar + +Este proyecto combina conceptos de fases anteriores: + +- modelado inmutable con `dataclass`; +- estados controlados con `StrEnum`; +- dinero exacto con `Decimal`; +- diccionarios como índices de consulta; +- sets para la unión de claves de conciliación; +- orden determinista; +- validación y excepciones deliberadas; +- funciones con fronteras claras de entrada y salida; +- cobertura con pytest; +- separación entre lógica de dominio y presentación. + +## Escenario ficticio + +Dos fuentes imaginarias deberían contener las mismas referencias e importes. + +Fuente Norte: + +| Referencia | Importe | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `275.50` | +| `REF-003` | `100.00` | + +Fuente Sur: + +| Referencia | Importe | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `270.50` | +| `REF-004` | `100.00` | + +Las clasificaciones esperadas son: + +```text +REF-001 -> matched +REF-002 -> amount_mismatch +REF-003 -> left_only +REF-004 -> right_only +``` + +Para registros encontrados en ambos lados, la diferencia con signo es: + +```text +difference = left.amount - right.amount +``` + +Así, `275.50 - 270.50` produce `5.00`. + +## Requisitos + +El flujo debe: + +1. aceptar dos iterables de `ReconciliationRecord`; +2. rechazar identificadores de referencia vacíos; +3. exigir importes `Decimal` finitos; +4. aceptar solo importes exactamente representables con precisión de centavos y con un máximo de 100 dígitos en la parte entera; +5. eliminar espacios alrededor de los identificadores; +6. canonicalizar los importes aceptados a dos decimales; +7. rechazar referencias duplicadas dentro de cualquiera de las fuentes; +8. comparar identificadores de forma exacta y sensible a mayúsculas/minúsculas; +9. clasificar cada referencia como `matched`, `amount_mismatch`, `left_only` o `right_only`; +10. conservar la diferencia con signo en divergencias de importe; +11. ordenar la salida por identificador; +12. generar conteos de resumen deterministas; +13. calcular la magnitud absoluta total de las divergencias; +14. renderizar un informe de texto estable. + +El límite de 100 dígitos enteros es un contrato explícito de seguridad de recursos para este proyecto educativo. Está muy por encima de los valores realistas de los ejemplos, pero impide que notaciones científicas compactas como `1e1000000` se expandan a enteros gigantescos en Python. + +## Alcance deliberado + +La primera versión comienza **después de la ingestión**. + +No procesa CSV, hojas de cálculo, APIs, bases de datos ni datos privados. Esas capas pueden añadirse después como extensiones. + +Separar la ingestión mantiene visible la pregunta principal: + +> Dadas dos colecciones ya validadas, ¿cómo debería comportarse la conciliación? + +## Estructura + +```text +07-fictional-reconciliation-workflow/ +├── README.md +├── README.pt-BR.md +├── README.es.md +├── demo.py +├── reconciliation.py +└── tests/ + ├── conftest.py + ├── test_decimal_precision.py + ├── test_reconciliation.py + └── test_text_safety.py +``` + +## Modelo principal + +### `ReconciliationRecord` + +```python +ReconciliationRecord( + reference_id="REF-001", + amount=Decimal("150.00"), +) +``` + +El registro: + +- elimina espacios alrededor del identificador; +- rechaza identificadores vacíos; +- exige un `Decimal` real; +- rechaza `NaN` e infinitos; +- rechaza valores más allá de la precisión de centavos; +- rechaza importes cuya parte entera supere 100 dígitos; +- almacena los importes aceptados en forma canónica de dos decimales. + +Los importes negativos están permitidos porque un flujo genérico puede representar reversiones o ajustes. + +### `ReconciliationStatus` + +Los estados controlados son: + +```python +MATCHED +AMOUNT_MISMATCH +LEFT_ONLY +RIGHT_ONLY +``` + +### `ReconciliationItem` + +Cada clave conciliada tiene una forma válida: + +| Estado | Izquierda | Derecha | Diferencia | +|---|---|---|---| +| `MATCHED` | sí | sí | cero | +| `AMOUNT_MISMATCH` | sí | sí | distinta de cero | +| `LEFT_ONLY` | sí | no | ausente | +| `RIGHT_ONLY` | no | sí | ausente | + +La dataclass valida estas invariantes en lugar de confiar en que el llamador construya un resultado coherente. + +### `ReconciliationSummary` + +El resumen almacena: + +- total de elementos; +- elementos conciliados; +- divergencias de importe; +- elementos exclusivos de la izquierda; +- elementos exclusivos de la derecha; +- diferencia absoluta total de las divergencias. + +Las diferencias individuales conservan su signo. El agregado usa valores absolutos para que una divergencia de `+5.00` y otra de `-5.00` no se cancelen incorrectamente. + +### `ReconciliationReport` + +El informe agrupa los nombres de las fuentes, los elementos ordenados y el resumen. El renderizado sucede después, por lo que la lógica de comparación no queda ligada al texto. + +## Pipeline de conciliación + +```text +validar etiquetas de las fuentes + ↓ +indexar fuente izquierda + ↓ +indexar fuente derecha + ↓ +rechazar duplicados + ↓ +unir todos los identificadores + ↓ +ordenar identificadores + ↓ +clasificar cada identificador + ↓ +calcular diferencias + ↓ +construir resumen + ↓ +devolver informe inmutable +``` + +Los diccionarios son útiles porque permiten consulta directa por clave de conciliación y hacen explícita la detección de duplicados. + +## Contrato de matching + +Los identificadores se comparan después de eliminar los espacios alrededor. + +El matching es exacto y sensible a mayúsculas/minúsculas: + +```text +REF-001 != ref-001 +``` + +Esta es una decisión del proyecto, no una regla universal. Si un dominio requiere normalización de caja, claves compuestas u otra regla, debe declararse antes de iniciar la conciliación. + +## ¿Por qué `Decimal`? + +Para importes monetarios, el proyecto usa: + +```python +Decimal("275.50") +``` + +en lugar de `float`. + +Crear `Decimal` a partir de texto conserva el valor decimal deseado. El registro aplica después la frontera monetaria de dos decimales y el máximo de 100 dígitos enteros antes de cualquier expansión a centavos enteros. + +## Ejemplo básico + +```python +from decimal import Decimal + +from reconciliation import ReconciliationRecord, reconcile + +left = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("275.50")), +) + +right = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("270.50")), +) + +report = reconcile(left, right) + +for item in report.items: + print(item.reference_id, item.status) +``` + +Salida lógica: + +```text +REF-001 matched +REF-002 amount_mismatch +``` + +## Demostración + +Ejecuta desde esta carpeta: + +```bash +python demo.py +``` + +La demo es determinista, no interactiva, sin red y usa únicamente datos ficticios en memoria. + +Produce los cuatro estados importantes y un resumen. + +## Caminos de fallo + +El flujo falla deliberadamente cuando su contrato de entrada es ambiguo o inválido. + +Ejemplos: + +```python +ReconciliationRecord("", Decimal("10.00")) +``` + +genera `ValueError`. + +```python +ReconciliationRecord("REF-001", 10.00) +``` + +genera `TypeError` porque los floats no se convierten silenciosamente. + +```python +ReconciliationRecord("REF-001", Decimal("10.001")) +``` + +genera `ValueError` porque el importe supera la precisión de centavos. + +```python +ReconciliationRecord("REF-001", Decimal("1e100")) +``` + +genera `ValueError` porque el importe requeriría 101 dígitos enteros, por encima del límite documentado de 100 dígitos. + +Las referencias duplicadas dentro de una fuente también generan `ValueError`. El flujo no intenta adivinar si debería prevalecer el primer o el último duplicado. + +## Errores comunes + +### Comparar filas por posición + +Los mismos registros lógicos pueden llegar en órdenes distintos. Concilia mediante una clave estable, no mediante la posición en la lista. + +### Sobrescribir duplicados silenciosamente + +Una asignación normal en un diccionario puede ocultar registros duplicados. Este proyecto detecta el duplicado antes de que la inserción lo sobrescriba silenciosamente. + +### Usar valor absoluto demasiado pronto + +`abs(left - right)` elimina la dirección. Conserva la diferencia con signo en cada elemento y usa valores absolutos solo en la métrica de resumen. + +### Mezclar comparación e impresión + +Devolver resultados estructurados facilita las pruebas y permite otros renderizadores en el futuro. + +### Añadir normalización sin contrato + +Cambiar la caja, usar fuzzy matching, eliminar puntuación o ceros iniciales puede fusionar identificadores distintos. Trata la normalización como una decisión explícita de dominio. + +## Pruebas + +Ejecuta la suite enfocada desde la raíz del repositorio: + +```bash +python -m pytest -q practical-projects/07-fictional-reconciliation-workflow/tests +``` + +Las pruebas iniciales cubren validación, duplicados, los cuatro estados, diferencias positivas y negativas, generators, orden determinista, etiquetas de fuentes, sensibilidad a caja, invariantes de elementos, entrada vacía, límites de precisión monetaria, límites de magnitud, resúmenes y renderizado determinista. + +## Ejercicio + +Añade `REF-005` a ambas fuentes de la demo con importes diferentes. + +Antes de ejecutar, predice: + +1. el estado; +2. la diferencia con signo; +3. la nueva cantidad de divergencias; +4. la nueva diferencia absoluta total. + +Después ejecuta la demo y compara tu predicción con el informe real. + +## Desafíos de extensión + +Después de que el contrato base esté claro, prueba una extensión a la vez: + +1. Añade una tolerancia `Decimal` configurable y prueba exactamente su límite. +2. Añade una capa de ingestión CSV que produzca registros validados antes de la conciliación. +3. Sustituye la referencia simple por una clave compuesta como `(reference_id, period)`. +4. Añade un renderer Markdown sin modificar `reconcile()`. +5. Exporta solo elementos no resueltos, conservando el informe canónico completo. +6. Introduce un objeto de política de conciliación en lugar de muchos flags booleanos no relacionados. + +## Discusión de portafolio + +Este proyecto demuestra cómo transformar un problema genérico de comparación en contratos explícitos de software. + +Puntos útiles para explicar en un portafolio: + +- claves de dominio estables; +- importes monetarios exactos; +- límites explícitos de magnitud; +- rechazo de duplicados; +- consulta indexada; +- estados y orden deterministas; +- diferencias con signo y métricas agregadas; +- resultados inmutables; +- separación entre conciliación y renderizado; +- pruebas automatizadas enfocadas en límites. + +## Referencia rápida + +```text +Entrada: dos iterables de ReconciliationRecord +Clave: reference_id normalizado +Matching: exacto y sensible a mayúsculas/minúsculas +Dinero: Decimal finito, precisión de centavos, <= 100 dígitos enteros +Estados: matched / amount_mismatch / left_only / right_only +Diferencia: left.amount - right.amount +Orden: reference_id ascendente +Duplicados: rechazados dentro de cada fuente +Salida: ReconciliationReport inmutable +``` + +## Próximo proyecto + +Después de que este proyecto sea revisado, la Fase 10 continúa con el **Proyecto 08: Flujo Simulado de Automatización**. diff --git a/practical-projects/07-fictional-reconciliation-workflow/README.md b/practical-projects/07-fictional-reconciliation-workflow/README.md new file mode 100644 index 0000000..749a222 --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/README.md @@ -0,0 +1,387 @@ +# Fictional Reconciliation Workflow + +[🇺🇸 English](README.md) · [🇧🇷 Português](README.pt-BR.md) · [🇪🇸 Español](README.es.md) + +[← Back to Practical Projects](../README.md) + +This is **Project 07 of Phase 10: Practical Projects**. It turns two fictional record collections into an explicit, deterministic reconciliation report. + +The example is original and fictional. It does not reproduce any real company, client, accounting system, or private workflow. + +## What you will practice + +This project combines concepts from earlier phases: + +- immutable data modeling with `dataclass`; +- controlled states with `StrEnum`; +- exact money with `Decimal`; +- dictionaries as lookup indexes; +- sets for the union of reconciliation keys; +- deterministic sorting; +- validation and deliberate exceptions; +- functions with clear input/output boundaries; +- pytest coverage; +- separation between domain logic and presentation. + +## Fictional scenario + +Two imaginary sources should contain the same references and amounts. + +Source North: + +| Reference | Amount | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `275.50` | +| `REF-003` | `100.00` | + +Source South: + +| Reference | Amount | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `270.50` | +| `REF-004` | `100.00` | + +The expected classifications are: + +```text +REF-001 -> matched +REF-002 -> amount_mismatch +REF-003 -> left_only +REF-004 -> right_only +``` + +For records found on both sides, the signed difference is: + +```text +difference = left.amount - right.amount +``` + +So `275.50 - 270.50` is `5.00`. + +## Requirements + +The workflow must: + +1. accept two iterables of `ReconciliationRecord`; +2. reject empty reference identifiers; +3. require finite `Decimal` amounts; +4. accept only amounts exactly representable at cent precision and with at most 100 integer digits; +5. normalize surrounding whitespace in reference identifiers; +6. canonicalize accepted amounts to two decimal places; +7. reject duplicate references inside either source; +8. match identifiers exactly and case-sensitively; +9. classify every reference as `matched`, `amount_mismatch`, `left_only`, or `right_only`; +10. preserve the signed difference for amount mismatches; +11. sort output by reference identifier; +12. build deterministic summary counts; +13. calculate total absolute mismatch magnitude; +14. render a stable text report. + +The 100-integer-digit boundary is an explicit resource-safety contract for this educational project. It is intentionally far above realistic sample values while preventing compact scientific notation such as `1e1000000` from expanding into enormous Python integers. + +## Deliberate scope + +The first version starts **after ingestion**. + +It does not parse CSV files, spreadsheets, APIs, databases, or private data. Those layers were studied elsewhere and can be added later as extensions. + +Keeping ingestion separate makes the core question easier to study: + +> Given two already validated collections, how should reconciliation behave? + +## Structure + +```text +07-fictional-reconciliation-workflow/ +├── README.md +├── README.pt-BR.md +├── README.es.md +├── demo.py +├── reconciliation.py +└── tests/ + ├── conftest.py + ├── test_decimal_precision.py + ├── test_reconciliation.py + └── test_text_safety.py +``` + +## Core model + +### `ReconciliationRecord` + +```python +ReconciliationRecord( + reference_id="REF-001", + amount=Decimal("150.00"), +) +``` + +The record: + +- trims surrounding identifier whitespace; +- rejects blank identifiers; +- requires an actual `Decimal`; +- rejects `NaN` and infinities; +- rejects values beyond cent precision; +- rejects amounts whose integer part exceeds 100 digits; +- stores accepted amounts in canonical two-decimal form. + +Negative amounts are allowed because a generic workflow may represent reversals or adjustments. + +### `ReconciliationStatus` + +The controlled states are: + +```python +MATCHED +AMOUNT_MISMATCH +LEFT_ONLY +RIGHT_ONLY +``` + +### `ReconciliationItem` + +Each reconciled key has one valid shape: + +| Status | Left | Right | Difference | +|---|---|---|---| +| `MATCHED` | yes | yes | zero | +| `AMOUNT_MISMATCH` | yes | yes | non-zero | +| `LEFT_ONLY` | yes | no | none | +| `RIGHT_ONLY` | no | yes | none | + +The dataclass validates these invariants instead of trusting callers to build a consistent result. + +### `ReconciliationSummary` + +The summary stores: + +- total items; +- matched items; +- amount mismatches; +- left-only items; +- right-only items; +- total absolute difference for amount mismatches. + +Per-item differences keep their sign. The aggregate uses absolute values so a `+5.00` mismatch and a `-5.00` mismatch do not incorrectly cancel each other. + +### `ReconciliationReport` + +The report groups source names, ordered items, and the summary. Rendering happens afterward, so comparison logic is not tied to text output. + +## Reconciliation pipeline + +```text +validate source labels + ↓ +index left source + ↓ +index right source + ↓ +reject duplicates + ↓ +union all reference ids + ↓ +sort ids + ↓ +classify each id + ↓ +calculate differences + ↓ +build summary + ↓ +return immutable report +``` + +Dictionaries are useful here because they provide direct lookup by reconciliation key and make duplicate detection explicit. + +## Matching contract + +Identifiers are compared after surrounding whitespace is removed. + +Matching is otherwise exact and case-sensitive: + +```text +REF-001 != ref-001 +``` + +That is a project decision, not a universal business rule. If a domain requires case folding, composite keys, or another normalization rule, that rule should be declared before reconciliation begins. + +## Why `Decimal`? + +For monetary values, the project uses: + +```python +Decimal("275.50") +``` + +instead of `float`. + +Creating `Decimal` from text preserves the intended decimal value. The record then enforces the project's two-decimal monetary boundary and a maximum of 100 integer digits before any integer-cent expansion occurs. + +## Basic example + +```python +from decimal import Decimal + +from reconciliation import ReconciliationRecord, reconcile + +left = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("275.50")), +) + +right = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("270.50")), +) + +report = reconcile(left, right) + +for item in report.items: + print(item.reference_id, item.status) +``` + +Logical output: + +```text +REF-001 matched +REF-002 amount_mismatch +``` + +## Demo + +Run from this directory: + +```bash +python demo.py +``` + +The demo is deterministic, non-interactive, network-free, and uses only fictional in-memory data. + +It produces the four important states and a summary. + +## Failure paths + +The workflow fails deliberately when its input contract is ambiguous or invalid. + +Examples: + +```python +ReconciliationRecord("", Decimal("10.00")) +``` + +raises `ValueError`. + +```python +ReconciliationRecord("REF-001", 10.00) +``` + +raises `TypeError` because floats are not silently converted. + +```python +ReconciliationRecord("REF-001", Decimal("10.001")) +``` + +raises `ValueError` because the amount exceeds cent precision. + +```python +ReconciliationRecord("REF-001", Decimal("1e100")) +``` + +raises `ValueError` because the amount would require 101 integer digits, beyond the documented 100-digit boundary. + +Duplicate references inside one source also raise `ValueError`. The workflow does not guess whether the first or last duplicate should win. + +## Common mistakes + +### Comparing rows by position + +The same logical records may arrive in different orders. Reconcile by a stable key, not by list position. + +### Silently overwriting duplicates + +A normal dictionary assignment can hide duplicate source records. This project detects duplicates before insertion wins silently. + +### Using absolute difference too early + +`abs(left - right)` removes direction. Keep the signed difference on each item and use absolute values only for the summary metric. + +### Mixing comparison and printing + +Returning structured results makes the workflow easier to test and allows other renderers later. + +### Adding normalization without a contract + +Case folding, fuzzy matching, punctuation removal, or leading-zero removal can merge different identifiers. Treat normalization as an explicit domain decision. + +## Tests + +Run the focused suite from the repository root: + +```bash +python -m pytest -q practical-projects/07-fictional-reconciliation-workflow/tests +``` + +The initial tests cover validation, duplicate detection, all four statuses, positive and negative differences, generators, deterministic ordering, source labels, case sensitivity, item invariants, empty input, exact-money precision boundaries, magnitude limits, summaries, and deterministic rendering. + +## Exercise + +Add `REF-005` to both demo sources with different values. + +Before executing the program, predict: + +1. the status; +2. the signed difference; +3. the new mismatch count; +4. the new total absolute difference. + +Then run the demo and compare your prediction with the actual report. + +## Extension challenges + +After the base contract is clear, try one extension at a time: + +1. Add a configurable `Decimal` tolerance and test its exact boundary. +2. Add a CSV ingestion layer that produces validated records before reconciliation. +3. Replace the simple reference with a composite key such as `(reference_id, period)`. +4. Add a Markdown renderer without changing `reconcile()`. +5. Export only unresolved items while preserving the canonical full report. +6. Introduce a reconciliation policy object instead of many unrelated Boolean flags. + +## Portfolio discussion + +This project demonstrates how to turn a generic comparison problem into explicit software contracts. + +Useful points to explain in a portfolio: + +- stable domain keys; +- exact monetary values; +- explicit magnitude boundaries; +- duplicate rejection; +- indexed lookup; +- deterministic states and ordering; +- signed differences and aggregate metrics; +- immutable results; +- separation of reconciliation and rendering; +- boundary-focused automated tests. + +## Quick reference + +```text +Input: two iterables of ReconciliationRecord +Key: normalized reference_id +Matching: exact and case-sensitive +Money: finite Decimal, cent precision, <= 100 integer digits +Statuses: matched / amount_mismatch / left_only / right_only +Difference: left.amount - right.amount +Ordering: ascending reference_id +Duplicates: rejected within each source +Output: immutable ReconciliationReport +``` + +## Next project + +After this project is reviewed, Phase 10 continues with **Project 08: Simulated Automation Flow**. diff --git a/practical-projects/07-fictional-reconciliation-workflow/README.pt-BR.md b/practical-projects/07-fictional-reconciliation-workflow/README.pt-BR.md new file mode 100644 index 0000000..4a66c01 --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/README.pt-BR.md @@ -0,0 +1,387 @@ +# Fluxo Fictício de Conciliação + +[🇺🇸 English](README.md) · [🇧🇷 Português](README.pt-BR.md) · [🇪🇸 Español](README.es.md) + +[← Voltar para Projetos Práticos](../README.pt-BR.md) + +Este é o **Projeto 07 da Fase 10: Projetos Práticos**. Ele transforma duas coleções fictícias de registros em um relatório de conciliação explícito e determinístico. + +O exemplo é original e fictício. Ele não reproduz nenhuma empresa real, cliente, sistema contábil ou fluxo privado. + +## O que você vai praticar + +Este projeto combina conceitos das fases anteriores: + +- modelagem imutável com `dataclass`; +- estados controlados com `StrEnum`; +- dinheiro exato com `Decimal`; +- dicionários como índices de consulta; +- sets para a união das chaves de conciliação; +- ordenação determinística; +- validação e exceções deliberadas; +- funções com fronteiras claras de entrada e saída; +- cobertura com pytest; +- separação entre lógica de domínio e apresentação. + +## Cenário fictício + +Duas fontes imaginárias deveriam conter as mesmas referências e valores. + +Fonte Norte: + +| Referência | Valor | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `275.50` | +| `REF-003` | `100.00` | + +Fonte Sul: + +| Referência | Valor | +|---|---:| +| `REF-001` | `150.00` | +| `REF-002` | `270.50` | +| `REF-004` | `100.00` | + +As classificações esperadas são: + +```text +REF-001 -> matched +REF-002 -> amount_mismatch +REF-003 -> left_only +REF-004 -> right_only +``` + +Para registros encontrados nos dois lados, a diferença com sinal é: + +```text +difference = left.amount - right.amount +``` + +Assim, `275.50 - 270.50` resulta em `5.00`. + +## Requisitos + +O fluxo deve: + +1. aceitar dois iteráveis de `ReconciliationRecord`; +2. rejeitar identificadores de referência vazios; +3. exigir valores `Decimal` finitos; +4. aceitar somente valores exatamente representáveis em precisão de centavos e com no máximo 100 dígitos na parte inteira; +5. remover espaços ao redor dos identificadores; +6. canonicalizar os valores aceitos para duas casas decimais; +7. rejeitar referências duplicadas dentro de qualquer fonte; +8. comparar identificadores de forma exata e sensível a maiúsculas/minúsculas; +9. classificar cada referência como `matched`, `amount_mismatch`, `left_only` ou `right_only`; +10. preservar a diferença com sinal nas divergências de valor; +11. ordenar a saída pelo identificador; +12. gerar contagens de resumo determinísticas; +13. calcular a magnitude absoluta total das divergências; +14. renderizar um relatório de texto estável. + +O limite de 100 dígitos inteiros é um contrato explícito de segurança de recursos deste projeto educacional. Ele fica muito acima dos valores realistas dos exemplos, mas impede que notações científicas compactas como `1e1000000` sejam expandidas para inteiros gigantescos em Python. + +## Escopo deliberado + +A primeira versão começa **depois da ingestão**. + +Ela não faz parsing de CSV, planilhas, APIs, bancos de dados nem dados privados. Essas camadas podem ser adicionadas depois como extensões. + +Separar a ingestão mantém visível a pergunta principal: + +> Dadas duas coleções já validadas, como a conciliação deve se comportar? + +## Estrutura + +```text +07-fictional-reconciliation-workflow/ +├── README.md +├── README.pt-BR.md +├── README.es.md +├── demo.py +├── reconciliation.py +└── tests/ + ├── conftest.py + ├── test_decimal_precision.py + ├── test_reconciliation.py + └── test_text_safety.py +``` + +## Modelo principal + +### `ReconciliationRecord` + +```python +ReconciliationRecord( + reference_id="REF-001", + amount=Decimal("150.00"), +) +``` + +O registro: + +- remove espaços ao redor do identificador; +- rejeita identificadores vazios; +- exige um `Decimal` real; +- rejeita `NaN` e infinitos; +- rejeita valores além da precisão de centavos; +- rejeita valores cuja parte inteira ultrapasse 100 dígitos; +- armazena valores aceitos no formato canônico de duas casas. + +Valores negativos são permitidos porque um fluxo genérico pode representar estornos ou ajustes. + +### `ReconciliationStatus` + +Os estados controlados são: + +```python +MATCHED +AMOUNT_MISMATCH +LEFT_ONLY +RIGHT_ONLY +``` + +### `ReconciliationItem` + +Cada chave conciliada possui uma forma válida: + +| Status | Esquerda | Direita | Diferença | +|---|---|---|---| +| `MATCHED` | sim | sim | zero | +| `AMOUNT_MISMATCH` | sim | sim | diferente de zero | +| `LEFT_ONLY` | sim | não | ausente | +| `RIGHT_ONLY` | não | sim | ausente | + +A dataclass valida essas invariantes em vez de confiar que o chamador monte um resultado consistente. + +### `ReconciliationSummary` + +O resumo armazena: + +- total de itens; +- itens conciliados; +- divergências de valor; +- itens exclusivos da esquerda; +- itens exclusivos da direita; +- diferença absoluta total das divergências. + +As diferenças individuais mantêm seu sinal. O agregado usa valores absolutos para que uma divergência de `+5.00` e outra de `-5.00` não se anulem incorretamente. + +### `ReconciliationReport` + +O relatório agrupa os nomes das fontes, os itens ordenados e o resumo. A renderização acontece depois, então a lógica de comparação não fica presa ao texto. + +## Pipeline de conciliação + +```text +validar rótulos das fontes + ↓ +indexar fonte esquerda + ↓ +indexar fonte direita + ↓ +rejeitar duplicados + ↓ +unir todos os identificadores + ↓ +ordenar identificadores + ↓ +classificar cada identificador + ↓ +calcular diferenças + ↓ +construir resumo + ↓ +retornar relatório imutável +``` + +Dicionários são úteis porque fornecem consulta direta pela chave de conciliação e tornam a detecção de duplicados explícita. + +## Contrato de matching + +Os identificadores são comparados depois da remoção dos espaços ao redor. + +O matching é exato e sensível a maiúsculas/minúsculas: + +```text +REF-001 != ref-001 +``` + +Essa é uma decisão do projeto, não uma regra universal. Se um domínio exigir normalização de caixa, chaves compostas ou outra regra, ela deve ser declarada antes da conciliação. + +## Por que `Decimal`? + +Para valores monetários, o projeto usa: + +```python +Decimal("275.50") +``` + +em vez de `float`. + +Criar `Decimal` a partir de texto preserva o valor decimal pretendido. O registro então aplica a fronteira monetária de duas casas e o máximo de 100 dígitos inteiros antes de qualquer expansão para centavos inteiros. + +## Exemplo básico + +```python +from decimal import Decimal + +from reconciliation import ReconciliationRecord, reconcile + +left = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("275.50")), +) + +right = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("270.50")), +) + +report = reconcile(left, right) + +for item in report.items: + print(item.reference_id, item.status) +``` + +Saída lógica: + +```text +REF-001 matched +REF-002 amount_mismatch +``` + +## Demonstração + +Execute a partir desta pasta: + +```bash +python demo.py +``` + +A demo é determinística, não interativa, sem rede e usa apenas dados fictícios em memória. + +Ela produz os quatro estados importantes e um resumo. + +## Caminhos de falha + +O fluxo falha deliberadamente quando seu contrato de entrada é ambíguo ou inválido. + +Exemplos: + +```python +ReconciliationRecord("", Decimal("10.00")) +``` + +gera `ValueError`. + +```python +ReconciliationRecord("REF-001", 10.00) +``` + +gera `TypeError`, pois floats não são convertidos silenciosamente. + +```python +ReconciliationRecord("REF-001", Decimal("10.001")) +``` + +gera `ValueError`, pois o valor ultrapassa a precisão de centavos. + +```python +ReconciliationRecord("REF-001", Decimal("1e100")) +``` + +gera `ValueError`, pois o valor exigiria 101 dígitos inteiros, acima do limite documentado de 100 dígitos. + +Referências duplicadas dentro de uma fonte também geram `ValueError`. O fluxo não tenta adivinhar se o primeiro ou o último duplicado deve prevalecer. + +## Erros comuns + +### Comparar linhas pela posição + +Os mesmos registros lógicos podem chegar em ordens diferentes. Concilie por uma chave estável, não pela posição na lista. + +### Sobrescrever duplicados silenciosamente + +Uma atribuição normal em dicionário pode esconder registros duplicados. Este projeto detecta a duplicidade antes que a inserção sobrescreva silenciosamente. + +### Usar valor absoluto cedo demais + +`abs(left - right)` remove a direção. Preserve a diferença com sinal em cada item e use valores absolutos somente na métrica de resumo. + +### Misturar comparação e impressão + +Retornar resultados estruturados facilita testes e permite outros renderizadores no futuro. + +### Adicionar normalização sem contrato + +Alterar caixa, usar fuzzy matching, remover pontuação ou zeros à esquerda pode unir identificadores diferentes. Trate normalização como decisão explícita de domínio. + +## Testes + +Execute a suíte focada a partir da raiz do repositório: + +```bash +python -m pytest -q practical-projects/07-fictional-reconciliation-workflow/tests +``` + +Os testes iniciais cobrem validação, duplicidade, os quatro status, diferenças positivas e negativas, generators, ordenação determinística, rótulos de fonte, sensibilidade a caixa, invariantes dos itens, entrada vazia, fronteiras de precisão monetária, limites de magnitude, resumos e renderização determinística. + +## Exercício + +Adicione `REF-005` às duas fontes da demo com valores diferentes. + +Antes de executar, preveja: + +1. o status; +2. a diferença com sinal; +3. a nova quantidade de divergências; +4. a nova diferença absoluta total. + +Depois execute a demo e compare sua previsão com o relatório real. + +## Desafios de extensão + +Depois que o contrato base estiver claro, tente uma extensão por vez: + +1. Adicione uma tolerância `Decimal` configurável e teste exatamente sua fronteira. +2. Adicione uma camada de ingestão CSV que produza registros validados antes da conciliação. +3. Substitua a referência simples por uma chave composta como `(reference_id, period)`. +4. Adicione um renderer Markdown sem alterar `reconcile()`. +5. Exporte somente itens não resolvidos, preservando o relatório canônico completo. +6. Introduza um objeto de política de conciliação em vez de muitos flags booleanos sem relação. + +## Discussão de portfólio + +Este projeto demonstra como transformar um problema genérico de comparação em contratos explícitos de software. + +Pontos úteis para explicar em um portfólio: + +- chaves de domínio estáveis; +- valores monetários exatos; +- limites explícitos de magnitude; +- rejeição de duplicados; +- consulta indexada; +- estados e ordenação determinísticos; +- diferenças com sinal e métricas agregadas; +- resultados imutáveis; +- separação entre conciliação e renderização; +- testes automatizados focados em fronteiras. + +## Referência rápida + +```text +Entrada: dois iteráveis de ReconciliationRecord +Chave: reference_id normalizado +Matching: exato e sensível a maiúsculas/minúsculas +Dinheiro: Decimal finito, precisão de centavos, <= 100 dígitos inteiros +Status: matched / amount_mismatch / left_only / right_only +Diferença: left.amount - right.amount +Ordenação: reference_id crescente +Duplicados: rejeitados dentro de cada fonte +Saída: ReconciliationReport imutável +``` + +## Próximo projeto + +Depois que este projeto for revisado, a Fase 10 continua com o **Projeto 08: Fluxo Simulado de Automação**. diff --git a/practical-projects/07-fictional-reconciliation-workflow/demo.py b/practical-projects/07-fictional-reconciliation-workflow/demo.py new file mode 100644 index 0000000..82e057d --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/demo.py @@ -0,0 +1,28 @@ +from decimal import Decimal + +from reconciliation import ReconciliationRecord, reconcile, render_text_report + + +def main() -> None: + source_north = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("275.50")), + ReconciliationRecord("REF-003", Decimal("100.00")), + ) + source_south = ( + ReconciliationRecord("REF-001", Decimal("150.00")), + ReconciliationRecord("REF-002", Decimal("270.50")), + ReconciliationRecord("REF-004", Decimal("100.00")), + ) + + report = reconcile( + source_north, + source_south, + left_name="Source North", + right_name="Source South", + ) + print(render_text_report(report), end="") + + +if __name__ == "__main__": + main() diff --git a/practical-projects/07-fictional-reconciliation-workflow/reconciliation.py b/practical-projects/07-fictional-reconciliation-workflow/reconciliation.py new file mode 100644 index 0000000..6140012 --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/reconciliation.py @@ -0,0 +1,452 @@ +"""Deterministic reconciliation for two fictional record sources.""" + +from __future__ import annotations + +from dataclasses import dataclass +from decimal import Decimal +from typing import Iterable + +try: + from enum import StrEnum +except ImportError: # Python < 3.11 + from enum import Enum + + class StrEnum(str, Enum): + """Minimal compatibility backport of enum.StrEnum behavior.""" + + def __str__(self) -> str: + return self.value + + +MAX_INTEGER_DIGITS = 100 + + +class ReconciliationStatus(StrEnum): + """Possible outcomes for one reference during reconciliation.""" + + MATCHED = "matched" + AMOUNT_MISMATCH = "amount_mismatch" + LEFT_ONLY = "left_only" + RIGHT_ONLY = "right_only" + + +def _digits_to_int(digits: tuple[int, ...]) -> int: + """Build an integer coefficient from Decimal digits without context math.""" + + coefficient = 0 + for digit in digits: + coefficient = coefficient * 10 + digit + return coefficient + + +def _integer_digit_count(value: Decimal) -> int: + """Return the number of digits in the integer part without expansion.""" + + _, digits, exponent = value.as_tuple() + if not any(digits): + return 1 + return max(1, len(digits) + exponent) + + +def _validate_amount_magnitude(value: Decimal) -> None: + """Reject monetary inputs beyond the project's explicit size boundary.""" + + if _integer_digit_count(value) > MAX_INTEGER_DIGITS: + raise ValueError( + f"amount must have at most {MAX_INTEGER_DIGITS} integer digits" + ) + + +def _decimal_to_cents(value: Decimal) -> int: + """Convert an exact cent-representable Decimal to integer cents. + + The conversion uses only the Decimal coefficient and exponent, so its + result is independent of the active Decimal arithmetic context. For + sub-cent exponents, discarded positions are inspected before converting + digits into an integer. That avoids building enormous temporary integers + when a value contains a very long tail of fractional zeros. + External record values are magnitude-checked before calling this helper. + """ + + sign, digits, exponent = value.as_tuple() + + if not any(digits): + return 0 + + if exponent < -2: + discarded_places = -2 - exponent + if discarded_places >= len(digits): + raise ValueError("amount must have at most two decimal places") + + split_at = len(digits) - discarded_places + if any(digits[split_at:]): + raise ValueError("amount must have at most two decimal places") + + cents = _digits_to_int(digits[:split_at]) + else: + coefficient = _digits_to_int(digits) + cents = coefficient * (10 ** (exponent + 2)) + + return -cents if sign else cents + + +def _cents_to_decimal(cents: int) -> Decimal: + """Build a two-decimal Decimal without context or integer-string limits.""" + + if cents == 0: + return Decimal("0.00") + + magnitude = Decimal(abs(cents)) + digits = magnitude.as_tuple().digits + sign = 1 if cents < 0 else 0 + return Decimal((sign, digits, -2)) + + +def _subtract_amounts(left: Decimal, right: Decimal) -> Decimal: + """Return an exact signed difference for canonical monetary amounts.""" + + return _cents_to_decimal(_decimal_to_cents(left) - _decimal_to_cents(right)) + + +def _validate_printable_text(value: str, *, field_name: str) -> str: + """Normalize surrounding whitespace and reject non-printable content.""" + + normalized = value.strip() + if not normalized: + raise ValueError(f"{field_name} must not be empty") + if not normalized.isprintable(): + raise ValueError(f"{field_name} must contain only printable characters") + return normalized + + +@dataclass(frozen=True, slots=True) +class ReconciliationRecord: + """One validated monetary record selected for reconciliation.""" + + reference_id: str + amount: Decimal + + def __post_init__(self) -> None: + if not isinstance(self.reference_id, str): + raise TypeError("reference_id must be a string") + if not isinstance(self.amount, Decimal): + raise TypeError("amount must be a Decimal") + + normalized_id = _validate_printable_text( + self.reference_id, + field_name="reference_id", + ) + + if not self.amount.is_finite(): + raise ValueError("amount must be finite") + + _validate_amount_magnitude(self.amount) + cents = _decimal_to_cents(self.amount) + normalized_amount = _cents_to_decimal(cents) + + object.__setattr__(self, "reference_id", normalized_id) + object.__setattr__(self, "amount", normalized_amount) + + +@dataclass(frozen=True, slots=True) +class ReconciliationItem: + """Reconciliation outcome for exactly one reference id.""" + + reference_id: str + status: ReconciliationStatus + left: ReconciliationRecord | None + right: ReconciliationRecord | None + difference: Decimal | None + + def __post_init__(self) -> None: + if not isinstance(self.reference_id, str) or not self.reference_id: + raise ValueError("reference_id must be a non-empty string") + if not self.reference_id.isprintable(): + raise ValueError("reference_id must contain only printable characters") + if not isinstance(self.status, ReconciliationStatus): + raise TypeError("status must be a ReconciliationStatus") + + if self.difference is not None and not isinstance(self.difference, Decimal): + raise TypeError("difference must be a Decimal or None") + + for record in (self.left, self.right): + if record is not None and record.reference_id != self.reference_id: + raise ValueError( + "item reference_id must match every attached record" + ) + + if self.status is ReconciliationStatus.LEFT_ONLY: + if ( + self.left is None + or self.right is not None + or self.difference is not None + ): + raise ValueError("left_only requires only a left record") + return + + if self.status is ReconciliationStatus.RIGHT_ONLY: + if ( + self.left is not None + or self.right is None + or self.difference is not None + ): + raise ValueError("right_only requires only a right record") + return + + if self.left is None or self.right is None or self.difference is None: + raise ValueError( + "matched comparisons require both records and a difference" + ) + + expected_difference = _subtract_amounts(self.left.amount, self.right.amount) + if self.difference != expected_difference: + raise ValueError("difference must equal left amount minus right amount") + + if self.status is ReconciliationStatus.MATCHED and self.difference != Decimal( + "0.00" + ): + raise ValueError("matched items require a zero difference") + if ( + self.status is ReconciliationStatus.AMOUNT_MISMATCH + and self.difference == Decimal("0.00") + ): + raise ValueError("amount_mismatch items require a non-zero difference") + + +@dataclass(frozen=True, slots=True) +class ReconciliationSummary: + """Aggregate counts for one reconciliation run.""" + + total_items: int + matched: int + amount_mismatches: int + left_only: int + right_only: int + total_absolute_difference: Decimal + + +@dataclass(frozen=True, slots=True) +class ReconciliationReport: + """Complete immutable result for one pair of fictional sources.""" + + left_name: str + right_name: str + items: tuple[ReconciliationItem, ...] + summary: ReconciliationSummary + + def __post_init__(self) -> None: + left_label = _validate_source_name(self.left_name, field_name="left_name") + right_label = _validate_source_name(self.right_name, field_name="right_name") + if left_label == right_label: + raise ValueError("left_name and right_name must be different") + + object.__setattr__(self, "left_name", left_label) + object.__setattr__(self, "right_name", right_label) + + +def _validate_source_name(name: str, *, field_name: str) -> str: + if not isinstance(name, str): + raise TypeError(f"{field_name} must be a string") + return _validate_printable_text(name, field_name=field_name) + + +def _index_records( + records: Iterable[ReconciliationRecord], + *, + source_name: str, +) -> dict[str, ReconciliationRecord]: + index: dict[str, ReconciliationRecord] = {} + + try: + iterator = iter(records) + except TypeError as exc: + raise TypeError(f"{source_name} must be an iterable of records") from exc + + for record in iterator: + if not isinstance(record, ReconciliationRecord): + raise TypeError( + f"{source_name} must contain ReconciliationRecord values" + ) + if record.reference_id in index: + raise ValueError( + f"duplicate reference_id in {source_name}: {record.reference_id}" + ) + index[record.reference_id] = record + + return index + + +def _build_summary( + items: Iterable[ReconciliationItem], +) -> ReconciliationSummary: + item_tuple = tuple(items) + + matched = sum( + item.status is ReconciliationStatus.MATCHED for item in item_tuple + ) + amount_mismatches = sum( + item.status is ReconciliationStatus.AMOUNT_MISMATCH + for item in item_tuple + ) + left_only = sum( + item.status is ReconciliationStatus.LEFT_ONLY for item in item_tuple + ) + right_only = sum( + item.status is ReconciliationStatus.RIGHT_ONLY for item in item_tuple + ) + total_absolute_difference_cents = sum( + abs(_decimal_to_cents(item.difference)) + for item in item_tuple + if item.status is ReconciliationStatus.AMOUNT_MISMATCH + and item.difference is not None + ) + total_absolute_difference = _cents_to_decimal( + total_absolute_difference_cents + ) + + return ReconciliationSummary( + total_items=len(item_tuple), + matched=matched, + amount_mismatches=amount_mismatches, + left_only=left_only, + right_only=right_only, + total_absolute_difference=total_absolute_difference, + ) + + +def reconcile( + left_records: Iterable[ReconciliationRecord], + right_records: Iterable[ReconciliationRecord], + *, + left_name: str = "Source A", + right_name: str = "Source B", +) -> ReconciliationReport: + """Compare two record collections by reference id and exact amount. + + Reference identifiers are matched exactly after surrounding whitespace is + removed by ``ReconciliationRecord``. Amount differences use the explicit + contract ``left.amount - right.amount``. + """ + + left_label = _validate_source_name(left_name, field_name="left_name") + right_label = _validate_source_name(right_name, field_name="right_name") + if left_label == right_label: + raise ValueError("left_name and right_name must be different") + + left_index = _index_records(left_records, source_name=left_label) + right_index = _index_records(right_records, source_name=right_label) + + items: list[ReconciliationItem] = [] + for reference_id in sorted(left_index.keys() | right_index.keys()): + left = left_index.get(reference_id) + right = right_index.get(reference_id) + + if left is None: + items.append( + ReconciliationItem( + reference_id=reference_id, + status=ReconciliationStatus.RIGHT_ONLY, + left=None, + right=right, + difference=None, + ) + ) + continue + + if right is None: + items.append( + ReconciliationItem( + reference_id=reference_id, + status=ReconciliationStatus.LEFT_ONLY, + left=left, + right=None, + difference=None, + ) + ) + continue + + difference = _subtract_amounts(left.amount, right.amount) + status = ( + ReconciliationStatus.MATCHED + if difference == Decimal("0.00") + else ReconciliationStatus.AMOUNT_MISMATCH + ) + items.append( + ReconciliationItem( + reference_id=reference_id, + status=status, + left=left, + right=right, + difference=difference, + ) + ) + + item_tuple = tuple(items) + return ReconciliationReport( + left_name=left_label, + right_name=right_label, + items=item_tuple, + summary=_build_summary(item_tuple), + ) + + +def render_text_report(report: ReconciliationReport) -> str: + """Render a stable, human-readable reconciliation report.""" + + if not isinstance(report, ReconciliationReport): + raise TypeError("report must be a ReconciliationReport") + + lines = [ + "Reconciliation Report", + f"Sources: {report.left_name} vs {report.right_name}", + "", + ] + + for item in report.items: + if item.status is ReconciliationStatus.MATCHED: + assert item.left is not None + assert item.right is not None + lines.append( + f"[MATCHED] {item.reference_id}: " + f"{item.left.amount:.2f} == {item.right.amount:.2f}" + ) + elif item.status is ReconciliationStatus.AMOUNT_MISMATCH: + assert item.left is not None + assert item.right is not None + assert item.difference is not None + lines.append( + f"[AMOUNT_MISMATCH] {item.reference_id}: " + f"{report.left_name}={item.left.amount:.2f}, " + f"{report.right_name}={item.right.amount:.2f}, " + f"difference={item.difference:.2f}" + ) + elif item.status is ReconciliationStatus.LEFT_ONLY: + assert item.left is not None + lines.append( + f"[LEFT_ONLY] {item.reference_id}: " + f"{report.left_name}={item.left.amount:.2f}" + ) + else: + assert item.right is not None + lines.append( + f"[RIGHT_ONLY] {item.reference_id}: " + f"{report.right_name}={item.right.amount:.2f}" + ) + + summary = report.summary + lines.extend( + [ + "", + "Summary", + f"Total items: {summary.total_items}", + f"Matched: {summary.matched}", + f"Amount mismatches: {summary.amount_mismatches}", + f"Left only: {summary.left_only}", + f"Right only: {summary.right_only}", + ( + "Total absolute difference: " + f"{summary.total_absolute_difference:.2f}" + ), + ] + ) + return "\n".join(lines) + "\n" diff --git a/practical-projects/07-fictional-reconciliation-workflow/tests/conftest.py b/practical-projects/07-fictional-reconciliation-workflow/tests/conftest.py new file mode 100644 index 0000000..6190fbf --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/tests/conftest.py @@ -0,0 +1,5 @@ +from pathlib import Path +import sys + +PROJECT_ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(PROJECT_ROOT)) diff --git a/practical-projects/07-fictional-reconciliation-workflow/tests/test_decimal_precision.py b/practical-projects/07-fictional-reconciliation-workflow/tests/test_decimal_precision.py new file mode 100644 index 0000000..2721b31 --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/tests/test_decimal_precision.py @@ -0,0 +1,105 @@ +from decimal import Decimal, localcontext + +import pytest + +import reconciliation +from reconciliation import MAX_INTEGER_DIGITS, ReconciliationRecord, reconcile + + +def record(reference_id: str, amount: str) -> ReconciliationRecord: + return ReconciliationRecord(reference_id, Decimal(amount)) + + +def test_record_accepts_valid_amount_under_low_decimal_precision() -> None: + with localcontext() as context: + context.prec = 3 + item = ReconciliationRecord("REF-001", Decimal("10.00")) + + assert item.amount == Decimal("10.00") + assert item.amount.as_tuple().exponent == -2 + + +def test_record_accepts_large_exact_amount_beyond_default_precision() -> None: + amount = Decimal("99999999999999999999999999.99") + + item = ReconciliationRecord("REF-001", amount) + + assert item.amount == amount + assert item.amount.as_tuple().exponent == -2 + + +def test_record_accepts_documented_integer_digit_boundary() -> None: + amount = Decimal(f"{'9' * MAX_INTEGER_DIGITS}.99") + + item = ReconciliationRecord("REF-001", amount) + + assert item.amount == amount + assert item.amount.as_tuple().exponent == -2 + + +@pytest.mark.parametrize("amount", ["1e100", "-1e100", "1e1000000"]) +def test_record_rejects_amount_above_integer_digit_limit(amount: str) -> None: + with pytest.raises( + ValueError, + match=rf"at most {MAX_INTEGER_DIGITS} integer digits", + ): + ReconciliationRecord("REF-001", Decimal(amount)) + + +def test_record_rejects_extreme_subcent_exponent_without_large_power() -> None: + amount = Decimal("1e-1000000000") + + with pytest.raises(ValueError, match="at most two decimal places"): + ReconciliationRecord("REF-001", amount) + + +def test_record_discards_long_fractional_zero_tail_before_integer_conversion( + monkeypatch: pytest.MonkeyPatch, +) -> None: + observed_digit_lengths: list[int] = [] + original_digits_to_int = reconciliation._digits_to_int + + def recording_digits_to_int(digits: tuple[int, ...]) -> int: + observed_digit_lengths.append(len(digits)) + return original_digits_to_int(digits) + + monkeypatch.setattr(reconciliation, "_digits_to_int", recording_digits_to_int) + amount = Decimal("1." + "0" * 200_000) + + item = ReconciliationRecord("REF-001", amount) + + assert item.amount == Decimal("1.00") + assert observed_digit_lengths + assert max(observed_digit_lengths) <= 3 + + +def test_reconcile_preserves_difference_beyond_decimal_context_precision() -> None: + amount = "99999999999999999999999999.99" + + report = reconcile( + [record("REF-001", amount)], + [record("REF-001", f"-{amount}")], + ) + + assert report.items[0].difference == Decimal( + "199999999999999999999999999.98" + ) + + +def test_summary_preserves_sum_beyond_decimal_context_precision() -> None: + amount = "99999999999999999999999999.99" + + report = reconcile( + [ + record("REF-001", amount), + record("REF-002", amount), + ], + [ + record("REF-001", "0.00"), + record("REF-002", "0.00"), + ], + ) + + assert report.summary.total_absolute_difference == Decimal( + "199999999999999999999999999.98" + ) diff --git a/practical-projects/07-fictional-reconciliation-workflow/tests/test_reconciliation.py b/practical-projects/07-fictional-reconciliation-workflow/tests/test_reconciliation.py new file mode 100644 index 0000000..3aa091a --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/tests/test_reconciliation.py @@ -0,0 +1,315 @@ +from decimal import Decimal + +import pytest + +from reconciliation import ( + ReconciliationItem, + ReconciliationRecord, + ReconciliationStatus, + reconcile, + render_text_report, +) + + +def record(reference_id: str, amount: str) -> ReconciliationRecord: + return ReconciliationRecord(reference_id, Decimal(amount)) + + +def test_record_trims_reference_id_and_canonicalizes_amount() -> None: + item = record(" REF-001 ", "10") + + assert item.reference_id == "REF-001" + assert item.amount == Decimal("10.00") + assert item.amount.as_tuple().exponent == -2 + + +@pytest.mark.parametrize("reference_id", ["", " ", "\t"]) +def test_record_rejects_empty_reference_id(reference_id: str) -> None: + with pytest.raises(ValueError, match="reference_id must not be empty"): + record(reference_id, "10.00") + + +def test_record_rejects_non_string_reference_id() -> None: + with pytest.raises(TypeError, match="reference_id must be a string"): + ReconciliationRecord(101, Decimal("10.00")) # type: ignore[arg-type] + + +def test_record_rejects_non_decimal_amount() -> None: + with pytest.raises(TypeError, match="amount must be a Decimal"): + ReconciliationRecord("REF-001", 10.00) # type: ignore[arg-type] + + +@pytest.mark.parametrize("amount", ["NaN", "Infinity", "-Infinity"]) +def test_record_rejects_non_finite_amount(amount: str) -> None: + with pytest.raises(ValueError, match="amount must be finite"): + record("REF-001", amount) + + +def test_record_rejects_more_than_two_decimal_places() -> None: + with pytest.raises(ValueError, match="at most two decimal places"): + record("REF-001", "10.001") + + +def test_record_normalizes_negative_zero() -> None: + item = record("REF-001", "-0.00") + + assert item.amount == Decimal("0.00") + assert f"{item.amount:.2f}" == "0.00" + + +def test_reconcile_matches_equal_records() -> None: + report = reconcile( + [record("REF-001", "150.00")], + [record("REF-001", "150.00")], + ) + + item = report.items[0] + assert item.status is ReconciliationStatus.MATCHED + assert item.difference == Decimal("0.00") + assert report.summary.matched == 1 + + +def test_reconcile_detects_amount_mismatch_with_signed_difference() -> None: + report = reconcile( + [record("REF-001", "275.50")], + [record("REF-001", "270.50")], + ) + + item = report.items[0] + assert item.status is ReconciliationStatus.AMOUNT_MISMATCH + assert item.difference == Decimal("5.00") + assert report.summary.total_absolute_difference == Decimal("5.00") + + +def test_reconcile_preserves_negative_signed_difference() -> None: + report = reconcile( + [record("REF-001", "20.00")], + [record("REF-001", "25.50")], + ) + + assert report.items[0].difference == Decimal("-5.50") + assert report.summary.total_absolute_difference == Decimal("5.50") + + +def test_reconcile_detects_left_only_and_right_only() -> None: + report = reconcile( + [record("REF-001", "10.00")], + [record("REF-002", "20.00")], + ) + + assert [item.status for item in report.items] == [ + ReconciliationStatus.LEFT_ONLY, + ReconciliationStatus.RIGHT_ONLY, + ] + assert report.summary.left_only == 1 + assert report.summary.right_only == 1 + + +def test_reconcile_sorts_results_by_reference_id() -> None: + report = reconcile( + [record("REF-003", "30.00"), record("REF-001", "10.00")], + [record("REF-002", "20.00"), record("REF-003", "30.00")], + ) + + assert [item.reference_id for item in report.items] == [ + "REF-001", + "REF-002", + "REF-003", + ] + + +def test_reconcile_accepts_single_pass_generators() -> None: + left = (record(f"REF-{number}", f"{number}.00") for number in (1, 2)) + right = (record(f"REF-{number}", f"{number}.00") for number in (1, 2)) + + report = reconcile(left, right) + + assert report.summary.matched == 2 + + +def test_reconcile_rejects_duplicate_reference_within_left_source() -> None: + with pytest.raises(ValueError, match="duplicate reference_id in Source A"): + reconcile( + [record("REF-001", "10.00"), record(" REF-001 ", "20.00")], + [], + ) + + +def test_reconcile_rejects_duplicate_reference_within_right_source() -> None: + with pytest.raises(ValueError, match="duplicate reference_id in Source B"): + reconcile( + [], + [record("REF-001", "10.00"), record("REF-001", "20.00")], + ) + + +def test_reconcile_rejects_invalid_record_type() -> None: + with pytest.raises(TypeError, match="must contain ReconciliationRecord"): + reconcile([object()], []) # type: ignore[list-item] + + +def test_reconcile_rejects_non_iterable_source() -> None: + with pytest.raises(TypeError, match="Source A must be an iterable"): + reconcile(None, []) # type: ignore[arg-type] + + +@pytest.mark.parametrize( + ("left_name", "right_name", "message"), + [ + ("", "Source B", "left_name must not be empty"), + ("Source A", " ", "right_name must not be empty"), + ("Same", "Same", "must be different"), + ], +) +def test_reconcile_validates_source_names( + left_name: str, + right_name: str, + message: str, +) -> None: + with pytest.raises(ValueError, match=message): + reconcile([], [], left_name=left_name, right_name=right_name) + + +def test_reconcile_rejects_non_string_source_name() -> None: + with pytest.raises(TypeError, match="left_name must be a string"): + reconcile([], [], left_name=123) # type: ignore[arg-type] + + +def test_reconcile_trims_source_names() -> None: + report = reconcile([], [], left_name=" Source A ", right_name=" Source B ") + + assert report.left_name == "Source A" + assert report.right_name == "Source B" + + +def test_empty_reconciliation_has_zeroed_summary() -> None: + report = reconcile([], []) + + assert report.items == () + assert report.summary.total_items == 0 + assert report.summary.matched == 0 + assert report.summary.amount_mismatches == 0 + assert report.summary.left_only == 0 + assert report.summary.right_only == 0 + assert report.summary.total_absolute_difference == Decimal("0.00") + + +def test_summary_counts_all_statuses_and_absolute_difference() -> None: + report = reconcile( + [ + record("REF-001", "10.00"), + record("REF-002", "30.00"), + record("REF-003", "40.00"), + ], + [ + record("REF-001", "10.00"), + record("REF-002", "25.00"), + record("REF-004", "12.00"), + ], + ) + + assert report.summary.total_items == 4 + assert report.summary.matched == 1 + assert report.summary.amount_mismatches == 1 + assert report.summary.left_only == 1 + assert report.summary.right_only == 1 + assert report.summary.total_absolute_difference == Decimal("5.00") + + +def test_reference_matching_is_case_sensitive() -> None: + report = reconcile( + [record("ref-001", "10.00")], + [record("REF-001", "10.00")], + ) + + assert [item.status for item in report.items] == [ + ReconciliationStatus.RIGHT_ONLY, + ReconciliationStatus.LEFT_ONLY, + ] + + +def test_reconciliation_item_rejects_inconsistent_left_only_shape() -> None: + left = record("REF-001", "10.00") + right = record("REF-001", "10.00") + + with pytest.raises(ValueError, match="left_only requires only a left record"): + ReconciliationItem( + reference_id="REF-001", + status=ReconciliationStatus.LEFT_ONLY, + left=left, + right=right, + difference=None, + ) + + +def test_reconciliation_item_rejects_wrong_difference() -> None: + left = record("REF-001", "15.00") + right = record("REF-001", "10.00") + + with pytest.raises(ValueError, match="difference must equal"): + ReconciliationItem( + reference_id="REF-001", + status=ReconciliationStatus.AMOUNT_MISMATCH, + left=left, + right=right, + difference=Decimal("4.00"), + ) + + +def test_render_text_report_is_deterministic() -> None: + report = reconcile( + [ + record("REF-001", "150.00"), + record("REF-002", "275.50"), + record("REF-003", "100.00"), + ], + [ + record("REF-001", "150.00"), + record("REF-002", "270.50"), + record("REF-004", "100.00"), + ], + left_name="Source North", + right_name="Source South", + ) + + assert render_text_report(report) == ( + "Reconciliation Report\n" + "Sources: Source North vs Source South\n" + "\n" + "[MATCHED] REF-001: 150.00 == 150.00\n" + "[AMOUNT_MISMATCH] REF-002: Source North=275.50, " + "Source South=270.50, difference=5.00\n" + "[LEFT_ONLY] REF-003: Source North=100.00\n" + "[RIGHT_ONLY] REF-004: Source South=100.00\n" + "\n" + "Summary\n" + "Total items: 4\n" + "Matched: 1\n" + "Amount mismatches: 1\n" + "Left only: 1\n" + "Right only: 1\n" + "Total absolute difference: 5.00\n" + ) + + +def test_render_empty_report_is_stable() -> None: + report = reconcile([], [], left_name="North", right_name="South") + + assert render_text_report(report) == ( + "Reconciliation Report\n" + "Sources: North vs South\n" + "\n" + "\n" + "Summary\n" + "Total items: 0\n" + "Matched: 0\n" + "Amount mismatches: 0\n" + "Left only: 0\n" + "Right only: 0\n" + "Total absolute difference: 0.00\n" + ) + + +def test_render_text_report_rejects_wrong_type() -> None: + with pytest.raises(TypeError, match="ReconciliationReport"): + render_text_report(object()) # type: ignore[arg-type] diff --git a/practical-projects/07-fictional-reconciliation-workflow/tests/test_text_safety.py b/practical-projects/07-fictional-reconciliation-workflow/tests/test_text_safety.py new file mode 100644 index 0000000..1643a55 --- /dev/null +++ b/practical-projects/07-fictional-reconciliation-workflow/tests/test_text_safety.py @@ -0,0 +1,36 @@ +from decimal import Decimal + +import pytest + +from reconciliation import ReconciliationRecord, reconcile + + +@pytest.mark.parametrize( + "reference_id", + [ + "REF-001\n[RIGHT_ONLY] SPOOF", + "REF-001\rSPOOF", + "REF-001\tSPOOF", + "REF-001\x00SPOOF", + ], +) +def test_record_rejects_non_printable_reference_id(reference_id: str) -> None: + with pytest.raises(ValueError, match="printable characters"): + ReconciliationRecord(reference_id, Decimal("10.00")) + + +@pytest.mark.parametrize( + ("left_name", "right_name"), + [ + ("Source A\n[RIGHT_ONLY] SPOOF", "Source B"), + ("Source A", "Source B\rSPOOF"), + ("Source A\tSPOOF", "Source B"), + ("Source A", "Source B\x00SPOOF"), + ], +) +def test_reconcile_rejects_non_printable_source_names( + left_name: str, + right_name: str, +) -> None: + with pytest.raises(ValueError, match="printable characters"): + reconcile([], [], left_name=left_name, right_name=right_name) diff --git a/practical-projects/README.es.md b/practical-projects/README.es.md index 550e51e..e2e0bf9 100644 --- a/practical-projects/README.es.md +++ b/practical-projects/README.es.md @@ -21,8 +21,8 @@ La Fase 10 combina conceptos de las fases anteriores en flujos completos y compr 3. ✅ [Registro de Usuarios](03-user-registration/README.es.md) 4. ✅ [Analizador CSV](04-csv-analyzer/README.es.md) 5. ✅ [Generador de Informes](05-report-generator/README.es.md) -6. 🚧 [Organizador de Archivos](06-file-organizer/README.es.md) -7. ⏳ Flujo Ficticio de Conciliación +6. ✅ [Organizador de Archivos](06-file-organizer/README.es.md) +7. 🚧 [Flujo Ficticio de Conciliación](07-fictional-reconciliation-workflow/README.es.md) 8. ⏳ Flujo Simulado de Automatización ## Contrato de los proyectos @@ -38,4 +38,4 @@ Cada proyecto debe incluir: - desafíos de extensión; - discusión de portafolio. -El Proyecto 01 establece el patrón de integración con registros monetarios validados y persistencia. El Proyecto 02 amplía ese patrón con políticas de calificación configurables, agregación ponderada exacta, estados parcial/final explícitos, informe estructurado y cobertura pytest centrada en límites. El Proyecto 03 añade datos de identidad canónicos, prevención de duplicados, índices secundarios de lookup, actualizaciones seguras de campos indexados y transiciones explícitas del ciclo de vida sin introducir autenticación. El Proyecto 04 añade schemas CSV exactos, conversión tipada, separación entre fallos estructurales y fallos de fila, parsing con éxito parcial, identificadores duplicados, filtros deterministas y agregación sin ocultar la ingestión detrás de pandas. El Proyecto 05 transforma registros operativos validados en artefactos de informe deterministas con ventanas de fecha explícitas, métricas de resumen exactas, renderizadores TXT/Markdown y escritura UTF-8, manteniendo separadas la agregación, la presentación y la persistencia. El Proyecto 06 añade descubrimiento superficial del filesystem, clasificación por sufijo, planificación inmutable de movimientos, políticas explícitas de colisión, fronteras de symlink, revalidación en el momento de ejecución y protección exacta no-replace del destino antes de organizar los archivos en carpetas por categoría. +El Proyecto 01 establece el patrón de integración con registros monetarios validados y persistencia. El Proyecto 02 amplía ese patrón con políticas de calificación configurables, agregación ponderada exacta, estados parcial/final explícitos, informe estructurado y cobertura pytest centrada en límites. El Proyecto 03 añade datos de identidad canónicos, prevención de duplicados, índices secundarios de lookup, actualizaciones seguras de campos indexados y transiciones explícitas del ciclo de vida sin introducir autenticación. El Proyecto 04 añade schemas CSV exactos, conversión tipada, separación entre fallos estructurales y fallos de fila, parsing con éxito parcial, identificadores duplicados, filtros deterministas y agregación sin ocultar la ingestión detrás de pandas. El Proyecto 05 transforma registros operativos validados en artefactos de informe deterministas con ventanas de fecha explícitas, métricas de resumen exactas, renderizadores TXT/Markdown y escritura UTF-8, manteniendo separadas la agregación, la presentación y la persistencia. El Proyecto 06 añade descubrimiento superficial del filesystem, clasificación por sufijo, planificación inmutable de movimientos, políticas explícitas de colisión, fronteras de symlink, revalidación en el momento de ejecución y protección exacta no-replace del destino antes de organizar los archivos en carpetas por categoría. El Proyecto 07 añade registros de conciliación validados, comparación monetaria exacta con `Decimal`, rechazo de claves duplicadas, clasificación determinista en cuatro estados, diferencias con signo, resúmenes inmutables y resultados de dominio separados de la presentación. diff --git a/practical-projects/README.md b/practical-projects/README.md index c989311..57840cc 100644 --- a/practical-projects/README.md +++ b/practical-projects/README.md @@ -21,8 +21,8 @@ Phase 10 combines concepts from the previous phases into complete, testable work 3. ✅ [User Registration](03-user-registration/README.md) 4. ✅ [CSV Analyzer](04-csv-analyzer/README.md) 5. ✅ [Report Generator](05-report-generator/README.md) -6. 🚧 [File Organizer](06-file-organizer/README.md) -7. ⏳ Fictional Reconciliation Workflow +6. ✅ [File Organizer](06-file-organizer/README.md) +7. 🚧 [Fictional Reconciliation Workflow](07-fictional-reconciliation-workflow/README.md) 8. ⏳ Simulated Automation Flow ## Project contract @@ -38,4 +38,4 @@ Each project should include: - extension challenges; - portfolio discussion. -Project 01 establishes the integration pattern with validated monetary records and persistence. Project 02 extends it with configurable grading policies, exact weighted aggregation, explicit partial/final states, structured reporting, and boundary-focused pytest coverage. Project 03 adds canonical identity-like data, duplicate prevention, secondary lookup indexes, safe indexed-field updates, and explicit user lifecycle transitions without introducing authentication. Project 04 adds exact CSV schemas, typed conversion, structural-versus-row failure handling, partial-success parsing, duplicate row identifiers, deterministic filtering, and aggregation without hiding ingestion behavior behind pandas. Project 05 turns validated operational records into deterministic reporting artifacts with explicit date windows, exact summary metrics, TXT/Markdown renderers, and UTF-8 file output while keeping aggregation, presentation, and persistence separate. Project 06 adds shallow filesystem discovery, suffix classification, immutable move planning, explicit collision policies, symlink boundaries, execution-time revalidation, and exact no-replace destination protection before files are organized into category folders. +Project 01 establishes the integration pattern with validated monetary records and persistence. Project 02 extends it with configurable grading policies, exact weighted aggregation, explicit partial/final states, structured reporting, and boundary-focused pytest coverage. Project 03 adds canonical identity-like data, duplicate prevention, secondary lookup indexes, safe indexed-field updates, and explicit user lifecycle transitions without introducing authentication. Project 04 adds exact CSV schemas, typed conversion, structural-versus-row failure handling, partial-success parsing, duplicate row identifiers, deterministic filtering, and aggregation without hiding ingestion behavior behind pandas. Project 05 turns validated operational records into deterministic reporting artifacts with explicit date windows, exact summary metrics, TXT/Markdown renderers, and UTF-8 file output while keeping aggregation, presentation, and persistence separate. Project 06 adds shallow filesystem discovery, suffix classification, immutable move planning, explicit collision policies, symlink boundaries, execution-time revalidation, and exact no-replace destination protection before files are organized into category folders. Project 07 adds validated reconciliation records, exact `Decimal` comparison, duplicate-key rejection, deterministic four-state classification, signed differences, immutable summaries, and presentation-independent reconciliation results. diff --git a/practical-projects/README.pt-BR.md b/practical-projects/README.pt-BR.md index bcdfa79..e36c5cb 100644 --- a/practical-projects/README.pt-BR.md +++ b/practical-projects/README.pt-BR.md @@ -21,8 +21,8 @@ A Fase 10 combina conceitos das fases anteriores em fluxos completos e testávei 3. ✅ [Cadastro de Usuários](03-user-registration/README.pt-BR.md) 4. ✅ [Analisador CSV](04-csv-analyzer/README.pt-BR.md) 5. ✅ [Gerador de Relatórios](05-report-generator/README.pt-BR.md) -6. 🚧 [Organizador de Arquivos](06-file-organizer/README.pt-BR.md) -7. ⏳ Fluxo Fictício de Conciliação +6. ✅ [Organizador de Arquivos](06-file-organizer/README.pt-BR.md) +7. 🚧 [Fluxo Fictício de Conciliação](07-fictional-reconciliation-workflow/README.pt-BR.md) 8. ⏳ Fluxo Simulado de Automação ## Contrato dos projetos @@ -38,4 +38,4 @@ Cada projeto deve incluir: - desafios de extensão; - discussão de portfólio. -O Projeto 01 estabelece o padrão de integração com registros monetários validados e persistência. O Projeto 02 amplia esse padrão com políticas de notas configuráveis, agregação ponderada exata, estados parcial/final explícitos, relatório estruturado e cobertura pytest focada em fronteiras. O Projeto 03 adiciona dados de identidade canônicos, prevenção de duplicidade, índices secundários de lookup, atualizações seguras de campos indexados e transições explícitas de ciclo de vida sem introduzir autenticação. O Projeto 04 adiciona schemas CSV exatos, conversão tipada, separação entre falhas estruturais e falhas de linha, parsing com sucesso parcial, identificadores duplicados, filtros determinísticos e agregação sem esconder a ingestão atrás de pandas. O Projeto 05 transforma registros operacionais validados em artefatos de relatório determinísticos com janelas explícitas de datas, métricas exatas de resumo, renderizadores TXT/Markdown e escrita UTF-8, mantendo agregação, apresentação e persistência separadas. O Projeto 06 adiciona descoberta rasa no filesystem, classificação por sufixo, planejamento imutável de movimentos, políticas explícitas de colisão, fronteiras de symlink, revalidação no momento da execução e proteção exata no-replace do destino antes da organização em pastas por categoria. +O Projeto 01 estabelece o padrão de integração com registros monetários validados e persistência. O Projeto 02 amplia esse padrão com políticas de notas configuráveis, agregação ponderada exata, estados parcial/final explícitos, relatório estruturado e cobertura pytest focada em fronteiras. O Projeto 03 adiciona dados de identidade canônicos, prevenção de duplicidade, índices secundários de lookup, atualizações seguras de campos indexados e transições explícitas de ciclo de vida sem introduzir autenticação. O Projeto 04 adiciona schemas CSV exatos, conversão tipada, separação entre falhas estruturais e falhas de linha, parsing com sucesso parcial, identificadores duplicados, filtros determinísticos e agregação sem esconder a ingestão atrás de pandas. O Projeto 05 transforma registros operacionais validados em artefatos de relatório determinísticos com janelas explícitas de datas, métricas exatas de resumo, renderizadores TXT/Markdown e escrita UTF-8, mantendo agregação, apresentação e persistência separadas. O Projeto 06 adiciona descoberta rasa no filesystem, classificação por sufixo, planejamento imutável de movimentos, políticas explícitas de colisão, fronteiras de symlink, revalidação no momento da execução e proteção exata no-replace do destino antes da organização em pastas por categoria. O Projeto 07 adiciona registros de conciliação validados, comparação monetária exata com `Decimal`, rejeição de chaves duplicadas, classificação determinística em quatro estados, diferenças com sinal, resumos imutáveis e resultados de domínio separados da apresentação. diff --git a/scripts/example_manifest.txt b/scripts/example_manifest.txt index dd58562..89f81cb 100644 --- a/scripts/example_manifest.txt +++ b/scripts/example_manifest.txt @@ -184,3 +184,4 @@ practical-projects/03-user-registration/demo.py practical-projects/04-csv-analyzer/demo.py practical-projects/05-report-generator/demo.py practical-projects/06-file-organizer/demo.py +practical-projects/07-fictional-reconciliation-workflow/demo.py