Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,12 @@ jobs:
- name: Install dependencies
run: npm ci --no-audit --no-fund

# The committed dist/ is the 0.1.17 release build (gulp/browserify): the
# contract tests and the Jasmine specs must keep passing on it
- name: Tests on the committed dist (0.1.17 build) and the sources
# The committed dist/ is the 0.1.17 release build (gulp/browserify, Math.js
# 9.5.0): tested against the contract of that release
- name: Committed dist against the 0.1.17 contract
run: npm run test:baseline

- name: Sources (browserify + the package transform) and Jasmine specs
run: npm test

# Isolated copy: npm ci, build, artifacts (ES5, source maps), tests on the
Expand Down Expand Up @@ -71,7 +74,11 @@ jobs:
run: npm run build

- name: Tests on the fresh dist (standalone and module bundles, plain and minified), the sources and the Jasmine specs
run: npm test
shell: bash
run: |
set -e
npm run test:dist
npm test

- name: The build changed nothing outside dist/
shell: bash
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,32 @@

## Unreleased

### Dependencies: Math.js 15.2.0 (from 9.5.0)

- **Runtime requirements**: Math.js 13+ requires ES2020 and `BigInt` (Chrome and
Edge 80+, Firefox 74+, Safari 14+ from the ES2020 features used, not tested in
those browsers; no Internet Explorer). The bundles contain
Math.js as published (ES2020 syntax); the package sources still compile to ES5.
- Fractions use fraction.js 5 (`bigint`): exact at any size (0.1.17: doubles,
exact up to 2^53). The arithmetic helpers stay exact; long sums such as the
mean of a probabilistic sensitivity analysis grow without limit; the
aggregation in sd-computations has to bound them (separate change there). `toFloat` returns the correctly rounded double (ties to
even) also for Fractions beyond the double range (fraction.js 5 `valueOf()`
gives Infinity or NaN there).
- `eval(expr, true)` rejects a `Unit` result as before (Math.js 14+ would convert
the value of the unit).
- The JSON form of a Fraction (`toJSON`, data model DTO of the job repository)
writes `n` and `d` as strings; the 0.1.17 form with numbers is still read.
Saved diagrams are unchanged (Fractions are saved as fraction strings).
- Behaviour of Math.js for user expressions changes in many places, some of them
silently: precedence of `%` (percent and modulus) and `mod`, implicit
multiplication after a signed constant division, tolerances of comparisons and
`isInteger`, `mod` round-off, matrix functions, `print` indices, `typeOf`, …
(`docs/MATHJS_UPGRADE.md`, with a proposal for old diagrams).
- Security: Math.js 15.2.0 fixes GHSA-jvff-x2qm-6286, GHSA-29qv-4j9f-fjw5
(CVE-2026-40897) and GHSA-5v89-rwgr-qj6g (CVE-2026-41139), which affect
13.1.0 - 15.1.x (not 9.5.0).

### Build

- The `dist/` bundles are built with Rollup and npm scripts instead of
Expand Down
45 changes: 32 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# sd-expression-engine
SilverDecisions expression engine module: evaluation of the expressions of
SilverDecisions diagrams with [Math.js](https://mathjs.org/) 9.5.0, exact
SilverDecisions diagrams with [Math.js](https://mathjs.org/) 15.2.0, exact
Fraction arithmetic and the sd-random distributions.

[![CI](https://github.com/SilverDecisions/sd-expression-engine/actions/workflows/ci.yml/badge.svg)](https://github.com/SilverDecisions/sd-expression-engine/actions/workflows/ci.yml)
Expand Down Expand Up @@ -29,15 +29,23 @@ Built files (`dist/`, committed, rebuilt by the release):

The plain files carry an inline source map, the minified ones a `.map` file.

Runtime requirements: Math.js 13+ requires ES2020 and `BigInt` (Chrome and Edge 80+,
Firefox 74+, Safari 14+ from the ES2020 features used, not tested in those
browsers; no Internet Explorer). The package sources compile to
ES5; the bundles and consumer bundles contain Math.js as published (ES2020).
Releases up to 0.1.17 (Math.js 9.5.0) ran on ES5 engines.

## Development

Node 24 (`.nvmrc`); CI runs Node 22 and 24 on Linux and a build smoke test on
Windows. Math.js is pinned to exactly 9.5.0.
Windows. Math.js is pinned to an exact version (15.2.0).

```bash
npm ci
npm run build # dist/ with Rollup (rollup.config.mjs), ~2 min (terser on the large bundles)
npm test # contract tests on dist/ and the sources, Jasmine specs in Node
npm test # contract tests on the sources (browserify + transform), Jasmine specs in Node
npm run test:dist # contract tests on the four bundles in dist/ (after npm run build)
npm run test:baseline # the committed dist/ (0.1.17 build) against the 0.1.17 contract
npm run test:build # isolated build check, see scripts/check-build.js
```

Expand All @@ -49,8 +57,8 @@ and sd-random are compiled first by `scripts/browserify-transform.mjs` with the
Babel 7 options of their own `browserify.transform` field and CommonJS modules,
exactly as babelify compiles them for consumers, so the module interop
(`src/mathjs.js` sets `module.exports`, `import * as math`) is Babel's.
Compilation target: ES5 syntax (`@babel/preset-env` without targets, no
browserslist config); class fields are assignments (Babel 6
Compilation target of the package sources: ES5 syntax (`@babel/preset-env`
without targets, no browserslist config); class fields are assignments (Babel 6
`transform-class-properties`). Math.js (`lib/cjs`, through its `require`
export condition, as browserify resolves `main`) and its dependencies are
bundled as published, not compiled; the `browser` fields are honoured
Expand All @@ -59,8 +67,8 @@ for compatibility with the consumers' transform; a migration of that
configuration (and Babel 8) is a separate task.

`npm run test:build` (`scripts/check-build.js`): copy of the working tree,
`npm ci`, build without warnings, the six artifacts exist and the bundles are
ES5, source maps are valid JSON with `sources`/`sourcesContent` and map a known
`npm ci`, build without warnings, the six artifacts exist and the bundles parse
as ES2020 (the package sources compiled by a consumer as ES5), source maps are valid JSON with `sources`/`sourcesContent` and map a known
statement back to `src/expression-engine.js`, nothing changed outside `dist/`,
contract tests on the four fresh bundles and the sources, Jasmine specs,
`npm pack` installed into an empty browserify 17 + babelify 10 consumer (contract
Expand All @@ -86,19 +94,28 @@ artifact, chosen with `SD_EXPRESSION_ENGINE_TARGET`:
| a file path | a CommonJS module exporting the package namespace |

`SD_EXPRESSION_ENGINE_DIST=<dir>` tests the bundles of another `dist/` directory.
`SD_EXPRESSION_ENGINE_BASELINE=<release>` (`0.1.17`) tests built artifacts of
that release against its contract: the Math.js version it contains (9.5.0) and
the version-dependent expectations; without it the current contract applies
(Math.js of `package.json`) and a bundle with another Math.js version fails.

```bash
npm test # all five targets on ./dist and the sources, then Jasmine
npm test # the sources (target source), then Jasmine
node test/contract/run.js # all five targets on ./dist and the sources
npm run test:baseline # committed dist/ against the 0.1.17 contract
SD_EXPRESSION_ENGINE_TARGET=module node --test test/contract/ # one target
node test/contract/run.js --targets=module,source
node test/contract/run.js --dist=<dir> # another dist/ directory
npm run test:published # the bundles of [email protected] from npm
npm run test:published # the bundles of [email protected] from npm (0.1.17 contract)
npx jasmine --config=test/jasmine.json # the Jasmine specs (test/specs) in Node
```

The committed `dist/` is the 0.1.17 release build (gulp/browserify): `npm test`
on a fresh checkout tests it (CI does so as well), `npm run test:build` tests a
fresh build. The Jasmine specs compile the sources with `@babel/register`
The committed `dist/` is the 0.1.17 release build (gulp/browserify, Math.js
9.5.0) until the next release: `npm run test:baseline` tests it against the
0.1.17 contract (CI does so as well), `npm test` the current sources and
`npm run test:build` a fresh build. Tests whose expectation depends on the Math.js
version (`test/contract/mathjs-version.test.js`, `[version difference]`) assert
both versions explicitly. The Jasmine specs compile the sources with `@babel/register`
(`test/jasmine-babel.js`) and run in Node; the bundles are not run in a real
browser or Web Worker by these tests (see the integration notes of the PRs).

Expand Down Expand Up @@ -129,7 +146,9 @@ pass while the defect is present and fail once it is fixed.
`eval(expr, false)` reject them.
- `serialize(±Infinity)` and `round(±Infinity)` throw; `add` / `divide` of
`Infinity` and a finite value throw (implicit conversion to Fraction).
- `median` and `std` do not accept Fractions (Math.js 9, `number` config).
- `median` and `std` do not accept Fractions (`number` config).
- An expression can change the shared Math.js configuration for all later
evaluations (`config({number: "BigNumber"})`).
- `isExpressionObject` is false for a live Fraction; the replacer works inside
`JSON.stringify`, which calls `toJSON()` first.
- The Math.js instance is shared: `createUnit` in an expression changes it for
Expand Down
149 changes: 149 additions & 0 deletions docs/MATHJS_UPGRADE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# Math.js 9.5.0 → 15.2.0: behaviour changes and old diagrams

Status: proposal for decision. Nothing of the handling of old diagrams below
is implemented; the engine (PR "upgrade Math.js to 15.2.0") only makes the
differences explicit in `test/contract/mathjs-version.test.js` (`DIFFERENCES`,
the complete list of reproduced cases, asserted for both versions).

## Where the differences come from

1. **Math.js itself** (HISTORY.md 10.0 - 15.2, each case reproduced):
- parser: `%` before a term is modulus and unary percent binds tighter
(15.0: `100 + 3% + 1` 104 → 100, `x% + 1` 1.02 → 0); `mod` has the
precedence of `*` and `/` (14.0: `10 / 4 mod 3` 10 → 2.5); implicit
multiplication after a signed constant division (11.0: `-1 / 2 x` -0.25 →
-1 for x = 2); empty true branch of `?:` is an error (15.1.1); a delimiter
after hex/bin/oct literals (15.0);
- tolerances: comparisons use `relTol` 1e-12 and `absTol` 1e-15 instead of a
relative 1e-12 and an absolute 2.2e-16 (13.0: `x > 0` for x = 1e-15 true →
false, `abs(x - 0.3) < 1e-15` for x = 0.1 + 0.2 true → false, `isZero`,
`isPositive`, `compare`); `isInteger` with tolerance (15.1:
`isInteger(0.1 * 3 * 10)` false → true); `mod` round-off (11.11.1:
`0.3 mod 0.1` 0.0999… → -5.6e-17);
- functions: element-wise `sqrt`/`exp`/`sin`/… of matrices removed (11.0),
`size` returns an array and subsets keep the shape of the index (15.0),
`print` uses one-based indices in expressions (11.11.1), `typeOf` of a
function is `function` (11.0), exact `combinations` (10.2), `mod` with a
negative divisor (12.1), US liquid units (13.0.3), new functions and
operators (`??`, `?.`, `isFinite`, `cumsum`, …).
2. **fraction.js 5** (Math.js 14.0): Fractions hold bigint numerators and
denominators, so they are exact at any size (0.1.17: doubles, exact up to
2^53); numeric strings keep every digit; the JSON form writes `n` and `d`
as strings; `valueOf()` overflows for large parts (the engine's `toFloat`
converts with bigint, correctly rounded); long sums grow without limit (the
mean of the probabilistic sensitivity analysis, handled in sd-computations).
3. **The engine adapter**: only `toFloat` (bigint conversion) and the rejection
of Unit results in `toNumber` / `eval(expr, true)` (as in 0.1.17). The
arithmetic helpers are exact (no rounding policy).

Expressions of the 16 saved diagrams in the test data of SilverDecisions and
sd-computations give the same values in both versions; the differences above
come from constructs those files do not use. Syntax alone does not show all of
them: tolerances, `isInteger`, `mod`, matrix functions depend on the values,
which change with sensitivity analyses and random draws.

## Proposal: old diagrams

### 1. Semantics version of a diagram

- Saved files carry the SilverDecisions version (`"SilverDecisions": "1.2.1"`);
all releases up to 1.2.x use the engine 0.1.x (Math.js ≤ 9.5). A file without
the new field below is a **legacy** file.
- New field written on save, e.g. `data.expressionSemantics: "mathjs-15"` (a
name of the expression semantics, not of the app version, so that a later
Math.js upgrade can add another one).
- Jobs (sd-computations `fix/job-resume-semantics`, a draft, not ready to be
merged on its own, see the decision on expressions below): it stores an expression
semantics profile in every job execution: profile version, full Math.js
version and the whole Math.js configuration (records without it: 0.1.17 with
its defaults). A stopped job is continued only under the same profile;
otherwise the old job is kept and a new one starts from the beginning, and a
worker with another profile than the page does not compute the page's jobs.
The profile is read from the shared Math.js instance, so a decision on the
comparison tolerances (`absTol`) changes it, and so does `config(...)` in an
expression: such a diagram cannot run restartable analyses in the
SilverDecisions worker (the worker evaluated its code, the page did not) until
expressions either may not change the configuration or each evaluation gets a
configuration owned by the diagram (a decision, see below). This refusal
applies with 0.1.17 too. The profile does not cover the rest of the mutable
Math.js state (`createUnit`, `import`).

### 2. Classification on opening a legacy file (before any recomputation)

Every expression (global code, node code, payoffs, probabilities, text
templates with `math(...)`) is parsed with the **legacy parser** (Math.js 9.5
`parse`, a separate bundle loaded only for legacy files) and with the current
parser.

- **S, syntax only** (precedence of `%`/`mod`, unary percent, implicit
multiplication): migratable. The legacy tree printed with all parentheses
(`node.toString({parenthesis: 'all', implicit: 'show'})`) is parsed by the
current parser; the migration is offered only if the two trees are
structurally equal (node types, operators, names, constants). Expressions that
parse to equal trees in both versions need no change.
- **F, changed functions** (matrix element-wise functions, `size`, matrix
indexing, `print`, `typeOf`, `mod`, `isInteger`, `combinations`, units, …):
found by the names in the tree; not migrated automatically (the result
depends on the argument values and types).
- **T, tolerance-dependent** (`==`, `!=`, `<`, `<=`, `>`, `>=`, `compare`,
`equal`, `larger`, `smaller`, `isZero`, `isPositive`, `isNegative`, `sign`):
value-dependent. Option T1 (engine-wide): keep the 0.1.17 comparisons by
creating Math.js with `absTol: Number.EPSILON` — reproduced: it restores
every comparison case of the table (not `isInteger`, `mod`, `floor`/`ceil`).
Option T2: no change, flag as F.
- Parse errors in one version only (`c ? : b`, `??`, …) are reported.

### 3. Dynamic check (value-dependent differences)

For a legacy file the base case is evaluated with both engines (legacy bundle
and current) and all values are compared: diagram and node scopes, payoffs,
probabilities, the computed values of the current rule, interpolated texts.
This finds differences the syntax cannot show (a comparison at 1e-15, an
`isInteger` of a computed value). It covers the evaluated values only: a
sensitivity analysis or random draws can reach other values, so expressions of
class F/T stay flagged even when the base case is equal.

### 4. Before recomputation

The file is shown with its saved computed values (marked as computed by
SilverDecisions x.y), not recomputed, until the user decides:

- nothing found (no S/F/T findings, equal base case): open normally, mark as
current semantics on the next save;
- only S: offer the migration (list old and new text of each expression), then
recompute;
- F/T findings or base-case differences: a report (node, field, expression,
value with 0.1.x, value now); choices: migrate S and accept the new
behaviour for the rest, keep the file read-only, or cancel. No silent
recomputation.

### 5. No repeated warnings

Accepting writes `expressionSemantics` (and e.g. `migratedFrom: "1.2.1"`) into
the diagram; the next save stores it and later openings do not warn. Without a
save the warning appears again (the file is still legacy). Jobs: results of
jobs completed by 0.1.x are data and are shown as they are; a re-run uses the
semantics of the current data model; a job stopped by 0.1.x is not continued
(section 1).

### Costs and open points

- The legacy parser bundle (Math.js 9.5) is loaded only for legacy files; size
and whether `parse` can be bundled without the rest of Math.js 9 are to be
measured.
- Templates of texts (`${...}`, lodash templates with `math("...")`) need a
parser of their own to find the expressions.

## Decisions needed

1. Runtime requirement ES2020 + BigInt (Math.js ≥ 13).
2. Comparisons: T1 (keep the 0.1.17 tolerances with `absTol: Number.EPSILON`)
or the Math.js 13+ tolerances.
3. Handling of legacy diagrams (sections 1-5) or an alternative, before a
SilverDecisions release with Math.js 15.
4. Expressions that change the shared Math.js instance (`config(...)`,
`import`, `createUnit`): allowed as now (their jobs in the worker are refused),
rejected, or isolated per diagram; this decides whether the expression
semantics profile of jobs can be guaranteed for every diagram.
5. JSON form of Fractions with string `n` / `d` in the job repository (read
back from 0.1.17 records works, the reverse does not).
Loading