EPA SWMM 5.2.4 in R, written by a translator that reads EPA's C, and judged against EPA itself: the binary .out
file byte for byte, and the .rpt report line for line except the three lines that print the wall clock.
Result: 465 of the 480 gate models identical to EPA 5.2.4; the other 15 not run (timed out after 1800 s), each identical to EPA's output as far as it had got; none differs. An independent rerun of a sample from a clean checkout: 102 of 103 identical, 1 not run (timed out after 1800 s). R runs SWMM about 400 times slower than EPA's C.
It is not a new engine and not a wrapper: there is no C in it (no .C, .Call or dyn.load), it calls no other
port, and it uses base R only. Every number it writes is EPA 5.2.4's, including the results of EPA's own defects
wherever EPA's behaviour is defined (lessons/02-epa-defects-found.md). R is one of the languages of Bob
Dickinson's Code Rosetta Stone app.
The same as the earlier ports' (see go/ or the swmm5-go repository): EPA SWMM 5.2.4 built from the USEPA v5.2.4
tag with gcc 13.3 -O2 on x86-64 Linux with glibc 2.39 (ref/Makefile), on a processor with FMA and AVX2; each
model run in its own folder; pass = .out byte-identical (when EPA writes one) and .rpt identical after removing
Analysis begun on, Analysis ended on and Total elapsed time, the exit code the same (EPA's command line always
exits 0). When an input error stops EPA before it writes results, the engine must not write a results file either.
Corpus: the 480 gate models of gate/models_all.tsv from SWMMBobSWMM6/1729-SWMM5-Models-2030.
Requires R 4.3.3 (Ubuntu's r-base-core, Rscript), base R only: the engine uses the base, compiler and
stats packages that come with R. gcc is used only by the runtime checks, to write glibc's reference values.
r/build.sh # once: byte-compiles the two engines (about 40 s)
r/runswmm model.inp model.rpt model.out # = Rscript --vanilla r/runswmm.R ...To regenerate the engines: python3 tools/c2x/emit_r.py r/swmm5 from the repository root (needs
pip install libclang==18.1.1, the clang 18 headers at /usr/lib/llvm-18/lib/clang/18/include, and Rscript,
which the translator asks once whether R reads each decimal literal exactly; about 1 minute). It writes
r/swmm5/engine.R and r/swmm5/engine_fast.R.
| File | Lines | What |
|---|---|---|
swmm5/engine.R |
35,667 | the safe engine, generated by tools/c2x/emit_r.py from EPA's C (52 C files; odesolve.c, mempool.c, six helpers and three rdii.c functions are hand ports) |
swmm5/engine_fast.R |
35,668 | the fast engine, generated from the same C by the same translator (below) |
swmm5/rt.R |
851 | the C library pieces SWMM uses: CPtr strings, boxes, reference arrays, x86's integer conversions, printf, exact strtod and sscanf, stdio, and log, log10 and pow where R's differ from glibc's |
swmm5/overrides.R |
259 | hand ports of odesolve.c, mempool.c and six helpers that use pointer arithmetic (from the Python port's overrides.py), and of three rdii.c functions whose arrays are kept short (below) |
swmm5/build.R, build.sh |
18, 6 | byte-compile both engines and save them (engine_fast.rds, engine_safe.rds, not committed) |
runswmm.R, runswmm |
51, 3 | EPA's run/main.c, and the shell launcher |
tools/c2x/emit_r.py |
1,691 | the translator back end |
tools/c2x/front.py parses EPA's C with libclang; tools/c2x/emit_r.py subclasses the Python back end
(emit_py.py) for its analysis (the front end's pointer modes, names made unique per function, string literals) and
writes all of its R itself. When the port disagreed with EPA, the fix went into the translator or the runtime, never
into the generated files. The mapping, in full in the emitter's header:
- Numbers. Every C integer and floating type is an R double. R's own integers are 32-bit, turn an overflow into
NA, and its
%/%and%%floor where C truncates; doubles hold every integer EPA computes exactly. C's integer/istrunc(a / b)(exact for 32-bit operands),%isa - trunc(a / b) * b, anunsignedresult is reduced with%% 4294967296,<<and>>are a multiplication and a floor division by a power of two, and&,|,^go through the runtime on 16-bit halves (R'sbitwAndtakes 32-bit integers and cannot hold 0x80000000, which is NA).(int)xand(long)xof a double are written inline as x86-64 does them: truncation, and the "integer indefinite" value INT_MIN or LONG_MIN for NaN and out-of-range values (EPA relies on it: lessons/02, defect 2).(float)xisrt_f32, a 4-bytewriteBin, which is C's cast (cvtsd2ss). - Values and references. R vectors and lists are values: assigning or passing one and then changing it changes
a copy. Environments are references. So a C struct is an environment (constructor
T_<struct>, cloneK_<struct>, copy-intoC_<struct>); a struct assignment copies into the existing environment, so an array of structs, a plain R list of environments, never changes after it is made. An array of numbers or pointers is a reference array: an environment whose fieldvholds the vector (rt_ra), so a function that fills an array it was given fills the caller's.char*is a CPtr,list(buffer, offset), the buffer a reference array of byte values. A pointer to a number is a box (an environment with fieldv). A C local whose address is passed to a function stays a plain R local and goes through a box for the call, copied in before and out after (no EPA function keeps such a pointer);&s.fand&a[i]go through a temporary box the same way. - Lists and NULL.
x[[i]] <- NULLdeletes an element of an R list; a store of a pointer into a list is writtenx[i] <- list(value). - Laziness. R evaluates a function's arguments when the function first uses them; C evaluates them before the
call. Every generated function starts by evaluating its parameters in order (
a; b; c), which forces them at once. - Assignment. A path rooted at a local is assigned with
<-, at a global with<<-(assigning<-to a global would make a local copy); a field of a struct reached through an expression is assigned through a temporary ({.e <- Node[[j + 1]]; .e$newDepth <- v}), so only the environment's own binding is touched. Assignments are expressions in R, so C'sif ((f = fopen(...)) == NULL)keeps its shape. - Statements. if / else if / else, while, repeat; do-while is
repeat { ...; if (!(c)) break };foris a while loop with the increment repeated before everynext;switchis an if / else if chain, or a one-passrepeatloop with a fall-through flag when a case falls through;return(x). - Names. R's reserved words and names that would hide a base function the engine calls get a trailing
_(length_,q_,next_); the translator's own names begin with a dot (.e1,.c), which C names cannot. - Constants. Enum constants and numeric macros are written as their values. R's parser does not always round a
decimal literal correctly (
as.numericdiffers from glibc's strtod on 30 of the 199,988 plain decimals of the runtime check), so the translator asks R once how it reads every non-integer literal it writes and would write any that R reads inexactly in hexadecimal; for EPA's literals the answer was none. - Speed without changing what is computed. A global that a function reads twice or in a loop, and that neither it
nor anything it can call assigns as a whole (call graph, a call through a function pointer counting as a call to
every function whose address is taken), is read once into a local at entry; a local is not zero-initialised when
the first statement that mentions it assigns it; the engine is byte-compiled once (
build.sh) and loaded compiled.
A comparison with NaN is false in C (true for !=); in R it is NA, and if (NA) is an error. Several gate models
compute NaN (two print -nan in EPA's own report), so every comparison of floating values must give C's answer.
Written out, if (a < b) becomes if (!is.na(.c <- (a < b)) && .c), and is.na is a call to a builtin: about 0.15
microseconds each, on every comparison of doubles in the engine.
The translator therefore writes two engines from the same C. The safe engine (engine.R) writes every
comparison of floating values, every double used as a condition and every (int) cast in that NaN-proof form. The
fast engine (engine_fast.R) writes them as R's plain comparisons, so a NaN that reaches one is an NA, and R stops
with an error ("missing value where TRUE/FALSE needed"). runswmm.R runs the model on the fast engine; if it stops
with an error, it closes the files the engine opened, removes its temporary files and runs the model again from the
start on the safe engine, whose outputs replace the first run's (EPA opens the report, results and SAVE files for
writing, which truncates them). The fast engine never gives a different answer silently: a NaN in an if, while,
?: or a comparison used as a number is an error, and NA && FALSE (FALSE) and NA || TRUE (TRUE) are what C
gives whatever truth value the NaN had. What R does differently is evaluation order: it evaluates x in both
NA && x and NA || x, where C evaluates it only after a true left operand of && or a false one of ||. A plain
comparison with NaN is false in C, so the left operand of an && whose right operand has side effects is written
NaN-proof in the fast engine too, and so is the left operand of such an || when it holds a !=, a ! or a double
tested as a condition (NaN is true there; EPA's code has none, lessons/45). The gate's two NaN models
(Hydraulics/Allow_Ponding_Low_CE.inp and Hydraulics/swmm5_shapes.inp) run this way.
R zero-fills a vector when it makes it; C's calloc only reserves address space, and nothing is resident until it is
written. rdii.c keeps, for every unit hydrograph, a circular array of past rainfall as long as the hydrograph's time
base over the rain interval; Hydrology/sewer_rdii_problem.inp has 613 hydrographs with a 9,900-hour base and a
60-second interval, 375 million values that EPA almost never touches, and the first R run was killed by the
out-of-memory killer at 6 GB (lessons/46). overrides.R has hand ports of allocRdiiMemory, updateDryPeriod and
getUnitHydConvol in which an empty array stands for an array of zeros: the arrays grow as EPA writes them (always
at the next index), the start of an RDII event empties the past rainfall instead of storing 0.0 in every entry, and
the convolution takes an entry past the end as 0 and adds the same terms in the same order, skipping those that add
nothing. These three functions run only when a node has RDII inflow.
- printf, strtod, sscanf (
rt.R). R'ssprintfhands a floating conversion to the C library'ssnprintf, sort_formatparses each C format and passes every finite%f,%eor%gto it one conversion at a time; R prints NaN and infinities its own way (NaN,Inf), so those, signs and padding included, are written by the runtime as glibc writes them (-nan,inf);%sand%care padded on the bytes; integers go tosprintfas R integers, and values beyond 32 bits are formatted from 16-bit limbs (%luof a negative long is its two's complement). A parsed format is cached only for a string literal (lessons/39).as.numericis R's ownR_strtod, not glibc's strtod, and is not always correctly rounded, sort_scan_doublefinds the extent of a number by glibc's rules (hexadecimal floats,inf,infinity,nan(n-char-sequence)with its payload) and converts it exactly: one correctly rounded multiplication or division of exact doubles when the number has at most 15 significant digits and a small exponent; otherwise an estimate fromas.numeric, corrected by comparing the exact decimal with the midpoints between neighbouring doubles in big-integer arithmetic (base-2^24 limbs; a midpoint is built from its integer parts, since a double cannot hold it). sscanf's%finto a Cfloatrounds once, as glibc's strtof does. - Checked against glibc 2.39 by
r/test/run_crtcheck.sh(the 1,394,252 cases fromtools/crt/test/mkfmt.cthat the other runtimes replay): 0 of 1,389,850 differ (r/results/crtcheck_result.txt),%#gincluded. The 4,402%ldcases whose value is 2^53 or more in magnitude were not run: an R double cannot hold such a long, and the engine never computes one. - Math.
libR.soimports exactly the libm symbols EPA imports, at the same versions (exp@GLIBC_2.29,log@GLIBC_2.29,pow@GLIBC_2.29,sin@GLIBC_2.2.5, ...;nm -D), but R does not always call them as C does:log(x)returns R's own NaN (sign bit clear) for x < 0 and for a NaN, where glibc's has the sign bit set or keeps the argument's;log10of a NaN is R's NaN, not the argument; andx ^ yisR_pow, which squares (x * x) when y is 2, where glibc'spow(x, 2)differs fromx * xin the last bit now and then (12,953 of 20,000,000 arguments, lessons/40), and which decides zero, infinite and NaN arguments by rules of its own ((-Inf) ^ 1.5is NaN in R, +Inf in C). So the engine calls R'sexp,sin,cos,tan,asin,acos,atan,sqrt,floor,ceilingandabsdirectly,rt_logandrt_log10for log and log10, and for pow:x ^ ywhere it is glibc's (finite nonzero x, finite y other than 2; for a constant exponent that is not an integer the translator writes it inline, guarded only against -Inf), the special cases of glibc'se_pow.cwritten out for zero, infinite and NaN arguments, and, forpow(x, 2), R'sqweibull(-|x|, 0.5, 1, FALSE, TRUE), which is1 * pow(-p, 1 / 0.5)computed by glibc's pow inside R's math library: glibc's ownpow(|x|, 2), which ispow(x, 2). No glibc code was translated for this port. - Checked bit for bit, NaN signs and payloads included, by
r/test/run_libm_tests.sh(test/libmcheck.R), each function as the engine calls it: 11,000,672 random arguments over SWMM's ranges plus special values (tools/libm/test/mkvectors.c), 3,300,000 edge cases (mkedge.c), and 8,000,360 cases (test/mkexact.c, from the Hy port's) for sqrt, floor, ceil, fabs, negation,(float), and the safe and fast forms of(int)and(long)(the fast forms must stop on NaN, and did): 0 differ (r/results/libm_tests.txt), except where the argument is a signaling NaN (exp, sin, cos, tan, asin, acos, atan, sqrt, floor, ceil: 663 cases in all), which R hands back unchanged without calling the C library and glibc returns quieted. No SWMM value can be a signaling NaN: arithmetic produces quiet NaNs and the runtime's strtod builds quiet ones.
Each model's last record counts (r/gate/summarize.py):
465 identical, 15 not run (timed out after 1800 s), 0 differ, 0 crashed.
| Run | Engine | Models | Result |
|---|---|---|---|
r_d1 |
de46ab3 | 247 | 247 identical |
r_1a |
de46ab3 | 14 | 13 identical, 1 not run (timed out after 1800 s) |
r_2a |
de46ab3 | 1 | 1 not run (timed out after 1800 s) |
r_2c |
de46ab3 | 19 | 19 identical |
r_2d |
de46ab3 | 8 | 8 identical |
r_2e |
de46ab3 | 4 | 3 identical, 1 not run (timed out after 1800 s) |
r_2f |
de46ab3 | 8 | 7 identical, 1 not run (timed out after 1800 s) |
r_2g |
de46ab3 | 9 | 9 identical |
r_2h |
de46ab3 | 5 | 4 identical, 1 not run (timed out after 1800 s) |
r_2i |
de46ab3 | 7 | 6 identical, 1 not run (timed out after 1800 s) |
r_2j |
de46ab3 | 17 | 17 identical |
r_2k |
de46ab3 | 27 | 27 identical |
r_2l |
de46ab3 | 3 | 2 identical, 1 not run (timed out after 1800 s) |
r_2m |
de46ab3 | 13 | 13 identical |
r_2n |
de46ab3 | 4 | 4 identical |
r_2o |
de46ab3 | 3 | 2 identical, 1 not run (timed out after 1800 s) |
r_2p |
de46ab3 | 17 | 16 identical, 1 not run (timed out after 1800 s) |
r_2q |
de46ab3 | 2 | 2 identical |
r_2r |
de46ab3 | 2 | 1 identical, 1 not run (timed out after 1800 s) |
r_2s |
de46ab3 | 2 | 1 identical, 1 killed by the out-of-memory killer (run again in r_4d) |
r_2t |
de46ab3 | 1 | 1 identical |
r_2u |
b9dbf76 | 12 | 11 identical, 1 not run (timed out after 1800 s) |
r_3a |
b9dbf76 | 14 | 14 identical; replaced by r_4a |
r_2va |
af6b367 | 9 | 9 identical |
r_2vb |
af6b367 | 9 | 9 identical |
r_4a |
af6b367 | 14 | 14 identical |
r_4b |
af6b367 | 3 | 3 identical |
r_4c |
af6b367 | 1 | 1 identical |
r_4d |
af6b367 | 1 | 1 not run (timed out after 1800 s) |
r_2w |
af6b367 | 9 | 9 identical |
r_2x |
af6b367 | 3 | 3 identical |
r_2y |
af6b367 | 5 | 4 identical, 1 not run (timed out after 1800 s) |
r_2z |
af6b367 | 5 | 4 identical, 1 not run (timed out after 1800 s) |
r_2za |
af6b367 | 2 | 2 identical |
r_2bb |
af6b367 | 12 | 12 identical |
r_2ba |
af6b367 | 1 | 1 not run (timed out after 1800 s) |
The engine changed twice during the gate, both times only in rdii.c's three functions (git diff de46ab3 af6b367 -- r/swmm5: the generated engines lose allocRdiiMemory, updateDryPeriod and getUnitHydConvol, and
overrides.R gains them; nothing else changes), which run only when a node has RDII inflow. So every deck with an
[RDII] section or RDII ... YES in EPA's report (23) was run on the final engine, af6b367: the 18 that had passed
before were run again in r_4a to r_4c and were identical both times, and the other 5 were first run after it.
The 15 not run, and how far each got (r/results/stopped_runs.txt; in every one, what R had written when it was
stopped is identical to EPA's output so far):
| Model | .out bytes written of EPA's |
.rpt lines written of EPA's |
|---|---|---|
| Hydraulics/HRSDtest_cim_7.inp | 115,027,968 of 229,761,803 | 144 of 1218 |
| EPA/429_H&H_Elements.inp | 864,256 of 2,908,675 | 1555 of 2850 |
| EPA/380_nodes.inp | 6,033,408 of 7,319,608 | 355 of 3101 |
| Hydrology/17_Sub_90_Link_USA_Units.inp | 1,318,912 of 3,992,879 | 0 of 634 |
| Hydrology/292_Storage_Nodes_200_Subs.inp | 1,323,008 of 2,917,344 | 0 of 2925 |
| Hydrology/short_trap_SW5.inp | 12,288 of 16,518 | 0 of 170 |
| EPA/72_H&H_Elements(2).inp | 0 of 440,275 | 1499 of 1935 |
| EPA/HRSDtest_cim_7.inp | 113,192,960 of 229,761,803 | 144 of 1218 |
| Hydrology/user5_sub_patterns.inp | 1,064,960 of 1,622,749 | 0 of 1392 |
| EPA/attenuation_DW_High_CE.inp | 22,540,288 of 57,546,553 | 132 of 378 |
| Hydrology/sewer_rdii_problem.inp | 3,391,488 of 19,966,686 | 0 of 397 |
| Hydrology/22_Subs.inp | 552,960 of 573,965 | 0 of 443 |
| Hydrology/CN_Horton.inp | 20,021,248 of 43,832,913 | 0 of 1442 |
| Hydrology/27_Subs_Horton.inp | 151,552 of 239,085 | 0 of 458 |
| Hydrology/SCANLON.inp | 11,669,504 of 78,975,850 | 992 of 2976 |
Each needs more than 1800 s in R. The Hy port, about 8 times faster, passed 14 of them in 170 to 1,742 s and timed out on the 15th as well.
Independent rerun (r_v): from a clean git archive of a1e4943 in /dev/shm,
python3 tools/c2x/emit_r.py regenerated both engines byte-identical to the committed ones (cmp; md5 a8e73c32...
and 487dcc4d...), build.sh compiled them, and the gate ran that checkout's runswmm_saves.sh on the 100 models
drawn by random.Random(20261110).sample(sorted(ids that passed), 100) from the 465, plus the three cho.txt decks
(Hydrology/save_rtk_long.inp, one_sub_one_rdii_node.inp and Use_SeasonalRDIIn.inp, none of which the draw picked):
r_va to r_ve, ids in r/gate/ids_v_all.txt. 102 identical, 1 not run (timed out after 1800 s):
EPA/small_wwf_dwf.inp, which took 1,740 s in r_2x; when it was stopped it had written 835,584 of EPA's 857,852
.out bytes and 217 of 717 .rpt lines, identical so far. The three cho.txt decks were identical.
The main session's check (r_main): a clean git clone of commit 29a590a in /dev/shm, where emit_r.py regenerated
both engines (no file changed: git status empty, before and after build.sh compiled them), then the gate ran that
clone's runswmm_saves.sh on 45 models drawn by random.Random(20261109).sample(sorted(ids that passed), 45) from
the 465, plus the cho.txt decks the draw had not picked (it had picked one of the three): 47 models, ids in
r/gate/ids_main_check.txt. 46 identical, 1 not run (timed out after 1800 s): EPA/small_wwf_dwf.inp again,
stopped with 765,952 of EPA's 857,852 .out bytes and 217 of 717 .rpt lines written, identical so far
(r/results/stopped_runs.txt). 5,199 s of running in all. Hydrology/cho.txt was unchanged afterwards (1,114,474
bytes, md5 5b0ca1fe78588202fa779dbfa417b089).
All runs: python3 gate/gate.py NAME r/runswmm_saves.sh --ids IDS -j 1 --timeout 1800 --prune inside flock on the
shared corpus lock, with the output folder in /dev/shm (lessons/22), started by r/gate/run_chunk.sh.
runswmm_saves.sh (the Ruby port's wrapper with the engine changed) backs up every file a deck SAVEs (listed by the
swmm6-rust repository's gate/savefiles.py, called read-only), including the backslash spelling of [LID_USAGE]
report files (lessons/29), restores them after the run, and runs the engine with exec when there is nothing to back
up, so the gate's timeout stops the engine itself. The chunks (r/gate/ids_*.txt, made by r/gate/make_chunks.py)
were cut from the Hy port's gate times times seven, so that each background job, queue included, fits the sandbox's
two-hour limit; 2v and 2b were later split in two (ids_2va.txt ...), the RDII reruns are ids_3a.txt to
ids_3d.txt (run as r_3a and r_4a to r_4d), and the sample is ids_va.txt to ids_ve.txt. Records: r/results/*.json, the logs r/gate/*_run.txt, queue, start and end times in
r/gate/*_times.txt.
On the 465 models that passed, the R port took 37,055 s in all (the gate's per-model times, which include starting R
and loading the compiled engine, about 0.6 s), EPA's C 91.4 s and the Hy port 4,402 s: about 400 times EPA's time
and 8.4 times Hy's; the median model took 6.2 s. Measured on the shared 2-core machine under nice -n 10, one model
at a time, with another agent's gate runs waiting for the same lock, so these are rough. The 480 took 64,056 s of
model time and 19.4 hours from the start to the end of the gate runs (69,760 s, queueing excluded), from 10 November
20:23 to 12 November 02:02 with the waits for the lock.
Where the time goes: every C operation is an interpreted R operation on a boxed double, every struct field is an
environment lookup, and every array element access goes through an environment's v. What was tried and kept is
listed under "Speed without changing what is computed" above; jump tables for switch, removing the argument
forcing and the compiler's optimisation level 3 made no measurable difference and were left out.
- That SWMM is right: identical to EPA means EPA's answers, good or bad.
- The 15 models that need more than 1800 s in R: they were stopped, and only what they had written by then was compared with EPA's output.
- Other platforms: R calls the system libm, so on another machine the port should follow EPA built there; only x86-64 Linux with glibc 2.39 and R 4.3.3 was run. Another R build could route a math function differently (the checks above would show it).
longvalues of 2^53 or more: they are R doubles here, exact only up to 2^53. EPA computes none.- Paths no gate model exercises: streets and inlets (one deck, which stops on input errors), climate files, USE HOTSTART, USE INFLOWS, USE RUNOFF; the stepping API is translated but only the command line was gated.
- A checked run in C's sense: R checks its list and vector bounds on reads, but a C overrun that stays inside an R vector would not show, and a write past the end of an R vector extends it silently.
- No person has reviewed the generated code line by line; the evidence is the gate and the runtime checks.
EPA SWMM is a work of the U.S. Environmental Protection Agency and is in the public domain; its engine is Lewis
Rossman's design, and this port keeps every algorithm as he wrote it. The port contains no glibc code: R calls the
system's libm (glibc, LGPL-2.1-or-later, ref/LGPL-2.1.txt) as a shared library at run time, as EPA's own build does.
R itself is GPL-2 | GPL-3; the port is R code that R runs, not a modification or a part of R, and R's licence does
not reach it. No licence has been chosen yet for the translator and the runtime.
The translator back end (tools/c2x/emit_r.py), the runtime (r/swmm5/rt.R, r/swmm5/overrides.R,
r/runswmm.R, r/swmm5/build.R), the generated engines, the launcher, gate wrapper and chunk runner, the runtime
and math checks, lessons/45 and lessons/46 and this README were written by Claude (Anthropic's AI model, Claude Opus 5.5) in a Claude
Code session from 10 to 12 November 2026, for Robert (Bob) Dickinson, who asked for R in his Code Rosetta Stone app
and set the byte-for-byte standard. Claude ran every gate and check reported here. The checks are mechanical; no
person has yet reviewed the code.