Add a reflection back end - #278
Draft
AaronWebster wants to merge 4 commits into
Draft
Conversation
`--generate` was declared with `choices=["cc"]` but `main()` never read it: the C++ back end was imported and run unconditionally. Restructure the driver around `_generate_code_and_log_errors()`, `_default_file_suffix()` and `_format_code()` so that adding a back end is a matter of extending three `match` statements. Add `--generate ir`, which writes the compiler's own serialized IR as JSON. That is the format back ends already consume -- the front end writes it via `--output-file`, the `emboss_library` Bazel rule produces `<src>.emb.ir` with it, and `emboss_codegen_cpp` reads it -- but `embossc` had no spelling for it, so anything wanting IR had to import `compiler.front_end.glue` directly. The bytes are identical to the front end's own output. Also drop `nargs=1` from `--generate`, `--output-path` and `--output-file`. All three were indexed as lists while defaulting to bare strings; the `--output-path` default survived only because `"."[0] == "."`. Verified with scripts/regenerate_goldens.py: all 34 C++ goldens regenerate byte-identically.
Renders an ir_data.Expression back to Emboss source text, the inverse of
expression_parser.parse. The only expression renderer in the tree today is
header_generator's, which emits C++ helper names (Sum, LessThanOrEqual)
rather than infix, and format_emb.py works on the parse tree rather than on
ir_data.Expression. A back end that reports expressions to a human -- a
`requires` clause in documentation, a non-constant field offset in reflection
output -- needs the Emboss spelling.
Parentheses are emitted only where the grammar needs them, which is a little
subtler than a precedence table:
* `&&` and `||` are grammar siblings, not tiers, so mixing them without
parentheses does not parse. They get equal precedence plus a rule that
parenthesizes a boolean child of a different boolean parent.
* `?:` takes logical-expressions, so a nested `?:` is always parenthesized.
* Comparisons do not re-associate, so a comparison inside a comparison is
always parenthesized.
* Unary minus parses to a binary SUBTRACTION against a synthesized
zero-width `0`; that form is detected and printed back as `-x`.
`compiler/back_end/reflection` emits a JSON description of a module's types
instead of code: every field with its bit offset and bit size, every enum
member with its value, every `let` value, plus documentation, `requires`
clauses, existence conditions, byte order, and runtime parameters.
The point is to let a program consume an `.emb` as data. The case that
motivated it: a host tool that has to speak two revisions of one FPGA register
interface at once cannot `#include` both generated headers, because the two
declare the same fully-qualified names. Reading the registers as data sidesteps
the collision entirely, and the same output serves documentation generators and
register-map catalogs.
Offsets are reported relative to the type a field is declared in. Anonymous
`bits` blocks are the exception -- they have no name for an offset to be
relative to -- so their fields are lifted into the enclosing type at absolute
bit offsets, which is where the compiler's own alias virtuals already point.
A field whose placement is not a compile-time constant reports a null offset or
size plus the Emboss expression that determines it, and, where the front end
knows one, the modulus it is aligned to.
Notes on a few decisions:
* Mixed byte order within one structure is legal Emboss, so a structure whose
fields disagree reports a null `byte_order` rather than failing. Every
field reports its own order regardless, so nothing is lost.
* `is_signed` comes from the enum's `is_signed` attribute for enum-typed
fields and from the expression's computed bounds for `let` fields, rather
than from a type-name test, which would miss both.
* The compiler's own virtuals -- the `$`-prefixed size fields and the alias
fields it synthesizes for anonymous `bits` members -- are not reported.
No new attributes, so no `.emb` needs an `expected_back_ends` update and the
front end is untouched.
Also adds `embossc --generate reflection` (output suffix `.json`), the
`emboss_reflection_library` Bazel macro, goldens under
`testdata/golden_reflection/`, and their entries in
`scripts/regenerate_goldens.py`.
A dynamically-sized structure has no one size, so `bit_size` is null for it -- which left a consumer with nothing at all. The compiler already knows the range: it computes bounds on the `$size_in_bytes` virtual whether or not the value folds to a constant. Report them as `min_bit_size` / `max_bit_size`, and null out a bound the compiler could not establish. Found by diffing this back end against the walker it was seeded from, over a real set of register maps: the seed reported a size for six structures where this one reported none. In one case the size was genuinely fixed and the bounds now pin it; in the other five the seed was quietly reporting the offset of the last fixed-position field, having dropped the variable-length tail.
Collaborator
|
Potentially bikeshed-y, but could we consider a different name for the backend, like |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a reflection back end:
embossc --generate reflection foo.embwrites a JSONdescription of the module's types instead of C++ code — every field with its bit
offset and bit size, every enum member with its value, every
letvalue, plusdocumentation,
requiresclauses, existence conditions, byte order, and runtimeparameters.
The point is to let a program consume an
.embas data. The case that motivatedit: a host tool that has to speak two revisions of one FPGA register interface at
once cannot
#includeboth generated headers, because the two declare the samefully-qualified names. Reading the registers as data sidesteps the collision
entirely, and the same output serves documentation generators and register-map
catalogs.
No new attributes, so no
.embneeds anexpected_back_endsupdate and thefront end is untouched.
Commits
embosschonor--generate, and add anirback end.--generatewas declared with
choices=["cc"]butmain()never read it — the C++ backend was imported and run unconditionally. Restructured around
_generate_code_and_log_errors()/_default_file_suffix()/_format_code()so adding a back end is a matter of extending three
matchstatements, anddropped the
nargs=1that made--generate/--output-path/--output-filelists while their defaults were bare strings.
--generate irwrites thecompiler's own serialized IR, byte-identical to the front end's
--output-file.compiler/util/expression_printer. Renders anir_data.Expressionback to Emboss source text, the inverse of
expression_parser.parse. The onlyrenderer in the tree today is
header_generator's, which emits C++ helpernames (
Sum,LessThanOrEqual) rather than infix.min_bit_size/max_bit_sizerather than nothing at all.Notes on a few decisions
bitsblocks are the exception — they have no name for an offset to berelative to — so their fields are lifted into the enclosing type at absolute
bit offsets, which is where the compiler's own alias virtuals already point.
or size plus the Emboss expression that determines it, and, where the front end
knows one, the modulus it is aligned to.
fields disagree reports a null
byte_orderrather than failing. Every fieldreports its own order regardless, so nothing is lost.
is_signedcomes from the enum'sis_signedattribute for enum-typed fieldsand from the expression's computed bounds for
letfields, rather than from atype-name test, which would miss both.
$-prefixed size fields and the alias fieldssynthesized for anonymous
bitsmembers — are not reported.Testing
//compiler/util:expression_printer_test(round-trips throughexpression_parser.parse),//compiler/back_end/reflection:reflection_generator_test,//:embossc_test.testdata/golden_reflection/, with aREFLECTION_GOLDENSlist in
scripts/regenerate_goldens.py.scripts/regenerate_goldens.pyreproduces all 34 existing C++ goldensbyte-identically, which is what establishes commit 1 as behavior-preserving.
against the ad-hoc IR walker it was seeded from: 398 types / 991 fields agree
exactly on
bit_offset,bit_size,enum_ref,is_arrayand documentation;every remaining difference is one of the intended fixes (named sub-structs no
longer inlined away, synthetic aliases de-duplicated, dynamic placement
reported rather than dropped).
Draft: opening it for CI. Happy to split commit 1 out into its own PR if that
reads better.