Repository navigation
Source-level LilyPond API and datasets for curated corpora; release 0.4.0 - #12
Merged
Merged
Conversation
lytk.LilyPondVersion compares numerically and accepts what LilyPond 2.24 accepts (x.y.z with a free fourth part, x.y for an even y). lilypond_version(text) reads the first \version statement from the tree; set_lilypond_version and strip_lilypond_version edit the statements. Score/MusicDocument.lilypond_version carry the source's version, and to_lilypond/to_lilypond_music take version=. An invalid \version is the new invalid-version error, as LilyPond's lexer rejects it. The corpus boards leave out LilyPond's tests of its own errors (expect-error = ##t) and report how many of them lytk catches (2 of 13). Co-Authored-By: Claude Opus 5.5 <[email protected]>
lytk.tokenize(text) gives the tokens of LilyPond text from the parse tree (kind, text, character span, line, column): comment, string, scheme, command, symbol, number, fraction, punctuation, and error for text the grammar cannot tokenize. Nothing but whitespace is lost, checked on the fixtures, broken input and the whole corpus board. lytk.strip_comments removes LilyPond comments as LilyPond skips them. Co-Authored-By: Claude Opus 5.5 <[email protected]>
lytk.info(score) is the library home of `lytk info --json` (which now calls it), computed in Rust: parts and notes as before, plus voices, bars, length in quarter notes, lyric syllables, chord symbols and grace notes. lytk.source_stats(text) counts bytes, lines, tokens, comments, Scheme expressions and error tokens. lytk.from_lilypond_music_movements gives every movement of a file as a MusicDocument. The devlog's test counts for K1 and K3 were two too high; corrected. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… request
flatten finds includes on the parse tree (anywhere in a line, never in a
comment or string; a one-line %{ %} no longer hides the rest of the file),
keeps every \language (keeping only the last changed the music before it),
accepts several \header blocks (LilyPond merges them; MultipleHeaders is
gone), and keeps includes of LilyPond's own files (list generated from its
ly/ directory). lytk.flatten_string flattens text.
Every LilyPond reader and check_lilypond take include_paths=: includes are
followed, and each diagnostic is mapped back into the caller's text (one
in an included file at its \include, naming the file). Without it,
includes are still not followed; SECURITY.md says what following them
reads.
Epic K is complete.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
lytk.tokenize descends into embedded Scheme, which the grammar already
parses: `scheme` is now the # or $ that opens an expression, followed by
its symbols, numbers, strings, comments, brackets and quotes, plus the new
boolean, character and keyword kinds; Token.scheme tells Scheme tokens from
LilyPond ones, and LilyPond inside #{ #} is LilyPond again.
strip_comments strips Scheme's comments too. Nothing but whitespace is
lost, on the fixtures and the whole corpus board.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The string twin of from_lilypond_movements, with the same language, strict and include_paths keywords. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ds, API reference) - RecordsDataset.from_jsonl / from_records: records holding music as text, keeping their ids and records; with split_field, split() returns the records' own splits instead of re-shuffling them. - on_error="raise"|"skip"|"warn" (iteration skips and records in dataset.errors; indexing raises), movements="all", and language, include_paths, strict and MIDI quantize passed to the readers. - The cache is keyed by content, reading options, lytk version and normalized converter arguments, written atomically; Subset uses its parent's cache. - ids on every dataset, return_ids on the torch and TensorFlow datasets and data loaders, split(groups=...) for group-aware ratio splits. - docs/python-api.md generated from the stubs by scripts/python_api.py; tests fail when it is stale or a public name has no stub. - lytk.from_abc_tunes_string for ABC records. Epic L ships in 0.4.0 with Epic K, as the owner chose. Co-Authored-By: Claude Opus 5.5 <[email protected]>
set_lilypond_version and the writers' version= check a string version
and write it as given ("2.24" stays "2.24"); a LilyPondVersion is written
as str() gives it. Rust: set_lilypond_version takes the version text.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Epics K and L (source-level LilyPond API and datasets for curated corpora), with Scheme tokens and from_lilypond_movements_string: version 0.4.0 in Cargo.toml (and Cargo.lock); the changelog's [Unreleased] becomes [0.4.0] - 2026-09-28; roadmap and development log updated. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LilyPond source you can inspect and edit, and datasets for curated corpora: Epics K and L, released together as 0.4.0. See
docs/changelog.mdfor the list of changes,docs/devlog.mdfor the notes and the newdocs/python-api.mdfor the API.What changes
\version:lytk.LilyPondVersioncompares numerically and accepts what LilyPond 2.24 accepts;lilypond_version,set_lilypond_versionandstrip_lilypond_versionwork on the parse tree;Score.lilypond_version, andversion=on the LilyPond writers;invalid-versionerror.lytk.tokenizeandstrip_comments. Embedded Scheme is tokenized as Scheme, and LilyPond inside it as LilyPond. Nothing but whitespace is lost, on the whole corpus board.lytk.info(score)(the library home oflytk info --json, computed in Rust) andlytk.source_stats(text).flatten_stringflattens text;check_lilypondfollow includes withinclude_paths=, with diagnostics mapped back into the caller's text;flattenno longer refuses several\headerblocks, no longer drops earlier\languagelines, and keeps LilyPond's own includes.from_lilypond_music_movements,from_lilypond_movements_string,from_abc_tunes_string.RecordsDataset.from_jsonl/from_records(ids, records, the records' own splits);on_error="skip"|"warn"withdataset.errors;movements="all";Subsetuses;ids,return_idson the torch and TensorFlow adapters;docs/python-api.md, generated from the stubs; tests fail when it is stale or a public name has no stub.Checks
🤖 Generated with Claude Code