ParticleList carries a species table, a charge and a mass

[michakraus]

ParticleList carries a species table, a charge and a mass

First-class charge, mass and species for the particle lists: a Species type, a species table and a
per-particle index on ParticleList, accessors that infer and allocate nothing, a lazy
charge-weighted weight matrix, and the HDF5 round trip of the new data.

What this does

Species{T} (new src/species.jl)

• fields name::Symbol, charge::T, mass::T; Species(name, charge, mass) promotes the two
  numbers to their common type.
• a mass that is not positive, NaN included, throws ArgumentError; a zero charge is allowed.
  The check lives in the inner constructor, so Species{T}(s::Species) and the conversion of a
  Float64 table into a Float32 list check again, and a mass that rounds to zero in T throws.
• accessors charge(s::Species) and mass(s::Species), both exported.

ParticleList (src/particle_list.jl)

• two fields: species::Vector{Species{T}} with T = eltype(list), and speciesindex::Vector{Int}
  of length np. One matrix, one list: the splitting method copies particles.z as one state and
  ParticleDistribution wraps one ParticleList, so a tuple of lists, one per species, would break
  both.
• the species and speciesindex keywords reach the (DT, np, nd) and (x, v, w) forms through
  kwargs, so no caller's shape changes and VlasovMethods needs no edit. The keyword constructor
  copies speciesindex, so a later change to the caller's vector does not reach the list.
• species(pl, i), charge(pl, i) and mass(pl, i): @inline, inferred, and no allocation
  after a warm call. They take any Integer index. Base.:(==) compares the two new fields as
  well.
• the inner constructor throws ArgumentError when length(speciesindex) ≠ np or when an index
  lies outside 1:length(species), so the positional form checks too. The keyword form also throws
  for more than one species without a speciesindex.
• default: a list built without a species holds one Species(:default, one(T), one(T)) with
  speciesindex = ones(Int, np), so q · w == w bit for bit and every existing result is
  unchanged. The index costs 8 bytes per particle at construction.
• species and speciesindex are now field names, so a variable or parameter with either name
  fails the constructor's AssertionError. The CHANGELOG lists this under Breaking Changes.

chargeweights(pl) (exported; ChargeWeights is not)

• a lazy ChargeWeights{T} <: AbstractMatrix{T} of size 1 × np whose [1, i] element is
  charge(pl, i) * w_i. That is the shape and the value a merged _deposit!(potential, vals, x, w)
  reads as w[1, i], so a later switch of a deposit to q · w changes one argument at each call
  site, needs no buffer and leaves the signature alone. It reads pl.list[r, i] at the one row of
  w, so a scalar w row and a one-row range agree.
• ArgumentError when the list has no variable w, or when w spans more than one row.
• No code calls it yet. See Known issues.

HDF5

• h5save writes a group species under path, beside particles, with name
  (Vector{String}), charge and mass (in T) and index (Vector{Int}); the reader reads it
  from the same group.
• a file without the group, as 0.1.2 writes, loads with the default species.
• the HDF5 floor rises to 0.16.16 (HDF5 = "0.16.16, 0.17"). The species names are the first
  variable-length strings the package reads, and HDF5.jl 0.16.0 to 0.16.15 cannot read one on
  Julia 1.10: their loader calls Base._memcpy!, which Julia 1.10 does not define. The Downgrade
  job fails on 0.16.0 and passes on 0.16.16.

Scope

This PR does not edit VlasovMethods. The switch of a deposit to a species weight belongs to the
later change that first deposits with a species charge.

Tests

• test/species.jl (new): the non-positive and NaN mass, the promotion of charge and mass,
  the conversion to another element type and its check.
• test/particle_list.jl: species table and accessors (Float32) and (Float64), chargeweights,
  HDF5 round trip of the species table (Float32) and (Float64), a file without a species group loads with the default, the species group follows path. They cover an Int32 index, an index
  below and above the range, a copied index, a mass that rounds to zero in Float32, the scalar and
  the range form of w, a two-row w, and the element type of the datasets on disk.

Known issues added

• K1 · ChargeWeights and chargeweights are exported API with no caller. Remove the pair
  before the next release, or keep it until the first deposit that uses a species charge calls it.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.69%. Comparing base (54899f7) to head (cce0d1b).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #20      +/-   ##
==========================================
+ Coverage   85.71%   94.69%   +8.98%     
==========================================
  Files           3        4       +1     
  Lines          98      132      +34     
==========================================
+ Hits           84      125      +41     
+ Misses         14        7       -7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[michakraus]

Review of add4dad (head of species/part-42)

Verdict: request changes (posted as a comment). One bug: this change makes the HDF5 [compat] floor false, and the Downgrade job shows it. Three correctness risks. And the code can be cut substantially without loss.

CI

• Required checks (classic protection and the ruleset agree: Julia min and Julia 1 on ubuntu, macOS and windows, and Doctests): all 7 green.
• Downgrade - ubuntu-latest (advisory, continue-on-error): red, and caused by this PR. Julia 1.10.12 with HDF5 v0.16.0 and HDF5_jll v1.10.5+7. Particle Lists errors with UndefVarError: _memcpy! not defined in HDF5._typed_load(::Type{Ptr{UInt8}}, ...) (HDF5 0.16.0 src/typeconversions.jl:315). The call comes from read(sgroup["name"]) at src/particle_list.jl:258, through the old round trip at test/particle_list.jl:91. The last Downgrade run on main (run 37980598372, the same Julia and HDF5 v0.16.0) is green, with Particle Lists at 44/44. The new species/name dataset is the first variable-length string that the suite reads. HDF5.jl calls Base._memcpy! without a version guard up to v0.16.15, and guards it from v0.16.16 on (@static if VERSION ≤ v"1.10.0-DEV.1390").

Findings

#	severity	file:line	claim	evidence
1	bug	Project.toml:10, src/particle_list.jl:258	HDF5 = "0.16, 0.17" is now a false floor. On Julia 1.10 with HDF5 0.16.0 to 0.16.15, no file that h5save writes can be loaded, because the read of species/name reaches Base._memcpy!, which Julia 1.10 does not define. Fix: HDF5 = "0.16.16, 0.17", then re-run Downgrade.	The Downgrade log above. main is green on the same versions. HDF5.jl v0.16.15 src/typeconversions.jl calls Base._memcpy! unguarded; v0.16.16 guards it.
2	correctness-risk	src/particle_list.jl:82	convert(Vector{Int}, speciesindex) returns the vector of the caller itself, so the list aliases it, and any later write to it bypasses the range check.	With idx = [1, 2, 1, 2, 2] passed as speciesindex: pl.speciesindex === idx is true. After idx[1] = 7, charge(pl, 1) throws BoundsError: attempt to access 2-element Vector{Species{Float64}} at index [7]. Fix: collect(Int, speciesindex).
3	correctness-risk	src/species.jl:28	m ≤ zero(T) passes a NaN mass, though the docstring says that the mass must be positive.	Species(:n, 1.0, NaN) returns Species{Float64}(:n, 1.0, NaN). Fix: throw unless the mass compares greater than zero, a test that is false for NaN.
4	correctness-risk	src/particle_list.jl:51-59, 71-90	All species validation is in the keyword constructor. The 8-argument inner constructor stores any table and any index without a check. Species keeps its invariant in its inner constructor; ParticleList should do the same, so that no path builds a list whose charge(pl, i) throws later.	Read: the inner constructor checks only the three name-disjointness assertions.
5	quality	src/particle_list.jl:198-238, src/ParticleMethods.jl:29, test/particle_list.jl:195-240	ChargeWeights, chargeweights and _wrow (41 lines of source, one exported name, 45 lines of tests) are exported API with no caller. The PR body puts the deposit that would use it in a later change. The lazy type is not faster than the product written inline. Remove it, and add it, if it is still wanted, together with the deposit that reads it. The hand-written deposit loop in the test (231-239) tests its own loop, and goes too.	One process, warm, single thread, Julia 1.13.1, median of 51 calls, machine load about 9: a loop over cw[1, i] against a loop over charge(pl, i) * pl.w[i]: 0.667 against 0.667 µs at np = 10³, 68.96 against 67.67 µs at 10⁵, 695 against 692 µs at 10⁶.
6	quality	src/particle_list.jl:71-90, 256-268	The species block of the keyword constructor (two nothing sentinels, a nested if, a hand loop, the locals stable and sind) and the (nothing, nothing) branch of the reader reduce to keyword defaults and one check in the inner constructor. The sketch below also closes #2 and #4.	The sketch below, run as a free function, gives the same result as this PR for the default, a missing index with two species, an Int32 index, an index of 0, an empty table with np = 5 and with np = 0, and does not alias the index.
7	quality	CHANGELOG.md:25-32	The entry leaves out the breaking part: species and speciesindex are now reserved, so a list with a variable or a parameter of either name now throws.	ParticleList(z; variables = (x = 1, v = 2, w = 3), parameters = (species = 1,)) throws AssertionError: isdisjoint(fieldnames(ParticleList), keys(params)) on this head, and builds on main. The same holds for a variable named speciesindex.
8	quality	test/particle_list.jl:243-335	The three HDF5 testsets compare charge, mass, index and table element by element, but this PR extends == to the table and the index. Each round trip is @test ParticleList(f, path) == pl, plus the four element-type checks at 263-268. The charge.(Ref(pl2), 1:np) and mass.(...) lines repeat what pl2.species == pl.species and pl2.speciesindex == pl.speciesindex already imply.	ParticleList(f) == pl, ParticleList(f, "a") == plA and ParticleList(f, "b") == plB are all true, in Float32 and Float64.
9	nit	test/particle_list.jl:110, 143	The test that allocations(_allocating, pl, 1) is positive tests that zeros(1) allocates. That is Base, not this package.	Read.
10	nit	src/species.jl:61, src/particle_list.jl:189, 196	species(s::Species) = s is an identity method with no caller. charge(pl, i) and mass(pl, i) read the fields .charge and .mass instead of the accessors charge and mass that this PR defines.	Read.
11	nit	src/particle_list.jl:64, 73, 256, 258	Local names shadow functions: the keyword species and the reader local species shadow the exported species, and names shadows Base.names. stable reads as an adjective.	Read.
12	nit	src/particle_list.jl:261-262, 293-295	The reader is Species.(Symbol.(read(s["name"])), read(s["charge"]), read(s["mass"])), and its convert(Vector{Int}, ...) repeats the conversion of the constructor. The writer can use charge.(p.species) and mass.(p.species).	Read.
13	nit	CHANGELOG.md:59	The eachslice bullet now gives that change 192.0 MB and 41.5 %. That figure contains the 8 MB index of this PR. The eachslice change brought it to 184.0 MB, and the new species bullet already states its own 8 bytes per particle.	Warm @allocated ParticleList(z; variables) at np = 10⁶, nd = 6, three variables: 184 010 768 B on main, 192 022 688 B on this head.
14	nit	CHANGELOG.md:52 (adjacent, not from this PR)	The bullet cites test/particle_list_tests.jl:81, which does not exist; temp.h5 is written at test/particle_list.jl:85.	Read.
15	nit	PR body	The body is written for an internal process, not for a reader on GitHub: part numbers (P42, P13, §5V.P42), sections for "the dispatcher" and "advisor decisions", and local script names (mutate.jl, round-1a/probe_blocking.jl). Its "Pre-PR verification" section opens with "Verdict: stop and ask — 2 unresolved verification gates". It names HDF5 0.16.0 as the floor that was checked for delete_object, but no run at that floor read the new string dataset (#1). Its "cold" allocation figures are warm figures: a cold first call measures 279.8 MB on main and 290.5 MB here, compilation included. The delta it gives is correct.	The PR body; the measurements under "Verified".

The simplification for #2, #4 and #6

# inner constructor: the one place that every path goes through
function ParticleList(list, views, params, particles, variables, indices, species, speciesindex)
    @assert isdisjoint(fieldnames(ParticleList), keys(views))
    @assert isdisjoint(fieldnames(ParticleList), keys(params))
    @assert isdisjoint(keys(views), keys(params))
    sind = collect(Int, speciesindex)
    length(sind) == size(list, 2) && all(in(eachindex(species)), sind) ||
        throw(ArgumentError("speciesindex needs one entry per particle, each an index into species"))
    new{...}(list, views, params, particles, variables, indices,
        Species{eltype(list)}.(species), sind)
end

function ParticleList(list::AbstractMatrix; variables = NamedTuple(), parameters = NamedTuple(),
        species = [Species(:default, one(eltype(list)), one(eltype(list)))],
        speciesindex = length(species) ≥ 2 ?
                       throw(ArgumentError("more than one species needs a speciesindex")) :
                       ones(Int, size(list, 2)))
    # ... views, vars, particles as before ...
    ParticleList(list, views, parameters, particles, vars, svariables, species, speciesindex)
end

# reader
kw = if haskey(group, "species")
    s = group["species"]
    (species = Species.(Symbol.(read(s["name"])), read(s["charge"]), read(s["mass"])),
        speciesindex = read(s["index"]))
else
    (;)
end
ParticleList(particles; variables, parameters = params, kw...)

This removes the nothing sentinels, the nested if, the hand loop and the (nothing, nothing) branch. It copies the index and checks it in one place.

Verified

• Downgrade attribution: the job log on this head, against the Downgrade job of run 37980598372 on main (Julia 1.10.12, HDF5 v0.16.0, green). HDF5.jl v0.16.0 and v0.16.15 call Base._memcpy! unguarded; v0.16.16 and v0.17.0 guard it.
• Required checks: branches/main/protection and rules/branches/main list the same 7 contexts, and all 7 pass on this head. Downgrade is not required.
• Construction allocation: the rise on this head is exactly the index and the table. Warm, at np = 10, 10³, 10⁵ and 10⁶, head minus main equals @allocated ones(Int, np) plus 80 B (224 B = 144 + 80 at np = 10; 8 011 920 B = 8 011 840 + 80 at 10⁶).
• ParticleList(f, path) == pl after h5save, at the root and under two nested paths, in Float32 and Float64.
• JuliaFormatter (sciml) reports all six changed .jl files as formatted.
• No closing issue references.
• The local checkouts of VlasovMethods and ReducedBasisMethods use no variable or parameter named species or speciesindex.

Not checked

• A local run of HDF5 0.16.16 on Julia 1.10: the probe environment could not download artifacts. The bump in #1 needs the Downgrade job to confirm it.
• The full test suite was not run locally; CI runs it.
• A name clash of the new exports charge, mass and species with other packages that a user loads together with this one.

[michakraus]

Review findings: resolution

This resolves every finding of the review on add4dad. Fixes are in 74953f0 (code and tests) and 2d85f15 (CHANGELOG). All 13 checks pass on 2d85f15, Downgrade included.

#	severity	finding	resolution
1	bug	HDF5 floor 0.16 false: 0.16.0 to 0.16.15 cannot read the new species/name string dataset on Julia 1.10	Fixed in 74953f0: HDF5 = "0.16.16, 0.17". HDF5 0.16.15 still calls Base._memcpy! unguarded; 0.16.16 guards it. The Downgrade job now resolves HDF5 v0.16.16 and passes (Particle Lists 145/145). Before the fix, the same job failed at src/particle_list.jl:258, and main's Downgrade job passed, so the failure came from this PR.
2	correctness-risk	convert(Vector{Int}, speciesindex) aliases the caller's vector	Fixed in 74953f0: collect(Int, speciesindex). New test: a write to the caller's vector after construction leaves pl.speciesindex unchanged.
3	correctness-risk	m ≤ zero(T) accepts a NaN mass	Fixed in 74953f0: m > zero(T) || throw(...); docstring and test (Species(:n, 1.0, NaN) throws) added.
4	correctness-risk	species checks only in the keyword constructor	Fixed in 74953f0: the length and range checks are in the inner constructor (all(in(eachindex(species)), speciesindex)). The positional constructor now rejects a wrong-length and an out-of-range index.
5	quality	ChargeWeights/chargeweights has no caller; remove it	Not fixed: needs a decision. The PR body records an author decision that chargeweights lands in this PR. Removing it is an API decision for the author, not for a fix round. Within the type, _wrow (two methods, 7 lines) is replaced by only(pl.indices.w), which throws the same ArgumentError for zero or several rows.
6	quality	constructor and reader can collapse	Fixed in 74953f0: the default species is a keyword default, the hand loop and the stable/sind locals are gone, and the reader passes the species keywords only when the group exists (no (nothing, nothing) branch). src/particle_list.jl loses 18 lines net (+33/−51), docstring included. Construction at np = 10⁶ allocates 80 B more than add4dad; the index is not copied twice.
7	quality	CHANGELOG omits that species and speciesindex become reserved names	Fixed in 2d85f15: entry under Breaking Changes.
8	quality	HDF5 tests compare element by element	Fixed in 74953f0: each round trip is one @test ParticleList(f, path) == pl; the stripped-group cases compare with a default-species list.
9	nit	allocations(_allocating, …) > 0 tests Base's zeros	Disputed. It is the control for the == 0 assertions: it shows that the allocations harness can report a non-zero figure, so a zero is not vacuous. Kept.
10	nit	species(::Species) identity method has no caller; accessors bypass charge/mass	Fixed in 74953f0: the method and its test are removed, and charge(pl, i)/mass(pl, i) call charge/mass of the species.
11	nit	local names shadow functions (species, names), stable misleading	Fixed in part in 74953f0: stable, sind and names are gone with #6 and #12. The keyword species still shadows the function inside the constructor, because the keyword name is public API.
12	nit	reader and writer verbose	Fixed in 74953f0: Species.(…) in the reader, no redundant convert, charge.(p.species) and mass.(p.species) in the writer.
13	nit	eachslice bullet now reports 192.0 MB / 41.5 %, which includes this PR's 8 MB	Fixed in 2d85f15: the bullet keeps its own 184.0 MB / 43.9 % and names the 8 bytes per particle of the species index separately.
14	nit	CHANGELOG cites test/particle_list_tests.jl:81 (pre-existing)	Not fixed: pre-existing, on a line this PR does not touch.
15	nit	PR body is internal-process text; claims the HDF5 floor was checked; "cold" figures are warm	Not fixed: the PR body is the author's. Note that its statement "A floor raise: not reached" is now out of date because of #1.