[transfix]

Why

LbieMesherTest.HexaVerticesStayInsideAVolumeTheSurfaceLeaves, #677's own regression test, hit an access violation (0xC0000005) in the recipe / windows-2022 / Release lane of #758 (run 39637628700, job 119352377930). It crashed in the r = 1.5 sphere, after "finished collapse" and before "finished compute qef". That is earlier than the hexa vertex walk #677 fixed. It is intermittent: the same lane passed on #758's previous head and in ~40 other recent runs.

Cause. Octree::interpRect3Dpts_x/y/z interpolate an edge crossing's normal between the gradients at two grid points: the one the crossing falls in and the next one. When the edge ends on the volume's far face exactly at the isovalue (ival == 1), the second point is one plane off the grid (index 17 of 17), and getVertGrad → getValue read past orig_vol. For a z edge that is up to ~1.1 KB past the end, far enough to reach an unmapped page now and then on Windows.

Why only the second sphere: sampled at 17³ in [-1, 1]³, the r = 1.5 sphere passes exactly through 24 grid points on the box's faces (8² + 4² + 8² = 12² in eighths), where r - d is exactly 0. The r = 1.2 sphere passes through none.

AddressSanitizer (Linux, GCC 13, RelWithDebInfo) flags it on every run:

heap-buffer-overflow READ of size 4, 12 bytes after the 19652-byte orig_vol (Octree::initMem)
  LBIE::Octree::getValue           octree.cpp:3598
  LBIE::Octree::getVertGrad        octree.cpp:3661
  LBIE::Octree::interpRect3Dpts_z  octree.cpp:3501
  LBIE::Octree::cell_comp          octree.cpp:606
  LBIE::Octree::compute_qef        octree.cpp:760
  LBIE::Mesher::do_duallib_extract LBIE_Mesher.cpp:84

What

getVertGrad() clamps its grid point onto the volume. In the case above the off-grid sample's interpolation weight is exactly 0, so meshes are unchanged:

• HEXA, r = 1.2: 344 vertices / 4374 quads;
• HEXA, r = 1.5: 64 vertices / 162 quads.

These are the counts #677 reported. Only the out-of-bounds read goes away.

Tests

• New LbieMesherTest.MeshesASurfaceThroughGridPointsOnTheFarFaces meshes the r = 1.5 sphere as TETRA, HEXA, DOUBLE and TETRA2. It checks the geoframe and that every normal is finite.
• Without the fix, under ASan:
  • HexaVerticesStayInsideAVolumeTheSurfaceLeaves is flagged in 40 of 40 runs (OMP_NUM_THREADS = 1, 2, 8 and the default, 10 runs each);
  • on the r = 1.5 sphere every mesh type is flagged: SINGLE, TETRA, QUAD and HEXA in compute_qef, DOUBLE and TETRA2 in compute_qef_interval.
• With the fix, under ASan:
  • both tests pass in 40 of 40 runs, with no reports;
  • lbie_mesher_test 37/37, clean apart from the pre-existing HexaDoMeshOptimize read listed below;
  • algorithm_test 10/10 and geometry_test 111/111, clean.
• Not run: Windows /fsanitize=address. This PR's Windows lanes are the first Windows run.

Found, not fixed (pre-existing; not this crash)

• QUAD: Octree::assign_refine_sign_quad (from polygonize_quad) reads index -1 of an octcell-indexed array wherever the surface leaves the volume. Both the r = 1.2 and r = 1.5 spheres reach it. It is the same family as LBIE: keep the hexa vertex walk on the grid (intermittent hexahedralize crash) #677. CI's QUAD tests use r = 0.5, so they never reach it, and that is why the new test leaves QUAD out.
• Octree::getGrad reads v1[3], v2[3] and v3[3] of float[3] arrays. ASan reports it as a stack-buffer-overflow in HexaDoMeshOptimize via Octree::optimization. A stack overread can't fault, but it feeds garbage into OPTIMIZATION's gradient, which may explain the arm64 Release NaNs that test's comment describes.
• SINGLE extracts no surface from the r = 1.5 sphere. This is not a memory error.