Skip to content

Keep integer true division, hypot() and nextafter() away from the bundled miniexpr until it is fixed #756

Open

Description

@FabianIsensee

To track the follow-up from #754 (#754 (comment)). The bundled miniexpr computes true division, hypot() and nextafter() of integer or bool operands in integer arithmetic (Blosc/miniexpr#26, Blosc/miniexpr#27). #754 gives these expressions NumPy's result dtype, but wherever miniexpr evaluates them the values are still wrong. With #741 and #754:

a = blosc2.asarray(np.array([1, 2, 3, 4]))
b = blosc2.asarray(np.array([2, 4, 4, 4]))
(a / b)[:]                   # [0. 0. 0. 1.]             NumPy: [0.5 0.5 0.75 1.]
(a / b).sum()                # 1.0                       NumPy: 2.75
(a / b + 0.25)[:]            # [0.25 0.25 0.25 1.25]     NumPy: [0.75 0.75 1. 1.25]
blosc2.hypot(a, b)[:]        # [2. 4. 5. 5.]             NumPy: [2.236 4.472 5. 5.657]
(blosc2.hypot(a, b) > 2)[:]  # [False True True True]    NumPy: all True
(a / b).compute()[:]         # right
(a / b)[:2]                  # right

Natively, expr[:] and reductions are wrong. Under WebAssembly, compute() and slices are wrong too; there test_lazyexpr_result_dtype_matches_numpy marks 30 such cases xfail(strict=True) (_WASM_KNOWN_FAILURES). (arctan2() is already handled in python-blosc2.)

Until a fixed miniexpr is bundled, python-blosc2 should send these expressions to an evaluator that gets them right. From the review of #754:

  • The guard covers expr[:], compute(), partial slices and reductions, natively and under WebAssembly.
  • It reaches an evaluator that is right for these expressions, such as NumPy per chunk. Disabling the prefilter alone may not be enough, if chunk evaluation then picks another miniexpr path.
  • It does not convert all inputs to float, which would break integer overflow, comparisons and integers above 2**53.
  • It finds the affected operations anywhere in the expression, also below other operators (a / b + 0.25, hypot(a, b) > 2).

When the fixed miniexpr is bundled, the guard goes, and so do the WebAssembly xfails, which then XPASS.

#744 (closed) had a guard for these and other miniexpr bugs (src/blosc2/miniexpr_guard.py), which sent them to numexpr. #741 removes numexpr, so a new guard would send them to NumPy.

AI disclosure: the analysis and the examples were done with Claude Code; I reviewed them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions