Repository navigation
Fix tanh′ interval enclosures in saturated tails - #192
Merged
Merged
Conversation
Use the outward-rounded stable 4*q/(1+q)^2 endpoint formula with q=exp(-2*abs(x)). Select extrema by distance to zero, preserve exact zero-domain value 1, clip bounds to [0,1], and handle underflow conservatively. Apply the scalar helper to the interval Jacobian/Hessian path and deep hybrid derivative interval factors, rounding again after tensor conversion. Expose tanh_prime_bounds, document the rounding argument, and fix the optional PyTorch import guard so scalar activation bounds work without PyTorch. Validation: all 333 tests passed, including high-precision references, saturated-tail regressions, subnormal inputs/outputs, tensor dtype conversion, and import without PyTorch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The interval tanh′ routine evaluates
1 - tanh(x)**2, which can round to zero and exclude the real derivative. On [20,21], the old enclosure is approximately [-1.4e-45, 1.4e-45], while the real range is approximately [2.2998e-18, 1.6993e-17].Use the stable
4*q/(1+q)**2formula withq=exp(-2*abs(x)), directed Decimal arithmetic, and extremum selection by distance to zero. The new publictanh_prime_boundspreserves [1,1] at zero, keeps bounds within [0,1], and handles subnormal values and underflow conservatively. The ordinary interval Jacobian/Hessian path retains float32 padding within [0,1]. Deep hybrid derivative interval factors use the same helper, with outward rounding after tensor conversion.Includes the rounding argument and high-precision cosh-based reference tests. Also corrects the optional PyTorch import guard so the scalar activation API can be imported without PyTorch.
Validation: full suite passed (333 tests), including saturated-tail Jacobian regression, extreme numerical cases, float32/float64 deep-factor conversion, and no-PyTorch import. The committed tree exactly matches the tested checkout.