feat: add seed via exception context - #113
Draft
pbrisbin wants to merge 1 commit into
Draft
Conversation
When supported (base-4.20), use `addExceptionContext` to add the seed to the failure. This ensures it happens, not just with expectation failures (`HUnitFailure`), but also with other thrown-exception failures, and does so without altering the exception type (as `annotated-exception` would). In base lower than 4.20, behavior is unchanged. Closes #44. **CAVEAT**: this doesn't work :( HUnit-based testing libraries (e.g hspec) have not updated themselves to use or display exception context in their output. They would need to incorporate it into `formatFailureMessage` for expectation failures. Even if/when they do, this also doesn't even work for its intended purpose (non-expectation-failure exceptions) because `hspec` (for example) uses `show` or (optionally) `displayException`, neither of which include context (which is wild, IMHO). So, while it would be nice if we could use exception context for this information, users would still need to update their test runners somehow to ensure its displayed. I know the `hspec` project is not amenable to displaying more information like this in its output _by default_, but I think they do plan to add some sort of `--verbose` flag. If/when that existed, I think we could get them to include any exception context in such output, and we could instruct our uses to use it if they want to see seed. But is that how we want it to work? Or should we keep this behavior of stuffing it into strings that we know are always displayed (and just give up on #44)? I'm leaning this way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When supported (base-4.20), use
addExceptionContextto add the seed tothe failure. This ensures it happens, not just with expectation failures
(
HUnitFailure), but also with other thrown-exception failures, anddoes so without altering the exception type (as
annotated-exceptionwould).
In base lower than 4.20, behavior is unchanged.
Closes #44.
CAVEAT: this doesn't work :(
HUnit-based testing libraries (e.g hspec) have not updated themselves to
use or display exception context in their output. They would need to
incorporate it into
formatFailureMessagefor expectation failures.Even if/when they do, this also doesn't even work for its intended
purpose (non-expectation-failure exceptions) because
hspec(forexample) uses
showor (optionally)displayException, neither ofwhich include context (which is wild, IMHO).
So, while it would be nice if we could use exception context for this
information, users would still need to update their test runners somehow
to ensure its displayed.
I know the
hspecproject is not amenable to displaying moreinformation like this in its output by default, but I think they do
plan to add some sort of
--verboseflag. If/when that existed, Ithink we could get them to include any exception context in such output,
and we could instruct our uses to use it if they want to see seed.
But is that how we want it to work? Or should we keep this behavior of
stuffing it into strings that we know are always displayed (and just
give up on #44)? I'm leaning this way.