Modify sig fig feedback priority order - #818
Conversation
If the user's answer is correct but has too many significant figures, specific feedback about this is more useful than the default feedback. The "too few s.f., regardless of correctness" feedback should still be lower priority than the default feedback, since we have no way of knowing whether the answer was rounded too much or is just wrong.
This isn't an issue for normal use since tags cannot be null in a real question, but it breaks some tests where tags aren't manually initialised.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #818 +/- ##
==========================================
+ Coverage 40.97% 41.01% +0.03%
==========================================
Files 550 550
Lines 23555 23583 +28
Branches 2837 2845 +8
==========================================
+ Hits 9652 9672 +20
- Misses 13032 13039 +7
- Partials 871 872 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
If we have exactly matched a known incorrect choice that also has the wrong number of sig figs, the custom feedback for that choice should take precedence over generic sig figs feedback.
This test needs to check that "too many sig figs" takes precedence over the correct explanation.
|
FWIW, even if this gets approved before next week's release, I'd advocate not merging this until the release after; I don't think that modifying this behaviour mid-STEM SMART residential when the content teams won't be available to test is sensible. |
There was a problem hiding this comment.
Thanks for the changes! I'm pretty happy with this PR now, and feel reasonably confident this will only create the desired change, which is to raise the priority of "too many s.f." messages above the priority of default feedback.
I find I can now quite easily explain to myself how this is achieved: you sank the early return (which returns the default feedback) so it only happens after the too "too many s.f." check. (A byproduct is that the "too many s.f." now runs before the "too few s.f." check, but this won't matter as we can't have both at the same time).
I still think the validator overall is a little too complex. I think validateQuestionResponse is long enough it should only return once, and thevalidateWith family of private methods should consistently mark answers with "tooMany" as well as "tooFew" s.f. problems. Mostly though, I'm just happy we found a small, unobtrusive code change that achieves the new behaviour, and I'm very keen not to touch the numeric validator.
Thanks for the test changes too, and for extracting the constants.
The feedback priority order should now be:
This makes numeric questions consistent with coordinate questions, where this was already the feedback order.
Note that we now always do sig fig checking (unless
diregardSignificantFigurersis set) and overwrite thesig_figs_too_fewfeedback with the default feedback if necessary, instead of short-circuiting with the default feedback before the sig figs check. IMO this makes the code easier to follow since we're now only callinguseDefaultFeedbackIfNecessaryin one place, but we could add the short-circuit option back in between the two types of sig fig check.Also moves the hardcoded
sig_figsfeedback tags to constants.