Skip to content

Fix missing failure details for strict XPASS in allure-pytest - #933

Open
nonatomiclabs wants to merge 2 commits into
allure-framework:masterfrom
nonatomiclabs:display-full-messages-on-strict-xfails
Open

nonatomiclabs wants to merge 2 commits into
allure-framework:masterfrom
nonatomiclabs:display-full-messages-on-strict-xfails

Conversation

@nonatomiclabs

Copy link
Copy Markdown

Context

Pytest’s xfail marker identifies tests that are expected to fail, for instance because of a known bug. Pytest supports a "strict" xfail mode through @pytest.mark.xfail(strict=True) or the xfail_strict configuration option. In this mode, an unexpectedly passing test (XPASS) is treated as a failure. This is useful to ensure that xfail markers are removed when a bug is fixed.

Currently, allure-pytest reports these tests as Failed, but provides no failure details, leaving out both the strict XPASS indication and the xfail reason.

This PR populates the failure details with pytest’s [XPASS(strict)] message, including the reason when present. This follows the approach already used in allure-pytest-bdd (see the code and the "Xfail support" section in #845).

Regression tests cover explicit and configured strict mode, with and without a reason, and verify that existing XFAIL and ordinary failure reporting remain unchanged.

Checklist

Verification

  • The new unit tests pass:
    $ PYTEST_DISABLE_PLUGIN_AUTOLOAD=true python3 -m pytest tests/allure_pytest/acceptance/status/xfail_call_status_test.py -q
    ...........                                                                                                                                                                                                                             [100%]
    11 passed in 0.28s
    
  • I created a small project and we can see that the new behavior works as intended:
    • before:
      Capture d’écran 2026-10-26 à 11 04 37

    • after:
      Capture d’écran 2026-10-26 à 11 05 12

Include pytest's strict XPASS message and reason in Allure status details
when an xfailed test unexpectedly passes.
@nonatomiclabs nonatomiclabs changed the title Fix missing failure details for strict XPASS in *allure-pytest* Fix missing failure details for strict XPASS in allure-pytest Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant