Skip to content

Get rid of unittest once and for all - #896

Merged
joanise merged 14 commits into
mainfrom
dev.ej/convert-to-pytest
Oct 15, 2026
Merged

joanise merged 14 commits into
mainfrom
dev.ej/convert-to-pytest

Conversation

@joanise

@joanise joanise commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

PR Goal?

Convert all out tests that are still based on unittest.TestCase to pytest.

Fixes?

Gives us more of the benefits of using pytest.

  • better output capturing
  • better test reports
  • the full pytest.fixture framework
  • pytest.parametrize
  • and more

Feedback sought?

Mostly rubber stamping, maybe sanity checking. This PR touches a lot of code, and the diff is very long, but it's just changing how we run tests, nothing in the real code.

Priority?

medium

Tests added?

only changes tests

How to test?

  • uv run pytest still runs all the tests -- note that there are now 407 instead of 403 due to some tests having been refactored using pytest.parametrize
  • coverage is unchanged
  • CI passes

Confidence?

high

Version change?

no

Related PRs?

EveryVoiceTTS/StyleTTS2#40
EveryVoiceTTS/FastSpeech2_lightning#170

Some of this refactoring was done with the assistance of Claude Opus 5.5, fully human reviewed.

@joanise
joanise requested a review from roedoejet October 14, 2026 20:59
@semanticdiff-com

semanticdiff-com Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor
CLI load time: 0:00.16
Pull Request HEAD: 92b21472aef867fb778c24675c49880617a710db

Imports that take more than 0.1 s:
import time: self [us] | cumulative | imported package
import time:      6799 |     153854 | everyvoice.cli
import time:      3330 |     141660 | typer.rich_utils

@codecov

codecov Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.25%. Comparing base (a6e1231) to head (92b2147).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #896   +/-   ##
=======================================
  Coverage   85.25%   85.25%           
=======================================
  Files          51       51           
  Lines        4789     4789           
  Branches      712      712           
=======================================
  Hits         4083     4083           
  Misses        534      534           
  Partials      172      172           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@roedoejet roedoejet left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mean get rid of unittest* once and for all 😆 but yes, skimmed this and it looks all standard. tests pass on a freshly installed env on my mac.

@joanise joanise changed the title Get rid of pytest once and for all Get rid of ~pytest~ unittest once and for all Oct 15, 2026
@joanise joanise changed the title Get rid of ~pytest~ unittest once and for all Get rid of unittest once and for all Oct 15, 2026
Convert test_evaluation, test_symbol_mapping, test_subsample,
test_wizard_helpers and test_utils from unittest.TestCase to pytest
style; assertLogs in test_utils is replaced by caplog.

Refactoring done with the assistance of Claude Opus 5.5, fully human
reviewed.
Refactoring done with the assistance of Claude Opus 5.5, fully human
reviewed.
WizardTestBase no longer derives from TestCase, so its subclasses
TestWizard and TestCustomG2p are converted in the same commit.
TestG2p's setUp/tearDown become an autouse fixture that saves and
restores AVAILABLE_G2P_ENGINES.

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
The subTest loops over model and config types become parametrized
tests, with a make_model() helper replacing the duplicated FastSpeech2
and HiFiGAN construction in the two TestLoadingModel tests.

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
The assertLogs check in test_config_lock is replaced by caplog,
cleared at the start of each fail_config_lock() call so that each
check only sees its own log messages. assertAlmostEqual(places=4)
becomes pytest.approx(abs=5e-5) to keep the same tolerance.

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
Also fix the TestDemoAppErros class name typo. The scripts are still
run as `coverage run -p ../test-demo-app.py` by regression-test.sh;
their __main__ blocks keep calling pytest.main(sys.argv).

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
The docstrings said to run test_demo_app_mix.py, which doesn't exist;
prep-datasets.sh copies each script to ../test-demo-app.py relative to
the regress directory, where regression-test.sh runs it.

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
We were using capture_stdout and capture_stderr extensively to
compensate for the fact that unittest leaks a lot of output to screen.

Pytest cleanly captures everything, so we no longer need to do that.
The test suite is now fully in pytest style; this local pygrep hook
keeps unittest.TestCase from creeping back in.

Refactoring done with the assistance of Claude Opus 5.5, fully human reviewed.
@joanise
joanise force-pushed the dev.ej/convert-to-pytest branch from 3f6c93e to 92b2147 Compare October 15, 2026 17:59
@joanise
joanise merged commit 86fcab8 into main Oct 15, 2026
13 checks passed
@joanise
joanise deleted the dev.ej/convert-to-pytest branch October 15, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants