Skip to content

refactor(analysis): remove fill_missing method (#97) - #167

Merged
Gerrrr merged 1 commit into
apache:masterfrom
adambernier:remove-fill-missing
Aug 12, 2026
Merged

refactor(analysis): remove fill_missing method (#97)#167
Gerrrr merged 1 commit into
apache:masterfrom
adambernier:remove-fill-missing

Conversation

@adambernier

@adambernier adambernier commented Aug 7, 2026

Copy link
Copy Markdown
Contributor
File Action Details
otava/analysis.py MODIFY Delete the fill_missing() function implementation.
otava/series.py MODIFY Remove from otava.analysis import fill_missing and all call sites of fill_missing(). Pass raw data series directly.
tests/analysis_test.py MODIFY Remove unit tests covering fill_missing().
tests/series_test.py MODIFY Update tests to ensure unpadded series data processes without errors. 
$ uv run tox
test: commands_pre[0]> uv sync --active --locked --all-extras -v
DEBUG Found workspace root: `/home/acbernier/otava`
DEBUG Adding root workspace member: `/home/acbernier/otava`
DEBUG Skipping `pyproject.toml` in `/home/acbernier/otava` (no `[tool.uv]` section)
DEBUG Searching for user configuration in: `/home/acbernier/.config/uv/uv.toml`
DEBUG uv 0.12.2 (x86_64-unknown-linux-gnu)
DEBUG Found project root: `/home/acbernier/otava`
DEBUG Using active virtual environment `build/.tox/test` instead of project environment `.venv`
DEBUG No Python version file found in workspace: /home/acbernier/otava
DEBUG Using Python request `>=3.10, <3.15` from `requires-python` metadata
DEBUG Using active virtual environment `build/.tox/test` instead of project environment `.venv`
DEBUG Checking for Python environment at: `build/.tox/test`
DEBUG The project environment's Python version satisfies the request: `Python >=3.10, <3.15`
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Found static `requires-dist` for: /home/acbernier/otava/
DEBUG Existing `uv.lock` satisfies workspace requirements
Resolved 72 packages in 0.88ms
DEBUG Computed cache info: Timestamp(SystemTime { tv_sec: 1786113363, tv_nsec: 458263624 }), None, None, {}, {"src": None}. Most recently modified: pyproject.toml
DEBUG Requirement already installed: apache-otava==0.8.0 (from file:///home/acbernier/otava)
DEBUG Requirement already installed: configargparse==1.7.1
DEBUG Requirement already installed: dateparser==1.2.0
DEBUG Requirement already installed: google-cloud-bigquery==3.38.0
DEBUG Requirement already installed: numpy==2.2.6
DEBUG Requirement already installed: pg8000==1.31.5
DEBUG Requirement already installed: pystache==0.6.8
DEBUG Requirement already installed: python-dateutil==2.9.0.post0
DEBUG Requirement already installed: requests==2.33.0
DEBUG Requirement already installed: ruamel-yaml==0.18.16
DEBUG Requirement already installed: scipy==1.16.3
DEBUG Requirement already installed: slack-sdk==3.39.0
DEBUG Requirement already installed: tabulate==0.9.0
DEBUG Requirement already installed: validators==0.35.0
DEBUG Requirement already installed: autoflake==2.3.1
DEBUG Requirement already installed: flake8==7.3.0
DEBUG Requirement already installed: isort==7.0.0
DEBUG Requirement already installed: pre-commit==4.5.0
DEBUG Requirement already installed: pytest==9.0.3
DEBUG Requirement already installed: pytest-benchmark==5.2.3
DEBUG Requirement already installed: pytz==2025.2
DEBUG Requirement already installed: ruff==0.14.8
DEBUG Requirement already installed: tox==4.32.0
DEBUG Requirement already installed: regex==2025.11.3
DEBUG Requirement already installed: tzlocal==5.3.1
DEBUG Requirement already installed: google-api-core==2.28.1
DEBUG Requirement already installed: google-auth==2.43.0
DEBUG Requirement already installed: google-cloud-core==2.5.0
DEBUG Requirement already installed: google-resumable-media==2.7.2
DEBUG Requirement already installed: packaging==25.0
DEBUG Requirement already installed: scramp==1.4.6
DEBUG Requirement already installed: six==1.17.0
DEBUG Requirement already installed: certifi==2025.10.5
DEBUG Requirement already installed: charset-normalizer==3.4.4
DEBUG Requirement already installed: idna==3.15
DEBUG Requirement already installed: urllib3==2.7.0
DEBUG Requirement already installed: ruamel-yaml-clib==0.2.14
DEBUG Requirement already installed: pyflakes==3.4.0
DEBUG Requirement already installed: mccabe==0.7.0
DEBUG Requirement already installed: pycodestyle==2.14.0
DEBUG Requirement already installed: cfgv==3.4.0
DEBUG Requirement already installed: identify==2.6.15
DEBUG Requirement already installed: nodeenv==1.9.1
DEBUG Requirement already installed: pyyaml==6.0.3
DEBUG Requirement already installed: virtualenv==20.36.1
DEBUG Requirement already installed: iniconfig==2.3.0
DEBUG Requirement already installed: pluggy==1.6.0
DEBUG Requirement already installed: pygments==2.20.0
DEBUG Requirement already installed: py-cpuinfo==9.0.0
DEBUG Requirement already installed: cachetools==6.2.1
DEBUG Requirement already installed: chardet==5.2.0
DEBUG Requirement already installed: colorama==0.4.6
DEBUG Requirement already installed: filelock==3.24.3
DEBUG Requirement already installed: platformdirs==4.5.0
DEBUG Requirement already installed: pyproject-api==1.10.0
DEBUG Requirement already installed: googleapis-common-protos==1.72.0
DEBUG Requirement already installed: proto-plus==1.26.1
DEBUG Requirement already installed: protobuf==6.33.5
DEBUG Requirement already installed: grpcio==1.76.0
DEBUG Requirement already installed: grpcio-status==1.76.0
DEBUG Requirement already installed: pyasn1-modules==0.4.2
DEBUG Requirement already installed: rsa==4.9.1
DEBUG Requirement already installed: google-crc32c==1.7.1
DEBUG Requirement already installed: asn1crypto==1.5.1
DEBUG Requirement already installed: distlib==0.4.0
DEBUG Requirement already installed: typing-extensions==4.15.0
DEBUG Requirement already installed: pyasn1==0.6.4
DEBUG Preserving seed package: pip==26.2
Checked 67 packages in 0.88ms
test: commands[0]> ruff check --diff .
test: commands[1]> autoflake --exclude build,.venv --recursive --remove-all-unused-imports --check .
./tests/graphite_test.py: No issues detected!
./hatch_build.py: No issues detected!
./tests/analysis_test.py: No issues detected!
./tests/report_test.py: No issues detected!
./tests/slack_notification_test.py: No issues detected!
./tests/cli_help_test.py: No issues detected!
./tests/util_test.py: No issues detected!
./tests/e2e_test_utils.py: No issues detected!
./tests/postgres_e2e_test.py: No issues detected!
./tests/graphite_e2e_test.py: No issues detected!
./otava/attributes.py: No issues detected!
./tests/tigerbeetle_test.py: No issues detected!
./otava/bigquery.py: No issues detected!
./tests/config_test.py: No issues detected!
./tests/csv_e2e_test.py: No issues detected!
./perf/perf_test.py: No issues detected!
./otava/util.py: No issues detected!
./tests/change_point_divisive_test.py: No issues detected!
./otava/graphite.py: No issues detected!
./otava/grafana.py: No issues detected!
./otava/change_point_divisive/__init__.py: No issues detected!
./otava/csv_options.py: No issues detected!
./otava/data_selector.py: No issues detected!
./tests/series_test.py: No issues detected!
./otava/change_point_divisive/significance_test.py: No issues detected!
./otava/change_point_divisive/detector.py: No issues detected!
./otava/postgres.py: No issues detected!
./otava/report.py: No issues detected!
./otava/config.py: No issues detected!
./otava/slack.py: No issues detected!
./otava/change_point_divisive/base.py: No issues detected!
./tests/importer_test.py: No issues detected!
./otava/change_point_divisive/calculator.py: No issues detected!
./otava/test_config.py: No issues detected!
./otava/analysis.py: No issues detected!
./otava/main.py: No issues detected!
./otava/series.py: No issues detected!
./otava/importer.py: No issues detected!
test: commands[2]> isort --check --diff .
Skipped 3 files
test: commands[3]> flake8 --count --show-source --statistics
0
test: commands[4]> pre-commit run --all-files --hook-stage pre-push insert-license
[INFO] Initializing environment for https://github.com/Lucas-C/pre-commit-hooks.
[INFO] Installing environment for https://github.com/Lucas-C/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Add license for all Python files.........................................Passed
Add license for all YAML files...........................................Passed
Add license for all Markdown files.......................................Passed
Add license for all Shell files..........................................Passed
Add license for all toml files...........................................Passed
Add license for all SQL files............................................Passed
Add license for all other files..........................................Passed
test: commands[5]> mkdir -p /home/acbernier/otava/build/test
test: commands[6]> pytest --verbose tests perf
======================== test session starts =========================
platform linux -- Python 3.12.3, pytest-9.0.3, pluggy-1.6.0 -- /home/acbernier/otava/build/.tox/test/bin/python3
cachedir: build/.tox/test/.pytest_cache
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/acbernier/otava
configfile: pyproject.toml (WARNING: ignoring pytest config in tox.ini!)
plugins: benchmark-5.2.3
collected 114 items                                                  

tests/analysis_test.py::test_single_series PASSED              [  0%]
tests/analysis_test.py::test_single_series_original PASSED     [  1%]
tests/analysis_test.py::test_significance_tester PASSED        [  2%]
tests/change_point_divisive_test.py::test_calculator_candidate PASSED [  3%]
tests/change_point_divisive_test.py::test_permutation_calculation PASSED [  4%]
tests/change_point_divisive_test.py::test_permutation_test PASSED [  5%]
tests/change_point_divisive_test.py::test_ttest PASSED         [  6%]
tests/change_point_divisive_test.py::test_get_intervals_requires_sorted_change_points PASSED [  7%]
tests/cli_help_test.py::test_otava_help_output PASSED          [  7%]
tests/cli_help_test.py::test_otava_analyze_help_output PASSED  [  8%]
tests/cli_help_test.py::test_otava_list_tests_help_output PASSED [  9%]
tests/cli_help_test.py::test_otava_list_metrics_help_output PASSED [ 10%]
tests/cli_help_test.py::test_otava_list_groups_help_output PASSED [ 11%]
tests/cli_help_test.py::test_otava_remove_annotations_help_output PASSED [ 12%]
tests/cli_help_test.py::test_otava_validate_help_output PASSED [ 13%]
tests/config_test.py::test_load_graphite_tests PASSED          [ 14%]
tests/config_test.py::test_load_csv_tests PASSED               [ 14%]
tests/config_test.py::test_load_test_groups PASSED             [ 15%]
tests/config_test.py::test_load_histostat_config PASSED        [ 16%]
tests/config_test.py::test_configuration_substitutions[slack_token] PASSED [ 17%]
tests/config_test.py::test_configuration_substitutions[bigquery_project_id] PASSED [ 18%]
tests/config_test.py::test_configuration_substitutions[bigquery_dataset] PASSED [ 19%]
tests/config_test.py::test_configuration_substitutions[bigquery_credentials] PASSED [ 20%]
tests/config_test.py::test_configuration_substitutions[grafana_url] PASSED [ 21%]
tests/config_test.py::test_configuration_substitutions[grafana_user] PASSED [ 21%]
tests/config_test.py::test_configuration_substitutions[grafana_password] PASSED [ 22%]
tests/config_test.py::test_configuration_substitutions[graphite_url] PASSED [ 23%]
tests/config_test.py::test_configuration_substitutions[postgres_hostname] PASSED [ 24%]
tests/config_test.py::test_configuration_substitutions[postgres_port] PASSED [ 25%]
tests/config_test.py::test_configuration_substitutions[postgres_username] PASSED [ 26%]
tests/config_test.py::test_configuration_substitutions[postgres_password] PASSED [ 27%]
tests/config_test.py::test_configuration_substitutions[postgres_database] PASSED [ 28%]
tests/config_test.py::test_config_section_yaml_parser_flattens_only_config_sections PASSED [ 28%]
tests/config_test.py::test_cli_precedence_over_env_vars PASSED [ 29%]
tests/config_test.py::test_unknown_argument_raises_error PASSED [ 30%]
tests/csv_e2e_test.py::test_analyze_csv PASSED                 [ 31%]
tests/csv_e2e_test.py::test_analyze_csv_multiple_branches_without_branch_flag_fails PASSED [ 32%]
tests/csv_e2e_test.py::test_analyze_csv_branch_flag_without_branch_column_fails PASSED [ 33%]
tests/csv_e2e_test.py::test_analyze_csv_with_branch_filter PASSED [ 34%]
tests/graphite_e2e_test.py::test_analyze_graphite PASSED       [ 35%]
tests/graphite_e2e_test.py::test_analyze_graphite_with_branch PASSED [ 35%]
tests/graphite_test.py::test_compress_target_paths PASSED      [ 36%]
tests/importer_test.py::test_import_csv PASSED                 [ 37%]
tests/importer_test.py::test_import_csv_with_metrics_filter PASSED [ 38%]
tests/importer_test.py::test_import_csv_with_time_filter PASSED [ 39%]
tests/importer_test.py::test_import_csv_with_unix_timestamps PASSED [ 40%]
tests/importer_test.py::test_import_csv_semicolon_sep PASSED   [ 41%]
tests/importer_test.py::test_import_csv_last_n_points PASSED   [ 42%]
tests/importer_test.py::test_import_histostat PASSED           [ 42%]
tests/importer_test.py::test_import_histostat_last_n_points PASSED [ 43%]
tests/importer_test.py::test_import_postgres PASSED            [ 44%]
tests/importer_test.py::test_import_postgres_with_time_filter PASSED [ 45%]
tests/importer_test.py::test_import_postgres_last_n_points PASSED [ 46%]
tests/importer_test.py::test_import_bigquery PASSED            [ 47%]
tests/importer_test.py::test_import_bigquery_with_time_filter PASSED [ 48%]
tests/importer_test.py::test_import_bigquery_last_n_points PASSED [ 49%]
tests/importer_test.py::test_graphite_substitutes_branch PASSED [ 50%]
tests/importer_test.py::test_graphite_branch_placeholder_without_branch_raises_error PASSED [ 50%]
tests/importer_test.py::test_postgres_branch_placeholder_without_branch_raises_error PASSED [ 51%]
tests/importer_test.py::test_bigquery_branch_placeholder_without_branch_raises_error PASSED [ 52%]
tests/importer_test.py::test_csv_no_branch_no_branch_column PASSED [ 53%]
tests/importer_test.py::test_csv_no_branch_single_branch_in_column PASSED [ 54%]
tests/importer_test.py::test_csv_no_branch_multiple_branches_raises_error PASSED [ 55%]
tests/importer_test.py::test_csv_branch_specified_no_branch_column_raises_error PASSED [ 56%]
tests/importer_test.py::test_csv_branch_specified_filters_rows PASSED [ 57%]
tests/postgres_e2e_test.py::test_analyze PASSED                [ 57%]
tests/postgres_e2e_test.py::test_analyze_and_update_postgres PASSED [ 58%]
tests/report_test.py::test_report PASSED                       [ 59%]
tests/report_test.py::test_json_report PASSED                  [ 60%]
tests/series_test.py::test_change_point_detection PASSED       [ 61%]
tests/series_test.py::test_change_point_min_magnitude PASSED   [ 62%]
tests/series_test.py::test_div_by_zero PASSED                  [ 63%]
tests/series_test.py::test_change_point_detection_performance PASSED [ 64%]
tests/series_test.py::test_get_stable_range PASSED             [ 64%]
tests/series_test.py::test_incremental_otava PASSED            [ 65%]
tests/series_test.py::test_validate PASSED                     [ 66%]
tests/series_test.py::test_can_append PASSED                   [ 67%]
tests/series_test.py::test_orig_edivisive PASSED               [ 68%]
tests/series_test.py::test_series_sparse_commits PASSED        [ 69%]
tests/series_test.py::test_series_irregular_timestamps PASSED  [ 70%]
tests/series_test.py::test_series_raw_initialization PASSED    [ 71%]
tests/slack_notification_test.py::test_blocks_dispatch PASSED  [ 71%]
tests/tigerbeetle_test.py::test_tb_old_defaults PASSED         [ 72%]
tests/tigerbeetle_test.py::test_tb_old_defaults_p05 PASSED     [ 73%]
tests/tigerbeetle_test.py::test_tb_old_defaults_p1 PASSED      [ 74%]
tests/tigerbeetle_test.py::test_tb_old_defaults_p2 PASSED      [ 75%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p2 PASSED        [ 76%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p1 PASSED        [ 77%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p01 PASSED       [ 78%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p001 PASSED      [ 78%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p0001 PASSED     [ 79%]
tests/tigerbeetle_test.py::test_tb_magnitude0_p00001 PASSED    [ 80%]
tests/util_test.py::test_merge_sorted PASSED                   [ 81%]
tests/util_test.py::test_remove_common_prefix PASSED           [ 82%]
tests/util_test.py::test_insert_multiple PASSED                [ 83%]
tests/util_test.py::test_sliding_window PASSED                 [ 84%]
tests/util_test.py::test_merge_dicts PASSED                    [ 85%]
tests/util_test.py::test_dict_list PASSED                      [ 85%]
tests/util_test.py::test_interpolate PASSED                    [ 86%]
perf/perf_test.py::test_tb_baseline0001 PASSED                 [ 87%]
perf/perf_test.py::test_tb_baseline001 PASSED                  [ 88%]
perf/perf_test.py::test_tb_baseline01 PASSED                   [ 89%]
perf/perf_test.py::test_tb_baseline1 PASSED                    [ 90%]
perf/perf_test.py::test_tb_baseline2 PASSED                    [ 91%]
perf/perf_test.py::test_tb_twice0001 PASSED                    [ 92%]
perf/perf_test.py::test_tb_twice001 PASSED                     [ 92%]
perf/perf_test.py::test_tb_twice01 PASSED                      [ 93%]
perf/perf_test.py::test_tb_twice1 PASSED                       [ 94%]
perf/perf_test.py::test_tb_twice2 PASSED                       [ 95%]
perf/perf_test.py::test_tb_incremental0001 PASSED              [ 96%]
perf/perf_test.py::test_tb_incremental001 PASSED               [ 97%]
perf/perf_test.py::test_tb_incremental01 PASSED                [ 98%]
perf/perf_test.py::test_tb_incremental1 PASSED                 [ 99%]
perf/perf_test.py::test_tb_incremental2 PASSED                 [100%]


----------------------------------------------------------------------------------- benchmark: 15 tests ------------------------------------------------------------------------------------
Name (time in ms)               Min                Max               Mean            StdDev             Median               IQR            Outliers       OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_tb_baseline0001         7.9230 (1.0)       9.2694 (1.0)       8.0740 (1.0)      0.1653 (1.0)       8.0401 (1.0)      0.0866 (1.18)          4;6  123.8546 (1.0)         114           1
test_tb_baseline001          8.5798 (1.08)     10.2639 (1.11)      8.7624 (1.09)     0.2499 (1.51)      8.7035 (1.08)     0.0731 (1.0)          7;10  114.1244 (0.92)        113           1
test_tb_incremental0001     11.1100 (1.40)     19.7569 (2.13)     12.3123 (1.52)     2.3668 (14.32)    11.3464 (1.41)     0.4484 (6.13)        10;17   81.2195 (0.66)         87           1
test_tb_baseline01          11.8020 (1.49)     13.5598 (1.46)     12.0242 (1.49)     0.2678 (1.62)     11.9714 (1.49)     0.1765 (2.41)          3;3   83.1657 (0.67)         83           1
test_tb_incremental001      11.8569 (1.50)     13.2494 (1.43)     12.1130 (1.50)     0.2127 (1.29)     12.0793 (1.50)     0.2214 (3.03)          9;2   82.5557 (0.67)         81           1
test_tb_twice0001           15.8170 (2.00)     19.2875 (2.08)     16.2970 (2.02)     0.5428 (3.28)     16.1524 (2.01)     0.4168 (5.70)          8;6   61.3611 (0.50)         60           1
test_tb_incremental01       16.4057 (2.07)     22.5458 (2.43)     16.7604 (2.08)     0.8075 (4.89)     16.6192 (2.07)     0.1726 (2.36)          2;3   59.6645 (0.48)         56           1
test_tb_twice001            17.2626 (2.18)     18.8334 (2.03)     17.5009 (2.17)     0.2744 (1.66)     17.4471 (2.17)     0.1566 (2.14)          4;4   57.1398 (0.46)         57           1
test_tb_twice01             23.8332 (3.01)     30.8713 (3.33)     24.7840 (3.07)     1.2522 (7.58)     24.4253 (3.04)     0.7926 (10.84)         4;4   40.3487 (0.33)         42           1
test_tb_baseline1           26.2696 (3.32)     29.4324 (3.18)     26.7632 (3.31)     0.7780 (4.71)     26.4512 (3.29)     0.3034 (4.15)          5;7   37.3648 (0.30)         38           1
test_tb_incremental1        35.6927 (4.50)     37.0809 (4.00)     36.1989 (4.48)     0.2729 (1.65)     36.2146 (4.50)     0.2590 (3.54)          5;3   27.6252 (0.22)         28           1
test_tb_baseline2           48.3863 (6.11)     51.1180 (5.51)     49.2043 (6.09)     0.8523 (5.16)     48.8588 (6.08)     1.4696 (20.09)         5;0   20.3234 (0.16)         21           1
test_tb_twice1              54.8857 (6.93)     60.1800 (6.49)     56.8059 (7.04)     1.8394 (11.13)    55.9817 (6.96)     3.5191 (48.12)         7;0   17.6038 (0.14)         17           1
test_tb_incremental2        60.6456 (7.65)     63.0708 (6.80)     61.4671 (7.61)     0.6465 (3.91)     61.2567 (7.62)     0.6884 (9.41)          4;1   16.2689 (0.13)         17           1
test_tb_twice2              95.7630 (12.09)    98.4475 (10.62)    97.1678 (12.03)    0.7144 (4.32)     97.3437 (12.11)    0.8321 (11.38)         2;0   10.2915 (0.08)         11           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
======================== 114 passed in 56.40s ========================
  test: OK (61.68=setup[0.03]+cmd[0.02,0.01,0.12,0.12,0.21,4.14,0.00,57.02] seconds)
  congratulations :) (61.70 seconds)

@Gerrrr

Gerrrr commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Hey @adambernier,

Can you please rebase on latest master?

@adambernier
adambernier force-pushed the remove-fill-missing branch from 1d55b12 to f79b714 Compare August 9, 2026 08:35
@adambernier

adambernier commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

rebased and uv tox run all passed

Gerrrr
Gerrrr approved these changes Aug 12, 2026
View reviewed changes

@Gerrrr Gerrrr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you for the contribution!

@Gerrrr
Gerrrr merged commit cd5bc1e into apache:master Aug 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Gerrrr Gerrrr Gerrrr approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adambernier @Gerrrr