Repository navigation
Revision updates - #118
Merged
Merged
Revision updates#118
Conversation
…d to sklearn logistic regressions only
Feat sklearn pca50
Feat sklearn single baseline
Feat survival task inclusion
…nd not through extra profile
…r inside profile of same name
… modules, update samplesheet
…ine, as it do not provide
…line to continue run
…st of paths, otherwise exceed arg length when long filenames
There was a problem hiding this comment.
🟡 Changes recommended
Several active survival paths produce incorrect metrics or fail due to unpropagated configuration, missing process inputs, and offline container provisioning gaps.
10 open findings
What changed in this PR
Adds survival-analysis support, single-modality benchmarking, and consolidated feature-selection hyperparameter reporting.
Changes:
- Adds scikit-survival, MOFA, and cooperative-learning survival workflows and metrics.
- Adds unimodal dataset expansion and outcome-aware preparation.
- Records and merges selected hyperparameters across methods.
| File | Description |
|---|---|
workflows/messi_benchmark.nf |
Wires unimodal data and outcome type. |
subworkflows/prepare_data/main.nf |
Adds modality splitting and survival preparation. |
subworkflows/methods/sksurv/main.nf |
Adds scikit-survival workflow. |
subworkflows/methods/sklearn/main.nf |
Adjusts reduction and result merging. |
subworkflows/methods/rgcca/main.nf |
Passes outcome type when merging. |
subworkflows/methods/mogonet/main.nf |
Passes outcome type when merging. |
subworkflows/methods/mofa/main.nf |
Adds survival-aware MOFA stages. |
subworkflows/methods/integrao/main.nf |
Passes outcome type when merging. |
subworkflows/methods/diablo/main.nf |
Passes outcome type when merging. |
subworkflows/methods/cooperative_learning/main.nf |
Adds survival-aware cooperative learning. |
subworkflows/methods/caret_multimodal/main.nf |
Passes outcome type when merging. |
subworkflows/feature_selection/main.nf |
Collects selected hyperparameters. |
subworkflows/cross_validation/r/main.nf |
Gates R methods by outcome. |
subworkflows/cross_validation/python/main.nf |
Adds survival and unimodal routing. |
subworkflows/cross_validation/main.nf |
Routes new CV inputs and outputs. |
nextflow.config |
Adds survival, modality, and container configuration. |
modules/split_train_test/resources/usr/bin/split_tr_te.py |
Adds survival-event stratification. |
modules/sksurv/train/resources/usr/bin/sksurv_train.py |
Implements survival model training. |
modules/sksurv/train/resources/usr/bin/manual_set_seed.py |
Adds reproducibility helper. |
modules/sksurv/train/resources/usr/bin/load_tr_te.py |
Loads survival fold data. |
modules/sksurv/train/resources/usr/bin/load_survival_model_class.py |
Defines survival model builders. |
modules/sksurv/train/resources/usr/bin/combine_mdata2df_survival.py |
Converts MuData for survival training. |
modules/sksurv/train/main.nf |
Defines survival training process. |
modules/sksurv/preprocess/resources/usr/bin/tr_te_split_mdata.py |
Splits survival MuData folds. |
modules/sksurv/preprocess/resources/usr/bin/sksurv_preprocess.py |
Prepares survival fold files. |
modules/sksurv/preprocess/resources/usr/bin/load_test_splits.py |
Loads fold indices. |
modules/sksurv/preprocess/main.nf |
Defines survival preprocessing process. |
modules/sksurv/predict/resources/usr/bin/sksurv_predict.py |
Produces survival predictions. |
modules/sksurv/predict/resources/usr/bin/manual_set_seed.py |
Adds prediction seed helper. |
modules/sksurv/predict/resources/usr/bin/load_survival_model_class.py |
Supplies prediction model definitions. |
modules/sksurv/predict/resources/usr/bin/generate_result_table_survival.py |
Formats survival results. |
modules/sksurv/predict/resources/usr/bin/combine_mdata2df_survival.py |
Converts prediction MuData. |
modules/sksurv/predict/main.nf |
Defines survival prediction process. |
modules/sklearn/train/resources/usr/bin/sklearn_train.py |
Uses shared MuData conversion. |
modules/sklearn/train/resources/usr/bin/combine_mdata2df.py |
Removes duplicated helper. |
modules/sklearn/select_feature/resources/usr/bin/sklearn_select_features.py |
Records tuned hyperparameters. |
modules/sklearn/select_feature/resources/usr/bin/run_random_search_cv.py |
Returns fitted randomized search. |
modules/sklearn/select_feature/resources/usr/bin/load_classifier_class.py |
Removes duplicated classifier loader. |
modules/sklearn/select_feature/resources/usr/bin/get_feats_df.py |
Handles unsupported MLP importance. |
modules/sklearn/select_feature/resources/usr/bin/combine_mdata2df.py |
Removes duplicated converter. |
modules/sklearn/select_feature/main.nf |
Emits hyperparameter JSON. |
modules/sklearn/predict/resources/usr/bin/sklearn_predict.py |
Uses shared converter interface. |
modules/sklearn/predict/resources/usr/bin/combine_mdata2df.py |
Removes duplicated converter. |
modules/rgcca/train/resources/usr/bin/run_rgcca.R |
Sets Horst scheme. |
modules/rgcca/select_feature/resources/usr/bin/rgcca_select_features.R |
Records RGCCA hyperparameters. |
modules/rgcca/select_feature/main.nf |
Emits RGCCA hyperparameters. |
modules/prepare_data/split_modality/resources/usr/bin/split_modalities.py |
Creates per-modality MuData files. |
modules/prepare_data/split_modality/main.nf |
Defines modality-splitting process. |
modules/prepare_data/prepare_mu_data/resources/usr/bin/transform_mudata_format.py |
Accepts outcome type. |
modules/prepare_data/prepare_mu_data/main.nf |
Passes MuData outcome type. |
modules/prepare_data/prepare_mae_data/resources/usr/bin/transform_mae_format.R |
Prepares survival metadata. |
modules/prepare_data/prepare_mae_data/resources/usr/bin/save_mae.R |
Preserves outcome columns. |
modules/prepare_data/prepare_mae_data/main.nf |
Passes MAE outcome type. |
modules/mogonet/select_feature/resources/usr/bin/mogonet_select_features.py |
Records MOGONET configuration. |
modules/mogonet/select_feature/main.nf |
Emits MOGONET hyperparameters. |
modules/mofa/train/resources/usr/bin/run_mofa.R |
Adds Cox survival training. |
modules/mofa/train/resources/usr/bin/run_mofa_survival.R |
Adds alternate survival trainer. |
modules/mofa/train/main.nf |
Passes outcome type to training. |
modules/mofa/select_feature/resources/usr/bin/mofa_select_features.R |
Records MOFA options. |
modules/mofa/select_feature/main.nf |
Emits MOFA hyperparameters. |
modules/mofa/preprocess/resources/usr/bin/preprocess_mofa.R |
Preserves survival outcomes in folds. |
modules/mofa/preprocess/main.nf |
Passes preprocessing outcome type. |
modules/mofa/predict/resources/usr/bin/predict_mofa.R |
Adds survival prediction output. |
modules/mofa/predict/main.nf |
Passes prediction outcome type. |
modules/merge_selected_hyperparameters/resources/usr/bin/merge_selected_hyperparameters.py |
Merges hyperparameter records. |
modules/merge_selected_hyperparameters/main.nf |
Defines hyperparameter merge process. |
modules/merge_result_table/resources/usr/bin/combine_tables.R |
Supports classification and survival tables. |
modules/merge_result_table/main.nf |
Adds outcome-aware result merging. |
modules/integrao/select_feature/resources/usr/bin/integrao_select_feature.py |
Records Integrao parameters. |
modules/integrao/select_feature/main.nf |
Emits Integrao hyperparameters. |
modules/diablo/train/resources/usr/bin/tune_diablo.R |
Uses Horst tuning scheme. |
modules/diablo/train/resources/usr/bin/run_diablo.R |
Uses Horst training scheme. |
modules/diablo/select_feature/resources/usr/bin/diablo_select_features.R |
Records DIABLO parameters. |
modules/diablo/select_feature/main.nf |
Emits DIABLO hyperparameters. |
modules/cooperative_learning/train/resources/usr/bin/run_cooperative_learning.R |
Adds Cox cooperative learning. |
modules/cooperative_learning/train/main.nf |
Passes training outcome type. |
modules/cooperative_learning/select_feature/resources/usr/bin/cplr_select_features.R |
Records selected parameters. |
modules/cooperative_learning/select_feature/main.nf |
Emits cooperative-learning parameters. |
modules/cooperative_learning/preprocess/main.nf |
Passes preprocessing outcome type. |
modules/cooperative_learning/predict/resources/usr/bin/predict_cooperative_learning.R |
Adds survival predictions. |
modules/cooperative_learning/predict/main.nf |
Passes prediction outcome type. |
modules/caret_multimodal/select_feature/resources/usr/bin/caret_multimodal_select_feature.R |
Records caret tuning details. |
modules/caret_multimodal/select_feature/main.nf |
Emits caret hyperparameters. |
modules/calculate_metrics/resources/usr/bin/calculate_survival_metrics.py |
Adds survival metrics. |
modules/calculate_metrics/resources/usr/bin/calculate_classification_metrics.py |
Separates classification metrics. |
modules/calculate_metrics/main.nf |
Selects metrics by outcome. |
Makefile |
Expands work-directory cleanup. |
launcher_local.sh |
Adds local HPC launcher. |
launch_MESSI_pipeline.sh |
Updates modules and survival profile. |
conf/simulated_data.config |
Enables classification modality baselines. |
conf/run_survival.config |
Adds survival execution profile. |
conf/real_data.config |
Enables real-data modality baselines. |
conf/modules.config |
Publishes merged hyperparameters. |
conf/dev-sockeye.config |
Revises development Apptainer settings. |
conf/base.config |
Increases retry count. |
bin/split_mae.R |
Preserves survival fold metadata. |
bin/rhelpers.R |
Adds hyperparameter JSON helpers. |
bin/python_utils/load_classifier_class.py |
Centralizes classifier definitions. |
bin/python_utils/combine_mdata2df.py |
Centralizes MuData conversion. |
bin/misc_utils/extract_Xy.R |
Extracts classification or survival outcomes. |
bin/misc_utils/checkers.R |
Strengthens sample alignment checks. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| input: | ||
| tuple val(method_name), path(list_result_tables) | ||
| val(saveMode) | ||
| val(outcome_type) |
Comment on lines
+38
to
+41
| echo ${dataset_name} | ||
| echo ${fold_name} | ||
| echo ${model_path} | ||
| echo ${test_path} |
Comment on lines
+43
to
+44
| echo ${dataset_name} | ||
| touch ${dataset_name}_fold_1 |
Comment on lines
+41
to
+43
| echo 'some text' > text.log | ||
| touch prediction.csv | ||
| touch model |
Comment on lines
+232
to
+233
| withLabel: sklearn { container = 'tonyliang19/eipy:latest' } // fix: should have a new sklearn container instead, for now use eipy bcz it contains XGBoost | ||
| withLabel: sksurv { container = 'tonyliang19/sksurv:latest' } |
| outcome_type = 'survival' | ||
|
|
||
| // Full test mode, run all methods and all data | ||
| single_modality_mode = true |
| exit 1 | ||
| else | ||
| set -o allexport | ||
| source .env # This sources the .env file |
Comment on lines
+79
to
+80
| time_col = "time" // colData column with follow-up time in days | ||
| event_col = "status" // colData column with event indicator (0/1) |
| // Survival model stuff | ||
| survival_model_names = ["rsf"] | ||
| // If sklearn wants to do single input mode, default: false | ||
| sklearn_single_mode = false |
| Defines input files and everything required to run on real datasets | ||
|
|
||
| Use as follows: | ||
| nextflow run main.nf -profile survival,<docker/singularity> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.