Repository navigation
Conversation
| @@ -0,0 +1,97 @@ | |||
| # Design Specification for Pyvale orchestration | |||
|
|
|||
| ## Module name suggestions? | |||
There was a problem hiding this comment.
I like "workflow" and "pipeline". I feel like "Orchestrate" is too cumbersome to pronounce every time, but maybe it's just me 😂
|
|
||
| Initially, it may be worth attempting a naïve HPC deployment that chains components together within a large Slurm-driven parameter sweep, which wiill hopefully be a useful baseline for understanding the computational cost and behaviour of the complete workflow. Each workflow instance should be capable of running independently, with the relevant parameters passed through the full chain and the outputs stored in a consistent format. This will also allow the individual stages to be profiled and potential computational bottlenecks to be identified in a multimodule pyvale call. | ||
|
|
||
| The main goal will then be to develop a 'smarter' and more adaptive workflow. This will link image deformation/FEA, rendering, DIC, strain extraction and VVUQ into a single process, with machine learning methods used to determine which workflow runs are most informative. Gaussian process-based surrogate models are one potential approach, with relationships between input and output can to be approximated without requiring every possible configuration to be evaluated. completed passes can be used to update the surrogate model and determine where additional samples should be taken. This could be based on uncertainty in the surrogate prediction, expected changes in the output, or a combination of both. Using this kind of approach the workflow can progressively focus computational effort on regions of an extensive design space that are required to imrpove inderstanding of the validation problem. |
There was a problem hiding this comment.
I agree, it would be good to start with GPR and GPR-based models. We can just use GPyTorch, PyApprox, or/and PyMC, they are all user-friendly. Not sure about whether it would be worthwhile to try other models, particularly NN-based ones, as they would require significantly more training data? In any case, I think it would be relatively straightforward to switch from GPyTorch to PyTorch.
| method="gaussian_process" | ||
| ) | ||
|
|
||
| result = optimiser.run( |
There was a problem hiding this comment.
Maybe also something like workflow.generate_model_data(...) ,workflow.fit_model(...), workflow.predict_model(...)?
|
|
||
| The aim will be to integrate Pyvale’s independently developed modules into a coherent, interoperable workflow despite their differing architectures and coding styles. The initial focus will be on establishing a consistent interface between the existing components, defining the data structures passed between them and identifying where additional wrappers or conversion steps are needed. | ||
|
|
||
| Initially, it may be worth attempting a naïve HPC deployment that chains components together within a large Slurm-driven parameter sweep, which wiill hopefully be a useful baseline for understanding the computational cost and behaviour of the complete workflow. Each workflow instance should be capable of running independently, with the relevant parameters passed through the full chain and the outputs stored in a consistent format. This will also allow the individual stages to be profiled and potential computational bottlenecks to be identified in a multimodule pyvale call. |
There was a problem hiding this comment.
| Initially, it may be worth attempting a naïve HPC deployment that chains components together within a large Slurm-driven parameter sweep, which wiill hopefully be a useful baseline for understanding the computational cost and behaviour of the complete workflow. Each workflow instance should be capable of running independently, with the relevant parameters passed through the full chain and the outputs stored in a consistent format. This will also allow the individual stages to be profiled and potential computational bottlenecks to be identified in a multimodule pyvale call. | |
| Initially, it may be worth attempting a naïve HPC deployment that chains components together within a large Slurm-driven parameter sweep, which will hopefully be a useful baseline for understanding the computational cost and behaviour of the complete workflow. Each workflow instance should be capable of running independently, with the relevant parameters passed through the full chain and the outputs stored in a consistent format. This will also allow the individual stages to be profiled and potential computational bottlenecks to be identified in a multimodule pyvale call. |
| pyvale.vvuq(...) | ||
| ) | ||
|
|
||
| workflow.add_parameter( |
There was a problem hiding this comment.
With parameters we will want to be able to sample them stochastically or deterministically or have the sampling driven by another stage like feedback for an optimiser.
We probably need: probabilistic samplers, latin hypercube, grid-based space filling, stratified random sampling etc.
|
|
||
| workflow.add_simulation(sim_data) # add some simdata stuff | ||
|
|
||
| workflow.add( |
There was a problem hiding this comment.
Not sure how linear this is going to be all the time - I think workflows might have branches and feedback loops with inner and outer loops.
|
|
||
| **### Example API** | ||
|
|
||
| some pseudo-code for a camera placement study? |
There was a problem hiding this comment.
We could start even simpler and just do a DIC convergence study as example 1? Then we start chaining things together after that?
No description provided.