Kubeflow¶
New Kubeflow Version 1.9¶
Kubeflow recently released Version 1.9 with a variety of feature improvements such as a centralized model registry, security enhancements, and integration/installation improvements. We have now deployed Kubeflow 1.9 in the MQS infrastructure and you can access Kubeflow with the Quantum & Machine Learning Tier: https://dashboard.mqs.dk/subscriptions
Here an overview of the important menu items under Pipelines which we currently utilize at MQS to design well thought out calculation pipelines:
- Pipelines
- Experiments
- Runs
- Recurring Runs
- Artifacts
- Executions
This allows us also to design the Cebule Application Programming Interface in a consistent way and to bridge MLOps together with quantum chemistry, quantum computing and machine learning tools.
Some of the changes between version 1.8 and 1.9 we found important with respect to navigating and applying our model library with Kubeflow:
The Central Kubeflow Dashboard UI has been improved, with all Kubeflow Pipelines links now being under the Pipelines tab as shown on the left below:
The UI has now been updated to support visualizing output artifacts such as the training loss plot from a MQS ML model fine-tuning pipeline:
Visualizing images in the UI allows one to conveniently view output results (ex. evaluate a training run with the loss plot) without having to download the output files and view them locally, as was previously needed.
Applying the MQS Model Library¶
As part of our Quantum & Machine Learning Tier, we provide quantum chemistry and ML models for on-demand use such as geometry optimization with different methods, and graph neural network training/fine-tuning to predict molecular target properties.
These models can be accessed through Pipelines -> Shared on the Central Dashboard:
Additionally, the models included in the Quantum & Machine Learning Tier, plus the Enterprise Graph Neural Network generating Hamiltonians of molecules, are available through the pay-per-usage Cebule API and can be used in a Jupyter Notebook with our SDK as described our GNN tutorial notebooks.
The following models/pipelines are available, with more being continuously added to the model library.
Overview of Kubeflow Pipelines¶
geometry_opt
Semi-empirical geometry optimization of molecule 3D coords after initial force field optimization.
Inputs:
force_field: str from [mmff94, ghemical].
optimization_method: str from [gfn2_xtb, am1].
smiles_list: List[str] of SMILES as JSON str.
Cebule TaskTypes applied:
GEOMETRY_OPT
Artifacts:
None
Output:
List[List[Tuple[float, float, float]]] as JSON str containing each molecule's optimized 3D coords.
model_training
Fine-tune a graph neural network on a custom dataset to predict a molecule property of interest.
Inputs:
model_name: str unique to this model (only letters, numbers, underscores).
query: str to select dataset molecules from MQS Database.
target_property: str for the model to learn to predict; currently [homo_lumo_gap] supported.
Cebule TaskTypes applied:
GNN_DATASET_CREATE
GEOMETRY_OPT
GNN_DATASET_EXTEND
GNN_TRAIN
Artifacts:
test_loss_mae: Metrics
loss_history: Metrics
loss_plot: HTML`
model: Model
pretrained_model: Model
Output:
None; The trained model is stored in MinIO object storage under the given model_name to be accessed via Kubeflow in the model_prediction pipeline.
model_prediction
Predict a property of interest with a pre-trained or fine-tuned model.
Inputs:
model_name: str to use for prediction; fine-tuned model or pretrained for pre-trained model.
smiles_list: List[str] of SMILES as JSON str.
Cebule TaskTypes applied:
GNN_PREDICT
Artifacts:
model: Model
Output:
List[float] in JSON containing predicted target property for given molecules.
Python SDK notebook examples¶
For a detailed example of running our Kubeflow pipelines, see our tutorial on the pipelines for training, fine-tuning and using our GNN models to predict molecule HOMO-LUMO gaps.
Further, we recommend to take a look at all Python-SDK example notebooks here: https://gitlab.com/mqsdk/python-sdk/-/tree/main/notebooks
We hope you find our integration of Kubeflow's latest version and our Quantum Chemistry/ML models library can be of value for interesting studies with molecular quantum information.
You are always welcome to reach out if you have any feedback, questions, ideas or even collaboration projects to push this kind of work and research further (contact (at) mqs [dot] dk).