Multi-GPU cloud training of 3D segmentation AI
A client needed an AI to perform 3D segmentation of brain tumors from MRI scans.
While they had some in-house experience with algorithm development, they did not have expertise in renting clusters of cloud GPUs, training an AI neural network model on more than one GPU at a time, checkpointing, hyperparameter optimization, or creating intuitive real-time dashboards.
Theta Tetch AI built the client’s pipeline and trained a bleeding-edge transformer-based 3D segmentation model on a cluster of eight NVIDIA A100 GPUs all training the same neural network.
We set up web dashboards for the client to watch the segmentation results live and monitor the training accuracy in real-time.
AI Tech
- Neural Network: 3D Swin-Unet Transformer
- Multi-GPU training in cloud using Torch Lightning framework
Data Analyzed
- 3D, volumetric brain MRI scans
- Multi-modal data (T2-w, T1-w, T1+Contrast, FLAIR scans)
- 3D volume meshes of tumors
Strategy & Execution
- Gathered a variety of MRI scans from multiple institutions
- Visualizing data and segmentation results in 3D
- Preprocessing (e.g. histogram equalization) to normalize scans
- Setting up cross-validation experiments
- Reporting Dice accuracy on Weights & Biases dashboard during live training
- Deployed model in Docker container