CellEmbed SAM Introduction
This repository provides a complete pipeline for multimodal microscopic cell or nuclei instance segmentation, based on the CellEmbed-SAM family of models. The workflow covers dataset preparation, model training, evaluation, and inference. These models are designed for multimodal microscopic cell or nuclei instance segmentation in biomedical images. The framework supports multiple backbones (CellEmbed, InstanSeg, CellPoseSAM, CellPose3, MicroSAM, CellViT, Mediar) and includes practical utilities for data loading, augmentation, post-processing, evaluation, and a GUI visualization interface.
https://github.com/zhangjbig/CellEmbed-SAM.git
CellEmbed-main/ ├── cellembed/ # Main framework │ ├── pretrained CEmbed-SAM/ │ │ └── CEmbed_SAM/ │ │ └── model_weights.pth # Trained CEmbed-SAM checkpoint (used by GUI) │ ├── scripts/ │ │ ├── train.py # Training entry │ │ └── test.py # Evaluation entry │ ├── datasets/ # Stores .pth files for training/validation/testing │ │ # Generated by running raw_datasets/load_datasets.py │ ├── All_train/ # Trained weights & logs for multi-modality experiments │ │ ├── CEmbed SAM/ # each contains: model_weights.pth + experiment_log.csv │ │ ├── CellPose SAM/ │ │ ├── Micro SAM/ │ │ ├── CellPose3/ │ │ ├── CellViT/ │ │ ├── Instanseg/ │ │ └── Mediar/ │ ├── TNBC_2018/ # Dataset-specific training results (TNBC_2018 dataset) │ │ ├── CEmbed SAM/ # each contains: model_weights.pth + experiment_log.csv │ │ ├── CellPose SAM/ │ │ ├── Micro SAM/ │ │ ├── CellPose3/ │ │ ├── CellViT/ │ │ ├── Instanseg/ │ │ └── Mediar/ │ └── ... # Core models, losses, utilities │ ├── inference/ # Inference utilities │ └── inference.py # Batch inference on arbitrary PNG folders │ ├── othermodel/ # Baseline/alternative frameworks │ ├── cellposesam/ # CellPose-SAM │ ├── cellpose/ # CellPose3 │ ├── cellvit/ # CellViT │ ├── mediar/ # MediarFormer │ └── microsam/ # Micro-SAM │ ├── raw_datasets/ # Raw dataset loaders/adapters │ └── load_datasets.py # Converts raw image folders into .pth datasets │ ├── segmentation_anything/ # SAM backbone dependencies │ ├── gui/ # Simple GUI for cell segmentation │ └── segment_gui.py # Loads weights from pretrained CEmbed-SAM/model_weights.pth │ ├── sam_vit_l.pth # Pretrained SAM ViT-L weights ├── requirements.txt └── README.md
Installation
git clone https://github.com/zhangjbig/CellEmbed-SAM.git cd cellembed-sam pip install -r requirements.txt
Dataset Preparation
Use load_datasets.py to convert raw Slide_/GT_ image pairs into a .pth dataset. The dataset should be stored as .pth files containing training and validation images with corresponding labels.
raw_datasets/TNBC_NucleiSegmentation/
└── TNBC_and_Brain_dataset/
├── Slide_001/
│ ├── img1.png
│ ├── img2.png
│ └── ...
└── GT_001/
├── img1.png
├── img2.png
└── ...
Training
python -m cellembed.scripts.train \ --data_path ".\\cellembed\\datasets" \ --dataset "TNBC_2018" \ --model_str "cellembed" \ --output_path ".\\cellembed\\TNBC_2018" \ --experiment_str "CEmbed SAM" \ --num_epochs 200 \ --length_of_epoch 10 \ --batch_size 1 \ --num_workers 2 \ --optimizer "adamw" \ --device "cuda:0"
Testing
python -m cellembed.scripts.test \ --data_path ".\\cellembed\\datasets" \ --dataset "TNBC_2018" \ --model_path ".\\cellembed\\TNBC_2018" \ --model_folder "CEmbed SAM" \ --output_folder "Results" \ --test_set "Validation" \ --save_ims True
Interactive Segmentation (GUI Mode)
To perform interactive segmentation and visualization, simply run the GUI interface:
python -m gui.segment_gui
Output
Model checkpoints: model_weights.pth
Training logs: experiment_log.csv
Metrics: experiment_metrics.csv
Plots: loss.png → Training and validation loss curves f1_metric.png → F1-score progression
Example Additional Commands
python -m cellembed.scripts.train --data_path ".\\cellembed\\datasets" --dataset "TNBC_2018" --model_str "mediar" --output_path ".\\cellembed\\111" --experiment_str "mediar" --num_epochs 200 --length_of_epoch 10 --batch_size 1 --num_workers 2 --optimizer "adamw" --device "cuda:0" python -m cellembed.scripts.test --data_path ".\\cellembed\\datasets" --dataset "TNBC_2018" --model_path ".\\cellembed\\All_train" --model_folder "CEmbed SAM" --output_folder "Results" --test_set "Validation" --save_ims True