'segmenteverygrain' is a Python package that aims to detect grains (or grain-like objects) in images. The goal is to develop an ML model that does a reasonably good job at detecting most of the grains in a photo, so that it will be useful for determining grain size and grain shape, a common task in geomorphology and sedimentary geology. 'segmenteverygrain' relies on SAM 2.1 (Segment Anything Model 2.1), developed by Meta, for getting high-quality outlines of the grains. However, SAM requires prompts for every object detected and, when used in 'everything' mode, it tends to be slow and results in many overlapping masks and non-grain (background) objects. To deal with these issues, 'segmenteverygrain' relies on a Unet-style, patch-based convolutional neural network to create a first-pass segmentation which is then used to generate prompts for the SAM-based segmentation. Some of the grains will be missed with this approach, but the segmentations that are created tend to be of high quality.
'segmenteverygrain' also includes a set of functions that make it possible to clean up the segmentation results: delete and merge objects by clicking on them, and adding grains that were not segmented automatically. The QC-d masks can be saved and added to a dataset of grain images. These images then can be used to improve the Unet model. Many of the images used in the dataset are from the sedinet project.
Python 3.10 or higher. All dependencies (TensorFlow, PyTorch, SAM 2, and others) are handled by the conda environment files or by pip — see the installation instructions below.
More documentation is available at https://zsylvester.github.io/segmenteverygrain/index.html.
The quickest way to try 'segmenteverygrain' — without installing anything — is to run the Segment_every_grain_colab.ipynb notebook in Google Colab.
For local use, we recommend cloning the repository (it contains the trained U-Net model and the example images) and creating a conda environment from the provided environment files. If you do not have conda, install miniforge first.
Clone the repository:
git clone --depth 1 https://github.com/zsylvester/segmenteverygrain.git
Create the environment (this also installs the segmenteverygrain package and JupyterLab).
Linux/Windows:
conda env create -f segmenteverygrain/environment.yml
Mac (Apple Silicon):
conda env create -f segmenteverygrain/environment_macos.yml
Activate the environment and launch JupyterLab from the repository folder:
conda activate segmenteverygrain
cd segmenteverygrain
jupyter lab
Then open notebooks/Segment_every_grain.ipynb and run the cells from the top; the first cells download the SAM 2.1 model checkpoint (~860 MB) automatically.
If you only need the library as a dependency in an existing environment (without the notebooks and model files), 'segmenteverygrain' is also available on PyPI:
pip install segmenteverygrain
Detailed step-by-step instructions — including conda setup from scratch, GPU notes for each platform, and troubleshooting — are in the installation guide.
See the Segment_every_grain.ipynb notebook for an example of how the models can be loaded and used for segmenting an image and QC-ing the result. The notebook goes through the steps of loading the models, running the segmentation, interactively updating the result using the GrainPlot class, and saving the grain data and the mask.
The interactive editing interface (GrainPlot) provides the following controls:
- Left-click on grain-free area: Instant grain creation
- Left-click on existing grain: Select/unselect
- Alt + Left-click: Foreground prompt for multi-prompt grain creation
- Alt + Right-click: Background prompt for multi-prompt grain creation
- Shift + drag: Draw scale bar for unit conversion
- c: Create grain from placed prompts
- d: Delete selected grains
- m: Merge selected grains
- z: Undo last created grain
- h: Toggle coverage mask (highlights unsegmented areas in red)
- Esc: Clear all selections and prompts
- Ctrl (hold): Temporarily hide grain masks
The screen recording below shows how new grains can be added and objects that are not proper grains can be deleted. The green dots are 'grain' prompts (Alt + Left-click); the red dots are background prompts (Alt + Right-click).
seg_new_editing_interface_optimized.mp4
The images below illustrate how a relatively large thin-section image of a sandstone can be segmented using segmenteverygrain. Image from Digital Rocks Portal.
If the base Unet model does not work well on a specific type of image, it is a good idea to generate some new training data (a few small images are usually enough) and to fine tune the base model so that it works better on the new image type. This can be done by running the cells in the last section ('Finetuning the base model') of the Segment_every_grain.ipynb notebook.
The grain_utils module provides tools for extracting individual grain images and clustering them for classification tasks:
- Extract grain images: Crop and normalize individual grains from segmented images
- Feature extraction: Use pre-trained CNNs (VGG16, ResNet50, InceptionV3) or color features
- Clustering: Group similar grains using K-means, DBSCAN, or hierarchical clustering
- Interactive labeling: Use
ClusterMontageLabelerto manually label grains by category - Quality control: Use
ClusterMontageSelectorto remove unwanted grains or clusters
See the documentation for detailed examples.
The Segment_every_grain_colab.ipynb has been adjusted so that the segmentation can be tested in Google Colab. That said, the interactivity in Colab is not as smooth as in a local notebook.
It takes 2 minutes and 40 seconds to run the full segmentation on a 3 megapixel (e.g., 1500x2000 pixels) image, on an Apple M2 Max laptop with 96 GB RAM. The same image takes the same amount of time to segment using Google Colab with a Nvidia A100 GPU.
Obviously, large images take longer to process. The segmentation of the ~20 megapixel example image that is provided in the repository ('mair_et_al_L2_DJI_0382_image.jpg') takes ~20 minutes with both hardware configurations mentioned before. As the processing of large images is done in patches, the increase in computational time is roughly linear.
We welcome contributions from anyone interested in improving the project. To contribute to the model use the following steps:
- Fork the repository.
- Create a new branch for your changes:
git checkout -b feature/my-feature- Make your changes and commit them:
git add .
git commit -m "Add my feature"- Push your changes to your forked repository:
git push origin feature/my-feature- Create a pull request from your forked repository back to the original repository.
If you encounter any issues or problems while using segmenteverygrain, we encourage you to report them to us. This helps us identify and address any bugs or areas for improvement.
To report an issue, please follow these steps:
- Check the Existing Issues: Before submitting a new issue, search our issue tracker to see if the problem you're experiencing has already been reported. If you find a similar issue, you can add any additional information or comments to that existing issue.
- Create a New Issue: If you don't find an existing issue that matches your problem, create a new issue by clicking the "New issue" button on the issues page. Provide a clear and descriptive title for your issue, and include the following information in the description:
- A detailed description of the problem you're experiencing, including any error messages or unexpected behavior.
- The steps to reproduce the issue, if possible.
- Your operating system and the version of the software you're using.
- Any relevant logs or screenshots that could help us understand the problem.
- Submit the Issue: Once you've provided all the necessary information, click the "Submit new issue" button to create the issue. Our team will review the issue and respond as soon as possible.
We appreciate you taking the time to report any issues you encounter. Your feedback helps us improve.
Dave Matthews wrote the interactions module from scratch and, by doing this, made the interactive part of segmenteverygrain much faster and more user friendly. Thanks to Danny Stockli, Nick Howes, Kalinda Roberts, Jake Covault, Matt Malkowski, Raymond Luong, Wilson Bai, Rowan Martindale, and Sergey Fomel for discussions and/or helping with generating training data. Funding for this work came from the Quantitative Clastics Laboratory industrial consortium at the Bureau of Economic Geology, The University of Texas at Austin.
If you use segmenteverygrain in your research, please cite the following paper:
Sylvester, Z., Stockli, D. F., Howes, N., Roberts, K., Malkowski, M. A., Poros, Z., Martindale, R. C., & Bai, W. (2025). Segmenteverygrain: A Python module for segmentation of grains in images. Journal of Open Source Software, 10(112), 7953. https://doi.org/10.21105/joss.07953
BibTeX:
@article{Sylvester2025,
doi = {10.21105/joss.07953},
url = {https://doi.org/10.21105/joss.07953},
year = {2025},
publisher = {The Open Journal},
volume = {10},
number = {112},
pages = {7953},
author = {Sylvester, Zoltán and Stockli, Daniel F. and Howes, Nick and Roberts, Kalinda and Malkowski, Matthew A. and Poros, Zsófia and Martindale, Rowan C. and Bai, Wilson},
title = {Segmenteverygrain: A Python module for segmentation of grains in images},
journal = {Journal of Open Source Software}
}segmenteverygrain is licensed under the Apache License 2.0.