# UCSD DSE MAS Capstone Project: Neuronal Circuit and Synapse Analysis with Deep Neural Networks


This objective of this capstone project is to improve the accuracy of automated synapse segmentations in electron microscopy datasets.

This project extends the existing CDeep3M tool with 3D image processing and flood-filling networks (FFN), which comprises the new tool known as CDense3M.

CDense3M Overview:
 * Provides a plug-and-play deep learning solution for large-scale image segmentation of light, electron, and X-ray microscopy
 * New models improve voxel-wise precision and recall of the dense segmentation of synaptic densities by 10% compared to the best CDeep3M2 synapse segmentation model
 * Distributed as a CloudFormation template for provisioning CDense3M-capable AWS cloud instances, as a Docker container for both cloud and local installations, and as a series of Colab notebooks for free, interactive use
 * Backwards compatible, allowing users to continue using models that have been trained with earlier versions of CDeep3M
 * Provides enhanced robustness using automated image enhancements such as 3D labeling, 3D vesicle filtering, and erosion models
 * Code implemented in Python 3
 * Generates automatically enhanced images and colored overlays of the organelle and cell-body segmentations with the enhanced images for visual verification

## Running CDense3M

| Name | Description | Link | Documentation
| ------ | ------ | ------ | ------ |
| **CDense3M-Docker:** | Local or remote, large runs, long trainings, simple installation, GPU with minimum 12GB video RAM required | [Link](https://hub.docker.com/r/ncmir/cdeep3m) | [Documentation](https://github.com/CRBS/cdeep3m2/wiki/CDeep3M-Docker) |
| **CDense3M-AWS:** | Remote, large runs, long trainings, simple installation, pay for GPU/hour (around $0.54/hour). Uses CDense3M-Docker image. | [Link](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=cdeep3m-stack-py3-docker&templateURL=https://cf-templates-1i8oypshb6jhq-us-west-2.s3-us-west-2.amazonaws.com/cloud_formation_cdeep3m_py3-docker.json) | [Documentation](https://github.com/CRBS/cdeep3m2/wiki/CDeep3M2-on-AWS-(via-Cloud-Formation-Template)) |
| **CDense3M-Colab:**  | Remote, short runs or re-training, simple installation, free GPU access with up to 12 hour usage limit | [Link](https://github.com/haberlmatt/cdeep3m-colab) | [Documentation](https://github.com/haberlmatt/cdeep3m-colab) |

## Steps to Process Images with CDense3M

* [Training CDense3M organelle model: train the CNN to recognize organelles in 3D image stacks (or 2D images)](https://github.com/CRBS/cdeep3m2/wiki/PreprocessTrainingData.py-and-runtraining.sh)

* [Training FFN model: train the 3D CNN to recognize organelles in 3D image stacks](https://github.com/google/ffn/#training)

* [Prediction: image segmentation](https://github.com/CRBS/cdeep3m2/wiki/runprediction.sh)

* [Transfer Learning: adapt a pre-trained model to a new dataset or different object](https://github.com/CRBS/cdeep3m2/wiki/Transfer-Learning)

For prediction and transfer learning you can use either a [pre-trained model from the modelzoo](https://cdeep3m.crbs.ucsd.edu/home/pre_trained_models) or your own model generated during training.

## Data

The data that was used for this capstone project is not publicly available. Unfortunately, the data cannot currently be shared because it will be used in a future publication. Please email haberlmatt@gmail.com if you would like access to the data.