TR&D3: Image processing and analysis, with an emphasis on analysis of cell and tissue organization in support of modeling

Microscope images provide information about biological systems that is typically unavailable from any other source. Over the past thirty years the development of new microscopy methods, the advent of digital recording, and the development of high throughput microscopes created the ability to routinely collect terabytes of images containing detailed molecular and structural information. However, the breaking of the logjam in acquiring images has led to a bottleneck in analyzing them.


Example synthetic 3D image for HeLa cells The image shows the nuclear membrane in red, the cell membrance in blue, and lysosomal membranes in green. It was randomly generated from a model learned from real 3D microscope images using the approach described in Peng and Murphy. (Peng,T. and R.F.Murphy. 2011. Image-derived, three-dimensional generative models of cellular organization. Cytometry A 79:383-391.)


In TR&D3, we are addressing three specific, critical challenges in image processing and analysis: methods to convert diverse, micron resolution images to use in the modeling of cellular processes at molecular resolutions; paths to reconstructing the three-dimensional connectivity of neuronal tissue and relating the structure to neuronal activity; and  tools to provide fast access to various types of processed images whose sizes strain current methods, especially for interactive visualization. Overcoming these challenges will enable a dramatic increase in the information that can be extracted from images and in the routine use of cutting-edge modeling tools by cell biologists, developmental biologists, and neurobiologists.

To meet these challenges, we are focusing on three areas:

  • The development and distribution of software to build models of cellular and subcellular organization from fluorescent microscope images

    This software will include interfaces which provide new capabilities to cell simulation software such as MCell, Virtual Cell, and Smoldyn. It will improve existing object-based models of protein distribution at a single time point, and include tools to estimate generative spatial models of single protein concentrations at a single time point, generative spatiotemporal models for single proteins, and dependencies between spatiotemporal models for different proteins.

  • The creation of high efficiency registration and analysis algorithms for petavoxel image sets, especially image sets from serial section electron microscopy (ssEM).

    These algorithms will be implemented across a wide range of computing platforms and will improve the detection of alignment points, leading toward the fully automatic assembly of datasets exceeding one petavoxel. These algorithms will also include semi-automated methods to align datasets from different sources (such as EM and optical microscopy), and to trace and segment neural pathways and other structures within registered ssEM datasets.

  • The development of a new multi-platform framework called the Virtual Volume Filesystem (VVFS), enabling the efficient delivery of images from large datasets (~100 gigbytes or larger), especially the volumetric data produced by DBP5.

    The VVFS will allow users to enter data in optimized VVFS formats and insert algorithms into the VVFS pipeline to create customized, on-the-fly transformations. Results will be delivered as virtual files to analysis programs on users’ computing platforms. As an example application that uses the system, a Virtual Volume Viewer will be implemented to provide interactive viewing of VVFS datasets while navigating in arbitrary 3D orientations.



An illustration of a model of the distribution of Arp3 in the helper T cell [being built in collaboration with DBP4]. Left: 40 seconds before immunological synapse formation. Going across rows left to right and progressing top to bottom, each subimage shows a cross section of the 3D protein distribution averaged across 17 cells. Each slice is perpendicular to the synapse (the face of the half ellipsoid template) and shows the synapse as the top edge of the shape shown. The synapse has not yet formed, so actin branching is occurring uniformly all around the periphery of the cell. Right: Model at the time of immunological synapse formation. Actin branching appears to be concentrated near the synapse as expected.


TR&D3 Research Highlights


Tools for determining the spatial relationships

An important task for understanding how cells are organized is determining which components have spatial patterns that are related to each other. Read more


4d rtd

Pipeline for creation of spatiotemporal maps

Using a combination of diffeomorphic methods and improved cell segmentation, we developed a CellOrganizer pipeline for use in DPB4 to construct models of the 4D distributions of actin and 8 of its regulators during the response of T cells to antigen presentation. Read more



Development of models of cell and nuclear shape

We added a major new capability to our open source CellOrganizer system, the ability to construct diffeomorphic models of cell and nuclear shape. The models were developed because of the needs of DBP4... Read More 


Mouse visual cortex
Anatomy and Function of an Excitatory Network in the Visual Cortex

MMBioS researcher Greg Hood’s collaboration with Wei-Chung Allen Lee of Harvard University and R. Clay Reid of the Allen Institute for Brain Science concerning the reconstruction of an excitatory nerve-cell network in the mouse brain cortex at a subcellular level using the AlignTK software has been published in Nature. Read more


CellOrganizerCellorganizer 2.0 Major Release

A major new release of the CellOrganizer system for creating image-derived models of cell shape and organization has just been published.  Read more.

View all Research Highlights