Capsule-Based System for Object Segmentation Offers Accuracy at Less Cost

Technology #34138

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Ulas Bagci, Ph.D.
External Link (
Rodney Lalonde
Managed By
Andrea Adkins
Assistant Director 407.823.0138
Patent Protection

US Patent Pending
Capsules for Object Segmentation
Cornell University Library, arXiv:1804.04241

New SegCaps deep learning architecture enables more cost-effective segmentation of medical images.

Researchers at the University of Central Florida have developed an innovative capsule-based deep learning system that requires far less parameterization to perform object segmentation more accurately and efficiently than state-of-the-art convolutional neural networks (CNNs) such as U-Net. The UCF deep convolutional-deconvolutional capsule network called SegCaps also reduces the cost, time and memory space needed to automate the segmentation process.  The system works with many computer vision applications, such as medical imaging, for effective cancer diagnosis.

In one example application, UCF researchers used the system’s SegCaps network architecture to segment pathological lungs from low-dose CT scans. Experimental results showed that compared to the U-Net architecture, SegCaps reduced the number of parameters by 95.4 percent while still providing better segmentation accuracy. Also, the system aptly handled large image sizes (512 x 512 pixels) as well as baseline sizes (less than 32 x 32).

Technical Details

The invention comprises a novel multi-task deep convolutional-deconvolutional capsule architecture called SegCaps and methods for using the system for improved, accurate object segmentation. To dramatically reduce the memory and parameter burden of a capsule implementation, SegCaps acts locally when routing children capsules to parent capsules and also allows for the segmentation of large image sizes. It shares transformation matrices (sets of parameters for children capsules) across capsules within the same capsule type. All of the capsule storage occurs at the neuron level as vectors rather than scalars. Vectors contain information about spatial orientation, magnitude/prevalence, and other attributes of an extracted feature and represent the “capsule types” within a layer.

The system’s object segmentation process starts when a computing device receives an input image and passes it through a convolutional layer to produce feature maps. The maps form a set of children capsules with associated sets of parameters to model the spatial relationships of objects. SegCaps uses the children capsules to create a set of prediction vectors that are locally constrained within eachof the parent capsules' kernels. The effort produces locally accurate predictions for components of the input image, thus forming meaningful part-to-whole relationships not found in standard CNNs.

Partnering Opportunity

The research team is looking for partners to develop the technology further for commercialization.

Stage of Development

Prototype available.


  • Reduces the space, time and costs associated with object segmentation
  • Allows for a computing device to recognize images in a batch that are similar to an input image
  • Provides better accuracy than existing methods and requires 95 percent less parameterization
  • The algorithm is usable in other platforms besides UNIX


  • Medical imaging
  • Computer vision
  • Image segmentation for social media
  • Artificial intelligence
  • Machine learning
  • Robotics

Copyright and Trademark Information

Pending registration.