UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

This paper proposes UNet++, a neural architecture for semantic and instance segmentation that overcomes two limitations of U-Net and FCN variants: their unknown optimal depth and their restrictive skip connections that force feature aggregation only at same-scale encoder and decoder maps. UNet++ alleviates the unknown-depth problem with an efficient ensemble of U-Nets of varying depths that partially share an encoder and co-learn simultaneously through deep supervision, and it redesigns skip connections to aggregate features of varying semantic scales at the decoder, yielding a highly flexible feature fusion scheme. A pruning scheme is added to accelerate inference.

Evaluated on six medical image segmentation datasets across CT, MRI, and electron microscopy, UNet++ consistently outperforms the baseline models for semantic segmentation across datasets and backbones, and it improves segmentation quality for objects of varying sizes over the fixed-depth U-Net. As Mask RCNN++, the design surpasses the original Mask R-CNN for instance segmentation, and pruned UNet++ models achieve significant speedups with only modest performance degradation. This mattered as a widely adopted, flexible improvement to encoder-decoder segmentation networks.