I. Introduction to ZS-DeconvNet
Zero-shot deconvolution network (ZS-DeconvNet) is a deep learning-based computational super-resolution approach, which fully exploits the powerful feature representation and noise suppression capabilities of deep neural networks in an unsupervised manner and can instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary SR imaging conditions without the need for any training data or data priors such as sparsity and temporal continuity.
Representative raw images (left), conventional SIM (right), and ZS-DeconvNet-SIM images (middle) of clathrin coated pits (CCPs) and microtubules (MTs) in COS-7 cells.
ZS-DeconvNet is versatilely compatible with multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional (3D) wide-field microscopy, confocal microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy, which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans.
This homepage includes a general introduction and some typical applications of ZS-DeconvNet (this page), theoretical illustrations on the basic principle (see Principle), and tutorials for the implementation of the open-source python codes and ZS-DeconvNet Fiji plugin (see Tutorial), intended to give more informative instructions for potential users of our methods.
* Get our open-source codes and Fiji plugin from https://github.com/TristaZeng/ZS-DeconvNet!
* Get our open-source raw data from Datasets for Zero-Shot DeconvNet!
II. Characrization of ZS-DeconvNet
1. More than 10-fold SNR improvement and 1.5-fold resolution enhancement
The proposed ZS-DeconvNet consists of a dual-stage architecture and conduct image denoising and deconvolution using an integrated physical model-consistency unsupervised training strategy. We characterized that ZS-DeconvNet is able to improve the resolution by more than 1.5-fold even when trained the low SNR input images only and without involving user defined parameters, and yields higher SR fidelity can other existing unsupervised computations SR methods.
(Left) The representative SR images of Lyso and MTs reconstructed by RL deconvolution (second column), sparse deconvolution (third column) and ZS-DeconvNet (fourth column) with clear WF images displayed for reference are listed below. (Right) statistical comparisons of RL deconvolution, sparse deconvolution and ZS-DeconvNet in terms of PSNR, FRC resolution (n=100) and full width at half maximum (FWHM).
2. Compatible with diverse imaging modalities
The implementation of ZS-DeconvNet is not dependent on certain microscopy (see basic principles of ZS-DeconvNet) and can be widely compatible with multiple imaging modalities including total internal reflection fluorescence microscopy, 2D/3D wide-field microscopy, confocal microscopy, lattice light-sheet (LLS) microscopy, and multimodal structured illumination microscopy (SIM). Representative ZS-DeconvNet enhanced images acquired by WF microscopy (C. elegans embryo), confocal microscopy (early mouse embryo) , and LLS-SIM (F-actin in a HeLa cell and mitochondrial outer membrane in a 293T cell) are shown below.
3. Effectively trained on a single input image or stack
ZS-DeconvNet can be effectively trained with noisy 2D/3D data, e.g., noisy time-lapse images or even a single noisy image/stack, while yielding better SR performance than existing deconvolution approaches. Some representative SR images processed with ZS-DeconvNet models trained with the single input image/stack itself and images generated via Richardson-Lucy (RL) deconvolution and sparse deconvolution are shown below.
4. No user-defined hyper-parameters
Before using most existing unsupervised computational SR methods, e.g., SRRF, RL deconvolution, and sparse deconvolution, users must predetermine several hyper-parameters such as radiality radius, deconvolution iterations, and sparsity weighting scalars, for each independent set of images, which are closely associated with the output resolution and fidelity. By contrast, the resolution enhancement is realized by the deconvolution loss defined on the optical imaging forward model and system’s points spread function (PSF) in ZS-DeconvNet. Therefore, there’s no user-defined hyper-parameters in both training and inference procedures.
III. Application and Results
1. Subcellular dynamics and interactions of mitochondria and ER during mitosis visualized via 3D ZS-DeconvNet
SNR and resolution enhanced three-color volumetric imaging via 3D ZS-DeconvNet for 937 time points at 10 sec intervals in a HeLa cell stably expressing calnexin-mEmerald (ER in grey), H2B-mCherry (chromosome in orange) and Mito-Halo (mitochondria in cyan).
Time-lapse three-color 3D rendering images reconstructed via 3D ZS-DeconvNet of ER, H2B, and Mito.
2. Comparison of different deconvolution methods on a four-color confocal stack of an early mouse embryo
3D rendering of an early mouse embryo immunostained for microtubule bridges (cyan, left panel), chromosomes (orange, left panel), actin rings (magenta, right panel), and apical domain (green, panel), showing the contrast and resolution comparison across the mouse embryo, from raw confocal microscopy, RL deconvolution, sparse deconvolution, and 3D ZS-DeconvNet.
.png?raw=true)
.png?raw=true)
.png?raw=true)
.png?raw=true)
Time-lapse three-color 3D rendering images reconstructed via 3D ZS-DeconvNet of ER, H2B, and Mito.
3. Remodeling dynamics of F-actin and Myosin IIA during adhesion process after dropping a U2OS cell onto the coverslip
ZS-DeconvNet enables time-lapse records of the adhesion and spreading dynamics of a U2OS cell co-expressing mEmerald-lifeact (cyan) and mCherry-myosin IIA (yellow) for 110 time points at ~150 nm resolution and 5 sec intervals.
4. Dynamics of recycling-endosomes (REs) and lysosomes or late endosomes (Lyso/LEs) revealed by ZS-DeconvNet
The rapid dynamics of REs and Lyso/LEs are captured at the spatiotemporal resolution of 150 nm and 3 Hz for ~1,500 time points in a gene-edited SUM159 cell endogenously expressing EGFP-Rab11 (Green) and mCherry-Lamp1 (magenta). Right panels present the magnified images of the boxed region in the left, showing the dynamic behaviors of Lyso/LEs (top) and REs (bottom) of interests and their motion trajectories.
5. Visualizing disassemble and reassemble process of nuclear speckles during mitosis as seen by 3D ZS-DeconvNet enhanced LLSM
Two-color volumetric super-resolution imaging of nuclear speckles (green) and chromosomes (magenta) labelled with HeLa-SC35 and H2B-mCherry, respectively, at the speed of 10 sec per volume over 318 time points within a FOV of a group of HeLa cells, recording the entire multi-phase separation dynamics of nuclear speckles during mitosis with a high spatiotemporal resolution.
6. Volumetric dynamics of lysosomes and hypodermal cell fusion during the development of a C. elegans embryo revealed by 3D ZS-DeconvNet
Long-term volumetric SR imaging of a C. elegans embryo via 3D ZS-DeconvNet enhanced wide-filed microscopy for over 213 time points at 30 sec intervals with negligible photobleaching and phototoxicity.