How Artificial Intelligence Enhances Confocal Imaging

Gentle Live-Cell Imaging with the STELLARIS confocal platform and AiviaMotion

 How_Artificial_Intelligence_Enhances_Confocal_Imaging_teaser.jpg

Live-cell imaging has come a long way. However, significant challenges still exist. These include balancing experimental gentleness with fast acquisition rates, high spatial resolution and large imaging dimensionality. Consequently, as a research scientist, you are struggling to get complete and accurate high-quality data when imaging fast biological processes. To break new ground and gain a deeper understanding of your scientific question, you need tools that can meet these challenges head on.

In this article, we show how artificial intelligence (AI) can enhance your imaging experiments. Namely, how AiviaMotion improves image quality while capturing the temporal dynamics of your live-cell specimen.

Introduction: An excellent standard for live-cell 3D imaging

Life is complex, dynamic and happens in 3D. The STELLARIS confocal platform is a great choice for capturing these hallmarks of life. Live-cell imaging, with STELLARIS enables you to capture the complex dynamics of the subcellular machinery and its microenvironment over time and with high 3D resolution.

Video 1: Truly simultaneous multicolor imaging of live cells (U2OS) in 3D. Mitochondria (MitoView Green), actin (SPY555-actin), microtubules (SiR-tubulin), and vesicles (CellBrite NIR750). Image series processed with Dynamic Signal Enhancement, AiviaMotion and LIGHTNING.

STELLARIS hardware for gentle and fast live-cell imaging

Confocal microscopy is generally considered a gentle, non-invasive method and well suited for live-cell studies. This fact is particularly true for STELLARIS. On the one hand, STELLARIS comes with a white light laser (WLL) that allows the matching of fluorophore excitation spectra to minimize adverse effects, such as phototoxicity. On the other hand, the sensitive Power HyD detector family enables you to faithfully detect even the faintest signals over a wide spectral range. Finally, employing the resonant scanner, STELLARIS is not only gentle to the sample, but also allows for capturing dynamic changes in live specimens with high temporal resolution.

Challenges of live-cell imaging

As researcher, you set the goal to discover novel insights and, thus, often push the limits of your imaging experiments. Accordingly, you may confront these challenges:

  • Can I get better temporal resolution to record neural activity even more accurately?
  • Can I visualize additional markers to get even more context from the immunological synapse?
  • Can I prolong my experiment to follow the behavior of cancer stem cells even longer?

These demands will inevitably require some trade-offs. The seemingly mutually exclusive limitations can be visualized with the imaging pyramid: gentleness, speed, image quality, and dimensionality. Consequently, tools which mitigate these trade-offs are highly sought after.

Dynamic Signal Enhancement (DSE) for increasing the signal-to-noise ratio (SNR)

In practice, the crucial trade-off for live-cell imaging is frequently between acquisition speed and image quality, because experimental gentleness and dimensionality often have certain requirements. Accordingly, since capturing the dynamics of living samples requires higher frame rates, live-cell imaging experiments commonly have a lower image quality than fixed samples due to less time for the detectors to capture photons from the specimen.

With STELLARIS, we introduce Dynamic Signal Enhancement (DSE) for real-time denoising of live-cell imaging series. DSE uses a rolling average calculation as its core algorithm. The basic idea of a rolling window is that noise between adjacent imaging frames is random while the biological signal is conserved. Accordingly, by averaging over several adjacent raw imaging frames with a rolling window, the relevant biological signal is accentuated while the unwanted noise is “averaged out”. In more expert terms, the DSE increases the signal-to-noise ratio (SNR) and ultimately leads to better image quality.

Video 2: Scheme to depict DSE calculation for vesicles (large red circles) using five raw data frames (upper images) for the rolling window (bracket). Noise (small red dots) gets averaged out while the biological signal (large red circles) is accentuated in the output frames (lower images). Note that for simplicity the weight parameter of the DSE is not shown. The weight parameter controls the contribution of raw data frames for the rolling window calculation (i.e., central frames are more weighted than peripheral frames) and helps to optimize the preservation of temporal dynamics.

Other “simple averaging” methods, i.e., averaging several raw images to output a single denoised image, reduce the temporal resolution by the averaging factor. For example, if you were averaging five raw image frames, the temporal resolution of the output image would be decreased by a factor of five. In contrast, DSE keeps the raw image frames and slides a fixed size averaging window over the entire raw image series. As result, the number of image frames and temporal resolution are identical between raw and DSE-processed imaging data.

AiviaMotion enhances the Dynamic Signal Enhancement: Better temporal dynamics and better signal-to-noise ratio at the same time

However, since we integrate over several raw data frames with the rolling average, the temporal dynamics could be reduced when applying DSE. For example, when using an unfavorable high number of frames for the rolling window, dynamic structures can begin to smear and show signs of doubled structures (“ghosting”). Accordingly, using a rolling window for denoising has its own trade-off: When averaging over more frames, the SNR gets better, but the temporal dynamics of fast processes are reduced. When averaging over less frames, the SNR is worse, but the temporal dynamics of fast processes are better preserved.

Here, the new DSE option AiviaMotion helps to boost your live-cell imaging results. With a single click, AiviaMotion uses AI to interpolate so-called “AiviaMotion frames” between the acquired “raw data frames”. Using the additional information from the AiviaMotion frames for rolling window calculation, DSE can now reach a higher SNR and better temporal dynamics at the same time.

Video 3: Scheme to depict DSE calculation with activated AiviaMotion option. Since both acquired raw data frames (upper images with white outline) and AiviaMotion frames (upper images with blue outline) are used for rolling window calculation, the temporal dynamics are better preserved with an activated AiviaMotion option (indicated by well-defined red circles in the lower images). In contrast to the previous example (of the “vanilla” DSE), now an average over only three acquired raw data frames could be used, since with the additional two AiviaMotion frames still a total of five frames is employed for the rolling window calculation. On top, because the AiviaMotion frames by themselves also filter noise, not only the temporal dynamics are better preserved, but also the SNR is higher than with the “vanilla” DSE.

In the animation from above, we have virtually doubled the temporal dynamics of the sample by only using three acquired raw images for the rolling window. At the same time, we still average over a total of five frames by using two additional AiviaMotion frames for the rolling window calculation. While it may seem intuitive that this should lead to the same SNR as for DSE with five raw frames and no AiviaMotion, in practice, we saw a further improvement of the SNR. This observation can be explained by how the AiviaMotion deep learning network works: AiviaMotion also acts as a filter by interpolating pixels, which are consistent over frames (i.e., biological signal), rather than noise.

Video 4: Comparison of raw data (left), Dynamic Signal Enhancement without AiviaMotion (middle), and Dynamic Signal Enhancement with AiviaMotion (right). An activated AiviaMotion option (right) leads to a higher signal-to-noise ratio and better preservation of temporal dynamics than the Dynamic Signal Enhancement without AiviaMotion (middle).

It is important to note that the number of frames you get as a result of DSE with AiviaMotion does not change. This number is identical to the number of raw data frames used as your input. So even though AiviaMotion may have “doubled” the number of frames used as your input for DSE, these frames are only transient, resulting in a higher SNR and better temporal dynamics for your live-cell experiment when using STELLARIS.

Combine with LIGHTNING

The STELLARIS software offers even more tools to improve your live-cell imaging experiment. The LIGHTNING detection concept is based on a fully automated, adaptive deconvolution method which you can easily combine with DSE and AiviaMotion in a single processing step. In this way, your live-cell imaging series gets improved in multiple ways – not only the noise gets removed but also the contrast is enhanced and the spatial resolution further refined. Thus, by combining DSE and LIGHTNING, we can mitigate the restrictions of the imaging pyramid even more.

Video 5: Gentle, volumetric multicolor imaging of U2OS live cells labelled with MitoView (green), SiR-lysosome (magenta), and CellBrite NIR750 (cyan). Image volume processed with Dynamic Signal Enhancement, AiviaMotion and LIGHTNING.

Conclusion

If you’re a scientist performing live-cell 3D imaging, now you can benefit from more advanced data acquisition during your experiments and collect more accurate and reliable results for publication. You just need to take advantage of the power of AI with Leica Microsystems’ brand new AiviaMotion software package for the STELLARIS confocal platform.

Related Pages

Interested to know more?

Talk to our experts. We are happy to answer all your questions and concerns.

Contact Us

Do you prefer personal consulting?

  • Leica Microsystems Inc.
    1700 Leider Lane
    Buffalo Grove, IL 60089 United States
    Office Phone : +1 800 248 0123
    Service Phone : 1 800 248 0223
    Fax : +1 847-236-3009

You will find a more detailed list of local contacts here.