AI Microscopy Enables the Efficient Detection of Rare Events

The essential reason to use imaging techniques in the life sciences is the generation of data to answer biological questions. This aim is in general accomplished by using a combination of image acquisition and advanced image analysis.

Conventional imaging follows the constant interaction of an operator, searching for suitable objects or regions of interest (ROIs) on the sample, with the microscopy system and making appropriate optimal acquisition settings to decisively scan these ROIs. Due to the nature of such a manually defined experimental workflow, only a manageable number of ROIs can be precisely localized, and the acquisition of ROIs requires a lot of time.

The rare event detection workflow performed by applying Autonomous Microscopy powered by Aivia works completely autonomous after the setup at the beginning of the experiment. As soon as the experiment is started, no human intervention is required from a low-resolution pre-scan to the detection of rare events and the acquisition and storage of the high-resolution 3D images. The clear benefit is the high speed of this process and the large number of detected rare events during the experiment

Autonomous microscopy enables the automated detection of such ROIs or rare events (REs) without the need for human interaction and, thus, the complete automation of complex microscopic workflows. Within this autonomous workflow, low-resolution 2-dimensional (2D) overview images are generated in a first step, which are immediately transferred to the connected AI-based image processing (Aivia) system. This detects the rare events, previously defined by the operator, by means of a pixel classifier and sends the rare event coordinates back to the imaging system, which scans the rare events according to the operator's specifications, such as high-resolution and 3-dimensional (3D) data stacks. In this way, the generated data are:

1. highly specific to the object of interest,
2. available in a statistically relevant number due to automation.

The highly specific scanning of rare events massively increases the overall quality of the acquired data, as only data really of interest is obtained. This approach ensures target scanning with an accuracy of up to 90 % of all existing rare events. At the same time, a highly economical operation of the microscope system is achieved, because there are no long "idle times" due to time-consuming and, thus, costly manual searches for rare events.

Besides highly specific rare event scanning, autonomous microscopy enables the acquisition of data completely independent of the rare events objectives. Objects of interest can be scanned in a target-oriented manner, which, on the one hand, massively prevents the generation of junk data and, on the other hand, provides the acquired data in a highly qualitative manner for subsequent image analysis. These advantages are achievable, because the acquired object is always centrally located in the field of view (FOV) of the scan area.

Reproducibility is one of the key elements of trustworthy life science research. Autonomously working experiment sequences can be restored at any time, equipped with the appropriate versioning, and run again with exactly the same settings. This process ensures a reproducibility that could never be achieved by a manual setup imaging procedures for experiments.

Furthermore, highly complex microscopy workflows can be set up and automated, making advanced applications accessible that could not be addressed before at all or only with considerable manual effort.

LAS X Navigator is a powerful tool for navigation and image acquisition that allows users to switch from searching image by image to seeing the full sample overview quickly and identifying the important specimen details instantly. It enables the setup of high-resolution image acquisition automatically using templates for slides, dishes, and multi-well plates. Like a GPS for the cells of a specimen, users always have a clear path to high-quality data, to navigate through your sample, create fast overviews, identify details instantly, and to run high-resolution image acquisition using templates for slides, dishes, and multi-well plates.

Navigator Expert is based on the Navigator and represents a synergistic fusion with AI-based image processing by Aivia that provides the rare event processor workflow. In addition, with Navigator Expert, multiple scan jobs can be defined and assigned to arbitrary scan geometries / scan fields for complex imaging procedures and experimental setups.

Navigator Expert consists of the two modules: 'Jobs' and 'Experiment'. Within the 'Jobs' module, any scan jobs can be set up via the usual STELLARIS LAS X user interface. This module includes, for example, the definition of an autofocus (AF) job, the job for a fast 2D overview scan with low resolution as well as an actual rare event job which may be in high resolution and 3D.

The ‘Pixel Positions’ finally list the rare event positions detected by the Aivia ‘Rare Event Processor’.

To create the Aivia ‘Rare Event Processor’, it is sufficient to simply train the Aivia ‘Pixel Classifier’ on a small number of images previously acquired via the overview scan. Using simple painting tools, the desired rare events are annotated, and the background of the images is defined. From this, the Pixel Classifier can independently detect rare events from the overview-scan images.

After the rare event detection workflow is started, the overview-scan images are transferred directly to Aivia which pushes the pixel coordinates of detected rare events back to Navigator Expert, allowing the rare-event scans to be executed.

Besides the specifications mentioned above, AI-based autonomous microscopy can massively improve and optimize experiments.

• Reduce the total time and effort spent in front of the microscope by up to 75%
• Radically shorten acquisition times of high-value data by up to 70%
• Eliminate the need to store low quality data by up to 90%
• Increase reproducibility of your rare event detection
• Increase data quality with AI
• Make rare event workflows feasible: Detect up to 90% of rare events

Leverage the power of targeted detection. Access data that matter!