Video 1: Intracranial Vascular Aneurysm seen under a GLOW800 filter.
The use of NIR fluorescence with ICG, such as the FL800 module from Leica Microsystems, has aided surgeons in vascular neurosurgery to perform countless life-saving operations. But, due to the different nature of the white light reflectance and fluorescence imaging modes, each mode requires different optical filters for illumination and imaging. The selection of an imaging mode triggers an automatic exchange to the appropriate filters in the microscope. Therefore, the different imaging modes are only offered in a sequential manner.
The surgeon is required to separately select and view each imaging mode in turn, observe and note the useful features observed with each mode, and then mentally merge all this information in order to generate a useable ‘picture’ about the observed tissue and blood flow. This activity has to be done before carrying out the intervention, such as clipping an aneurysm.
Videos 2 and 3: Pre-clipping images from an aneurysm clipping operation showing what the surgeon has to mentally merge (FL800 NIR fluorescence and white light) in order to carry out the surgical intervention.
This task creates the problem of overburdening the surgeon’s time and effort, while some details from the fluorescence image might be overlooked due to difficulty in remembering all the spatial features of the black and white NIR fluorescence image. This challenge continues even after the clip is placed.
Videos 4 and 5: Post-clipping images from an aneurysm clipping operation showing what the surgeon has to mentally merge (FL800 NIR fluorescence and white light) to assess the surgical intervention.
The challenge of using NIR fluorescence is further compounded because fluorescence only produces a weak signal and its measurement is strongly affected by a microscope’s working distance and magnification levels. Surgeons can only compare the fluorescence intensity within an image, but not between different positions or surgeries.
With the GLOW800 technology, Leica Microsystems has set out to overcome this very significant challenge that occurs when operating with surgical microscopes.
GLOW800, the first product available from the GLOW AR platform, is an Augmented Reality module that overcomes the above mentioned limitations for an improved operating experience. It combines the NIR fluorescence signal of Indocyanine Green (ICG) with traditional white light illumination into a single view. This capability allows the surgeon to visualize the information of both ICG fluorescence and white light in a single image at the same time – he or she can assess vessel flow and anatomy all in full color.
GLOW AR platform employs a multispectral imaging technology which allows the capture of multiple images simultaneously at different spectral bands, in reflectance or fluorescence mode. This technology is using a spectral multiplexing technique for both the white light and fluorescence illumination. This allows illumination of the tissue with a precise mixture of white and fluorescence excitation light, while the multispectral imaging sensor utilizes appropriate filtering to eliminate any potential interference between different spectral bands.
Specifically, for the GLOW800 module, the white light reflectance imaging illumination consists of visible light with a wavelength of circa 400-700 nm and the ICG fluorescence excitation illumination of near infrared light, circa 700-790 nm. All light above 790 nm is filtered out to avoid interference with the fluorescence emission signal. The GLOW800 multispectral sensor employs filters to allow independent measurements in four spectral bands: red, green and blue for the reconstruction of the white light reflectance image and NIR emission at 835-880 nm for the visualization of ICG fluorescence. The GLOW AR software combines the visible light and non-visible NIR video streams. The user sees a high definition video of the surgical site displaying both the emitted fluorescence signal and anatomical information.
The spectral graph (figure 1) below illustrates the light wavelengths which are and are not visible to the human eye. By exciting the fluorophores in the non-visible range, they become ‘visible’ to the human eye through the GLOW800 filter.
|Fluorescence excitation|| |
The GLOW800 module uses a video processing unit (VPU) to acquire, process and display the data of two video streams from cameras embedded in the microscope optical head. This image processing provides the key features described below.
The standard image of NIR fluorescence is only black and white, but a pseudo (false) color (figure 2 and table 2 below) can be chosen by the end user, based on personal preference, for observation of the emission fluorescence signal. The color image from white light illumination is constructed using a 'debayering' (or demosaicing) process which interpolates color values from the filter array output of the GLOW800 sensor chip.
To overcome inhomogeneity of the luminance distribution delivered by the surgical microscope system, a homogenization filter is applied to each incoming frame which normalizes the measured luminance by applying pixel-wise division based on the microscope settings (working distance, magnification, iris, illumination) as seen in Figure 6 below. The system applies a compensation factor proportional to each image. With homogenization, the effect of center-weighted illumination can be compensated. The homogenization values range from 0% (none) to 100% (maximum).
In addition, a rotational, translational and scaling transformation is applied to the fluorescence images before homogenization to compensate for any differences in the beam paths from the surgical site to the two sensor chips. Bilinear interpolation is applied at run time to derive the fluorescence intensity at non-integer origin co-ordinates from surrounding pixels.
Brightness of acquired images decreases with growing working distance and magnification, as well as with decreasing light intensity. To compensate for this decrease, a scale factor is applied to the fluorescence image.
The intensity adjusts the fluorescence contrast, brightness and transparency in relation to the object details. The intensity values range from 0% (fluorescence just visible/object is dominant) to 100% (fluorescence intensively visible and dominant).
The GLOW800 module enables the user to utilize NIR fluorescence to assess blood flow in full multispectral color, using the M530 surgical microscopes from Leica Microsystems. When activating the GLOW800 mode, the white light illumination of the Leica microscope is extended to near infrared to excite the fluorophore (ICG). The filtered NIR fluorescence signal of the fluorophore (ICG) is acquired by a NIR sensitive video camera and processed in the GLOW800 VPU.
Video 6: The same aneurysm from video 1 after it has been clipped. The color highlighted blood flow and anatomy allows the surgeon to better assess the aneurysm clip and blood flow post clipping.
GLOW800 offers two different observation modes: the white light object view with the embedded fluorescence signal in pseudo color or black & white (B&W) NIR fluorescence view. In both views the fluorescence video signal can be observed on an attached Leica surgical microscope video monitor or through the microscope oculars (this requires the CaptiView image injection from Leica Microsystems, which is sold separately).
The GLOW800 module uses augmented reality fluorescence algorithms to achieve a high level of synchronization that ensures the color is natural and fully integrated into the image, irrespective of which viewing source is used – 2D screen, 3D1 screen or the microscope oculars (requires the CaptiView module). These algorithms also ensure there are no asynchronous movements or delays.
These features allow the GLOW800 module to serve as a valuable tool for surgeons so they can simultaneously visualize blood flow and anatomy.
Videos 7-9: The aneurysm before it is clipped.
Videos 10-12: The aneurysm after it has been clipped.
GLOW AR is an augmented reality platform that forms the foundation of the GLOW800 AR fluorescence module. It offers a new surgical imaging experience by acquiring both imaging modes, white light and fluorescence, simultaneously, enabling visualization of all the workflow-related information in a single augmented reality view.
In the absence of any fluorescence signal, the tissue is visualized in its natural white light appearance, and, if fluorescence occurs, it can be visualized via a vivid color which contrasts with the natural tissue tones. Special color-blending algorithms ensure faithful presentation of the tissue anatomy combined with accurate visualization of fluorescence intensity using an intuitive tissue-glowing effect. Multiple fluorescence signals can be visualized simultaneously using different colors, without the need to switch modes or any other user intervention.
Moreover, the GLOW AR platform provides accurate visualization of fluorescence intensity, independent of working distance, magnification, and inhomogeneities in the illumination and imaging optics. This accuracy allows surgeons to compare fluorescence intensities between different imaging positions, operations and even patients.
The GLOW800 module is also GLOW400-ready and, pending appropriate regulatory clearance, can be upgraded to also provide GLOW400 capabilities. Generic features of GLOW AR, such as correction of inhomogeneities and normalization of fluorescence intensity, are included.
Other GLOW AR platform technologies are currently in development and will be released in due course2.
Learn about the neurosurgical treatment of spinal arterio-venous fistulas, including classification,…Nov 23, 2022Read article
Surgery of skull base tumors and diseases, such as cavernomas, epidermoid cysts, meningiomas and…Oct 10, 2022Read article
The neurosurgical operating room is increasingly digital, with multiple sources of data providing…Sep 19, 2022Read article