By Dr Maria Ada Prusicki
Advances in whole-slide scanning technology enabled the automation of microscopy. Researchers can run image acquisition overnight and check the results remotely, minimizing the need for supervision and optimizing their use of time. Nonetheless, a few steps still require intense human supervision, especially for sample identification and scan area definition, where classic algorithms do not suffice in covering the intrinsic variability of biological samples. Deep learning approaches tackle this issue, enabling accurate sample detection and leading to sample specific solutions.
Whole-slide imaging (WSI) has been firmly established as a practical microscopy method in recent years in many scientific fields – including clinical research, drug development, pathology and neurobiology, among others – demonstrated by these two studies in cancer research by Song et al.  and Burn et al. .
The core concept of WSI is the acquisition of large scan areas – ideally the whole slide – in a digital format. The main advantages of WSI include enabling researchers to easily visualize, analyse, share and, not least of all, save the digitized information of unique biological samples without needing to physically store large volumes of material or, for example, send slides overseas to a collaborators.
Slide scanners are now fully integrated into the daily workflow of biology branches where microscopy has always been the main methodology, such as the previously mentioned pathology. Moreover, they are also entering biological fields that were traditionally based on different techniques and which are now shifting toward a more visual approach, such as spatial genomics and transcriptomics.
As the use of slide scanners spreads, the interest in automating the WSI workflow increases. Automation offers the potential to not only increase throughput, but also optimize time management. The development of new technologies in this direction frees the scientific staff from cumbersome and repetitive tasks, enabling them to focus on results analysis and higher-end activities. Moreover, automation of a scientific workflow is reflected in higher precision and reproducibility, which are pillars of scientific research.
Microscopy workflow steps and advances in automation
The workflow of a microscopist can be largely summarized as:
• position the slides on the microscope stage;
• find the specimen;
• set up the acquisition (i.e. the observation method, e.g. transmitted light path, magnification, exposure time);
• select the correct focal plane;
• acquire the image;
• save the data;
• process the files; and
• analyse/annotate/share the results.
On the one hand, some of these steps have been successfully automated with new mechanical technology and software controls. For example, slides are now loaded autonomously on the microscope stage. This is achieved with robotic arms handling trays or slide cassettes. The image acquisition step can also be performed somewhat automatically, as imaging settings for a group of slides can be adjusted in advance and applied later, allowing the running of unsupervised overnight experiments. Moreover, images can be saved and uploaded on databases automatically.
On the other hand, the automation of tasks such as sample detection and focal plane adjustment is lagging behind, and these steps are still strongly dependent on human supervision. A possible solution for the focal plane issue, as well as limitations, have been extensively discussed . Here, though, we will focus on the issue of sample detection.
A classic algorithm for sample detection separates the specimen from its background based on pixel intensity and colour saturation. The algorithm identifies a potential sample among the pixels that are neither too bright (background) nor too dark (dust/coverslip borders). In addition, filters on the sample size and colour can be used to refine the detection.
This conventional sample detection method is used in many slide scanners, and it works very accurately for well-stained samples with high-contrast images, such as samples prepared for brightfield microscopy and stained with preparations such as H&E (Fig. 1a) and immunostaining mediated by horseradish peroxidase (Fig. 1b) or alkaline phosphatase.
Unfortunately, biological samples present intrinsic variability. They have different morphologies and thicknesses, and the stain does not penetrate every tissue in a uniform way, leading to irregularly coloured or faint samples. Moreover, specimens stained with fluorescent labels are often difficult to visualize when observed with a transmitted light path. All these factors can influence the efficacy of the sample detection algorithm, leading to false results. Figure 1 shows an example of a detection algorithm that performs well on a highly contrasted specimen (Fig. 1a). The same algorithm applied to a sample with non-uniform staining and varying transparency levels results in an inaccurate sample mask that fails to detect faint areas of the tissue (Fig. 1b).
When automatic sample detection fails, there are a few possible paths to follow, including the following two options that have inherent impractical implications.
• Enlarge the scan area, choosing, for example, to include the complete slide in the scan. Consequently, the image acquisition will require more time and the data will include more noninformative pixels. Bigger file sizes will increase the complexity of downstream steps, such as image processing and analysis. Moreover, this might increase the risk of acquiring misfocused images, as the system will search for an in-focus specimen where there is none.
• Forego automation and adjust the scan area manually.