Do hlubin digitální patologie - Jakub Kantner (MSD) [seminář MPN 14.10.2026]

Summary

Jakub Kantner presents an internal MSD tool for automating the annotation of digitized mouse-brain tissue images. This work supports research into how diseases or experimental treatments affect brain tissue. The central problem is that manually locating brain regions and tracing them on slides is slow, repetitive, and difficult to scale.

Why automate pathology annotation?

• Digital pathology involves digitizing tissue images so they can be examined and analyzed on a computer.
• Scientists annotate images by outlining specific brain regions, then compare their size or shape across samples.
• A trained scientist may spend 45–90 minutes annotating one slide. A study can involve many animals and multiple slides per animal, making manual work a substantial bottleneck.
• The tool transfers region annotations from a reference mouse-brain atlas to experimental images. Because tissue can be distorted or damaged, the process must both identify the right atlas section and align it to the sample.

The annotation workflow

The system has four main stages:

1. Detect and crop individual brains

   • Brain sections appear among other material on a slide and may not be perfectly aligned; some may be missing.
   • The system uses Mask R-CNN for object detection and instance segmentation, identifying each brain and separating it from the rest of the image.
   • The model was fine-tuned using about 40 images. The reported detection accuracy was 98%.

2. Select the corresponding atlas section

   • The reference is the Allen Mouse Brain Atlas, which contains annotated brain sections.
   • An EfficientNet-B0 classifier selects the most likely section from a set of 50 coronal atlas sections.
   • The model uses visual features of the cropped brain to score the candidate sections.
   • Exact section selection was correct in 59% of cases. However, it was within one section in 86% of cases and within two sections in 94%. The speaker notes that neighboring sections are very thin and similar, so near-matches can still be useful.

3. Assign labels to the cropped images

   • Once the brains are separated from the original slide, their positions and the original layout may no longer be obvious.
   • The software uses an experiment-specific or standard table of labels, arranged to correspond to the slide layout.
   • It matches detected images to expected positions using the Hungarian algorithm, which finds the assignment with the lowest total matching cost. In the example shown, the cost is based on distances between image detections and label positions.
   • This lets the system attach the appropriate labels automatically, including when some positions are missing.

4. Register the atlas to each brain

   • First, an affine transformation moves, rotates, and scales the atlas image to roughly match the sample.
   • Then a local elastic deformation refines the fit so the atlas follows the brain’s shape.
   • The local transformation uses diffomorphic B-splines: control points on a mesh are moved, and the image is smoothly adjusted between them. The goal is to avoid folds or implausible distortions.
   • Registration can use manually selected landmarks or image-based matching. The described system uses normalized cross-correlation to compare image patterns, along with a penalty for excessive deformation.
   • Parameters are iteratively optimized, for example using gradient descent or the Adam optimizer, until the mismatch is minimized.

Results and practical use

• The system reduces the time spent on manual annotation and is designed to fit into the existing research workflow.
• In the reported results:
  • 66% of cases needed no further human intervention.
  • 15% needed a person to make corrections to an existing, approximate annotation.
  • 9% were substantially misaligned and required manual work from the beginning.
  • 10% were unsuitable images—for example, because tissue was missing, torn, or poorly stained—and would generally be excluded from later analysis.
• The tool is semi-automatic, not fully unattended. Scientists still review the output and correct or reject problematic cases.
• It was deployed on MSD’s high-performance computing infrastructure and made available through an API and a graphical interface, with usability for non-IT researchers as an explicit consideration.

Other topics discussed

• Kantner briefly describes his work background, including a move from data management and visualization into computer vision and AI research. He says his team focuses in part on AI safety and evaluating models for medical applications.
• In the Q&A, he explains that the classifier learns from labeled examples rather than relying on a separately defined similarity measure. He also discusses the possibility of using external annotated datasets for further training.
• He says the broader research project is ongoing and connected to work such as spatial transcriptomics, which can help researchers examine gene expression around disease-related brain regions.
• He mentions other examples of applied research at MSD, including monitoring chicken behavior and detecting parasites on salmon.
• He also outlines student opportunities and a graduate rotation program at MSD, and comments that interest, potential, and teamwork matter alongside technical skills.

Speakers and sources featured

Speakers

• Jakub Kantner — MSD AI researcher and main presenter; also answers audience questions.
• Seminar host/moderator — introduces the speaker and closes the event; not identified by name in the subtitles.
• Audience members — ask questions during the discussion; not identified by name.

Named sources, tools, and methods mentioned

• Allen Mouse Brain Atlas
• Mask R-CNN
• EfficientNet-B0
• Hungarian algorithm
• Normalized cross-correlation (NCC)
• Diffomorphic B-splines
• Gradient descent and Adam optimizer
• Halo — software mentioned as an example of a tool used for manual image annotation.