MIKAIA: build H&E analysis using multiple AIs

MIKAIA is specifically designed for researchers in academia, CROs, biotech, pharma. One of the challenges is that researchers by design do unique things. For this reason, MIKAIA Apps do not represent individual side-by-side analyses that stand in isolation, but instead they can be used in conjunction to build comprehensive workflows. The output of one App can serve as the input to another. Let’s see, how to build H&E analysis using multiple AIs.

A generic workflow for H&E analysis could look like this:

1. Automatic tissue (foreground) detection
2. Division of slide into Scan Areas to support multiple specimens per slide (e.g. TMA cores)
3. Further subdivision into ROIs (e.g., “metastasis”)
   •   …using custom AI created with AI Author
   •   … using dedicated AI such as Crypt Segmentation AI
   •   … by hand
   •   …concentric distance margins
4. Cell recognition AI + assign cells to ROIs
5. Spatial neighborhood analysis
6. Visualization using interactive markup and heatmaps
7. export to CSV for downstream statistical analysis
8. export of markup files, experiment settings, etc., for documentation and repeatability (GLP)

Here are 2 concrete examples for H&E analysis using multiple AIs

1. Comparison of cell populations in mucosal crypts vs. inter-crypt stroma

This workflows makes use of 3 Apps:

1. AI Author App: used to obtain a mask of the mucosa
2. H&E Crypt AI: used to outline the individual crypts
3. H&E Cell AI: used to detect and classify cells

Storyline (with mm:ss timecodes):

• AI Author App
  • 0:00 – Create new AI model “My Colon AI”.
    Set up 4 classes: Mucosa, Tumor, Inflammation and Stroma
  • 0:13 – outline typical regions for each class that are used to train the new AI model.
    Since the AI Author uses “Few Shot Learning” AI technology inside, it already yields good results when only very few annotations are provided
  • 0:25 – click “Train”. The AI Author will recognize the new training annotations and analyze them. The t-SNE plot visualizes the feature space and indicates that the 4 tissue types are well separable
  • 0:37 – click analyze “FoV” to test the newly trained AI model on the currently visible field of view (FoV). Adjacent areas that have been assigned to the same class are automatically fused, so that now a single polygon that covers the mucosa is available.
  • skipped in the video for brevity: the remaining classification mistakes should now be tackled, either by drawing new training annotations and training again (possibly multiple cycles), or by using the class changer tool to manually re-classify the false areas.
• H&E Crypt AI App
  • 0:59 – select the Crypt AI App in the App center. Select the mucosa mask. Then start the analysis in the mucosa by clicking the analyze-“ROI” button.
• Add Margins feature
  • 1:25 – now the set of crypts is dilated to obtain a mask of the inter-crypt stroma. Select one of the crypts. Then open the “Add Margins…” dialog. Configure 1 outward margin and click “Add margins to all in the current class”.
• H&E Cell AI
  • 1:34 – select the Cell AI App in the App center. Configure that the “Crypt” and “Inter crypt stroma” classes should be treated as ROI classes, i.e. for each detected cell, test in which ROI it is located and assign it to a class “<ROI>: <cell type>”.
  • 2:06 – analysis is done. Cells/mm² are now available for each cell type and separately for the 2 ROIs “crypts” and “inter-crypt-stroma”.

2. Quantification of tumor cell composition and visualization of tumor-infiltrating-lymphocytes (TILs)

Storyline (with mm:ss timecodes):

• Colon Tissue Cartography App – mask tumor
  • (for other organs, alternatively the AI Author App can be used to train a tumor detection AI and obtain a mask of the tumor)
  • 0:00 – select Colon Tissue Cartography App from App center and run on current fielf of view by clicking the analyze-“FoV” button
  • 0:17 – analysis is finished. Adjacent areas that were assigned to the same tissue class and merged into large masks.
• H&E Cell AI* – cell analysis of tumor
  • 0:21 – select H&E Cell AI from App center. Select the tumor mask. Then start analyzing only the selected tumor area by clicking the analyze-“RoI” button.
  • 1:05 – cell analysis is finished. Now the tumor composition is available: abundance and density (cells/mm²) for the individual cell types*.
• Heatmap of TILs
  • 1:19 – the interactive Density Heatmap functionality is used to create a heatmap based on the local density (cells/mm²) of lymphocytes.
• Cell-Cell-Connections App
  • 1:33 – select Cell-Cell-Connections App in the App center. Select all annotation classes that contain annotation objects that represent cells and shall be connected to one another (algorithm: Delauney triangulation + outlier removal). The cell classes generated by the Cell AI App all have the “cell” tag, so it is convenient to select all these classes by using the “cell” tag button.
  • 1:50 results are available. Now, for instance, the abundance of connections between lymphocytes and neighboring tumor cells is available. The table shows for every combination of 2 cell types: average number of neighborhoods (“bystander analysis”), average distance (in µm) + standard deviation, percentage of this connection type (both per class and in total).

*: cell types recognized by H&E Cell AI: epithelial cells, dysplastic cells, lymphocytes, fibroblasts, eosinophiles, neutrophiles, plasma cells, nerve cells, macrophages. AI was trained mainly on colon sections and will be extended to other organs next. Release of H&E Cell AI planned for v1.7 (coming soon).

For more information about MIKAIA^®, check out the MIKAIA® university.