Ryoichi Koga

Ph. D. in Engineering

Assistant Professor at Department of Computer Science, Nagoya Institute of Technology

E-mail: koga [a t] nitech.ac.jp

Research Area: Medical Image Analysis, Pattern Recognition

Link: https://researchmap.jp/ryoichi-koga
Link: https://sites.google.com/view/pattern-analysis 

Under review.

Pathology foundation models are increasingly used as common feature extractors in whole slide image (WSI) analysis. Virchow2, a pathology foundation model, is pre-trained on a mixed-magnification image dataset, but it remains unclear how embeddings from different magnifications are organized in the shared representation space. In this paper, we characterize the multi-scale embedding geometry of Virchow2 using linear probing of input magnification, distributional distance analysis, low-dimensional visualization, and scale-shift vector analysis. The results consistently demonstrate that Virchow2 embeddings are not scale-invariant: input magnification can be accurately identified from embeddings, distributional differences increase with scale gap, and relationships among magnifications depend not only on distance but also on direction in the embedding space. In particular, the shifts from an intermediate magnification toward lower and higher magnifications exhibit largely independent directions. Based on this observation, we present a compact scale-plane interpretation around the intermediate reference magnification. This work clarifies the scale-dependent organization of Virchow2 embeddings learned under mixed magnification pretraining and provides empirical insights for designing multi-resolution WSI analysis pipelines.

We propose a framework that explicitly links attention in multiple instance learning (MIL) to a specific direction vector in the Virchow2 embedding space and trains an embedding-conditioned image generator to render traversals along this direction. Many conventional MIL pipelines use ImageNet-pretrained models (e.g., ResNet) for embeddings, and their attention is computed as the inner product between a weight vector and an embedding after a non-linear transformation. In contrast, our proposed framework extracts patch embeddings using Virchow2, a pathology foundation model, and then computes the inner product between the embedding vector h and the weight vector w to determine attention strength. Since our framework does not involve nonlinear transformations for embedding vectors, the learned direction w has a clear meaning as an attention-increasing direction in the Virchow2 embedding space. As a result, a linear traversal that increases attention can be represented in the embedding space for each patch. Our proposed framework trains a diffusion autoencoder conditioned on embeddings to render patch sequences along the linear traversal. These sequences provide a morphology-oriented visualization of changes associated with increasing attention.

Link: https://openreview.net/pdf?id=0clGvkoeoQ

We investigate whether modeling cellular context within transcriptomic spots improves cell-based prediction of spatial gene expression from histology images. Existing cell-based approaches independently map each cell embedding to a gene score and therefore do not account for relationships among nuclei within the same spot. We introduce a Contextual Cell Transformer that jointly processes the cell embeddings in each spot and averages the resulting context-dependent gene scores to predict spot-level expression. We compare the proposed predictor with non-contextual linear and multilayer perceptron predictors under identical cell embeddings, target genes, aggregation, and optimization settings. Experiments on the HER2ST and ccRCC datasets show that the proposed predictor achieves lower mean squared error and higher mean Pearson and Spearman correlations than both non-contextual cell-based predictors. It also achieves the lowest mean squared error and highest mean Pearson correlation among the compared patch- and cell-based methods on both datasets, although the highest Spearman correlation on each dataset is obtained by a patch-based baseline. These results provide a proof-of-concept that intra-spot cellular context can improve cell-based histology-to-spatial transcriptomics prediction.

Link: https://openreview.net/pdf?id=pePDlXMuJ9

We propose a criterion for grading follicular lymphoma that is consistent with the intuitive evaluation, which is conducted by experienced pathologists. A criterion for grading follicular lymphoma is defined by the World Health Organization (WHO) based on the number of centroblasts and centrocytes within the field of view. However, the WHO criterion is not often used in clinical practice because it is impractical for pathologists to visually identify the cell type of each cell and count the number of centroblasts and centrocytes. Hence, based on the widespread use of digital pathology, we make it practical to identify and count the cell type by using image processing and then construct a criterion for grading based on the number of cells. Here, the problem is that labeling the cell type is not easy even for experienced pathologists. To alleviate this problem, we build a new dataset for cell type classification, which contains the pathologists’ confusion records during labeling, and we construct the cell type classifier using complementary label learning from this dataset. Then we propose a criterion based on the composition ratio of cell types that is consistent with the pathologists’ grading. Our experiments demonstrate that the classifier can accurately identify cell types and the proposed criterion is more consistent with the pathologists’ grading than the current WHO criterion. 

Link: https://www.sciencedirect.com/science/article/pii/S0968432824000805
Link: https://www.jstage.jst.go.jp/article/mit/43/4/43_110/_article/-char/ja

We propose a method of attention induction to improve an attention mechanism in a whole slide image (WSI) classifier. Generally, only some regions in a WSI are useful for lesion classification, and the WSI classifier is required to find and focus on such regions for the classification. Multiple instance learning and hierarchical representation learning are widely employed for WSI processing and both use attention mechanisms to automatically find the useful regions and then conduct the class prediction. Here, it is impractical to collect a large number of WSIs, and when the attention mechanism is trained with a small number of training WSIs, the resultant attention often fails to focus on the useful regions. To improve the attention mechanism without increasing the number of training WSIs, we propose a method of attention induction for a hierarchical representation of WSI that guides attention to focus on the regions useful for lesion classification based on pathologists’ coarse annotations. Our experimental results demonstrate that the proposed method improves the attention mechanism, thereby enhancing the performance of WSI classification. 

Link: https://www.sciencedirect.com/science/article/pii/S215335392400052X

We propose a method that modifies encoding in DDIM (Denoising Diffusion Implicit Model) to improve the quality of counterfactual histopathological images of malignant lymphoma. Counterfactual medical images are widely employed for analyzing the changes in images accompanying disease. For the analysis of pathological images, it is desired to accurately represent the types of individual cells in the tissue. We employ DDIM because it can refer to exogenous variables in causal models and can generate counterfactual images. Here, one problem of DDIM is that it does not always generate accurate images due to approximations in the forward process. In this paper, we propose a method that reduces the errors in the encoded images obtained in the forward process. Since the computation in the backward process of DDIM does not include any approximation, the accurate encoding in the forward process can improve the accuracy of the image generation. Our proposed method improves the accuracy of encoding by explicitly referring to the given original image. Experiments demonstrate that our proposed method accurately reconstructs original images, including microstructures such as cell nuclei, and outperforms the conventional DDIM in several measures of image generation. 

Link: www.scitepress.org/Papers/2024/123661/123661.pdf

We propose a new method for cancer subtype classification from histopathological images, which can automatically detect tumor-specific features in a given whole slide image (WSI). The cancer subtype should be classified by referring to a WSI, i.e., a large-sized image (typically 40,000 × 40,000 pixels) of an entire pathological tissue slide, which consists of cancer and non-cancer portions. One difficulty arises from the high cost associated with annotating tumor regions in WSIs. Furthermore, both global and local image features must be extracted from the WSI by changing the magnifications of the image. In addition, the image features should be stably detected against the differences of staining conditions among the hospitals/specimens. In this paper, we develop a new CNN-based cancer subtype classification method by effectively combining multiple-instance, domain adversarial, and multi-scale learning frameworks in order to overcome these practical difficulties. When the proposed method was applied to malignant lymphoma subtype classifications of 196 cases collected from multiple hospitals, the classification performance was significantly better than the standard CNN or other conventional methods, and the accuracy compared favorably with that of standard pathologists 

Link: https://openaccess.thecvf.com/content_CVPR_2020/papers/CVPR_2020_paper.pdf