Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks
Yuansong Zeng, Zhuoyi Wei, Weijiang Yu, Rui Yin, Bingling Li, Zhonghui Tang, Yutong Lu, Yuedong Yang*
Here, we have developed Hist2ST, a deep learning-based model using histology images to predict RNA-seq expression. At each sequenced spot, the corre-sponding histology image is cropped into an image patch, from which 2D vision features are learned through convolutional operations. Meanwhile, the spatial relations with the whole image and neighbored patches are captured through Transformer and graph neural network modules, respectively. These learned features are then used to predict the gene expression by following the zero-inflated negative binomial (ZINB) distribution. To alleviate the impact by the small spatial transcriptomics data, a self-distillation mechanism is employed for efficient learning of the model. Hist2ST was tested on the HER2-positive breast cancer and the cutaneous squamous cell carcinoma datasets, and shown to outperform existing methods in terms of both gene expression prediction and following spatial region identification.
import torch
from HIST2ST import Hist2ST
model = Hist2ST(
depth1=2, depth2=8, depth3=4,
n_genes=785, learning_rate=1e-5,
kernel_size=5, patch_size=7, fig_size=112,
heads=16, channel=32, dropout=0.2,
zinb=0.25, nb=False,
bake=5, lamb=0.5,
policy='mean',
)
# patches: [N, 3, W, H]
# coordinates: [N, 2]
# adjacency: [N, N]
pred_expression = model(patches, coordinates,adjacency) # [N, n_genes]
Note: the detailed parameters instructions please see HIST2ST_train
Required package:
- PyTorch >= 1.10
- pytorch-lightning >= 1.4
- scanpy >= 1.8
- python >=3.7
- tensorboard
See tutorial.ipynb
NOTE: Run the following command if you want to run the script tutorial.ipynb
- Please run the script
download.sh
in the folder data
or
Run the command line git clone https://github.com/almaan/her2st.git
in the dir data
- Run
gunzip *.gz
in the dirHist2ST/data/her2st/data/ST-cnts/
to unzip the gz files
- human HER2-positive breast tumor ST data https://github.com/almaan/her2st/.
- human cutaneous squamous cell carcinoma 10x Visium data (GSE144240).
- you can also download all datasets from here
All Trained models of our method on HER2+ and cSCC datasets can be found at synapse
Please cite our paper:
@article{zengys,
title={Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks},
author={ Yuansong Zeng, Zhuoyi Wei, Weijiang Yu, Rui Yin, Bingling Li, Zhonghui Tang, Yutong Lu, Yuedong Yang},
journal={biorxiv},
year={2021}
publisher={Cold Spring Harbor Laboratory}
}