- From PyPI
pip install pathologyfoundationimport pathologyfoundation as pf
plip = pf.model_zoo("PLIP-ViT-B-32")
example_image = "https://pbs.twimg.com/media/F3puqTTW8AEnB2F?format=jpg&name=large"
image_embeddings = plip.embed_images(example_image, normalize=True)The image format can be:
- numpy array (W * H * 3)
- numpy arrays (N * W * H * 3)
- PIL Images (Image, or list of Images)
- local/http URL (string)
- local/http URLs (list of strings)
import pathologyfoundation as pf
plip = pf.model_zoo("PLIP-ViT-B-32")
example_text = "An image of colorectal adenocarcinoma."
text_embeddings = plip.embed_texts(example_text, normalize=True)The text format can be:
- String
- list of Strings
Step 1. Download example data CRC-VAL-HE-7K.
import pathologyfoundation as pf
from PIL import Image
import torch
import os
plip = pf.model_zoo("PLIP-ViT-B-32")
df = pf.dataset.load_data.load_example(dataset_name="CRC-VAL-HE-7K")
df_image_label = df[["image", "label"]].sample(45) # Subsample few images in this tutorial.
print(df_image_label)Step 2. Initialize a classifier with PLIP model as backbone.
clf = pf.models.finetuner.FineTuner(backbone=plip.model,
preprocess=plip.preprocess,
num_classes=len(df_image_label["label"].unique()),
freeze_vit=False,
random_state=0
)Step 3. Start training the model.
clf.train(df_image_label,
validation_split=0.1,
batch_size=32,
num_workers=1,
lr=1e-5,
nepochs=10
)Optional 1: Save current fine-tuned model.
clf.save_model(os.path.join(pf.utils.get_default_cache_dir(), 'model_statedict_epoch.pth'))Optional 2: Predict a single image.
image = Image.open(df_image_label["image"].values[0])
proba = clf.predict(image)
predicted_class_name = df.loc[df['label'] == torch.argmax(proba).item(), 'class'].values[0]
print(f"Prediction: {predicted_class_name}")Optional 3: Generate image embedding.
image = Image.open(df_image_label["image"].values[0])
image_embeddings = clf.extract_embedding(image)