Segmentations from zarr are not visualised in SPATIAL view

[Artur-man]

Describe the bug:

I have tried to replicate the Visualization of a Loom file tutorial using VitessceR on a custom zarr file with a Visium dataset

1. Although the Python version (conda env with Vitessce) does work, the R wrapper
   does not seem to behave the same way.
2. It seems CoordinationType$SPATIAL_SEGMENTATION_LAYER is not
   defined in vitessceR although Python wrapper has it.
3. "select lasso" seems to work on the R widget as I can see that
   some entities are selected on the umap on the right as I select
   arbitrary regions.

NOTE: I can also arrange a link to send around the zarr file if needed.

Environment:

• Release or git hash: 3177e17
• Operating system: MacOS Monterey v12.5.1
• R version: 4.3.1
• RStudio version: 2023.06.1+524

To Reproduce and Screenshots:

Here is the R script where no segmentations are visualised on the viewer pane:

devtools::install_github("vitessce/vitessceR")
library(vitessceR)

vc <- VitessceConfig$new(schema_version = "1.0.15", 
                         name='MBrain Example', 
                         description='mBrain')
w <- AnnDataWrapper$new(
  adata_path="mbrain.zarr/",
  obs_set_paths = c("obs/clusters"),
  obs_set_names = c("Clusters"),
  obs_locations_path = "obsm/spatial",
  obs_segmentations_path="obsm/segmentation",
  obs_embedding_paths=c("obsm/umap")
)
dataset <- vc$add_dataset("MBrain")$add_object(w)
umap <- vc$add_view(dataset = dataset, Component$SCATTERPLOT, mapping = "umap")
cell_sets <- vc$add_view(dataset, Component$OBS_SETS)
spatial <- vc$add_view(dataset, Component$SPATIAL)

spatial_segmentation_layer_value <- list(opacity = 1, radius = 0, visible = TRUE, stroked = FALSE)

vc$link_views(views = c(spatial),
              c_types = c(CoordinationType$SPATIAL_ZOOM, CoordinationType$SPATIAL_TARGET_X, CoordinationType$SPATIAL_TARGET_Y, "spatialSegmentationLayer"),
              c_values = c(0, 300.00, 300.00, spatial_segmentation_layer_value))

vc$layout(
  hconcat(spatial, vconcat(umap, cell_sets))
)
vc$widget()

Here is the Python equivalent (let me know if this is not accurate).

# import modules
import os
import zarr
from os.path import join, isfile, isdir
from urllib.request import urlretrieve
from anndata import read_loom
import numpy as np

from vitessce import (
    VitessceConfig,
    Component as cm,
    CoordinationType as ct,
    AnnDataWrapper,
)
from vitessce.data_utils import (
    optimize_adata,
    to_diamond,
    VAR_CHUNK_SIZE,
)

# import zarrdata 
zarrdata = zarr.open("data/mbrain.zarr/", mode = "r+")
zarrdata.tree()

Lets check if segmentations are here.

# check segmentations data
zarrdata["obsm/segmentation"][:]

array([[[438.89847, 216.07916],
        [440.89847, 214.07916],
        [438.89847, 212.07916],
        [436.89847, 214.07916]],

       [[143.9587 , 160.41652],
        [145.9587 , 158.41652],
        [143.9587 , 156.41652],
        [141.9587 , 158.41652]],

       [[410.49915, 438.67813],
        [412.49915, 436.67813],
        [410.49915, 434.67813],
        [408.49915, 436.67813]],

       ...,

       [[350.08606, 333.54904],
        [352.08606, 331.54904],
        [350.08606, 329.54904],
        [348.08606, 331.54904]],

       [[225.69708, 166.5611 ],
        [227.69708, 164.5611 ],
        [225.69708, 162.5611 ],
        [223.69708, 164.5611 ]],

       [[172.40964, 246.95697],
        [174.40964, 244.95697],
        [172.40964, 242.95697],
        [170.40964, 244.95697]]], dtype=float32)

Attempting widget on Python wrapper:

vc = VitessceConfig(schema_version="1.0.15", 
                    name='MBrain Example', 
                    description='mBrain')
w = AnnDataWrapper(adata_path="mbrain.zarr/", 
                   obs_set_paths=["obs/clusters"], 
                   obs_set_names=["Clusters"], 
                   obs_locations_path="obsm/spatial", 
                   obs_segmentations_path="obsm/segmentation",
                   obs_embedding_paths=["obsm/umap"])
dataset = vc.add_dataset(name='MBrain').add_object(w)
umap = vc.add_view(cm.SCATTERPLOT, dataset=dataset, mapping="umap")
cell_sets = vc.add_view(cm.OBS_SETS, dataset=dataset)
spatial = vc.add_view(cm.SPATIAL, dataset=dataset)

spatial_segmentation_layer_value = {
  "opacity": 1,
  "radius": 0,
  "visible": True,
  "stroked": False
}

vc.link_views([spatial], 
              [ct.SPATIAL_ZOOM, ct.SPATIAL_TARGET_X, ct.SPATIAL_TARGET_Y, ct.SPATIAL_SEGMENTATION_LAYER], 
              [0, 300.00, 300.00, spatial_segmentation_layer_value])

vc.layout(spatial | (umap / cell_sets));
vw = vc.widget()
vw

[ashishjain1988]

Hi,

I am also facing the same issue as I am not able to see the segmentations on shiny app using the VitessceR package but able to see it using the Vitessce package for the same data. I used the osmFISH_SScortex_mouse_all_cells.loom data from https://loom.linnarssonlab.org/ as suggested/shown in the Vitessce package tutorial.

Regards,
Ashish Jain

[Artur-man]

I forked and did a quick update to the version of the Vitessce js pulled from https://unpkg.com/, specifically from https://unpkg.com/[email protected], hence switching from 3.2.1 to 3.4.6.

See here: https://github.com/Artur-man/vitessceR/blob/72537c0cb64afeb27ed31a1279e2447a71ff676b/inst/htmlwidgets/vitessceR.js#L36

Now it works!