Prov-gigapth Tutorial¶
Please down load github repsitory https://github.com/prov-gigapath/prov-gigapath
Please fowllow the github Install guide
git clone https://github.com/prov-gigapath/prov-gigapath
cd prov-gigapath
conda env create -f environment.yaml
conda activate gigapath
pip install -e .
Introduction¶
Prepocessing WSI¶
tile imges format¶
- input:
px: 256x256 position: 123x_123y.png
RGB
- output:
Array 1 x 1536 dims [1,2.....1536] position 1 x 2 dims [x, y]
- save as:
.pt for image embedding .csv for labeling, slide_id, image_position
slide imges format¶
- input:
tile embedding tile position embedding
- output:
images embedding 1 x 768 dims
- save as:
.h5 for whole slide image embedding .csv for labeling, slide_id,
File Location¶
1. tile encoder¶
prov-gigapath/gigapth
prov-gigapath/gigapth/pipepline.py
1.1 tile encoder_finetune¶
prov-gigapath/linear_probe/main.py > tile level fine-tuning for 2 labels | Dataset pcam/pcam.csv
2. slide encoder¶
slide_encoder.py & pos_embed.py > how to embed (x,y) position for tiles and embed at slide level.
2.1 slide encoder_finetune¶
prov-gigapath/finetune & prov-gigapath/gigapth/classification_head.py > slide level fine-tuning for 5 labels | Dataset PANDA
Important
Please set up Nvidia driver
nvidia-smi to chekc if you have or not.
Demo¶
There is demo located in /demo/ as well
1. download models¶
1.1 download tile encoder¶
In [1]:
import timm
tile_encoder = timm.create_model("hf_hub:prov-gigapath/prov-gigapath", pretrained=True)
1.2 check tile encoder¶
In [ ]:
from torchsummary import summary
summary(tile_encoder)
In [ ]:
2.1 donlowad slide encoder¶
In [1]:
## this is downloading from huggingface, please obtain your personal token. https://huggingface.co/docs/hub/en/security-tokens
import os
os.environ["HF_TOKEN"] = ""
In [ ]:
# put gigapth folder with .ipynb at same directory.
import gigapath.slide_encoder as encoder
In [ ]:
## download slide encoding model
## THere are 3 models (please check slide_encoder.py Line 254 to 269
# 1. enc12l768d --> embed_dim = 768, depth = 12, mlp_ratio = 4.
slide_encoder = encoder.create_model("hf_hub:prov-gigapath/prov-gigapath", "gigapath_slide_enc12l768d", 1536)
2.2 check slide encoder¶
In [ ]:
summary(slide_encoder)
In [ ]:
2. Embedding WSI¶
2.0 Preprocessing Data | reference (https://www.youtube.com/watch?v=QntLBvUZR5c)¶
In [1]:
from openslide import open_slide
from PIL import Image
import numpy as np
from matplotlib import pyplot as plt
In [2]:
img = "./test_ground/000026.ndpi"
In [3]:
slide = open_slide(img)
In [4]:
## show image as thumb
slide_thumb = slide.get_thumbnail(size=(600,600)) ## resize to 600x600 px
print(type(slide_thumb))
print(slide_thumb.size) ## width x height
slide_thumb.show()
<class 'PIL.Image.Image'> (600, 461)
In [5]:
## convert to numpy to show
slide_num = np.array(slide_thumb)
print(slide_num.shape) ## height x width x channels
(461, 600, 3)
In [6]:
plt.figure(figsize=(8,8))
plt.imshow(slide_num)
Out[6]:
<matplotlib.image.AxesImage at 0x74c3298471f0>
In [7]:
## different level
factor = slide.level_downsamples
level_dim = slide.level_dimensions
level = 0
print(f"Level Image Size Scaling Factor")
for i in level_dim:
print(f"{level}\t{i}\t{factor[level]}")
level+=1
Level Image Size Scaling Factor 0 (96000, 73728) 1.0 1 (48000, 36864) 2.0 2 (24000, 18432) 4.0 3 (12000, 9216) 8.0 4 (6000, 4608) 16.0 5 (3000, 2304) 32.0 6 (1500, 1152) 64.0 7 (750, 576) 128.0 8 (375, 288) 256.0
In [8]:
## different level;
level3_dim = level_dim[2]
print(level3_dim)
## load level 3 image
level3_img =slide.read_region((0,0), 2, level_dim[2])
level3_rgb = level3_img.convert('RGB')
(24000, 18432)
In [9]:
level3_np = np.array(level3_rgb)
print(level3_np.shape)
(18432, 24000, 3)
In [10]:
plt.imshow(level3_np)
Out[10]:
<matplotlib.image.AxesImage at 0x74c328ba9940>
In [ ]:
In [11]:
### tile processing
from openslide.deepzoom import DeepZoomGenerator
In [12]:
## segement as 256 x 256 px
tiles = DeepZoomGenerator(slide, tile_size=256, overlap=0, limit_bounds=False)
In [13]:
## check number of level
layer = 0
for i in tiles.level_dimensions:
print(f"{layer} layer: {i} ")
layer+=1
print(f"Total tiles for this WSI: {tiles.tile_count }")
0 layer: (1, 1) 1 layer: (2, 2) 2 layer: (3, 3) 3 layer: (6, 5) 4 layer: (12, 9) 5 layer: (24, 18) 6 layer: (47, 36) 7 layer: (94, 72) 8 layer: (188, 144) 9 layer: (375, 288) 10 layer: (750, 576) 11 layer: (1500, 1152) 12 layer: (3000, 2304) 13 layer: (6000, 4608) 14 layer: (12000, 9216) 15 layer: (24000, 18432) 16 layer: (48000, 36864) 17 layer: (96000, 73728) Total tiles for this WSI: 144124
In [14]:
## different tiles and different level
level_num = 16
print(f"tile shape at level {level_num} is: {tiles.level_tiles[level_num]}")
print(f"tiles : {tiles.level_tiles[level_num][0] * tiles.level_tiles[level_num][1]}")
tile shape at level 16 is: (188, 144) tiles : 27072
In [ ]:
In [ ]:
### save a whole slide image into tiles
In [ ]:
cols, rows = tiles.level_tiles[16] ## 27k tiles
tile_dir = "./test_groundtiles_practice/"
In [ ]:
## loop through
for row in range(rows):
for col in range(cols):
tile_name = os.path.join(tile_dir, '%d_%d' %(row,col))
temp_tile = tiles.get_tile(16,(col,row))
temp_rgb = temp_tile.convert('RGB') ## rgba > rgb
temp_np = np.array(temp_rgb)
plt.imsave(tile_name+".png", temp_np)
print(f"finish saving {row}")
In [ ]:
In [1]:
## reference to demo/run_gigapath.ipynb
# please download this pre-processing data [https://hanoverprod.z21.web.core.windows.net/gigapath/PANDA_sample_tiles.zip]
2.1 Prepare Data (using example data from github)¶
In [4]:
## prepare data
slide_dir = "PANDA_sample_tiles/054b6888604d963455bfff551518ece5/"
images =[os.path.join(slide_dir, img) for img in os.listdir(slide_dir) if img.endswith('.png')]
print(f"Found {len(images)} tiles")
Found 810 tiles
2.2 Tile embedding¶
In [14]:
## Encoding Part
## tile encoding with image and store coord from preprocessing
from gigapath.pipeline import run_inference_with_tile_encoder
tile_encoder_outputs = run_inference_with_tile_encoder(images, tile_encoder) ## batch size = 128
Running inference with tile encoder: 100%|███████████████████████████████████████████████████████████████| 7/7 [00:08<00:00, 1.19s/it]
In [19]:
## return dic structure
print(tile_encoder_outputs.keys())
dict_keys(['tile_embeds', 'coords'])
In [17]:
for k in tile_encoder_outputs.keys():
print(f"tile_encoder_outputs[{k}].shape: {tile_encoder_outputs[k].shape}")
tile_encoder_outputs[tile_embeds].shape: torch.Size([810, 1536]) tile_encoder_outputs[coords].shape: torch.Size([810, 2])
In [24]:
print(tile_encoder_outputs['tile_embeds'][0]) # image
print(tile_encoder_outputs['coords'][0]) # position
tensor([-0.8817, -0.9265, -0.7653, ..., -0.3008, -0.4806, -0.5998]) tensor([ 6040., 26952.])
2.3 slide embedding¶
In [ ]:
## slide encoding with position
In [28]:
from gigapath.pipeline import run_inference_with_slide_encoder
In [29]:
slide_embeds = run_inference_with_slide_encoder(slide_encoder_model=slide_encoder, **tile_encoder_outputs)
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 13.69it/s]
In [30]:
print(slide_embeds.keys())
dict_keys(['layer_0_embed', 'layer_1_embed', 'layer_2_embed', 'layer_3_embed', 'layer_4_embed', 'layer_5_embed', 'layer_6_embed', 'layer_7_embed', 'layer_8_embed', 'layer_9_embed', 'layer_10_embed', 'layer_11_embed', 'layer_12_embed', 'last_layer_embed'])
In [32]:
for i in slide_embeds.keys():
print(f"{i} .Shape=> {slide_embeds[i].shape}")
layer_0_embed .Shape=> torch.Size([1, 768]) layer_1_embed .Shape=> torch.Size([1, 768]) layer_2_embed .Shape=> torch.Size([1, 768]) layer_3_embed .Shape=> torch.Size([1, 768]) layer_4_embed .Shape=> torch.Size([1, 768]) layer_5_embed .Shape=> torch.Size([1, 768]) layer_6_embed .Shape=> torch.Size([1, 768]) layer_7_embed .Shape=> torch.Size([1, 768]) layer_8_embed .Shape=> torch.Size([1, 768]) layer_9_embed .Shape=> torch.Size([1, 768]) layer_10_embed .Shape=> torch.Size([1, 768]) layer_11_embed .Shape=> torch.Size([1, 768]) layer_12_embed .Shape=> torch.Size([1, 768]) last_layer_embed .Shape=> torch.Size([1, 768])
In [34]:
## check layer 1 and layer 12
print(slide_embeds['layer_0_embed'])
print('-------------')
print(slide_embeds['last_layer_embed'])
tensor([[-4.5210e-01, 1.7855e+00, 2.5990e-01, -9.2623e-02, -7.9379e-02,
-4.9030e-01, 5.2150e-01, 2.0427e-02, -8.7831e-01, -1.0923e+00,
9.1779e-01, 1.9031e+00, -1.8882e-01, 4.0819e+00, -9.9635e-01,
6.6815e-01, 9.3187e-01, 4.6784e-01, 6.5349e-01, 4.8378e-01,
-8.1687e-02, 5.9407e-02, 3.2209e+00, -3.5371e-01, -1.0952e+00,
7.4126e-01, -1.5051e-01, -1.1173e+00, -3.1420e-01, -8.2539e-01,
-4.4671e-02, 3.6703e-01, 1.6183e-01, 1.8128e-01, 1.3408e-01,
1.5553e+00, -2.4995e+00, -3.0818e-01, 3.6680e-01, 7.0053e-01,
-2.4750e-01, -2.1626e-01, -4.1587e-01, -4.6226e-01, -2.6574e-01,
3.5805e-01, 1.5961e-01, 2.3117e-01, -1.8021e-01, -1.3967e-02,
-2.4106e-01, -5.3606e-01, -2.0481e-01, -4.8502e-01, -1.9846e-01,
2.6415e-01, 6.1616e-02, -1.9794e-01, 1.9629e-01, 2.4304e-02,
-1.2413e-02, 1.7330e-01, 2.2374e-01, 1.9147e-02, -6.4836e-01,
2.2571e-01, 2.3290e-02, 4.8940e-01, 5.0804e-01, -1.8679e-01,
-3.3605e-01, 9.7664e-01, -1.8251e-01, 6.2380e-01, 2.2746e-01,
-1.2193e-01, 4.1913e-01, 1.5899e-01, -3.1110e-01, -4.3871e-01,
-1.9955e-01, -4.2973e-01, -5.6255e-01, 4.7870e-01, 2.5511e-01,
1.6640e-01, -3.5910e-01, -5.2204e-01, 1.5549e-01, -1.1235e+00,
-3.5336e-01, -9.4648e-01, 6.5775e-02, -5.2909e-01, 4.8724e-02,
-4.2012e-01, 1.7835e+00, -1.0192e-01, -4.4476e-01, 1.6462e-01,
-6.9707e-01, 1.7902e-01, 5.4059e-01, -5.7155e-01, 5.1882e-01,
1.5895e+00, -6.5196e-01, -7.1919e-02, -3.4362e-01, -1.4979e-01,
1.1727e-01, 2.9567e-01, -2.3110e-01, 2.0275e-01, 6.8011e-01,
1.1323e+00, -6.0740e-01, -2.6564e-02, 1.0097e+00, -4.1150e-01,
-2.2152e-01, 9.6508e-02, -4.4581e-01, 1.9913e-02, 8.3275e-01,
8.5658e-01, -4.8016e-01, -1.0270e-01, -7.2395e-01, 1.3258e+00,
8.9414e-01, -2.1775e-02, 5.0311e-01, 4.1735e-01, 1.1492e+00,
1.3205e+00, 5.4107e-01, -6.5949e-03, 6.5017e-01, 3.1715e-01,
2.7267e-01, 4.8378e-01, -3.0058e-01, 5.4396e-01, 5.8496e-01,
1.8413e-01, 3.1589e-01, -6.4351e-01, -1.3796e+00, 3.4586e-01,
1.3460e-01, -9.6449e-01, 1.7127e-01, 9.6740e-02, 5.0191e-01,
6.9694e-01, -7.4558e-01, -4.2157e-01, -6.6286e-01, -3.4442e-01,
-3.2163e-02, 7.4291e-01, 7.8034e-01, 1.0056e-01, -2.0979e-01,
-4.7121e-01, 2.2513e-01, -6.9803e-01, 1.9003e-01, -7.8334e-02,
5.9580e-01, 6.5451e-01, 4.1943e-01, 1.0541e+00, 2.1776e-01,
4.4065e-01, -1.6912e-01, 1.2728e+00, 2.5968e-01, 3.5075e-01,
-1.3095e-01, -2.6191e-01, 1.6659e-01, 7.3039e-01, 2.3895e-01,
-5.9750e-01, -1.5467e+00, -1.1446e-01, -1.1480e-01, 8.7548e-01,
1.6501e-01, 5.3064e-01, -4.2036e-01, 1.0562e-01, -1.8419e+00,
2.1793e-01, 1.5452e+00, 2.2596e+00, 3.6780e+00, -1.7516e+00,
-5.1068e+00, 6.9256e-01, -2.0463e+00, 3.4414e+00, 6.0913e-01,
-2.7580e+00, -1.3877e+00, 3.6491e+00, 9.0664e-01, 5.9650e-01,
-5.2873e+00, 2.7276e+00, 1.5299e+00, 2.0928e+00, 2.8855e-01,
9.0883e-02, 4.5446e-01, -1.1399e+00, -1.2227e+00, -2.3733e-02,
6.4067e-01, 3.5359e-01, 1.7500e-01, 1.0138e-01, 8.5462e-01,
1.8084e-01, -4.4042e-01, 8.7066e-01, -2.1352e-02, -2.0795e-01,
-2.3622e-01, 1.0714e-01, -5.7031e-01, 7.8763e-02, 4.6406e-01,
1.5986e-01, -8.3928e-02, -3.3554e-01, -9.4353e-02, -1.5627e-01,
4.3915e-02, -3.3141e-01, -4.3529e-01, -3.6728e-01, 2.2967e-01,
5.5715e-02, -4.4543e-02, 2.1356e-01, 5.4012e-02, 1.2812e-01,
5.7026e-01, -1.8449e-01, 2.7647e-01, 2.8464e-01, 3.3869e-01,
-4.4471e-01, 2.7815e-01, 1.5218e-01, 8.1389e-02, 3.9135e-01,
-3.1414e-01, 1.8227e-01, 2.3324e-01, 1.2150e-01, 1.6102e-01,
1.6883e-01, 6.1170e-01, -2.7265e-01, 2.6056e-01, 1.1096e-01,
3.6061e-01, -1.2543e-01, 5.6875e-01, 7.9587e-02, 2.8133e-02,
-4.3987e-01, 7.4524e-01, 8.1850e-01, 3.3244e-01, -9.6828e-03,
-6.6014e-01, -4.6687e-01, 1.8872e-02, -1.4596e+00, -5.6165e-01,
-2.6647e-01, -6.8108e-01, -8.9338e-02, 6.2322e-01, -3.4848e-01,
1.3080e+00, 3.1034e-01, -5.6990e-01, 6.3610e-01, 4.1913e-01,
-2.2799e-01, -3.4887e-01, 1.6637e-01, 2.2691e-01, -1.4747e-01,
1.1307e-02, -1.4700e+00, -4.9403e-01, -2.5610e+00, 4.0959e-01,
-1.7832e-02, 2.8175e-01, -1.1772e+00, -3.9658e-01, -5.8043e-01,
-3.8035e-01, -2.4175e+00, -6.5694e-01, -1.1885e+00, 4.2394e-01,
-7.9300e-01, -6.9536e-01, 1.9480e-01, 5.3906e-01, 4.5429e-01,
5.5419e-01, -7.8385e-01, -6.3982e-01, -9.8756e-01, -6.4319e-01,
9.8521e-01, 1.3515e-01, -6.0872e-01, 4.6865e-01, -5.9693e-01,
1.1639e+00, 4.5721e-01, -4.8116e-01, -1.0266e+00, 8.0050e-01,
-8.6543e+00, -7.7487e-01, -1.7354e+00, 5.0774e-01, -1.5116e-01,
6.5581e-01, 1.0535e+00, -1.1288e-01, 9.0740e-01, -1.5008e-01,
-2.3286e-01, -6.1092e-01, -4.2080e-01, 1.9577e-01, -7.4521e-02,
-9.0685e-01, -9.5202e-01, -1.4556e-01, 1.1909e+00, -5.2183e-01,
5.2400e-01, 5.4804e-01, -2.6359e-02, 7.1852e-02, -1.3642e+00,
-4.7307e-01, -1.3460e+00, 1.4403e+00, 5.8677e-01, -6.2046e-01,
6.3720e-01, 3.6557e-01, 1.5390e-01, -3.4337e-01, -6.1884e-01,
5.1864e-01, -8.6654e-01, -7.3285e-01, -3.6331e-01, 2.1983e-02,
-3.1849e-01, 4.3553e-01, -2.9508e-01, -6.1034e-01, 2.7714e-01,
-1.0299e-02, -1.3007e-01, -7.9403e-01, -1.4354e-02, -1.9658e-01,
1.2445e-01, -7.1796e-01, 1.0185e+00, 2.3027e-01, 7.9509e-02,
1.6290e+00, 3.3397e-02, -1.2053e+00, -5.5172e-01, 8.5064e+00,
3.6699e-01, 9.4890e-01, -4.9857e-02, 1.6379e+00, 1.7442e+00,
-1.7512e+00, -1.3697e+00, -8.1510e-01, -7.8216e-01, -1.6073e+00,
2.4169e+00, -2.6893e+00, -6.7954e-01, -5.5251e-01, -4.1289e-01,
6.7903e-01, -6.2132e+00, 5.0589e-01, -8.5332e-01, -3.1098e-01,
4.7847e-01, 1.0729e+00, -2.6948e+00, -3.3724e-01, -2.1517e-01,
-8.0831e-01, 4.4977e-01, 1.4663e-01, 4.0207e-01, -1.7856e-01,
-3.8278e-01, -2.3807e-01, 4.4171e-01, -1.3126e-01, -9.1115e-02,
-3.1845e-01, -4.2943e-01, 3.1140e-01, -5.9271e-01, 3.6972e-01,
4.2026e-01, -1.0126e-01, 1.1651e-01, -7.0632e-03, -2.7237e-02,
7.3954e-02, -1.8809e-02, -1.5508e-01, -2.4741e-01, -2.2450e-01,
-2.5280e-01, -9.3249e-02, 2.0116e-01, -2.4234e-01, -1.1933e-01,
9.6005e-02, -1.1471e-01, 4.6700e-01, -1.0877e+00, -1.0630e-01,
-7.0045e-01, -4.4478e-02, 7.0627e-03, -2.3293e-01, -1.6952e-01,
-2.7771e-02, 3.0962e-02, 8.5340e-01, 4.2607e-01, 6.4577e-03,
2.1383e-01, 1.7657e-01, -5.2605e-02, 3.9513e-01, -1.6461e-01,
5.7022e-01, -4.8996e-01, 2.1916e-01, -4.2963e-01, 2.6680e-01,
-1.4892e-01, 5.0345e-01, -1.5079e+00, -5.0591e-01, -7.4365e-01,
-4.9540e-02, -6.2029e-01, -3.2874e-01, -7.6654e-01, 3.0008e-01,
-6.2198e-01, 3.1901e-01, -7.8671e-02, -4.3953e-01, -1.7092e-01,
-5.0774e-01, 5.9035e-01, -6.5870e-01, -5.2818e-02, -6.6234e-01,
-4.5778e-01, -8.5810e-02, -3.7528e-01, 4.1572e-01, -3.5546e-01,
-6.6395e-02, 2.2828e-01, 4.3746e-01, -2.8033e-01, 8.5752e-02,
-3.5769e-02, -5.2495e-01, -5.3248e-01, 7.3542e-02, 5.2541e-02,
-7.2728e-01, 8.6862e-01, 6.1096e-01, 1.8481e-01, 2.4711e-02,
1.1197e+00, 8.9652e-02, -9.9660e-02, -7.2695e-02, -6.0828e-01,
-5.7953e-01, 1.5475e-01, 6.2944e-01, 5.7360e-01, 3.0312e-01,
1.3165e-01, 3.9286e+00, -2.6326e-01, 5.6148e-01, 6.3572e-01,
-2.4615e-01, -2.9317e-01, 4.3956e-01, -4.1583e-01, 4.3967e-01,
7.5817e-01, 3.9044e-01, 8.8932e-01, -3.6930e-02, -3.0395e-01,
-1.0312e-02, -1.2368e+00, -7.4400e-02, 2.6703e-01, 1.1823e+00,
5.5611e-01, 3.3376e-01, -3.8320e-02, 6.1927e-01, 1.0627e-01,
-2.8700e-01, 8.5333e-01, 9.7134e-01, 1.3335e-01, 5.5171e-01,
2.6774e-01, -9.6888e-01, -1.8453e-01, -1.1297e-01, 8.5712e-01,
-3.7611e-01, -1.2098e+00, 7.4872e-01, 1.8454e-01, 3.5428e-01,
3.3605e-01, -1.4440e-01, -2.4086e-01, 1.2324e+00, 4.6899e-01,
2.9121e-01, -6.7357e-01, -1.7190e-01, 4.8392e-01, -1.9781e-01,
-7.5126e-01, 4.0947e-01, 2.1045e+00, -1.2726e-01, -2.3006e-01,
5.0114e-01, 7.3735e-01, -4.5818e-01, -4.2784e-01, 4.2079e-02,
-5.9962e-01, -2.4089e+00, 2.3286e+00, 4.3583e+00, -1.6604e-02,
7.4824e-01, -3.2635e-01, 1.4836e-02, -6.5412e-03, 1.8046e+00,
1.1204e+00, 5.7502e+00, -3.5545e+00, 4.8219e+00, 1.9261e+00,
4.4971e-01, -2.3580e-01, 1.3023e+00, 6.7736e-01, 1.5877e-01,
-4.8377e-02, -1.2184e+00, -1.0612e+00, 9.4124e-01, 6.9500e-01,
7.2990e-01, 2.2573e-01, 1.7875e-01, 6.6879e-01, 4.0617e-01,
6.6001e-01, 2.1380e-01, 4.1321e-01, 7.6323e-02, -3.7210e-01,
-2.6931e-01, 6.6018e-02, -1.2173e-01, -2.9774e-01, 3.8813e-01,
5.8198e-01, 3.1442e-01, -4.2824e-01, -1.6255e-01, 1.5531e-01,
6.5715e-02, -2.8626e-01, 1.5100e-02, 2.1823e-02, -7.1460e-02,
-1.2019e-01, 9.4595e-02, 8.6030e-02, 5.7020e-01, -7.5068e-02,
-1.1433e-01, -1.8602e-01, -3.5941e-02, 2.2753e-01, 2.1493e-01,
1.3542e-01, 2.7423e-01, -1.3808e-01, -4.0036e-01, 4.2396e-01,
8.3237e-01, 1.0710e-01, 9.0594e-01, -7.8732e-01, -1.7216e-01,
-1.9548e-01, -2.6920e-01, 7.3795e-02, 3.4235e-01, 1.0182e-01,
1.1253e-01, 3.8861e-01, -5.2161e-01, -3.1576e-01, -9.5424e-01,
4.3329e-02, 5.6380e-01, 4.8834e-01, 1.1312e-01, 2.5585e-01,
1.7790e-01, -1.2985e-01, -8.0982e-02, -1.6345e-01, -5.7320e-01,
7.0335e-01, -2.4985e-01, -3.5911e-01, -3.0396e-01, -2.4768e-01,
-8.5159e-01, -3.7665e-02, 2.2901e-01, -3.2200e-01, -3.0409e-01,
-3.1189e-01, 5.7338e-02, 1.0383e+00, 2.4201e-01, -3.8273e-01,
-8.3808e-02, 1.1346e-01, -2.7166e-01, 1.0038e+00, -3.4995e-01,
-9.1717e-01, -2.3629e+00, -1.0566e+00, -4.8967e-01, -3.3856e-01,
-2.5526e-01, -6.6082e-01, 8.9148e-01, -1.0915e+00, -1.3361e+00,
-4.4893e-01, 1.5452e-02, -3.7852e-01, -8.9101e-01, -5.6950e-01,
-1.7580e+00, 1.2561e-01, 1.4734e-01, -2.2845e-01, -9.6657e-01,
-5.6488e-01, -2.2788e-01, -6.3057e-01, 2.6368e-02, 2.8124e-01,
-4.7570e-01, 3.3287e-01, -2.0889e+00, -7.2370e-01, 4.9773e-01,
-9.2107e-01, 3.1974e-01, 7.2370e-02, -1.1143e-01, 1.4512e-01,
-7.4272e-02, 1.3406e+00, 1.6424e-01, 6.1287e-02, -8.3602e-02,
6.9165e-01, -1.1855e+00, -1.9538e-01, -1.6059e+00, 3.1074e-01,
-1.7713e+00, -7.1275e-02, -3.8238e-01, -1.7204e-02, -5.5786e-01,
-5.5917e-01, -9.0503e-01, 2.3129e-01, -5.4579e-01, -7.2321e-01,
-5.4893e-01, -7.2457e-01, -4.0298e-01, 1.3188e-01, -8.7769e-02,
-7.0746e-01, -3.8019e-01, 2.0796e-01, -4.7922e-01, 1.4855e-01,
1.4827e+00, -5.4571e-01, -1.2159e-01, -5.3450e-02, 2.4224e-01,
-5.0215e-01, 3.2604e-01, -1.0368e+00]])
-------------
tensor([[-3.4813e-01, 2.0567e-01, 1.1198e+00, -7.6697e-01, 8.0420e-01,
-1.9985e-01, 1.5050e+00, 3.6373e-01, -8.4349e-01, 1.0604e+00,
-1.6847e-03, 3.8358e-01, -5.0110e-02, 8.6829e-03, 9.9880e-01,
-2.3323e-01, -1.2491e-01, 9.6177e-01, 6.1365e-01, 1.0828e+00,
1.2108e+00, -5.9665e-01, 3.6451e-01, 1.7379e+00, -1.0191e+00,
2.8054e-02, 1.0104e+00, -4.7329e+00, 1.7480e-01, -5.9130e-01,
1.3921e+00, 8.7543e-01, 1.1291e+00, 5.8056e-01, -1.5969e+00,
4.3268e+00, 2.0635e-01, -2.9404e-01, -4.0857e-01, -1.1726e-01,
-4.0359e-01, 1.2989e+00, 1.4539e+00, -8.6118e-02, -2.5519e-01,
-2.1269e-01, -3.3352e-02, 4.3396e-01, -3.6227e-01, -3.5778e-01,
-3.2265e-02, -7.7707e-01, -1.3238e-01, -8.4862e-01, -9.6902e-02,
-6.2703e-01, -9.4705e-02, 2.2783e-01, -5.2093e-01, 1.6962e+00,
6.6151e-01, -4.3671e-01, 2.6962e-01, 6.7134e-02, -1.6922e+00,
-6.3866e-01, -2.6890e-02, 1.3045e+00, 6.2257e-01, 3.0609e-01,
-4.7825e-01, 1.7611e+00, -3.3672e+00, -7.7393e-01, 4.0487e-01,
-7.4480e-01, -5.9025e-01, 9.7720e-01, -3.1626e-01, -8.8742e-01,
9.6683e-01, 3.5197e-01, 7.6611e-02, -2.6025e-01, 4.9337e-01,
-1.3028e+00, -6.4966e-01, -3.0904e-01, 1.1566e-01, -1.3597e+00,
3.6609e-01, -2.0978e-01, -3.9352e-02, -1.0408e+00, 6.4481e-01,
-8.7822e-01, 8.2662e-01, -1.5772e-01, 6.0465e-02, -2.7535e+00,
-5.2105e+00, 5.8370e-01, 1.5931e+00, 5.4748e-01, 2.9027e-01,
-1.1941e+00, -6.7923e-01, 7.8292e-01, -5.1848e-01, -4.7592e-01,
1.6911e-01, -6.4140e-01, 1.2979e+00, -2.1467e+00, -4.4913e-01,
-1.0146e+00, 1.3549e-01, 1.1974e-01, -3.4173e-01, 2.0884e-01,
1.1952e+00, -4.6521e-01, -1.6883e-01, -6.1835e-01, 1.8646e-02,
1.0804e+00, -1.6815e-01, -3.0126e-01, 1.1650e-01, -2.8620e-01,
1.0174e+00, 6.9934e-01, 5.4898e-01, -4.2590e-01, 8.3970e+00,
-8.5070e-03, 6.6272e-01, -3.1643e-01, 2.2381e+00, 7.6352e-01,
-8.8733e-01, 2.1807e-01, -1.7618e+00, 4.5878e-01, 7.7188e-01,
-1.7815e+00, 1.1668e-01, -5.2732e-01, -9.5365e-01, -8.1243e-01,
3.3305e-01, -5.4493e-01, -1.5769e-01, 7.5794e-01, 1.1380e+00,
7.4210e-01, -6.9392e-01, -2.1022e+00, 3.0827e-01, 2.1535e+00,
8.4265e-01, 1.2759e+00, 7.5762e-01, 3.5451e-02, 1.7083e+00,
9.2172e-01, -5.3087e-01, 2.5303e-01, -2.2834e-04, -1.0250e-01,
1.2776e+00, 3.0638e-02, 3.2619e-01, -4.0159e-01, -1.2273e+00,
1.5906e-01, 6.5872e-01, 1.2371e+00, -7.6748e-01, 1.3451e+00,
-5.1231e-01, -1.0270e-01, -6.5145e-03, 5.5793e-01, 2.2064e+00,
6.4659e-01, -1.5841e+00, 7.4834e-01, 2.2196e-01, 1.3080e+00,
9.1690e-01, 3.9154e-01, 8.5474e-02, 2.7225e-01, -5.3079e-02,
4.1462e-01, -2.0329e-01, 1.6161e+00, -4.7893e-01, -1.0734e+00,
5.9383e-01, -1.7099e+00, 6.2909e-01, -2.7005e-01, 5.0200e-01,
-9.1901e-02, -7.2574e-01, -1.0055e+00, -1.3700e+00, -1.8188e-02,
6.6036e+00, -3.6362e-01, -3.9388e-01, 1.7845e-01, -3.8350e-01,
-3.6312e-01, -1.1015e+00, -6.8798e-01, 5.6410e-01, -2.9650e+00,
-9.5680e-01, 9.1157e-01, -7.3791e-01, -2.8096e-01, 3.2821e-01,
1.1094e-01, 1.0264e-01, -1.2200e-01, -1.5305e+00, -2.2308e-02,
1.2022e+00, -3.0021e-01, 4.4868e-01, -1.7616e-01, -3.2443e-01,
-2.4681e-01, -8.8532e-01, 7.0337e-01, -8.6574e-01, -1.2170e+00,
7.5557e-02, -1.5184e+00, 9.3093e-01, -8.0685e-01, 4.2268e-01,
5.6668e-01, -2.5511e-01, 1.2286e+00, -8.2452e-01, -7.9402e-01,
-8.0377e-02, -7.3588e-01, -6.7720e-01, -1.0961e+00, 3.5913e-01,
2.2695e-01, 8.3689e-01, 3.1688e-01, 3.3786e-01, -1.7791e-01,
7.2241e-01, 1.2199e+00, -4.0950e-01, 9.0252e-01, -8.4963e-01,
7.1447e-01, 2.4751e+00, 1.4018e+00, 9.4643e-01, 1.1977e+00,
-2.0829e-01, 5.8284e-01, -7.7632e-02, -7.0110e-01, 8.3377e-01,
-3.4768e-01, 1.8975e-01, 9.3699e-01, -5.7122e-01, 5.0952e-01,
2.6974e-01, 9.5232e-01, 7.6480e-01, 3.3528e-01, -5.9050e-01,
2.1423e-01, 7.6098e-01, 3.5594e-01, -5.9158e-01, -4.3236e+00,
-2.3408e+00, -1.9314e-01, -1.0072e+00, 2.7498e-01, 8.0217e-01,
4.1695e-01, 1.0866e+00, 3.1961e-01, -7.7914e-01, -3.9838e+00,
3.8185e-01, 6.4260e-01, -6.5978e-01, 5.5445e-01, 1.0432e-01,
4.1820e-01, 5.7183e-01, 9.9554e-01, -6.6937e-01, 1.0036e-01,
-2.3801e-01, -5.7794e-01, 3.6589e-01, -1.4862e+00, -3.7305e+00,
-9.1693e-01, -8.4865e-01, -3.0791e-01, 1.1304e+00, -2.0416e-01,
2.4343e-01, 1.2942e-01, -5.1435e-01, -1.3415e-01, 7.0296e-01,
1.6261e+00, -5.8394e-01, 9.0721e-02, -3.0127e-01, -1.8367e-01,
-4.0652e-01, 5.6997e-01, 3.1324e-01, 1.9543e-01, 1.3412e+00,
-1.0137e+00, 8.5265e-02, 4.5186e-01, -4.7791e-01, 6.6931e-01,
5.2295e-01, -1.1835e+00, -1.1139e+00, 1.0672e+00, -4.2333e-01,
5.0702e-01, -8.9741e-01, 5.9155e-01, 8.3508e-01, -2.1873e+00,
-4.1383e-01, -6.4959e-02, 3.9494e-01, 1.8686e-01, -3.0738e-01,
1.0184e+00, 8.6245e-01, -4.9785e-01, 8.8178e-01, 5.2227e-01,
2.5437e-01, 3.4015e-01, 8.8109e-01, -2.2000e-01, 5.0073e-01,
8.6852e-01, 1.5609e-01, 2.5910e-01, 5.0248e-01, 3.7036e-01,
1.2450e+00, -5.9712e-01, 5.4946e-01, 1.6803e-02, 2.1828e+00,
3.5344e-01, -1.4539e+00, -8.8693e-01, 5.7637e-02, -4.0915e-01,
-8.3213e-01, 4.5347e-01, -1.6956e-01, -6.0734e-01, 5.8257e-01,
-1.4884e-01, 4.0022e-01, 9.7919e-01, -2.9748e-01, -2.2484e-01,
3.2519e-01, 2.7024e-01, 5.0929e-01, -5.9550e-02, -1.0012e+00,
-1.3135e+00, 2.2016e-04, -1.7333e-01, -1.0410e-01, -2.6027e-01,
-8.2319e-01, -8.3118e-02, 3.0932e-01, 7.4527e-01, 1.0045e-01,
1.0745e+00, -3.3992e-01, 5.8400e-01, -3.7636e-02, 2.9397e-01,
-1.0027e+00, 7.1944e-01, -3.9056e-01, -4.9528e-01, -7.9961e-01,
4.5255e-01, 1.1398e+00, -8.9698e-02, -3.4009e-01, -2.2956e-01,
3.8484e-01, 7.6305e-01, -6.1467e-01, -1.7440e-01, -8.7346e-01,
-8.1845e-02, -1.1355e+00, -4.3620e-01, -2.9867e-02, -3.9152e-01,
1.9816e-01, 2.1522e-01, -9.8091e-02, -2.8077e-02, -9.1162e-01,
-1.1340e-01, -8.8686e-01, -6.1723e-01, -8.8788e-03, -1.8461e-01,
1.4909e+00, -3.9125e-01, -7.1782e-01, 9.7443e-01, 1.3088e+00,
3.1069e-01, -3.7232e-01, 2.5268e-01, -3.5614e-01, -5.4891e-02,
-1.4297e+00, 9.3509e-01, 7.1550e-01, 1.2757e-01, 1.2531e-01,
1.4145e-01, -9.2239e-01, -7.8556e-01, -2.1314e-01, -7.7443e-01,
8.8775e-01, -1.3814e+00, -8.2674e-01, -1.1904e+00, 5.6500e-01,
-1.4106e-01, 1.4624e+00, 9.1496e-01, 2.9025e-01, -9.5909e-01,
9.2260e-01, 6.4493e-02, -1.2465e-01, -2.0094e+00, 9.3044e-01,
1.2729e-01, -1.0760e+00, -6.6359e-01, -6.0642e-01, -7.3617e-02,
-3.2029e-01, 6.7818e-01, 2.0279e-01, -1.1754e+00, 6.0223e-01,
7.7994e-01, 9.0384e-01, -5.5044e-01, 5.9588e-02, -1.2183e+00,
6.5162e-01, 8.2186e-01, -1.5056e+00, 1.2215e+00, -8.0109e-01,
-6.8082e-01, -8.8348e-01, 4.4649e-01, 3.2998e-01, 7.1524e-01,
-1.7936e-01, -1.2808e+00, -3.8634e-01, -9.7096e-02, 1.7621e-01,
8.9458e-01, -7.2052e-01, -3.3643e+00, -6.9678e-01, 6.4796e-01,
-4.3859e-01, -3.5114e-01, -1.6527e-01, 1.1502e+00, -1.0907e+00,
4.3750e-02, 1.3387e+00, 5.3176e-01, -3.8411e-01, 3.3800e-01,
-2.3536e-01, 2.5217e-01, 6.6257e-01, -2.7705e-01, 1.9945e-01,
1.4662e+00, 8.0927e-01, 1.2329e+00, 1.3495e-02, 2.9029e-02,
6.6076e-01, -6.3162e-01, -4.2206e-01, 1.0871e+00, -5.6084e-01,
-8.4602e-01, 7.8891e-01, -4.0589e-01, -5.9161e-01, -1.7212e+00,
-9.0162e-01, -1.7410e+00, -6.9864e-01, 4.2097e-01, -4.4049e-01,
8.4982e-01, 1.4221e+00, -8.5526e-01, -1.0523e+00, -1.9903e-02,
-3.2509e-01, -3.1833e-01, -1.3884e+00, 8.1081e-01, 6.6353e-01,
-4.3236e-01, 1.1110e-01, 3.8525e-01, 3.7393e-01, 3.5350e-01,
-4.4797e-01, -6.3131e-01, -5.9597e-01, -1.2599e+00, 7.3669e-01,
2.0802e-01, 8.7950e-02, -2.7246e-01, -3.4595e-01, -7.0211e-02,
7.3376e-01, 1.0929e+00, -8.5560e-01, -1.4409e+00, 7.8775e-01,
-1.2244e+00, 6.8799e-01, 3.7250e-01, -8.0667e-01, -2.4792e-01,
3.6830e-01, -1.4776e+00, -9.1936e-01, -1.8799e-02, -7.1844e-01,
1.1552e-01, -3.8574e+00, 2.6220e-01, -1.6299e-01, -5.9341e-01,
-1.8762e-01, -8.4764e-01, -2.8735e-01, 5.2838e-01, -3.0892e-02,
-6.3573e-01, 4.0009e-01, -1.8742e-03, 2.6570e-01, -2.9478e-01,
-1.6617e+00, 1.1095e+00, -1.0892e+00, 9.0501e-01, 6.9013e-01,
6.5222e-01, -2.9603e+00, 2.7552e-01, 3.7823e-01, -4.8048e-01,
-1.1328e-01, 9.5024e-01, -6.2560e-01, 1.1036e+00, -2.7692e-01,
-4.1032e-01, 2.3083e+00, -9.0611e-01, -8.1959e-01, 1.7053e-01,
1.0951e+00, -4.3046e-01, 4.0264e-01, 1.1148e+00, 4.8053e-01,
-1.2339e+00, -1.3231e+00, 1.0122e+00, 4.4393e-01, 1.4085e+00,
-6.9475e-02, -1.7194e+00, -6.7711e-01, -1.3572e-01, -1.3110e-01,
-3.7889e-01, 8.2807e-01, 1.0463e-01, 1.9614e-01, 2.5452e-02,
-4.2329e-01, 8.0391e-01, 4.2201e-01, -1.8933e-02, -1.5100e+00,
5.0569e-01, 3.5948e-01, 4.6802e-01, 8.1692e-01, 4.3526e-01,
5.7782e-01, 4.2392e-01, -6.8392e-01, -3.3318e-01, 9.5535e-01,
-9.9570e-02, -4.1224e-01, 2.6090e-01, 3.3855e-01, -1.0511e-01,
-2.4586e-01, 7.0579e-02, -2.8246e-01, 4.3424e+00, -7.0079e-01,
-1.0838e-02, 1.0038e+00, -3.1166e-01, 5.4678e-01, 4.4886e-01,
-5.4054e-01, -7.9610e-01, 1.9553e-01, 1.9221e+00, 3.0019e-01,
9.0937e-02, -5.8753e-02, 2.7907e-01, 4.4700e-01, 1.3425e+00,
4.2148e-01, 1.8954e-01, -3.5193e-02, -2.0193e+00, 1.3882e-01,
-2.1958e-01, -5.6894e-01, -5.5701e-02, 5.7875e-01, 1.4129e-01,
3.9483e-01, -2.1259e+00, 1.6142e+00, 3.5990e-01, -1.1435e+00,
-9.3953e-01, -1.9983e-01, 2.1357e-01, -1.3327e+00, 4.1222e-01,
-3.0913e-01, -3.8733e-01, -5.3323e-01, 1.2022e-01, -1.1312e+00,
-1.5004e-01, -4.2470e-01, 1.3373e+00, 2.6895e-01, 1.0954e-01,
-2.6285e-01, 1.4255e+00, -7.9770e-01, -1.6346e+00, -1.3298e-01,
7.2792e-03, 1.5195e-01, -1.5816e-01, 7.0274e-01, -3.2218e-01,
2.5528e-01, 1.9823e-01, -2.3153e-01, -3.5111e-01, -1.0063e+00,
-9.3071e-01, -8.4325e-01, 1.0345e+00, -4.4011e-01, 1.0357e+00,
-4.7383e-01, -4.3529e-01, -4.6581e-01, -1.0314e+00, -5.5068e-01,
-1.0455e+00, -4.6042e-01, 1.0848e+00, 4.7743e-01, -7.8016e-01,
-1.4927e-01, 5.0406e-01, -6.5103e-01, -7.0877e-01, 7.8158e-01,
2.4137e-01, 3.1456e-01, 1.3761e+00, 8.8048e-01, 4.7309e-01,
-3.9214e-01, 6.2577e-01, 1.5551e+00, 3.4581e-01, -6.0743e-01,
1.6307e-01, 4.1183e-01, 3.6736e-01, 7.1847e-02, 9.0911e-01,
8.5306e-01, 7.2854e-01, -1.4751e-01, 1.2149e-01, 1.2044e-01,
2.1714e+00, -7.5083e-01, -1.1921e+00]])
In [41]:
print(cpu['layer_0_embed'])
tensor([[-4.5210e-01, 1.7855e+00, 2.5990e-01, -9.2623e-02, -7.9379e-02,
-4.9030e-01, 5.2150e-01, 2.0427e-02, -8.7831e-01, -1.0923e+00,
9.1779e-01, 1.9031e+00, -1.8882e-01, 4.0819e+00, -9.9635e-01,
6.6815e-01, 9.3187e-01, 4.6784e-01, 6.5349e-01, 4.8378e-01,
-8.1687e-02, 5.9407e-02, 3.2209e+00, -3.5371e-01, -1.0952e+00,
7.4126e-01, -1.5051e-01, -1.1173e+00, -3.1420e-01, -8.2539e-01,
-4.4671e-02, 3.6703e-01, 1.6183e-01, 1.8128e-01, 1.3408e-01,
1.5553e+00, -2.4995e+00, -3.0818e-01, 3.6680e-01, 7.0053e-01,
-2.4750e-01, -2.1626e-01, -4.1587e-01, -4.6226e-01, -2.6574e-01,
3.5805e-01, 1.5961e-01, 2.3117e-01, -1.8021e-01, -1.3967e-02,
-2.4106e-01, -5.3606e-01, -2.0481e-01, -4.8502e-01, -1.9846e-01,
2.6415e-01, 6.1616e-02, -1.9794e-01, 1.9629e-01, 2.4304e-02,
-1.2413e-02, 1.7330e-01, 2.2374e-01, 1.9147e-02, -6.4836e-01,
2.2571e-01, 2.3290e-02, 4.8940e-01, 5.0804e-01, -1.8679e-01,
-3.3605e-01, 9.7664e-01, -1.8251e-01, 6.2380e-01, 2.2746e-01,
-1.2193e-01, 4.1913e-01, 1.5899e-01, -3.1110e-01, -4.3871e-01,
-1.9955e-01, -4.2973e-01, -5.6255e-01, 4.7870e-01, 2.5511e-01,
1.6640e-01, -3.5910e-01, -5.2204e-01, 1.5549e-01, -1.1235e+00,
-3.5336e-01, -9.4648e-01, 6.5775e-02, -5.2909e-01, 4.8724e-02,
-4.2012e-01, 1.7835e+00, -1.0192e-01, -4.4476e-01, 1.6462e-01,
-6.9707e-01, 1.7902e-01, 5.4059e-01, -5.7155e-01, 5.1882e-01,
1.5895e+00, -6.5196e-01, -7.1919e-02, -3.4362e-01, -1.4979e-01,
1.1727e-01, 2.9567e-01, -2.3110e-01, 2.0275e-01, 6.8011e-01,
1.1323e+00, -6.0740e-01, -2.6564e-02, 1.0097e+00, -4.1150e-01,
-2.2152e-01, 9.6508e-02, -4.4581e-01, 1.9913e-02, 8.3275e-01,
8.5658e-01, -4.8016e-01, -1.0270e-01, -7.2395e-01, 1.3258e+00,
8.9414e-01, -2.1775e-02, 5.0311e-01, 4.1735e-01, 1.1492e+00,
1.3205e+00, 5.4107e-01, -6.5949e-03, 6.5017e-01, 3.1715e-01,
2.7267e-01, 4.8378e-01, -3.0058e-01, 5.4396e-01, 5.8496e-01,
1.8413e-01, 3.1589e-01, -6.4351e-01, -1.3796e+00, 3.4586e-01,
1.3460e-01, -9.6449e-01, 1.7127e-01, 9.6740e-02, 5.0191e-01,
6.9694e-01, -7.4558e-01, -4.2157e-01, -6.6286e-01, -3.4442e-01,
-3.2163e-02, 7.4291e-01, 7.8034e-01, 1.0056e-01, -2.0979e-01,
-4.7121e-01, 2.2513e-01, -6.9803e-01, 1.9003e-01, -7.8334e-02,
5.9580e-01, 6.5451e-01, 4.1943e-01, 1.0541e+00, 2.1776e-01,
4.4065e-01, -1.6912e-01, 1.2728e+00, 2.5968e-01, 3.5075e-01,
-1.3095e-01, -2.6191e-01, 1.6659e-01, 7.3039e-01, 2.3895e-01,
-5.9750e-01, -1.5467e+00, -1.1446e-01, -1.1480e-01, 8.7548e-01,
1.6501e-01, 5.3064e-01, -4.2036e-01, 1.0562e-01, -1.8419e+00,
2.1793e-01, 1.5452e+00, 2.2596e+00, 3.6780e+00, -1.7516e+00,
-5.1068e+00, 6.9256e-01, -2.0463e+00, 3.4414e+00, 6.0913e-01,
-2.7580e+00, -1.3877e+00, 3.6491e+00, 9.0664e-01, 5.9650e-01,
-5.2873e+00, 2.7276e+00, 1.5299e+00, 2.0928e+00, 2.8855e-01,
9.0883e-02, 4.5446e-01, -1.1399e+00, -1.2227e+00, -2.3733e-02,
6.4067e-01, 3.5359e-01, 1.7500e-01, 1.0138e-01, 8.5462e-01,
1.8084e-01, -4.4042e-01, 8.7066e-01, -2.1352e-02, -2.0795e-01,
-2.3622e-01, 1.0714e-01, -5.7031e-01, 7.8763e-02, 4.6406e-01,
1.5986e-01, -8.3928e-02, -3.3554e-01, -9.4353e-02, -1.5627e-01,
4.3915e-02, -3.3141e-01, -4.3529e-01, -3.6728e-01, 2.2967e-01,
5.5715e-02, -4.4543e-02, 2.1356e-01, 5.4012e-02, 1.2812e-01,
5.7026e-01, -1.8449e-01, 2.7647e-01, 2.8464e-01, 3.3869e-01,
-4.4471e-01, 2.7815e-01, 1.5218e-01, 8.1389e-02, 3.9135e-01,
-3.1414e-01, 1.8227e-01, 2.3324e-01, 1.2150e-01, 1.6102e-01,
1.6883e-01, 6.1170e-01, -2.7265e-01, 2.6056e-01, 1.1096e-01,
3.6061e-01, -1.2543e-01, 5.6875e-01, 7.9587e-02, 2.8133e-02,
-4.3987e-01, 7.4524e-01, 8.1850e-01, 3.3244e-01, -9.6828e-03,
-6.6014e-01, -4.6687e-01, 1.8872e-02, -1.4596e+00, -5.6165e-01,
-2.6647e-01, -6.8108e-01, -8.9338e-02, 6.2322e-01, -3.4848e-01,
1.3080e+00, 3.1034e-01, -5.6990e-01, 6.3610e-01, 4.1913e-01,
-2.2799e-01, -3.4887e-01, 1.6637e-01, 2.2691e-01, -1.4747e-01,
1.1307e-02, -1.4700e+00, -4.9403e-01, -2.5610e+00, 4.0959e-01,
-1.7832e-02, 2.8175e-01, -1.1772e+00, -3.9658e-01, -5.8043e-01,
-3.8035e-01, -2.4175e+00, -6.5694e-01, -1.1885e+00, 4.2394e-01,
-7.9300e-01, -6.9536e-01, 1.9480e-01, 5.3906e-01, 4.5429e-01,
5.5419e-01, -7.8385e-01, -6.3982e-01, -9.8756e-01, -6.4319e-01,
9.8521e-01, 1.3515e-01, -6.0872e-01, 4.6865e-01, -5.9693e-01,
1.1639e+00, 4.5721e-01, -4.8116e-01, -1.0266e+00, 8.0050e-01,
-8.6543e+00, -7.7487e-01, -1.7354e+00, 5.0774e-01, -1.5116e-01,
6.5581e-01, 1.0535e+00, -1.1288e-01, 9.0740e-01, -1.5008e-01,
-2.3286e-01, -6.1092e-01, -4.2080e-01, 1.9577e-01, -7.4521e-02,
-9.0685e-01, -9.5202e-01, -1.4556e-01, 1.1909e+00, -5.2183e-01,
5.2400e-01, 5.4804e-01, -2.6359e-02, 7.1852e-02, -1.3642e+00,
-4.7307e-01, -1.3460e+00, 1.4403e+00, 5.8677e-01, -6.2046e-01,
6.3720e-01, 3.6557e-01, 1.5390e-01, -3.4337e-01, -6.1884e-01,
5.1864e-01, -8.6654e-01, -7.3285e-01, -3.6331e-01, 2.1983e-02,
-3.1849e-01, 4.3553e-01, -2.9508e-01, -6.1034e-01, 2.7714e-01,
-1.0299e-02, -1.3007e-01, -7.9403e-01, -1.4354e-02, -1.9658e-01,
1.2445e-01, -7.1796e-01, 1.0185e+00, 2.3027e-01, 7.9509e-02,
1.6290e+00, 3.3397e-02, -1.2053e+00, -5.5172e-01, 8.5064e+00,
3.6699e-01, 9.4890e-01, -4.9857e-02, 1.6379e+00, 1.7442e+00,
-1.7512e+00, -1.3697e+00, -8.1510e-01, -7.8216e-01, -1.6073e+00,
2.4169e+00, -2.6893e+00, -6.7954e-01, -5.5251e-01, -4.1289e-01,
6.7903e-01, -6.2132e+00, 5.0589e-01, -8.5332e-01, -3.1098e-01,
4.7847e-01, 1.0729e+00, -2.6948e+00, -3.3724e-01, -2.1517e-01,
-8.0831e-01, 4.4977e-01, 1.4663e-01, 4.0207e-01, -1.7856e-01,
-3.8278e-01, -2.3807e-01, 4.4171e-01, -1.3126e-01, -9.1115e-02,
-3.1845e-01, -4.2943e-01, 3.1140e-01, -5.9271e-01, 3.6972e-01,
4.2026e-01, -1.0126e-01, 1.1651e-01, -7.0632e-03, -2.7237e-02,
7.3954e-02, -1.8809e-02, -1.5508e-01, -2.4741e-01, -2.2450e-01,
-2.5280e-01, -9.3249e-02, 2.0116e-01, -2.4234e-01, -1.1933e-01,
9.6005e-02, -1.1471e-01, 4.6700e-01, -1.0877e+00, -1.0630e-01,
-7.0045e-01, -4.4478e-02, 7.0627e-03, -2.3293e-01, -1.6952e-01,
-2.7771e-02, 3.0962e-02, 8.5340e-01, 4.2607e-01, 6.4577e-03,
2.1383e-01, 1.7657e-01, -5.2605e-02, 3.9513e-01, -1.6461e-01,
5.7022e-01, -4.8996e-01, 2.1916e-01, -4.2963e-01, 2.6680e-01,
-1.4892e-01, 5.0345e-01, -1.5079e+00, -5.0591e-01, -7.4365e-01,
-4.9540e-02, -6.2029e-01, -3.2874e-01, -7.6654e-01, 3.0008e-01,
-6.2198e-01, 3.1901e-01, -7.8671e-02, -4.3953e-01, -1.7092e-01,
-5.0774e-01, 5.9035e-01, -6.5870e-01, -5.2818e-02, -6.6234e-01,
-4.5778e-01, -8.5810e-02, -3.7528e-01, 4.1572e-01, -3.5546e-01,
-6.6395e-02, 2.2828e-01, 4.3746e-01, -2.8033e-01, 8.5752e-02,
-3.5769e-02, -5.2495e-01, -5.3248e-01, 7.3542e-02, 5.2541e-02,
-7.2728e-01, 8.6862e-01, 6.1096e-01, 1.8481e-01, 2.4711e-02,
1.1197e+00, 8.9652e-02, -9.9660e-02, -7.2695e-02, -6.0828e-01,
-5.7953e-01, 1.5475e-01, 6.2944e-01, 5.7360e-01, 3.0312e-01,
1.3165e-01, 3.9286e+00, -2.6326e-01, 5.6148e-01, 6.3572e-01,
-2.4615e-01, -2.9317e-01, 4.3956e-01, -4.1583e-01, 4.3967e-01,
7.5817e-01, 3.9044e-01, 8.8932e-01, -3.6930e-02, -3.0395e-01,
-1.0312e-02, -1.2368e+00, -7.4400e-02, 2.6703e-01, 1.1823e+00,
5.5611e-01, 3.3376e-01, -3.8320e-02, 6.1927e-01, 1.0627e-01,
-2.8700e-01, 8.5333e-01, 9.7134e-01, 1.3335e-01, 5.5171e-01,
2.6774e-01, -9.6888e-01, -1.8453e-01, -1.1297e-01, 8.5712e-01,
-3.7611e-01, -1.2098e+00, 7.4872e-01, 1.8454e-01, 3.5428e-01,
3.3605e-01, -1.4440e-01, -2.4086e-01, 1.2324e+00, 4.6899e-01,
2.9121e-01, -6.7357e-01, -1.7190e-01, 4.8392e-01, -1.9781e-01,
-7.5126e-01, 4.0947e-01, 2.1045e+00, -1.2726e-01, -2.3006e-01,
5.0114e-01, 7.3735e-01, -4.5818e-01, -4.2784e-01, 4.2079e-02,
-5.9962e-01, -2.4089e+00, 2.3286e+00, 4.3583e+00, -1.6604e-02,
7.4824e-01, -3.2635e-01, 1.4836e-02, -6.5412e-03, 1.8046e+00,
1.1204e+00, 5.7502e+00, -3.5545e+00, 4.8219e+00, 1.9261e+00,
4.4971e-01, -2.3580e-01, 1.3023e+00, 6.7736e-01, 1.5877e-01,
-4.8377e-02, -1.2184e+00, -1.0612e+00, 9.4124e-01, 6.9500e-01,
7.2990e-01, 2.2573e-01, 1.7875e-01, 6.6879e-01, 4.0617e-01,
6.6001e-01, 2.1380e-01, 4.1321e-01, 7.6323e-02, -3.7210e-01,
-2.6931e-01, 6.6018e-02, -1.2173e-01, -2.9774e-01, 3.8813e-01,
5.8198e-01, 3.1442e-01, -4.2824e-01, -1.6255e-01, 1.5531e-01,
6.5715e-02, -2.8626e-01, 1.5100e-02, 2.1823e-02, -7.1460e-02,
-1.2019e-01, 9.4595e-02, 8.6030e-02, 5.7020e-01, -7.5068e-02,
-1.1433e-01, -1.8602e-01, -3.5941e-02, 2.2753e-01, 2.1493e-01,
1.3542e-01, 2.7423e-01, -1.3808e-01, -4.0036e-01, 4.2396e-01,
8.3237e-01, 1.0710e-01, 9.0594e-01, -7.8732e-01, -1.7216e-01,
-1.9548e-01, -2.6920e-01, 7.3795e-02, 3.4235e-01, 1.0182e-01,
1.1253e-01, 3.8861e-01, -5.2161e-01, -3.1576e-01, -9.5424e-01,
4.3329e-02, 5.6380e-01, 4.8834e-01, 1.1312e-01, 2.5585e-01,
1.7790e-01, -1.2985e-01, -8.0982e-02, -1.6345e-01, -5.7320e-01,
7.0335e-01, -2.4985e-01, -3.5911e-01, -3.0396e-01, -2.4768e-01,
-8.5159e-01, -3.7665e-02, 2.2901e-01, -3.2200e-01, -3.0409e-01,
-3.1189e-01, 5.7338e-02, 1.0383e+00, 2.4201e-01, -3.8273e-01,
-8.3808e-02, 1.1346e-01, -2.7166e-01, 1.0038e+00, -3.4995e-01,
-9.1717e-01, -2.3629e+00, -1.0566e+00, -4.8967e-01, -3.3856e-01,
-2.5526e-01, -6.6082e-01, 8.9148e-01, -1.0915e+00, -1.3361e+00,
-4.4893e-01, 1.5452e-02, -3.7852e-01, -8.9101e-01, -5.6950e-01,
-1.7580e+00, 1.2561e-01, 1.4734e-01, -2.2845e-01, -9.6657e-01,
-5.6488e-01, -2.2788e-01, -6.3057e-01, 2.6368e-02, 2.8124e-01,
-4.7570e-01, 3.3287e-01, -2.0889e+00, -7.2370e-01, 4.9773e-01,
-9.2107e-01, 3.1974e-01, 7.2370e-02, -1.1143e-01, 1.4512e-01,
-7.4272e-02, 1.3406e+00, 1.6424e-01, 6.1287e-02, -8.3602e-02,
6.9165e-01, -1.1855e+00, -1.9538e-01, -1.6059e+00, 3.1074e-01,
-1.7713e+00, -7.1275e-02, -3.8238e-01, -1.7204e-02, -5.5786e-01,
-5.5917e-01, -9.0503e-01, 2.3129e-01, -5.4579e-01, -7.2321e-01,
-5.4893e-01, -7.2457e-01, -4.0298e-01, 1.3188e-01, -8.7769e-02,
-7.0746e-01, -3.8019e-01, 2.0796e-01, -4.7922e-01, 1.4855e-01,
1.4827e+00, -5.4571e-01, -1.2159e-01, -5.3450e-02, 2.4224e-01,
-5.0215e-01, 3.2604e-01, -1.0368e+00]])
In [ ]:
In [ ]:
In [ ]:
3. Finetune¶
3.1 tile level¶
3.2 slide level¶
Please check
Before running slide level, please enable your huggingface token under/gigapath/slide_encoder.py def __init__
In [ ]:
In [ ]:
In [ ]: