Reproducting Image Perception in Zeiss Zen from Raw Data

TL; DR

I’ve created a package (czi-shader) to achieve what is mentioned in the title.

Introduction

Most of the lab’s slices are scanned using Zeiss microscopes. Previously, when working with CZI files outside Zen, you either had to export them within the software or directly convert them to pseudocolor using Python. But how are the appropriate colors assigned to different fluorescence channels when displaying the images?

Principles

After experimentation, the process of going from raw light to the final display can be divided into two steps: single-channel coloring and multi-channel merging.

Single-Channel Coloring

CTB488 is usually specified as fluorescence green, with RGB saturation values of (0, 255, 91), which translates to (141, 100%, 50%) in the HSL color space. Let the actual brightness obtained from scanning be denoted as $l_r$. The resulting color is calculated as follows:

$$
\begin{equation}
\begin{aligned}
H &= 141 \
S &= 100% \
L &= \frac{l_r - \text{low}}{\text{high} - l_r} * 50%
\end{aligned}
\end{equation}
$$

This process is applied to each channel, resulting in multiple (h, w, 3) matrices, where 3 represents the three components of the HSL color space.

Multi-Channel Merging

Zen’s merging is done in the RGB space. The HSL space images obtained above are converted to RGB and then summed and clipped. The equivalent numpy code is as follows:

1
2
3
4
merged_image = np.clip(
np.sum(rgb_imgs, axis=0),
0, 255
)

In practice, overflow needs to be considered.

Channel Information

Now that we have the calculation method, where can we find the raw data for the channels? Based on experiments, in the CZI metadata under .//DisplaySetting/Channels/, a sample data entry looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<Channel Id="Channel:2" Name="EGFP">
<Low>0.0043691845281902137</Low>
<High>0.14451909533769555</High>
<BitCountRange>16</BitCountRange>
<PixelType>Gray16</PixelType>
<DyeName>EGFP</DyeName>
<ShortName>EGFP</ShortName>
<IlluminationType>Fluorescence</IlluminationType>
<DyeMaxEmission>509</DyeMaxEmission>
<DyeMaxExcitation>488</DyeMaxExcitation>
<DyeId>McNamara-Boswell-0828</DyeId>
<DyeDatabaseId>66071726-cbd4-4c41-b371-0a6eee4ae9c5</DyeDatabaseId>
<Color>#FF00FF5B</Color>
<ColorMode>Palette</ColorMode>
<OriginalColor>#FF00FF5B</OriginalColor>
<PaletteName>HeatMap</PaletteName>
<IsSelected>false</IsSelected>
</Channel>

The data format is self-explanatory, but parsing this XML and converting it into appropriate data for coloring the raw light can be repetitive. Therefore, I have developed a Python package that allows you to perform these operations directly.

1
2
3
4
5
6
7
8
import cv2
from czi_shader import CZIChannel, shading_czi

p = '/mnt/inner-data/sc-C057-146-O4213.czi'
[print(c) for c in CZIChannel.from_czi(p)]

res = shading_czi(p, scale_factor=0.01)
cv2.imwrite(p + '.png', cv2.cvtColor(res, cv2.COLOR_RGB2BGR))
1
2
3
4
CZIChannel(id=0, name='Cy5', bit_count_range=16, pixel_type='Gray16', dye_name='Cy5', short_name='Cy5', illumination_type='Fluorescence', dye_max_emission=673, dye_max_excitation=650, dye_id='McNamara-Boswell-0774', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFF0014', original_color='#FFFF0014', color_mode=None, palette_name=None, gamma=None, low=0.0059662775616083005, high=0.03865110246433204, is_selected=None)
CZIChannel(id=1, name='Cy3', bit_count_range=16, pixel_type='Gray16', dye_name='Cy3', short_name='Cy3', illumination type='Fluorescence', dye_max_emission=561, dye_max_excitation=548, dye_id='McNamara-Boswell-0615', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFFAD00', original_color='#FFFFAD00', color_mode=None, palette_name=None, gamma=None, low=0.006240939955748837, high=0.13965056839856566, is_selected=None)
CZIChannel(id=2, name='EGFP', bit_count_range=16, pixel_type='Gray16', dye_name='EGFP', short_name='EGFP', illumination_type='Fluorescence', dye_max_emission=509, dye_max_excitation=488, dye_id='McNamara-Boswell-0828', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00FF5B', original_color='#FF00FF5B', color_mode=None, palette_name=None, gamma=None, low=0.004196231021591516, high=0.1739833676661326, is_selected=None)
CZIChannel(id=3, name='DAPI', bit_count_range=16, pixel_type='Gray16', dye_name='DAPI', short_name='DAPI', illumination_type='Fluorescence', dye_max_emission=465, dye_max_excitation=353, dye_id='McNamara-Boswell-0434', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00A0FF', original_color='#FF00A0FF', color_mode=None, palette_name=None, gamma=None, low=0.003936827649347677, high=0.15408560311284047, is_selected=None)

Comparison Result

The package contains only two important APIs, as shown above. To install it, run pip install czi-shader.

通过原始数据重建 Zeiss Zen 中所见的图像观感

TL; DR

我写了个包 (czi-shader) 用来做标题所说的事情

前言

实验室的切片几乎都是用 Zeiss 显微镜扫出来的, 之前要在 Zen 外使用 CZI 文件时, 要么是在软件里点导出, 要么是直接通过 Python 读出灰度图然后上伪彩. 但是这东西是怎么在展示的时候给不同的荧光通道上合适的颜色的呢?

原理

经尝试, 从 raw light 到最终展示分两步, 分别是单通道上色、多通道合并

单通道上色

CTB488 通常指定为荧光绿, 其 RGB 饱和色为 (0, 255, 91), 转换到 HSL 空间为 (141, 100%, 50%). 令扫描得到的实际亮度为 $l_r$, 那么最终的表现色为:

$$
\begin{equation}
\begin{aligned}
H &= 141 \
S &= 100% \
L &= \frac{l_r - \text{low}}{\text{high} - l_r} * 50%
\end{aligned}
\end{equation}
$$

那么对于每个通道都如此上色, 可以得到多个 (h, w, 3) 大小的矩阵, 此处的 3 是 HSL 色彩空间的三个分量.

多通道合并

Zen 合并是在 RGB 空间进行的, 将上述 HSL 空间的图片转为 RGB, 之后相加再 clip 的, 等价的 numpy 代码如下:

1
2
3
4
merged_image = np.clip(
np.sum(rgb_imgs, axis=0),
0, 255
)

实际情况中需要考虑求和溢出

通道信息

现在有了计算方法, 那么关于通道的原始数据在哪里呢? 根据尝试, 在 czi meta 的 .//DisplaySetting/Channels/, 一份样例数据为:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<Channel Id="Channel:2" Name="EGFP">
<Low>0.0043691845281902137</Low>
<High>0.14451909533769555</High>
<BitCountRange>16</BitCountRange>
<PixelType>Gray16</PixelType>
<DyeName>EGFP</DyeName>
<ShortName>EGFP</ShortName>
<IlluminationType>Fluorescence</IlluminationType>
<DyeMaxEmission>509</DyeMaxEmission>
<DyeMaxExcitation>488</DyeMaxExcitation>
<DyeId>McNamara-Boswell-0828</DyeId>
<DyeDatabaseId>66071726-cbd4-4c41-b371-0a6eee4ae9c5</DyeDatabaseId>
<Color>#FF00FF5B</Color>
<ColorMode>Palette</ColorMode>
<OriginalColor>#FF00FF5B</OriginalColor>
<PaletteName>HeatMap</PaletteName>
<IsSelected>false</IsSelected>
</Channel>

数据格式不言自明, 不过解析这个 XML 转换成合适的数据、对 raw light 上色还是有不少重复之事的, 因此我写了一个 Python 包, 你可以直接操作.

1
2
3
4
5
6
7
8
import cv2
from czi_shader import CZIChannel, shading_czi

p = '/mnt/inner-data/sc-C057-146-O4213.czi'
[print(c) for c in CZIChannel.from_czi(p)]

res = shading_czi(p, scale_factor=0.01)
cv2.imwrite(p + '.png', cv2.cvtColor(res, cv2.COLOR_RGB2BGR))
1
2
3
4
CZIChannel(id=0, name='Cy5', bit_count_range=16, pixel_type='Gray16', dye_name='Cy5', short_name='Cy5', illumination_type='Fluorescence', dye_max_emission=673, dye_max_excitation=650, dye_id='McNamara-Boswell-0774', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFF0014', original_color='#FFFF0014', color_mode=None, palette_name=None, gamma=None, low=0.0059662775616083005, high=0.03865110246433204, is_selected=None)
CZIChannel(id=1, name='Cy3', bit_count_range=16, pixel_type='Gray16', dye_name='Cy3', short_name='Cy3', illumination type='Fluorescence', dye_max_emission=561, dye_max_excitation=548, dye_id='McNamara-Boswell-0615', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFFAD00', original_color='#FFFFAD00', color_mode=None, palette_name=None, gamma=None, low=0.006240939955748837, high=0.13965056839856566, is_selected=None)
CZIChannel(id=2, name='EGFP', bit_count_range=16, pixel_type='Gray16', dye_name='EGFP', short_name='EGFP', illumination_type='Fluorescence', dye_max_emission=509, dye_max_excitation=488, dye_id='McNamara-Boswell-0828', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00FF5B', original_color='#FF00FF5B', color_mode=None, palette_name=None, gamma=None, low=0.004196231021591516, high=0.1739833676661326, is_selected=None)
CZIChannel(id=3, name='DAPI', bit_count_range=16, pixel_type='Gray16', dye_name='DAPI', short_name='DAPI', illumination_type='Fluorescence', dye_max_emission=465, dye_max_excitation=353, dye_id='McNamara-Boswell-0434', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00A0FF', original_color='#FF00A0FF', color_mode=None, palette_name=None, gamma=None, low=0.003936827649347677, high=0.15408560311284047, is_selected=None)

对比结果

包里只有两个重要 API, 都展示在上面了, 要安装, 则运行 pip install czi-shader