2024-02-01发表2024-02-02更新13 分钟读完 (大约1975个字)

MATALB R2023a for Windows:

magnet:?xt=urn:btih:a78d6cbf64e91568e8a2a7a4bb94ca2ea532e685&dn=MathWorks%20MATLAB%20R2023a%20v9.14.0.2206163%20%5BFileCR%5D&tr=https%3A%2F%2Ftracker1.520.jp%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker-udp.gbitt.info%3A80%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.moeking.me%3A6969%2Fannounce&tr=udp%3A%2F%2Fuploads.gamecoast.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker1.bt.moack.co.kr%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.4.babico.name.tr%3A3131%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=http%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fsanincode.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker01-msk-virt.corbina.net%3A80%2Fannounce&tr=udp%3A%2F%2Fprivate.anonseed.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=udp%3A%2F%2Fmovies.zsw.ca%3A6969%2Fannounce&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce

MATLAB R2023b for Linux:

magnet:?xt=urn:btih:56f878ec3c9fa85c4e137dbf48f9094eed99a30d&dn=MathWorks%20MATLAB%20R2023b%20v23.2.0.2459199%20Update%205_x64_LINUX&tr=http%3a%2f%2ftracker.ipv6tracker.ru%3a80%2fannounce&tr=udp%3a%2f%2ftracker.cubonegro.lol%3a6969%2fannounce&tr=http%3a%2f%2ftracker.renfei.net%3a8080%2fannounce&tr=udp%3a%2f%2f1c.premierzal.ru%3a6969%2fannounce&tr=udp%3a%2f%2f49.12.76.8%3a8080%2fannounce&tr=udp%3a%2f%2f91.216.110.52%3a451%2fannounce&tr=udp%3a%2f%2f%5b2a01%3a4f8%3ac012%3a8025%3a%3a1%5d%3a8080%2fannounce&tr=http%3a%2f%2fshare.camoe.cn%3a8080%2fannounce&tr=udp%3a%2f%2ftracker2.itzmx.com%3a6961%2fannounce&tr=http%3a%2f%2ftracker.electro-torrent.pl%3a80%2fannounce&tr=http%3a%2f%2furaniumhexafluori.de%3a1919%2fannounce&tr=udp%3a%2f%2ftracker3.itzmx.com%3a6961%2fannounce&tr=udp%3a%2f%2fipv6.fuuuuuck.com%3a6969%2fannounce&tr=udp%3a%2f%2f119.28.71.45%3a8080%2fannounce&tr=udp%3a%2f%2foh.fuuuuuck.com%3a6969%2fannounce&tr=udp%3a%2f%2fbt2.archive.org%3a6969%2fannounce&tr=udp%3a%2f%2fr.l5.ca%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a6969%2fannounce&tr=https%3a%2f%2ftr.burnabyhighstar.com%3a443%2fannounce&tr=udp%3a%2f%2fbt.rer.lol%3a6969%2fannounce&tr=udp%3a%2f%2fopen.tracker.ink%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.auctor.tv%3a6969%2fannounce&tr=udp%3a%2f%2ffree.publictracker.xyz%3a6969%2fannounce&tr=http%3a%2f%2fch3oh.ru%3a6969%2fannounce&tr=udp%3a%2f%2ftk1.trackerservers.com%3a8080%2fannounce&tr=udp%3a%2f%2fconcen.org%3a6969%2fannounce&tr=https%3a%2f%2ftracker.loligirl.cn%3a443%2fannounce&tr=http%3a%2f%2fopen.acgnxtracker.com%3a80%2fannounce&tr=https%3a%2f%2ft1.hloli.org%3a443%2fannounce&tr=http%3a%2f%2fv6-tracker.0g.cx%3a6969%2fannounce&tr=udp%3a%2f%2f207.241.226.111%3a6969%2fannounce&tr=udp%3a%2f%2fmovies.zsw.ca%3a6969%2fannounce&tr=https%3a%2f%2ftr.qfruiti.com%3a443%2fannounce&tr=udp%3a%2f%2fopen.dstud.io%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.filemail.com%3a6969%2fannounce&tr=udp%3a%2f%2fns1.monolithindustries.com%3a6969%2fannounce&tr=https%3a%2f%2fwww.peckservers.com%3a9443%2fannounce&tr=http%3a%2f%2fnyaa.tracker.wf%3a7777%2fannounce&tr=udp%3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fwepzone.net%3a6969%2fannounce&tr=udp%3a%2f%2f207.241.231.226%3a6969%2fannounce&tr=udp%3a%2f%2fblack-bird.ynh.fr%3a6969%2fannounce&tr=udp%3a%2f%2fopen.xxtor.com%3a3074%2fannounce&tr=http%3a%2f%2f%5b2001%3a1b10%3a1000%3a8101%3a0%3a242%3aac11%3a2%5d%3a6969%2fannounce&tr=http%3a%2f%2fbt.okmp3.ru%3a2710%2fannounce&tr=https%3a%2f%2ftracker.tamersunion.org%3a443%2fannounce&tr=udp%3a%2f%2ftracker.mirrorbay.org%3a6969%2fannounce&tr=http%3a%2f%2ftracker.dler.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.anima.nz%3a6969%2fannounce&tr=udp%3a%2f%2ftracker4.itzmx.com%3a2710%2fannounce&tr=udp%3a%2f%2fu4.trakx.crim.ist%3a1337%2fannounce&tr=udp%3a%2f%2fy.paranoid.agency%3a6969%2fannounce&tr=http%3a%2f%2ft.overflow.biz%3a6969%2fannounce&tr=udp%3a%2f%2fsu-data.com%3a6969%2fannounce&tr=http%3a%2f%2ftracker4.itzmx.com%3a2710%2fannounce&tr=udp%3a%2f%2ffe.dealclub.de%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.srv00.com%3a6969%2fannounce&tr=udp%3a%2f%2fmail.artixlinux.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker1.myporn.club%3a9337%2fannounce&tr=udp%3a%2f%2fretracker.lanta.me%3a2710%2fannounce&tr=http%3a%2f%2f49.12.76.8%3a8080%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.com%3a6969%2fannounce&tr=http%3a%2f%2ft.nyaatracker.com%3a80%2fannounce&tr=https%3a%2f%2ftracker.gbitt.info%3a443%2fannounce&tr=udp%3a%2f%2f%5b2a03%3a7220%3a8083%3acd00%3a%3a1%5d%3a451%2fannounce&tr=udp%3a%2f%2fepider.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.edkj.club%3a6969%2fannounce&tr=udp%3a%2f%2fmoonburrow.club%3a6969%2fannounce&tr=https%3a%2f%2f1337.abcvg.info%3a443%2fannounce&tr=http%3a%2f%2ftracker2.itzmx.com%3a6961%2fannounce&tr=http%3a%2f%2faboutbeautifulgallopinghorsesinthegreenpasture.online%3a80%2fannounce&tr=udp%3a%2f%2f%5b2a04%3aac00%3a1%3a3dd8%3a%3a1%3a2710%5d%3a2710%2fannounce&tr=https%3a%2f%2ftr.qfruiti.in%3a443%2fannounce&tr=udp%3a%2f%2fopentracker.io%3a6969%2fannounce&tr=https%3a%2f%2ftracker.ipfsscan.io%3a443%2fannounce&tr=https%3a%2f%2ftracker.imgoingto.icu%3a443%2fannounce&tr=udp%3a%2f%2fthinking.duckdns.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.ddunlimited.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.qu.ax%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrust-demo.com%3a6969%2fannounce&tr=udp%3a%2f%2fretracker.hotplug.ru%3a2710%2fannounce&tr=http%3a%2f%2ftracker.files.fm%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.swateam.org.uk%3a2710%2fannounce&tr=udp%3a%2f%2ftorrents.artixlinux.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.t-rb.org%3a6969%2fannounce&tr=https%3a%2f%2ftracker.netmap.top%3a8443%2fannounce&tr=udp%3a%2f%2fd40969.acod.regrucolo.ru%3a6969%2fannounce&tr=http%3a%2f%2fbvarf.tracker.sh%3a2086%2fannounce&tr=udp%3a%2f%2ftracker.birkenwald.de%3a6969%2fannounce&tr=https%3a%2f%2ftrackers.mlsub.net%3a443%2fannounce&tr=https%3a%2f%2fyolo.liberbear.com%3a443%2fannounce&tr=http%3a%2f%2ft.acg.rip%3a6699%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net%3a6969%2fannounce&tr=https%3a%2f%2ftracker.cloudit.top%3a443%2fannounce&tr=udp%3a%2f%2fnew-line.net%3a6969%2fannounce&tr=http%3a%2f%2ftracker.edkj.club%3a6969%2fannounce&tr=http%3a%2f%2f1337.abcvg.info%3a80%2fannounce&tr=http%3a%2f%2fipv6.rer.lol%3a6969%2fannounce&tr=http%3a%2f%2ftracker.birkenwald.de%3a6969%2fannounce&tr=http%3a%2f%2ftracker.bt4g.com%3a2095%2fannounce&tr=http%3a%2f%2f207.241.226.111%3a6969%2fannounce&tr=https%3a%2f%2fshahidrazi.online%3a443%2fannounce&tr=udp%3a%2f%2f6.pocketnet.app%3a6969%2fannounce&tr=http%3a%2f%2ftracker.gbitt.info%3a80%2fannounce&tr=http%3a%2f%2fwww.peckservers.com%3a9000%2fannounce&tr=udp%3a%2f%2ffh2.cmp-gaming.com%3a6969%2fannounce&tr=udp%3a%2f%2f184.105.151.166%3a6969%2fannounce&tr=udp%3a%2f%2f46.17.46.112%3a8080%2fannounce&tr=udp%3a%2f%2f6ahddutb1ucc3cp.ru%3a6969%2fannounce&tr=udp%3a%2f%2f%5b2a00%3ab700%3a1%3a%3a3%3a1dc%5d%3a8080%2fannounce&tr=http%3a%2f%2f%5b2a01%3a4f8%3ac012%3a8025%3a%3a1%5d%3a8080%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.to%3a6969%2fannounce&tr=udp%3a%2f%2f%5b2a0f%3ae586%3af%3af%3a%3a81%5d%3a6969%2fannounce&tr=http%3a%2f%2f%5b2a04%3aac00%3a1%3a3dd8%3a%3a1%3a2710%5d%3a2710%2fannounce&tr=https%3a%2f%2ftracker.gcrreen.xyz%3a443%2fannounce&tr=udp%3a%2f%2fhz.is%3a1337%2fannounce&tr=http%3a%2f%2fwepzone.net%3a6969%2fannounce&tr=udp%3a%2f%2fodd-hd.fr%3a6969%2fannounce&tr=udp%3a%2f%2fbt1.archive.org%3a6969%2fannounce&tr=udp%3a%2f%2fpublic.tracker.vraphim.com%3a6969%2fannounce&tr=https%3a%2f%2ftracker.lilithraws.org%3a443%2fannounce&tr=udp%3a%2f%2ftracker.artixlinux.org%3a6969%2fannounce&tr=wss%3a%2f%2ftracker.openwebtorrent.com%3a443%2fannounce&tr=http%3a%2f%2ftracker.qu.ax%3a6969%2fannounce&tr=http%3a%2f%2fwww.all4nothin.net%3a80%2fannounce.php&tr=http%3a%2f%2ftracker.mywaifu.best%3a6969%2fannounce&tr=http%3a%2f%2f207.241.231.226%3a6969%2fannounce&tr=udp%3a%2f%2fec2-18-191-163-220.us-east-2.compute.amazonaws.com%3a6969%2fannounce&tr=http%3a%2f%2ftorrentsmd.com%3a8080%2fannounce&tr=udp%3a%2f%2fmartin-gebhardt.eu%3a25%2fannounce&tr=udp%3a%2f%2fbt.ktrackers.com%3a6666%2fannounce&tr=http%3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fwww.torrent.eu.org%3a451%2fannounce&tr=http%3a%2f%2f%5b2a00%3ab700%3a1%3a%3a3%3a1dc%5d%3a8080%2fannounce&tr=https%3a%2f%2ftracker.kuroy.me%3a443%2fannounce&tr=udp%3a%2f%2fopentor.org%3a2710%2fannounce&tr=udp%3a%2f%2f%5b2001%3a470%3a1%3a189%3a0%3a1%3a2%3a3%5d%3a6969%2fannounce&tr=udp%3a%2f%2f52.58.128.163%3a6969%2fannounce&tr=udp%3a%2f%2ftamas3.ynh.fr%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.cyberia.is%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.picotorrent.one%3a6969%2fannounce&tr=udp%3a%2f%2fttk2.nbaonlineservice.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.tryhackx.org%3a6969%2fannounce&tr=https%3a%2f%2ftracker.renfei.net%3a443%2fannounce&tr=udp%3a%2f%2fapi.alarmasqueretaro.com%3a3074%2fannounce&tr=udp%3a%2f%2fopen.u-p.pw%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=http%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.fnix.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.0x7c0.com%3a6969%2fannounce&tr=udp%3a%2f%2fu6.trakx.crim.ist%3a1337%2fannounce&tr=https%3a%2f%2ftracker.lilithraws.cf%3a443%2fannounce&tr=http%3a%2f%2f46.17.46.112%3a8080%2fannounce&tr=udp%3a%2f%2fryjer.com%3a6969%2fannounce&tr=https%3a%2f%2ftracker.yemekyedim.com%3a443%2fannounce&tr=udp%3a%2f%2fevan.im%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.skyts.net%3a6969%2fannounce&tr=udp%3a%2f%2fjutone.com%3a6969%2fannounce&tr=http%3a%2f%2fretracker.hotplug.ru%3a2710%2fannounce&tr=http%3a%2f%2ftracker3.itzmx.com%3a6961%2fannounce&tr=udp%3a%2f%2faegir.sexy%3a6969%2fannounce&tr=udp%3a%2f%2f%5b2001%3a1b10%3a1000%3a8101%3a0%3a242%3aac11%3a2%5d%3a6969%2fannounce&tr=http%3a%2f%2fwww.wareztorrent.com%3a80%2fannounce&tr=https%3a%2f%2ftracker1.520.jp%3a443%2fannounce&tr=https%3a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2fopen.stealth.si%3a80%2fannounce&tr=udp%3a%2f%2fexodus.desync.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fannounce&tr=udp%3a%2f%2fuploads.gamecoast.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.4.babico.name.tr%3a3131%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2fsanincode.com%3a6969%2fannounce&tr=udp%3a%2f%2fretracker01-msk-virt.corbina.net%3a80%2fannounce&tr=udp%3a%2f%2fprivate.anonseed.com%3a6969%2fannounce&tr=udp%3a%2f%2fp4p.arenabg.com%3a1337%2fannounce&tr=http%3a%2f%2fmovies.zsw.ca%3a6969%2fannounce

2024-01-18发表2024-01-18更新6 分钟读完 (大约930个字)

注入代码到 steam UI 及禁用 steam 更新检查

当前 steam 版本: 8.63.11.84, 更新于 2024年1月13日，8:58:56

注意到 steam 的 UI 是网页格式的, 检查后确定其使用 CEF 展示, 在 steam 根目录游走发现steamui\localization下放了 steam 的本地化文件, 且steamui\chunk~2dcc5aaf7.js是入口文件.

尝试随机修改一些翻译后触发了 steam 的更新, 更新后相关文件复原. 注意到 logs 下有许多日志, bootstrap_log.txt描述了启动流程. 当一切正常时, 最后几行如下:

1
2
3

[2024-01-18 19:40:21] 正在验证安装...
[2024-01-18 19:40:21] Performing checksum verification of executable files
[2024-01-18 19:40:22] Verification complete

修改任何关键文件后, 则会有如下类似输出:

[2024-01-18 19:43:16] Downloaded new manifest: /steam_client_win32 version 1705108172, installed version 1705108172, existing pending version 0
[2024-01-18 19:43:16] Package file bins_cef_win32_win7.zip.vz.314ded663999d66f03023b3a12690d93d9d34974_66447549 missing or incorrect size
[2024-01-18 19:43:16] Package file bins_webhelpers_win32_win7.zip.vz.fe15dbe6e1b60812fbaee813884270a7b58aac0e_2518032 missing or incorrect size
[2024-01-18 19:43:16] Add pending download: https://client-update.akamai.steamstatic.com/bins_cef_win32_win7.zip.vz.314ded663999d66f03023b3a12690d93d9d34974_66447549
[2024-01-18 19:43:16] Manifest download: send request
[2024-01-18 19:43:16] Add pending download: https://client-update.akamai.steamstatic.com/bins_webhelpers_win32_win7.zip.vz.fe15dbe6e1b60812fbaee813884270a7b58aac0e_2518032
[2024-01-18 19:43:16] Manifest download: send request
[2024-01-18 19:43:16] 正在下载更新 (已下载 0，共 67,349 KB)...
...省略进度行
[2024-01-18 19:43:23] 正在下载更新 (已下载 35,320，共 67,349 KB)...

注意到大部分 bootstrap 里的输出都是英文的, 而正在验证安装是中文的, 那么肯定在某个地方放置了翻译文件. 全局搜索可知public\steambootstrapper_schinese.txt下有一些启动时翻译文件, 找到相关行如下:

1
2
3

"SteamBootstrapper_UpdateInstalling"			"正在安装更新..."
"SteamBootstrapper_InstallVerify"				"正在验证安装..."
"SteamBootstrapper_DownloadComplete"			"下载完成。"

遂全局搜索SteamBootstrapper_InstallVerify发现在steam.exe中有引用, 使用 IDA Pro 打开 steam.exe.

在 String 视图中有:

Alt text

循其引用, 只见其一, 所在片段如下:

Alt text

那么sub_466120肯定是一个翻译函数, 随意修改 steam 一些字节, 重新运行可见:

...
[2024-01-18 20:23:35] 正在验证安装...
[2024-01-18 20:23:36] BVerifyInstalledFiles: bad CRC on steam.exe (da921668 expected, ef3173b5 actual)
....

那么只要把验证函数空置, 应该就能处理掉不少问题, 搜索 BVerifyInstalledFiles 可见:

Address	Length	Type	String
.rdata:006E729C	00000028	C	BVerifyInstalledFiles: bad directory %s
.rdata:006E72C4	00000037	C	BVerifyInstalledFiles: %s is %lld bytes, expected %lld
.rdata:006E72FC	00000026	C	BVerifyInstalledFiles: can't read %s\n
.rdata:006E7324	0000003F	C	BVerifyInstalledFiles: bad CRC on %s (%x expected, %x actual)\n

第一个是处理错误目录的, 第二个是检查长度的, 第三个是报错不能读取的, 第四个是检查文件 CRC 的.

打开此函数反汇编伪代码, 前几行是:

v4 = (int)this;
v76 = this;
sub_457870(v47);
v77 = 0;
sub_4616F0(v46, 778, v4, 1);
if ( !(_BYTE)a2 && !(unsigned __int8)sub_49A920(v46) )
{
  v5 = 1;
  goto LABEL_87;
}

LABEL_87在尾巴上, 接近返回, 大概就是兜底. 因此反转此条件. 将两个jnz改成jz, 之后修改翻译文件, 将Login_PickUser改做:

1	"Login_PickUser":"谁要玩游戏 Hello World？"

启动 steam 可见:

Alt text

被糊掉的是我的头像

至此修改成功. 查看 Login_PickUser 附近代码有:

Alt text

这附近引用到了一个s.Config, 导出看看:

Alt text

现在可以一键替换「steam」为「斯迪姆」了

2023-09-10发表2024-06-04更新4 分钟读完 (大约637个字)

Reproducting Image Perception in Zeiss Zen from Raw Data

TL; DR

I’ve created a package (czi-shader) to achieve what is mentioned in the title.

Introduction

Most of the lab’s slices are scanned using Zeiss microscopes. Previously, when working with CZI files outside Zen, you either had to export them within the software or directly convert them to pseudocolor using Python. But how are the appropriate colors assigned to different fluorescence channels when displaying the images?

Principles

After experimentation, the process of going from raw light to the final display can be divided into two steps: single-channel coloring and multi-channel merging.

Single-Channel Coloring

CTB488 is usually specified as fluorescence green, with RGB saturation values of (0, 255, 91), which translates to (141, 100%, 50%) in the HSL color space. Let the actual brightness obtained from scanning be denoted as $l_r$. The resulting color is calculated as follows:

$$
\begin{equation}
\begin{aligned}
H &= 141 \
S &= 100% \
L &= \frac{l_r - \text{low}}{\text{high} - l_r} * 50%
\end{aligned}
\end{equation}
$$

This process is applied to each channel, resulting in multiple (h, w, 3) matrices, where 3 represents the three components of the HSL color space.

Multi-Channel Merging

Zen’s merging is done in the RGB space. The HSL space images obtained above are converted to RGB and then summed and clipped. The equivalent numpy code is as follows:

merged_image = np.clip(
    np.sum(rgb_imgs, axis=0), 
    0, 255
)

In practice, overflow needs to be considered.

Channel Information

Now that we have the calculation method, where can we find the raw data for the channels? Based on experiments, in the CZI metadata under .//DisplaySetting/Channels/, a sample data entry looks like this:

<Channel Id="Channel:2" Name="EGFP">
    <Low>0.0043691845281902137</Low>
    <High>0.14451909533769555</High>
    <BitCountRange>16</BitCountRange>
    <PixelType>Gray16</PixelType>
    <DyeName>EGFP</DyeName>
    <ShortName>EGFP</ShortName>
    <IlluminationType>Fluorescence</IlluminationType>
    <DyeMaxEmission>509</DyeMaxEmission>
    <DyeMaxExcitation>488</DyeMaxExcitation>
    <DyeId>McNamara-Boswell-0828</DyeId>
    <DyeDatabaseId>66071726-cbd4-4c41-b371-0a6eee4ae9c5</DyeDatabaseId>
    <Color>#FF00FF5B</Color>
    <ColorMode>Palette</ColorMode>
    <OriginalColor>#FF00FF5B</OriginalColor>
    <PaletteName>HeatMap</PaletteName>
    <IsSelected>false</IsSelected>
</Channel>

The data format is self-explanatory, but parsing this XML and converting it into appropriate data for coloring the raw light can be repetitive. Therefore, I have developed a Python package that allows you to perform these operations directly.

import cv2
from czi_shader import CZIChannel, shading_czi

p = '/mnt/inner-data/sc-C057-146-O4213.czi'
[print(c) for c in CZIChannel.from_czi(p)]

res = shading_czi(p, scale_factor=0.01)
cv2.imwrite(p + '.png', cv2.cvtColor(res, cv2.COLOR_RGB2BGR))

CZIChannel(id=0, name='Cy5', bit_count_range=16, pixel_type='Gray16', dye_name='Cy5', short_name='Cy5', illumination_type='Fluorescence', dye_max_emission=673, dye_max_excitation=650, dye_id='McNamara-Boswell-0774', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFF0014', original_color='#FFFF0014', color_mode=None, palette_name=None, gamma=None, low=0.0059662775616083005, high=0.03865110246433204, is_selected=None)
CZIChannel(id=1, name='Cy3', bit_count_range=16, pixel_type='Gray16', dye_name='Cy3', short_name='Cy3', illumination type='Fluorescence', dye_max_emission=561, dye_max_excitation=548, dye_id='McNamara-Boswell-0615', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFFAD00', original_color='#FFFFAD00', color_mode=None, palette_name=None, gamma=None, low=0.006240939955748837, high=0.13965056839856566, is_selected=None)
CZIChannel(id=2, name='EGFP', bit_count_range=16, pixel_type='Gray16', dye_name='EGFP', short_name='EGFP', illumination_type='Fluorescence', dye_max_emission=509, dye_max_excitation=488, dye_id='McNamara-Boswell-0828', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00FF5B', original_color='#FF00FF5B', color_mode=None, palette_name=None, gamma=None, low=0.004196231021591516, high=0.1739833676661326, is_selected=None)
CZIChannel(id=3, name='DAPI', bit_count_range=16, pixel_type='Gray16', dye_name='DAPI', short_name='DAPI', illumination_type='Fluorescence', dye_max_emission=465, dye_max_excitation=353, dye_id='McNamara-Boswell-0434', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00A0FF', original_color='#FF00A0FF', color_mode=None, palette_name=None, gamma=None, low=0.003936827649347677, high=0.15408560311284047, is_selected=None)

Comparison Result

The package contains only two important APIs, as shown above. To install it, run pip install czi-shader.

2023-09-10发表2023-09-10更新5 分钟读完 (大约754个字)

通过原始数据重建 Zeiss Zen 中所见的图像观感

TL; DR

我写了个包 (czi-shader) 用来做标题所说的事情

前言

实验室的切片几乎都是用 Zeiss 显微镜扫出来的, 之前要在 Zen 外使用 CZI 文件时, 要么是在软件里点导出, 要么是直接通过 Python 读出灰度图然后上伪彩. 但是这东西是怎么在展示的时候给不同的荧光通道上合适的颜色的呢?

原理

经尝试, 从 raw light 到最终展示分两步, 分别是单通道上色、多通道合并

单通道上色

CTB488 通常指定为荧光绿, 其 RGB 饱和色为 (0, 255, 91), 转换到 HSL 空间为 (141, 100%, 50%). 令扫描得到的实际亮度为 $l_r$, 那么最终的表现色为:

$$
\begin{equation}
\begin{aligned}
H &= 141 \
S &= 100% \
L &= \frac{l_r - \text{low}}{\text{high} - l_r} * 50%
\end{aligned}
\end{equation}
$$

那么对于每个通道都如此上色, 可以得到多个 (h, w, 3) 大小的矩阵, 此处的 3 是 HSL 色彩空间的三个分量.

多通道合并

Zen 合并是在 RGB 空间进行的, 将上述 HSL 空间的图片转为 RGB, 之后相加再 clip 的, 等价的 numpy 代码如下:

merged_image = np.clip(
    np.sum(rgb_imgs, axis=0), 
    0, 255
)

实际情况中需要考虑求和溢出

通道信息

现在有了计算方法, 那么关于通道的原始数据在哪里呢? 根据尝试, 在 czi meta 的 .//DisplaySetting/Channels/, 一份样例数据为:

<Channel Id="Channel:2" Name="EGFP">
    <Low>0.0043691845281902137</Low>
    <High>0.14451909533769555</High>
    <BitCountRange>16</BitCountRange>
    <PixelType>Gray16</PixelType>
    <DyeName>EGFP</DyeName>
    <ShortName>EGFP</ShortName>
    <IlluminationType>Fluorescence</IlluminationType>
    <DyeMaxEmission>509</DyeMaxEmission>
    <DyeMaxExcitation>488</DyeMaxExcitation>
    <DyeId>McNamara-Boswell-0828</DyeId>
    <DyeDatabaseId>66071726-cbd4-4c41-b371-0a6eee4ae9c5</DyeDatabaseId>
    <Color>#FF00FF5B</Color>
    <ColorMode>Palette</ColorMode>
    <OriginalColor>#FF00FF5B</OriginalColor>
    <PaletteName>HeatMap</PaletteName>
    <IsSelected>false</IsSelected>
</Channel>

数据格式不言自明, 不过解析这个 XML 转换成合适的数据、对 raw light 上色还是有不少重复之事的, 因此我写了一个 Python 包, 你可以直接操作.

import cv2
from czi_shader import CZIChannel, shading_czi

p = '/mnt/inner-data/sc-C057-146-O4213.czi'
[print(c) for c in CZIChannel.from_czi(p)]

res = shading_czi(p, scale_factor=0.01)
cv2.imwrite(p + '.png', cv2.cvtColor(res, cv2.COLOR_RGB2BGR))

CZIChannel(id=0, name='Cy5', bit_count_range=16, pixel_type='Gray16', dye_name='Cy5', short_name='Cy5', illumination_type='Fluorescence', dye_max_emission=673, dye_max_excitation=650, dye_id='McNamara-Boswell-0774', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFF0014', original_color='#FFFF0014', color_mode=None, palette_name=None, gamma=None, low=0.0059662775616083005, high=0.03865110246433204, is_selected=None)
CZIChannel(id=1, name='Cy3', bit_count_range=16, pixel_type='Gray16', dye_name='Cy3', short_name='Cy3', illumination type='Fluorescence', dye_max_emission=561, dye_max_excitation=548, dye_id='McNamara-Boswell-0615', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FFFFAD00', original_color='#FFFFAD00', color_mode=None, palette_name=None, gamma=None, low=0.006240939955748837, high=0.13965056839856566, is_selected=None)
CZIChannel(id=2, name='EGFP', bit_count_range=16, pixel_type='Gray16', dye_name='EGFP', short_name='EGFP', illumination_type='Fluorescence', dye_max_emission=509, dye_max_excitation=488, dye_id='McNamara-Boswell-0828', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00FF5B', original_color='#FF00FF5B', color_mode=None, palette_name=None, gamma=None, low=0.004196231021591516, high=0.1739833676661326, is_selected=None)
CZIChannel(id=3, name='DAPI', bit_count_range=16, pixel_type='Gray16', dye_name='DAPI', short_name='DAPI', illumination_type='Fluorescence', dye_max_emission=465, dye_max_excitation=353, dye_id='McNamara-Boswell-0434', dye_database_id='66071726-cbd4-4c41-b371-0a6eee4ae9c5', color='#FF00A0FF', original_color='#FF00A0FF', color_mode=None, palette_name=None, gamma=None, low=0.003936827649347677, high=0.15408560311284047, is_selected=None)

对比结果

包里只有两个重要 API, 都展示在上面了, 要安装, 则运行 pip install czi-shader

2023-08-07发表2024-06-04更新7 分钟读完 (大约1122个字)

Compile numpy without openblas to reduce pack size

Summary

Compiling numpy on Windows by self can save 36 MiB of openblas dependency. Building numpy from source on Windows is not as difficult as imagined.

Preface

I recently needed to package a tool that uses some numpy computations. At first glance with pyinstaller, there is a huge openblas dependency:


36.4 MiB [##########]  libopenblas64__v0.3.23-246-g3d31191b-gcc_10_3_0.dll
10.6 MiB [##        ]  main.exe
 5.5 MiB [#         ]  python311.dll 
 4.9 MiB [#         ] /numpy
 4.8 MiB [#         ] /pydantic_core
 ...

I admire those powerful tools that could do a lot with just a few megabytes in the past. Besides, the performance requirements for numpy in this program are not high. So I tried to remove this dependency. There are many discussions online about packaging and reducing package size, but they only replace mkl with openblas. However, openblas is still quite large.

After searching, this site provides builds without openblas and mkl, but the last update was in 2022, and the version is still 1.22.4, which is a bit old. Since third-party builds already exist, I guessed this might not be too difficult.

Preparation

Compiler

On Windows, just download Microsoft C++ Build Tools, start it and select Desktop development with C++. Wait for it to finish and you are good to go. No need to set environment variables as numpy will handle it later.

Source Code

1
2
3

git clone --recurse-submodules https://github.com/numpy/numpy.git
cd numpy
git checkout maintenance/1.24.x # or 1.25.x, doesn't matter

Disable openblas

In numpy/distutils/, add a file site.cfg with the following content:

[openblas]
libraries =
library_dirs = 
include_dirs =

Refer to site.cfg.example in root for complete description.

According to official docs, you can also disable openblas by setting environment variables.

Virtual environment and dependencies

1
2
3

python -m venv .#env
./.#env/Scripts/activate
pip install -r build_requirements.txt

Later we will run commands in the .#env virtual environment by default.

Build

For <=1.24.x, you can build with bare msvc like this:

1	python setup.py build -j 16

However, for 1.25+, this will silently fail on Windows by generating compilation commands longer than 32768 lines.

A viable solution is to switch to using cibuildwheel.

Modify pyproject.toml, find [tool.cibuildwheel], and comment out before-build, before-test, test-command by adding # in front or just delete them. It should look like:

[tool.cibuildwheel]
skip = "cp36-* cp37-* pp37-* *-manylinux_i686 *_ppc64le *_s390x *-musllinux_aarch64"  
build-verbosity = "3"
# before-build = "bash {project}/tools/wheels/cibw_before_build.sh {project}"
# before-test = "pip install -r {project}/test_requirements.txt"
# test-command = "bash {project}/tools/wheels/cibw_test_command.sh {project}"

The before-build script is for downloading openblas. The two test steps don’t work for me.

Then build in PowerShell:

1
2
3

$env:CIBW_BUILD="cp311-win_amd64" # cp311 means CPython3.11, change to other versions if needed
$env:CIBW_ENVIRONMENT="NPY_USE_BLAS_ILP64=0" # Don't know why it doesn't read site.cfg, use env var to emphasize
cibuildwheel --platform windows

If everything goes well, you should see a whl file generated under wheelhouse:

1 2	-rwxrwxrwx 1 root root 6.1M Aug 7 23:37 wheelhouse/numpy-1.25.2-cp310-cp310-win_amd64.whl -rwxrwxrwx 1 root root 6.1M Aug 7 23:44 wheelhouse/numpy-1.25.2-cp311-cp311-win_amd64.whl

Test

After installing the whl, check debug info:

> python -c "import numpy as np; np.show_config(); print(np.__version__)"
blas_armpl_info:
  NOT AVAILABLE  
blas_mkl_info:
  NOT AVAILABLE
blas_ssl2_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
accelerate_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE  
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
blas_info:
  NOT AVAILABLE
blas_src_info:
  NOT AVAILABLE
blas_opt_info:
  NOT AVAILABLE
lapack_armpl_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
lapack_ssl2_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE  
atlas_info:
  NOT AVAILABLE
lapack_info:
  NOT AVAILABLE
lapack_src_info:
  NOT AVAILABLE
lapack_opt_info:
  NOT AVAILABLE
numpy_linalg_lapack_lite:
    language = c
    define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3  
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
1.25.2

Very good, no openblas at all. Run tests:

> python runtests.py -v --no-build 

...
=============== 35028 passed, 1100 skipped, 1308 deselected, 29 xfailed, 2 xpassed in 345.30s (0:05:45) ===============

For normal binary installs, the result is:

> python -c "import numpy as np; np.show_config(); print(np.__version__)"
openblas64__info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['openblas\\lib']
blas_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']  
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['openblas\\lib']
openblas64__lapack_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['openblas\\lib']
lapack_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['openblas\\lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2 
    not found = AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
1.25.2

Packaging

# test.py

import numpy as np

print(np.random.rand(5, 5))  
np.show_config()
print(np.__version__)

Without openblas install, pyinstaller test.py gives:

   6.9 MiB [##########] /numpy
   5.5 MiB [#######   ]  python311.dll
   3.9 MiB [#####     ]  test.exe
   3.3 MiB [####      ]  libcrypto-1_1.dll
   1.7 MiB [##        ]  base_library.zip 
   1.1 MiB [#         ]  unicodedata.pyd
 996.0 KiB [#         ]  ucrtbase.dll
 ...
Total disk usage:  26.1 MiB  Apparent size:  25.9 MiB  Items: 79

With default install, the result is:

  36.4 MiB [##########]  libopenblas64__v0.3.23-246-g3d31191b-gcc_10_3_0.dll
   5.5 MiB [#         ]  python311.dll
   4.9 MiB [#         ] /numpy  
   3.9 MiB [#         ]  test.exe
   3.3 MiB [          ]  libcrypto-1_1.dll
   1.7 MiB [          ]  base_library.zip
   1.1 MiB [          ]  unicodedata.pyd
 996.0 KiB [          ]  ucrtbase.dll
 ...
Total disk usage:  60.6 MiB  Apparent size:  60.5 MiB  Items: 81

The difference is about 35 MiB, quite significant. Removing openblas, my tool can fit in a single 15 MiB file. Although it’s still some way from my goal of a few megabytes, this is Python after all, so I’m quite satisfied.

Misc

The commit I built 1.25.x on is ea677928332c37e8052b4d599bf6ee52cf363cf9. Reset to it if yours is different.
My Windows version is 22H2 19045.3271
The pure build takes around 2 minutes on my trashy E5-2678.
Sizes above are from ncdu output.

2023-08-07发表2024-06-04更新9 分钟读完 (大约1396个字)

编译不带加速的 numpy 降低打包分发大小

摘要

在 Windows 中通过自行编译 numpy, 可以省去 36 MiB 的 openblas 依赖. 自行编译 numpy 并没有想象中那么困难.

前言

近日需要打包一个使用了一些 numpy 计算的工具, pyinstaller 起手一看, 一个硕大的 openblas 依赖:

36.4 MiB [##########]  libopenblas64__v0.3.23-246-g3d31191b-gcc_10_3_0.dll
10.6 MiB [##        ]  main.exe
 5.5 MiB [#         ]  python311.dll
 4.9 MiB [#         ] /numpy
 4.8 MiB [#         ] /pydantic_core
 ...

我很崇尚过去那些几兆就能完成很多事情的强力工具, 加之这个程序里对 numpy 性能的要求并不算高, 因此我尝试着去除这个依赖, 网上有很多讨论打包和减小打包大小的, 但无非是把 mkl 换成 openblas, 但是 openblas 也够大的了.

经过检索, 此处提供了不带 openblas 和 mkl 的构建, 但是最后一次更新已是 2022 年, 其版本还在 1.22.4, 有点老了. 既然已有第三方构建, 那我猜此事应该也不会太难.

准备

编译器

在 Windows 下构建 numpy, 只需在此下载Microsoft C++ 生成工具, 启动后勾选使用 C++ 的桌面开发, 等待完成即可. 不需要设置什么环境变量, 后面 numpy 会自己处理.

源代码

1
2
3

git clone --recurse-submodules https://github.com/numpy/numpy.git
cd numpy
git checkout maintenance/1.24.x # 或者 1.25.x 随便

关闭 openblas

在numpy/distutils/添加一个文件site.cfg, 内容为:

[openblas]
libraries =
library_dirs =
include_dirs =

此文件完整描述可参考根目录的site.cfg.example.

按照官方文档, 也可以通过设置环境变量来关闭

虚拟环境和依赖包

1
2
3

python -m venv .#env
./.#env/Scripts/activate
pip install -r build_requirements.txt

之后默认在 .#env 的虚拟环境中执行命令.

构建

在 <=1.24.x 时, 可以裸 msvc 构建, 如下:

1	python setup.py build -j 16

但是 1.25+ 之后, 这种方式在 Windows 会生成单行超过 32768 的编译命令, 然后静默失败.

可行的方案是换用 cibuildwheel.

修改 pyproject.toml, 找到[tool.cibuildwheel], 在before-build, before-test, test-command 这三行前面都加上#, 或者删掉也行. 结果如下:

[tool.cibuildwheel]
skip = "cp36-* cp37-* pp37-* *-manylinux_i686 *_ppc64le *_s390x *-musllinux_aarch64"
build-verbosity = "3"
# before-build = "bash {project}/tools/wheels/cibw_before_build.sh {project}"
# before-test = "pip install -r {project}/test_requirements.txt"
# test-command = "bash {project}/tools/wheels/cibw_test_command.sh {project}"

before-build 的脚本是用来下载 openblas 的, 后面的两个 test 我这边运行不成.

之后在 PowerShell 中构建:

1
2
3

$env:CIBW_BUILD="cp311-win_amd64" # cp311 意为 CPython3.11, 你可以改成其他版本
$env:CIBW_ENVIRONMENT="NPY_USE_BLAS_ILP64=0" # 我不知道为什么后面的流程不吃 site.cfg, 这里再给一个环境变量强调下
cibuildwheel --platform windows

如果一切正常, 应当在 wheelhouse 目录下生成一个 whl 文件, 就是我们所要的了.

1 2	-rwxrwxrwx 1 root root 6.1M Aug 7 23:37 wheelhouse/numpy-1.25.2-cp310-cp310-win_amd64.whl -rwxrwxrwx 1 root root 6.1M Aug 7 23:44 wheelhouse/numpy-1.25.2-cp311-cp311-win_amd64.whl

测试

安装 whl 后看看调试输出:

> python -c "import numpy as np; np.show_config(); print(np.__version__)"
blas_armpl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
blas_ssl2_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
accelerate_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
blas_info:
  NOT AVAILABLE
blas_src_info:
  NOT AVAILABLE
blas_opt_info:
  NOT AVAILABLE
lapack_armpl_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
lapack_ssl2_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_info:
  NOT AVAILABLE
lapack_src_info:
  NOT AVAILABLE
lapack_opt_info:
  NOT AVAILABLE
numpy_linalg_lapack_lite:
    language = c
    define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
1.25.2

很好, 完全没有 openblas, 跑一下 tests:

> python runtests.py -v --no-build

...
=============== 35028 passed, 1100 skipped, 1308 deselected, 29 xfailed, 2 xpassed in 345.30s (0:05:45) ===============

如果是普通的二进制安装, 则结果为

> python -c "import numpy as np; np.show_config(); print(np.__version__)"
openblas64__info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['openblas\\lib']
blas_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['openblas\\lib']
openblas64__lapack_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['openblas\\lib']
lapack_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['openblas\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['openblas\\lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
1.25.2

打包

# test.py

import numpy as np

print(np.random.rand(5, 5))
np.show_config()
print(np.__version__)

在无 openblas 安装中, pyinstaller test.py 得到结果:

   6.9 MiB [##########] /numpy
   5.5 MiB [#######   ]  python311.dll
   3.9 MiB [#####     ]  test.exe
   3.3 MiB [####      ]  libcrypto-1_1.dll
   1.7 MiB [##        ]  base_library.zip
   1.1 MiB [#         ]  unicodedata.pyd
 996.0 KiB [#         ]  ucrtbase.dll
 ...
Total disk usage:  26.1 MiB  Apparent size:  25.9 MiB  Items: 79

在默认安装中得到结果:

  36.4 MiB [##########]  libopenblas64__v0.3.23-246-g3d31191b-gcc_10_3_0.dll
   5.5 MiB [#         ]  python311.dll
   4.9 MiB [#         ] /numpy
   3.9 MiB [#         ]  test.exe
   3.3 MiB [          ]  libcrypto-1_1.dll
   1.7 MiB [          ]  base_library.zip
   1.1 MiB [          ]  unicodedata.pyd
 996.0 KiB [          ]  ucrtbase.dll
 ...
Total disk usage:  60.6 MiB  Apparent size:  60.5 MiB  Items: 81

差不多差了 35 MiB, 还是很可观的. 去掉 openblas, 我的工具单文件 15 MiB 就够了, 虽然离心目中的几兆还有些距离, 但这毕竟是 Python, 我很满足了.

拾遗

我编译 1.25.x 时的 HEAD 是 ea677928332c37e8052b4d599bf6ee52cf363cf9, 如果有哪里不同, 可以git reset ea677928332c37e8052b4d599bf6ee52cf363cf9 过来
我的 Windows 版本是 22H2 19045.3271
你需要准备一把顺畅的梯子, cibuildwheel 会需要从 Python 官方网站下载全新的 Python
在我的洋垃圾 E5-2678 下纯编译过程需要两分钟左右
上面的大小是用 ncdu 输出的

2023-05-27发表2023-05-27更新2 分钟读完 (大约287个字)

2023 年从 Ghost 迁移到 Hexo

此处博客系统原是 Ghost, 是我很久以前选的, 现在越来越臃肿, 每次升级胆颤心惊, 放置自己的静态文件也麻烦, 遂生了迁移的想法. 网上一搜, 几乎都是五六七八年前的迁移脚本和文章, 而当下 Ghost 都是导出 plain-text 而没有格式了. 但是幸亏在 Ghost 官方有一个教程:「使用 Ghost 作为无头 CMS 启动站点，并使用静态站点生成器 Hexo 构建完全自定义的前端」, 按教程随意尝试后, 注意到在 source/_posts/ 生成了所有带格式的 md 文件.

至此就大概完成了, 不过仔细研读后发现还有个问题:

表征代码块时用了<pre><code></code></pre>而不是``, 导致行间距混乱和折行错误.
表征引用块时用了<blockquote></blockquote>而不是>, 亦致行间距混乱.

我遂写了脚本, 通过 Ghost 教程生成 md 文件后, 将脚本放置到source下, 运行即原地替换文件.

脚本可以在这里下载: trim-ghost.py.py, 也可见 gist:

2022-10-03发表2024-06-04更新3 分钟读完 (大约399个字)

使用 Python+AT 指令接收并解码中文短信

我手上的设备型号是「EC600N」, 使用「联通4G」卡, 查看短信流程如下:

> AT+CMGF=1
OK # 这一句用于设置读取短信时为文本模式, 由于未知原因, 在我的设备上PDU模式总是空的
> AT+CMGL="ALL"
+CMGL: 0,"REC UNREAD","180********",,"22/10/03,11:35:58+32"
4E2D658777ED4FE16D4B8BD5002000680065006C006C006F00200077006F0072006C0064

OK # 这一句用于查阅所有短信, 无论已读未读
> AT+CMGD=1,4
OK # 这一句用于删除储存1里的所有短信, 阅后即焚

收到的短信格式是这样的:

> AT+CMGL="ALL"
+CMGL: 0,"REC UNREAD","180********",,"22/10/03,11:35:58+32"
4E2D658777ED4FE16D4B8BD5002000680065006C006C006F00200077006F0072006C0064

OK

这串字符是「utf-16-be」编码的十六进制表示, 在Python里应当使用如下解读方式:

1
2
3

>>> line = '4E2D658777ED4FE16D4B8BD5002000680065006C006C006F00200077006F0072006C0064'
>>> bytes.fromhex(line).decode('utf-16-be')
中文短信测试 hello world

综合来讲, 我的代码是这样的:

async def main():
    ec600n = ES600N()
    await ec600n.write('AT')
    assert await ec600n.wait_ok()

    await ec600n.write_wait_ok('AT+CMGF=1')
    res = await ec600n.write_wait_ok('AT+CMGL="ALL"')
    for line in res.splitlines(keepends=False):
        if not re.match("^[0-9A-F]+$", line):
            print(line)
        else:
            print(bytes.fromhex(line).decode('utf-16-be'))
    
    inputs = input('删除所有短信吗? [yes/NO]\n')
    if inputs.lower() == 'yes':
        print(await ec600n.write_wait_ok('AT+CMGD=1,4'))

运行结果是这样的:

$ python 收短信.py
+CMGL: 0,"REC UNREAD","180********",,"22/10/03,11:35:58+32"
中文短信测试 hello world

OK
删除所有短信吗? [yes/NO]

$

2022-07-23发表2024-06-04更新1 分钟读完 (大约177个字)

Dynamic enum with FastApi

Sometimes you need dynamic enumerations, such as some selector options from database, but you will find that it seems like an the enumeration can only be modified at app startup. Try this:

myuan@jxtkfuwuqi ~> curl 10.242.155.222:8000
{"detail":[{"loc":["query","e"],"msg":"field required","type":"value_error.missing"}]}⏎
myuan@jxtkfuwuqi ~> curl 10.242.155.222:8000/rand-enum
{"message":{"0-7":"value-85","1-8":"value-6","2-5":"value-10","3-5":"value-99","4-4":"value-14"}}⏎       
myuan@jxtkfuwuqi ~> curl '10.242.155.222:8000?e=value-10'
{"e":"value-10"}⏎
myuan@jxtkfuwuqi ~> curl '10.242.155.222:8000?e=value-9'
{"detail":[{"loc":["query","e"],"msg":"value is not a valid enumeration member; permitted: 'value-85', 'value-6', 'value-10', 'value-99', 'value-14'","type":"type_error.enum","ctx":{"enum_values":["value-85","value-6","value-10","value-99","value-14"]}}]}⏎
myuan@jxtkfuwuqi ~> curl '10.242.155.222:8000?e=value-11'
{"detail":[{"loc":["query","e"],"msg":"value is not a valid enumeration member; permitted: 'value-85', 'value-6', 'value-10', 'value-99', 'value-14'","type":"type_error.enum","ctx":{"enum_values":["value-85","value-6","value-10","value-99","value-14"]}}]}⏎

Enumeration constraints are still in effect while the enumeration is modified

2022-07-20发表2024-03-08更新6 分钟读完 (大约866个字)

让 docker easyconnect 与 Clash 和平共处

近日尝试将了https://github.com/Hagb/docker-easyconnect, 但是我不想每次想要访问内网时还手动切换代理, 这简直比原来用 Windows 客户端还糟糕. 于是我找到了 Clash.

Clash 是一个基于规则的隧道转发器, 自然也能支持我们分流内网/国内网络/墙外网络, 最初我尝试修改飞机场提供的配置文件, 但是机场的配置文件会每天更新, 于是翻阅了文档后发现了proxy-providers这个功能

于是基本实现原理为: 将飞机场配置作为一个代理提供器, 在代理中添加来自 easyconnect 的代理, 首先匹配内网网段转发到 easyconnect, 之后按顺序匹配到机场. Clash 配置如下:

点击展开完整配置

mode: Script
mixed-port: 7890
external-controller: 127.0.0.1:19953
secret: 4616077c-8348-40f7-b29e-dd3aeb78595f
proxy-providers:
  机场提供器:
    type: http
    path: "./ruleset/westworld.yaml"
    url: "<你的机场订阅url>"
    health-check:
      enable: true
      url: http://www.gstatic.com/generate_204
      interval: 300

proxies:
  - { 
      type: socks5, name: "easyconnect_vpn", server: "<你的easyconnect地址>", port: <端口>;
    }
proxy-groups:
  - { name: 机场, type: url-test, use: [机场提供器] }
rules:
  - "IP-CIDR,10.20.0.0/16,easyconnect_vpn"
  - "IP-CIDR,10.10.0.0/16,easyconnect_vpn"
  - RULE-SET,applications,DIRECT
  - DOMAIN,clash.razord.top,DIRECT
  - DOMAIN,yacd.haishan.me,DIRECT
  - RULE-SET,private,DIRECT
  - RULE-SET,reject,REJECT
  - RULE-SET,google,DIRECT
  - RULE-SET,proxy,机场
  - RULE-SET,direct,DIRECT
  - RULE-SET,lancidr,DIRECT
  - RULE-SET,cncidr,DIRECT
  - RULE-SET,telegramcidr,机场
  - GEOIP,LAN,DIRECT
  - GEOIP,CN,DIRECT
  - MATCH,机场

rule-providers:
  reject:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/reject.txt"
    path: ./ruleset/reject.yaml
    interval: 86400

  icloud:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/icloud.txt"
    path: ./ruleset/icloud.yaml
    interval: 86400

  apple:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/apple.txt"
    path: ./ruleset/apple.yaml
    interval: 86400

  google:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/google.txt"
    path: ./ruleset/google.yaml
    interval: 86400

  proxy:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/proxy.txt"
    path: ./ruleset/proxy.yaml
    interval: 86400

  direct:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/direct.txt"
    path: ./ruleset/direct.yaml
    interval: 86400

  private:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/private.txt"
    path: ./ruleset/private.yaml
    interval: 86400

  gfw:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/gfw.txt"
    path: ./ruleset/gfw.yaml
    interval: 86400

  tld-not-cn:
    type: http
    behavior: domain
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/tld-not-cn.txt"
    path: ./ruleset/tld-not-cn.yaml
    interval: 86400

  telegramcidr:
    type: http
    behavior: ipcidr
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/telegramcidr.txt"
    path: ./ruleset/telegramcidr.yaml
    interval: 86400

  cncidr:
    type: http
    behavior: ipcidr
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/cncidr.txt"
    path: ./ruleset/cncidr.yaml
    interval: 86400

  lancidr:
    type: http
    behavior: ipcidr
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/lancidr.txt"
    path: ./ruleset/lancidr.yaml
    interval: 86400

  applications:
    type: http
    behavior: classical
    url: "https://cdn.jsdelivr.net/gh/Loyalsoldier/clash-rules@release/applications.txt"
    path: ./ruleset/applications.yaml
    interval: 86400

注1: proxy-providers 不能直接使用, 需要在 proxy-groups 中使用 use 来引用
注2: rules-providers 中一大堆来自于 https://github.com/Loyalsoldier/clash-rules, 这里使用了 Github 镜像

20240129 update:

对于我来说, 我的启动命令是:

1
2
3

docker run --device /dev/net/tun --cap-add NET_ADMIN -ti -p 3333:1080 -p 3334:8888 \
    -e EC_VER=7.6.7 -e CLI_OPTS="-d https://vpn.xxxxx.xx.cn -u [email protected] -p <password>" \
    --name econn --restart=always -d hagb/docker-easyconnect:cli

这个命令在 Windows 和 Linux 都可以用, 在 Windows 时应当在 wsl2 下关联 docker.

注3: 如果你使用 TUN 模式, TUN 配置中应当忽略 vpn host 和内网 ip, 否则会出现无尽回环风暴. 如果已经出现风暴, 把 Clash 模式调整为直连即可
注4: 上面的配置在虚空终端 Meta 也可用

20240308 update:

把代理供子里的地址改成了 jsdelivr 托管的.

TL; DR

Introduction

Principles

Single-Channel Coloring

Multi-Channel Merging

Channel Information

TL; DR

前言

原理

单通道上色

多通道合并

通道信息

Summary

Preface

Preparation

Compiler

Source Code

Disable openblas

Virtual environment and dependencies

Build

Test

Packaging

Misc

摘要

前言

准备

编译器

源代码

关闭 openblas

虚拟环境和依赖包

构建

测试

打包

拾遗

20240129 update:

20240308 update:

最新文章

归档

标签