Show HN: Streaming gigabyte medical images from S3 without downloading them
61 points
5 hours ago
| 7 comments
| github.com
| HN
invaderJ1m
7 minutes ago
[-]
How does this compare to things like COGs (Cloud Optimised GeoTIFFs) or other binary blob + index raster pyramid formats?

Was there a requirement to work with these formats directly without converting?

reply
matthberg
3 hours ago
[-]
Seems very similar to how maps work on the web these days, in particular protomap files [0]. I wonder if you could view the medical images in leaflet or another frontend map library with the addition of a shim layer? Cool work!

0: https://protomaps.com/

reply
el_pa_b
3 hours ago
[-]
Thanks! Indeed, digital pathology, satellite imaging and geospatial data share a lot of computational problems: efficient storage, fast spatial retrieval/indexing. I think this could be doable.

As for digital pathology, the field is very much tied to scanner-vendor proprietary formats (SVS, NDPI, MRXS, etc).

reply
rwmj
2 hours ago
[-]
https://dicom.nema.org/dicom/dicomwsi/

Interesting guide to the Whole Slide Images (WSI) format. The surprising thing for me is that compression is used, and they note does not affect use in diagnostics.

Back in the day we used TIFF for a similar application (X-ray detector images).

reply
tokyovigilante
2 hours ago
[-]
This is really a job for JPEG-XL, which supports decode of portions of larger images and has recently been added to the DICOM standard.
reply
dmd
1 hour ago
[-]
Or IIIF.
reply
lametti
2 hours ago
[-]
Interesting - I'm not so familiar with S3 but I wonder if this would work for WSI stored on-premises. Imposing lower network requirememts and a lightweight web viewer is very advantageous in this use case. I'll have to try it out!
reply
el_pa_b
2 hours ago
[-]
When WSI are stored on-premise, they are typically stored on hard drives with a filesystem. If you have a filesystem, you can use OpenSlide, and use a viewer like OpenSeaDragon to visualize the slide.

WSIStreamer is relevant for storage systems without a filesystem. In this case, OpenSlide cannot work (it needs to seek and open the file).

reply
Nora23
2 hours ago
[-]
How does this handle images with different compression formats?
reply
tonyhart7
2 hours ago
[-]
hey, I need this
reply