▲Last time I worked with parquet, I had to make an entire internal library to achieve performant logical row wise reads, since the file format and most open source readers are column wise. Cool to see row reader is supported out of the box
reply▲Great! I will give it a try. I found that using DuckDB to select from the parquet files and using the Apache Arrow API to read the results is also a very fast method
reply▲willtemperley3 hours ago
[-] Respect for doing this. I recently implemented a Parquet reader in Swift using parquet-java as a reference and it was by a long way the hardest bit of coding I’ve done. Your bit unpacking is interesting, is it faster then the 74 KLOC parquet-java bit unpacker?
reply