Meta has announced the public release of OpenZL, an open source data compression framework for structured data.
OpenZL offers lossless compression for structured data, using a user-provided description as a starting point. “As a user, you provide OpenZL with the data shape (via a preset or a thin format description),” then an optimization component builds a compression configuration that can be reused on similar data.
According to the OpenZL whitepaper, the tool “compresses data into a self-describing wire format, any configuration of which can be decompressed by a universal decoder.” Additionally, OpenZL “enables rapid development of tailored compressors with minimal code.”
Check out the quick start guide for an overview of the process and learn more at OpenZL.
