LIC-Audit: How Comparable Are Reported Gains in Learned Image Compression?
arXiv:2611.23419v1 Announce Type: cross Abstract: Learned image compression (LIC) papers often report BD-rate improvements of only a few percent. They are measured against VTM reference software whose version keeps changing: the claims we audit name 7 versions, 5 in 2025 papers alone. But how comparable are these numbers across papers, and what evidence should future papers give for such small gains? We introduce LIC-Audit, an analysis of 353 released rate-distortion curves from 36 repositories, and a controlled re-evaluation of eight public codec releases with 28,193 encode-decode records. We identify three problems. First, different VTM reference runs change BD-rates by up to 3.56 percentage points, and on CLIC Pro published numbers order 10 of 55 method pairs differently from a common computation; comparing codecs directly over a fixed quality range removes this dependence. Second, releases use different and mostly undocumented conventions for counting bits, computing PSNR, and padding images. Under consistent…
Covered by 1 source
- AarXiv CS.AI↗Haisheng Fu, Jie Liang, Runyu Yang, Feng Ding, Xin Dong, Zixiao Jin, Jingning Han3d ago