Are we compressed yet? How to cut through codec hype.

25 Sep 2015

Are you confused by contradictory claims about video codec technologies? An online resource might help you find the answer.

Finding out which codec is "best" can be difficult and challenging. Luckily, the team developing the open-source Daala video codec have a solution. "AreWeCompressedYet.com", an online resource that allows you to compare the performance of popular codecs.

How it works:
Codec developers submit "jobs" to the site. Each job involves coding and decoding a set of common test videos at a range of bitrates. This can be a very time consuming process, but the site collects and stores the results from many previous test runs. The results of each job can be viewed as a rate-distortion curve, with bitrate on the x-axis and "quality" on the y-axis:

This example compares the performance of five video codecs, under the following conditions:
  • Test video set: "ntt-short-1", a set of 1-second video clips with varying characteristics.
  • Video resolution: 720p, 24 frames per second
  • Picture quality metric: Peak SNR (PSNR)
In this example, HEVC and VP9 have the "best" performance (highest curves), followed by Thor, H.264 and Daala. There are a few things you should be aware of:
  • Measuring picture quality is a notoriously inexact science. Changing the metric from PSNR to a different metric such as SSim may - and often will - change the ranking of the codecs.
  • Changing the video set or resolution may affect the results, since each codec's performance depends on the actual video content.
  • The online tool is comparing specific implementations of each codec. For example, different software implementation of HEVC will probably perform differently.
  • There is probably significant scope for further improvements to the HEVC, VP9, Thor and Daala codecs, since these are all relatively new (or still-developing) formats.
With all this in mind, arewecompressedyet.com provides a quick and easy way to compare codec technologies under identical experimental conditions. Will it resolve the endless arguments about which codec is best? Maybe!


---- Professor Iain Richardson is the author of four books on video compression. He leads Vcodex, providers of expert advice on video compression and multimedia communications. http://vcodex.com

Comments