Scalable pixel product

I've spotted the first real SVC product announcement. At NAB, MainConcept, a DivX daughter, presented their SVC implementation. SVC is the new video coding extension to H.264 that doesn't bring higher coding efficiency, but actually worsens it. I wrote about it earlier. The big benefit though is that you can decode parts of the bit stream in case you only need a smaller resolution picture. MainConcept writes "creating an SVC file only causes an approximate 10% file size increase compared to a regular H.264/AVC file".

10% is a lot in video compression.

Pixel compression over time

Here's an interesting graph from Harmonic that I sometimes use in presentations. I often misplace it, so I figured I'd stick it here on this blog. That way I can always find it. The graph shows that video compressors are not all the same. They can be improved over time. This is an important fact for chip makers, since developing a complex chip these days often takes well over a year, and is then sold in the market for a year or so after. The longer you can keep your chip in the market, the more you will sell! If your chip includes a software programmable video subsystem, you can still take advantage of algorithmic improvements, just like Harmonic did, and deliver better video quality.

Perfect pixel patent

As early as 1929, Ray Davis Kell described a form of video compression and was granted a patent for it. He wrote, "It has been customary in the past to transmit successive complete images of the transmitted picture. [...] In accordance with this invention, this difficulty is avoided by transmitting only the difference between successive images of the object." Although it would be many years before this technique would actually be used in practice, it is still a cornerstone of many video compression standards today. It's the reason why video using MPEG can be compressed roughly a factor of 10 better than JPEG-compressed still images.

What technique can provide another magnitude of improvement in video compression?

My prediction is that we need to change focus from optimizing for best peak signal to noise performance to optimizing for psycho-visual perception. I.e. "how good do the compressed images look" instead of looking at minimizing the mathematical difference between the original and compressed imagery.

Acquired pixels

Recently, DivX acquired MainConcept for approximately $22M. MainConcept is a developer of mostly PC-based video codecs, which is the business that DivX is in also.

Which video company will be acquired next?

Crummy pixels on my iPod Touch

I recently bought an iPod Touch. The WiFi integration is neat and worked straight out of the box. I now have a pocketable Internet browser, and it even connects directly to YouTube, using the new H.264 codec instead of YouTube's default and inferior Flash codec that the PC-based website uses.

After playing a few videos something interesting happened. The video codec shows very crude artefacts. See the picture below. I haven't found many other iPod users on the web complaining yet, but it's hard to believe I am the only one. Will Apple be able to fix this with a firmware upgrade? If the rumoured Samsung chip at the heart of this device uses a hard-wired video coding subsystem, they likely won't be able to fix the issue quickly in software. Instead, they will have to respin the chip and people will have to return their devices and get new ones months later. Chip inventory will have to be trashed. A reset fixed the issue for me, but it has shown up again.

Are you an SOC designer that still uses hard-wired video codecs? Can you risk designing an SOC that requires a silicon respin to resolve issues that could have been solved in software if a programmable approach had been chosen?

Scalable pixels

H.264 was standardized quite a while ago in 2003 and brought a "back to basics" compress-video-only approach compared to the feature-laden MPEG-4. So what are the ISO and ITU video standardization gurus working on these days? They're working on SVC, short for scalable video coding. The goal this time is not to achieve a higher coding gain, but instead to make the bitstream scalable.

In SVC, a same single bitstream can be decoded at different resolutions or frame rates. If you're watching the stream on your 1080p big screen TV at home you decode all the bits, and in case you want to play the same stream on your mobile phone, just decode those pixels that you need for the small screen. There are many applications for scalable video coding, but I've seen the concept many times before and people just don't use it. JPEG2000 is scalable. MPEG-4 had a few scalable profiles, even MPEG-2 had scalable extensions. None of them are widely used today. Why? Because there's overhead involved in making a bitstream scalable. A scalable bitstream is larger than a non-scalable one. Also, most system engineers find it simply more practical to just recompress the bitstream for each specific target device. Pixar and Dreamworks even completely re-render their 3D movies dependent on whether it's for the theater or for a DVD. Compressing the resulting video sequence another time doesn't seem that much of an extra burden.

Will things be different this time? Will SVC become a prevalent standard?