|
5000+ Posts? I Should OWN This Site!
Join Date: Jun 2007
Posts: 5,067
|
|
The H.264 (a.k.a. MPEG-4 Part 10 and AVC) Bible
Many of you may have already heard of H.264 (Wiki page HERE) or AVC (for Advanced Video Coding), the latest-and-greatest video standard, widely used in everywhere where the best possible video quality is required with the least possible storage and/or bandwidth usage. It�s pretty much comparable to HE-AAC v2, the latest-and-greatest encoding in audio technology, which allows for quality (!) stereo Web radio streaming at even 24 kbps, and CD-quality recordings at 48 kbps. (Pretty much unbelievable, particularly at 24 kbps, with traditional compressed formats like MP3, WMA or even OGG, isn�t it?)
AVC is way better than the �old� MPEG-4 Part 2 or �ASP�. The latter is more commonly referred to as �DivX, Xvid�; in this article, I only refer to it as �ASP�, while I refer to the above-introduced H.264 format as �AVC�). While it gives you the same (or even better!) video quality, it is, in general, between 50 to 100% smaller and decidedly more flexible.
A lot of misconceptions or plain false info is circulating in the Windows Mobile, Palm and Symbian community; this is why I�ve found it extremely important to publish my AVC guide well before finally publishing my long-promised all-in-one Multimedia Bible.
Now, I take a look at whether you want to use it at all.
1.1 Pros and cons
First, let�s elaborate on why you would want to go for AVC files instead of the well-established and supported, plain, �old� ASP ones.
1.1.1 Cons
1.1.1.1 Battery life considerations
Decoding AVC, depending on the special AVC features used, the bit speed and the resolution, can require up to five times more CPU cycles than doing the same to ASP. In this section, I elaborate on what this results in in practice.
You may know it well enough that the more CPU cycles a given app uses, the less battery life you�ll have. This in itself may be a stumbling block for you.
Of course, this wildly differs between different CPU�s. For example, TI OMAP-based devices (particularly newer, latest-generation ones like the Nokia N95) consume far less power than Xscale-based ones. To demonstrate this, some examples.
A Nokia N95 playing an 320*144
* ASP encoded at 83 kbps (resulting in about ~30% CPU usage)
* AVC encoded at ~400 kbps (resulting in slightly less than full, 100% CPU usage)
results in the following power consumption:
(the first half of the chart shows playing back the first, the second half the second video).
As can clearly be seen, the difference really isn�t much � about 28 mW, which roughly corresponds to ~90 minutes battery life decrease using the stock 950 mAh battery. That's about 30-35% battery life decrease.
(Note that I couldn�t make the same on TI OMAP-based Windows Mobile devices as it�s not at all possible to measure the current on them � �only� the CPU usage.)
Now, let�s see how other CPU�s behave in this respect. Let�s see the same test on the 624 MHz Intel Xscale PXA 270-based Dell Axim x51v. (Backlight level: as with the Nokia, the lowest.) In here, I�ve tested a 320*144 AVC encoded at ~400 kbps (about 30% CPU usage at 624 MHz) without any manual quality degradation and a 640*272 AVC encoded at ~460 kbps, without deblocking (near 100% CPU usage). For the test, I�ve enabled dynamic CPU scaling (that is, I didn�t run it on external power) so that the CPU could switch down to a lower frequency while playing back the lower-resolution video to increase the battery life to some degree.
The upper chart shows the power (in mA), the lower the CPU usage. As can clearly be seen, the power usage of playing the high-res AVC video at ~100% CPU usage required about 70% more power (480 / 280 mA) than doing the same with the QVGA-resolution AVC video. That is, while, on the TI OMAP-based Nokia, you will �only� encounter a 35% battery life decrease, on anything (!) Xscale-based, about 70% (!!!!). Yes, the (newer) TI OMAP platform does have a lot of advantages, one of them being not chewing through the battery when running CPU-intensive tasks, while still delivering excellent performance.
What does this all mean?
- If you have an Xscale-based handset and have plenty of battery / a spare or an extended one OR you�re running a (particularly new-generation) TI OMAP-based handset (Nokia N95 etc.), you don�t need to be afraid of the battery life: go for the best quality; that is, AVC providing you with the best watching experience.
- If, on the other hand, battery life is of extreme importance for you and you use a Xscale-based device, make you will still want to prefer ASP (or, low-quality AVC) to AVC.
1.1.1.2 You�ll be forced for (slow!!) recoding � unlike with ASP
First and foremost, as can also be clearly seen based on my very thorough tests, you in no way can play your familiar, �torrented�, full PAL/NTSC-resolution video clips / movies on current hardware. This is diametrically opposed to the case of ASP, where you, in most cases (except for the slowest TI OMAP-based Windows Mobile devices), can.
This means you MUST recode your videos before watching. You can�t just quickly drop your fresh-torrented, PAL/NTSC full-resolution (720*480 or 576) AVC�s on a memory card and just use CorePlayer (or TCPMP) to watch them. When they become available, that is; currently, on the Torrent scene, mostly, �only� HD (720p / 1080p / 1080i) videos are encoded in AVC, traditional, �low-res� PAL/NTSC rips are (still) all ASP�s.
Not even the best, most powerful handheld devices are able to play full PAL/NTSC-resolution videos (let alone 720p / 1080p / 1080i ones!). You must recode everything to either the native screen width of your device (which, again, isn�t the case with traditional ASP videos) or less and, in cases (for example, with VGA devices and/or slow (200 MHz) TI OMAP-based Windows Mobile handsets), you must deactivate some of the advanced features. This, again, isn't the case with "traditional" ASP files.
As noone not having a mobile device him or herself will deliberately rescale his videos (primarily meant for desktop watching) to a 640 or, God forbid, 320-wide one and, probably, even remove some features, which results in a visible quality decrease. With ASP files, you can just watch files originally meant for desktops on a handset; with AVC files, you can�t.
In addition, recoding, as opposed to creating ASP files, is a time-consuming process. In general, creating an AVC video takes 2-3 times longer than doing the same with an ASP one.
1.1.2 Pros
If you MUST use the least possible video sizes (because, for example, you don�t have an SDHC-enabled handset) with the best possible quality, H.264 is the way to go. It�s simply unbeatable and is way better than ASP at the same (low) bit speed. Again, it�s like how HE-AAC v2 compares to MP3.
Also, the MP4 container format used by the recommended Nero Recode makes it possible to have two (!) sound tracks in the same file. This was very hard to achieve with AVI files without some manual work. To my knowledge, there aren�t any Windows Mobile (Symbian, Palm etc.) related tools that let for storing two sound tracks in an AVI file. (It�s, technically, not impossible.)
Finally, once you learn how to navigate around in Nero Recode, ripping DVD�s or converting your other videos becomes very easy. No command-line tools are necessary � albeit, of course, you can use them too. For free, I should add � the most important command-line tool to encode into AVC, x264, is highly recommended and is of high quality. It, however, takes a while to learn � this is why I tend to recommend Nero Recode instead.
Nero Recode is, at first, seems to be a bit more complicated to use than well-known, established Windows Mobile DVD / video conversion tools like Pocket DVD Studio, Pocket DVD Wizard etc. Also, it�s a lot more expensive (around US$ 90). However, taken into account that several other AVC encoders are more than five times more expensive and you also get a complete suite of for example CD / DVD burners, the price can be justified. And, again, it only takes, say, an hour to completely learn to master the tool - unlike with x264.
(Note that, as the aim of this Bible is NOT showing you all the necessary encoding tools and tricks, I "only" discuss Nero Recode. It's readily available for download as a pretty much usable trial, isn't much overpriced and is MUCH easier to use than free tools. It's particularly because of the latter that I've chosen it to be featured in this Bible. Should you need another tool, look around HERE. I particularly recommend the decoder comparisons linked from HERE.)
1.1.2.1 Sample videos
In order to show you how much better AVC videos are than plain ASP ones, I've uploaded several of my test videos for you to evaluate. They're all 29-second, bilingual (English & German) and with two subtitle streams (English & German) and have been encoded for best quality (two-pass encoding with the best quality defaults).
In order to play them, you'll most probably need an AVC-compliant video player (if you don't already have a H.264 decoder on your system; for example, Cyberlink's newer PowerDVD versions do have a pretty good one - albeit in no way as efficient as CoreAVC, which I'll talk later. To play back these videos, it's more than sufficient). The easiest is getting VideoLAN VLC. Just download & install it (no need to fuss around with separate codecs - it contains all), start and open the video files.
320*144 videos:
87 kbps:
ASP (the worst - at 87 kbps and 320*144, using traditional ASP produces pretty bad results). As with the other bitrates & resolutions, this file is called "ASP.mp4".
AVC with all AVC goodies enabled - now, compare the quality of this title to the above-linked ASP one. Quite different, isn't it? Yes, AVC is WAY better at really low bitspeeds like 87 kbps. The file is named "default.mp4" - as will be the case with other bitrates.
AVC without bilinear prediction / CABAC support: this video has been encoded without two major AVC features to heavily reduce the load at runtime (and help speed up playback). (Note that I'll speak of bilinear prediction / CABAC later.) Nevertheless, even without these features, it's way better than ASP.
The same videos, encoded at 273 kbps and 320*144. As can clearly be seen, ASP produces much better results at this speed than at 87 kbps; still, AVC still has the lead, even without bilinear prediction / CABAC.
Sample videos encoded at three different bitrates (204k, 366k and 464 kbps) and at 640*272. Note that, with the second group, I've also made available two other videos. THIS one demonstrates the video quality degradation with bilinear prediction disabled; THIS with CABAC disabled. (Again, the 363k group also has the video that shows the "combined" effect of disabling both features - it's, again, named "nobilinNoCabac.mp4").
I really recommend scrutinizing these videos. For example, in the next scene at second 13 into the sample movie:
it's really worth checking out the following:
- the wall (it'll be "moving" all the time, producing a very bad effect, particularly with ASP and/or deblocking disabled)
- the effective resolution of the balls on the pool board (it'll be decidedly lower with 320-wide ASP videos than AVC ones, showing ASP not only sacrifies at quality (blockiness), but also at resolution at such low bitrates)
- the "blockiness" of the green pool board around the balls.
As you'll see, ALL these test videos show how much better AVC is, image quality-wise.
Now that you�ve seen both the advantages and disadvantages of AVC, let�s take a look at what players there are to play them.
1.2 Available players for mobile platforms
(Note that this guide doesn't cover the features not directly related to AVC playback - for example, AVRCP support, equalizer etc. because they can also be utilized playing back other content. They'll be all elaborated on in my forthcoming Multimedia Bible.)
1.2.1 Windows Mobile
1.2.1.1 CorePlayer
Without doubt this is the best player out there to play H.264 content. It�s by far the fastest, most compatible and featureful. If you�re seriously into H.264, you MUST buy it: it isn�t THAT expensive.
1.2.1.2 TCPMP
For this free app to be able to play your AVC videos, you MUST download THIS (originally linked from HERE; mirror HERE) add-on CAB file. Install it after having installed TCPMP. The CAB file contains a beta version of an early Windows Mobile CoreAVC port and the well-known AAC decoder already discussed and linked to HERE. (A quick note: CoreAVC is the main AVC decoder of all multimedia products of the CoreCodec folks. CoreAVC, on desktop operating systems, is unrivalled in speed � it�s even faster than some hardware-supported, much more expensive solutions. The speed advantage is certainly visible with the mobile ports as well.)
Note that there is another AVC decoder, the official ffmpeg codec, which is WAY slower than this beta and is, therefore, in no way recommended.
While the app has certain strengths (most importantly, it�s free), I don�t recommend it for serious AVC freaks � it�s just not powerful enough.
1.2.1.3 Nero Mobile Pro
Unfortunately, while the encoder (�Nero Recode�, part of the Nero 8 Ultra Edition) of the same developer is without doubt excellent and highly recommended, their Windows Mobile player, the commercial Nero Mobile Pro, is not recommended for playing back AVC content. (Or, for that matter, currently for anything else either: in my tests, other media players have turned out to be much more efficient and featureful in almost every area. It seems its only advantage is the native ability to play back HE-AAC v2 audio. To do the same, however, you can always use the free TCPMP with the AAC plug-in � and it�ll still result in better battery life than Nero Mobile Pro.)
Unfortunately, there�s no trial version of this app any more. While Nero 7 Ultra Edition Enhanced did have a trial of Nero Mobile Pro, Nero 8 doesn�t. Before a trial is (re)introduced, you can safely take my word: it�s, currently (as of version 1.4.0.9), just not worth the price (I�ve paid some 15 euros + VAT for it). And, it costs the same as the technically VASTLY superior CorePlayer � why would you, then, go for it at all?
As far as its performance is concerned, on VGA Pocket PC�s, it�s only usable with 320-only (QVGA) videos; of them, ONLY with ones without CABAC and Bidirectional Prediction. This pretty much rules it out as a decent player. If you do enable these (important) features, there will be a lot of dropped frames � on even the fastest handsets like the Dell Axim x51v.
640-pixel-wide videos, even ASP ones, badly lag; the AVC ones, in addition, have stuttering sound, even at the lowest stream speeds. That is, it�s impossible to play back VGA content under Nero, not even if you do all the recommended performance tweaks.
On QVGA devices, the situation isn�t better either: it�s only able to play the lowest-speed QVGA AVC movies. On my 400 MHz HP iPAQ h2210, the video was stuttering even when using a 83 kbps video stream (a ~230 kbps one was even worse). I haven�t tested it with deblocking-disabled videos but I don�t think it�d improve the situation.
All in all, stay away from this app. While their encoder (Nero 8 Ultra Edition) is certainly worth purchasing (if you don�t want to manually convert your DVD�s or other files to AVC with alternative tools like x264), Nero Mobile Pro is in no way.
1.2.1.4 In no way recommended, other apps
Philips� Platform4 player (no longer developed, entirely abandoned, low-quality)
GPAC Osmo4/Osmophone
VideoLan � long-abandoned and has never really worked. When it did, it only produced at least an order of magnitude worse speed than TCPMP / CorePlayer.
1.2.2 Symbian
To play AVC videos under Symbian, your only real choice is CorePlayer.
Note that the built-in RealPlayer is also stated to be AVC-compliant (not only ASP). If it is, then, it must only be compliant with the simple Baseline profile, which isn�t what you will really prefer. I�m absolutely sure it�s not compatible with the Main profile, not even without Bidirectional Prediction and CABAC.
As far as other Symbian players are concerned, SmartMovie (as of version 3.41) doesn�t support AVC videos at all (it doesn�t even list MP4 files). The recently-released MMPlayer (as of version 1.01) doesn�t support AAC sound (see their official list of what�s already supported and what is planned HERE). This means it can�t play back AVC videos either because AVC videos generally use AAC sound tracks.
1.2.3 Palm OS
Here, CorePlayer is your only choice. MMPlayer doesn�t support AVC and, unlike with Windows Mobile, the Palm version of TCPMP doesn�t have an AVC add-on.
1.3 MP4 as a container; compatibility
AVC videos can be delivered in many so-called �containers�. The most widely used is the MP4 container, which, in addition to the video itself, can also have two (AAC) sound tracks, two subtitle streams and chapter support. Nero Recode supports these features. If you would really want to include two sound tracks in your videos and plan selecting your players based on these needs, then, you will find this section useful.
In the following chart, I list the compatibility of mobile players with these features.
As can clearly be seen, both Nero and Symbian�s RealPlayer are unable to switch to the second soundtrack. They don�t support chapter information either. The latter may turn out to be a letdown with direct DVD conversions.
Finally, none of these players support the Nero Recode subtitle format � unlike for example the free VideoLAN on desktop operating systems.
1.3.1 Matroska (.MKV) support
There is another, compared to MP4, more advanced container format called Matroska, which is especially popular on the High Definition ripper / Torrent scene (as opposed to �plain� DVD-only resolutions). Matroska files generally contain AVC video. CorePlayer supports these containers.
Note that, however, CorePlayer is only able to play back videos no wider than 1008 pixels. That is, it will NOT play back for example torrented 720p (meaning 1280-wide videos) content � most of current MKV files contain these kinds of videos. This is pretty much understandable if you take into account that most (particlarly QVGA) mobile hardware is simply unable to play back even 640-wide videos, let alone ones with much higher (twice the size!) resolution.
1.4 Fixing the frame drop problem
Unfortunately, as has already been pointed out, playing back AVC files requires a lot of computing power. On especially slower and/or (W)VGA or other hi-res devices, you MUST make some tradeoffs in order to be able to play back your AVC contents without problems (dropped frames).
In addition to lowering the resolution if you use a non-QVGA (read: (W)VGA on Windows Mobile, 320*480 on Palm OS or the screen resolution of communicators like the E90 on Symbian), you have four choices:
1.4.1 Using the simplest (Baseline) profiles instead of Nero�s �Standard � AVC�
Sticking with the simplest (Baseline) profiles means avoiding using the (standard) Main profile. This will still result in considerably better-quality results than with ASP, but, in my opinion, isn�t the best way to go because it�s definitely an overkill. That is, by fine-tuning the much more featureful Main profile, you can get much better results.
That is, if you stick to the Baseline profile (by, say, using the �Mobile AVC� or �Portable AVC� profiles in Nero Recode � as opposed to �Standard AVC�, which roughly corresponds to the �Main� profile of the H.264 standard), you will not have access to a lot of goodies that, otherwise, wouldn�t decrease your playback performance that much but still add a lot of additional functionalities and subtly increased image quality.
For example, if you don�t use the �Standard � AVC� Nero profile, you won�t be able to select HE-AAC for audio encoding, only the substantially worse AAC-LC. As has already been explained in the recently published 2nd Multimedia Bible sneak peek: crossfade / gapless playback, (audio) media compatibility and power usage, CorePlayer (as opposed to most other players; for example, TCPMP) doesn�t use considerably more CPU time for decoding HE-AAC audio; therefore, you�ll want to use HE-AAC with CorePlayer and not AAC-LC. If you, however, use a �dumb� profile, you won�t even have a chance to select HE-AAC.
However, when you do plan to watch your videos with TCPMP only (or, other, technically not so advanced multimedia players), you must keep in mind that the situation is pretty much the opposite. Then, you WILL want to switch back to AAC-LC by selecting the �Settings� radio button (highlighted in HERE, the mouse hovering over it), clicking the �Custom profile:� radio button and selecting AAC-LC as can be seen in HERE.
There will be other cases (for example, playing back AVC on slow TI OMAP CPU�s), when this (using �simple� profiles) is what you will want to prefer. Note that, again, the video quality will still be better than that of traditional ASP at the same bitrate � that is, it�s still a usable tradeoff. On other (faster) platforms (practically, anything non-TI OMAP-based, except for the newer Nokias, which already are based on a vastly enhanced TI OMAP), you will ALWAYS want to stick to the Main H.264 profile (accessible as �Standard AVC� in Nero).
Note that Nero Recode also has some other, device-specific profiles. Of them, many recommend for example the iPod 5.5G profiles for VGA users (it, using the default, automatic settings, encodes 640-wide video at 728kbps). However, I don�t really recommend it because the iPod 5.5G profile doesn�t support a number of advanced features (for example, CABAC and/or Bidirectional Prediction) � why not stick with the Standard profile, then?
Also, many recommend the Sony PSP profiles of Nero. I didn�t find them particularly useful either, particularly if you�re a high-res (VGA etc.) user. The sole reason for this is that it only encodes in low resolution.
1.4.2 Switching off Deblocking
Deblocking is a genuine AVC (not available in ASP) feature to heavily (!!!) reduce blocking effects and really increase image quality. It�s really-really useful, particularly with slow (80�120 kbps QVGA or 200-250 kbps VGA videos), while �only� consumes about 10-20% CPU cycles.
Disabling it will, therefore, increase playback performance by 10-20%. Note the following:
1. You should ONLY disable it when there is simply nothing else to do to increase performance. The two other (encoding-time) tweaks I�ll introduce in the next two sections, that is, disabling CABAC and/or Bidirectional Prediction, result in a much better performance gain, while retaining much better playback quality.
2. Some people state (example HERE) that switching off deblocking at video (re)coding time should be preferred to (plain) runtime switch-off. I�ve thoroughly benchmarked this and found out that, with the recommended CorePlayer, this is not the case (unlike what the original poster stated) - you won�t see almost any performance increase of not encoding your videos with deblocking support at all.
That is, with CorePlayer, where deblocking can be disabled by hand when you play back your video, disabling deblocking at encoding time isn�t necessary � there�re simply no advantages of doing it. With TCPMP or other players that don�t allow for disabling deblocking at runtime, however, you might still want to do this at encoding time. The speed increase will be pretty much similar to that of CorePlayer. (But, again, if you�re seriously into AVC, I simply don�t see the point in sticking with alternative and, technically, inferior products like TCPMP or Symbian�s RealPlayer � CorePlayer is THE fastest and THE best AVC player definitely worth its price.)
Disabling deblocking at runtime (again, only in CorePlayer) is pretty easy: just go to Tools / Preferences, select the Advanced page and scroll down to the �Disable AVC deblocking filter� checkbox. Tick it. Screenshots of this:
(Windows Mobile)
(Symbian)
Disabling deblocking at encoding time (again, if you do NOT use CorePlayer but (inferior) alternatives) is pretty easy too. If you use Nero Recode, after clicking Next on the main screen, click �Nero Digital Settings� in the left list (over the ? and the More >> buttons) and, after enabling the �Expert mode� checkbox under the tree view in the center (it contains a single item, �Encoder�, in non-expert mode, only allowing for switching between one- and two-pass modes) go to AVC Encoder / Encoding Tools. Then, just untick the Deblocking checkbox in the lower center.
1.4.3 Switch off CABAC at encoding time
CABAC is another, new and advanced technology used in the Main profile of AVC. Unfortunately, enabling it also consumes some additional CPU cycles at decoding time. Therefore, you might want to disable it � at encoding time only.
To do this in Nero Recode, go to the same AVC Encoder / Encoding Tools as was the case with disabling Deblocking, and untick �CABAC� (the uppermost checkbox) as can be seen in HERE.
Note that if you don�t see this checkbox, make sure you use the Standard � AVC profile inside the Nero Digital AVC category. In simpler profiles / categories like iPOD, PSP, Nero Digital AVC�s Portable etc., they are inaccessible (because they aren�t used at all by simple profiles).
1.4.4 Switch off Bidirectional Prediction at encoding time
As with Deblocking and CABAC, Bidirectional Prediction is another brand new feature of the non-basic profiles of H.264 (and, consequently, Nero). Unfortunately, it also consumes some additional CPU cycles at runtime and it can, as with CABAC (and unlike Deblocking), only be disabled at encoding time.
To do this in Nero Recode, go to the same AVC Encoder / Encoding Tools as was the case with disabling Deblocking and CABAC, and untick �Bidirectional prediction� (the uppermost checkbox) as can be seen in HERE.
(Note that you can disable both CABAC and Bidirectional Prediction at the same time. Do this if you do need the maximum performance. Don�t forget that, while disabling both results in a huge speed increase, disabling them both will still produce better results than disabling deblocking. Only disable the latter when you�ve already disabled the former two and there still are dropped frames. If it still isn�t working, then, consider using a lower resolution or entirely switching from AVC to the traditional ASP.)
1.4.5 Effects of en/disabling deblocking, CABAC and Bidirectional Prediction
The following chart shows the effect on performance of doing all these hacks. The results have all been measured using the current version (1.1.1 on Windows Mobile, b2 on Symbian) of CorePlayer. The test machine was a VGA Dell Axim x51v � video-wise, probably the fastest handheld around.
The first chart shows the performance gain of not only the combined disabling of CABAC and Bidirectional Prediction, but also the separated results. As can clearly be seen, the performance gain of not using CABAC roughly equals to disabling deblocking - while, again, it maintains MUCH better visual quality and should ALWAYS be preferred over disabling deblocking. The latter should always be the last resort to (try to) get rid of (heavily) dropped frames or bad performance. Disabling Bidirectional Prediction, on the other hand, results in a much higher performance gain: it�s like disabling deblocking AND CABAC at the same time.
The chart also shows the results of disabling the Intel 2700g hardware accelerator acceleration (a single checkbox in CorePlayer). As can clearly be seen, 2700g helps a LOT when playing back ASP videos � particularly high-resolution ones. With the latter, the performance gain can be even 60%. As the Intel 2700g doesn�t help at decoding AVC at all, the results would have been the same in both cases.
I�ve used several test videos with either 640 or 320 width. They all had two HE-AAC soundtracks (default of the Standard � AVC profile of Nero � as opposed to lower-quality profiles, where only AAC-LC is accessible) and two subtitle streams. The latter, of course, (still?) aren�t displayed by CorePlayer. All of the videos have been encoded using two passes and all optimizations. Note that removing one of the sound tracks and the two subtitle streams wouldn�t have resulted in a major speed gain (probably a 0.5% one � at most), which will also be shown in a later chart.
Note that most AVC benchmarks contain two values. The first shows the benchmark with enabled deblocking; the second (after a slash) with disabled one. (Again and again, disabling deblocking, unless your source is high-quality, is NOT recommended. Try creating your AVC videos with either bilinear prediction or CABAC (or both) disabled if you REALLY need some additional speed � the quality degradation will be far less visible than without deblocking.)
The chart also shows the effect of the bitstream speed on the decoding efficiency. It�s a well-known fact that the faster the bitstream, the more CPU it takes to decode it. As can clearly be seen, there is some difference. For example, a 204 kbps 640*272 AVC stream can be decoded with a 91% benchmark, while, when using a 464 kbps stream, this decreases to 82%, which, visually, is much worse (about two times more dropped frames). Fortunately, as AVC behaves extraordinary good at low bitrates (again, just like HE-AAC v2 in sound encoding), you will want to strive for using as low stream speeds as possible. Again, a 204 kbps 640*272 AVC stream is pretty much enjoyable � no need for using a faster stream. Let me emphasize again: a faster stream will only decrease performance.
1.4.5.1 Some other, bitrate-dependent tests
To prove my point and show how increasing the video stream speed decreases performance, I�ve made several other, bitrate-dependent tests with fixed streams. Note that, in here, the stream was a 640 * 384 (with QVGA, 320 * 192) one � that is, considerably thicker than the 640 * 272 stream used in the previous test. The results certainly show this has reduced the performance.
HP iPAQ hx4700:
HP iPAQ h2210:
HTC Universal:
Nokia N95:
1.4.5.2 Dell Axim x51v + TCPMP
The following chart shows how the x51v behaves with TCPMP (the free and, for playing back AVC files, not really recommended predecessor of CorePlayer). In here, I�ve benchmarked both the beta CoreAVC codec (the third and fourth columns) and the official (and much slower) ffmpeg codec (fifth column). As can clearly be seen, the ffmpeg codec is about 100% slower than the beta CoreAVC and the latter is about 26% slower than the one in the commercial CorePlayer. Note that this 26% also contains the additional slowdown introduced by TCPMP�s far less efficient HE-AAC audio decoder. With an AAC-LC test video, the difference wouldn�t have been this bad. (Again, as can be seen in HERE, the (old) HE-AAC decoder of TCPMP is 2.5 times slower than the AAC-LC one. As opposed to the CorePlayer case: There is almost no difference with CorePlayer�s AAC decoders.)
1.4.5.3 Other VGA Pocket PC�s
Let�s continue with some other test devices � in this section, Windows Mobile only. Let�s take a look at two other VGA devices, the 624 MHz HP iPAQ hx4700 (with an ATi chipset) and the 520 MHz HTC Universal phone (the latter without any hardware graphics accelerator). The tests, of course, have all been made with the latest CorePlayer (TCPMP was only used with x51v to show how slower it is compared to CorePlayer). As can clearly be seen, in AVC mode, the two devices performed equally well � and slightly (but not much!) worse than the �speedking� Dell Axim x51v. This also means if you have any of these devices (just like me), you may want to prefer them to x51v because of the far better-quality screen (much better color reproduction, no polarization problems etc.)
As can also be seen in these charts, the lack of any (previous-generation like the Intel 2700g in the Dell Axim x51v or the ATI chipset used in the hx4700) hardware accelerator in a VGA device isn�t a problem. The hardware acceleration of ATI and Intel 2700g (currently, the two chipsets supported by CorePlayer and TCPMP � no support for GoForce 5500 and Qualcomm 7200 yet) only helps ASP playback, not AVC one. It�s only the latest Marvell (ex-Intel) XScale 3xx (Monahan) series that has support for AVC decoding � not earlier designs. (Unfortunately, currently, only the brand new iPAQ�s (will) have the new Xscale CPU�s and nothing else.)
This also explains why, it�s only in the not recommended, old ASP mode that the (hardware accelerated) Dell and iPAQ are considerably faster than the non-accelerated Universal (or, the same devices themselves with disabled acceleration).
HTC Universal:
HP iPAQ hx4700:
Note that the �Video� setting dialog also contains a �Video quality� drop-down list. It should never be used because in the �Medium� setting, it severely degrades video quality (it effectively halves (!) the original (!) resolution � that is, VGA source becomes QVGA) and, in �Low� setting, almost noting will be visible. The speed gain its usage results in is pretty low too. Finally, it only has an effect on non-AVC (for example, ASP) source; this is why I�ve added the MQ (�Medium Quality�) remarks in there. (For example, (MQ: 319) with the HTC Universal playing back a 204 kbps VGA movie means 319% benchmark when the video quality is set to Medium.)
Halving the original resolution means that, if you play back a VGA ASP movie on a QVGA device that supports the quality setting, in general, you can give a try to switching to Medium quality � you won�t see much image degradation (as opposed to what you would see on a VGA device). This especially pays off if, otherwise, you couldn�t play back the video without dropped frames.
1.4.5.4 QVGA Pocket PC�s and MS Smartphones
Now, from VGA Pocket PC�s, let�s move to QVGA devices (Pocket PC�s and MS Smartphones alike): the pretty old, WM2003 HP iPAQ 2210 (run by a 400 MHz Intel Xscale PXA-255) and the new, low-speed HTC Vox (s710) MS Smartphone, run by a 200 MHz TI OMAP CPU. (Note that, as far as other TI OMAP-based models are concerned, I�ve also benchmarked the HTC Wizard. I�ll show the results in a later chart.)
HP iPAQ 2210:
HTC Vox / s710:
As can clearly be seen, the 400 MHz iPAQ 2210 is just able to play back QVGA movies without having to resort to manual quality degradation (bilinear prediction / CABAC / deblocking). With the 200 MHz TI OMAP Vox, the situation is far worse � you cannot watch QVGA-width AVC videos on Windows Mobile models based on this CPU without manually decreasing the quality in encoding time (in runtime, just disabling deblocking won�t suffice).
1.4.5.5 Nokia N95
Finally, let�s see how probably the best multimedia smartphone of today, the TI OMAP-based (running at 330 MHz) Nokia N95 (firmware version v20, which is a bit faster at video playback (too) than v12) fares:
As can clearly be seen, it�s just able to play back 640*272 ASP videos without dropped frames � unlike the Vox or anything Windows Mobile based on the TI OMAP platform (without heavy overclocking).
1.4.5.6 Note that�
- I haven�t benchmarked any Qualcomm MS7200-based devices because the next, soon-to-be-released, 1.2 version of CorePlayer will have much better support for it. Also, GoForce 4000 / 5500-based devices like the O2 XDA Flame are promised hardware support sometimes next year � see THIS for more info. Therefore, I haven�t tested them either. According to serveral users, both Acer (GoForce 4000) and the Flame (5500) have severe problems with playing back video � as is the case with the Qualcomm MS7200-based devices right now.
- The source video I used was a 640*272 / 320*144 one � that is, a traditional 2.39:1 (35 mm anamorphic / Panavision / 'Scope) movie. With, vertically, considerably bigger movies (1.85:1, 16:9 (1.78:1) or even plain 4:3 � see THIS for more info) will result in a somewhat worse results.
For example, with a real 4:3 (640*480, VGA) 550 kbps ASP source, the Nokia N95 benchmarks at about 93%, as opposed to the ~111% of the �thin� 2.39:1 movie resized to 640*272 (keeping the aspect ratio).
1.4.6 Other tweaking
I�ve also thoroughly tested the performance gain using other checkboxes in the already-known (it�s there that you need to disable deblocking) Advanced tab of CorePlayer. Note that the test video was a fast one (the Standard - AVC 640-wide one was 1995 kbps, the Standard 320-wide one was 1468 kbps etc.); with much lower (and, therefore, much more recommended) bitrates (between 80�450 kbps, depending on the resolution), the absolute numbers would have been considerably higher, while the relative rations would have stayed approximately the same.
As can clearly be seen, trying to increase the performance with the other, Advanced checkboxes are pretty much futile. As a rule of thumb, it�s only the three parameters (Deblocking both runtime and encoding time and CABAC / Bidirectional Prediction at encoding time) that you should pay attention to. Also note that, by default, CorePlayer defaults to the best and most effective playback method (video output). It�s only with some ATI-based devices that you MAY want to override its decisions, should you encounter for example the infamous greenish effect.
1.4.7 Resolution-dependent benchmarks
I�ve also made some resolution-dependent benchmarks to see how the players behave with non-QVGA / VGA source videos. In this test, I�ve let Nero Recode set the target resolution depending on the manually set bitrate.
Incidentally, this (Nero Recode itself decides the best resolution for a given bitrate) is the default how encoding works. This may be an overkill in a lot of cases. For example, even a 200 kbps 640*272 AVC stream looks great, while Nero Recode insists of only allowing for this size at the bitrate of 533 kbps. This is a big overkill if your premium concern is storage space. Again and again, even 200 kbps 640-wide streams can look great, particularly those of �traditional� �35 mm� (2:39:1) movies. No need for wasting almost three times more storage on the video stream.
Therefore, if you do want to ALWAYS force the Nero Recode to convert your videos / DVD�s to either 320- or 640-wide videos, you must manually click the Video button (the lowermost large button on the right), go to the Resize tab and fill in both the horizontal and vertical size. In THIS screenshot, I�ve filled in 640 for the horizontal and 272 the vertical size. Uncheck the �Letterboxing� checkbox (if ticked).
Note that, as there�s no �keep aspect ratio� functionality in here, you must manually compute the vertical size, based on the original one. That is, if the original is, say, 720*300, then, you�ll need to
1, divide 300 (the original size) by the result of 720/640 = 1.125 (or, if you plan to watch the video on a QVGA screen, 720/320 = 2.25). The result is 266 (or, with the QVGA case, 133).
2, now, you�ll need to find the least multiple of 16 closest to the result. To find it, divide 266 (133) by 16 and round up. The results are:
266 / 16 = 16.625; rounded up: 17
133 / 16 = 8.3125; rounded up: 9
3, multiply the rounded-up integer with 16 (272 and 144, respectively); this will be the new vertical size.
Now, the chart:
With exactly the same video files (and, in addition, with a one pass, one soundtrack, no subtitle file and a iPod 5.5G �specific one to see whether they result in a better performance), I�ve also made some other tests with the three VGA devices to see how they play them back. As can clearly be seen, there isn�t much difference.
First, the results clearly show including a second soundtrack and/or subtitles doesn�t really cause any real speed hit. That is, feel free to do it if you�re either a language buff (and like watching movies in different languages) or would like to listen to the commentary soundtrack, if any. (Note that, currently, few desktop-based media players can display the Nero subtitle streams: of course, Nero�s own Showtime and VideoLAN.)
As can also be clearly seen (just compare the 640-wide result to that of the 624 and the 688-wide: as can clearly be seen, there isn�t much difference), the overhead caused by having to resize the video to (horizontally) fill in the entire screen is pretty low (as opposed to what some people state).
Still, as you won�t take advantage of the extra 80 pixels of the original 720-wide video, there isn�t much point in NOT reconverting it. Again and again, with AVC videos and current hardware (that is, before the VGA iPAQ 2xx series with the Marvel Xscale CPU�s hits the street and CorePlayer receives support for it), you MUST convert your videos to be enjoyable on your handsets � as opposed to most ASP videos played on most Windows Mobile / hi-end Symbian handsets.
The other direction (not using a full 640-wide stream), on the other hand, can pay off. If you don�t want to disable any of the special AVC features (deblocking / CABAC / Bidirectional Prediction), then, resizing your videos to be 544 pixels wide will still yield an above-100 benchmark, meaning flawless playback (mostly) without dropped frames.
There is another usage area where you can make good use of the almost non-existing performance degradation caused by CorePlayer�s reisizing the non-320/640-pixels-wide content to the screen. For example, take the example of the QVGA Nokia N95, which has an analogue TV output. A 320 pixels wide video is pretty much pixelizated; a 400-pixel-wide isn�t so. The latter is still on the verge of playability without (many) dropped frames (and, if you�re lucky enough, even without having to disable deblocking). Therefore, in cases like this, it�s preferable to create a 400-pixels wide video, which can be played back pretty well on both the built-in QVGA (320 pixels wide) screen and an external TV set. (Also see THIS for more info on this subject.)
1.4.8 High-bitrate tests
Earlier in this article, I�ve already presented some bitrate-related tests to find out what effect the stream bitrate has on the playback speed. I�ve also decided to test some additional situations � extremely large stream bitrates to find out how the players react.
Note that, in these tests, I used the ffmpeg codec (instead of the much more recommended beta CoreAVC port) to benchmark TCPMP�s AVC playback.
Also note that the first two tests show how official trailers like those of Get Smart can be played back. This also shows that high-res, official AVC trailers created for desktop players (read: no CPU optimizations took place: with CABAC, deblocking and Bidirectional Prediction are all enabled), in general, can�t be played back, not even on the, otherwise, fastest x51v.
As the previous chart, Windows Mobile-wise, only contains data on the x51v (as far as CorePlayer, the recommended player is concerned), I�ve repeated the tests on all the other test Windows Mobile Pocket PC�s, including the 400 MHz Samsung & ATI-based HTC Trinity.
1.5 Using Nero Recode
There is an excellent and pretty much up-to-date tutorial on using Nero Recode HERE. This is why I haven�t elaborated on basic subjects like importing a video file or a DVD. Keep in mind, however, that the subjects not discussed in the Doom9 tutorial (forcing the 320/640-wide output with manual resizing; unticking the two (three) checkboxes to dramatically increase performance etc.) are only discussed in my tutorial.
1.6 Verdict
With the hacks / performance improvements I�ve thoroughly explained in section �1.4 Fixing the frame drop problem�, I�m absolutely sure you�ll love AVC, particularly if you don�t mind having to recode your movies and/or you absolutely need the least possible storage usage. AVC, sometimes together with the hacks, is indeed a killer video compression format, even on low-power mobile platforms.
Last edited by Menneisyys; 12-16-2007 at 08:50 PM..
|
Threaded Mode