04-12-2004, 08:30 PM
|
|
Executive Editor
Join Date: Aug 2006
Posts: 29,160
|
|
Extreme Tech's Audio Codec Quality Shootout
"Millions of people convert their CD collections to a compressed digital audio format, often without considering the format or settings they use. With several different popular formats and lots of quality settings, many users just pick an application they like and use whatever the default settings are. ExtremeTech readers -- generally a bit more hands-on and do-it-yourself -- are more likely to seek out other applications or players and encode their audio exactly how they want.
Certainly a top consideration should be compatibility. We all have different audio devices, and it doesn't matter how good your music sounds if it won't play on your hardware. We'll leave that decision up to you; you know what software and devices you have and what they're capable of playing. Encoding time is another possible consideration, but honestly, modern computers encode audio so quickly that the difference in speed between one format and another is practically a non-issue."
This is one of the best, most hard-core looks at audio codec quality that I've seen. If you've been wondering about what audio format to use for your portable music player (whether it be an iPod, Pocket PC, or Smartphone), this article is a good substitution for doing all the testing yourself. I won't ruin the article for you, but let's just say that I wasn't surprised by the final result - I made the switch myself about a year ago, and have been very pleased with the results. It's also greatly simplified my ripping/encoding process, saving me time, which is a good thing.
Ultimately, however, it's important to realize that the best codec for me might not be the best one for you - it depends on factors such as speakers, headphones, music type, and the most important one: your ears. Check out the article - it's a great read!
|
| |
|
|
|
04-12-2004, 10:55 PM
|
|
Pupil
Join Date: Apr 2004
Posts: 29
|
|
This is a very biased review, so much so that I wonder if MS paid them to do the test. The article gets blasted in the comments and over at Hydrogen Audio - http://www.hydrogenaudio.org/index.php?showtopic=20443.
Some major flaws of the test:- *Only 6 listeners
*They try to make the listening group out to be both a group of experts (which they're not- they admit one of the listeners is "even a young child") and a "real-world listening test" (which it's not- with a sample that small, their results are irrelevant)
*Of those 6 listeners, some of them gave 5 (the highest rating in their scoring system) to the 64k samples
*They are very unspecific as to their settings (huh?! 98% quality? Is that 313.6 ABR? --alt-preset insane with 2% of the audio spectrum cut off? what is it?)
*Their "98% quality" VBR scores were lower than their 128k scores (they speculated that it was "because they are actually more accurate than the 128k versions")
Sorry for the rant, but it really irks me when people try to pass off advertising as unbiased journalism. I do agree that WMA has good compatibility, but it is not as leaps and bounds ahead of MP3 as they claim. In *unbiased* listening tests, AAC has proven itself to be a much more transparent codec than WMA, especially at higher bitrates (which they conveniently left off because "iTunes doesn't support VBR AAC"). They also freely admit that OGG sounds the best of all codecs at 64k (in the low bitrates, where it was optimized to sound best). If device support was a non-issue, I'd do all my high bit-rate encoding in AAC and all my low bit-rate encoding in OGG. However, since I am concerned with compatibility, I just do everything in MP3 because it is even more ubiquitous than WMA (it plays in my car CD MP3 player, my portable MP3 CD players, my Pocket PC, my computer, and my DVD player).
|
| |
|
|
|
04-12-2004, 11:08 PM
|
|
Executive Editor
Join Date: Aug 2006
Posts: 29,160
|
|
Quote:
|
Originally Posted by SubFuze
This is a very biased review...
|
Heh. Interesting - I found it to be the exact opposite. I think if anything can be learned from this article, looking at the scores, the reality is that at 128 kbps and higher, it's difficult for the average person with average audio speakers/headphones to tell the difference between what each codec offers. If you have an iPod, you're going to use AAC. If you don't, WMA offers better compatibility with devices while still maintaining great audio quality. MP3 still has the best compatibility though, and if you're encoding at 160 kbps, I doubt most people could tell the difference - so I think they shorted MP3 a bit in the article. All sorts of car stereo decks play MP3, but finding one that supports VBR WMA 9 is a bit of a challenge (at least in these parts).
Saying that it's a MS-sponsored piece is pretty insulting BTW. :?
|
| |
|
|
|
04-12-2004, 11:38 PM
|
|
Neophyte
Join Date: Feb 2003
Posts: 7
|
|
This article seems to have really stirred the passions of audio enthusiasts and I'm not sure that I understand why. The author clearly stated at the beginning that the tests were based on a very small sample size (and here I do agree that six people really don't create any kind of significance, statistically speaking) and that it was specifically done without the usual controls and constraints of a scientific inquiry. They were really just seeking some real-world opinions and that's exactly what they published. It's simply another way of comparing audio codec performance and seems like a worthy look to me, yet there is so much fervor over it's lack of merit as a comparison.
According to the author's instructions, it's meant to be more of an editorial piece than a comparative study, but people seem to refuse to take it for what it's intended to be <insert heavy sigh here>. Oh well.
|
| |
|
|
|
04-12-2004, 11:52 PM
|
|
Sage
Join Date: Mar 2004
Posts: 735
|
|
Re: Extreme Tech's Audio Codec Quality Shootout
Quote:
|
Originally Posted by Jason Dunn
Ultimately, however, it's important to realize that the best codec for me might not be the best one for you - it depends on factors such as speakers, headphones, music type, and the most important one: your ears.
|
Well said and cannot agree you with your more.
If you cannot tell the difference between a 64k MP3 and 256k WMA, great, you save space! If you have to have play vinyl with a vaccum tube amp and nothing else, it's great that you appreciate and enojoy your hobby!
As soon as people realize that they should be their own best judge on what codec to use, the less forum battles need to be devoted to audio codec and be then used to talk about the never-ending topics of PPC vs Palm, Apple vs Windows vs Linux, religion, politics, etc! 
|
| |
|
|
|
04-12-2004, 11:55 PM
|
|
Pupil
Join Date: Apr 2004
Posts: 29
|
|
Quote:
|
Originally Posted by Jason Dunn
Heh. Interesting - I found it to be the exact opposite. I think if anything can be learned from this article, looking at the scores, the reality is that at 128 kbps and higher, it's difficult for the average person with average audio speakers/headphones to tell the difference between what each codec offers. If you have an iPod, you're going to use AAC. If you don't, WMA offers better compatibility with devices while still maintaining great audio quality. MP3 still has the best compatibility though, and if you're encoding at 160 kbps, I doubt most people could tell the difference - so I think they shorted MP3 a bit in the article. All sorts of car stereo decks play MP3, but finding one that supports VBR WMA 9 is a bit of a challenge (at least in these parts).
Saying that it's a MS-sponsored piece is pretty insulting BTW. :?
|
I guess one of the reasons I found it to be so biased is that it contradicts almost every other listening test that I've ever seen. I agree that for the most part, above 128k, most people can't tell much of a difference, however I can and so I'm much more picky about my codecs than most. Because of my pickiness, I've been reading about encoding methods and codecs for years. Currently, I encode all of my CDs with LAME MP3 and use --alt-preset extreme which gives me about a 224k average bit rate (which is a good balance between quality and size for me since most of my portable devices are CD based). At that high of a rate, the difference between codecs is so small that it's not worth losing compatibility.
I in no way meant my speculation about their motives for writing the article to be a slam on either MS or your site. As far as enthusiast communities go, PPCT is one of the best at realizing where to draw the line between enthusiasm and propaganda. If there are problems with MS, they're freely discussed and never censored. The ads are always clearly marked as such and I've never felt like anyone here was paid-off to be a supporter. That's why this is one of very few sites that I visit daily  .
Honestly, I'd be happy if WMA was the best codec because of the widespread support, but from almost every other listening test I've seen, it just hasn't been. WMA has come a long way, but it's still not the top codec in my opinion. As such, I'd love to see wider support for more codecs in more devices (I'd love to see that regardless, having more choices is (almost) always a good thing). That's one of the reasons I love my Pocket PC- it doesn't just play WMA and MP3...
|
| |
|
|
|
04-13-2004, 12:21 AM
|
|
Pupil
Join Date: Sep 2003
Posts: 40
|
|
I thought the review was pretty decent. I'm not surprised Ogg did well in the low end, that's generally recognized. WMA usually sounds quite a bit better than mp3 (esp at low-end, where mp3 just fails imo). And i haven't listened to aac much. at lower bitrates i prefer to use wma because it's convenient.
On computer i rip to 192 kbps mp3, with 160 sometimes i can hear a few artifacts, 192 is nearly unnoticeable from original (for me).
|
| |
|
|
|
04-13-2004, 01:45 AM
|
|
Thinker
Join Date: Dec 2004
Posts: 315
|
|
my opinion on wmv9 has changed some what since I technicaly see its actually pretty hot at high bitrates. Im now wondering if players like my iriver 250's wma codec properly decodes wma9. I know it CAN decode wma9, but is it chinsing on quality in a sort of compatability mode? or does it fully decode it like media player 9 would?
As for the real world test, 6 subjectes was so poor they should have saved the time. I have NO idea why they didn't just burn a crap load of these cd's off, and offer them for free with a checklist. 6 is just..sad.
|
| |
|
|
|
04-13-2004, 04:41 AM
|
|
Ponderer
Join Date: Mar 2004
Posts: 99
|
|
i don't really consider myself that much of an audiophile, but i can DEFINITELY hear the difference between 160kbps mp3 and anything lower through even cheap headphones... in fact i would go so far as to say that i would call anyone out if they say they CAN'T hear the difference on headphones.
what i REALLY don't get is why anyone willing to plunk down a couple hundred bills would then decide to shortchange their music, when it's easy to upload music to any mp3 player made... especially if it's a hdd based model which already holds more music than anyone could possibly listen to on a month long vacation...
just my opinion though... i could certainly be wrong...
|
| |
|
|
|
04-14-2004, 12:35 AM
|
|
Intellectual
Join Date: Feb 2004
Posts: 152
|
|
Same here, anything encoded below 160kbps sounds terrible to my ears (except maybe AAC), so that someone could give a "5" to something encoded at 64kbps is cause for concern.
I remember when WMA first came out in 1999 (1998?) they grabbed people in supermarkets and made them listen to mono samples and they said it was better than mp3...
|
| |
|
|
|
|
|
Linear Mode
