android
Go Back   abi>>forums > Software & Media > Software

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 03-27-2012, 04:36 PM
Enigmatic Enigmatic is offline
Senior Member
 
Join Date: Mar 2008
Posts: 731
Default *Discussion Thread* FLAC or MP3? How Can I Determine Which Is Best for My DAP?

************************************************** ******************************************
These posts were here moved from the original sticky. Some of the references will therefore seem as if this was the original. That became too unwieldy in that form as a reference so most of it was moved here.
************************************************** ******************************************

Our members requested that this thread be made a sticky. Did you ever wonder why some of our members recommend against using lossless files on digital-audio players? If yes, this is the post for you. The information in this post was spread across several dozen threads.

This thread is just to let you know what is the best way to determine which is the best audio format for you—nothing more, nothing less. You can accomplish this by using your ears and your equipment to listen for differences once variables that can fool you have been removed. This is done with ABX testing. Let us make one thing clear.

No one is trying to tell you what files to use under any circumstances. You should use the files that suit your purposes best. This sticky was created so that an easily referenced set of directions is available. You can now decide what is best by using ABX testing to make an apples-to-apples comparison of various audio formats.

One of the early titles for this thread was FLACs and MP3s. Unfortunately, some read that as FLACs versus MP3s and felt that there was an attempt being made to demonstrate that lossless and lossy files were equal under all circumstances. No. Sorry for the long introduction, but considering the way that this thread has been derailed in the past, we felt that it was best to try to clarify the purpose of this thread.

The difference in sound quality between FLACs and MP3s appears to be a popular topic. This post, therefore, will attempt to answer some of the questions about the sound quality of FLACs compared to the sound quality of MP3s. We have taken part in many discussions on this subject in these forums. Many members have raised objections to what we are going to present.

So, for this reason, we are presenting this post as a conversation between ourselves and a notional member. This notional member will be a composite of some of the members that have raised objections to what we are about to present. What the notional member will have to say will be in italics.
---------------------------------------------------------------------------------------------------------------------------------------



I can hear a big difference between FLACs and MP3s. So can many other individuals. So why all these so-called discussions?

This is actually an excellent question, because the answer is rather complicated. First, you must be aware of the placebo effect. Because of the placebo effect, it is not enough to just listen to a FLAC and then just listen to an MP3. What is needed is an ABX test.



But I have done blind listening before, either by myself or with help from my friends.

That is good, but an ABX test is better because it is one of the most rigorous ways to evaluate the sound quality of lossy files. It is a true double-blind test, and the best way to listen for extremely subtle compression artefacts.



But no one listens to music this way. Sorry, but I really do not see the need to do any ABX tests at all. I have better things to do with my time.

Please read item 4 in this PDF. Because of the placebo effect, many myths are associated with audio. Some unscrupulous individuals have seized this as an opportunity to make money from these individuals who are experiencing the placebo effect.

ABX tests can, therefore, be very useful in leveling the playing field. Yes, ABX tests take time and effort. But it really is the better way to evaluate sound quality. ABX tests have been used to debunk audio myths. Please see, for example, the following.

Fancy wires sound better than ordinary wires debunked.
• The so-called high-resolution formats sound better than CDs debunked here and here.
Digital-audio recordings sound bad debunked.
• Expensive amplifiers sound better than ordinary amplifiers debunked here and here.
More audio myths debunked.



Well, OK. So how does one use an ABX test to evaluate the sound-quality difference between FLACs and MP3s?

Please download foobar2000 and its ABX comparator. They are free downloads. To ABX, please do the following. But first, a couple of recommendations before you start.

• We recommend that you use headphones, Exact Audio Copy or something similar, the newest stable version of LAME—e.g., 3.99.5—and quality settings –V0 to –V9. If you use an MP3 encoder other than LAME, use the newest stable version of that encoder. You should also use a wide range of quality or bitrate settings.

• You need an uncompressed WAV and a perceptually encoded—lossy—copy of the uncompressed WAV. A FLAC or other types of lossless copies of the uncompressed WAV is also OK instead of using an uncompressed WAV. The audio information in any lossless format is the same as in the uncompressed WAV, so using any lossless format instead of an uncompressed WAV allows the same comparison.

1. Open foobar2000, then the File menu. From there, click Add Files. Navigate to where you have the files you want to compare stored. Click on the lossy or lossless file that you want to compare. Repeat to add the other file that you want to compare. Some may find it easier to just drag the files into an open playlist pane.

2. In the open playlist pane, select both of the files that you want to compare and then right click. From the context menu that opens, select Utils. Select ABX Two Tracks. A new window will open.

3. In that window, select the options that you want to apply to your test. You can choose to use ReplayGain if the tracks have that information in their tags. You can also choose to use digital-signal processing (DSP). Once you have made your choices, click OK.

4. In the new window that opens, make the final choices that you want to apply to your testing. You can choose to keep the playback position during track changes or not to keep the playback position during track changes. We recommend Keep playback position when changing track. You can also choose to see the results as you test or to hide them until you are finished with the test.

saratoga recommends that the Hide results box be checked, because “seeing how good or bad you're doing can really mess with your head and throw off your concentration.” “It'll work much better if you don't see the numbers since you won't bias yourself.”

5. Play A and B to get a good idea of how they sound. Unless there is a big difference between the two files, it is not unusual to not be able to hear a difference at this point. Select Play X or Play Y.

6. Compare X or Y to A and B. You can choose to listen to the entire track or just a part of it. You can choose to listen from the beginning or any starting point. These choices can be changed at any time during any part of a single trial in the course of the test.

They can then be changed again in the next or any other following trial in the course of the test. Once you believe that you can hear a difference, choose one of X is A, Y is B or Y is A, X is B. This choice is not locked in until you select Next Trial.

7. Once you have completed the number of trials you have decided on before the start of the ABX test—it must be at least 5 trials—select Exit. A window will come up to allow you to save the results. The saved results will include the files used and the percentage of correct identifications among other information.

8. If Probability that you were guessing reaches 5% or lower, you can hear a difference. If Probability that you were guessing does not reach 5% or lower, you cannot hear a difference.



Why is the minimum number of trials 5?

First, you must know a little about the p-value. For you to legitimately claim to be able to hear a difference, the p-value must be 0.05 or lower. Values like 0.07 or 0.06 are not good enough. Next, please take a look at this ABX binomial probability table. So from this table, we can see that 5 is the minimum number of trials.



Is it also possible to ABX using loudspeakers?

Yes. But, as mentioned earlier, because the compression artefacts can sometimes be extremely subtle, many individuals prefer to use headphones. But if you prefer to use loudspeakers, there is no reason not to. You can also try ABXing with loudspeakers and then with headphones or first with headphones and then later with loudspeakers.



Do I need extremely expensive and high-quality sound cards, headphones, or loudspeakers?

No. You can use any sound cards, headphones, or loudspeakers.



Why use Exact Audio Copy?

http://wiki.hydrogenaudio.org/index....act_Audio_Copy.



Why use LAME?

http://wiki.hydrogenaudio.org/index.php?title=LAME.



Why use the newest stable version of LAME?

LAME is always undergoing improvements.



Why use the quality settings –V0 to –V9?

http://wiki.hydrogenaudio.org/index....te.29_settings.



So why is FLAC such a popular format?

FLAC is great for external hard drives and for archiving. And it is also great if you must have a perfect copy of the music on your digital-audio player. But if you are only after great sound quality, MP3 is often a better choice because of the much smaller size. If you have followed all of the recommendations given above and have actually done some ABX tests, I think you too will realize that properly encoded MP3s sound very similar or the same as FLACs.



What do you mean by very similar?

There are rare cases of some music tripping up lossy encoders no matter how good the lossy encoder.



So should I not then be using FLACs on my digital-audio players instead of a lossy format like MP3?

How many songs did you ABX at –V0? Out of all of these ABX tests at –V0, how many times has the p-value reached 0.05 or lower? Can you honestly claim that, from the music you typically listen to, you always or often hear a difference in ABX tests done at –V0?



What about AAC or Vorbis? Should I also ABX these types of files? What about other lossy formats?

Sure. If you want to, why not? We at anythingbutipod always encourage members to expand their knowledge.



If I still believe that MP3s sound bad or that FLACs sound much better than lossy formats such as MP3, is anyone here going to give me a hard time?

We might if, after reading this far, you still refuse to do ABX tests. If you are going to start making claims, you better have some evidence to support your claims such as the results of ABX tests that you have done.



Why are you guys so insistent on ABX tests?

We take pride in trying to give good advice.



---------------------------------------------------------------------------------------------------------------------------------------
This post was written by skip252 and Enigmatic and edited by Enigmatic. This post may undergo further edits. Thank you, WalkGood, dfkt, saratoga, and The DarkSide.

Last edited by skip252; 05-21-2012 at 07:00 PM.
Reply With Quote

Advertisement [Remove Advertisement]

  #2  
Old 03-27-2012, 06:58 PM
The DarkSide's Avatar
The DarkSide The DarkSide is offline
Ultra Senior Member
 
Join Date: Feb 2008
Location: Qns Vlg, NYC
Posts: 16,054
Default

A necessity,...thank you for the information.
Reply With Quote

  #3  
Old 03-27-2012, 07:00 PM
WalkGood's Avatar
WalkGood WalkGood is offline
Moderator
 
Join Date: Aug 2006
Location: FL
Posts: 12,579
Default

Quote:
Originally Posted by The DarkSide View Post
A necessity,...thank you for the information.
Good point TDS, you actually gave me the idea to ask for it, thus the sticky ...
__________________
WalkGood,
Ramón

abi >> | Forum Rules | Glossary | Why Rockbox | FLAC or MP3? | irc
Reply With Quote

  #4  
Old 03-27-2012, 07:42 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

Damn never thought I'd see a sticky thread for this. I can't hear a difference between FLAC and 320MP3 for crap lol. I just convert ripped FLAC files to 400kbps AAC for my DAPs and thats it,also I'm too lazy to do most of these ABX stuff,and I don't really care much about the space being used much lol I don't like using Foobar2000 much thus don't bother setting up the ABX thing. I kinda became fond of the AAC format after buying some songs from iTunes and then started converting FLAC files to the highest AAC bitrate instead of MP3 like I used to. So AAC became my main convert to format for my DAPs. I don't need lossless on my DAPs,however I DO want the highest quality lossy audio files on them.
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #5  
Old 03-27-2012, 07:46 PM
McDougal's Avatar
McDougal McDougal is offline
Ultra Senior Member
 
Join Date: Aug 2009
Posts: 2,375
Default

A sticky that was needed, thanks for adding, WalkGood.

@Lagoo56, yeah, AACs are decent. However I still rip to MP3 due to universal compatibility.

But, I prefer downloading music from iTunes now. Their 256VBR AAC format I believe is more efficient than Amazon MP3's CBR files. I know, Amazon does use -V0 which is great, but I hardly ever get -V0 files . If Amazon only used VBR/ABR, then I'd still be shopping there.
__________________
iPod Touch 4G 32GB; Google Nexus 7 32GB
JVC HAS600B;Pioneer SE-M390;Audio-Technica ATH-P5

Retired Devices:
Sansa Fuze 8+16GB Rockboxed
Reply With Quote

  #6  
Old 03-27-2012, 07:53 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

Quote:
Originally Posted by McDougal View Post
A sticky that was needed, thanks for adding, WalkGood.

@Lagoo56, yeah, AACs are decent. However I still rip to MP3 due to universal compatibility.

But, I prefer downloading music from iTunes now. Their 256VBR AAC format I believe is more efficient than Amazon MP3's CBR files. I know, Amazon does use -V0 which is great, but I hardly ever get -V0 files . If Amazon only used VBR/ABR, then I'd still be shopping there.
The problem with Amazon is that all of their music files varies,so you never know what bitrate your MP3 file will be when you purchase one,thats why I use 7digital since they only sell 320CBR files. I was pissed when I got a 256CBR file from Amazon once and it sounded like crap to my ears,then I bought the same song again from 7digital and it sounded much better.
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #7  
Old 03-27-2012, 08:14 PM
McDougal's Avatar
McDougal McDougal is offline
Ultra Senior Member
 
Join Date: Aug 2009
Posts: 2,375
Default

Quote:
Originally Posted by Lagoo56 View Post
The problem with Amazon is that all of their music files varies,so you never know what bitrate your MP3 file will be when you purchase one,thats why I use 7digital since they only sell 320CBR files.
Yes, this is the reason why I don't buy from them anymore. The only pattern I have found is any music that is sold by "SONY Music Entertainment" is LAME3.98b ABR 256, which is ok with me. It would be fairly easy for them to tell you what bitrate it's at before you buy, they're just too lazy to do so.

I actually try to avoid 320CBR files, and CBR files in general (waste of space imo). With iTunes, I know what I'm getting, and I know how Apple encodes their music as well (Amazon I have no idea). If only iTunes prices were as good as Amazon, lol .
__________________
iPod Touch 4G 32GB; Google Nexus 7 32GB
JVC HAS600B;Pioneer SE-M390;Audio-Technica ATH-P5

Retired Devices:
Sansa Fuze 8+16GB Rockboxed
Reply With Quote

  #8  
Old 03-27-2012, 08:18 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

I'm content with 320CBR and 400VBR AAC files. My problem with LAME -V0 is that,since its varied,I don't trust that the end result will always end up sounding its best,therefore I just go for the highest lossy bitrate I can since I'm too lazy to convert to -V0 then have to listen to all of the songs individually to check if they sound good enough,forget good enough,I'd rather go for insane guaranteed quality and not have to think about whether the end converted result will be good enough
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #9  
Old 03-27-2012, 08:32 PM
saratoga saratoga is offline
Rockbox Developer / Moderator
 
Join Date: Apr 2007
Posts: 3,627
Default

Quote:
Originally Posted by Lagoo56 View Post
I'm content with 320CBR and 400VBR AAC files. My problem with LAME -V0 is that,since its varied,I don't trust that the end result will always end up sounding its best,
But you're ok with VBR AAC?

Quote:
Originally Posted by Lagoo56 View Post
therefore I just go for the highest lossy bitrate I can since I'm too lazy to convert to -V0 then have to listen to all of the songs individually to check if they sound good enough,forget good enough,I'd rather go for insane guaranteed quality
This is silly. Just use V0. The codec will pick a high enough bitrate for each frame.
__________________
Interested in Google's Summer of Code ? PM me.
Reply With Quote

  #10  
Old 03-27-2012, 08:35 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

I've used -V0 before and don't like it so I threw away the idea. As for 400VBR,its fine because its already only staying in the near 400 bitrate range. -V0 doesn't work well for all music,therefore I go for something that I'll be content with. It may seem silly to you but I prefer a simple 3 step system and not have to worry about bad sound quality in the end.
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #11  
Old 03-27-2012, 08:37 PM
McDougal's Avatar
McDougal McDougal is offline
Ultra Senior Member
 
Join Date: Aug 2009
Posts: 2,375
Default

Yeah Lagoo56, I'd go with saratoga's advice and use -V0. I've never had a problem.....once in the 256 range everything starts sounding identical to me anyway. But if you have the file space then you could use 320CBR, I guess....
__________________
iPod Touch 4G 32GB; Google Nexus 7 32GB
JVC HAS600B;Pioneer SE-M390;Audio-Technica ATH-P5

Retired Devices:
Sansa Fuze 8+16GB Rockboxed
Reply With Quote

  #12  
Old 03-27-2012, 08:44 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

I have all the space I need for high bitrate files so I'm happy. I'm more concerned about having enough space for all the video files I always download lol but thats a different topic for another day lol

But anyways who cares about bitrate? Find your favorite bitrate and audio type and stick with it then shut up and just enjoy the music because you DID buy the music so you can enjoy it,right?

I found my favorite bitrate and audio file type and I'm content with what I picked.
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #13  
Old 03-27-2012, 09:29 PM
steinburger1109's Avatar
steinburger1109 steinburger1109 is offline
Ultra Senior Member
 
Join Date: Feb 2009
Location: Naperville, Illinois
Posts: 2,456
Default

I'd heard that v3 is the drop off point for many. Upon ABXing, however, I determined within a fifteen percent chance (which is good enough for me) that v4 its when I notice sometimes quality. I know most people consider 5 percent to be the point when you're sure, but I don't care enough lol

While I think Lagoo's methods are silly, as well, oh well. At least he's not using FLAC. Haha
__________________
- Justin
Glossary for Newbies Everybody
Babble. On.
Reply With Quote

  #14  
Old 03-27-2012, 10:11 PM
Satellite_6's Avatar
Satellite_6 Satellite_6 is offline
Senior Member
 
Join Date: Feb 2009
Location: VA, USA
Posts: 871
Default

FLAC would probably make more sense than 400 kbps AAC.

LAME preset insane ftw. . . :P

Actually I just use -v0 now for use on a car's stereo and nothing else.
__________________
satelliteaudioblog.blogspot.com
Reply With Quote

  #15  
Old 03-27-2012, 10:15 PM
Marvin the Martian's Avatar
Marvin the Martian Marvin the Martian is offline
Ultra Senior Member
 
Join Date: Oct 2008
Location: east central NY state
Posts: 10,602
Default

Quote:
Originally Posted by Satellite_6 View Post
FLAC would probably make more sense than 400 kbps AAC.
What about Vorbis at Q10?
__________________
iPod Touch 5G 32GB, Touch 4G 32GB, Clip Sport 8GB. Rockbox-> Clip Zip 4GB, iPod Nano 2G 4GB, iPod 5.5G 80GB
2012 Nexus 7 32GB, Asus MeMoPad 8 16+64GB, LG Optimus G Pro, Nokia Lumia 900 and Lumia 520
Reply With Quote

  #16  
Old 03-27-2012, 10:33 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

Quote:
Originally Posted by Marvin the Martian View Post
What about Vorbis at Q10?
Now thats just overkill lol

I kinda pick my bitrates based on favorite numbers alone lol

I hate the number one because its too alone,and two kinda sucks,but 3 is a cool number and 4 is my favorite number aside from 3 lol 5 is just ridiculous because it multiplies too much.

/Of course sarcasm is involved here,don't take this favorite number stuff seriously lol
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

  #17  
Old 03-28-2012, 07:00 AM
The DarkSide's Avatar
The DarkSide The DarkSide is offline
Ultra Senior Member
 
Join Date: Feb 2008
Location: Qns Vlg, NYC
Posts: 16,054
Default

I only do LAME VBR V~0 FTW!!! Not because I hear a difference, but just because I like it like that.
Reply With Quote

  #18  
Old 03-28-2012, 02:05 PM
McDougal's Avatar
McDougal McDougal is offline
Ultra Senior Member
 
Join Date: Aug 2009
Posts: 2,375
Default

Lame -v2/-v0 FTW !!
__________________
iPod Touch 4G 32GB; Google Nexus 7 32GB
JVC HAS600B;Pioneer SE-M390;Audio-Technica ATH-P5

Retired Devices:
Sansa Fuze 8+16GB Rockboxed
Reply With Quote

  #19  
Old 03-28-2012, 02:38 PM
Syndrome's Avatar
Syndrome Syndrome is offline
Redneck Moderator
 
Join Date: Dec 2007
Posts: 1,278
Default

Most noticable thing about FLAC over mp3 for me is the sound stage. I can't really tell on a portable, but with my PC and my speakers then I can hear the difference.
__________________

Ruger M77 , Howa 1500 , Czech Republic 452-2E
Ruger .22 Single Six, Springfield Armory XD-9 , Springfield Armory XD45ACP
2007 Elite Archery Synergy

Gamo CFX, Gamo PT80
Large Rock


Need your headphones fixed or something soldered? Syndrome's Soldering Service
Reply With Quote

  #20  
Old 03-28-2012, 03:25 PM
Lagoo56's Avatar
Lagoo56 Lagoo56 is offline
Senior Member
 
Join Date: Dec 2008
Location: SI,NY
Posts: 737
Default

Quote:
Originally Posted by Syndrome View Post
Most noticable thing about FLAC over mp3 for me is the sound stage. I can't really tell on a portable, but with my PC and my speakers then I can hear the difference.
+1,Glad I'm not the only one who notices the soundstage difference.
__________________
DAPs I own: Rockboxed Sansa C250 2GB+64GB MicroSDXC
Tablets I own:Rooted 8GB Samsung Galaxy Tab 2 7.0 running Sungsonic 2 HD JB 4.1.2
Rooted 80GB Asus Memopad HD 7
Reply With Quote

Reply

Tags
flac vs mp3

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 09:50 AM.