Double Blind Test (DBT)

Ambio

Banned
Banned
Joined
Jul 21, 2014
Messages
411
Points
0
Location
M'sia
Re: Are CD Player still relevant?

Which one should we poor audiophiles believe? Glassman, a regular forum contributor, or research papers approved by peers?

Unable to perceive in a random double blind test does not mean the problem does not exist. There was a paper recently that pretty much no one could distinguish between 16/44 and 24/96 in an ABX test. Does that mean no one should buy high res audio?
 
Re: Are CD Player still relevant?

Unable to perceive in a random double blind test does not mean the problem does not exist. There was a paper recently that pretty much no one could distinguish between 16/44 and 24/96 in an ABX test. Does that mean no one should buy high res audio?

I just buy music. If the hiRez sounds better than I will buy them. As a general rule, after buying over 100 SACD, DSD, DFF and HQ, I hardly buy hirez unless the music is faboulous.
 
Re: Are CD Player still relevant?

The question is not whether the problem exists or not, its about the sensitivity of its impact on the overall desired outcome, in this case, SQ. If there is NO significant, statistically speaking, variability in perception by human hearing in a randomized double blinded test, it simply means that for all practical purposes, the two samples/sources are equal in their intended impact/outcome. Are they equal in absolute terms? NO!, is the difference worth taking into consideration? NO!, unless of-course you want to tighten the tolerance of your double blinded test design itself, which is a moot point in my humble opinion.



On a lighter note without any offence to anyone, if one uses his power of reasoning and objectivity, the answer is a BIG NO! if one uses ears, ear drums, eustachian canal & other parts of human body you can very well buy anything that sells in the name of High res or audiophile goods & services :)

And there lies the problem with armchair objectivists. When presented with real world objective data about something being mathematically different and the reasons why, they stand behind the shield of a DBT which is completely a subjective test :lol:. Also FYI statistics is not science - it is an engineering solution to a potential unsolvable problem.

Not sure there's any point arguing this any more as this is purely a waste of time.
 
Last edited:
Re: Are CD Player still relevant?

...When presented with real world objective data about something being mathematically different and the reasons why...... Also FYI statistics is not science - it is an engineering solution to a potential unsolvable problem.
....

I must have missed the real world data. Are you referring to Glassman's post? And why do you say statistics is not science? Isn't it a branch t of mathematics which is also know as absolute science? :confused:
 
Re: Are CD Player still relevant?

And there lies the problem with armchair objectivists. When presented with real world objective data about something being mathematically different and the reasons why, they stand behind the shield of a DBT which is completely a subjective test :lol:. Also FYI statistics is not science - it is an engineering solution to a potential unsolvable problem.

Not sure there's any point arguing this any more as this is purely a waste of time.

Sorry ROC, don't intend to get personal but its not that there is no point in furthering the discussion, its just that you don't have anything worthwhile to consider. I would've thought twice before calling statistics as not being science! By extension you are saying probability theory is not science, modern drugs are not science, weather prediction that saves lives is not science, then what is science?

Here is the definition of the voodo, back magic, pseudo science as defined by American Statistical Association,

Statistics is the science of learning from data, and of measuring, controlling, and communicating uncertainty; and it thereby provides the navigation essential for controlling the course of scientific and societal advances (Davidian, M. and Louis, T. A., 10.1126/science.1218685).

FYI, the armchair objectivist makes a living out of working for RDBT and Aadaptive design industry that brings out novel drugs to patients. The next time around when you are in a hospital being administered a drug or therapy please recall that the life saver is a product of RDBT/statistics and not some pseudo science and dare you to ask the doctor to not give you any medicine that has been through RDB clinical trial.

Pulling out some random post from google is not real world data, the data you shared doesn't have any antecedent or precedent, share something credible.
 
Re: Are CD Player still relevant?

To bring some sanity into this highly polarized discussion, can the DBT supporters throw some light about its practicality. For an audiophile, it would make sense to know how to use this methodology to help choose an audio equipment which he is going to entertain himself.

For example, the man has a stellar pair of speakers and an excellent room. Now he is faced with this prolonged exercise of extended sighted auditions to decide what is a good amplifier and digital source.

To make his life easier, can you let us know an easier method ?
 
DBT is meant for elements of the chain,which are hard to distinguished in normal scenario. normally amp and speaker of different brands are not entertained in this, whereas cables, interconnect,hirez matters are considered.
Now this thread is doing competition with cable thread in terms of controversy. So why not this thread be closed, because IMO, DIGITAL stuff is still not in reach with many of us to understand, since for a simple 0 and 1 of digital there are several views and i will not be surprised if someone says that digital domain should entertain voodoo things.
 
Last edited:
Re: Are CD Player still relevant?

To bring some sanity into this highly polarized discussion, can the DBT supporters throw some light about its practicality. For an audiophile, it would make sense to know how to use this methodology to help choose an audio equipment which he is going to entertain himself.

Did I ever suggest that we should personally double-blind test every drug we take against a placebo? No. And I entirely admit, not only to the practical aspects, but to the pleasurable aspects of audition, sharing, buying. it's fun.

But why does all that mean that double-blind testing should be spat on and ridiculed?
There was a paper recently that pretty much no one could distinguish between 16/44 and 24/96 in an ABX test. Does that mean no one should buy high res audio?
a paper? It's worse than that: show me a double-blind test result in which people could tell the difference.

None so far. Not if properly conducted.

OK, it is not that I don't believe that it is possible. I believe the guy who reported that, with levels pumped up to almost painful, he could detect certain artifacts/differences, but I also believed him when he says that at normal, or even loud, levels, he couldn't.

But... these test results... don't they suggest that, vaguely, possibly, something is wrong? yes they do.

Now this thread is doing competition with cable thread in terms of controversy.

I don't know why, as neither CD playback nor PC playback are inherently controversial. Of course, for those that choose the high-sample-rate ladder, it has to be PCs, because they are not going to get that out of their CD player.
 
Re: Are CD Player still relevant?

For an audiophile, it would make sense to know how to use this methodology to help choose an audio equipment which he is going to entertain himself.

I don't know what the word Audiophile is supposed to mean in this context, so i'm not going to speak to that. Let's step back and clear the air a bit. DBT is not a "Decision making tool" its a simple way to remove biases in any comparison testing so that the results are objective, by "binding" the object & the observer from the subject being studied/compared and recording the response variables. DBT is a scientific testing methodology used in highly regulated industries, where human lives are involved, like Clinical trials - Please Google for more information.

Practical use: If someone told me that TEAC VRDS is a superior transport compared to Takai DV338. We will put both units in a similar black box ramdomize it so that no one knows what's inside which box, and then blind the listeners/testers if we plan on recording output data e.g. "jitter" and then use statistical tools on the recorded results/output data to either accept our initial hypothesis that VRDS is superior or reject it. [There are different statistical tools available for hypothesis testing, usually p-value is used when the data mimics normal distribution or bell curve]

Made the above example simple and generic for easy assimilation. The bottom line, its NOT a decision making tool, its a testing tool, based upon which one could arrive at a decision whether to buy Takai or VRDS.

Hope this helps.
 
Re: Are CD Player still relevant?

Did I ever suggest that we should personally double-blind test every drug we take against a placebo? No.

This may appear to be tangential to what we are discussing, but for the sake of information & knowledge, every drug that FDA or other regulatory bodies approves go through a Randomized DBT at a minimum, not just that, post marketing testing too involves complex designs of RDBT, so far that is the only proven scientific way drugs are brought to market. This is the main bone of contention between allopathy and homepathy (Thad, relax & let go of this one :eek:hyeah:) so the warring factionalism is not just limited to audio world, it extends beyond....
 
Last edited:
Re: Are CD Player still relevant?

I don't know what the word Audiophile is supposed to mean in this context, so i'm not going to speak to that. Let's step back and clear the air a bit. DBT is not a "Decision making tool" its a simple way to remove biases in any comparison testing so that the results are objective, by "binding" the object & the observer from the subject being studied/compared and recording the response variables. DBT is a scientific testing methodology used in highly regulated industries, where human lives are involved, like Clinical trials - Please Google for more information.

Practical use: If someone told me that TEAC VRDS is a superior transport compared to Takai DV338. We will put both units in a similar black box ramdomize it so that no one knows what's inside which box, and then blind the listeners/testers if we plan on recording output data e.g. "jitter" and then use statistical tools on the recorded results/output data to either accept our initial hypothesis that VRDS is superior or reject it. [There are different statistical tools available for hypothesis testing, usually p-value is used when the data mimics normal distribution or bell curve]

Made the above example simple and generic for easy assimilation. The bottom line, its NOT a decision making tool, its a testing tool, based upon which one could arrive at a decision whether to buy Takai or VRDS.

Hope this helps.

@Thx911,

I think I understand the DBT concept...having fought many battles over this on audio asylum when such arguments was the rage a decade ago :eek:

One of the reasons why I ( and many others for whom high quality audio reproduction at home ) decided to ignore the concepts therein was the absence of clear guidelines about its usage in our hobby.

If it cannot be used for making practical decisions for choosing audio equipment, it pretty much has only academic use. Hence the question.

For example, how do I choose between the Takai and VRDS using the dbt method?

- Do I need to do DBT in the prescribed format ?

- Can I keep both of them in another room and ask someone to switch between them without me knowing and use each device for 1 full day playing full albums and really live with the beasts for a while ? Remember, in this case I will use each of the devices for a full 1 day playing whatever I want and I may choose to do this exercise many times for a total of 20 days. So both devices are tested a total of 10 times. Lets say in this test, I choose the VRDS 10 times out of ten..Does this qualify as DBT ?
 
Last edited:
Re: Are CD Player still relevant?

..
- Do I need to do DBT in the prescribed format ?

- Can I keep both of them in another room and ask someone to switch between them without me knowing and use each device for 2 days playing full albums and really live with the beasts for a while ? Remember, in this case I will use each of the devices for a full 2 days playing whatever I want and I may choose to do this exercise many times for a total of 20 days. So they tested a total of 10 times. Lets say in this test, I choose the VRDS 10 times out of ten..Does this qualify as DBT ?

That will work as long you level match the volume. However, when you come to DAC, Preamp and Amplifier the mismatched output and input gain causes a pronounced difference. Here it is going to get technical but then I would just buy whichever sounds the best unless you know how to remove the variables.
 
Re: Are CD Player still relevant?

At the end of the day buying a music system is not a question of life and death. Each one of us have preferences. You are going to buy what you want to buy.

If a DBT shows one cable is better than the other, will we buy it? Not necessarily. If we don't want to buy, we will start questioning the basis of DBT. Obviously Asians hear differently from the Europeans. We like more treble. Europeans like it more mellow. Blah. Blah. Blah.

Would we buy a drug basis a DBT? Absolutely not. Its only if our doc recommends it, will we take it. In matters of safety, be it drugs or electrical appliances there are Government bodies sanctioning permits for their use. These bodies may use extensive technical and statistical tests to confirm their safety.

But for mundane products like audio, why would anyone do it? Ultimately the buyer is going to buy what sounds good to him. If i like vinyl, i will give links to 4 experts saying vinyl is the best and conveniently overlook 6 other experts opinion which challenges it. But thats human nature.

I only believe that one should indulge within ones budget and not overspend
 
Re: Are CD Player still relevant?

Prem, not all Europeans have same taste in sound. Brits like it mellow, but the Dutch (the Germans) are opposite. French like yet another type of sound. They like an aroma of romance in their sound. Swiss like perfection in details. And so on...
 
Re: Are CD Player still relevant?

Hi ranjeetrain

I just used it as an example saying people will start questioning a DBT, basis nationality :) Thats why i added a blah, blah in the end.
 
Re: Are CD Player still relevant?

That will work as long you level match the volume. However, when you come to DAC, Preamp and Amplifier the mismatched output and input gain causes a pronounced difference. Here it is going to get technical but then I would just buy whichever sounds the best unless you know how to remove the variables.

Yes, the following will be controlled:

Volume levels
Music
Listener
 
Re: Are CD Player still relevant?

Prem, Yes to most of your post, especially to the sense of perspective. Some of us certainly can loose that when we get into the heat of an argument, big-->:eek: from me!

But your thinking is not quite right here...

Would we buy a drug basis a DBT? Absolutely not. Its only if our doc recommends it, will we take it. In matters of safety, be it drugs or electrical appliances there are Government bodies sanctioning permits for their use. These bodies may use extensive technical and statistical tests to confirm their safety.
Double-blind tests are very much a part of that confirmation. This is why I ask for double-blind testing, if not by us in our homes, then further up the chain. The marketing men would hate it, but at least some of the engineers and real developers would probably welcome it. To paraphrase something I read from Sean Olive recently: better tests lead to better products. That should be an all-round win.
 
Re: Are CD Player still relevant?

But to some A US700 system sounds identical to US12000's.
 
Re: Are CD Player still relevant?

But to some A US700 system sounds identical to US12000's.

The same is true in every hobby. Eg people buy 32"TVs while others buy projector setups worth thousands of dollars, or US$100 use and throw cameras vs medium format system which cost US$10k+ for a basic kit.
 
Buy from India's official online dealer!
Back
Top