A practical blind test : Just for kicks

[Thad E Ginathom]

There is no reason not to do this! There is no reason not to reproduce the Matrix HiFi group test. It would be interesting to do so, The more tests and experiments the merrier!

It seems like you are hell bent on making a killer system for sysA and a very lousy system for sysB and hence claiming that blind testing is crap since it produced a bad system (using coat hangers and dvd players and AVRs?). I seriously doubt your 'unbiased' intentions. Looks to me like you have already made up your mind. Me, I have already made up my mind, feel free to change it by giving a demo that it makes a difference.

It's a curious world. System A could be a $5 system. Matrix used 700 against 12,000 (with speakers and speaker cable being constant). We will still go out and spend our thousands, either actually or in dreams.

We are trying to figure out if any other methods other than the blind testing one can yield a better or worse sounding system.

I'm not sure this is the right question. The methodology is proven, and, where, for instance, health is concerned, is demanded (and still fudged for commercial motives!) in all [?] technical fields. It's just that the hifi industry/community refuses to play. It wants to play the science game, but won't play by science rules.

 

[captrajesh]

@ capn,
These are results from various blind testing experiments.

With the coat hanger, I said it will since it has been proven in a Blind test that one cannot any more differences.

With the dvd player and amp I said probably. So I am not sure. It may go further although I have read somewhere they proved that some ~ 200 $ pioneer receiver and an esoteric amplifier could not be differentiated in a double blind test.

As far as my experience goes, different people have different agenda and at times, hidden and even mala fide. Hence I take in all the information with a fair bit of caution. I feel one should have an open mind and go by one's own experience in matters like these.

The cable and source differences are so evident to me that it is a no brainer AFA I'm concerned. I'm ready to stick my neck out.

 

[square_wave]

It seems like you are hell bent on making a killer system for sysA and a very lousy system for sysB and hence claiming that blind testing is crap since it produced a bad system (using coat hangers and dvd players and AVRs?). I seriously doubt your 'unbiased' intentions. Looks to me like you have already made up your mind. Me, I have already made up my mind, feel free to change it by giving a demo that it makes a difference.

A source contains a DAC as well as a transport. How about using oppo bdp se95 for the blind test system. its just a stupid bluray player with a decent dac right. Anyway, how can a video player with tons of circuit for a/v processing beat a cd player (same logic used by audiophiles for saying amp is better than AVR). how about using a marantz cd6004, a 'pure' cd player for the audiophile's system. Fair comparison? Anyway, as per your criteria, price doesnt matter right?

@doors..cool down

I dont think you have understood what is being done. You will need to re-read the whole thread. You dont need to use using coat hangers and dvd players and AVRs for System A. You can use anything that passes the test in the blind testing procedure.

The audiophile is free to choose anything for System B.

The constant is the speaker and room. The speaker will be state of the art capable of revealing the minutest changes in the upstream gear. There will be electrical synergy among the gear in both the cases.

 

[square_wave]

I'm not sure this is the right question. The methodology is proven, and, where, for instance, health is concerned, is demanded (and still fudged for commercial motives!) in all [?] technical fields. It's just that the hifi industry/community refuses to play. It wants to play the science game, but won't play by science rules.

Okay.I think we both are not on the same page here. I think we need to come to some common ground before we can proceed.

Double blind testing is a tool used to eliminate components which do not actually perform better than some standard right ? So what is this standard ? Who sets it ?

Now, if we arrive at this standard, is it not possible to test each component against this standard using double blind method to come up with a system ? If we arrive at such a system, isnt that system the best in the world according to double blind testing believer camp ?

If we are not in agreement, there is no point in proceeding any further.

 

[doors666]

Then why are you bringing up coat hangers and dvd players:lol:
Such experiments have been done in the past. who cared? did we learn? nope.
I did post in one of the thread where a boutique audio shop owner, audiophile having some 25k$ system, the amp in his system was changed with a 200$ AVR and he couldnt tell the difference. There was one other test, i think it was boston audiophile society, similar to yours, where IIRC the speakers were constant and the rest of the chain was changed (expensive audiophile gear changed with dvd player kind of stuff) and some 70% of the people got it consistently got wrong which is which.

 

[square_wave]

Then why are you bringing up coat hangers and dvd players:lol:
Such experiments have been done in the past. who cared? did we learn? nope.
I did post in one of the thread where a boutique audio shop owner, audiophile having some 25k$ system, the amp in his system was changed with a 200$ AVR and he couldnt tell the difference. There was one other test, i think it was boston audiophile society, similar to yours, where IIRC the speakers were constant and the rest of the chain was changed (expensive audiophile gear changed with dvd player kind of stuff) and some 70% of the people got it consistently got wrong which is which.

Point noted. No arguments with you Doors It was a spur of the moment post since there is a history to those particular tests. For example, in the Matrix test you can see that in a blind test you cannot make out differences between a Wadia state of the art player and some sony cheapo dvd player. So what do I do ? Can I not appoint a sony dvd player as the player of choice for System B ?

 

[doors666]

Can I not appoint a sony dvd player as the player of choice for System B ?

I think I already answered that, but let me paraphrase that again here.

Why is an integrated amplifier better than AVR. AVR has lots of things to do, lots of other circuitry, money spent on that etc etc etc. Audiophiles frown upon the AVRs as inherently inferior right?

Same yardstick should be applicable with a source right?

Can I not appoint an oppo bdp se95 for the blind test system. Can I not appoint a marantz cd6004 for the audiophile system, a 'pure' cd player.

 

[doors666]

I was just thinking about it, what are the things that people can hear a difference in. I havent tried a lot of this stuff myself, so decide for yourself what all can you hear a difference with or not.

Speakers, headphones - yes I can. I am sure everyone can.

Amplifier - I think I can, but have a very limited experience with it. can hear a clear difference when switching to pure direct mode from 2ch stereo mode on my avr.

Source and dac- can differentiate in mobo soundcard and a good sc. could also make out differences between my m-audio and asus essence sc. never did ab though. could hear changes when i switched between the cd player and avr dac. I should try my cd player and asus and see if its different.

Masterings - I can clearly hear the difference between a poorly recorded vs well recorded cd. Never did blind testing.

Mp3 vs cd - some 8-10 years ago did a test with 256 kbps mp3s and cds. I was switching the material and he was blind. I could hear the difference most of the times (of course, I was the one switching cds), my friend who was blind got it right 100%, not a single mistake with multiple songs and cds. he was a lot more familiar with the material than I am. A musician friend of mine claims he can make out the difference on his altec lansing atp3.

Cables - Very limited experience, I have a few of mx cables. I also got belden 1505 and 1694 cables, these are the ones blue jeans uses, instead of canare connectors, I used neutrik connectors. cant hear a difference. Didnt try A'Bing them or blind test. In casual listening, I cant hear a difference, maybe I should look for differences, but then whats the point?

Capacitors, resistors, inductors - anywhere, amp, preamp, crossover - havent tried it. are boutique items better if all other values are same?

24-96/192 - havent tried with the same set of songs. Its not the case that if i hear a song i can tell that - hey this sounds great, much better than the usual stuff i listen to. Not like a dvd vs bluray where one look and i know its a bluray playing or dvd.

 

[Thad E Ginathom]

OK, lets try to get a bit more on the same page...

Double blind testing is a tool used to eliminate components which do not actually perform better than some standard right ?

I haven't mentioned double-blind testing. I'm not a scientist, but so far as I know, double blind testing is where neither the tested nor the tester know, this, trials of a drug against a placebo, for instance, cannot be influenced even subconciously by the testers.

So what is this standard ? Who sets it ?

No, there is no standard, only results.

The only standard is in how the tests are conducted. When it comes to the drugs you take, you expect this to be rigorous. When it comes to a pastime, even friendly, informal testing (of the ask-the-wife-not-to-tell-me) type is fine by me, although I'd expect better of, say, a major publication.

It's a tool we can use. That's all. It is something we can embrace, and even have fun with. It is a also a tool we can use, where there is controversy, to find objective answers. For me, things like mains and digital cables come in this category, whereas things like speakers, do not.

Friends can have fun with it. As consumers, I think we should demand that the industry embrace it. The industry, especially the cables and peripherals section, does not dare.

 

[Hiten]

Brothers,
If anyone honestly want to learn, here is what one can do. Get single audio system known to bring out subtle differences in proper place. Make audition room such that there are no excuses. Get CD MAT, Ordinary and High End power cord and Cable Lifters. Blindfolded or by any other method listen without knowing to see if CD MAT is present or not. same for other things mentioned. Take 5 or 7 revisions as all agrees upon. just audition and put their opinion on nameless paper. No one has to tell what is better. No one has to prove anything. No one would be right or wrong.
Chat and have a drink, if everyone agrees .
And put results in the forum. Fair enough ??
Regards

 

[captrajesh]

We can argue till the land's end without any tangible result.

Seek volunteers, seek equipment. Get on ground and get real. Do some real testing. That is the way to go.

 

[square_wave]

OK, lets try to get a bit more on the same page...



I haven't mentioned double-blind testing. I'm not a scientist, but so far as I know, double blind testing is where neither the tested nor the tester know, this, trials of a drug against a placebo, for instance, cannot be influenced even subconciously by the testers.

No, there is no standard, only results.

The only standard is in how the tests are conducted. When it comes to the drugs you take, you expect this to be rigorous. When it comes to a pastime, even friendly, informal testing (of the ask-the-wife-not-to-tell-me) type is fine by me, although I'd expect better of, say, a major publication.

It's a tool we can use. That's all. It is something we can embrace, and even have fun with. It is a also a tool we can use, where there is controversy, to find objective answers. For me, things like mains and digital cables come in this category, whereas things like speakers, do not.

Friends can have fun with it. As consumers, I think we should demand that the industry embrace it. The industry, especially the cables and peripherals section, does not dare.

In the matrix test case, what is the inference ?

The sony dvd player is as good as the wadia ! Right ?

And the behringer amp is as good as the YBA / Classe combination ! Right ?

If there is no inference, what is the point of such a test for an audiophile ?

If the inference is that they are all the same quality, are you saying that in the hands of a capable audiophile, these two categories of gear cannot be made to sound vastly different?

 

[square_wave]

I think I already answered that, but let me paraphrase that again here.

Why is an integrated amplifier better than AVR. AVR has lots of things to do, lots of other circuitry, money spent on that etc etc etc. Audiophiles frown upon the AVRs as inherently inferior right?

Same yardstick should be applicable with a source right?

Can I not appoint an oppo bdp se95 for the blind test system. Can I not appoint a marantz cd6004 for the audiophile system, a 'pure' cd player.

Bro...i don't think you are understanding the test

To clarify, you cannot use an oppo because the blind panel will not let you since they cannot hear the difference between a sony dvd player and the oppo. Please remember that the blind testing panel cannot hear the difference between the sony dvd player and a state of the art wadia. So you have already hit the ceiling with the sony

 

[asliarun]

Perhaps one reason why there are such differing opinions on this subject is because of the McGurk Effect: Try The McGurk Effect! - Horizon: Is Seeing Believing? - BBC Two - YouTube

As an audio lay person, I find it fascinating that our brain can interpret the same audio signal in completely different ways depending on different visual cues, even if we know or convince ourselves that the visual cues are merely visual cues.

 

[doors666]

Bro...i don't think you are understanding the test

To clarify, you cannot use an oppo because the blind panel will not let you since they cannot hear the difference between a sony dvd player and the oppo. Please remember that the blind testing panel cannot hear the difference between the sony dvd player and a state of the art wadia. So you have already hit the ceiling with the sony

Let the testers be a judge of that. if they cant hear a difference in sony dvd player and oppo, they'll take the sony. But I seriously doubt that. a sony dvd player will have sound hardware like a mobo sc hardware probably. Even i can distinguish between mobo sc and asus. oppo has sabre 9018 dedicated dac for 2 channel audio, which is currently arguably the best dac in the world with least distortion. So yeah, maybe the wadia was such a piece of shit that it lost to a sony (high price tag and big brand name doesnt guarantee great performance). Maybe sony will beat the oppo too, we'll see how it turns out.

But the blind testers will be the judge of that, and to answer your question, no you cant nominate gear for the other camp. If the results from previous tests are to be considered, audiophiles have already lost as in those tests, people couldnt make out the difference.:lol:

And I agree with captain, talking wont get us anywhere, so lets get testing.

 

[square_wave]

I have seen this before Asilrun. This is a very interesting effect.

BTW: I have nothing against a blind test. But I have a problem with how it is usually done. Something like the one below is better

1.Identify one person who has years of listening experience and actually owns a very resolving / revealing system which he has painfully setup. This guy needs many years of experience comparing many types of components. Do the test with his existing system.

2.Select a piece of music and give him couple of days to get familiar with this piece and how it sounds with his components.

3.Identify one different component. Give this new component to him for two days to familiarize along with the selected piece of music.

4.On the day of the test, play the piece with his current component first. Then keep playing the same music piece many times each time switching or not switching the component without him knowing which one is playing.

5.Ask him to identify HIS component each time you play.

 

[square_wave]

Let the testers be a judge of that. if they cant hear a difference in sony dvd player and oppo, they'll take the sony. But I seriously doubt that. a sony dvd player will have sound hardware like a mobo sc hardware probably. Even i can distinguish between mobo sc and asus. oppo has sabre 9018 dedicated dac for 2 channel audio, which is currently arguably the best dac in the world with least distortion. So yeah, maybe the wadia was such a piece of shit that it lost to a sony (high price tag and big brand name doesnt guarantee great performance). Maybe sony will beat the oppo too, we'll see how it turns out.

But the blind testers will be the judge of that, and to answer your question, no you cant nominate gear for the other camp. If the results from previous tests are to be considered, audiophiles have already lost as in those tests, people couldnt make out the difference.:lol:

And I agree with captain, talking wont get us anywhere, so lets get testing.

Not a problem...let the panel decide. No arguments

 

[doors666]

I have seen this before Asilrun. This is a very interesting effect.

BTW: I have nothing against a blind test. But I have a problem with how it is usually done. Something like the one below is better

1.Identify one person who has years of listening experience and actually owns a very resolving / revealing system which he has painfully setup. This guy needs many years of experience comparing many types of components. Do the test with his existing system.

2.Select a piece of music and give him couple of days to get familiar with this piece and how it sounds with his components.

3.Identify one different component. Give this new component to him for two days to familiarize along with the selected piece of music.

4.On the day of the test, play the piece with his current component first. Then keep playing the same music piece many times each time switching or not switching the component without him knowing which one is playing.

5.Ask him to identify HIS component each time you play.

Now my question to you is, why does it have to be so painful? Why cant you show the differences in a normal system on the spot. Use your own cds, change a component on your system and hear. Why a 25k amp, with say 50k speakers are not enough to show the differences. Whats a revealing system, whos' going to decide if its revealing enough or not. I have heard this excuse before, 'the system is not revealing enough to resolve the differences between these cables'.

There was another thread where Asit said that the differences with even cables can be heard on any system, he could identify different cables while being in a different room in a normal system. You can hear the difference between 256 kbps mp3 and an flac on a 2000 rs headphones, speakers/source changes with a topping. So why do you need such a convoluted setup and lengthy procedure. Do you think with 685, you will not be able to detect and amp change from say a yamaha amp to a denon AVR?

Pick a normal system, say a B&W 6 series speakers, or a usher x718, a yamaha, marantz amp, a marantz/denon cd player. Change the cables/amp and show me the difference.

If you need a thiel or maggie or 8 series speakers with accuphase or classe or some other 2 lakh amp and then only you can show the differences, I am not interested in changing my cables. I couldnt give 2 hoots about cables even if they made a world of difference because at that kind of quality level, it doesnt matter if you slightly improve or not, you will be satisfied. And I dont think I will spend that kind of cash.:lol:

If you cannot differentiate between different amps without the help of a very revealing system, without the help of a guy with years of experience in listening and identifying components, I think the audiophile camp has already lost. We can rephrase the statements to something like this - all the amps almost sound the same, most of the times you cant make out a difference between a 20k amp and a 200k amp, so no point wasting a lot of money, just get the cheaper one.:lol:

 

[Thad E Ginathom]

In the matrix test case, what is the inference ?

Inference? There were results.

If the inference is that they are all the same quality, are you saying that in the hands of a capable audiophile, these two categories of gear cannot be made to sound vastly different?

so... what? You think that these testers were tone-deaf boneheads? I don't know them: you could be right! hyeah:

My impression is that they are as serious, and probably as experienced, as many of the HFVers that I know, on- and off-line.

 

[manoj.p]

Inference? There were results.


so... what? You think that these testers were tone-deaf boneheads? I don't know them: you could be right! hyeah:

My impression is that they are as serious, and probably as experienced, as many of the HFVers that I know, on- and off-line.

+1.

You summed it very well.

We all are missing one thing. In the matrix hifi test, they kept two components which contribute the most to the listening experience same - The speakers and room treatments. The electronics, in this case just the DAC and amp will make some difference. But it won't be as big contributor as the speakers and room treatments.

As for Wadia, if it fails to impress bunch of people, then so be it. After all "Trust only your ears", says the audiophiles. Or is the audio dependent on the brand name too?