A practical blind test : Just for kicks

[square_wave]

System A
Assemble a system only based on blind testing. This means that you cannot choose anything that does not pass the blind testing each time. So this system will be assembled based by a blind testing panel. For example: for the source equipment, you will start from the ipod and then move to a 1500 dvd player etc...till you reach the ceiling beyond which the panel cannot hear a difference. You do this for all the components including the wires. So this will be a random mix of gear. One can of course make sure the electrical specs are compatible for the chosen gear for the test.

System B
Now ask an experienced audiophile to choose whatever he wants including all these so called snake oil devices of his choice.


The room and speaker remains the same. In this test, cost should not be a factor since anything which does not pass the blind testing is snake oil and cannot contribute to improvement right?


Now listen to both

This will be the mother of all tests. Who do you guys think will win ?

 

[anm]

depends on whether results are declared with closed eyes or open eyes.
I think the blind system will win, because the second setup is imaginary. A real audiophile will wait for the next version of each equipment

 

[santhosh]

This will be the ultimate Blind testhyeah:. To make it more interesting, the components should not be auditioned in any particular order and the panel should listen to everything, they should only rate the components in the order that they listened. For Eg, we can start off with a DVD player #2, then Ipod, then CDP#2, Music PC#1 and so on.

 

[captrajesh]

Synergy between components will yield far better results than choosing the best of the components and forming a system out of them.

For example, in a team game like soccer, you cannot guarantee the championship just by making a team of all star players.

 

[square_wave]

Please keep in mind that the System B will consist of state of the art (according to current industry trends) and put together by an experienced audiophile who understands synergy.

The system A will also have synergy but it will be proven electrical synergy according to engineering concepts.

 

[manoj.p]

If the audiophile who chooses the components in second setup is himself the blind tester, then its a no brainer.
Everytime a new component is chosen by Audiophile is used, he WILL hear the difference. The real test would be - let the audiophile select a component, but don't put in the system or put another components without Audiophile's knowing. Then lets find out if he can hear the difference. On quality/genuine components all will hear the difference. On snake oil based components, it will be interesting to know what the the Audiophile hears.

 

[square_wave]

I think I wasnt clear enough about the methodology.

System A
The blind testing team is an independent panel of people and the blind testing is done according to blind testing standards. Each component is blind tested against industry leading studio equipment or even state of the art home audio gear. Digital source, amplification and cables are the ones being tested for inclusion into the system. For any of the gear, once the blind testing team reaches the stage wherein they cannot discern any more differences against the reference grade component, the testing stops for that particular component and the one being tested then is chosen. So we do this for all the three and voila .......!! you have the best system that can be had according to the blind testing camp ! Anything beyond this is snake oil.

The system B
The experienced audiophile is free to choose anything under the sun including any accessories which does not conform to accepted engineering practices since anything beyond what is chosen by the blind testing panel is apparently snake oil.

The room and speaker remains the same.

Now listen to both systems ( blind or not) and which one will sound better ?

 

[Thad E Ginathom]

This has, pretty much, been done, and discussed here too!

Have a look at the work these guys do: Matrix HiFi

 

[Moviela]

A camel is a horse designed by the protocol of system A

 

[square_wave]

This has, pretty much, been done, and discussed here too!

Have a look at the work these guys do: Matrix HiFi

Hey Thad,

The website is in a language which I do not understand

Has this been done? Can you elaborate?

The thought about "reverse engineering" the blind testing method is something which intrigues me. If blind testing is valid, then reverse engineering the method should produce a system which cannot be beat.

 

[square_wave]

A camel is a horse designed by the protocol of system A

I dont think so Moviela.

Please note that all electrical synergies are taken care of in this method. For example, power requirements, pre amp + power amp (impedance issues), source output voltage or others if I have missed out any.

The outcome of this method is the best system derived from the blind testing method. Anything beyond this is snake oil.

 

[square_wave]

The blind testing camp do not believe in gear synergy related to voicing, power conditioning, cables matching, high cost esoteric audio gear, component isolation techniques and all those fairy tales. All these will be implemented to the ultimate level in System B.

The members can take upto an year to setup both the systems. This is the time an audiophile typically takes to arrive at correct synergy so we have to accomodate this time.

 

[Thad E Ginathom]

The blind testing camp do not believe...

It's not a religion! It is a basic and essential scientific method. Doesn't everyone appreciate that the whole can be greater than the sum of the parts?

On the contrary, the opposite point of view appeals to faith, and we may as well be arguing with the pope (hey, sorry to pick on one religion, it's just an example!).

Nobody has to blind test anything. The only reason for it is comparison. Are you going to use file format "A" or file format "B?" Are you going to invest huge amounts of money in speaker cable, use some mains lead --- or twist some coat hangers* together ? Is to make more informed decision, and to protect the consumer from the possible baseless claims of manufacturers. Anyone is free to make any of those choices, but, unless they can tell the difference without knowing which is the coat hangers and which is the, err... monstrous... cable, then they should not make dogmatic claims. Leave out the dogma, and one can leave out the claims ...and the testing.

The very basis of your A v. B test seems to be to discredit the idea of blind testing. Why? To put it bluntly, what's so scary about it? It might prove you wrong about something; it might prove me wrong about something; we should mind that?

The industry should be scared, especially certain cable manufacturers and purveyors of gadgets --- by which I do not mean everyone who makes expensive cables, but I do mean those that use heavy and misleading marketing, or, for instance, publish factually inaccurate supposed guides (that's two companies I won't ever buy a cable from!). Us consumers have nothing to fear.

Hey Thad,

The website is in a language which I do not understand

That's odd, I see only English! Try this link. I note "ENG" in the URL, so perhaps the site's server thinks you are visiting from a non-English-speaking country.

By the way... Very honest and straighforward report from a high-end manufacturer who does not "believe" in expensive cables, and could not, in any electrical or listening test prove that a set were better, but found a certain something that made them worth keeping. Somewhere here: can't dig for the exact page just now. Given the cost, though, he could think of better ways to improve a system by spending that much.


*Source: Audioholics, although it was picked up by a number of other sites and is easy to find

.

 

[doors666]

The blind testing camp do not believe in gear synergy related to voicing, power conditioning, cables matching, high cost esoteric audio gear, component isolation techniques and all those fairy tales. All these will be implemented to the ultimate level in System B.

The members can take upto an year to setup both the systems. This is the time an audiophile typically takes to arrive at correct synergy so we have to accomodate this time.

Gear synergy is real, its based on pure science, not hocus pocus like cables. you dont match impedences, voltage levels etc, you are toast.

Power conditioning. Yeah, that a load of crap, unless of course you are talking about feeding with 180v or a sawtooth waveform instead of sine wave. What good is that monster torroidal transformer and that power supply rectifier circuit if it cant even handle a little bit of fluctuation in the supply line. Dont most of the boards run on DC. Dont you convert your 220v ac to dc using rectifiers and other circuits. Does your ps circuit or rectifier gives two hoots as to what is the input voltage as long as its in the range. Does your preamp circuit cares what input voltage was converted to the 18v dc its running on.

Cables, yeah, give me least lcr, well insulated, good build quality cables and I am very happy.

 

[Thad E Ginathom]

Gear synergy is real, its based on pure science,

Exactly. And there is no reason why blind testing should not be applied to whole systems.

Further to my last post, here is Frank van Alstine on the results of his "cable challenge," but please read the whole article.

Anyway, to make a long story very short, I am now using Kimber Kables in my own reference system. Why? Because I believe the system sounds slightly more musical with them than without them. The differences I think I hear are subtle. not at all day and night, and are not anything like the purple prose cable test write-ups in the underground magazines. I have real reservations about the cost-benefit ratio of the improvements (the retail price of all the cables I am using would nearly pay for a complete Fet-Valve 500 amplifier and that will make a really big improvement in your system - or if you have enough audio equipment already it would pay for a Sony 32HSR10 big screen TV (make your value judgements carefully).

...
...

Then Why Am I Using the Kimber Kable?

Because when all the formal tests were done I went ahead and installed the Kimber Kables from input to output of both channels of my reference system and went back to simply relax and listen to the music and that is when I started to "hear something" or more appropriately, "not hear something." Everything sounded the same, but everything sounded a tiny bit closer to live real music. Not all on my staff agree. Some think I am simply hearing the "sugar pill effect" of all those expensive cables being there. Others don't disagree with my very subjective evaluation. They remember that we designed some widely accepted audio equipment (the Dyna 416 and Double 400 for example) "by ear" long before we had full time electrical engineers on the staff and that the test bench and formal A-B testing doesn't give us all the answers (or always the right answer).

Maybe I am "hearing things" and certainly I would not want to pay as much for audio cables as these cost, but the Kimber Kables are staying in my system for now. They met my challenge fairly and surprised me by making my system seem to work that little bit better and in the quest for perfect music reproduction, each "little bit" counts.

So, we should not be split into "camps." We should not deny the possibility of a .0001% difference (and it's our money, if we feel we are justified in spending it) but we should certainly not decry scientific or engineering measurement, and published testing (especially commercial, eg magazine, reviews) should begin with blind testing, not run away from it like the plague. Trouble is, to sectors of the industry it is a plague, and they will fight it tooth and nail. Why-oh-why should we, the buyers, support them in this?

Let me cherry-pick a couple of highlights from the Matrix Hifi test in my link (btw: I found this site via HFV only!):

system A: Estimated value when new: 700 U$D
system B: Estimated value when new: approx. 12,000U$D (or more)

... ... ...

The human testers were all trained ears and used to extensively listening to high end equipments, a good number of them participated, each with his own conception of the high end world, some totally subjectivists, some completely objectivists, some in between.

(This is the beauty of this event, we are all toghether sharing our experiencies and visions around this hobby)

... ... ...

38 persons participated on this test
14 chose the "A" system as the best sounding one
10 chose the "B" system as the best sounding one
14 were not able to hear differences or didn't choose any as the best.

But it is important to read the whole page, and to check out the site, to know the aims, philosophy and methodology of this particular group of music/hifi lovers

.

 

[square_wave]

At everyone,

This thread is not to discredit anyone or any camp. I do not belong to any camp. It is meant to initiate discussion on:

Which system will win!
OR
Thoughts about the methodology !

Try to contain discussion within this domain.

For system A

The cable testing will start from coat hanger. It will not go any further since it has been proven that coat hanger is all that is needed.

Source testing will probably stop at a dvd player.

The amp testing will probably stop at a random 800 $ muscle amp which has enough muscle to power the speaker.

The setup will be on a rickety table. It will not go any further since it has been proven that a rickety table is all that is needed.

The speaker will be a 50,000 $ model. Take your pick.

 

[captrajesh]

The cable testing will start from coat hanger. It will not go any further since it has been proven that coat hanger is all that is needed.

Source testing will probably stop at a dvd player.

Coming from you, I find it perplexing Square_wave.

I'm sticking my neck out here.

This is a no brainer.

 

[doors666]

At everyone,

This thread is not to discredit anyone or any camp. I do not belong to any camp. It is meant to initiate discussion on:

Which system will win!
OR
Thoughts about the methodology !

Try to contain discussion within this domain.

For system A

The cable testing will start from coat hanger. It will not go any further since it has been proven that coat hanger is all that is needed.

Source testing will probably stop at a dvd player.

The amp testing will probably stop at a random 800 $ muscle amp which has enough muscle to power the speaker.

The setup will be on a rickety table. It will not go any further since it has been proven that a rickety table is all that is needed.

The speaker will be a 50,000 $ model. Take your pick.

Your first half of the mail where you claim that you have an open mind and dont want to discredit any of the camps is directly and badly contradicted by the second half of your mail.

Since coat hanger is pretty much free, and cables cost a fortune, on the value for money scale, coat hanger wins and cables loose badly.

Why same speakers man, let sysB have 50K$ speakers and let sysA have the cheapest ones possible. maybe then you can easily prove your point.

 

[square_wave]

@ capn,
These are results from various blind testing experiments.

With the coat hanger, I said it will since it has been proven in a Blind test that one cannot any more differences.

With the dvd player and amp I said probably. So I am not sure. It may go further although I have read somewhere they proved that some ~ 200 $ pioneer receiver and an esoteric amplifier could not be differentiated in a double blind test.

@doors,
Did you read the methodology? We have to keep the speakers and room the same or the test is not valid.

We are trying to figure out if any other methods other than the blind testing one can yield a better or worse sounding system.

Yes, sir, I am keeping an open mind.

 

[doors666]

It seems like you are hell bent on making a killer system for sysA and a very lousy system for sysB and hence claiming that blind testing is crap since it produced a bad system (using coat hangers and dvd players and AVRs?). I seriously doubt your 'unbiased' intentions. Looks to me like you have already made up your mind. Me, I have already made up my mind, feel free to change it by giving a demo that it makes a difference.

A source contains a DAC as well as a transport. How about using oppo bdp se95 for the blind test system. its just a stupid bluray player with a decent dac right. Anyway, how can a video player with tons of circuit for a/v processing beat a cd player (same logic used by audiophiles for saying amp is better than AVR). how about using a marantz cd6004, a 'pure' cd player for the audiophile's system. Fair comparison? Anyway, as per your criteria, price doesnt matter right?