Why We Measure Audio Equipment Performance

[keith_correa]

One can produce all the graphs and plots one likes but do we listen to graphs and plots? No!

Why should anyone use graphs and plots to validate what one likes? Why would "validation" even be necessary? So subjectively if one likes something and objectively that produces horrible graphs and plots, what is one supposed to do? Change the like to dislike? I find that absurd!

 

[0verkilled]

Graphs are really important, at least for the people who cant decide. Its like shopping in a mall with countless items on display. Once you start looking into too many things, you loose track of what you like or dont like. And then the salesman comes and suggests the black jeans is better than others.

Graphs are the results of measurements, but who governs what a measurement is. This can go as far back as - 'Is mathematics discovered or invented?' There are proofs for both. If the graph says your equipment is good, congrats, feel happy. If it does not, shun the results and enjoy the music!

 

[Hari Iyer]

One can produce all the graphs and plots one likes but do we listen to graphs and plots? No!

Why should anyone use graphs and plots to validate what one likes? Why would "validation" even be necessary? So subjectively if one likes something and objectively that produces horrible graphs and plots, what is one supposed to do? Change the like to dislike? I find that absurd!

@ Keith
Saying it again - objective measurements are not for those who do not anyway understand them. If you dont like subjectively about what you listen, then how are you going to apply your tweaks? Where are you going to start from. It will be like firing a missile without any coordinates. If you hit the target then god saved the world, else - everything is fair in love and war.

You may start appreciating importance of objective measurements only when you realize its importance - else its just a piece of graph paper for the novice.

You can check some of my recent DIY speaker where i have posted some images for your review if you are keen to know more http://www.hifivision.com/speakers/62581-time-advance-tl-floor-stander-dayton-audio-drivers-new.html. If you are really interested to discover more then you are welcome for a listen by this weekend (you stay at Mumbai).

 

[keith_correa]

Graphs are really important, at least for the people who cant decide. Its like shopping in a mall with countless items on display. Once you start looking into too many things, you loose track of what you like or dont like. And then the salesman comes and suggests the black jeans is better than others.

Agreed!

Graphs are the results of measurements, but who governs what a measurement is. This can go as far back as - 'Is mathematics discovered or invented?' There are proofs for both. If the graph says your equipment is good, congrats, feel happy. If it does not, shun the results and enjoy the music!

Well said!

 

[keith_correa]

@ Keith
Saying it again - objective measurements are not for those who do not anyway understand them.

That goes without saying. But my point was different. Let me try again:
One heard.
One liked what they heard.
One sees bad measurement results of the equipment that one heard and liked.
Does one now dislike?
All this comes stems from what you said:

Ibelive that subjective listening should get validated by objective measurements too.

If you dont like subjectively about what you listen, then how are you going to apply your tweaks? Where are you going to start from.

NoNoNo. Tweaks? This is taking it into a different direction. Let's not even go there.

It will be like firing a missile without any coordinates. If you hit the target then god saved the world, else - everything is fair in love and war.

Don't understand this but never mind.

You may start appreciating importance of objective measurements only when you realize its importance - else its just a piece of graph paper for the novice.

Novices will not even look at measurements not should they do so - what's the point. Many "non novices" do the same.

You can check some of my recent DIY speaker where i have posted some images for your review if you are keen to know more http://www.hifivision.com/speakers/6...ivers-new.html. If you are really interested to discover more then you are welcome for a listen by this weekend (you stay at Mumbai).

Hari, this is not isolated to speakers. "Audio Equipment" is more than just speakers even though IMO speakers are the weakest and most important link in the chain. Thanks for the invite. I will surely drop in when I'm in the vicinity.

 

[Hari Iyer]

Stereophile magazine had published a series of articles to discuss this very topic almost 18 years ago- "Measuring Loudspeakers" which gives important insights about measurements. Some of the key factors infact -

1. All Measurements lie
2. None of the subjective parameters can ever be objectively measured ie. Objective measurements and subjective listening should not be confused with or cant be interrelated with each other.
3. The human ear is still the best judge of the sound quality - which again is very subjective and very very relative.

Given the 3 above statements what can we say about both objective and subjective measurements?

Check out the entire series here Measuring Loudspeakers, Part One | Stereophile.com

 

[shaan]

One can produce all the graphs and plots one likes but do we listen to graphs and plots? No!

We definitely do not listen to graphs and plots. But a collection of graphs and plots taken from a particular system will have some measurable and perceptible effect on the SQ; we listen to that effect.

Why should anyone use graphs and plots to validate what one likes? Why would "validation" even be necessary? So subjectively if one likes something and objectively that produces horrible graphs and plots, what is one supposed to do? Change the like to dislike? I find that absurd!

Half of the world's population would find that absurd too. Although the other half might demand that when their systems are playing music, nothing is added or removed to or from the material and take the plots as a proof. Belief plays a great role here. While you may believe your ears to be the correct judge of SQ (which is totally valid), they know that our hearing sense is the easiest to trick and they place their belief in plots while thinking things like "this 0.000001% THD amplifier must give best SQ from this speaker which is totally flat from 1Hz to 100KHz" (which is also totally valid). And naturally they would find that system sounding perfect even if you find it sounding horrible, and they will reject the idea of changing their like to dislike for their system even if they are presented a system that might sound good to the ears but has poor graphs and plots.

IMO, both half is right.

 

[keith_correa]

We definitely do not listen to graphs and plots. But a collection of graphs and plots taken from a particular system will have some measurable and perceptible effect on the SQ; we listen to that effect.

Right. No dispute there.

Half of the world's population would find that absurd too. Although the other half might demand that when their systems are playing music, nothing is added or removed to or from the material and take the plots as a proof. Belief plays a great role here. While you may believe your ears to be the correct judge of SQ (which is totally valid), they know that our hearing sense is the easiest to trick and they place their belief in plots while thinking things like "this 0.000001% THD amplifier must give best SQ from this speaker which is totally flat from 1Hz to 100KHz" (which is also totally valid). And naturally they would find that system sounding perfect even if you find it sounding horrible, and they will reject the idea of changing their like to dislike for their system even if they are presented a system that might sound good to the ears but has poor graphs and plots.

IMO, both half is right.

I understand that but that was not my point. OK once again - this time differently. Let's talk only once piece of the chain - speakers.

We have 2 sets of different speakers - A and B. The person listening knows how to read measurements and to interpret them.

Speaker A.

Heard
Liked
Measures bad​

Speaker B.

Heard
Disliked
Measures good​

Would the majority buy A or B? I say A.

 

[shaan]

We have 2 sets of different speakers - A and B. The person listening knows how to read measurements and to interpret them.

Speaker A.

Heard
Liked
Measures bad​

Speaker B.

Heard
Disliked
Measures good​

Would the majority buy A or B? I say A.

I think the whole crowd should avoid both speakers as both are tailored to excel at particular aspect and all other aspects ignored. Both disagree to the following which you also agreed to:

We definitely do not listen to graphs and plots. But a collection of graphs and plots taken from a particular system will have some measurable and perceptible effect on the SQ; we listen to that effect.

But to answer your question, I think all (not just majority) who have the same realization as above, will go for A.

Btw, don't you think the situation is narrowed down and doesn't cover other possibilities as well?

 

[Sumanta]

I am thoroughly enjoying the discussion.
Thanks to all.

 

[Sumanta]

But to answer your question, I think all (not just majority) who have the same realization as above, will go for A.

Won't Placebo effect affect (at least tries to) the decision here?

 

[Hari Iyer]

Right. No dispute there.

I understand that but that was not my point. OK once again - this time differently. Let's talk only once piece of the chain - speakers.

We have 2 sets of different speakers - A and B. The person listening knows how to read measurements and to interpret them.

Speaker A.

Heard
Liked
Measures bad​

Speaker B.

Heard
Disliked
Measures good​

Would the majority buy A or B? I say A.

@ Keith.

You seem to have a huge experience in listening speakers and evaluating them. Please let all of us know from your vast experience a very badly measured speaker sounding good and also a very good sounding speaker measured badly for further discussions. Else we are all we all playing with our own egos here.

 

[shaan]

Won't Placebo effect affect (at least tries to) the decision here?

That's a good question. And the facts are dangerous. Can start a war if not handled with extreme care.

 

[keith_correa]

@ Keith.

You seem to have a huge experience in listening speakers and evaluating them. Please let all of us know from your vast experience a very badly measured speaker sounding good and also a very good sounding speaker measured badly for further discussions..

"Huge" experience? How did you presume that?
"Vast" experience? How did you presume that?

I have neither "huge" nor "vast" experience. If you are trying to be sarcastic, please save it for someone who cares.

Else we are all we all playing with our own egos here.

You seem to be using sentences just for effect without understanding what it means. There are no egos here - certainly not mine. Yours? Only you can answer that. We were having a good thread until you started turning this into something else by nasty posts like this.

I'm out of here!

 

[regeHA]

I think this discussion is really a moot point.

The subjective folks will actually take the side of measurements when we bring in speakers like Bose.

The objective folks will start using "subjective phrases" like warmth, forward, colored, etc. when we bring in speakers like Genelec.

Seriously if measurement matters then something like Genelec is all you need. Yet many are not sold on it as its too "uncolored". I myself let go of my Genelecs for 2 primary reasons, the connection options were too pro-studio and the sound was too flat. The detail was excellent but exitement was missing i.e. what the English folks call PRAT.

And if subjective is a good enough yardstick then we should not ridicule the likes of Bose. Yet my "audiophile" friends actually would lose no opportunity to ridicule the Bose brand though to (mine and others) ears it actually sounded as good as most mid-fi separates. It happens here too... I read a thread a couple of days ago where someone was being guided to a Yahama HTIB instead of Bose.

I have owned both Bose speakers and Genelec speakers as well as even home theater speakers from Bose (Lifestyle series). With hindsight I can say neither delivered 100% or even to my satisfaction. Neither did the KEF speakers that I own currently, at least not initially. The thing that made them sing had nothing to do with measurements or even subjective listening. It was actually changing the source to a $35 Raspberry Pi. I have no doubt that even the performance of the Genelec as well as Bose speakers would have improved vastly with my present source/transport, the mighty Pi.

What that tells me is neither subjective listening nor objective measurements really have the final say in how well or how bad speakers perform. The amp, the source/transport, the sound material, even cables, etc. all have a say in ultimately how a speaker sounds. In that sense neither objective or subjective really matters, the entire chain and synergy is what really matters.

 

[raghupb]

Measurements are only required for initial guidance and possibly post purchase nit-picking.
Over the past few months on the forum, I have made some friends and either invited them home or gone to their home.
Each time one or more elements in the chain (at my place or fellow FMs place) was replaced and the chain sounded slightly different or vastly different.

The last experiment was with Hegel DAC, Rega IA used as pre, my power amps and Harbeths.
Frankly, it sounded horrible. Adele's "Hello" sounded like she was desperately in need of a throat lozenge.
Now I am sure that each of the element in the chain has been measured by the company and independent 3rd parties and nothing alarming was found.
I know for a fact that my power amp's graph is pretty flat across the frequency range (3rd party validation).
But for some bizarre reason, it just did not fit into the above chain + room. The same amps sound great in my chain + room.
Other elements in my chain have respectable measurements too (I have checked 3rd party sites that have published them).

So content/system/room synergy is paramount. If synergy is lacking, then you get a situation similar to what I posted here earlier:
http://www.hifivision.com/av-lounge...audio-equipment-performance-4.html#post685398

All things being fairly normalized, including cost (can't really say equal here), in my personal opinion:
It is easier to live with a good sounding system with not so convincing measurements than the other way around.

A good sounding system (to one's ears) with good measurements is a great system at its price point.
If one can put this together, the system will stay with you a long time.
I think I am almost there or my expectation is very low
None of the elements in my system have been bought solely on objective or subjective basis.
Here is a link to my stuff (earlier in this very thread; some of the buy decisions are actually not very logical; but then it works for me)
http://www.hifivision.com/av-lounge...audio-equipment-performance-3.html#post684599

Cheers,
Raghu

 

[asliarun]

I am surprised to see no one has brought out the car analogy yet. (I think there should be something similar to Godwin's law - that when the length of a discussion thread tends to infinity, a car analogy will invariably popup). So I will oblige.

A big section of people will buy a car based on no analysis (of objective measurements). Or 2, namely - mileage, and size of the car. They will eventually make their decision based on a test drive, and possibly some feedback from friends (with friends over drinks - you: "I'm planning to buy a Skoda". your friends: "Oh, cool, that is an awesome looking car" - net result, you feel reaffirmed about your decision)

A smaller but more discerning section of people will pore over the technical specs of the car. Engine CC, horsepower, number of valves per cylinder, etc. They will also discuss their intended purchase in car forums and get feedback from other enthusiasts about the car, the dealer, the car's reliability, long term maintenance implications etc.

Still, no one in their sane mind would buy a car blind - without taking a test drive or two. Or without trying out multiple dealers. And ultimately, their decision will be equally governed with the quality of the test drive as much as the technical specs.

An even smaller sub-section of people, the proverbial gear-heads would custom build their enthusiast car. They would take a stock car, maybe a cheap used one, and would reconfigure and significantly upgrade the car. For more power, they would focus on not just engine horsepower (say, add a turbo) but on improving the overall power bandwidth of the car, and ensure that the system as a whole is upgraded with similar levels of performance parts. Upgrade the air intake and air filter, upgrade the engine and harden it as well with better quality components that can withstand much higher performance without breaking down or causing a bottleneck, widen and optimize the exhaust.

Now I would imagine that for gear-heads, developing a deep understanding of the technical specification of the parts is essential and of paramount importance. If they are buying an expensive part to upgrade their car, they need to know exactly "how much better" the component is going to be (in a quantified metric, not a subjective "it will be heaps better"), and consequently how much closer it will take them towards their higher performance goal.

How can this kind of a performance upgrade be done without really digging deep into the technical specs of the various components available for an upgrade? How can this be done without having a very good overall understanding of the car and how it works and how the components work with each other?

In other words, how can a purely subjective approach be taken to make significantly complex upgrades to either a car or to a music system? Maybe, just maybe, through sheer force of will and countless trial and errors. But wouldn't it be easier to do it in a much more easier way by being objective about measurements and technical specifications?

Another sub-section of people just want a high performance car or high performance audio system to begin with. They don't want to get super geeky - they just want to listen to really good music or drive a really good car. Period. In the car world, it is easy. Spend money, and buy a supercar or a high performance car from a marquee brand. You get performance exceeding expectations, pride of ownership, great dealership experience and support - with a big sticker price, which is fair enough.

I again find it hard to understand why these options do not exist in the audio world. You would not dream of buying a Ferrari and then changing its engine with an aftermarket engine. Then, why would you buy a high end audio system with a big sticker price and then even contemplate changing a major component?

And if you are a gear head and an "optimizer" who wants similar high end performance on a beer budget instead of a cognac budget, why would you even attempt to do that without very very strong technical knowledge of specs and theory?

The two target audiences are very different and perhaps the whole problem is that in the audio world, they are not clearly differentiated or properly catered to. Again, just my two paise.

 

[philipreji]

The proverbial gear-heads are a very small subsection of car buyers and the best among them should have a deep understanding of the possible mods and their effects (quoting asliarun). This is exactly so in audio too. Irony is, by the time people reach that level of wisdom the underlying science will become very clear to them, and as the mystery has unveiled half of them will lose interest and spend the rest of their lives playing music through a two-in-one system hyeah:

As a famous Mallu poet (Kunjunni Maash) wrote: "Son, light is grief; darkness is comfort"

 

[Hari Iyer]

As a famous Mallu poet (Kunjunni Maash) wrote: "Son, light is grief; darkness is comfort"

Dont know who this poet is - My guess is he must be a utter failure in life - or must be facing terrific grief when penning this poem, though there may be some relative truth in what he says.

 

[shanmune]

When I read this thread, majority of the posts convey the same message, but that is conveyed in different ways. If I have understood correctly, let me state in simple terms :-
1. Whether a sound signature is liked or not is determined by subjective analysis.
2. Whether a sound signature is of true high fidelity or not is determined by objective analysis. Of course, the current objective methods used may not be completely exhaustive, but at least they can get you more closer to high fidelity than using the subjective method.

Now, what is the core issue of this debate ? Why We Measure Audio Equipment Performance ?
In general, I feel few of us (or majority ??) will tend to incline more towards the subjective way of analyzing things. The others will tend to incline more towards the objective way. It is an individual's choice to choose the way he/she wants. At the same time, we must have audio performance measurement. Why ? I have a different point of view here. That is - The general audiophile world should not fall prey to the marketing nonsense.

By having the objective data, we, at least, have an option to verify the marketing claims before we end up shelling our hard earned money. So for the general public, objective/performance data is required for not getting cheated by the hi-fi companies. As many have suggested, I prefer (my personal opinion/choice) to have the objective data as a baseline to start with and use subjectivity as a secondary one.