Why We Measure Audio Equipment Performance

[keith_correa]

Measurements: If one knows what to measure, one can determine why a component "sounds" the way it does. For example if we just measure the resistance of a resistor, we're not going to learn or determine anything by that number. But measure what it is doing to the signal and the answer is there. IMO, it is all attributable to harmonic distortion. Reji is right in what he said. We use terms like "airy", "precise", "punchy" but all of these can be explained technically. Like it or not, some distortion sounds good. Ask the "toob" guys.

 

[prem]

To me the Hifi industry is an industry of opinions. When someone's opinions is based on his or experience you can't argue with that. Some are happy that they have an amp or dac which measures fabulously while some base it on their listening experience. Either ways you can't argue with that.

Recently I got myself a new amp. The designer had two sets of the same tubes. One was Russian and the other American. The Russian tubes had a distortion of 0.6 percent whereas the American tubes had a standard 1% distortion. The designer couldn't hear a difference though. So he called over some of his friends. They too couldn't hear the difference. So here was a case where distortion was reduced by 40% and yet the designer could not hear any sound difference. I am sure this must be happening with a lot of equipment. If the designer can't hear a difference he might ignore a better speced part.

 

[Fantastic]

The ear isn't sensitive in a linear fashion. It is closer to the log value. 1% is -40dB and 0.6% is about -44 dB. The ear can't distinguish between that. Remember that -40dB is already almost inaudible with a signal at 0db ( the reference level).

 

[philipreji]

So here was a case where distortion was reduced by 40% and yet the designer could not hear any sound difference. I am sure this must be happening with a lot of equipment.

Very true. Guess this was the Total harmonic Distortion (THD) of the amplifier. In fact a limited amount of THD will add timbre and fullness to the sound (the tube guys know!), so reducing THD to minuscule levels may not really help, though it might satisfy the purists and the marketing division.

On the other hand, the designer should take extreme care to minimize intermodulation distortion to the absolute minimum which is very annoying and non-musical because the newly generated frequencies in IMD don't have any harmonic relationship whatsoever to the original frequencies.

cheers,
Reji

 

[square_wave]

I do not think anyone in their right mind would contradict the importance of measuring the right things in audio. If this was the aim for this thread, you would probably not see these opposing viewpoints..But then again, it would be nice to see manufacturers or designers participating in such a conversation with actual users contributing their experiences. I would call such a discussion a healthy one.

Probably, almost everything that happens in the audio chain can be eventually measured and probably explained. But do such a tool exist in the hands of a listener for taking buying decisions? In the absence of such tools, what should they do? The competence to make such calls may exist in the hands of a very competent manufacturer. There is one I know who vehemently recommends that the entire chain of gear in a system must be from his stable or the results can vary . I do not blame him..he designs gear with a certain goal in mind. A very well thought out one though.

The extreme objectivist tend to use spec sheets and published measurements and dbt validation as their tool for taking buying decisions. Many such people argue that there is no difference between a 200 $ receiver and a 2000 $ amp in a dbt. But they however buy themselves a 1200 $ emotiva. Bear in mind that they will not buy the 200 $ from bestbuy. Have they validated the 1200 $ emotiva in a dbt to take the buying decision to spend the extra 1000 $ ?

Some of the well-known objectivist manufactures and audio gear designers have some of the best listener validated gear in their personal systems. They also test their gear with such systems.

 

[raghupb]

Nikhil & Fantastic, you both have validated the fundamental principles which I too believe firmly. But many of the forum members don't seem to have enough faith on their own ears. How do we deal with the situation!?

This debate is turning out to be like two lovers on a empty railway track. Subjectivists on one rail and objectivists on the other.
Sometimes they hold hands and whisper sweet nothings to each other, other times they will argue to death. Most of the time they are oblivious to each other.

Captain,

Personally, I think I am an opportunistic buyer. Every purchase in my AV chain, except the TV, has been an almost blind buy.
This is because I have not had the opportunity or sometimes the patience to audition stuff. Here is the list
Onkyo DV-SP500 (in 2003) - Needed a DVD player
Marantz PM7001 (in 2007) - Needed an amp; auditioned Denon/HK/Marantz with whatever speakers they had been paired with in demo room
B&W DM303 (in 2001) - bought by my dad and handed down to me when I acquired the IA
Outlaw M2200 Amps and OSB-1 Soundbar (in 2012) - totally blind buy (friend was moving back from US; had them bring it along with their shipment)
ASUS media player (in 2014) - impulse buy; needed something that would play a wide range of audio/video formats
Marantz NR1605 (in 2015) - totally blind buy; needed it for HDMI and network capability; really did not care about power as I had Outlaws for L/R
Schiit M2U and SYS (in 2015) - totally blind buy; needed a DAC and a switcher; friend got them from US
KEF R300 (in 2015) - almost blind buy; KEF Q series had impressed me; cousin was moving back from UK; had her get a demo pair that was on sale
Speaker stands (in 2016) - this was a "weighted" buy; went, saw, lifted them and bought them

Now there are certain areas where my ensemble can be better, had I staged it in a different order or paid more attention/money.
But the important thing is nothing sounds bad or is under performing for their respective price points, feature set and spec.
To arrive at these decisions to buy, I trusted reviews (collectively others' subjective/objective mindset) and finally my gut feeling.

Cheers,
Raghu

 

[arj]

As in everything the truth is somewhere in the middle.

Assuming the musical content is good, If it measures bad and sounds good there is something wrong..either the wrong thing is measured or we are hearing something which only sounds good but is far away from reality

If it measures good but sounds wrong the equipment is a piece of junk

But very rarely do we have such extremes. mostly in this industry we do not have standardized parameters or norms. a 100W amp can sound less powerful to a 40 W amp as the ability of the 40 watt'er to act as a constant current may be higher or a 88 db speaker can go louder/more dynamic with lower power amps than a 95 db one simply because of its impedance and phase measurements are tougher ...and which no one publishes.

So practically we can measure but in reality its never published and there is no standardization of what to measure

 

[Fantastic]

...... a 100W amp can sound less powerful to a 40 W amp as the ability of the 40 watt'er to act as a constant current may be higher or a 88 db speaker can go louder/more dynamic with lower power amps than a 95 db one simply because of its impedance and phase measurements are tougher ...and which no one publishes.

Don't quite get what you mean. All commercial amps are voltage sources. A few dedicated ones built by experimenters are current sources but those aren't available for commercial sale. Maybe you mean that some amplifiers can handle lower loads ( 4 ohms) which draws higher current than a typical 8 ohm load and isn't specifically mentioned in the spec sheet. There are no constant current audio amplifiers.
About the 88dB speaker bit I lost it completely. Maybe you haven't expressed yourself properly. Try writing it again differently.

 

[captrajesh]

Personally, I think I am an opportunistic buyer. Every purchase in my AV chain, except the TV, has been an almost blind buy.

To arrive at these decisions to buy, I trusted reviews (collectively others' subjective/objective mindset) and finally my gut feeling.

You are one lucky guy. I for one have burned my fingers, (nah read pockets) to get to where I am.

 

[philipreji]

Let us take the example of those "exotic cables" which sound "better" than the ordinary speaker wire connected between the amplifier and the speaker. In this cable the "electrons are moving more smoothly through the metallic lattice owing to the patent pending quantum relativistic superconductor technology" adopted by the manufacturer.

The objectivist will insist that cable X is not going to make ANY difference, while the subjectivist will swear that he heard a difference.

Both are right!

Since the objectivist believes that all cables are pure resistances, his brain will not hear a difference. But in fact cable X is not pure resistance: the manufacturer has deliberately made it in such a way that it has substantial capacitive reactance. This will change the frequency response of the amp+speaker combination. If both the amp and the speaker are very good, the vigilant listener (subjectivist) may hear a difference. From this point onwards if is not too difficult for the Snake Oil salesman to talk his gullible prey into believing that the "different sound" is a "better sound".

No intention here to undermine the value of subjective tweaking of an otherwise excellent audio chain. Experienced people can certainly do this. For instance there was a report somewhere in the net about the DBT validation of the supremacy of a certain cable over ordinary wire in a certain system. Fact is, most of the time the customer is talked into believing that the "improvement" he heard in the showroom can be replicated in his home system, which is just a false assumption.

Measurement is the foundation of any decent audio component. Once quality components are put together to form an audio system in a given room, subjective tweaks may be attempted. In the end, this may or may not give good results. In either case, for most people the returns will not be worth the extra money spent.

cheers,
Reji

 

[keith_correa]

You are one lucky guy. I for one have burned my fingers, (nah read pockets) to get to where I am.

I've burned much more and I still haven't got to where I want to be. And where is that? Damned if I know!

 

[captrajesh]

I've burned much more and I still haven't got to where I want to be. And where is that? Damned if I know!

Don't want to go into how much is too much because it is all in one's own perspective..

I resolved to be happy at where I am and just enjoying the journey.

 

[raghupb]

Maybe off topic, but wanted to share this cartoon someone forwarded to me.
Seems very apt when we sometimes unwittingly analyze a single component or a single measurement/spec in isolation.

Cheers,
Raghu

 

[shaan]

Measurements: If one knows what to measure, one can determine why a component "sounds" the way it does. For example if we just measure the resistance of a resistor, we're not going to learn or determine anything by that number. But measure what it is doing to the signal and the answer is there. IMO, it is all attributable to harmonic distortion. Reji is right in what he said. We use terms like "airy", "precise", "punchy" but all of these can be explained technically. Like it or not, some distortion sounds good. Ask the "toob" guys.

Passive components do have particular sonic signatures to them. And there are certain nodes in a system where they affect the sonics to an audible degree. IMO this is a fault of the system that it can't eliminate the effect of that passive part and that's where the "magic" lies though some find it to be "garbage" instead of magic.

I think distortion is only half of the contributing factor to a system's SQ. The other half is the equalization of frequency response to the listener's liking, salt to taste. Pure current-output amplifiers have a rep of shocking newcomers and it is almost always not due to the THD readings. Some feel that the sound of an SE amp is to kill for and it is almost always not due to response.

I have met people with the idea of "punchy" and "precise" which are unique to each of them even when they are listening to the same system. Which of these people's describing terms should we take as a reference and why? And what if it contradicts with my thoughts about the system being or not being "punchy" and "precise"?

 

[Hari Iyer]

As per my experience phase coherence is the way to go forward anf has rewarding returns with minimal tweaks later to the placements and rooms.

 

[sidvee]

Which of these people's describing terms should we take as a reference and why? And what if it contradicts with my thoughts about the system being or not being "punchy" and "precise"?

So isn't this contradictory to the title of this thread "Why We Measure Audio Equipment Performance?" as Capt.Rajesh - the OP mentioned- so ultimately it is all subjective? Even hard objective measurements do not correspond to subjective opinions? To be honest this is what drives equipment purchases around the globe - at-least for listeners who take the trouble to go and listen to the equipment before buying - as opposed to armchair googlers. Perhaps even the best measurements will not appeal to some, myself included. Hence I will not buy anything without listening to it first - secondly and most importantly I WILL NOT comment on any equipment till I listen to it first!!!!
Cheers,
Sid

 

[shaan]

...Perhaps even the best measurements will not appeal to some, myself included. Hence I will not buy anything without listening to it first - secondly and most importantly I WILL NOT comment on any equipment till I listen to it first!!!!
Cheers,
Sid

Say I do a really complex math at home, then go to moon and perform the same math - the result will be same. Meanwhile auditory perception changes by moving to the next room and a lot of people know that their system sounds best when playing from under the sofa and and listening from the bathroom. :lol:

This is not to say that objective measurements are useless. These tell you the truth. But whether the truth corresponds to listening experience or not doesn't matter if the listener is happy with the sound. And if he is not, someone else will be coz there is no ideal sound, only your sound and my sound.

 

[philipreji]

In this time and age it will be atrocious if we can't have objective references in audio

A true high-fidelity system - where the output is just an amplified replica of the input and nothing else - will necessarily meet the requirements of zero distortion and complete phase coherence, and therefore, can be considered as a reference system. This will be an ideal, unachievable system, but fortunately nowadays it is possible to make realistic systems which are pretty close in performance.

Any departure from the above reference system (by virtue of selective choice of passive components and/or engineered phase modifications) is inevitably distortion, as Keith has pointed out. Many people like distortions - to each one his own distortion. I often use a Radioshack 15+15 band graphic equalizer in my system, and I'm quite happy with it, even though I know it will mess up the overall phase coherence of my system.

cheers,
Reji

 

[asliarun]

There's one more aspect here. For some strange unknown reason, no one makes an attempt to measure the quality of sound in a realistic listening room or living room setting, with a benchmark song or songs, with standard equipment placement.

If things like speaker placement, furniture, listening position, size of room, mastering quality, type of music - plays such a big role in the quality of audio reproduction, then it is quite an oversight to not establish standardized baselines for these aspects.

No audio manufacturer tells you that a certain speaker performs optimally if you listen to a certain set of classical or jazz or rock recordings, in a 15x15 room with certain furniture in place, with the speakers 8 feet apart and 2 feet from the wall, and the listener sitting at a certain height and 8 feet away from the speakers.

While this does not eliminate other variables, at least this kind of benchmarking would vastly simplify this whole voodoo cult notion of how to extract the best possible sound from your equipment and room.

And most importantly, the measurements and specifications should be measured and shared with these benchmark settings, not in some anechoic chamber that no listener possesses.

Side-note: A more sophisticated alternative would be for the manufacturer to use EQ and specific benchmarked response curves in the listening position to ensure that other variables get taken out of the equation - only via software instead.

Without this kind of benchmarking and standardization, I feel that the pursuit of audio fidelity becomes an exercise of blind people touching various parts of an elephant and trying to deduce what they are touching, and if there is more, and if so, how much more. And even if we don't jump to conclusions of calling the elephant a donkey (or do, like me sometimes, and realize we are making an ass of ourselves), we are still left with an overly painstaking approach to gaining insights into the true physical form of the elephant.

For what it is worth, this is ultimately a passive hobby. We don't create anything. We are not playing a musical instrument. We are trying to reproduce the sound of a musical instrument. The objective vs subjective debate become very clear in an active hobby. Say, you buy a cricket bat because it has 9 grains in it. Or you buy a car because it has certain specs and measurements and ratings. Or you buy a guitar because it has a solid maple top. You will consider all that but still rely on actual performance, actual driveability, actual musical quality of the guitar etc. to determine how good that piece of equipment is - and how well *it suits you*.

But listening to music has much less variables because we are not interacting with the components when listening to music, unlike playing an instrument. So it should be a lot easier for the objective and subjective to come closer to each other and get demystified. I just don't understand why no one tries to do it however.

 

[Hari Iyer]

In my experiences talking with people, i have seen that most of the guys who dont respect objective measurements are the ones who dont understand anything about it. Objective measurements are not only frequency response and impedance as FM think. It should include step response, impulse response, phase response, waterfall plot, cumulative spectral decay, spectrograms and distortion measurements.

Agreed a layman will not understand or be able to anslyze them, but that does not mean that they are humbug. I am not telling that subjective listening is not important. Ibelive that subjective listening should get validated by objective measurements too.

Imo chances of a well measured speakers sounding good is very high compared to a poorly measured one.