The fallacy of audio reviews

ajinkya

Active Member
Joined
Oct 27, 2007
Messages
506
Points
43
I always listen to equipment before making a final jugdement. However, I do rely on audio magzine reviews to zone in on particular components. I came across this article about the PSB Alpha B1s:
PSB Speakers - Stereophile Product of the Year
It mentions them being 'Product of the Year' in this US magazine.

Now check out their review at WhatHiFi:
PSB Alpha B1 review - whathifi.com
The mag gave them THREE stars out of 5. I can understand 5/5 or even a 4/5, saying product is good but there are better. But 3? That stands for a mediocre component. So what gives?

I think it has a lot to do with British vs. American tastes in music and what sounds sweet to one set of ears sounds 'dull' to the other. Which really makes me wonder about the accuracy of any reviewers, since we're finally getting an opinion...not an objective assesment. Which is also why I'm so wary of according the 1909 AVR 'Superking' status, just because WHF says so!
I am completely disillusioned by all these self-styled reviewers and their opinions, which are put forth as final judgement on the components they test. And unfortunately, since we users do base a lot of our buying decisions on such magazines, the audio companies are forced to pander to these whimsical reviews as well. Isn't something wrong in the chain somewhere?
No, I don't have an alternative, efficient system to review and judge the plethora of products out there...but when a component gets Best of the Best in one mag and Mediocre in another, someone is hard of hearing somewhere in this chain. Maybe its us, the users.
 
I hear you, loud and clear, Ajinkya. If you look around you would be hard put to find a review actually thrash the living daylights out of a product. And we know that not all speakers are fantastic sounding. This is indeed a huge problem. That is precisely why, nothing beats your ears. Forget objectivity, there is only subjectivity here.

But as far as Whathifi is concerned, I would personally take their advice with a pinch of salt. Take a look at their system recommendations, for example. For a low end system, their recommendation of a receiver is Onkyo. For a mid end system the recommendation still is Onkyo and for a high end system, dont hold your breath, it is again Onkyo.

Rest my case.
 
Hi,

Its a game of probabilities! If there are atleast 5 reviews on a product and 4 of them says good things about it, there are 80% chances that you will like it :cool:

Regards,
Prasad.
 
I suppose, in the end, it all boils down to the number of ads the companies place. Check What Hi Fi. You may find lot of Onkyo ads (full page - multi-colour). These magazines have to protect their commercial interests more
than the readers'.
 
There are a number of factors that we have to consider both for and against these reviewers:

1. The reviewers are human beings after all.
2. They are paid to do their jobs and have some factors affecting what they write. And you know where the payments come from.
3. They have been doing this for a long time, and will have a fair idea of what is good and bad.
4. They have good equipment and a decent environment to test these products.
5. There is a vast different in tastes of the Americans and the British. The British always end up using Orchestral music as test material and that is very difficult to use for judgment. The Yanks, on the other hand, use rock, male and female voices, movie sound, etc.
6. In the particular case of the Alpha, the WhatHifi review seems to have been written in a hurry and is very brief. The Stereophile review, on the other hand, is detailed and is followed by a technical testing of the product.
7. The equipment you use for a particular test also makes a difference. Amplifiers, cables, players, and of course the test material go a long way in affecting the tests.
8. Blind tests, as many leading magazines have themselves accepted, end up being very confusing. When a professional reviewer does not see what he is testing, he ends up giving comments which he himself will not believe when he opens his eyes. It just goes to show that all reviewers enter in to a test with some pre conceived notion. Unfortunately the testing of audio and video equipment is very subjective and this can never be changed.

For us Indians the points become even more acute. None of these equipment have been made for Indian music. Even Indian magazines use either western music to test equipment. Their use of Indian music is always 'after' the western music. What about the superb male and female Indian singers, what about the veena, jaltarang, sitar? What about Alla Rakha and Zakhir's Hussain's rendition of the tabla. What about the emotion in the voices of Bhimsen Joshi, MS Subbalakshmi, and even film singers such as Mukesh and Manna Dey?

We have to use these reviews in filtering the good from the bad. After that we have to trust our own ears in deciding which equipment is good.

Cheers
 
Well put, Venkat, as always. I have always found Stereophile to be detailed and the fact that they include the measurement means that it is not all about vague words like'warmth', 'richness' etc.

That the measurements are accompanied by explanations by the fantastic technical man there is a huge boon. When you read a review you are always hit by how much of a close knit team it is out there. At least that is what it looks like from outside:). The reviewer points a few things out in a speaker and surmises that the measurements probably would show why it is behaving this way and also makes qualitative observations. The technical man doing the measurements then chips in with actual measurement data to support the qualitative observations - making it quantitative and therefore somewhat more reliable.

Over and above everything though, it is upto the individual and whether the speaker satisfies one's own requirements. But reviews are the proverbial entrance tests. They help whittle things down to manageable sizes. Otherwise the world of hifi audio is simply too overwhelming for a new enthusiast. And yes, I am speaking out of experience:)

As I think about whether I want to add a subwoofer to my system or expand to a better bookshelf or a floorstander, I find out that there are still loads of ways to get confused. The only way to make a decision is to go and actually hear stuff. Everything else, as they say, is incidental.
 
I agree with all of Venkat's points and Vortex's too. In fact, Venkat's observation about no real testing of Indian music is spot on. And yes, all audio equipment sound is subjective...it's a fundamental human limitation.

What would be interesting in the Stereophile reviews is whether the two (listener and measurement man) test/review the equipment independently without talking to each other and after their observations are on paper, do they talk to each other. If this is the case, then it would make a very strong case for scientific/quantitative testing of audio equipment, because as Vortex pointed out, the Stereophile audio and measurement reviews are very matched in what they predict the equipment will sound like.
Does anyone know if this is the case?...i.e the listening and measurement tests are independently done first?
 
For us Indians the points become even more acute. None of these equipment have been made for Indian music. Even Indian magazines use either western music to test equipment. Their use of Indian music is always 'after' the western music. What about the superb male and female Indian singers, what about the veena, jaltarang, sitar? What about Alla Rakha and Zakhir's Hussain's rendition of the tabla. What about the emotion in the voices of Bhimsen Joshi, MS Subbalakshmi, and even film singers such as Mukesh and Manna Dey?



Cheers

Venkat,

I am surprised to see your statement: "Even Indian magazines use either western music to test equipment. ...."

During an audition Music pieces are selected that can test bass, mid and high, sound staging/ imaging, separation, tonal accuracy etc. So any such piece of music selected for testing, must be well recorded in the first place regardless of artist/s, genre and musical instruments used .

Unfortunately most of music produced in India (till recently) was poorly recorded, as well as poorly pressed, Most of the great work in Indian Music (read Film music) from Lata, Mukesh , Manna Day etc is in mono format , which can't be used for testing music, do you know that the first Stereo LP (read : Recording) done in India was in 1968! Before that all recording was in Mono format. Even today, recording and pressing quality is still so and so.

That's why the bias is towards well recorded western music (Rock, Pop, Jazz, Classical) .

Your one more statement is also quite surprising: "None of these equipment have been made for Indian music........"


Although certain design topology suits better for certain music genre, no manufacture designs his product tailored to a specific genre and further to music from any specific region / period / culture. If an amp can do Jazz well, it can do Indian Classical also, if an amp can do well Rock it can play R.D's tune with equal zeal. There is no need or demand of any audio gear designed especially for Indian Music. And if at all one wants to design any such gear then what kind of Indian Music to be used as a reference point? Hindi Film, South Indian Film, Hindustani Classical , Carnatic Classical, Robindra Shongeet, Gazal, Bhajan, Folk ?
 
Thanks all for your views!
when these kind of intelligent and meaningful discussions/debates happen, it enriches the collective wisdom and widens the horizons of all the forum participants.
Great! keep it coming.
You all have been wonderful.. and i must say, that among so many forums which i visit, HIFIVIsion stands out.. I have been around for an year and have seen it evolve leaps and bounds.
 
hi Suhas and Venkat....I think I have a middle ground to set out here.....true none of the systems are made with Indian music in mind....in which case, what an Indian listener should do is opt for components that introduce the minimum possible coloration or bias into the sound......Some speakers are designed to do a better job with rock music, and there's bump in the mid-bass provided for that purpose......that might work well with rock, but not so well with classical or Indian music.....so for Indian music, I think the trick is to pick the most neutral components possible, that provide the most faithful reproduction across the frequency spectrum......once that is done, the only reason your Bhimsen Joshi would sound bad is the quality of the recording.....and there's not too much you can do about that.....


Venkat,

I am surprised to see your statement: "Even Indian magazines use either western music to test equipment. ...."

During an audition Music pieces are selected that can test bass, mid and high, sound staging/ imaging, separation, tonal accuracy etc. So any such piece of music selected for testing, must be well recorded in the first place regardless of artist/s, genre and musical instruments used .

Unfortunately most of music produced in India (till recently) was poorly recorded, as well as poorly pressed, Most of the great work in Indian Music (read Film music) from Lata, Mukesh , Manna Day etc is in mono format , which can't be used for testing music, do you know that the first Stereo LP (read : Recording) done in India was in 1968! Before that all recording was in Mono format. Even today, recording and pressing quality is still so and so.

That's why the bias is towards well recorded western music (Rock, Pop, Jazz, Classical) .

Your one more statement is also quite surprising: "None of these equipment have been made for Indian music........"


Although certain design topology suits better for certain music genre, no manufacture designs his product tailored to a specific genre and further to music from any specific region / period / culture. If an amp can do Jazz well, it can do Indian Classical also, if an amp can do well Rock it can play R.D's tune with equal zeal. There is no need or demand of any audio gear designed especially for Indian Music. And if at all one wants to design any such gear then what kind of Indian Music to be used as a reference point? Hindi Film, South Indian Film, Hindustani Classical , Carnatic Classical, Robindra Shongeet, Gazal, Bhajan, Folk ?
 
suhas said:
Unfortunately most of music produced in India (till recently) was poorly recorded, as well as poorly pressed, Most of the great work in Indian Music (read Film music) from Lata, Mukesh , Manna Day etc is in mono format , which can't be used for testing music, do you know that the first Stereo LP (read : Recording) done in India was in 1968! Before that all recording was in Mono format. Even today, recording and pressing quality is still so and so.

That's why the bias is towards well recorded western music (Rock, Pop, Jazz, Classical) .

I have been hearing this for some time and even in this forum. And I must say I am surprised.

I think, stereo recording came to India within 5 or 6 years of it's coming to the US and Europe. After all the main recording company in India of those days - HMV - was a subsidiary of a British company. All original Beatles numbers and many Orchestral numbers were all recorded in mono. They were later converted to stereo. These same things have happened in India. Accepted, original recording were in mono. But, I have heard over a 1000 Hindi film songs that have pristine quality and are re-mastered in stereo. I have referred elsewhere to a release by HMV called 'The Classic Revival' which is a case in point.

When the first AV-Max exhibition was run, I had taken a special Yatra to Mumbai with over a 100 CDs. That was the first time that I auditioned the B&W's legendary 303s. I remember all the companies had just entered the fray and were small operations. When I entered the room with the 303s, they were as usual playing some very western rock. One of the most popular number those days was Hotel California. I quietly waited for the crowd to thin out, and requested the demo guy whether I can play my own CDs. He instantly agreed, but regretted it later. I gave him a CD of a Carnatic Classical singer called OS Arun. The reason I chose Arun was that he is a school mate of mine, and I know how his voice sounds exactly. When Arun was singing Meera Bjhajans through the system, one of the Cooper brothers came in, and literally screamed. "What are you guys playing?' Fortunately for me and the poor guy demonstrating, there was a huge crowd in the room who had all come in from other demo rooms. I spent the next hour or so playing a series of Indian CDs featuring Ravi Shankar, Hariprasad Chaurasia, MS Subbalakshmi, Shakthi with John McLaughlin, solo tabla by Zakhir Hussain, and a couple of CDs with film songs of the 1970s and 1980s. Actually Mehul Mepani (of AV Max) sauntered by and we started talking to each other, He tagged along with me for some time listening to the music I was using for auditioning. At least two of the demo guys wanted to grab my CDs.

The OS Arun CD was recorded by a small Chennai based company called Charsur (wwww,charsur.com). I have seen their recording methodology. They use multiple highly sensitive Shure mikes linked to a Nagra tape deck. The Analogue recordings are taken back to their small studio and carefully mixed for Stereo and converted to Digital. I have gone up and down between the live recordings and the studio produced CD, and the clarity and capacity to duplicate the environment is really good.

Frankly even on the most expensive systems with matching speakers, I found nothing wrong with the Indian CDs. Companies such as HMV, Times Music, Magnasound etc have released some very good Indian numbers. I do like Western music, but I also like to listen to Indian music regularly. Even on my trips abroad when I visit shops to audition system, I carry a large percentage of Indian music. Strangely, many shop keeps have expressed that they 'like' the music, and found it pleasant.

Some of the modern recordings are really exceptional. Some examples are Breathless by Shankar Mahadevan, Strings by Dhaani, Shaan featuring Bhool Na and other hits, Lucky Ali's Sifar, Golden Kirthis Colours featuring Zakhir Hussain, and Kunnakudi Vaidyanathan, ReDhoom by Euphoria. There are excellent CDs in Hindustani and Carnatic which I have not mentioned. CDs of films such as Main Hoon Na, Kal Ho Naa Ho, etc are very good. In terms of old film songs, in addition to The Classic Revival, HMV has a series called 'The Golden Collection' I find these also quite good.

I would really like to sit with one of you, listen to an Indian CD, and understand what is that you find missing. Maybe mu understanding is not enough.

suhas said:
Your one more statement is also quite surprising: "None of these equipment have been made for Indian music........"

Although certain design topology suits better for certain music genre, no manufacture designs his product tailored to a specific genre and further to music from any specific region / period / culture. If an amp can do Jazz well, it can do Indian Classical also, if an amp can do well Rock it can play R.D's tune with equal zeal. There is no need or demand of any audio gear designed especially for Indian Music. And if at all one wants to design any such gear then what kind of Indian Music to be used as a reference point? Hindi Film, South Indian Film, Hindustani Classical , Carnatic Classical, Robindra Shongeet, Gazal, Bhajan, Folk ?

Maybe I used the wrong words. But here is my thinking and my worries.

You take a electronics designer from the US or Europe. What music is he familiar with? Western music, of course. So when he designs a circuit, he will tune it in such a way that the output sounds familiar and pleasant to him. Not all designs are identical and sterile. That is why reviewers find some warm and some harsh. The same reason why British reviewers find US design harsh, while American reviewers find British designs as stale.

Take Hindustani or Carnatic. The base is either a tabla or a mridungam. Now the base in Western music, particularly rock, are heavy drums. So if you tune a system to emphasise the drums, how will it sound with a tabla or a mridungam? Will it color the high frequencies of the accompanying sitar, violin, or the female voice?

If you leave out the film music which has a large shade of western influence, most Indian singing are based upon Hindustani or Carnatic ragas. And there is a lot of commonality between these two, The primary notes are actually the same, and each style of singing has evolved based on local custom. There is a lot of commonality in the instruments also - for the sitar you have the veena, for the tabla you have the mridungam, for the Sarod you have the violin, etc. Actually you can take a Sitar and play Carnatic numbers as well as you can play Hindustani on a Veena. Zakhir Hussain can accompany a Hindustani or Carnatic singer/artist with equal aplomb.

I do know mid HiFi and high end audio system will never be 'tuned' for Indian music. But what troubles me is the sheer lack of musical knowledge of our Indian designers and dealers. At least in the lot I have interacted with.

When I auditioned Audire in Chennai some time ago, the Chief Designer said he was a Physicist and does not understand music at all !!! Other companies are, of course low end, and their emphasis is always on volume and enhanced base. All the demos I have seen use standard numbers such as Hotel California. Is that all there is to music?

You take modern fusion that has voices and a lot of electronic sounds, we know it will sound good on a reasonable system. But can we listen to a classical Hindustani, Carnatic or a old film song with equal confidence?

These are my worries.

Cheers
 
For us Indians the points become even more acute. None of these equipment have been made for Indian music. Even Indian magazines use either western music to test equipment. Their use of Indian music is always 'after' the western music. What about the superb male and female Indian singers, what about the veena, jaltarang, sitar? What about Alla Rakha and Zakhir's Hussain's rendition of the tabla. What about the emotion in the voices of Bhimsen Joshi, MS Subbalakshmi, and even film singers such as Mukesh and Manna Dey?

We have to use these reviews in filtering the good from the bad. After that we have to trust our own ears in deciding which equipment is good.

Cheers[/QUOTE]

Hi Venket, hey man i HV high regard for u.........any system which can play western classic will do justice with African or Indian or any classical music. It only depends on the audio signal quality of that particular recording in question ! And again any good system which can play rock or pop can play indi pop :)
My personal rig have Jamo + dussun amp for country/ blues/ jazz/ classic (Indian & Western) at times i do listen to pop on this !
JBL + CA for pop (Indian & Western)
some cds sound fantastic some bad............all due to recording only.
I have no problems & complains with my systems for the money spent & the performance getting !
Hope it clears
Regards to all
RB9
 
Sorry to jump in again but I second Venkat completely. He has expressed exactly what I feel more eloquently than I could have. And yes, I have seen marked differences in American vs. British musical tastes and we are using amps tuned to Western tastes (at the risk of sounding part of the 'Hindutva' brigade :D).
Of course, unless an indigenous company comes up with Indian audio designers, this situation will not change. However, it is certainly true that a system which performs beautifully for Western music may fall short when playing back Indian classical. The widely disparate reviews on the PSB A B1 just strengthens my opinion. It's one of several reviews I've compared between reviewers of different customs and how their tastes affect their critique. An audio amp designer is similarly human.

I remember reading a review on the Quad 11L2 where there is a pronounced dip in the low-mid frequency response. The rest of the response was flat so this was obviously a design feature. The American reviewer did not much mind it but the British reviewers found the mid dry, compared to the 11L. Obviously, the designer was trying to 'tune' the Quads more for a certain segment of musical tastes. I do not listen to a lot of Indian classical but from what little I do listen to, I find the dynamic range of voices much broader and the midrange nuances more subtle than in Western music. So I find it entirely conceivable that one can tune a musical component to better reproduce Indian music, just like one can tune any product to suit it's particular audience's tastes.

In fact, I just thought of an analogy from the electromechanical world: cars. Here's a quote from a news report on Audi entering the Indian market:
http://www.autoobserver.com/2008/01...for-slim-slice-of-small-india-luxury-pie.html
"Tuning the cars to meet Indian market demands is also something Audi has learned during its initial three years here in India. Weyler said many Audi A4 owners are driven by chauffeurs. Therefore, there is a unique demand for more rear seat entertainment options, stronger air-conditioning units with rear controls and greater rear legroom. Audi also must tune suspensions handle India??s less developed roads."

Similarly, BMW and Honda both tune their car's engine and performance to meet Indian conditions and Indian driving tastes. And I think we all can agree that BMW is one beautiful driving machine. Then why retune an a 'near-perfect' product when entering India (or any new region, for that matter). It's because Indian tastes are different from their Western counterparts.

I don't say that any good system will fail miserably at playing Indian music. All I'm saying is that the same system 'may' be tuned to play Indian music even better.
 
Last edited:
Venkat - I agree one hundred percent with your thoughts. Almost always I take along a large number of Indian music for speaker testing and invariably end up getting a few snide glances at the showroom. Never mind that.

You make a very good point about our percussion instruments. When considering our own brand of stringed instruments and unique ones like the Jaladharangam, Shehnai, Nadhaswaram, Thavil etc. things start getting more and more complex. And unique to Indian music.

While I dont believe that speaker manufacturers all over the world must make equipment which is tested on and cater to every genre of music, I believe that it falls upon us, the buyers, to test speakers using the kind of music that we most listen to. The music that we grew up with. The music that we go back to when in need of rhythm or peace or whatever it is we seek in music.

For testing speakers, our traditional Indian music does not have too much bass in it - as compared to Western music. If this is important this alone may be tested with appropriate sounds or music. Treble and midrange and the manner of their handling in Indian music - album or film - is indeed unique and deserves specific listening in order to determine the speakers' suitability.
 
Buy from India's official online dealer!
Back
Top