There are reports of better sound with higher sampling rates. No doubt, the folks that like the "sound of a 192KHz" converter hear something. Clearly it has nothing to do with more bandwidth: the instruments make next to no 96KHz sound, the microphones don't respond to it, the speakers don't produce it, and the ear can not hear it.
Moreover, we hear some reports about "some of that special quality captured by that 192KHz is retained when down sampling to 44.1KHz. Such reports neglect the fact that a 44.1KHz sampled material can not contain above 22.05KHz of audio.
Some claim that that 192K is closer to the audio tape. That same tape that typically contains "only" 20KHz of audio gets converted to digital by a 192K AD, than stripped out of all possible content above 22KHz (down sample to CD).
If you hear it, there is something there is an artistic statement. If you like it and want to use it, go ahead. But whatever you hear is not due to energy above audio. All is contained within the "lower band". It could be certain type of distortions that sound good to you. Can it be that someone made a real good 192KHz device, and even after down sampling it has fewer distortions? Not likely. The same converter architecture can be optimized for slower rates and with more time to process it should be more accurate (less distortions).
The danger here is that people who hear something they like may associate better sound with faster sampling, wider bandwidth, and higher accuracy. This indirectly implies that lower rates are inferior. Whatever one hears on a 192KHz system can be introduced into a 96KHz system, and much of it into lower sampling rates. That includes any distortions associated with 192KHz gear, much of which is due to insufficient time to achieve the level of accuracy of slower sampling.
Conclusion:
There is an inescapable tradeoff between faster sampling on one hand and a loss of accuracy, increased data size and much additional processing requirement on the other hand.
AD converter designers can not generate 20 bits at MHz speeds, yet they often utilize a circuit yielding a few bits at MHz speeds as a step towards making many bits at lower speeds.
The compromise between speed and accuracy is a permanent engineering and scientific reality.
Sampling audio signals at 192KHz is about 3 times faster than the optimal rate. It compromises the accuracy which ends up as audio distortions.
While there is no up side to operation at excessive speeds, there are further disadvantages:
1. The increased speed causes larger amount of data (impacting data storage and data transmission speed requirements).
2. Operating at 192KHz causes a very significant increase in the required processing power, resulting in very costly gear and/or further compromise in audio quality.
The optimal sample rate should be largely based on the required signal bandwidth. Audio industry salesman have been promoting faster than optimal rates. The promotion of such ideas is based on the fallacy that faster rates yield more accuracy and/or more detail. Weather motivated by profit or ignorance, the promoters, leading the industry in the wrong direction, are stating the opposite of what is true.