The problem is cable, that together with carrying you 0 and 1s, can carry some impurities too (EMI/RFI). These special cable manufacturers try to minimize this phenomena. A very simple analogy will be - when you open door for guests, together with guests, some mosquitoes too fly in and become nuisance later on.
I too had similar kind of opinion as yours a few years ago but things are different now.
Best analogy lol. But yeah into the depths of it, each signal (especially fast switching digital signals) will carry noise on the data line and also the return currents and emi on the ground line. There is line to line, line to ground effects and also emi from external environment. Assuming some random "isolator" at the receiving end is going to eliminate this completely is more fairytale than reality. If you think of transformer coupled galvanic isolation, you cannot guarantee full isolation because Maxwell's 4th law (which by the way has been fairly well established to be a valid model for analysis) ensures things will still creep in through and also the fact that real world magnetic properties of materials deviate a lot from ideality. If you're thinking optical isolation would fix it, you'd again be wrong since photon emission is a probabilistic process and same goes for absorption, it is quite hard to build a high speed optical isolator and even if you build, there is likely to be significant jitter introduced and if you try to cancel out jitter by special electronic design or signal regeneration, it'll yet again be a digital switching logic which will once again spur noise into the stream!
You don't need to flip a bit to cause issues with analog circuits like dacs and amplifiers. Ground stability problems, problems in clocks, etc will manifest in the output fidelity. If you try to make a design insensitive to these changes, more often than not such a design will also be filtering out useful signals along with it which means loss of fidelity. This can be thought of as a case of source insensitivity as opposed to absolute immunity.
And no, sine sweep tests, FFT donot necessarily characterize these effects. All of those are approximations that don't necessarily convey the full random signal performance of any system. They are useful for basic functionality testing, but for proper analysis of fidelity of a system, more rigorous methods may be needed, which may or may not be possible with limited budget, but thankfully we have an amazing hearing mechanism which can be the final arbiter for our choice/preference.
Best place to fix a source of noise is at the point where it is introduced. This is the reason I am more into pc+low noise software+decent cables. Even in this there cannot be guaranteed complete immunity/noise free structure (though better than letting it behave noisy here and trying to filter later, which will always leave residuals) since real world is always random, and what we could do further is try to detune/decorrelate these by specific buffering/cable length/cable shield changing schemes that can be adapted to the particular environment you are in. Ifi gemini cables come with the option to move a slider that can help in changing the antenna effect pattern on the wires. Phasure lush cable has options to change the grounding/shielding patterns on the fly.