IMO, in a practical scenario, the biggest difference in SQ in a digital audio setup comes from the Drivers > Amplification>> DAC>>>> Cables
My subjective view:
Drivers: Cheap speakers with weak magnets , shoddy cones and bad casing will always yield poor sound - The SQ difference improvement continues to be quite linear from 2000/- boomboxes to 15K entry level bookshelves to 75-100K towers/bookshelves - Beyond that you hit the diminishing returns curve
Amplifier While I am far from an expert, I have extensively tested the SQ difference in a stereo setup between a mid-high end integrated Amp (Marantz PM17SA), a mid end AVR (Marantz SR6010), a low end AVR (Denon 1611) a cheap class D DIY Amp (TPA3116) and a class AB cheap stereo Amp in a box (the kind you get at the local electrical shops)
With the exception of the last one (which was horrible), all of the others were admirable performers
And while there was definitely a delta between them, for someone working on a fixed budget, a much better improvement can be found by spending the extra money on better speakers
DAC Tested specifically this weekend on a KEF Q300 paired with the Marantz PM 17 (a mid-end setup)
Compared the SQ of an old (but acclaimed) DAC viz M-Audio , Chromecast Audio (known to use a decent DAC), Airport Express (known to use a good DAC), the 3.5mm out from a Raspberry Pi and the 3.5mm out from a 150/- USB sound card
The source in all cases was 320kbps MP3s
The story was similar to what I saw with amps, with the exception of the 3.5mm out from the RPi (which was very bad), all other DACs performed at nearly the same level - I personally found the chromecast audio to be the best of the lot - But the SQ diff between all the rest was too small to be bothered with
Cables I will leave this one aside - I truly believe that as long as you don't do silly things (like frayed /poorly shielded cables at the line-in stage or the 3Rs /meter red/black wires for your speakers, you won't hear a difference
Moral of the long rant: If you are out to get the best sound from your money on a stereo setup (and as long as your budget is south of say 75K for just stereo), please allocate most of the money to the speakers
If it is between 75K-150K, start looking at Amp upgrades
If it is between 150K-250K, then only should you start looking at DAC upgrades