It's easy to say that data is data and the ones and zeros are all the same --- but this completely overlooks the A in DAC. Every DAC, sound card, or sound interface (assuming analogue output) has analogue circuitry.
Added to which, even the digital side, and especially USB, is not implemented equally. S/PDIFF and USB carry the same digital data, don't they? So why does one sound different to other on the same DAC? This is not one of those "audiophile" controversial things, but widely reported as being the case. There are technology differences; there are implementation differences. There are even high-end proprietary USB implementations, like RME. Which pisses me off, because even I saved up enough to buy it, I would not get the benefit in Linux --- but that is another story.
In a decade of PC audio, I have never used a USB interface/DAC. This is partly because, in the early days, USB1.0 was not really good for sound (and some of that reputation has stained USB2.0). Very likely that I will be trying it in the next year or two.
If a person wants to only play (not record) music, exclusively from PC(s) then a USB DAC must be the way to go for economy and simplicity. If the S/PDIFF implementation of a particular box is so much better than USB that it is worth buying a converter, then isn't it the wrong box to buy for the purpose?
Always possible: take analogue-out from a PC sound card. ESI Julai@ or better will surprise you, as my RME card surprised me years ago. Do bear in mind that the designers of decent sound cards have thought about internal box electrical noise --- and having a well-made PSU probably helps too. There are also sound "cards" (simpler to call them "interfaces") which do the business externally (mine does) from dedicated PCI card or with Firewire.
There is a bigger range of PC audio possibilities. One huge advantage to that USB DAC, though, is being able to plug it into any PC/Laptop ...anywhere.