My List of things which Impact the final Outcome , in this order of preference. And this order of investment also.
Room Characteristics : Most Speakers don't sound the same when they're in different rooms, it happens so often that a speaker that sounds excellent in audition room, sounds very badly in your room. Room also includes the other parameters like acoustics , speaker positioning & speaker isolation etc which come into picture. Two totally different speakers sound nearly similar (Sound signature wise) due to room characteristics, though the difference can still be heard by well trained ears. Again there is no limit to investment here, from inexpensive thing like optimum speaker positioning and spikes to professional acoustic treatment, it varies. We need to remember that we are not only listening to the speakers, but to the room and all the things inside. But flexibility and budget with alteration of room acoustics is mostly limited to most of us. So Speakers become the next most important investment which will impact our SQ by quite a margin.
Speakers : I was long under this impression that Contribution of speakers is the maximum, but my current realization is that the speakers , matching with the room under consideration are more useful. you just have to Select the type of speakers that perform better in your room, rest everything Can be manipulated to further optimize the SQ. But if your Speakers are of inferior quality/room mismatch , there will be a ceiling to the performance, and any amount of good gear can't supplement for it. Speakers Matching to your room and Your choice of music is equally important.
Unfortunately there is no easy way to know which speakers suit your room best, so always look for home audition whenever possible , or listening at other members rooms , which closely mimic your listening scenario. luckily the regular branded speakers are designed to suit most of the genre of music , You wouldn't go horribly wrong with your choices. As you go Higher say above 3L it becomes much more subjective and a tougher choice to make.
Source & Synergy: this is self explanatory I think, If the source file or Player doesn't retrieve the information adequately no amount of Post Processing will improve the SQ significantly.
Finally the Synergy between Electronics where other components come into play; Dac/ Pre/Power / Cables etc.. If they are horribly mismatched you will end up with less optimal performance. The other electronics like amp etc can negatively affect the SQ by mismatch , But the Positive effect is not as impactful as the change of speakers. So they take a second preference as far as budget is concerned. One more thing is the Pure Power supply that needs to be emphasized because of fluctuations which adversely affect the performance of various electronics.