Of course as long as we don't notice something really bad, maybe it's cool.

Let us take an skewed example. let us say you have an amp that has a maximum capacity of 50 watts per channel. And, let us say, you are driving speakers that can be driven upto 100 watts.
As you go on increasing the volume, the amplifier will reach its limits before the speaker does. As you continue to increase the volume, the amplifier will simply cut off or 'clip' the sound signal. What usually happens is the that low frequencies first and then the high frequencies start getting chopped off from the sound signal. What started as a sine wave will become a distorted square wave.
Unfortunately humans have a great capacity to listen to clipped signals without realising what they are hearing. In hard rock, for example, guitar amplifiers are usually clipped far beyond their capacity as that distorted sound is what the artists are looking for.
Cheers