YES, it is possible.
Clearly, as you have mentioned, it is a Constraint and not something that you are opting for over an RCA to RCA or XLR to XLR interconnect (which should be your Logical and 1st Preference).
XLR output is usually Twice as high as an RCA output, but it should not be a concern at all, except in the most extreme cases where say a DAC outputs 8 Volts RMS... Even then, it is workable with the amplifier volume control way low.
I presume the Amplifier that you are referring to has a volume control ...
You will probably have to get XLR to RCA interconnects made for you, as they are not easily available.
There are adapters available locally for XLR to RCA Conversion. You then use an XLR To XLR Interconnect with the adapters at one one. These adapters, (with their nickel plating) may not be the last word in Hi Fi signal purity, but you may not be able to hear a difference on many setups.
I am posting pictures of the adapters (available at Lamington Road Mumbai for Rs 125 each)