Quotes in XML. Single or double?

I've heard that using single quotes to surround XML attribute values is a "bad style". Is this correct?

Should I always write:

<element attr="value">

Or is it acceptable to write:

<element attr='value'>

Or does it not matter which style I use?