How to implement fallback for SVG `<use href>` element for readers with normal vision?

I have an HTML5 document with fragment

<p class="special">Some text with an icon
<svg class=\"icon\">
<title>alternative text content for accessibility</title>
<use href="icons.svg#foo"></use>
</svg> in the middle of sentence.</p>

The <title> element is used to specify the alternative content for accessibility purposes. However, how should I markup the alternative content for readers with normal vision in case <use href> fails to load the required referenced resource to correctly render the image? It seems that the default action is to render element <use> fully transparent.

I would have expected to see <use alt="..."> to define fallback for the referenced resource but that doesn't seem to exist. How to specify fallback content for readers with normal vision (that is, not using accessibility tools that can access <title>)?

I'm trying to use inline CSS to allow referenced graphics to use style="fill:currentColor" to inherit correct color from the context (the color of which is defined with CSS using selector p.special). I know that if I used HTML5 <img> element I could just use the attribute alt.