Compiling C# with non-standard language features

I need to compile user-typed expressions into .NET assemblies that can be invoked. The expression language (inherited from version 1 of the application) is very limited and is almost valid C#, but it does have some unusual properties.

I want to switch to full C# in version 2, but I'm having trouble converting the idiosyncrasies into a compileable SyntaxTree using Roslyn. At the moment I'm trying to figure out how to get powers and factorials to compile, i.e.

double n = 1.5² / 0.9³ + 6!;

has to be converted into something like:

double n = Pow(1.5, 2) / Pow(0.9, 3) + Factorial(6);

However when I parse the original expression it all breaks down immediately at the ² character:

public static void TestLanguageAdditions()
{
  const string code = "double n = 1.5² / 0.9³ + 6!;";

  var syntaxTree = CSharpSyntaxTree.ParseText(code);
  var syntaxRoot = syntaxTree.GetRoot();

  var sb = new StringBuilder();
  sb.AppendLine(code);
  sb.AppendLine();

  foreach (var node in syntaxRoot.DescendantNodes())
  {
    var span = node.FullSpan;
    var spanLine = 
      new string(' ', span.Start) + 
      new string('●', span.Length);

    spanLine = spanLine.PadRight(30);
    spanLine += ((CSharpSyntaxNode)node).Kind();

    sb.AppendLine(spanLine);
  }

  string report = sb.ToString();
  Console.Write(report);
}

Yields:

double n = 1.5² / 0.9³ + 6!;

●●●●●●●●●●●●●●●●●●●●●●●●●●●●  FieldDeclaration
●●●●●●●●●●●●●●                VariableDeclaration
●●●●●●●                       PredefinedType
       ●●●●●●●                VariableDeclarator
         ●●●●●                EqualsValueClause
           ●●●                NumericLiteralExpression

I want to know if there's any way to coax Roslyn to parse the rest of the string as-is, or if not, what a good strategy would be for preparing the string. Obviously I could replace the ² and ³ characters with some parseable placeholder, but not when those chars occur inside string or char literals.

----------------------------------- Edit -----------------------------------

Converting the illegal characters into block comments allows for parsing, although weirdly they appear as trivia after many of the nodes:

double n = 1.50/*²*/ / (0.90)/*³*/ + 6/*!*/;
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● CompilationUnit
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● FieldDeclaration
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●· VariableDeclaration
······································●●●●●·  ˃ trivia: /*!*/
●●●●●●●····································· PredefinedType
·······●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●· VariableDeclarator
······································●●●●●·  ˃ trivia: /*!*/
·········●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●· EqualsValueClause
······································●●●●●·  ˃ trivia: /*!*/
···········●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●· AddExpression
······································●●●●●·  ˃ trivia: /*!*/
···········●●●●●●●●●●●●●●●●●●●●●●●●········· DivideExpression
·····························●●●●●··········  ˃ trivia: /*³*/
···········●●●●●●●●●●······················· NumericLiteralExpression
···············●●●●●························  ˃ trivia: /*²*/
·······················●●●●●●●●●●●●········· ParenthesizedExpression
·····························●●●●●··········  ˃ trivia: /*³*/
························●●●●················ NumericLiteralExpression
·····································●●●●●●· NumericLiteralExpression
······································●●●●●·  ˃ trivia: /*!*/