Azure CloudFile - "The specifed resource name contains invalid characters."

I'm trying to download a file from Azure File Storage to a local file and get this exception:

"The specifed resource name contains invalid characters."

Here's the code:

if (_cloudFileShare.Exists())
{
      CloudFileDirectory rootDir = _cloudFileShare.GetRootDirectoryReference();    
      CloudFileDirectory tempDir = rootDir.GetDirectoryReference("temp");
      if (tempDir.Exists())
      {
        var file = tempDir.GetFileReference(saveFrom);
        file.DownloadToFile(saveTo, FileMode.Open);// OFFENDING LINE
      }
 }

The saveTo argument is a string and the value is something like this:

"C:\Users\Me\AppData\Local\Temp\tmpF2AD.tmp"

The saveFrom argument is something like this:

https://storageaccount.file.core.windows.net:443/fileshare/temp/tmpA2DA.tmp

I'm creating the argument using this function:

var saveTo = Path.GetTempFileName();

What am I doing wrong? I don't have much experience with Azure.