How can I safely encode a string in Java to use as a filename?

user_2959 0
Viewed 95323

I'm receiving a string from an external process. I want to use that String to make a filename, and then write to that file. Here's my code snippet to do this:

    String s = ... // comes from external source
    File currentFile = new File(System.getProperty("user.home"), s);
    PrintWriter currentWriter = new PrintWriter(currentFile);

If s contains an invalid character, such as '/' in a Unix-based OS, then a java.io.FileNotFoundException is (rightly) thrown.

How can I safely encode the String so that it can be used as a filename?

Edit: What I'm hoping for is an API call that does this for me.

I can do this:

    String s = ... // comes from external source
    File currentFile = new File(System.getProperty("user.home"), URLEncoder.encode(s, "UTF-8"));
    PrintWriter currentWriter = new PrintWriter(currentFile);

But I'm not sure whether URLEncoder it is reliable for this purpose.

encoding file java string
11 Answers

My suggestion is to take a "white list" approach, meaning don't try and filter out bad characters. Instead define what is OK. You can either reject the filename or filter it. If you want to filter it:

String name = s.replaceAll("\\W+", "");

What this does is replaces any character that isn't a number, letter or underscore with nothing. Alternatively you could replace them with another character (like an underscore).

The problem is that if this is a shared directory then you don't want file name collision. Even if user storage areas are segregated by user you may end up with a colliding filename just by filtering out bad characters. The name a user put in is often useful if they ever want to download it too.

For this reason I tend to allow the user to enter what they want, store the filename based on a scheme of my own choosing (eg userId_fileId) and then store the user's filename in a database table. That way you can display it back to the user, store things how you want and you don't compromise security or wipe out other files.

You can also hash the file (eg MD5 hash) but then you can't list the files the user put in (not with a meaningful name anyway).

EDIT:Fixed regex for java

cletus_183930

It depends on whether the encoding should be reversible or not.

Reversible

Use URL encoding (java.net.URLEncoder) to replace special characters with %xx. Note that you take care of the special cases where the string equals ., equals .. or is empty!¹ Many programs use URL encoding to create file names, so this is a standard technique which everybody understands.

Irreversible

Use a hash (e.g. SHA-1) of the given string. Modern hash algorithms (not MD5) can be considered collision-free. In fact, you'll have a break-through in cryptography if you find a collision.

¹ You can handle all 3 special cases elegantly by using a prefix such as "myApp-". If you put the file directly into $HOME, you'll have to do that anyway to avoid conflicts with existing files such as ".bashrc". 
public static String encodeFilename(String s)
{
    try
    {
        return "myApp-" + java.net.URLEncoder.encode(s, "UTF-8");
    }
    catch (java.io.UnsupportedEncodingException e)
    {
        throw new RuntimeException("UTF-8 is an unknown encoding!?");
    }
}

vog_191630

If you want the result to resemble the original file, SHA-1 or any other hashing scheme is not the answer. If collisions must be avoided, then simple replacement or removal of "bad" characters is not the answer either.

Instead you want something like this. (Note: this should be treated as an illustrative example, not something to copy and paste.)

char fileSep = '/'; // ... or do this portably.
char escape = '%'; // ... or some other legal char.
String s = ...
int len = s.length();
StringBuilder sb = new StringBuilder(len);
for (int i = 0; i < len; i++) {
    char ch = s.charAt(i);
    if (ch < ' ' || ch >= 0x7F || ch == fileSep || ... // add other illegal chars
        || (ch == '.' && i == 0) // we don't want to collide with "." or ".."!
        || ch == escape) {
        sb.append(escape);
        if (ch < 0x10) {
            sb.append('0');
        }
        sb.append(Integer.toHexString(ch));
    } else {
        sb.append(ch);
    }
}
File currentFile = new File(System.getProperty("user.home"), sb.toString());
PrintWriter currentWriter = new PrintWriter(currentFile);

This solution gives a reversible encoding (with no collisions) where the encoded strings resemble the original strings in most cases. I'm assuming that you are using 8-bit characters.

URLEncoder works, but it has the disadvantage that it encodes a whole lot of legal file name characters.

If you want a not-guaranteed-to-be-reversible solution, then simply remove the 'bad' characters rather than replacing them with escape sequences.

The reverse of the above encoding should be equally straight-forward to implement.

user_1399850

If you don't care about reversibility, but want to have nice names in most circumstances that are cross platform compatible, here is my approach.

//: and ? into .
name = name.replaceAll("[\\?:]", ".");

//" into '
name = name.replaceAll("[\"]", "'");

//\, / and | into ,
name = name.replaceAll("[\\\\/|]", ",");

//<, > and * int _
name = name.replaceAll("[<>*]", "_");
return name;

This turns:

This is a **Special** "Test": A\B/C is <BETTER> than D|E|F! Or?

Into:

This is a __Special__ 'Test'. A,B,C is _BETTER_ than D,E,F! Or.
user_20755370

If your system stores files in a case sensitive filesystem (where it is possible to store a.txt and A.txt in the same directory), then you could use Base64 in the variant "base64url". It is "URL- and filename-safe" according to https://en.wikipedia.org/wiki/Base64#Variants_summary_table because it uses "-" and "_" instead of "+" and "/".

Apache commons-codec implements this: https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html#encodeBase64URLSafeString-byte:A-

If your filename / directory name is too long then split it into multiple directories: [first 128 characters]/[second 128 characters]/...

As there is no dot in the Base64 charset you don't have to care about special filenames like . or .. or about a final dot at the end of the filename. Also you don't have to care about trailing spaces, ...

If there are reserved words/filenames in your filesystem (or your operating system) like LPT4 in Windows and the result of Base64url-encoding is equal to a reserved word like this you could mask it with e.g. an @ character (@LPT4) and removing the masking @ character before decoding. Have a look for reserved words here: https://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words

In a Linux system this could work forwards and backwards without loss of data/characters, I guess. Windows will reject having two files named e.g. "abcd" and "ABCD".

Admin1

Convert your String hexadecimal (e.g. with this https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Hex.html#encodeHexString-byte:A- ). Works forwards and backwards ( https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Hex.html#decodeHex-char:A- ).

Split the resulting String into chunks of 128 characters with one (sub)directory for every chunk.

Even in case-insensitive filesystems / operating systems there is no collision (like it could be in Base64).

At the moment I don't know any reserved filename (like COM, LPT1, ...) that would have a collision with a HEX value, so I guess that there is no need for masking. And even if masking would be needed then use e.g. a @ in front of the filename and remove it when decoding the filename into the original String.

Admin1
Related