Determining binary/text file type in Java?

There's no guaranteed way, but here are a couple of possibilities:

1. Look for a header on the file. Unfortunately, headers are file-specific, so while you might be able to find out that it's a RAR file, you won't get the more generic answer of whether it's text or binary.

2. Count the number of character vs. non-character types. Text files will be mostly alphabetical characters while binary files - especially compressed ones like rar, zip, and such - will tend to have bytes more evenly represented.

3. Look for a regularly repeating pattern of newlines.

Using Java 7 Files class http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#probeContentType(java.nio.file.Path)

boolean isBinaryFile(File f) throws IOException {
        String type = Files.probeContentType(f.toPath());
        if (type == null) {
            //type couldn't be determined, assume binary
            return true;
        } else if (type.startsWith("text")) {
            return false;
        } else {
            //type isn't text
            return true;
        }
    }

Have a look at the JMimeMagic library.

jMimeMagic is a Java library for determining the MIME type of files or streams.

If the file consists of the bytes 0x09 (tab), 0x0A (line feed), 0x0C (form feed), 0x0D (carriage return), or 0x20 through 0x7E, then it's probably ASCII text.

If the file contains any other ASCII control character, 0x00 through 0x1F excluding the three above, then it's probably binary data.

UTF-8 text follows a very specific pattern for any bytes with the high order bit, but fixed-length encodings like ISO-8859-1 do not. UTF-16 can frequently contain the null byte (0x00), but only in every other position.

You'd need a weaker heuristic for anything else.

Just to let you know, I've chosen quite a different path. I my case, there are only 2 types of files, chances that any given file will be a binary one are high. So

1. presume that file is binary, try doing what's supposed to be done (e.g. deserialize)
2. catch exception
3. treat file as textual
4. if that fails, something is wrong with file itself

See http://en.wikipedia.org/wiki/Magic_number_(programming)

You could try Apache Tika, I've opened a request specifically for this feature

but for now, I think this might work... needs more thorough testing, may also have the problem of other mime type libraries in which you still really need a mapping from the type to whether or not it's binary.

var config = TikaConfig.getDefaultConfig();
var tika = new Tika( config );
var mimeTypes = config.getMimeRepository();

var mimetype = tika.detect(Path.of("my/foo"));
var rootType = mimeTypes.forName( mime ).getType().getType();
rootType.endsWith( "text" ); // text and x-text

You could try the DROID tool.

Building on the suggestion from xenoterracide, here is an implementation using Tika Core.

    public boolean isText(byte[] contentBytes) {
        Tika tika = new Tika();
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(contentBytes);
        try {
            Set<MediaType> mediaTypes = new HashSet<>();
            MediaType mediaType = MediaType.parse(tika.detect(byteArrayInputStream));
            MediaTypeRegistry mediaTypeRegistry = MediaTypeRegistry.getDefaultRegistry();
            while(mediaType != null) {
                mediaTypes.addAll(mediaTypeRegistry.getAliases(mediaType));
                mediaTypes.add(mediaType);
                mediaType = mediaTypeRegistry.getSupertype(mediaType);
            }
            return mediaTypes.stream().anyMatch(mt -> mt.getType().equals("text"));
        } catch (IOException e) {
            e.printStackTrace();
            return false;
        }
    }

There are several different Tika.detect methods you can use depending on if you are starting from a File, have a file name, etc. See https://javadoc.io/static/org.apache.tika/tika-core/2.4.1/org/apache/tika/Tika.html