java api design - NULL or Exception

Josh Bloch recommends returning empty objects or "unit" objects in place of null values. Check out Effective Java for a ton of useful Java tidbits.

I think the answer is entirely dependent on the context of your application, and what you consider to be an "exceptional circumstance" versus programmer error.

For example, if writing an XML parser you may choose to implement methods like:

/**
 * Returns first child element with matching element name or else
 * throws an exception.  Will never return null.
 */
Element getMandatoryChildElement(Element parent, String elementName)
throws MissingElementException;

... as this approach allows you to implement your message parsing code in a single block of logic, without having to check that the message is well-formed after retrieving each element or attribute.

If null is an acceptable return value (ie, it can result from valid inputs rather than an error case) then return null; otherwise throw an exception.

I don't at all understand your statement that a checked exception may be a bad idea because the caller is forced to handle them - this is the whole point of checked exceptions. It guards against error conditions propagating into the code without being handled properly.

In Practial API Design, the author advises to not return null value (see p.24).

His main point is that it leads to the proliferation of checks against null value and more documentation.

I also strongly recommend reading "Null References: The Billion Dollar Mistake" by an authority in the field (T. Hoare).

My point is that

Never ever return NULL when return type is collection or an array. If no return value , then return an empty collection or an empty array.

This eliminates the need for null check.

Good practice:

public List<String> getStudentList()
{
     int count = getStudetCount();
     List<String> studentList = new ArrayList<String>(count);
     //Populate studentList logic
     return studentList;
}

Bad Practice:. Never ever write the code like below.

public List<String> getStudentList()
{
     int count = getStudetCount();
     if(count == 0)
     {
         return null;
     }
     //Populate studentList logic
     return studentList;
}

I tend to return null to "getXXX" methods that, for example, fetch something from the database example : User getUserByName(String name) is OK for me to return null

But, if you NEED what you are fetching by calling the method, throw an exception : ex : getDatabaseConnection() must always return a connection, and never return null

And, for methods that return a collection or an array, NEVER return null. return an empty collection/array instead

It really depends on how you want that situation to be handled. If it is truly an error condition then throw an exception. If null is an acceptable output then merely document it and let the user handle it.

If you have internal library errors (unexpected), let the runtime exceptions propagate. If there is a bug in the library code, the client is expected to crash. Fix the bug in the library. Don't catch all and return a library-specific exception, this won't do any good.

If you expect things to (sometimes) go wrong, build this into the API. This is based on the principle that you shouldn't use exceptions for normal program flow.

For instance:

ProcessResult performLibraryTask(TaskSpecification ts)

This way you can have ProcessResult indicate error conditions:

ProcessResult result = performLibraryTask(new FindSmurfsTaskSpecification(SmurfColor.BLUE));
if (result.failed()) {
    throw new RuntimeException(result.error());
}

This approach is similar to the return null approach, but you can send more information back to the client.

EDIT:

For interaction which doesn't conform to the agreed protocol, you can throw runtime errors, which you can document. For instance:

if (currentPeriod().equals(SmurfColor.BLUE) && SmurfColor.GREEN.equals(taskSpecification.getSmurfColor()) {
    throw new IllegalStateException("Cannot search for green smurfs during blue period, invalid request");
}

Please note that this is due to an interaction which breached the contract, and is not expected to happen.

A agree with the others, if null is an acceptable return value, return it, but otherwise not.

What I would like to add, is that you can also distinguish between checked and unchecked exceptions. In some cases it may be advisable to use unchecked exception, e.g. the user of your API is probably not in the situation to deal with that exception anyways. Spring Framework uses unchecked Exceptions very often, which makes the code often easier to read, but its harder to track down errors sometimes.