What's the difference between failure and error in JUnit?

If your test throws an exception which does not get bubbled up through the Assertion framework in Junit, it gets reported as an error. For example, a NullPointer, or a ClassNotFound exception will report an error:

String s = null;
s.trim();

or,

try {

    // your code
} catch(Exception e) {
    // log the exception
    throw new MyException(e);
}

Having said that, the following will report a failure:

Assert.fail("Failure here");

or,

Assert.assertEquals(1, 2);

or even:

throw new AssertionException(e);

It depends on the Junit version you are using. Junit 4- will make the distinction between a failure and an error, but Junit 4 simplifies it as failures only.

Following link provides more interesting inputs:

http://www.devx.com/Java/Article/31983/1763/page/2

You are right that failures come from the AssertionErrors thrown by the JUnit assertion methods, or by throwing an AssertionError, or by throwing an exception that you declared in your @Test annotation, and Errors come from other, unexpected Exceptions. But there's an important distinction between them:

A failure means your test ran correctly, and identified a defect in your code.

An error could mean a bug in your code, but one that you weren't even testing for. It could also mean that the bug is in the test itself.

In short, a failure means you need to rewrite the code that is being tested. An error means that it may be the unit test that you need to rewrite. It may mean this even if the failure was in your code, such as a NullPointerException, because you detected a flaw that you weren't even testing for, so it might be wise to test for that.

• The assert methods just throw AssertionExceptions.
• An exception is declared expected in the @Test annotation.
• Code outside of test methods can be stuff annotated with @Before, @BeforeClass and so on.
• The counts in the IDE reports count tests (test methods), not assertions, because the first failed assertion terminates the test.
• As a rule of thumb, if the test method terminates abnormally (not with a return statement), it is unsuccessful, and vice versa.

Ironically, junit and other testing related frameworks (testng, hamcrest) provide assert operations which verify condition and if it fails then "under-the-hood" a java.lang.AssertionError is being thrown, which btw extends java.lang.Error.

But it no way contradicts with answers above which are fully valid of course. So in order to mark specific test flow as failure one can throw AssertionError, however I'm not sure it's really documented in corresponding manuals, because more appropriate to use dedicated fail() API. Other kinds of Throwable will be considered as errors, not failures.

Basically, failures refer to unfulfilled assertions while errors are due to abnormal test execution. and I think each IDE has symbolic icons with different colors for passed, failed, and with error tests.

For more information, check this.