After a lot of searching, the best explanation I've found is from Java Performance Tuning website in Question of the month: 1.4.1 Garbage collection algorithms, January 29th, 2003

Young generation garbage collection algorithms

The (original) copying collector (Enabled by default). When this collector kicks in, all application threads are stopped, and the copying collection proceeds using one thread (which means only one CPU even if on a multi-CPU machine). This is known as a stop-the-world collection, because basically the JVM pauses everything else until the collection is completed.

The parallel copying collector (Enabled using -XX:+UseParNewGC). Like the original copying collector, this is a stop-the-world collector. However this collector parallelizes the copying collection over multiple threads, which is more efficient than the original single-thread copying collector for multi-CPU machines (though not for single-CPU machines). This algorithm potentially speeds up young generation collection by a factor equal to the number of CPUs available, when compared to the original singly-threaded copying collector.

The parallel scavenge collector (Enabled using -XX:UseParallelGC). This is like the previous parallel copying collector, but the algorithm is tuned for gigabyte heaps (over 10GB) on multi-CPU machines. This collection algorithm is designed to maximize throughput while minimizing pauses. It has an optional adaptive tuning policy which will automatically resize heap spaces. If you use this collector, you can only use the the original mark-sweep collector in the old generation (i.e. the newer old generation concurrent collector cannot work with this young generation collector).

From this information, it seems the main difference (apart from CMS cooperation) is that UseParallelGC supports ergonomics while UseParNewGC doesn't.

UseParNewGC usually knowns as "parallel young generation collector" is same in all ways as the parallel garbage collector (-XX:+UseParallelGC), except that its more sophiscated and effiecient. Also it can be used with a "concurrent low pause collector".

See Java GC FAQ, question 22 for more information.

Note that there are some known bugs with UseParNewGC

I will try to answer this question using official Oracle docs:

What is the purpose of -XX:+UseParNewGC

-XX:+UseParallelGC v/s -XX:+UseParNewGC

• -XX:+UseParallelGC is parallel collector and -XX:+UseParNewGC is parallel young generation collector.
• Parallel young generation collector is similar to the parallel garbage collector in intent and differs in implementation. Unlike the parallel collector (­XX:+UseParallelGC) this parallel young generation collector can be used with the concurrent low pause collector that collects the tenured generation.

When -XX:+UseParNewGC should be enabled/used?

By default this option is disabled and it is automatically enabled when you set the -XX:+UseConcMarkSweepGC option. Using the -XX:+UseParNewGC option without the -XX:+UseConcMarkSweepGC option was deprecated in JDK 8.

In Summary