What is the difference between concurrency and parallelism?

Concurrency is when two or more tasks can start, run, and complete in overlapping time periods. It doesn't necessarily mean they'll ever both be running at the same instant. For example, multitasking on a single-core machine.

Parallelism is when tasks literally run at the same time, e.g., on a multicore processor.

Quoting Sun's Multithreaded Programming Guide:

• Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism.

• Parallelism: A condition that arises when at least two threads are executing simultaneously.

Parallelism is simultaneous execution of processes on a multiple cores per CPU or multiple CPUs (on a single motherboard).

Concurrency is when Parallelism is achieved on a single core/CPU by using scheduling algorithms that divides the CPU’s time (time-slice). Processes are interleaved.

Units:

• 1 or many cores in a single CPU (pretty much all modern day processors)
• 1 or many CPUs on a motherboard (think old school servers)
• 1 application is 1 program (think Chrome browser)
• 1 program can have 1 or many processes (think each Chrome browser tab is a process)
• 1 process can have 1 or many threads from 1 program (Chrome tab playing Youtube video in 1 thread, another thread spawned for comments section, another for users login info)
• Thus, 1 program can have 1 or many threads of execution
• 1 process is thread(s)+allocated memory resources by OS (heap, registers, stack, class memory)

Concurrent programming execution has 2 types : non-parallel concurrent programming and parallel concurrent programming (also known as parallelism).

The key difference is that to the human eye, threads in non-parallel concurrency appear to run at the same time but in reality they don't. In non - parallel concurrency threads rapidly switch and take turns to use the processor through time-slicing. While in parallelism there are multiple processors available so, multiple threads can run on different processors at the same time.

Reference: Introduction to Concurrency in Programming Languages

concurency: multiple execution flows with the potential to share resources

Ex: two threads competing for a I/O port.

paralelism: splitting a problem in multiple similar chunks.

Ex: parsing a big file by running two processes on every half of the file.

They solve different problems. Concurrency solves the problem of having scarce CPU resources and many tasks. So, you create threads or independent paths of execution through code in order to share time on the scarce resource. Up until recently, concurrency has dominated the discussion because of CPU availability.

Parallelism solves the problem of finding enough tasks and appropriate tasks (ones that can be split apart correctly) and distributing them over plentiful CPU resources. Parallelism has always been around of course, but it's coming to the forefront because multi-core processors are so cheap.

Concurrency vs Parallelism

Rob Pike in 'Concurrency Is Not Parallelism'

Concurrency is about dealing with lots of things at once.

Parallelism is about doing lots of things at once.

[Concurrency theory]

Concurrency - handles several tasks at once
Parallelism - handles several thread at once

My vision of concurrency and parallelism

[Sync vs Async]

If at all you want to explain this to a 9-year-old.

From the book Linux System Programming by Robert Love:

Concurrency, Parallelism, and Races

Threads create two related but distinct phenomena: concurrency and parallelism. Both are bittersweet, touching on the costs of threading as well as its benefits. Concurrency is the ability of two or more threads to execute in overlapping time periods. Parallelism is the ability to execute two or more threads simultaneously. Concurrency can occur without parallelism: for example, multitasking on a single processor system. Parallelism (sometimes emphasized as true parallelism) is a specific form of concurrency requiring multiple processors (or a single processor capable of multiple engines of execution, such as a GPU). With concurrency, multiple threads make forward progress, but not necessarily simultaneously. With parallelism, threads literally execute in parallel, allowing multithreaded programs to utilize multiple processors.

Concurrency is a programming pattern, a way of approaching problems. Parallelism is a hardware feature, achievable through concurrency. Both are useful.

This explanation is consistent with the accepted answer. Actually the concepts are far simpler than we think. Don't think them as magic. Concurrency is about a period of time, while Parallelism is about exactly at the same time, simultaneously.

I really liked this graphical representation from another answer - I think it answers the question much better than a lot of the above answers

Parallelism vs Concurrency When two threads are running in parallel, they are both running at the same time. For example, if we have two threads, A and B, then their parallel execution would look like this:

CPU 1: A ------------------------->

CPU 2: B ------------------------->

When two threads are running concurrently, their execution overlaps. Overlapping can happen in one of two ways: either the threads are executing at the same time (i.e. in parallel, as above), or their executions are being interleaved on the processor, like so:

CPU 1: A -----------> B ----------> A -----------> B ---------->

So, for our purposes, parallelism can be thought of as a special case of concurrency

Source: Another answer here

Hope that helps.

"Concurrent" is doing things -- anything -- at the same time. They could be different things, or the same thing. Despite the accepted answer, which is lacking, it's not about "appearing to be at the same time." It's really at the same time. You need multiple CPU cores, either using shared memory within one host, or distributed memory on different hosts, to run concurrent code. Pipelines of 3 distinct tasks that are concurrently running at the same time are an example: Task-level-2 has to wait for units completed by task-level-1, and task-level-3 has to wait for units of work completed by task-level-2. Another example is concurrency of 1-producer with 1-consumer; or many-producers and 1-consumer; readers and writers; et al.

"Parallel" is doing the same things at the same time. It is concurrent, but furthermore it is the same behavior happening at the same time, and most typically on different data. Matrix algebra can often be parallelized, because you have the same operation running repeatedly: For example the column sums of a matrix can all be computed at the same time using the same behavior (sum) but on different columns. It is a common strategy to partition (split up) the columns among available processor cores, so that you have close to the same quantity of work (number of columns) being handled by each processor core. Another way to split up the work is bag-of-tasks where the workers who finish their work go back to a manager who hands out the work and get more work dynamically until everything is done. Ticketing algorithm is another.

Not just numerical code can be parallelized. Files too often can be processed in parallel. In a natural language processing application, for each of the millions of document files, you may need to count the number of tokens in the document. This is parallel, because you are counting tokens, which is the same behavior, for every file.

In other words, parallelism is when same behavior is being performed concurrently. Concurrently means at the same time, but not necessarily the same behavior. Parallel is a particular kind of concurrency where the same thing is happening at the same time.

Terms for example will include atomic instructions, critical sections, mutual exclusion, spin-waiting, semaphores, monitors, barriers, message-passing, map-reduce, heart-beat, ring, ticketing algorithms, threads, MPI, OpenMP.

Gregory Andrews' work is a top textbook on it: Multithreaded, Parallel, and Distributed Programming.

(I'm quite surprised such a fundamental question is not resolved correctly and neatly for years...)

In short, both concurrency and parallelism are properties of computing.

As of the difference, here is the explanation from Robert Harper:

The first thing to understand is parallelism has nothing to do with concurrency. Concurrency is concerned with nondeterministic composition of programs (or their components). Parallelism is concerned with asymptotic efficiency of programs with deterministic behavior. Concurrency is all about managing the unmanageable: events arrive for reasons beyond our control, and we must respond to them. A user clicks a mouse, the window manager must respond, even though the display is demanding attention. Such situations are inherently nondeterministic, but we also employ pro forma nondeterminism in a deterministic setting by pretending that components signal events in an arbitrary order, and that we must respond to them as they arise. Nondeterministic composition is a powerful program structuring idea. Parallelism, on the other hand, is all about dependencies among the subcomputations of a deterministic computation. The result is not in doubt, but there are many means of achieving it, some more efficient than others. We wish to exploit those opportunities to our advantage.

They can be sorts of orthogonal properties in programs. Read this blog post for additional illustrations. And this one discussed slightly more on difference about components in programming, like threads.

Note that threading or multitasking are all implementations of computing serving more concrete purposes. They can be related to parallelism and concurrency, but not in an essential way. Thus they are hardly good entries to start the explanation.

One more highlight: (physical) "time" has almost nothing to do with the properties discussed here. Time is just a way of implementation of the measurement to show the significance of the properties, but far from the essence. Think twice the role of "time" in time complexity - which is more or less similar, even the measurement is often more significant in that case.

Excerpt from this amazing blog:

Differences between concurrency and parallelism:

Concurrency is when two tasks can start, run, and complete in overlapping time periods. Parallelism is when tasks literally run at the same time, eg. on a multi-core processor.

Concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of (possibly related) computations.

Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.

An application can be concurrent – but not parallel, which means that it processes more than one task at the same time, but no two tasks are executing at same time instant.

An application can be parallel – but not concurrent, which means that it processes multiple sub-tasks of a task in multi-core CPU at same time.

An application can be neither parallel – nor concurrent, which means that it processes all tasks one at a time, sequentially.

An application can be both parallel – and concurrent, which means that it processes multiple tasks concurrently in multi-core CPU at same time .

Simply, concurrency is dealing lots of things at once.

Word ‘dealing’ is bold to show the difference between concurrency and parallelism. Dealing many things at once means completing many things at once, but it does not matter whether they are executed in the same time or not. On the other hand, doing parallelism means doing lots of things at once (executed in same time). Hence, concurrency context can be achieved with one or more processing resources. Dealing many things at once with one processing resource means that you are doing many things as they are executed in the same time by doing context switching between tasks. On the other side, concurrency context with many processing resources means doing parallelism. It means we are doing concurrency by doing parallelism, but not vice versa.

You might want to learn more about concurrency and parallelism and their relationship with nowadays technology in my article.

Merely to add even more clarification to other good answers:

Basing on the premise that an abstraction of processing (a CPU as a quite imaginable example) is able to run an only task at the same instant,

Concurrency is a story about the very abstraction of processing: it can switch between different tasks.

Parallelism is a story about we have more than one abstractions of processing (for example our CPU has multiple cores). So it's the cause of our system's ability to do several tasks at the same time (literally). But nothing is said here about the particular abstractions of processing (are they concurrent or not).

The emphasis here is on what these stories about.

So be aware when you are reading the accepted answer:

Concurrency is when two or more tasks can start, run, and complete in overlapping time periods.

Strickly speaking, one can conclude based on that definition that parallelism presupposes concurrency per se.

I think there is a confusion due to two different points of view on this question: point of view of the programmer (concurrent/parallel programming) vs point of view of the computer/operating system (concurrent/parallel execution).

Point of view of the computer is answered here.

Point of view of the programmer:

Concurrent programming: The programmer writes code knowing that the code will be executed by multiple threads, for whatever reason. The reasons could be: exploiting CPU better when waiting for I/O, handling Web requests by different threads, making GUI responsive by running calculations in a thread separate from the main one, running periodic background tasks. The programmer must apply mutual exclusion constructs, locking/unlocking, waiting on conditions/signaling, handle deadlocks etc. The multiple threads may run either on a single processor/core (concurrency from the computer’s point of view) or on multiple cores (parallelism from the computer’s point of view).

Parallel programming: The programmer knows that the program will run on a computer with multiple processors/cores and wants to exploit multiple cores. The programmer divides a CPU-intensive computation into multiple subtasks, runs each sub task in a thread and once the threads finish, their results are combined into the total result (divide-and-conquer). For example, dividing some matrix-processing code into tasks that process parts of a matrix in parallel. Each core will execute a thread with the subtask (or several threads concurrently, if number of threads is larger than the number of cores). The programmer must apply the programming constructs for concurrency here as well, but she is also concerned with dividing the task into sub-tasks and combining the results. In Java, for example, the programmer can use ParallelStreams that will split the data and combine the results automatically. If the programmer knows that the program will be executed on a single-core processor, there is no gain to split a CPU-intensive task into multiple threads. From Concurrent Programming in Java : Design Principles and Patterns, 2nd Edition, 1999, by Doug Leah, page 343:

Parallel programs are specifically designed to take advantage of multiple CPUs for solving computation-intensive problems.