What do the terms "CPU bound" and "I/O bound" mean?

It's pretty intuitive:

A program is CPU bound if it would go faster if the CPU were faster, i.e. it spends the majority of its time simply using the CPU (doing calculations). A program that computes new digits of π will typically be CPU-bound, it's just crunching numbers.

A program is I/O bound if it would go faster if the I/O subsystem was faster. Which exact I/O system is meant can vary; I typically associate it with disk, but of course networking or communication in general is common too. A program that looks through a huge file for some data might become I/O bound, since the bottleneck is then the reading of the data from disk (actually, this example is perhaps kind of old-fashioned these days with hundreds of MB/s coming in from SSDs).

CPU Bound means the rate at which process progresses is limited by the speed of the CPU. A task that performs calculations on a small set of numbers, for example multiplying small matrices, is likely to be CPU bound.

I/O Bound means the rate at which a process progresses is limited by the speed of the I/O subsystem. A task that processes data from disk, for example, counting the number of lines in a file is likely to be I/O bound.

Memory bound means the rate at which a process progresses is limited by the amount memory available and the speed of that memory access. A task that processes large amounts of in memory data, for example multiplying large matrices, is likely to be Memory Bound.

Cache bound means the rate at which a process progress is limited by the amount and speed of the cache available. A task that simply processes more data than fits in the cache will be cache bound.

I/O Bound would be slower than Memory Bound would be slower than Cache Bound would be slower than CPU Bound.

The solution to being I/O bound isn't necessarily to get more Memory. In some situations, the access algorithm could be designed around the I/O, Memory or Cache limitations. See Cache Oblivious Algorithms.

CPU bound means the program is bottlenecked by the CPU, or central processing unit, while I/O bound means the program is bottlenecked by I/O, or input/output, such as reading or writing to disk, network, etc.

In general, when optimizing computer programs, one tries to seek out the bottleneck and eliminate it. Knowing that your program is CPU bound helps, so that one doesn't unnecessarily optimize something else.

[And by "bottleneck", I mean the thing that makes your program go slower than it otherwise would have.]

Another way to phrase the same idea:

• If speeding up the CPU doesn't speed up your program, it may be I/O bound.

• If speeding up the I/O (e.g. using a faster disk) doesn't help, your program may be CPU bound.

(I used "may be" because you need to take other resources into account. Memory is one example.)

When your program is waiting for I/O (ie. a disk read/write or network read/write etc), the CPU is free to do other tasks even if your program is stopped. The speed of your program will mostly depend on how fast that IO can happen, and if you want to speed it up you will need to speed up the I/O.

If your program is running lots of program instructions and not waiting for I/O, then it is said to be CPU bound. Speeding up the CPU will make the program run faster.

In either case, the key to speeding up the program might not be to speed up the hardware, but to optimize the program to reduce the amount of IO or CPU it needs, or to have it do I/O while it also does CPU intensive stuff.

See what Microsoft says.

The core of async programming is the Task and Task objects, which model asynchronous operations. They are supported by the async and await keywords. The model is fairly simple in most cases:

• For I/O-bound code, you await an operation which returns a Task or Task inside of an async method.

• For CPU-bound code, you await an operation which is started on a background thread with the Task.Run method.

The await keyword is where the magic happens. It yields control to the caller of the method that performed await, and it ultimately allows a UI to be responsive or a service to be elastic.

I/O-Bound Example: Downloading data from a web service

private readonly HttpClient _httpClient = new HttpClient();

downloadButton.Clicked += async (o, e) =>
{
    // This line will yield control to the UI as the request
    // from the web service is happening.
    //
    // The UI thread is now free to perform other work.
    var stringData = await _httpClient.GetStringAsync(URL);
    DoSomethingWithData(stringData);
};

CPU-bound Example: Performing a Calculation for a Game

private DamageResult CalculateDamageDone()
{
    // Code omitted:
    //
    // Does an expensive calculation and returns
    // the result of that calculation.
}

calculateButton.Clicked += async (o, e) =>
{
    // This line will yield control to the UI while CalculateDamageDone()
    // performs its work.  The UI thread is free to perform other work.
    var damageResult = await Task.Run(() => CalculateDamageDone());
    DisplayDamage(damageResult);
};

Examples above showed how you can use async and await for I/O-bound and CPU-bound work. It's key that you can identify when a job you need to do is I/O-bound or CPU-bound, because it can greatly affect the performance of your code and could potentially lead to misusing certain constructs.

Here are two questions you should ask before you write any code:

Will your code be "waiting" for something, such as data from a database?

• If your answer is "yes", then your work is I/O-bound.

Will your code be performing a very expensive computation?

• If you answered "yes", then your work is CPU-bound.

If the work you have is I/O-bound, use async and await without Task.Run. You should not use the Task Parallel Library. The reason for this is outlined in the Async in Depth article.

If the work you have is CPU-bound and you care about responsiveness, use async and await but spawn the work off on another thread with Task.Run. If the work is appropriate for concurrency and parallelism, you should also consider using the Task Parallel Library.

An application is CPU-bound when the arithmetic/logical/floating-point (A/L/FP) performance during the execution is mostly near the theoretical peak-performance of the processor (data provided by the manufacturer and determined by the characteristics of the processor: number of cores, frequency, registers, ALUs, FPUs, etc.).

The peek performance is very difficult to be achieved in real-world applications, for not saying impossible. Most of the applications access memory in different parts of the execution and the processor is not doing A/L/FP operations during several cycles. This is called Von Neumann Limitation due to the distance that exists between the memory and the processor.

If you want to be near the CPU peak-performance a strategy could be to try to reuse most of the data in the cache memory in order to avoid requiring data from the main memory. An algorithm that exploits this feature is the matrix-matrix multiplication (if both matrices can be stored in the cache memory). This happens because if the matrices are size n x n then you need to do about 2 n^3 operations using only 2 n^2 FP numbers of data. On the other hand matrix addition, for example, is a less CPU-bound or a more memory-bound application than the matrix multiplication since it requires only n^2 FLOPs with the same data.

In the following figure the FLOPs obtained with a naive algorithms for the matrix addition and the matrix multiplication in an Intel i5-9300H, is shown:

Note that as expected the performance of the matrix multiplication in bigger than the matrix addition. These results can be reproduced by running test/gemm and test/matadd available in this repository.

I suggest also to see the video given by J. Dongarra about this effect.

I/O Bound process:- If most part of the lifetime of a process is spent in i/o state, then the process is a i/o bound process.example:-calculator,internet explorer

CPU Bound process:- If most part of the process life is spent in cpu,then it is cpu bound process.