11 Concurrency

Your computer does a lot of work and does it so fast that you don’t usually realize how much it’s doing. Now and then, though — especially on an older computer or phone — you might notice an app slow down or even freeze. This might express itself during an animation as jank: that annoying stutter that happens when the device does so much work that some animation frames get dropped.

Long-running tasks generally fall into two categories: I/O tasks and computationally intensive tasks. I/O, or input-output, includes reading and writing files, accessing a database or downloading content from the internet. These all happen outside the CPU, so the CPU has to wait for them to complete. On the other hand, computationally intensive tasks happen inside the CPU. These tasks might include decrypting data, performing a mathematical calculation or parsing JSON.

As a developer, you must consider how your app, and particularly your UI, will respond when it meets these time-consuming tasks. Can you imagine if a user clicked a download button in your app, and the app froze until the 20 MB download was complete? You’d be collecting one-star reviews in a hurry.

Thankfully, Dart has a powerful solution baked into the very core of the language, allowing you to handle delays gracefully without blocking your app’s responsiveness.

Concurrency in Dart

A thread is a sequence of commands that a computer executes. Some programming languages support multithreading — running multiple threads simultaneously — but others don’t. Dart is a single-threaded language.

“What? Was it designed back in 1990 or something?”

No, Dart was created in 2011, well into the age of multicore CPUs.

“What a waste of all those other processing cores!”

Ah, but no. The developers deliberately made Dart single-threaded, providing significant advantages, as you’ll soon see.

Parallelism vs. Concurrency

To understand Dart’s model for handling long-running tasks and to see why Dart’s creators decided to make Dart single-threaded, it helps to understand the difference between parallelism and concurrency. In common English, these words mean about the same thing, but a distinction exists in computer science.

Parallelism is when multiple tasks run at the same time on multiple processors or CPU cores; concurrency is when multiple tasks take turns running on a single CPU core. When a restaurant has a single person alternately taking orders and clearing tables, that’s concurrency. But a restaurant that has one person taking orders and a different person clearing tables, that’s parallelism.

“It seems like parallelism is better.”

It can be — when there’s a lot of work to do and that work is easily split into independent tasks. However, parallelism has some disadvantages, too.

A Problem With Parallelism

Little Susie has four pieces of chocolate left in the box next to her bed. She used to have ten, but she’s already eaten six of them. She’s saved the best ones for last because three friends are coming home with her after school today. She can’t wait to share the chocolates with them. Imagine her horror, though, when she gets home and finds only two pieces of chocolate left in the box! After a lengthy investigation, it turns out that Susie’s brother had discovered the stash and helped himself to two of the chocolates. From then on, Susie locked the box whenever she left home.

The same thing can happen in parallel threads with access to the same memory. One thread saves a value in memory and expects the value to be the same when the thread checks the value later. However, if a second thread modifies the value, the first thread gets confused. It can be a major headache to track down those kinds of bugs because they come from a source completely separate from the code that reports the error. A language that supports multithreading needs to set up a system of locks so values won’t change at the wrong time. The cognitive load of designing, implementing and debugging a system with multiple threads can be heavy.

So the problem isn’t with parallelism but rather with multiple threads having access to the same state in memory.

Dart Isolates

Dart’s single thread runs in what it calls an isolate. Each isolate has its own allocated memory, ensuring that no isolate can access any other isolate’s state. That means there’s no need for a complicated locking system. It also means sensitive data is much more secure. Such a system greatly reduces the cognitive load on a programmer.

“But isn’t concurrency slow?”

If you’re running all of a program’s tasks on a single thread, it seems like it would be really slow. However, it turns out that’s not usually the case.

In the following image, you can see multiple tasks running on two threads in parallel. A rectangle represents each task, and longer rectangles represent longer-running tasks. A flat line represents an idle state where the thread isn’t doing anything:

The next image shows the same tasks running concurrently on a single thread:

The concurrent version does take a little longer, but it isn’t much longer. The reason is that the parallel threads were idle for much of the time. A single thread is usually more than enough to accomplish what needs to be done.

Flutter has to update the UI 60 times a second. Each update timeslice is called a frame. That leaves about 16 milliseconds to redraw the UI on each frame. It typically doesn’t take that long, giving you time to perform other work while the thread is idle. The user won’t notice any problems as long as that work doesn’t block Flutter from updating the UI on the next frame. The trick is to schedule tasks during the thread’s downtimes.

Synchronous vs. Asynchronous Code

The word synchronous consists of syn, meaning “together”, and chron, meaning “time”, thus together in time. Synchronous code executes each instruction in order, one line of code immediately following the previous one.

This contrasts with asynchronous code, which means not together in time. Asynchronous code reschedules certain tasks to run in the future when the thread isn’t busy.

All the code you’ve written so far in the book has been synchronous. For example:

print('first');
print('second');
print('third');

Run that, and it prints:

first
second
third

Because the code executes synchronously, it’ll never print in a different order like third first second.

For many tasks, order matters:

• You have to open the bottle before you can take a drink.
• You have to turn on the car before you can drive it.
• Multiplying before adding is different than adding before multiplying.

For other tasks, though, the order doesn’t matter:

• It doesn’t matter if you brush your teeth first or wash your face first.
• It doesn’t matter if you put a sock on the right foot first or the left foot first.

As in life, so it is with Dart. Although some code must execute in order, other tasks can be temporarily postponed. The postponable tasks are where the Dart event loop comes in.

The Event Loop

You’ve learned that Dart employs concurrency on a single thread, but how does Dart manage to schedule tasks asynchronously? Dart uses what it calls an event loop to execute tasks that had been postponed.

The event loop uses a data structure called a queue. Think of a queue like waiting in line at the grocery store. When you first join the line, you stand at the back of the line. Then, you slowly move to the front of the line as people before you leave. The first one in line is the first to leave. For that reason, developers call a queue a first-in-first-out, or FIFO, data structure. Dart uses queues to schedule tasks to execute on the main isolate.

The event loop has two queues: an event queue and a microtask queue. The event queue is for events like a user touching the screen or data coming in from a remote server. Dart primarily uses the microtask queue internally to prioritize certain small tasks that can’t wait for the tasks in the event queue to finish.

Look at the following image:

• Synchronous tasks in the main isolate thread are always run immediately. You can’t interrupt them.
• If Dart finds any long-running tasks that agree to be postponed, Dart puts them in the event queue.
• When Dart finishes running the synchronous tasks, the event loop checks the microtask queue. If the microtask queue has any tasks, the event loop puts them on the main thread to execute next. The event loop keeps checking the microtask queue until it’s empty.
• If the synchronous tasks and microtask queue are both empty, the event loop sends the next waiting task in the event queue to run on the main thread. Once it gets there, the code executes synchronously. Like any other synchronous code, nothing can interrupt it after it starts.
• If new microtasks enter the microtask queue, the event loop handles them before the next event in the event queue.
• This process continues until all the queues are empty.

Typically, if all the tasks are finished, this would indicate that it’s time to exit the main isolate and terminate the application. However, the isolate will stay around if it’s waiting for a response from the outside world. Maybe that’s a timer that the isolate previously started, or perhaps it’s listening for a response from a user or remote server.

Running Code in Parallel

When people say Dart is single-threaded, they mean Dart only runs on a single thread in the isolate. However, that doesn’t mean you can’t have tasks running on another thread. One example of this is when the underlying platform performs some work at the request of Dart. For example, when you ask to read a file on the system, that work isn’t happening on the Dart thread. The system is doing the work inside its own process. Once the system finishes its work, it passes the result back to Dart, and Dart schedules some code to handle the result in the event queue. A lot of the I/O work from the dart:iolibrary happens this way.

Another way to perform work on other threads is to create a new Dart isolate. The new isolate has its own memory and thread working in parallel with the main isolate. The two isolates are only able to communicate through messages, though. They have no access to each other’s memory state. The idea is like messaging a friend. Sending your friend a text message doesn’t give you access to the internal memory of their mobile device. They simply check their messages and reply to you when they feel like it.

You won’t often need to create a new isolate. However, if you have a task that’s taking too long on your main isolate thread, which you’ll notice as unresponsiveness or jank in the UI, then this work is likely a good candidate for handing it off to another isolate. Chapter 14, “Isolates”, will teach you how to do that.

Observing the Event Loop

Theory is nice, but it’s time for some cold, hard code. In this chapter, you’ll use the Future class to observe the event loop by adding tasks to the event and microtask queues. In Chapter 12, “Futures”, you’ll learn to use Future for more practical applications.

Adding a Task to the Event Queue

Passing a block of code to Future causes Dart to put that code on the event queue rather than running it synchronously.

Write the following code in main:

print('first');

Future(
  () => print('second'),
);

print('third');

The constructor of Future takes an anonymous function. Future then adds that function to the event queue.

Run the code above. This is what you’ll get:

first
third
second

second comes last. Why is that? Think about what’s happening:

1. Dart always runs the synchronous code first. print('first') is synchronous, so Dart executes it immediately on the main isolate.
2. Then Dart comes to Future. Dart takes the function inside Future and adds it to the event queue. The event queue code has to wait for all the synchronous code to finish before it can go.
3. print('third') is also synchronous, so Dart executes that next.
4. Finally, there’s no more synchronous code, so Dart takes print('second') off the event queue and executes it.

Adding a Task to the Microtask Queue

You’ll only need to add a task to the microtask queue once in a blue moon.

Like, literally.

Count how many times you’ve seen a blue moon in your life. How many did you get? Zero? Yeah, that’s about how often you’ll need to explicitly put something on the microtask queue in your Dart career. Unless you’re writing some low-level library, you can forget about this queue and trust Dart to handle the events there.

However, should the need arise, scheduling tasks on the microtask queue is possible.

Replace the code in main with the following:

print('first');

Future(
  () => print('second'),
);

Future.microtask(
  () => print('third'),
);

print('fourth');

Adding an anonymous function to the Future.microtask constructor puts this code on the microtask queue.

Run the code above and check the results:

first
fourth
third
second

Here’s the step-by-step:

1. Dart always runs synchronous code first, so that’s why first and fourth are first.
2. Dart added print('second') to the event queue and print('third') to the microtask queue.
3. Once the synchronous code finishes, Dart prioritizes any code in the microtask queue. That’s why third is next.
4. Finally, when the microtask queue is empty, Dart gives the code in the event queue a chance. That’s why second is last.

Running Synchronous Code After an Event Queue Task

Sometimes you might want to perform a task immediately after a task from the event queue finishes.

Replace the code in main with the following:

print('first');

Future(
  () => print('second'),
).then(
  (value) => print('third'),
);

Future(
  () => print('fourth'),
);

print('fifth');

Here are a few notes:

• The then method of a Future instance will execute an anonymous function immediately after the future completes. This code is synchronous.
• When futures complete successfully, they return a value. You’ll learn more about that in the next chapter. For now, ignore value in the then callback.
• You’ll also learn about async/await syntax in Chapter 12, “Futures”. It’s a little easier to use than then.

Run the code above. This is what you’ll see:

first
fifth
second
third
fourth

By now, you should know why first and fifth are first. They’re both synchronous, and synchronous code always goes first. second and fourth both get added to the event queue, but because then runs its code synchronously after second finishes, third jumps in before fourth has a chance to come off the event queue.

Intentionally Delaying a Task

Sometimes, it’s useful to simulate a long-running task. You can accomplish this with Future.delayed. Dart will add a task to the event queue after some time.

Replace the contents of main with the following code:

print('first');

Future.delayed(
  Duration(seconds: 2),
  () => print('second'),
);

print('third');

Future.delayed takes two parameters. The first is the duration of time you want to wait before starting the task. The second is the function you want to run after completing the duration. Dart adds the function to the event queue at that point.

Run the code above. First, you only see the following:

first
third

But two seconds later, Dart adds second to the list:

first
third
second

By the time the duration passed, all the synchronous code was long finished, so print('second') didn’t have to wait very long on the event queue. Dart executed it right away.

Is this all starting to make sense? If not, the challenge below and its accompanying explanation should help you.

Challenge

Before moving on, here’s a challenge to test your understanding of how Dart handles asynchronous tasks. An explanation follows the challenge, but try to figure out the solution yourself before looking.

Challenge 1: What Order?

In what order will Dart print the numbered statements? Why?

void main() {
  print('1 synchronous');
  Future(() => print('2 event queue')).then(
    (value) => print('3 synchronous'),
  );
  Future.microtask(() => print('4 microtask queue'));
  Future.microtask(() => print('5 microtask queue'));
  Future.delayed(
    Duration(seconds: 1),
    () => print('6 event queue'),
  );
  Future(() => print('7 event queue')).then(
    (value) => Future(() => print('8 event queue')),
  );
  Future(() => print('9 event queue')).then(
    (value) => Future.microtask(
      () => print('10 microtask queue'),
    ),
  );
  print('11 synchronous');
}

Write your answer down before reading the solution that follows.

Solution to Challenge 1

For brevity, the explanations below will refer to each task by its number. For example, print('1 synchronous') is abbreviated as 1.

Step 0

void main() {
  // ...
}

Dart creates the main isolate and calls your main function:

Step 1

print('1 synchronous');

1 is synchronous, so Dart executes it immediately in the main isolate:

Step 2

Future(() => print('2 event queue')).then(
  (value) => print('3 synchronous'),
);

Dart adds 2 to the event queue.

Step 3

Future.microtask(() => print('4 microtask queue'));

Dart adds 4 to the microtask queue:

Step 4

Future.microtask(() => print('5 microtask queue'));

Dart adds 5 to the microtask queue:

Step 5

Future.delayed(
  Duration(seconds: 1),
  () => print('6 event queue'),
);

Dart starts an internal timer for one second. The queues remain unchanged:

Step 6

Future(() => print('7 event queue')).then(
  (value) => Future(() => print('8 event queue')),
);

Dart adds 7 to the event queue:

Step 7

Future(() => print('9 event queue')).then(
  (value) => Future.microtask(
    () => print('10 microtask queue'),
  ),
);

Dart adds 9 to the event queue:

Step 8

print('11 synchronous');

11 is synchronous, so Dart executes it immediately:

Step 9

print('4 microtask queue');

All the synchronous tasks have finished, so Dart executes the first task in the microtask queue:

Step 10

print('5 microtask queue');

Dart then executes the next task in the microtask queue:

Step 11

print('2 event queue');

The microtask queue is empty now, so Dart takes the first task off of the event queue and executes it in the main isolate:

Step 12

Future(() => print('2 event queue')).then(
  (value) => print('3 synchronous'),
);

As soon as 2 finishes, Dart executes 3 synchronously:

Step 13

print('7 event queue');

Dart takes 7 off of the event queue and executes it:

Step 14

Future(() => print('7 event queue')).then(
  (value) => Future(() => print('8 event queue')),
);

When 7 finishes, Dart schedules 8 at the end of the event queue:

Step 15

print('9 event queue');

Dart takes 9 off of the event queue and executes it:

Step 16

Future(() => print('9 event queue')).then(
  (value) => Future.microtask(
    () => print('10 microtask queue'),
  ),
);

When 9 finishes, Dart adds 10 to the microtask queue:

Step 17

print('10 microtask queue');

The microtask queue has priority over the event queue, so Dart executes 10 before 8:

Step 18

print('8 event queue');

The microtask queue is empty now, so Dart takes 8 off of the event queue and executes it:

Step 19

The queues are all empty now:

However, Dart is still waiting on the Future.delayed timer it set back in Step 5, so the isolate doesn’t exit yet.

Step 20

Future.delayed(
  Duration(seconds: 1),
  () => print('6 event queue'),
);

Sometime later, the duration finally completes, so Dart adds 6 to the event queue:

Step 21

print('6 event queue');

There’s nothing to wait for, so Dart takes 6 off the event queue and executes it:

Step 22

The queues are all empty again:

Dart isn’t waiting for anything either, so the isolate exits and the application terminates.

Result

Here is the final output:

1 synchronous
11 synchronous
4 microtask queue
5 microtask queue
2 event queue
3 synchronous
7 event queue
9 event queue
10 microtask queue
8 event queue
6 event queue

If you wrote down the correct answer, give yourself a well-deserved pat on the back!

Key Points

• Dart is single-threaded and handles asynchronous programming through concurrency rather than parallelism.
• Concurrency refers to rescheduling tasks to run later on the same thread, whereas parallelism refers to running tasks simultaneously on different threads.
• Dart uses an event loop to schedule asynchronous tasks
• The event loop has an event queue and a microtask queue.
• A queue is a first-in-first-out (FIFO) data structure.
• Synchronous code always runs first and cannot be interrupted. After this comes anything in the microtask queue, and when these finish, any tasks in the event queue.
• You can run code in parallel by creating a new isolate.

Where to Go From Here?

You learned about queues as first-in-first-out data structures in this chapter. If you’d like to learn more, as well as how to build a queue, check out the “Queues” chapter in Data Structures & Algorithms in Dart.