Tasks, Microtasks and Batched Tasks

Promises are hard. I love them, and when paired with async/await they are my favorite feature in JavaScript. But they are really hard.

Promises are spececed in ECMAscript. The spec says that promises should be queued into jobs. What exactly jobs are is ambiguous, but the major ECMAScript embedders have converged on having two queues. In the web these are called tasks and microtaks.

Tasks

Here is some fake code that matches my mental model of whats going on in the browser.

// shared among threads
/** @type {Array<Function>} */
const tasks = [];

function mainThread() {
  while(true) {
    const task = tasks.pop();
    // Tasks can be added by another thread, so this isn't an infite loop.
    if (!task)
      continue;
    task();
  }
}

The tasks array can be modified by any thread. So clicking would work something like this:

// on io thread
operatingSystem.addEventListener('click', event => {
  // queue a task to be run on the main thread
  tasks.add(() => {
    document.activeElement.dispatchEvent(event);
  });
});

Originally when some browsers implemented Promises, they queued all callbacks into new tasks, like so:

class Promise {
  constructor(resolveGetter) {
    this._callbacks = [];
    this._resolved = false;
    resolveGetter(resolve.bind(this));

    function resolve(value) {
      this._resolved = true;
      this._value = value;
      for (const callback of this._callbacks)
        tasks.push(() => callback(value));
    }
  }

  then(continuation) {
    if (this._resolved)
      tasks.push(() => continuation(this._value));
    else
      this._callbacks.push(continuation);
  }
}

Microtasks

But today, promises use a different queue, called microtasks.

// shared among threads
/** @type {Array<Function>} */
const tasks = [];

/** @type {Array<Function>} */
const microtasks = [];

function mainThread() {
  while(true) {
    const task = tasks.pop();
    // Tasks can be added by another thread, so this isn't an infite loop.
    if (!task)
      continue;
    task();
    runQueuedMicrotasks();
  }
}

function runQueuedMicrotasks() {
  while (microtasks.length) {
    const microtask = microtasks.pop();
    microtask();
  }
}

Microtasks are always run before the next task. To use them, the Promise implementation only has to replace tasks with microtasks.

class Promise {
  constructor(resolveGetter) {
    this._callbacks = [];
    this._resolved = false;
    resolveGetter(resolve.bind(this));

    function resolve(value) {
      this._resolved = true;
      this._value = value;
      for (const callback of this._callbacks)
        microtasks.push(() => callback(value));
    }
  }

  then(continuation) {
    if (this._resolved)
      microtasks.push(() => continuation(this._value));
    else
      this._callbacks.push(continuation);
  }
}

See Jake Archibald’s amazing article about microtasks for a great explaination of all of this. But there is still more to the story.

Batched Tasks

setTimeout runs a callback on the main thread after a delay. My mental model for setTimeout looks something like this:

function setTimeout(callback, delay) {
  const time = Date.now() + delay;
  // operatingSystem.runWhenItIs calls the callback on a different thread when it is time
  operatingSystem.runWhenItIs(time, () => {
    // push a new task to call callback on the main thread
    tasks.push(callback);
  }, delay);
}

NodeJS does something fancy though. In order to avoid setting lots of timers on the operating system, they will batch timeouts together if they should run at about the same time.

const runningTimers = new Map()
function setTimeout(callback, delay) {
   const time = Date.now() + delay;
   if (runningTimers.has(time)) {
    runningTimers.get(time).push(callback);
    return;
  }

  runningTimers.set(delay, [callback]);

  // operatingSystem.runWhenItIs calls the callback on a different thread when it is time
  operatingSystem.runWhenItIs(time, () => {
    const timers = runningTimers.get(time);
    runningTimers.remove(time);
    // push a new task to call callback on the main thread
    tasks.push(() => {
      // on the main thread, run all of the batched timer callbacks
      for (const callback of timers)
        callback();
    });
  }, delay);
}

NodeJs has another function setImmediate which runs a callback in the next task. It is also batched, but without the delay its much easier to follow.

let queuedImmediates = null;
function setImmediate(callback) {
  if (!queuedImmediates) {
    queuedImmediates = [];
    tasks.push(() => {
      for (const callback of queuedImmediates)
        callback();
    });
  }
  queuedImmediates.push(callback);
}

This is how things looked until Node v11. Considered a bugfix, now node calls microtasks in between every queued immediate/timeout. This better matches the browser behavior.

let queuedImmediates = null;
function setImmediate(callback) {
  if (!queuedImmediates) {
    queuedImmediates = [];
    tasks.push(() => {
      for (const callback of queuedImmediates) {
        callback();
        runQueuedMicrotasks(); // The important change! Run microtasks after every callback.
      }
    });
  }
  queuedImmediates.push(callback);
}

const runningTimers = new Map()
function setTimeout(callback, delay) {
   const time = Date.now() + delay;
   if (runningTimers.has(time)) {
    runningTimers.get(time).push(callback);
    return;
  }

  runningTimers.set(delay, [callback]);

  // operatingSystem.runWhenItIs calls the callback on a different thread when it is time
  operatingSystem.runWhenItIs(time, () => {
    const timers = runningTimers.get(time);
    runningTimers.remove(time);
    // push a new task to call callback on the main thread
    tasks.push(() => {
      // on the main thread, run all of the batched timer callbacks
      for (const callback of timers) {
        callback();
        runQueuedMicrotasks(); // The important change! Run microtasks after every callback.
      }
    });
  }, delay);
}

This same pattern of batching events in a single task can still occur in user code though. I’ve run into it in the ws package for node, and in Playwright’s own pipe transport. The bug occurs when we want to emit multiple small events from a single event :

class MessageEmitter extends EventEmitter {
  constructor() {
    super();
    // every 'bigmessage' event arrives in its own task
    somethingElse.on('bigmessage', this._bigMessage.bind(this));
  }

  _bigMessage(bigmessage) {
    for (const message of bigMessage.split('\n'))
      this.emit('message', message); // microtasks will not run between messages
  }
}

In Node >=11, this can be fixed by simply wrapping your emits in setImmediate to give them their own tasks:

class MessageEmitter extends EventEmitter {
  constructor() {
    super();
    // every 'bigmessage' event arrives in its own task
    somethingElse.on('bigmessage', this._bigMessage.bind(this));
  }

  _bigMessage(bigmessage) {
    for (const message of bigMessage.split('\n'))
      setImmediate(() => this.emit('message', message)); // microtasks will successfully run between messages
  }
}

However in Node 10 and below, Node won’t run microtasks between your setImmediates. So you need to call setImmediate from setImmediate.

let spinning = false;
const callbacks = [];
const loop = () => {
  const callback = callbacks.shift();
  if (!callback) {
    spinning = false;
    return;
  }
  setImmediate(loop);
  callback();
};

function waitForNextTask(callback) {
  callbacks.push(callback);
  if (!spinning) {
    spinning = true;
    setImmediate(loop);
  }
};

class MessageEmitter extends EventEmitter {
  constructor() {
    super();
    // every 'bigmessage' event arrives in its own task
    somethingElse.on('bigmessage', this._bigMessage.bind(this));
  }

  _bigMessage(bigmessage) {
    for (const message of bigMessage.split('\n'))
      waitForNextTask(() => this.emit('message', message)); // microtasks will successfully run between messages
  }
}

In the browser, there is no setImmediate. You can use setTimeout(callback, 0) as a replacement. But there is a somehwat large performance impact, as the delay is often capped to 4ms.

Why Care?

When users think an event will be dispatched in its own task, but it is not, very subtle bugs can happen. Consider a Playwright user that wants to turn an event into a promise:

function waitForConsoleMessage(page) {
  return new Promise(resolve => page.once('console', resolve));
}

Will this loop recieve all of the messages?

while (true) {
  const message = await waitForConsoleMessage();
  console.log(message);
}

If every message is in its own task, yes. If not, no.

Consider another user library user who wants to return an active websocket connection:

async function connectToWebsocketWithBackups(...urls) {
  for (const url of url) {
    const websocket = new WebSocket(url);
    await Promise.all([
      new Promise(resolve => websocket.onopen = resolve),
      new Promise(resolve => websocket.onerror = resolve),
    ]);
    if (websocket.readyState === websocket.OPEN)
      return websocket;
  }
  throw new Error('Could not connect to any websocket server');
}

const websocket = await connectToWebsocketWithBackups('wss://foo.example.com', 'wss://bar.example.com', 'wss://baz.example.com');
websocket.addEventListener('message', event => {
  console.log(event)
});

Will this websocket miss any messages? If the open event happens in the same task as the first message, it will miss that messasge. But if every event gets its own task, then it will not miss any messages.

Even though I like this code, I think it is reasonable to consider it brittle for relying on events not being batched into a task. But knowingly or not lots of people write code that relies on events being dispatched in separate tasks. The best way not to break them is to dispatch events in separate tasks.