Streaming promises in the order they resolve

I have recently been obsessed with the idea of HTML streaming, in particular out-of-order streaming after reading this article on how out-of-order streaming can be done with declarative shadow DOM. I’ve been working on an experimental templating language/framework/monstrosity in my free time called Tempeh. I would consider the project’s current state to be extremely rough and not ready for public consumption, so click that link at your own risk.

However, I wanted to share a cool thing I put together while exploring out-of-order HTML streaming possibilities.

Click here to skip right to the code.

Context

So, let’s say I have 3 pieces of lazy content on a page. I have an html template string function which will unwrap promises and stream the html string contents in order as they resolve. An unfortunate limitation of this implementation is that it’s order-dependent; the async lazy content at the end of the template literal can only be streamed as fast as the first promise resolves.

return html`
  <div la-z-id="vCz0LY_0">
    Loading...
  </div>
  <div la-z-id="vCz0LY_1">
    Loading...
  </div>
  <div la-z-id="vCz0LY_2">
    Loading...
  </div>
  ${waitFor(500).then(()=>html`
    <la-z id="vCz0LY_0" hidden>
      I'm first, but I'm the slowest!
    </la-z>
  `)}
  ${waitFor(300).then(()=>html`
    <la-z id="vCz0LY_1" hidden>
      I'm second, and I'm in the middle.
    </la-z>
  `)}
  ${waitFor(100).then(()=>html`
    <la-z id="vCz0LY_2" hidden>
      I'm third, but I'm the fastest!
    </la-z>
  `)}
`.arrayBufferStream();

So, we can use streaming to accomplish a cool effect where the static placeholder content can load in immediately, and then we stream in the lazy content as it’s ready. Each piece of lazy content is wrapped in a <la-z> web component which, when it mounts on the page, will find its corresponding placeholder element and swap in the final lazy content.

In the scenario above, the first piece of lazy content is the slowest, so it will act as a bottleneck and prevent the other pieces of content from streaming sooner even if their promises have already resolved first. I want a way to stream the resolved values from an array of promises as they resolve, in the order that they resolve! Unfortunately there are not built-in friendly ways to accomplish this; the main methods available for dealing with a group of promises are either Promise.all/Promise.allSettled, which force you to wait for all promises to resolve together and preserves their order, or Promise.race/Promise.any, which can get the value of the first promise in the group which resolves, but nothing aside from that.

So, it’s time to build something ourselves! My html template string function can also unwrap iterators, so I decided that should probably be the move.

The implementation

Alright, here’s the code:

/**
 * Takes an array of promises and yields the resolved values in the order they resolve.
 * @template TResolveValue
 * @param {Promise<TResolveValue>[]} promises
 */
export async function* unorderedPromiseIterator(promises) {
  const resultStream = new ReadableStream({
    async pull(controller) {
      /** @param {TResolveValue} value */
      const onResolve = (value) => {
        controller.enqueue(value);
      };

      /** @param {unknown} err */
      const onError = (err) => {
        controller.enqueue(
          new Error("unorderedPromiseIterator: An unexpected error occurred.", {
            cause: err,
          })
        );
      };

      for (const promise of promises) {
        promise.then(onResolve, onError);
      }

      await Promise.allSettled(promises);
      controller.close();
    },
  });

  const reader = resultStream.getReader();

  try {
    /** @type {Awaited<ReturnType<typeof reader.read>>} */
    let readResult;
    while (!(readResult = await reader.read()).done) {
      yield readResult.value;
    }
  } finally {
    reader.releaseLock();
  }
}

Excuse the JSDoc annotations, I’m experimenting with being able to use TypeScript without a build step. Although it’s a touch clunky, I like it overall!

So obviously there is some real deep cut neo-JavaScript stuff in here. The ReadableStream API and generator functions aren’t something that you see super frequently day to day, although they’ve been around long enough to use comfortably both in Node and the browser. But both are extremely powerful and very fun to work with!

As it turns out, building something like this is pretty hard because generator functions only allow you to yield within the body of the generator function; this means that yielding from a promise.then callback is a no-go, and any obvious implementation ends up falling back into the same trap of having to wait for each promise to resolve in order.

On my first attempt at this, I went through a couple other iterations where I was tracking the index of each unresolved promise, waiting for any of them to resolve with Promise.race, removing that resolved promise from the array, adjusting all of the tracked indices, and then repeating that process until all promises where settled. The code was pretty gnarly and much more inefficient than I think something like this should be, so I kept searching for something better. I finally realized the ReadableStream API was exactly what I was looking for.

ReadableStreams provide us with a way to set up a custom stream which we can fully control the underlying implementation of. This makes it easy for us to simply pass a method to each promise’s .then callback which enqueues the resolved value, and then we can close the stream one all promises have settled. Now we can yield each value in a generator-friendly way as they roll in from the stream’s reader.

Pretty cool! Here’s how it looks in action:

const promises = [
  waitFor(300).then(()=>"first (middle)"),
  waitFor(500).then(()=>"second (slowest)"),
  waitFor(100).then(()=>"third (fastest)"),
];

for await (const value of unorderedPromiseIterator(promises)) {
  console.log(value);
}

// console log:
// "third (fastest)"
// "first (middle)"
// "second (slowest)"

This can then be applied to the html template string example I mentioned above like this:

return html`
  <div la-z-id="vCz0LY_0">
    Loading...
  </div>
  <div la-z-id="vCz0LY_1">
    Loading...
  </div>
  <div la-z-id="vCz0LY_2">
    Loading...
  </div>
  ${unorderedPromiseIterator([
    waitFor(500).then(()=>html`
      <la-z id="vCz0LY_0" hidden>
        I'm first, but I'm the slowest!
      </la-z>
    `),
    waitFor(300).then(()=>html`
      <la-z id="vCz0LY_1" hidden>
        I'm second, and I'm in the middle.
      </la-z>
    `),
    waitFor(100).then(()=>html`
      <la-z id="vCz0LY_2" hidden>
        I'm third, but I'm the fastest!
      </la-z>
    `),
  ])}
`.arrayBufferStream();