Web Streams

The WHATWG Streams Standard (nodejsdp.link/web-streams) provides a standardized API for working with streaming data, known as “Web Streams.” While inspired by Node.js streams, it has its own distinct implementation and is designed to be a universal standard for the broader JavaScript ecosystem, including browsers.

About a decade after the initial development of Node.js streams, Web Streams emerged to address the lack of a native streaming API in browser environments, something that made it difficult to efficiently work with large datasets on the frontend.

Today, most modern browsers support the Web Streams standard natively, making it an ideal choice for building streaming pipelines within the browser. In contrast, Node.js streams are not natively available in browsers. You could bring Node.js streams to the browser by installing them as a library in your project, but their utility is limited since native APIs like fetch use Web Streams to send requests or read responses incrementally. Given this, using Web Streams in the browser is the recommended choice.

Web Streams have also been implemented in Node.js, effectively giving us two competing APIs to deal with streaming data. However, at the time of writing, Web Streams is still relatively new and hasn’t yet reached the same level of adoption as native Node.js streams within the large Node.js ecosystem. That’s why this chapter focused mainly on Node.js streams, but understanding Web Streams is still an important piece of knowledge, and we expect it to become more relevant in the coming years.

Fortunately, getting started with Web Streams should be easy if you have been following this chapter. Most of the concepts are aligned, and the primary differences lie in function names and arguments, which is something that can be easily learned by checking the Web Streams API documentation.

One aspect that is worth exploring here is the interoperability between Node.js and Web Streams. Fortunately, it’s possible to convert Node.js stream objects to equivalent Web Stream objects and vice versa. This makes it easy to transition or work with third-party libraries that use Web Streams in the context of Node.js.

Let’s briefly discuss how this interoperability works.

In the Web Streams standard, we have 3 primary types of objects:

• ReadableStream: Source of streaming data and pretty much equivalent to a Readable Node.js stream.
• WritableStream: Destination for streaming data; equivalent to a Node.js Writable stream.
• TransformStream: Allows you to transform streaming data in a streaming pipeline. Equivalent to a Node.js Transform stream.

Note how these concepts match almost perfectly. Also note how, thanks to the Stream suffix of the Web Streams classes, we don’t have naming conflicts between equivalent streaming abstractions.

Converting Node.js streams to Web Streams

You can easily convert Node.js streams to equivalent Web Streams objects by using the .toWeb(sourceNodejsStream) method available respectively in the Readable, Writable, and Transform classes.

Let’s see what the syntax looks like:

import { Readable, Writable, Transform } from 'node:stream'
const nodeReadable = new Readable({/*...*/}) // Readable
const webReadable = Readable.toWeb(nodeReadable) // ReadbleStream
const nodeWritable = new Writable({/*...*/}) // Writable
const webWritable = Writable.toWeb(nodeWritable) // WritableStream
const nodeTransform = new Transform({/*...*/}) // Transform
const webTransform = Transform.toWeb(nodeTransform) // TransformStream

Converting Web Streams to Node.js streams

The Readable, Writable, and Transform classes also expose methods to convert a Web Stream to an equivalent Node.js stream. These methods, unsurprisingly, have the following signature: .fromWeb(sourceWebStream).

Let’s see a quick example to clarify the syntax:

import { Readable, Writable, Transform } from 'node:stream'
import {
  ReadableStream,
  WritableStream,
  TransformStream,
} from 'node:stream/web'
const webReadable = new ReadableStream({/*...*/}) // ReadableStream
const nodeReadable = Readable.fromWeb(webReadable) // Readable
const webWritable = new WritableStream({/*...*/}) // WritableStream
const nodeWritable = Writable.fromWeb(webWritable) // Writable
const webTransform = new TransformStream({/*...*/}) // TransformStream
const nodeTransform = Transform.fromWeb(webTransform) // Transform

The last two snippets illustrate how easy it is to convert stream types between Node.js streams and Web Streams.

One important detail to keep in mind is that these conversions don’t destroy the source stream but rather wrap it in a new object that is compliant with the target API. For example, when we convert a Node.js Readable stream to a web ReadableStream, we can still read from the source stream while also reading from the new Web Stream. The following example should help to clarify this idea:

import { Readable } from 'node:stream'
const nodeReadable = new Readable({
  read() {
    this.push('Hello, ')
    this.push('world!')
    this.push(null)
  },
})
const webReadable = Readable.toWeb(nodeReadable)
nodeReadable.pipe(process.stdout)
webReadable.pipeTo(Writable.toWeb(process.stdout))

In the preceding example, we are defining a Node.js stream that emits the string “Hello, world!” in 2 chunks before completing. We convert this stream into an equivalent Web Stream, then we pipe both the source Node.js stream and the newly created Web Stream to standard output.

This code will produce the following output:

Hello, Hello, world!world!

This is because, every time that the source Node.js stream emits a chunk, the same chunk is also emitted by the associated Web Stream.

The .fromWeb() and .toWeb() methods are implementations of the Adapter pattern that we will discuss in more detail in Chapter 8, Structural Design Patterns.