Readable and writable streams with Node's core stream module

If we wanted our own readable stream, we would need the stream.Readable base constructor.

This base class will call a special method called _read. It's up to us to implement the _read method. Since Node 4, we can also supply a read property to an options object, which will be the supplied function to be added as the _read method of the returned instance.

Whenever this method is called, the stream expects us to provide more data available that can be consumed by the stream. We can add data to the stream by calling the push method with a new chunk of data.

Using readable-stream instead of stream
To allow universal behavior across Node modules, if we ever use the core stream module to create streams, we should actually use the readable-stream module available on npm. This an up-to-date and multi-version compatible representation of the core streams' module and ensures consistency.

Let's create a folder called core-streams, initialize it as a package, install readable-stream, and create an index.js file inside:

$ mkdir core-streams 
$ cd core-streams 
$ npm init -y 
$ npm install readable-stream 
$ touch index.js 

At the top of index.js, we write:

const { Readable, Writable } = require('readable-stream') 
 
const rs = Readable({ 
  read: () => { 
    rs.push(Buffer.from('Hello, World!')) 
    rs.push(null)  
  } 
}) 

Each call to push sends data through the stream. When we pass null to push, we're informing the stream.Readable interface that there is no more data available.

The use of the read option (instead of attaching a _read method) is only appropriate for scenarios where our code is expected to be used by Node 4 and above (the same goes for the use of destructing context and fat arrow lambda functions).

To create a writable stream, we need the stream.Writable base class. When data is written to the stream, the writable base class will buffer the data internally and call the _write method that it expects us to implement. Likewise, from Node 4, we can use the write option for a terser syntax. Again this approach isn't appropriate for modules which are intended to be made publicly available, since it doesn't cater to legacy Node users (Node 0.10 or 0.12).

Now let's add the following to the bottom of our index.js file:

const ws = Writable({ 
  write: (data, enc, cb) => { 
    console.log(`Data written: ${data.toString()}`) 
    cb() 
  } 
}) 

We can either write data to the stream manually using the write method or we can pipe (or rather pump) a readable stream to it.

If we want to move the data from a readable to a writable stream, the pipe method available on readable streams is a much more elegant solution than using the data event on the readable stream and calling write on the writable stream (but remember we should use pump in production).

Let's add this final line to our index.js file:

rs.pipe(ws) 

Now we can run our program:

$ node index.js 

This should print out Data written: Hello, World!.