Upcoming Node.js Features Improve Experience for CLI App Authors

by Christopher Hiller

Node.js will ship a few features in the current release line (v10.x; see release schedule) which may be of interest to those writing command-line applications. Let's take a closer look-see.

fs.readdir() optionally outputs file types

fs.readdir() is simple; it outputs a list of filenames.  Of course, those filenames may represent directories, symbolic links, sockets, devices, hot dogs, etc.  

If you want to know, say, which of those filenames represent directories, you will have to call fs.stat().  This is ultimately kind of silly, because the underlying method in libuv actually provides this information; Node.js simply discards it.

Silliness is forbidden, so Bryan English created nodejs/node#22020 to address this malfeasance.  fs.readdir(), fs.readdirSync(), and fs.promises.readdir() (experimental) will now accept a new option, withFileTypes.  Use it like this:

// Reads contents of directory `/some/dir`, providing an `Array` of
// `fs.Dirent` objects (`entries`)
fs.readdir('/some/dir', { withFileTypes: true }, (err, entries) => {
  if (err) throw err;
  entries.filter((entry) => entry.isDirectory())
    .forEach((entry) => {
      console.log(`${entry.name} is a directory`);
    });
});

// or
const entries = await fs.promises.readdir('/some/dir', {
  withFileTypes: true
});

// or
const entries = fs.readdirSync('/some/dir', { withFileTypes: true });

Above, entries is an array of fs.Dirent objects, which are similar to fs.Stats objects.  These objects contain the same methods as fs.Stats objects, but no other properties except name.  You cannot get information about file’s size from an fs.Dirent object, for example.

The withFileTypes feature provides a more performant and convenient API to work with if you need file type information after reading a directory.  Since it’s optional behavior, it introduces no breaking changes.

fs.mkdir() optionally creates directories recursively

This is mkdir -p.  Over the years of Node.js’ existence, the mkdirp module and its ilk have courageously provided this functionality from userland.  In fact, mkdirp has become ubiquitous, leading some to wonder why it’s not part of the core API.  This author wondered that as well!

The short answer is that Node.js has a “small core” philosophy.  Whether you agree with that philosophy or not, we can all agree mkdirp is a wildly popular module which provides a common filesystem operation.  It’s proven its necessity, and that’s why Node.js merged the Benjamin Coe-created PR nodejs/node#21875.

fs.mkdir(), fs.mkdirSync and fs.promises.mkdir() will now support the recursive option.  Use it like this:

// Creates /tmp/a/apple, regardless of whether `/tmp` 
// and /tmp/a exist.
fs.mkdir('/tmp/a/apple', { recursive: true }, (err) => {
  if (err) throw err;
});

// or
await fs.promises.mkdir('/tmp/a/apple', { recursive: true });

// or
fs.mkdirSync('/tmp/a/apple', { recursive: true });

Note: A recursive fs.mkdir() is not an atomic operation.  It’s…unlikely…to fail halfway through, but it could.

Another Note: There's an open PR concerning how to support feature detection here.

process.allowedNodeEnvironmentFlags: Queryable & Iterable Flags

NODE_OPTIONS is an environment variable supported as of Node.js v8.0.0. If present, it works just like flags passed to the node executable.  The flags allowed in NODE_OPTIONS have a unique property: they do not fundamentally alter the default behavior of the node executable.  What does that mean, exactly?

Flags which don’t fundamentally alter node’s behavior retain two properties:

  • Executing node --some-flag will open a REPL
  • Executing  node --some-flag file.js will run file.js
A handful of flags are excluded from NODE_OPTIONS, such as --preserve-symlinks, due to security concerns.

This means flags like --help, --version, --check, etc., aren’t supported by NODE_OPTIONS.

Now, if you’re using node with flags in production, you’re probably using flags supported by NODE_OPTIONS.  And you probably have some tests.  Assuming you do have tests, if you’re using a test runner which wraps the node executable (such as Mocha), you will need to pass those same flags to the test runner’s executable.  For example:

# production app
$ node --experimental-modules ./app.js
# test runner
$ mocha --experimental-modules "test/**/*.spec.js"

That’s a nice user experience, and it’s fine and good as long as the test runner supports the flags you want to use.  But it also means  the test runner must add support for any given flag in NODE_OPTIONS and pass it along to Node.js.  This is more manual labor to maintain than you’d expect; many flags are considered “temporary,” and only some flags support swapping any dash (-) for an underscore (_) or vice-versa.

This author created PR nodejs/node#19335 which adds   process.allowedNodeEnvironmentFlags. Using this, test runners and other CLI apps needing to wrap the node executable won’t need to manually track new flags as they are added to (and removed from) Node.js.

To detect whether or not a flag as provided in process.argv is a NODE_OPTIONS flag—and thus a useful one, for our purposes—we can do this:

// cli.js
const command = ['cli2']; 
process.argv.slice(2).forEach((arg) => {
  if (process.allowedNodeEnvironmentFlags.has(arg)) {
    command.unshift(arg);
  } else {
    command.push(arg);
  }
});
command.unshift(process.execPath);
// `command` looks like:
// ['node', '--node-flag', 'cli2', '--cli2-flag']

process.allowedNodeEnvironmentFlags is a Set-like object.  It can't be mutated—add(), delete() and clear() operations will silently fail—and its has() method will return true for any allowed permutation of a NODE_OPTIONS flag.  For example:

process.allowedNodeEnvironmentFlags.has('--stack-trace-limit') // true

process.allowedNodeEnvironmentFlags.has('--stack_trace_limit') // true

process.allowedNodeEnvironmentFlags.has('--stack_trace-limit') // true

has() will also return true for a disallowed (but convenient) permutation: omitted leading dashes.  This means:

process.allowedNodeEnvironmentFlags.has('--experimental-modules') // true

process.allowedNodeEnvironmentFlags.has('experimental-modules') // true

// careful with your underscores!
process.allowedNodeEnvironmentFlags.has('--experimental_modules') // false

Since it’s a Set-like object, you can iterate over it using forEach and other typical methods:

process.allowedNodeEnvironmentFlags.forEach((flag) => {
  // --enable-fips
  // --experimental-modules
  // --experimental-repl-await
  // etc.
});

Only the canonical format (as shown in node --help) of each flag will appear when iterated over; it won’t contain any duplicates.

This addition enables tooling authors to more easily wrap the node executable, and provides runtime insight into the nature of flags in process.argv.

Horn-Tooting

If you’re interested in helping make Node.js a better experience for authors of command-line tools, click on these links, dammit: