An Overview of the Node Package Manager

Exclusive offer: get 50% off this eBook here
Node Web Development

Node Web Development — Save 50%

A practical introduction to Node, the exciting new server-side JavaScript web development stack

€13.99    €7.00
by David Herron | August 2011 | Open Source Web Development

NPM is a package management and distribution system for Node. It has become the de-facto standard for distributing modules (packages) for use with Node. Conceptually it's similar to tools like apt-get (Debian), rpm/yum (Redhat/Fedora), MacPorts (Mac OS X), CPAN (Perl), or PEAR (PHP). It's purpose is publishing and distributing Node packages over the Internet using a simple command-line interface. With npm you can quickly find packages to serve specific purposes, download them, install them, and manage packages you've already installed. npm defines a package format for Node largely based on the CommonJS Package spec.

In this article by David Herron, author of Node Web Development, we will take a look at the npm package management system.

 

Node Web Development

Node Web Development

A practical introduction to Node, the exciting new server-side JavaScript web development stack

        Read more about this book      

(For more resources on Web Development, see here.)

 

npm package format

An npm package is a directory structure with a package.json file describing the package. This is exactly what we just referred to as a Complex Module, except npm recognizes many more package.json tags than does Node. The starting point for npm's package.json is the CommonJS Packages/1.0 specification. The documentation for npm's package.json implementation is accessed with the following command:

$ npm help json

A basic package.json file is as follows:

{ name: "packageName",
   version: "1.0",
   main: "mainModuleName",
   modules: {
      "mod1": "lib/mod1",
      "mod2": "lib/mod2"
   }
}

The file is in JSON format which, as a JavaScript programmer, you should already have seen a few hundred times.

The most important tags are name and version. The name will appear in URLs and command names, so choose one that's safe for both. If you desire to publish a package in the public npm repository it's helpful to check and see if a particular name is already being used, at http://search.npmjs.org or with the following command:

$ npm search packageName

The main tag is treated the same as complex modules. It references the module that will be returned when invoking require('packageName'). Packages can contain many modules within themselves, and those can be listed in the modules list.

Packages can be bundled as tar-gzip tarballs, especially to send them over the Internet.

A package can declare dependencies on other packages. That way npm can automatically install other modules required by the module being installed. Dependencies are declared as follows:

"dependencies":
  { "foo" : "1.0.0 - 2.9999.9999"
      , "bar" : ">=1.0.2 <2.1.2"
  }

The description and keywords fields help people to find the package when searching in an npm repository (http://search.npmjs.org). Ownership of a package can be documented in the homepage, author, or contributors fields:

"description": "My wonderful packages walks dogs",
  "homepage": "http://npm.dogs.org/dogwalker/",
  "author": dogwhisperer@dogs.org

Some npm packages provide executable programs meant to be in the user's PATH. These are declared using the bin tag. It's a map of command names to the script which implements that command. The command scripts are installed into the directory containing the node executable using the name given.

bin: {
  'nodeload.js': './nodeload.js',
     'nl.js': './nl.js'
},

The directories tag documents the package directory structure. The lib directory is automatically scanned for modules to load. There are other directory tags for binaries, manuals, and documentation.

directories: { lib: './lib', bin: './bin' },

The script tags are script commands run at various events in the lifecycle of the package. These events include install, activate, uninstall, update, and more. For more information about script commands, use the following command:

$ npm help scripts

This was only a taste of the npm package format; see the documentation (npm help json) for more.

 

Finding npm packages

By default npm modules are retrieved over the Internet from the public package registry maintained on http://npmjs.org. If you know the module name it can be installed simply by typing the following:

$ npm install moduleName

But what if you don't know the module name? How do you discover the interesting modules?

The website http://npmjs.org publishes an index of the modules in that registry, and the http://search.npmjs.org site lets you search that index.

npm also has a command-line search function to consult the same index:

$ npm search mp3
mediatags Tools extracting for media meta-data tags =coolaj86 util
m4a aac mp3 id3 jpeg exiv xmp
node3p An Amazon MP3 downloader for NodeJS. =ncb000gt

Of course upon finding a module it's installed as follows:

$ npm install mediatags

After installing a module one may want to see the documentation, which would be on the module's website. The homepage tag in the package.json lists that URL. The easiest way to look at the package.json file is with the npm view command, as follows:

$ npm view zombie
...
{ name: 'zombie',
description: 'Insanely fast, full-stack, headless browser testing
using Node.js',
...
   version: '0.9.4',
   homepage: 'http://zombie.labnotes.org/',
...
npm ok

You can use npm view to extract any tag from package.json, like the following which lets you view just the homepage tag:

$ npm view zombie homepage
http://zombie.labnotes.org/

 

Using the npm commands

The main npm command has a long list of sub-commands for specific package management operations. These cover every aspect of the lifecycle of publishing packages (as a package author), and downloading, using, or removing packages (as an npm consumer).

Node Web Development

Getting help with npm

Perhaps the most important thing is to learn where to turn to get help. The main help is delivered along with the npm command accessed as follows:

Node Web Development

For most of the commands you can access the help text for that command by typing the following:

$ npm help <command>

The npm website (http://npmjs.org/) has a FAQ that is also delivered with the npm software. Perhaps the most important question (and answer) is: Why does npm hate me? npm is not capable of hatred. It loves everyone, even you.

 

Node Web Development A practical introduction to Node, the exciting new server-side JavaScript web development stack
Published: August 2011
eBook Price: €13.99
Book Price: €22.99
See more
Select your format and quantity:

 

        Read more about this book      

(For more resources on Web Development, see here.)

 

Viewing package information

The npm view command treats the package.json file as data, letting you query that data using a dot-notation for JSON tags such as viewing the package dependencies:

$ npm view google-openid dependencies
{ express: '>= 0.0.1',
   openid: '>= 0.1.1 <= 0.1.1' }

The package.json file can include the package repository URL. Therefore, if you just want to retrieve the package source, use the following:

$ npm view openid repository.url
git://github.com/havard/node-openid.git
$ git clone git://github.com/havard/node-openid.git
Cloning into node-openid...
remote: Counting objects: 253, done.
remote: Compressing objects: 100% (253/253), done.
remote: Total 253 (delta 148), reused 0 (delta 0)
Receiving objects: 100% (253/253), 63.29 KiB, done.
Resolving deltas: 100% (148/148), done.

What version of Node is required for a package?

$ npm view openid engines
node >= 0.4.1

Installing an npm package

The npm install command makes it easy to install packages upon finding the one of your dreams as follows:

$ npm install openid
openid@0.1.6 ./node_modules/openid
$ ls node_modules/
openid

Notice that the package is installed in a local node_modules directory. Packages can be installed in other locations either by changing the current directory, or by telling npm to make a global install.For example, the following will set up a directory, /var/www, where /var/www/node_modules stores modules to be shared among several websites:

$ cd /var/www
$ npm install openid
openid@0.1.6 ./node_modules/openid

npm makes a distinction between global mode and local mode. Normally it operates in local mode and installs packages into a local node_modules directory next to your application code. In global mode packages are installed globally, meaning that they're installed into the Node installation (directories in require.paths) rather than a local node_modules directory.

The first method to install packages in global mode is to use the -g flag as follows:

$ npm install -g openid
openid@0.1.6 /usr/local/node/0.4.7/lib/node_modules/openid
$ which node
/usr/local/node/0.4.7/bin/node

The installation directory in global mode is based on where Node is installed for you.

The second method for global mode installation is to change npm configuration settings. There are many configuration settings, which we'll discuss in some time, and the relevant one for now is as follows:

$ npm set global=true
$ npm get global
true
$ npm install openid
openid@0.1.6 /usr/local/node/0.4.7/lib/node_modules/openid

To learn about all the folders npm uses enter the following command:

$ npm help folders

Using installed packages

The point of installing a package is to enable a Node program to access the module like the following:

var openid = require('openid');

What npm does is to help you set up conditions for this to work smoothly.

Some packages include inner modules that could themselves be useful to other software. For example, the current version of this openid module we've been picking on includes a base64 encode/decode module that could be useful for other software:

var base64 = require('openid/lib/base64').base64;

This runs a risk the openid module could change its implementation of its base64 encode/decode function, breaking your application. Some packages structured themselves to provide a family of related sub-modules accessed this way, and provide some guarantee of a stable API for the exposed sub-modules.

What packages are currently installed?

The npm list command lists the installed packages, based on a search from the current directory. Remember that Node searches for modules starting at the current directory of the code being executed. Therefore, the installed packages are relatively based on your current directory, depending on the content of node_modules directories above the current directory.

For example, notice how the listed modules changes based on which directory you are in:

Node Web Development

By default the list is shown in a tree structure, that isn't terribly useful as data to other commands as shown in the previous screenshot. The parseable configuration setting can make the output usable as data as follows:

$ npm set parseable=true
$ npm list
/home/david/Node/chap06
/home/david/Node/chap06/node_modules/ejs
/home/david/Node/chap06/node_modules/express
/home/david/Node/chap06/node_modules/express/node_modules/connect
/home/david/Node/chap06/node_modules/express/node_modules/mime
/home/david/Node/chap06/node_modules/express/node_modules/qs
/home/david/Node/chap06/node_modules/mongodb
/home/david/Node/chap06/node_modules/mongoose
/home/david/Node/chap06/node_modules/sqlite3

Package scripts

npm allows for package scripts to automatically run at various times in the package lifecycle. Currently there are four lifecycle events: test, start, stop, and restart.

An npm package can include tests which are run as follows:

$ npm test <packageName>

The start, stop, and restart lifecycle events don't have a defined meaning. An obvious use is starting or stopping daemon processes associated with the package.

Editing and exploring installed package content

npm includes a pair of commands to let you look at or change package content. For example, you could use this during development to read the package source (say, to understand what it's doing), look in the package examples directory, or make modifications to test patches.

For example:

Node Web Development

As the command output implies, the explore command spawns a sub-shell whose current directory is the location where the module is installed. Typing exit or control-D ends the sub-shell returning you to your login shell.

You can edit files while browsing the package, if you like. If you do, the package may need to be rebuilt as follows:

$ npm rebuild mongoose
mongoose@1.3.3 /home/david/Node/chap06/node_modules/mongoose

 

Node Web Development A practical introduction to Node, the exciting new server-side JavaScript web development stack
Published: August 2011
eBook Price: €13.99
Book Price: €22.99
See more
Select your format and quantity:

 

        Read more about this book      

(For more resources on Web Development, see here.)

 

Updating outdated packages you've installed

The coder codes, updating their package, leaving you in their dust unless you keep up.

To find out if your installed packages are out of date use the following command:

$ npm outdated
express@2.3.6 ./node_modules/express current=2.3.3
mongoose@1.3.6 ./node_modules/mongoose current=1.3.3

This shows the current installed version as well as the current version in the npm repository. Updating the outdated packages is very simple:

$ npm update express
connect@1.4.1 ./node_modules/express/node_modules/connect
mime@1.2.2 ./node_modules/express/node_modules/mime
qs@0.1.0 ./node_modules/express/node_modules/qs
express@2.3.6 ./node_modules/express

Uninstalling an installed npm package

It may come to pass that the package of your dreams turns into a nightmare, and even if it does not there are plenty of reasons to remove installed packages. This can be done as follows:

$ npm list
/home/david/Node
   openid@0.1.6
$ npm uninstall openid
$ npm list
/home/david/Node
(empty)

Developing and publishing npm packages

Now that we have a good idea of how to use npm let's get to the other end of the process and look at how to develop npm packages. Some of the npm commands serve the development process.

The first step is creating the package.json file, and the npm init command helps you create the initial version. It interrogates you with a few questions and quickly helps you create something like the following:

{
  "author": "I.M. Awesome <awesome@example.com>",
  "name": "tmod",
  "description": "Test Module",
  "version": "0.0.1",
  "repository": {
      "url": ""
  },
  "engines": {
      "node": ">0.4.1"
  },
  "dependencies": {},
  "devDependencies": {}
}

The next step is obviously to create the package source. npm doesn't have any way to help you with this. You are the coder so you do the coding. Just make sure to keep updating the package.json file as you add things to the package. npm does have a couple of commands you'll be using while developing the package.

One of these commands is npm link, a lighter-weight method of installing packages. The difference between this and npm install is that npm link simply sets up a symbolic link to your source directory, and you can freely edit package files without having to repackage and update the package on every change. You can iteratively work on the package, and test it, without having to continually rebuild.

Using npm link is a two step process, where first you link your project into the Node installation as follows

$ cd tmod
$ npm link
../../0.4.7/lib/node_modules/tmod -> /home/david/Node/chap03/tmod

In the second step you link that package into your application:

$ npm link tmod
../node_modules/tmod -> /home/david/Node/0.4.7/lib/node_modules/
tmod -> /home/david/Node/chap03/tmod

The arrows (->) show you the symbolic link chain that's set up by these commands.

The npm install command has a couple of modes that are useful during development. The first is that, if executed in the root of a package directory, it installs the current directory and dependencies into the local node_modules directory.

The second is to install tarball's either from a local file or over the network from a URL. Most source code control systems support a URL providing a tarball (compressed tar file) of the source tree. For example, the downloads page on github projects gives a URL like this one:

$ npm install https://github.com/havard/node-openid/tarball
openid@0.1.6 ../node_modules/openid

When you're satisfied that your package works, you might want to publish it in the public npm repository so others can use it.

The first step is to register an account with the npm repository. It's done by running the npm adduser command, which asks you a series of questions to establish a username, password, and e-mail address:

$ npm adduser
Username: my-user-name
Password:
Email: me@example.com

After this step run the npm publish command in the root directory of your package:

$ npm publish

If all has gone well, after running npm publish, you can go to http://search.npmjs.org and search for the package. It should show up pretty quick.

The npm unpublish command, as the name implies, removes the package from the npm repository.

npm configuration settings

There are a number of other settings to fine-tune npm behavior. Let's first look at the ways to make configuration settings.

First is the npm set and npm get commands, or:

npm config set <key> <value> [--global]
npm config get <key>
npm config delete <key>
npm config list
npm config edit
npm get <key>
npm set <key> <value> [--global]

For example:

$ npm set color true
$ npm set global false
$ npm config get color
true
$ npm config get global
false

Environment variables can be used to set configuration values. Any variables which start with NPM_CONFIG_ are interpreted for configuration values. For example, the variable NPM_CONFIG_GLOBAL will set the global configuration value.

Configuration values can be put into configuration files:

  • $HOME/.npmrc
  • <Node Install Directory>/etc/npmrc

The configuration file contains name=value pairs like the following, and is updated by the npm config set command:

$ cat ~/.npmrc
global = false
color = true

 

Package version strings and ranges

Node doesn't know anything about version numbers. It knows about modules, and can treat a directory structure as if it were a module, and it has a fairly rich system of looking for modules, but it doesn't know version numbers. npm however knows about version numbers. It uses the Semantic Versioning model (see further) and as we've seen you can install modules over the Internet, query for out-of-date modules, and update them with npm. All of this is version controlled, so let's take a closer look at the things npm can do with version numbers and version tags.

Earlier we used npm list to list installed packages, and the listing includes version numbers of installed packages. If instead, you wish to see the version number of a specific module, type the following command:

$ npm view express version
2.4.0

Whenever npm commands take a package name, you can append a version number or version tag to the package name. This way you can deal with specific package versions if needed; for example, if you've tested and qualified your application against a specific version in a staging environment, you can ensure that version is used in the deployment environment:

$ npm install express@2.3.1
mime@1.2.2 ./node_modules/express/node_modules/mime
connect@1.5.1 ./node_modules/express/node_modules/connect
qs@0.2.0 ./node_modules/express/node_modules/qs
express@2.3.1 ./node_modules/express

npm has a "tag" concept that might be used as shown to install the latest stable release of a package:

$ npm install sax@stable

Tag names are arbitrary and are not required. The package author designates the tag names and not all packages will use tag names.

Packages list dependencies to other packages in their package.json, which you can view in this way:

$ npm view mongoose dependencies
{ hooks: '0.1.9' }

$ npm view express dependencies
{ connect: '>= 1.5.1 < 2.0.0',
  mime: '>= 0.0.1',
  qs: '>= 0.0.6' }

The package dependencies is the way npm knows which additional modules to install. When installing a module, it looks at the dependencies and downloads any which are currently not installed.

The sharp-eyed will see the less-than and greater-than signs in this example. npm supports version number ranges, and for example if Express is declaring it will work with any version of Connect between 1.5.1 and 2.0.0.

While this will be straightforward and unsurprising to anybody who has dealt with software at any time, there is a sound model behind the scenes. The npm author used the Semantic Versioning spec at http://semver.org to guide the npm version numbering system. It is as follows:

  • Version strings are normally integers arranged as X.Y.Z; X is the Major version, Y is the Minor version, and Z is the Patch (for example, 1.2.3).
  • The version string can have an arbitrary text appended immediately after the Patch number for what are called "special versions" (for example, 1.2.3beta1).
  • Comparing version strings is not a string comparison, but a numerical comparison of the X, Y, and Z values. For example, 1.9.0 > 1.10.0 > 1.11.3. Further 1.0.0beta1 > 1.0.0beta2 > 1.0.0.
  • Compatibility is documented through a version numbering convention:
    • Packages with major version 0 (X = 0) are completely unstable and can change any API at any time.
    • The Patch number (Z) must be incremented when the only change is backwards-compatible bug fixes.
    • The Minor number (Y) must be incremented when backwardscompatible functionality is introduced (for example, a new function, and all other functions remain compatible).
    • The Major number (X) must be incremented when incompatible changes are made.

Summary

In this article we took a look at the npm package management system


Further resources on this subject:


About the Author :


David Herron

David Herron has worked as a software engineer and software quality engineer in Silicon Valley for over 20 years. Recently he worked for Yahoo! as an architect of the Quality Engineering team for their new Node.js based web app platform (Manhattan and Mojito).

While a staff engineer at Sun Microsystems, David worked as an architect of the Java SE Quality Engineering team where he focused on test automation tools, including the AWT Robot class that's now widely used in GUI test automation software. He was involved with launching the OpenJDK project, and other open source activities related to Java.

Before Sun he worked for VXtreme on the video streaming stack which eventually became Windows Media Player when Microsoft bought that company. At The Wollongong Group he worked on both the e-mail client and server software and was part of several IETF working groups improving e-mail-related protocols.

David is interested in electric vehicles, world energy supplies, climate change, and environmental issues, and is a co-founder of Transition Silicon Valley. As an online journalist, he writes about electric cars and other green technology for PlugInCars.com, TorqueNews.com, Examiner.com, LongTailPipe.com, and ElectricRaceNews.com. He runs a large electric vehicle discussion website on VisForVoltage.org, and blogs about other topics including Node.js, Drupal, and Doctor Who on DavidHerron.com. Using Node.js, he has developed a Content Management System called AkashaCMS (akashacms.com) that produces static HTML websites.

Books From Packt


Agile Web Application Development with Yii1.1 and PHP5
Agile Web Application Development with Yii1.1 and PHP5

Python 3 Web Development Beginner's Guide
Python 3 Web Development Beginner's Guide

YUI 2.8: Learning the Library
YUI 2.8: Learning the Library

Grok 1.0 Web Development
Grok 1.0 Web Development

Kentico CMS 5 Website Development: Beginner's Guide
Kentico CMS 5 Website Development: Beginner's Guide

Building Websites with DotNetNuke 5
Building Websites with DotNetNuke 5

Building Websites with Expression Engine 2
Building Websites with Expression Engine 2

MODx Web Development - Second Edition
MODx Web Development - Second Edition


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software