Using Node.js To Create Real-Time Web Applications

Update: 02/01/17: Much has changed since the early days of node.js when this post was written. The Now.js library featured below has long been abandoned and should not be confused with the Now client used for deployments. The functionality Now.js provided can now be achieved directly via socket.io.

In case you haven't heard, Node.js is the hottest new technology that is revolutionizing the web. Some of you don't seem convinced. After all, Ruby on Rails failed to achieve global domination, so why should server-side JavaScript be any different? The biggest distinction is that Node.js is not a server-side language; it is the server. Secondly, it's an incredibly fast and efficient server that scales well. If you've configured Apache's httpd.conf file, you've no doubt seen the StartServers and MinSpareServers settings which keep a specified number of idle Apache servers running in order to bypass the time required to start servers for new connections. In contrast, Nodejs.org explains:

Node tells the operating system (through epoll, kqueue, /dev/poll, or select) that it should be notified when a new connection is made, and then it goes to sleep. If someone new connects, then it executes the callback. Each connection is only a small heap allocation.

The Node.js + Socket.io + Now.js Formula

A faster, more responsive web server is only one building block to creating real-time web applications. Next we need to examine how the client browser and web server communicate, a.k.a. the transport layer. Traditionally, all communications are initiated by the client browser and the web server responds to requests. So, for example, in a traditional "real-time" web application, the client browser sends a constant stream of requests to the web server asking for updates (a process known as "polling"). If an update is available, the server responds and sends the new information to the client. The application is only "near" real-time because it is dependent on the frequency of requests and as the number of requests grow, the web server needs to greatly increase resources to handle them. Socket.io is a Node.js package that provides WebSockets functionality, allowing the server to push information to connected clients without the need for polling. Socket.io also provides a number of fallback transport options to achieve the widest range of browser and platform support possible.

The final piece of the stack is Now.js. Now.js provides a shared namespace between the web server and the client browser. This allows servers to directly call functions and access variables on the client and vice versa. A simple chat application in Now.js would involve the client browser calling a server function to send a message to everyone connected. The server function would in turn call a function on all connected clients to display the message.

Installation

Node.js is available as a pre-built binary package for many Linux distributions and Windows. You can also get the source code from Github and compile it by following the installation instructions.

Once Node.js is installed, you can install NPM, the Node Package Manager, by running:

curl http://npmjs.org/install.sh | sh

NPM makes installing Node packages extremely simple. Since Now.js includes a copy of Socket.io, you can install them both by running:

npm install now

Hello World

As mentioned, Node.js is the web server, so your JavaScript files will contain both the content or logic for any dynamic content and the information needed to start the server. Here's an example of a static content server:

var http = require('http');
httpServer = http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
});
httpServer.listen(1337, "127.0.0.1");    
console.log('Server running at http://127.0.0.1:1337/');

For this server, we're first creating an http object. Next we create a server on the HTTP object and set the response object to write an HTTP 200 code with a text content-type that returns "Hello World". The HTTP Server object is set to listen on port 1337 of the IP address 127.0.0.1. Finally, we send a message that the server is running back to the console.

To start this server, we simply run node followed by our JavaScript server file:

node helloWorld.js

Adding Now.js

To include Now.js functionality, we need to create a nowjs object. Then we initialize it, passing our existing HTTP Server object:

var nowjs = require('now');
var everyone = nowjs.initialize(httpServer);

The everyone object becomes the server's gateway to directly interacting with clients. Remember that Now.js provides a shared namespace (aptly called "now") between clients and the server. On the server, the shared now namespace is a property of everyone.

/* declare a function in the now namespace */
everyone.now.serverFunction = function(foo) {
  ...
}
 
/* call a function in the now namespace */
everyone.now.clientFunction(bar);

Clients can call functions that are methods on the now object and can declare new functions/methods to be shared.

/* call a function in the now namespace */
now.serverFunction(foo);
 
/* declare a function in the now namespace */
now.clientFunction = function(bar) {
  ...
}

Creating a Chatroom

The shared namespace concept can become a bit confusing, so let's put everything together to better understand how everything interacts. Here's the client-side code:

<html>
 <head>
    <!-- include jquery -->
    <script src="https://127.0.0.1/jquery/jquery.min.js"></script>
 
    <!-- include nowjs -->
    <script src="http://127.0.0.1:1337/nowjs/now.js"></script>
 
    <script>
      /**
       * create a function in the now namespace
       * that the server can call 
       */
      now.receiveMessage = function(message) {
        $('#messages').append('<p>' + message + '</p>');
      }
 
       /**
       * add a click event to call a function in 
       * the server's now namespace 
       */
      $('#send').click( function() {
        now.sendMessage($('#message').val());
      });
    </script>
  </head>
 
   <body>
    <div id="messages"></div>
    <input type="text" id="message"/>
    <input type="button" id="send" value="Send"/>
  </body>
</html>

Here's the Node.js server code:

    /**
     * Setup the http server 
     * The variable clientCode is the code above and
     * can be set in this file or you could read it in from 
     * a file. See the node.js documentation for 
     * filesystem interaction
     */
    var http = require('http');
    httpServer = http.createServer(function (req, res) {
      res.writeHead(200, {'Content-Type': 'text/html'});
      res.end(clientCode);
    });
    httpServer.listen(1337, "127.0.0.1");
 
    /**
     * Add Now.js
     */
    var nowjs = require('now');
    var everyone = nowjs.initialize(httpServer);
 
    /**
     * Receive incoming messages and distribute them
     */
    everyone.now.sendMessage = function(message) {
      everyone.now.receiveMessage(message);
    }

So, when a user enters a message and clicks the send button, it calls the function sendMessage() on the server. The server in turn, calls the receiveMessage() function on every connected client.

Tips

Since Node.js servers are typically run from the command line as:

node path/to/yourFile.js

Running a continuous server and capturing anything logged to the console is a bit challenging. You could background the process and append stdout to a log file, but that doesn't provide any protection if your node server quits unexpectedly. A better solution is to run the server using Forever. You can use NPM to install Forever:

npm install forever

Then to start your Node.js server, run:

forever start path/to/yourFile.js

Forever will automatically restart your server if it unexpectedly exits and you can pass it arguments to stop, restart and list running servers. It also includes a number of options, including the ability to specify your log file.

While Socket.io provides WebSockets functionality, actual support by browsers is limited by either failure to include it (Internet Explorer) or intentional disabling due to security concerns (Firefox). Socket.io's ability to invisibly fallback to other transports works well, however some transports such as XHR-Polling may introduce unexpected problems. When using polling, clients send a frequent "heartbeat" in order to maintain contact with the server. Clients with poor connections will frequently fail to send heartbeats within the allowed time period and be disconnected from the server. As of version 0.6, Socket.io will automatically reconnect clients, but you may need to write some buffering to account for it. For example, a chat message may need to be stored and delivery reattempted upon reconnect.

Revolutionize Your Web

Obviously, my example code is an extremely basic chat application, but if you were one of the unconvinced who didn't believe that Node.js was revolutionizing the web (or at least has a huge potential to do so), hopefully the meager 11 lines of server code helped change your mind and perhaps inspired you to give Node.js a try. More documentation is available from the respective Node.js, Socket.io and Now.js websites. Go make a real-time web application and revolutionize your web!