NodeJs

Setup

Running start up scripts in node

"scripts": {
  "start": "run-p start:dev start:api",
  "start:dev: "webpack-dev-server --config webpack.config.dev.js --port 3000",
  "prestart:api": "node tools/createMockDb.js",
   "start:api": "node tools/apiServer.js"
};

The run-p runs multiple jobs, pre<value> runs the command before the value without pre

Adding consts in webpack

To pass dev/production consts we can do this using webpack. Include webpack in you webpack.config.dev.js and webpack.DefinePlugin e.g.

const webpack = require("webpack")
...
plugins: [
    new webpack.DefinePlugin({
        "process.env.API_URL": JSON.stringify("http:://localhost:3001")
    })
],
...

Introduction

Why nodejs

Node is popular because

• Node supports JavaScript
• Non Blocking
• Virtual Machine Single-Thread

Resources

There is a site https://medium.com/edge-coders/how-well-do-you-know-node-js-36b1473c01c8 where u can test your knowledge.

Resource for the course were at https://github.com/jscomplete/advanced-nodejs

Closures

I am really bad at remembering names for things for here is my understanding of Closures

Lexical Scoping

Basically this is driving at the fact there is such a time called scope, maybe new to people in JS at one time. ha ha. I.E. the var is available in init() but not outside of init()

This is an example of lexical scoping, which describes how a parser resolves variable names when functions are nested. The word lexical refers to the fact that lexical scoping uses the location where a variable is declared within the source code to determine where that variable is available. Nested functions have access to variables declared in their outer scope.

function init() {
  var name = 'Mozilla'; // name is a local variable created by init
  function displayName() { // displayName() is the inner function, a closure
    alert(name); // use variable declared in the parent function
  }
  displayName();
}
init();

Closure Example

And repeat this almost, is that we return the function to be executed later. The values at the time of calling makeFunc are retained. i.e. name is 'Mozilla' if we execute myFunc().

A closure is the combination of a function and the lexical environment within which that function was declared. This environment consists of any local variables that were in-scope at the time the closure was created. In this case, myFunc is a reference to the instance of the function displayName that is created when makeFunc is run. The instance of displayName maintains a reference to its lexical environment, within which the variable name exists.

function makeFunc() {
  var name = 'Mozilla';
  function displayName() {
    alert(name);
  }
  return displayName;
}

var myFunc = makeFunc();
myFunc();

Node Architecture

V8

V8 is Google’s open source high-performance JavaScript and WebAssembly engine, written in C++. It is used in Chrome and in Node.js, among others. It implements ECMAScript and WebAssembly, and runs on Windows 7 or later, macOS 10.12+, and Linux systems that use x64, IA-32, ARM, or MIPS processors. V8 can run standalone, or can be embedded into any C++ application. libuv is a multi-platform C library that provides support for asynchronous I/O based on event loops. It supports epoll, kqueue, Windows IOCP, and Solaris event ports. It is primarily designed for use in Node.js but it is also used by other software projects.[3] It was originally an abstraction around libev or Microsoft IOCP, as libev supports only select and doesn't support poll and IOCP on Windows.

Chackra

This is an alternative to V8. I could find much on this cos I had other stuff to do but here is a diagram to make the page look nice. Note the picture is deliberately smaller.

V8 Node Options

The v8 engine has three feature groups, Shipping, Staged and In Progress. These are like channels and you can see which feature are supported with

node --v8-options | grep "staged"

Node CLI and REPL

REPL

• autocomplete
• up down history
• _ provides last command value
• .blah, .break will get you out of autocomplete, .save will write to history

Global Object

Introduction

The global object supports many functions. This is the core of Node. For instance you can create a value in global which can be accessed from anywhere

  
global.myvar = 10000

An example of usage can be global.process which holds the information for a process started in node. So global.process.env will provide the environment variables.

Taking Process As an Example

In this example we can see that process.on has registered a uncaughtException. If we did not call process.exit(1) this would result in an infinite loop.

  
// process is an event emitter

process.on('exit', (code) => {
  // do one final synchronous operation
  // before the node process terminates

  console.log(`About to exit with code: ${code}`);
});

process.on('uncaughtException', (err) => {
  // something went unhandled.
  // Do any cleanup and exit anyway!

  console.error(err); // don't do just that.

  // FORCE exit the process too.
  process.exit(1);
});

// keep the event loop busy
process.stdin.resume();

// trigger a TypeError exception
console.dog();

And the rest

The other functions are like a bash api or c api. They are functions which can be called using javascript.

Require

When we envoke require the following happens

• Resolving
• Evaluating
• Loading
• Wrapping
• Caching

By using the repl we can see how it finds the module.

  
console.log(module)
Module {
  id: '<repl>',
  path: '.',
  exports: {},
  parent: undefined,
  filename: null,
  loaded: false,
  children: [],
  paths: [
    '/home/iwiseman/dev/scratch/android_frag/repl/node_modules',
    '/home/iwiseman/dev/scratch/android_frag/node_modules',
    '/home/iwiseman/dev/scratch/node_modules',
    '/home/iwiseman/dev/node_modules',
    '/home/iwiseman/node_modules',
    '/home/node_modules',
    '/node_modules',
    '/home/iwiseman/.node_modules',
    '/home/iwiseman/.node_libraries',
    '/usr/local/lib/node'
  ]
}

With require it will try

• something.js
• something.json
• something.node (compiled add-on file)

  
// hello.cc
#include <node.h>

namespace demo {

using v8::FunctionCallbackInfo;
using v8::Isolate;
using v8::Local;
using v8::Object;
using v8::String;
using v8::Value;

void Method(const FunctionCallbackInfo<Value>& args) {
  Isolate* isolate = args.GetIsolate();
  args.GetReturnValue().Set(String::NewFromUtf8(
      isolate, "world").ToLocalChecked());
}

void Initialize(Local<Object> exports) {
  NODE_SET_METHOD(exports, "hello", Method);
}

NODE_MODULE(NODE_GYP_MODULE_NAME, Initialize)

}

npm

With npm we can install from github directly. E.g. npm i expressjs/express will install the latest express. you can confirm this with npm ls express. You can even install from branches e.g. npm i expressjs/express#4.14.0

• npm updated

Event Emitters

Synchronous Example

These can be used in Node Js just like listeners in other languages. "emit" to send and "on" to perform an action. Below is an example of synchronous events

const EventEmitter = require('events');

class WithLog extends EventEmitter {
  execute(taskFunc) {
    console.log('Before executing');
    this.emit('begin');
    taskFunc();
    this.emit('end');
    console.log('After executing');
  }
}

const withLog = new WithLog();

withLog.on('begin', () => console.log('About to execute'));
withLog.on('end', () => console.log('Done with execute'));

withLog.execute(() => console.log('*** Executing task ***'));

Asynchronous Example

const fs = require('fs');
const EventEmitter = require('events');

class WithTime extends EventEmitter {
  execute(asyncFunc, ...args) {
    console.time('execute');
    this.emit('begin');
    asyncFunc(...args, (err, data) => {
      if (err) {
        return this.emit('error', err);
      }

      this.emit('data', data);
      console.timeEnd('execute');
      this.emit('end');
    });
  }
}

const withTime = new WithTime();

withTime.on('begin', () => console.log('About to execute'));
withTime.on('end', () => console.log('Done with execute'));

withTime.execute(fs.readFile, __filename);

Client Server

Really liked how easy it was to built this example. Probably will never use it but the simplicity made me smile. I also like the process.nextTick() to wait for the load to complete from require('./server')(client);

Client

const EventEmitter = require('events');
const readline = require('readline');

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

const client = new EventEmitter();
const server = require('./server')(client);

server.on('response', (resp) => {
  process.stdout.write('\u001B[2J\u001B[0;0f');
  process.stdout.write(resp);
  process.stdout.write('\n\> ');
});

let command, args;
rl.on('line', (input) => {
  [command, ...args] = input.split(' ');
  client.emit('command', command, args);
});

Server

const EventEmitter = require('events');

class Server extends EventEmitter {
  constructor(client) {
    super();
    this.tasks = {};
    this.taskId = 1;
    process.nextTick(() => {
      this.emit(
        'response',
        'Type a command (help to list commands)'
      );
    });
    client.on('command', (command, args) => {
      switch (command) {
      case 'help':
      case 'add':
      case 'ls':
      case 'delete':
        this[command](args);
        break;
      default:
        this.emit('response', 'Unknown command...');
      }
    });
  }

  tasksString() {
    return Object.keys(this.tasks).map(key => {
      return `${key}: ${this.tasks[key]}`;
    }).join('\n');
  }

  help() {
    this.emit('response', `Available Commands:
  add task
  ls
  delete :id`
    );
  }
  add(args) {
    this.tasks[this.taskId] = args.join(' ');
    this.emit('response', `Added task ${this.taskId}`);
    this.taskId++;
  }
  ls() {
    this.emit('response', `Tasks:\n${this.tasksString()}`);
  }
  delete(args) {
    delete(this.tasks[args[0]]);
    this.emit('response', `Deleted task ${args[0]}`);
  }
}

module.exports = (client) => new Server(client);

Chat Server

Again just for reference a Chat Server. It shows just how easy software can be. Never used nc before but willing to drop telnet and be an old dog learning new tricks.

const server = require('net').createServer();
let counter = 0;
let sockets = {};

function timestamp() {
  const now = new Date();
  return `${now.getHours()}:${now.getMinutes()}`;
}

server.on('connection', socket => {
  socket.id = counter++;

  console.log('Client connected');
  socket.write('Please type your name: ');

  socket.on('data', data => {
    if (!sockets[socket.id]) {
      socket.name = data.toString().trim();
      socket.write(`Welcome ${socket.name}!\n`);
      sockets[socket.id] = socket;
      return;
    }
    Object.entries(sockets).forEach(([key, cs]) => {
      if (socket.id == key) return;
      cs.write(`${socket.name} ${timestamp()}: `);
      cs.write(data);
    });
  });

  socket.on('end', () => {
    delete sockets[socket.id];
    console.log('Client disconnected');
  });
});

server.listen(8000, () => console.log('Server bound'));

DNS

There is a dns module which support dns.reverse and dns.lookup.

HTTP

Simple HTTP Server Example

This demonstrate that the server will stream by default

const server = require('http').createServer();

server.on('request', (req, res) => {
  res.writeHead(200, { 'content-type': 'text/plain' });
  res.write('Hello world\n');

  setTimeout(function () {
    res.write('Another Hello world\n');
  }, 10000);

  setTimeout(function () {
    res.write('Yet Another Hello world\n');
  }, 20000);

  // res.end('We want it to stop');
});

// server.timeout = 100000 No go on forever!
server.listen(8000)

Simple HTTPS Server Example

And with https

const fs = require('fs');
const server = require('https')
  .createServer({
    key: fs.readFileSync('./key.pem'),
    cert: fs.readFileSync('./cert.pem'),
  });

server.on('request', (req, res) => {
  res.writeHead(200, { 'content-type': 'text/plain' });
  res.end('Hello world\n');
});
server.listen(443);

Simple Routing Server Example

Just loving how simple this all is and comparing it to my golang solution. I should be easy because it is but this is reminding me of what I already know. Top tip node -p http.STATUS_CODES

const fs = require('fs');
const server = require('http').createServer();
const data = {};

server.on('request', (req, res) => {
  switch (req.url) {
  case '/api':
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify(data));
    break;
  case '/home':
  case '/about':
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end(fs.readFileSync(`.${req.url}.html`));
    break;
  case '/':
    res.writeHead(301, { 'Location': '/home' });
    res.end();
    break;
  default:
    res.writeHead(404);
    res.end();
  }
});

server.listen(8000);

Simple HTTPS Client Example

const https = require('https');

// req: http.ClientRequest
const req = https.get(
  'https://www.google.com',
  (res) => {
    // res: http.IncomingMessage
    console.log(res.statusCode);
    console.log(res.headers);

    res.on('data', (data) => {
      console.log(data.toString());
    });
  }
);

req.on('error', (e) => console.log(e));

console.log(req.agent); // http.Agent

Simple HTTPS Client Example

Here is a diagram from the node documentation showing the various parts of a URL. We can use url.parse to see the varsion parts in an object. e.g.

const urlString = 'https://www.pluralsight.com/search?q=buna';
...
// Output 
Url {
  protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'www.pluralsight.com',
  port: null,
  hostname: 'www.pluralsight.com',
  hash: null,
  search: '?q=buna',
  query: [Object: null prototype] { q: 'buna' },
  pathname: '/search',
  path: '/search?q=buna',
  href: 'https://www.pluralsight.com/search?q=buna'
}

Equal we can reverse this with

const urlObject = {
  protocol: 'https',
  host: 'www.pluralsight.com',
  search: '?q=buna',
  pathname: '/search',
};
 url.format(urlObject)

// Output
// 'https://www.pluralsight.com/search?q=buna'

Built in Libraries

OS Module

This provides a ton of information around the os. E.g. version. From my old days I did like the node -p os.constants.signals

FS Module

This provides sync and async function for reading and writing. Here is an example script showing how to use it. It watches and log file changes to the directory.

const fs = require('fs');
const path = require('path');

const dirname = path.join(__dirname, 'files');
const currentFiles = fs.readdirSync(dirname);

const logWithTime = (message) =>
  console.log(`${new Date().toUTCString()}: ${message}`);

fs.watch(dirname, (eventType, filename) => {
  if (eventType === 'rename') { // add or delete
    const index = currentFiles.indexOf(filename);
    if (index >= 0) {
      currentFiles.splice(index, 1);
      logWithTime(`${filename} was removed`);
      return;
    }

    currentFiles.push(filename);
    logWithTime(`${filename} was added`);
    return;
  }

  logWithTime(`${filename} was changed`);
});

Console

You can use console to time the functions <syntaxhighlight lang="js"> console.time("test 1") // My large jobby ... console.timeEnd("test 1")

Streams

Types of Streams

• Readable (fs.createReadStream)
• Writable (fs.createWriteStream)
• Duplex (net.Socket)
• Transform (zlib.createGzip)

Functions and Events

The following show a list of some of the functions and events streams support. Import events for readable are data and end. For Writable it would be drain event to signal it and consume more data and finish

Clusters and Child Process

Node js supports the linux concepts of

• spawn (no new shell)
• exec (creates a shell)
• execFile
• fork

The cluster module allows up to split the processing using the fork. We can use it for load balancing.