Skip to main content

Lesson 7 - Node.js, NPM, and Yarn

Introduction

You are very familiar with Javascript by now, and you may notice that we can only run Javascript in our browser using the <script> tag. What if we want to write scripts using Javascript and run them using the local machines, just like how python is.

This is where Node comes into play. It is a JavaScript runtime engine built on Chrome's V8 Javascript engine that allows you to run JavaScript even without a browser. This means that you are able to do a lot more with JavaScript: managing local files, connecting and setting up a server, and much more.

Learning Objectives

  1. How npm & node works
  2. Create Backend APIs using Node & Express
  3. How to make API calls using axios
  4. File system with node & express

Getting Started

Let's start by creating a folder for us to put our code in. Let's call this folder example and then let's cd` into it using the console.

mkdir example
cd example

NPM

With the ability to run JavaScript locally on our machine (with the help of Node) and the contribution of a rich number of packages from a large community, the possibilities for JavaScript are endless!

To manage many of the packages that give Node its most powerful abilities, we use NPM. Node package manager (NPM) is a place where you can find and download node packages.

npm

Installing Node

Go to https://nodejs.org/en/ to install node.js for free.

Let us confirm that you have correctly downloaded Node.js and npm.

In the terminal, type

node -v
npm -v

If both lines output the version number then we are good.

Now, let us install out first package. The remainder of this example we will be using Axios to make HTTP requests as necessary, so let us install Axios.

Navigate to the folder where we are storing our project, in this case example/ while there, type the following command:

npm init -y
npm install axios

The npm init command is used to create a new npm package, essentially every node script that we will write can be treated as a npm package. The -y flag just skip all the configuration step and creates a most basic npm package with all default settings.

Now when we list the files in our example/ directory, we should see three things:

  • node_modules folder
  • package.json file
  • package-lock.json file (optional)

Essentially, each of these helps npm organize the packages we installed. In the node_modules folder is where we can find our actual packages. Many times, our node_modules file grows incredibly large and can take up quite a bit of space. It is discouraged to go directly into the node_modules, instead, npm provides a variety of utilities which we can use to keep our projects clean and efficient. I won't go into them here, but as you develop, you will run across them or you can read the docs here. The other file is the package.json file. The package.json file is the file which record and defines the top-level packages and dependencies for our project. package-lock.json is a similar idea but with some nuances, you do not need to worry about those.

Hello World

Now let's move back to Node for a second. Let's create a file called helloworld.js in the example/ folder. Enter console.log("hello world.") in the file.

You can run any node file by using the Node command.

node helloworld.js

The console should print "hello world." and then exit.

JS in Browser vs. Node.js

Node in browser and Node.js are technically the same form of JavaScript. Both run on the browser's JavaScript engine, usually Chrome V8. However, there are some fundamental difference which make Node.js just different enough to be annoying. The most prominent difference is that Node.js uses the require module to import other modules which define functionality. All Node projects are organized into modules, and we use require to import them. For example, the following code imports Axios, a package commonly used to make HTTP requests.

const axios = require('axios');

Another difference is that since there is no web page to be manipulated, there is no document object for Node.js, which means that DOM manipulation cannot be carried out in Node.js (with the help of some packages, it can).

Since Node is a server side language, it can interact with the local file system easily, it also provides ways to setup a server with ease.

If you are familiar with the fetch function in JS, there is no fetch function in Node. You can use the http package or some other third party package to handle sending a request.

Web Server with Node & Express

Install Express

Let's build a web server using Node! Since building a server can be very hard using Node, a package called Express is often used to set up a web server in a faster and simpler way.

Let's install it by npm install express.

After installing express, you would find it under the dependencies attribute in the package.json file.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/2e0ac7d2-506e-4963-948a-6b38c28908a5/Untitled.png

Reminder: this is how node keeps track of all the packages we used.

Hello World Web Server

const axios = require('axios');
const express = require('express');
const port = 3000;

const app = express();

app.get('/', (req, res) => {
res.send('Hello World');
});

app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});

Above is the code for the most basic web server. Let's analyze it line by line.

To import any npm packages that we download, we use the require() function provided by Node. Thus, the const express = require('express')

const app = express() creates an app object that provides various useful methods.

app.get('/', (req, res) => {
res.send('Hello World!');
});

The get method will return "Hello world!" once a web browser visits http://localhost:3000 Essentially, we are telling node that when the user makes a GET request to the / url (this is the root url), define this response and send it.

app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});

The listen method will set up and start the server that listens for any requests at a specific port.

Let's start our application by typing node index.js then visiting http://localhost:3000

You would see a Hello World displayed on the screen, which means that our server is up and running.

Adding routes and sending requests

Now let's define some more functionality for our server.

Before we jump into actually writing code, let's take a moment to consider design. With smarter design, we can make our code intuitive, clean, and efficient. The goal of software is to make life intuitive, clean, and efficient!

We will be implementing a "dumb" version of a database, meaning that we will be emulating what a database would do with a very stupid implementation. We will use a JSON object to store the items at a certain index and then return those items as needed.

First, let's think about what types of data we will be interfacing with. There are two main types: metadata and the actual database data. Our metadata is the data that we use to define the state of our database and its characteristics, and the actual database data will be the items we put.

Some examples of a metadata are the database capacity and current index. Let's define two routes in our API: /info and /db . These are where we will make our requests.

GET Requests

const axios = require('axios');
const express = require('express');
const port = 3000;
const app = express();

app.use(express.json()); // Utilities for request bodies
app.use(express.urlencoded({ extended: true })); // Utilities for query params

let counter = 0; // inner implmentation
stupidDB = {}; // initialize our dumb database

// GET Requests

app.get('/', (req, res) => {
// homepage
res.send('stupidDB API');
});

app.get('/info/index', (req, res) => {
// get the current index
res.send({ counter: counter });
});

app.get('/info/capacity', (req, res) => {
// get the capacity
res.send({ capacity: Object.keys(stupidDB).length });
});

app.get('/db/all', (req, res) => {
// get all items from the db
res.send(stupidDB);
});

app.get('/db/:id', (req, res) => {
// get a certain item from the db
const id = req.params.id;
if (id in stupidDB) {
res.send(stupidDB[id]);
} else {
res.send({ error: 'no object found with this id' });
}
});

// Server Setup

app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});

GET requests have a lot of flexibility and power in the way that we make them. We can get information generically or we can encode the information that we want into our API. Theoretically, we could also put that information into the body of the request, but in general, that is not in line with our REST API principles. GET returns information that we query to it.

POST & PUT Requests

const axios = require('axios');
const express = require('express');
const port = 3000;
const app = express();

app.use(express.json()); // Utilities for request bodies
app.use(express.urlencoded({ extended: true })); // Utilities for query params

let counter = 0; // inner implmentation
stupidDB = {}; // initialize our dumb database

// GET Requests

app.get('/', (req, res) => {
// homepage
res.send('stupidDB API');
});

app.get('/info/index', (req, res) => {
// get the current index
res.send({ counter: counter });
});

app.get('/info/capacity', (req, res) => {
// get the capacity
res.send({ capacity: Object.keys(stupidDB).length });
});

app.get('/db/all', (req, res) => {
// get all items from the db
res.send(stupidDB);
});

app.get('/db/:id', (req, res) => {
// get a certain item from the db
const id = req.params.id;
if (id in stupidDB) {
res.send(stupidDB[id]);
} else {
res.send({ error: 'no object found with this id' });
}
});

// POST Requests

app.post('/db', (req, res) => {
console.log(req);
const item = req.body.item; // access our request body
stupidDB[counter] = item; // add body item at index
counter += 1; // increment counter
res.send(`POST Request Successful. Item placed: ${item}`); // Send HTTP response
});

// PUT Requests

app.put('/db/:id', (req, res) => {
const id = req.params.id; // get the index of the data to update
const item = req.body.item; // access the body of the request which holds new data
if (id in stupidDB) {
stupidDB[id] = item; // insert destructively
res.send({ newItem: item });
} else {
res.send({ error: 'no object found with this id' });
}
});

// Server Setup

app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});

POST and PUT are, in my opinion, what separate a static website from a true web application. POST and PUT are how we actually collect and store relevant data for the user. They are the methods which give our website functionality for the user.

Let's look at POST first. To pass data along a POST request, we place data in the body. Because we don't know the "shape" and size of our data, we utilize the body as a sort of empty container to hold the data that we want to send. (Note: It would be overkill to put request parameters into this space for GET requests).

Exercise: Go to Postman and set up a POST request. Go to body and look at all of the potential options for request bodies. By and large, we will be using "raw" in this class. Click on raw and view the options in the dropdown for data formats. Lot's of options! In this class, we will largely stick to JSON but if you are curious, you can always explore others. (Note: other formats may require extra setup on the Express side).

Now let's look at PUT. PUT is an excellent example of how versatile a request is. In our PUT request, we use both url encoding and the body to send relevant data to our server. In the URL, we put where we want the data to go, and in our body we define what data we want to go there. This is exactly aligned to the design principles we laid our for our GET and POST requests! With smarter design we can vastly simplify the complexity of our systems.

Let's get some pictures from NASA using this API that they are providing and send the descriptions back.

To send a request to a particular api, we use the package called Axios. Remember that we have already npm installed axios so it should be available for us to use.

const express = require('express')
const app = express()
const port = 3000

const axios = require('axios');

app.get('/nasa', (req, res) => {
axios.get('https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY')
.then(response => {
res.send(response.data);
})
.catch(error => {
res.send(error);
});
})

We will try to send an API request to NASA whenever a user visits the /nasa route of our website.

Now to send an API request, we will use axios.get(). This will send a get request to the particular url provided.

Now go ahead and visit the http://localhost:3000/nasa url, you would find this displaying on the screen.

expected output

We will learn more about API requests and web server in the future.

Node and File System

Nodejs just like python, provides a very easy way to handle file system.

Let's build on top of the web server that we have. Say we want to build a API where the user can input the name and type of a file and the server can create it.

app.get('/create', (req, res) => {
let name = req.query.name;
let type = req.query.type;
...
})

In the /create route, we simply can access all the parameter like this.

Now, let us create a file using the file system package that node provides.

const fs = require("fs");

...

app.get('/create', (req, res) => {
let name = req.query.name;
let type = req.query.type;
let filename = name + '.' + type;
fs.appendFile(filename, ' ', (err) => {
if (err) res.send(err);
res.send({
"message": "File created!",
"name": name,
"type": type
})
})
})

The fs.appendFile() method creates a file with the given name in the current folder that the server is in. Now try to visit http://localhost:3000/create/?name=resume&type=txt and see whether a file named resume.txt is created.

Resources

Express "Hello World" example

NASA Open APIs

Get Query Strings and Parameters in Express.js


Contributors