Lesson 7 - Node.js, NPM, and Yarn
Introduction​
You are very familiar with Javascript by now, and you may notice that we can
only run Javascript in our browser using the <script> tag. What if we want to
write scripts using Javascript and run them using the local machines, just like
how python is.
This is where Node comes into play. It is a JavaScript runtime engine built on Chrome's V8 Javascript engine that allows you to run JavaScript even without a browser. This means that you are able to do a lot more with JavaScript: managing local files, connecting and setting up a server, and much more.
Learning Objectives​
- How npm & node works
- Create Backend APIs using Node & Express
- How to make API calls using axios
- File system with node & express
Getting Started​
Let's start by creating a folder for us to put our code in. Let's call this
folder example and then let's cd` into it using the console.
mkdir example
cd example
NPM​
With the ability to run JavaScript locally on our machine (with the help of Node) and the contribution of a rich number of packages from a large community, the possibilities for JavaScript are endless!
To manage many of the packages that give Node its most powerful abilities, we use NPM. Node package manager (NPM) is a place where you can find and download node packages.
Installing Node​
Go to https://nodejs.org/en/ to install node.js for free.
Let us confirm that you have correctly downloaded Node.js and npm.
In the terminal, type
node -v
npm -v
If both lines output the version number then we are good.
Now, let us install out first package. The remainder of this example we will be using Axios to make HTTP requests as necessary, so let us install Axios.
Navigate to the folder where we are storing our project, in this case example/
while there, type the following command:
npm init -y
npm install axios
The npm init command is used to create a new npm package, essentially every
node script that we will write can be treated as a npm package. The -y flag
just skip all the configuration step and creates a most basic npm package with
all default settings.
Now when we list the files in our example/ directory, we should see three
things:
node_modulesfolderpackage.jsonfilepackage-lock.jsonfile (optional)
Essentially, each of these helps npm organize the packages we installed. In the node_modules folder is where we can find our actual packages. Many times, our node_modules file grows incredibly large and can take up quite a bit of space. It is discouraged to go directly into the node_modules, instead, npm provides a variety of utilities which we can use to keep our projects clean and efficient. I won't go into them here, but as you develop, you will run across them or you can read the docs here. The other file is the package.json file. The package.json file is the file which record and defines the top-level packages and dependencies for our project. package-lock.json is a similar idea but with some nuances, you do not need to worry about those.
Hello World​
Now let's move back to Node for a second. Let's create a file called
helloworld.js in the example/ folder. Enter console.log("hello world.") in
the file.
You can run any node file by using the Node command.
node helloworld.js
The console should print "hello world." and then exit.
JS in Browser vs. Node.js​
Node in browser and Node.js are technically the same form of JavaScript. Both
run on the browser's JavaScript engine, usually Chrome V8. However, there are
some fundamental difference which make Node.js just different enough to be
annoying. The most prominent difference is that Node.js uses the require
module to import other modules which define functionality. All Node projects are
organized into modules, and we use require to import them. For example, the
following code imports Axios, a package commonly used to make HTTP requests.
const axios = require('axios');
Another difference is that since there is no web page to be manipulated, there
is no document object for Node.js, which means that DOM manipulation cannot be
carried out in Node.js (with the help of some packages, it can).
Since Node is a server side language, it can interact with the local file system easily, it also provides ways to setup a server with ease.
If you are familiar with the fetch function in JS, there is no fetch function
in Node. You can use the http package or some other third party package to
handle sending a request.
Web Server with Node & Express
Install Express​
Let's build a web server using Node! Since building a server can be very hard using Node, a package called Express is often used to set up a web server in a faster and simpler way.
Let's install it by npm install express.
After installing express, you would find it under the dependencies attribute in
the package.json file.
Reminder: this is how node keeps track of all the packages we used.
Hello World Web Server​
const axios = require('axios');
const express = require('express');
const port = 3000;
const app = express();
app.get('/', (req, res) => {
res.send('Hello World');
});
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
Above is the code for the most basic web server. Let's analyze it line by line.
To import any npm packages that we download, we use the require() function
provided by Node. Thus, the const express = require('express')
const app = express() creates an app object that provides various useful
methods.
app.get('/', (req, res) => {
res.send('Hello World!');
});
The get method will return "Hello world!" once a web browser visits
http://localhost:3000 Essentially, we are telling node
that when the user makes a GET request to the / url (this is the root url),
define this response and send it.
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
The listen method will set up and start the server that listens for any
requests at a specific port.
Let's start our application by typing node index.js then visiting
http://localhost:3000
You would see a Hello World displayed on the screen, which means that our server is up and running.
Adding routes and sending requests​
Now let's define some more functionality for our server.
Before we jump into actually writing code, let's take a moment to consider design. With smarter design, we can make our code intuitive, clean, and efficient. The goal of software is to make life intuitive, clean, and efficient!
We will be implementing a "dumb" version of a database, meaning that we will be emulating what a database would do with a very stupid implementation. We will use a JSON object to store the items at a certain index and then return those items as needed.
First, let's think about what types of data we will be interfacing with. There are two main types: metadata and the actual database data. Our metadata is the data that we use to define the state of our database and its characteristics, and the actual database data will be the items we put.
Some examples of a metadata are the database capacity and current index. Let's
define two routes in our API: /info and /db . These are where we will make
our requests.
GET Requests​
const axios = require('axios');
const express = require('express');
const port = 3000;
const app = express();
app.use(express.json()); // Utilities for request bodies
app.use(express.urlencoded({ extended: true })); // Utilities for query params
let counter = 0; // inner implmentation
stupidDB = {}; // initialize our dumb database
// GET Requests
app.get('/', (req, res) => {
// homepage
res.send('stupidDB API');
});
app.get('/info/index', (req, res) => {
// get the current index
res.send({ counter: counter });
});
app.get('/info/capacity', (req, res) => {
// get the capacity
res.send({ capacity: Object.keys(stupidDB).length });
});
app.get('/db/all', (req, res) => {
// get all items from the db
res.send(stupidDB);
});
app.get('/db/:id', (req, res) => {
// get a certain item from the db
const id = req.params.id;
if (id in stupidDB) {
res.send(stupidDB[id]);
} else {
res.send({ error: 'no object found with this id' });
}
});
// Server Setup
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
GET requests have a lot of flexibility and power in the way that we make them. We can get information generically or we can encode the information that we want into our API. Theoretically, we could also put that information into the body of the request, but in general, that is not in line with our REST API principles. GET returns information that we query to it.
POST & PUT Requests​
const axios = require('axios');
const express = require('express');
const port = 3000;
const app = express();
app.use(express.json()); // Utilities for request bodies
app.use(express.urlencoded({ extended: true })); // Utilities for query params
let counter = 0; // inner implmentation
stupidDB = {}; // initialize our dumb database
// GET Requests
app.get('/', (req, res) => {
// homepage
res.send('stupidDB API');
});
app.get('/info/index', (req, res) => {
// get the current index
res.send({ counter: counter });
});
app.get('/info/capacity', (req, res) => {
// get the capacity
res.send({ capacity: Object.keys(stupidDB).length });
});
app.get('/db/all', (req, res) => {
// get all items from the db
res.send(stupidDB);
});
app.get('/db/:id', (req, res) => {
// get a certain item from the db
const id = req.params.id;
if (id in stupidDB) {
res.send(stupidDB[id]);
} else {
res.send({ error: 'no object found with this id' });
}
});
// POST Requests
app.post('/db', (req, res) => {
console.log(req);
const item = req.body.item; // access our request body
stupidDB[counter] = item; // add body item at index
counter += 1; // increment counter
res.send(`POST Request Successful. Item placed: ${item}`); // Send HTTP response
});
// PUT Requests
app.put('/db/:id', (req, res) => {
const id = req.params.id; // get the index of the data to update
const item = req.body.item; // access the body of the request which holds new data
if (id in stupidDB) {
stupidDB[id] = item; // insert destructively
res.send({ newItem: item });
} else {
res.send({ error: 'no object found with this id' });
}
});
// Server Setup
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
POST and PUT are, in my opinion, what separate a static website from a true web application. POST and PUT are how we actually collect and store relevant data for the user. They are the methods which give our website functionality for the user.
Let's look at POST first. To pass data along a POST request, we place data in the body. Because we don't know the "shape" and size of our data, we utilize the body as a sort of empty container to hold the data that we want to send. (Note: It would be overkill to put request parameters into this space for GET requests).
Exercise: Go to Postman and set up a POST request. Go to body and look at all of the potential options for request bodies. By and large, we will be using "raw" in this class. Click on raw and view the options in the dropdown for data formats. Lot's of options! In this class, we will largely stick to JSON but if you are curious, you can always explore others. (Note: other formats may require extra setup on the Express side).
Now let's look at PUT. PUT is an excellent example of how versatile a request is. In our PUT request, we use both url encoding and the body to send relevant data to our server. In the URL, we put where we want the data to go, and in our body we define what data we want to go there. This is exactly aligned to the design principles we laid our for our GET and POST requests! With smarter design we can vastly simplify the complexity of our systems.
Let's get some pictures from NASA using this API that they are providing and send the descriptions back.
To send a request to a particular api, we use the package called Axios. Remember
that we have already npm installed axios so it should be available for us to
use.
const express = require('express')
const app = express()
const port = 3000
const axios = require('axios');
app.get('/nasa', (req, res) => {
axios.get('https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY')
.then(response => {
res.send(response.data);
})
.catch(error => {
res.send(error);
});
})
We will try to send an API request to NASA whenever a user visits the /nasa
route of our website.
Now to send an API request, we will use axios.get(). This will send a get
request to the particular url provided.
Now go ahead and visit the
http://localhost:3000/nasa url, you would find
this displaying on the screen.
We will learn more about API requests and web server in the future.
Node and File System​
Nodejs just like python, provides a very easy way to handle file system.
Let's build on top of the web server that we have. Say we want to build a API where the user can input the name and type of a file and the server can create it.
- http://localhost:3000/create/?name=resume&type=txt
- The API will look something like this, where the ? indicates parameters, and name=resume and type=txt are two of the parameters included in this API, they are connected by the & sign.
app.get('/create', (req, res) => {
let name = req.query.name;
let type = req.query.type;
...
})
In the /create route, we simply can access all the parameter like this.
Now, let us create a file using the file system package that node provides.
const fs = require("fs");
...
app.get('/create', (req, res) => {
let name = req.query.name;
let type = req.query.type;
let filename = name + '.' + type;
fs.appendFile(filename, ' ', (err) => {
if (err) res.send(err);
res.send({
"message": "File created!",
"name": name,
"type": type
})
})
})
The fs.appendFile() method creates a file with the given name in the current
folder that the server is in. Now try to visit
http://localhost:3000/create/?name=resume&type=txt
and see whether a file named resume.txt is created.
Resources​
Get Query Strings and Parameters in Express.js
Contributors