Building a gRPC Server in NodeJS
In this article we will learn how to build a simple gRPC server in NodeJS.
What is gRPC
gRPC is a way for clients and servers to communicate with each other over the Internet. It is intended as an upgrade from REST APIs, with some great features:
- It transmits faster by using a binary format instead of JSON.
- It uses a language-agnostic data format, which can be defined and passed. around to clients without the need for per-language developer documentation.
- It translates its common data format into the native data formats of the server and client.
- It supports versioning, so future versions of server code don’t require a total rewrite of the client.
- It defines how to find the endpoints, so URL naming is no longer an issue.
- It works over HTTP/3, which enables bidirectional streaming with less network latency.
The down sides (at the time of writing) are:
- It requires custom compiler code for each language to perform all the conversions and binary format conversions.
- The mechanics of operation are totally obscure to developers. It is not human-readable the way JSON and REST are.
- The toolchains are new and not very well supported. Sometimes getting a simple call and response to work can take hours or days to get working.
- It isn’t yet well supported or well documented on all platforms.
- HTTP/3 is poorly supported by browsers and app software, especially over at Apple.
In this tutorial, I will attempt to walk you through the process I went through to get a simple gRPC client working in React Native on iOS.
How gRPC Works
Basically how gRPC works is that a developer creates a Protobuffer definition (.proto) file, which describes objects and methods which will be provided by a service. This file acts as a sort of developer documentation and a reference for a protobuffer compiler which converts the .proto definitions into the local language’s data types. This is similar to C++ header files or structs, where the the functions and data structures of a class are defined separately from their implementation.
The protobuffer compiler creates a library that the developer can interact with, abstracting the messaging layer of the API.
This allows a developer to write API logic without having to manage API versions, HTTP endpoints, or JSON parsing.
Terminology
gRPC uses slightly different terminology than most developers are familiar with.
In gRPC:
- A Message is a data structure, which contains primitive properties (known as fields) such as int, bool, or string. Developers can think of a Message as a class or a struct. A list of variable types and their corresponding language implementations can be found in Protocol Buffers Language Guide: Scalar Value Types
The convention is to name these in camel case, lowercase first letter, for examplebigNumber - A Remote Procedure Call or rpc is a method or function that can be called remotely. rpcs can take or return zero or more Message parameters. Developers can think of this as a method, function.
The convention is to name these in camel case, uppercase first letter, for exampleRpcName - A Service is a collection of rpcs. The service acts as a named container for rps, as well as a namespace. So different rpcs can have the same name in different Services, but not the same name name in the same Service. Developers can think of this as a class.
The convention is to name these in camel case, uppercase first letter, for exampleServiceName
Project Overview
The server will be called AuthService and it will contain two routes (routes are like endpoints if you are a web developer):
Login()Logout()
The Protobuffer File
Here we create a simple Protobuffer definition called authService.protothat provides two methods, Login() and Logout().
For now we will put this in ~/sandbox/protos/authService.proto:
syntax = "proto3";
package authService;
option objc_class_prefix = "RTG"; // important for iOS / Xcode// Define a service
// Define data types here
message AccountCredentials {
string username = 1;
string password = 2;
}message OauthCredentials {
string token = 3;
uint32 timeoutSeconds = 4;
}// Define the service containing methods here
service AuthServiceRoutes {
// Basic function call, makes request and returns value
rpc Login(AccountCredentials) returns (OauthCredentials) {}
rpc Logout(OauthCredentials) returns (OauthCredentials) {}
}
Notice that this file creates a service called AuthServiceRoutes:
service AuthServiceRoutes {
// ...
}You’ll see that the Login() rpc takes an AccountCredentials Message and returns an OauthCredentials .
rpc Login(AccountCredentials) returns (OauthCredentials) {}The actual implementation is not defined. Only the function name and parameters. The implementation will be done in the native language of the program, in this case NodeJS.
Likewise the Logout rpc takes an OauthCredentials message and returns one also.
rpc Logout(OauthCredentials) returns (OauthCredentials) {}The AccountCredentials message is defined as a structure with two properties, username and password . Both are strings. Each property is assigned an ID, in this case 1 and 2.
message AccountCredentials {
string username = 1;
string password = 2;
}Finally we have this line:
option objc_class_prefix = "AGS"; // important for iOS / XcodeIt’s a best practice in Swift to prefix class names with an acronym that represents the name of a class. in this case we will call the prefix AGS (short for “Authentication gRPC Service.”
This authService.proto file will be available to both the client and server at the time of development, so that the protobuffer compiler can create RPC endpoints.
Setup
Now that we have a .proto file, we can start our project. We will need to create a new project:
$ mkdir -p ~/sandbox/nodejs-grpc-auth-service
$ cd ~/sandbox/nodejs-grpc-auth-service
$ npm initFollow the steps in the guided setup of the project, then install dependencies. We will be using the official grpc-js module, which runs the gRPC service over HTTP/3.
We will also use the official proto-loader module, which compiles the .proto file into your project at runtime.
$ npm install --save @grpc/grpc-js
$ npm install --save @grpc/proto-loaderImplementing
To get this running, we need to:
- Load the Protobuffer file
- Implement the RPC calls
- Provide the gRPC service
Load the Protobuffer file
Let’s presume you put the .proto file into the ../protos folder, relative to the server path. This is common if you have the server and multiple clients sharing the same .proto file during development.
Create a server.js and enter the following to load the required libraries and load the ProtoBuffer file:
const PROTO_PATH = __dirname + '/../protos/authService.proto';
const grpc = require('@grpc/grpc-js')
const protoLoader = require('@grpc/proto-loader')Now we can load the .proto file and tell NodeJS how to interpret it:
// suggested options for similarity to loading grpc.load behavior
const packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{
keepCase: true,
longs: String, // JavaScript doesn't support long ints
enums: String, // JavaScript doesn't support enum types
defaults: true,
oneofs: true
}
)
const protoDescriptor = grpc.loadPackageDefinition(packageDefinition)
// grab the authService package from the protobuf file
const authService = protoDescriptor.authServiceImplement the business logic
We will create an AuthServer class, which corresponds to the name of the .proto file. Since we will be tracking logins and keeping track of authentication tokens, let’s create an .authTokens and .activeUsers hash maps to track our logged-in users.
class AuthServer {
constructor () {
this.authTokens = {}
this.activeUsers = {}
}
}for .login() , let’s take a username and password. We can verify the username against currently the .activeUsers to decide if we should do a login or an auth token refresh. Then we can create an object compatible with the Protobuffer file’s OauthCredentials to return. We will add the token into the .authTokens hash so that the .logout() function can look up the user by authentication token.
class AuthServer {
// constructor() {}
login (username, password) {
let loginData = {}
if (username in this.activeUsers) {
loginData = this.activeUsers[username]
this.authTokens[loginData.authToken].refreshExpiration()
} else {
loginData = new AuthLogin(username, password)
this.authTokens[loginData.authToken] = loginData
this.activeUsers[username] = loginData.authToken
}
const currentTime = new Date()
const oauthCredentials = {
token: loginData.authToken,
timeoutSeconds: Math.floor((loginData.expires - currentTime) / 1000)
}
return oauthCredentials
}To support this, let’s create a AuthLogin object which can do neat things like generating magic numbers for the token and refresh the authorization expiration time:
class AuthLogin {
constructor (username, password) {
this.username = username
this.authToken = Math.random().toString(16).substr(2, 8)
this.refreshExpiration()
}
refreshExpiration () {
const currentTime = new Date()
this.expires = currentTime.setDate(currentTime.getDate() + 1)
}
}The .logout() method is simpler: it attempts to look up the token in the .activeTokens and then removes the resulting user information from both .activeTokens and .activeUsers if it’s found. It returns blank data for the OauthCredentials Message.
class AuthServer {
// constructor() {}
// login(username, password) {}
logout () {
if (authToken in this.authTokens) {
const loginData = this.authTokens[authToken]
delete this.activeUsers[loginData.username]
delete this.authTokens[authToken]
}
return { token: '', timeoutSeconds: 0 }
}
}Then we can instantiate this Authentication server :
const authServer = new AuthServer()Binding the logic to gRPC calls
We must implement two rpc calls: Login() and Logout(). The convention in NodeJS is to name methods using a lowercase first letter, which is amicable to the protocol buffer compiler. So our NodeJS methods will be named .login() and .logout()
Each gRPC receives two parameters:
call, which contains the function parameters and other information about the client, etc, andcallback, which you will use to respond to the client.
This is similar to ExpressJS’s req and res parameters on REST API endpoints.
The login() rpc will simply call the authServer.login() method:
const login = (call, callback) => {
let username = call.request.username
let password = call.request.password
console.log(`Login for username: '${username}'`)
const authData = authServer.login(username, password)
callback(null, authData)
}and likewise the logout() rpc will call the authServer.logout() method:
const logout = (call, callback) => {
let authToken = call.request.token
console.log(`Logout for token: '${authToken}'`)
const authData = authServer.logout(authToken)
callback(null, authData)
}Provide the gRPC service
Now that we have defined these methods, we can bind them to a gRPC service:
function getGrpcServer () {
const grpcServer = new grpc.Server()
grpcServer.addService(authService.AuthServiceRoutes.service, {
login: login,
logout: logout
})
return grpcServer
}And finally host the gRPC service on port 50051 . Since we are testing we will use an insecure channel ( ServerCredentials.createInsecure() )
const grpcServer = getGrpcServer()
console.log('Starting gRPC server on port 0.0.0.0:50051...')
grpcServer.bindAsync(
'0.0.0.0:50051',
grpc.ServerCredentials.createInsecure(), () => {
grpcServer.start()
}
)If you wanted to host an SSL-encrypted gRPC service, you can use the ServerCredentials.createSsl() method by providing root certs, keyCertPairs, and a boolean option to check client certificates.
Run
Now that you’ve got the code, you can run a gRPC service on localhost. This service will reset all login data when restarted since we aren’t yet using persistent storage, but it is enough to get started.
$ node server.js
Starting gRPC server on port 0.0.0.0:50051...Need a client? Try this tutorial to build a gRPC Client in iOS / iOS Swift.
