“Building APIs for APIs” sounds a bit like infinite recursion, but actually I’m talking about one of the cooler aspects of LoopBack: the ability to define a server API that maps to another server. Essentially your API acts as a proxy for another API. There are a lot of reasons you may want to do this, including:

  • Supplementing the set of APIs you already provide. Perhaps you’re a sports company that can provide APIs for every sport but golf. If you can find a third-party provider for golf data, you can then add it to your own library and offer a more complete solution to your users.
  • Modifying API results to fit your needs.  Maybe you want to use an API that is a bit inflexible in the data it returns. By creating your own proxy, you can modify the result sets to return only what you need.
  • To improve performance you can add your own caching layer.
  • Perhaps you want to use an API in your mobile app but don’t want to embed sensitive information, like an API key, in your source code. You can use your own server, and this LoopBack feature, to keep the key hidden in your Node.js code.

The feature we’re discussing is provided by the LoopBack “REST Connector.” When you first begin learning LoopBack, you use a connector that stores data in memory. This is great for testing and quick prototyping. You then switch to a persisted connector, like the MongoDB one perhaps, but in general, you’re still doing the same thing. Your models have basic CRUD with the connector in the back to handle persistence.

The REST connector is different. Instead of handling persistence, it handles the connection to the remote API. Let’s take a look at a basic example of the connector in action.

Begin with an existing LoopBack application (or create a new one!), and then install the REST connector:

npm install --save loopback-connector-rest

As a reminder, a standard LoopBack app includes only the in-memory connector, which makes sense if you think about it. There’s no need to include code for Oracle if you are using MySQL.

Next, create a new datasource:

slc loopback:datasource

For the name, let’s call it swapi. Our demo is going to make use of the excellent Star Wars API which provides information about everything related to the best thing ever (after cats.)

When prompted to select the connector, scroll down to the REST connector:

sbblog

Next it will prompt you for the base URL for the API. Here is where things can get tricky. The Star Wars API supports a number of different end points for movies, ships, characters, and more. You’ll want to specify one particular end point. Why?

Unlike the persistence-based datasources, the REST connector is meant to work with one model, not many. In other words, I may set up one MongoDB database for my application and one datasource for it in my LoopBack application. I’ll then add many different models all using that one Mongo-based connector.

For the REST connector, we need to point to one API and match it up with one particular model. For this demo then we’ll focus on spaceship data because spaceships are awesome.

According to the Star Wars API documentation, we can get a list of spaceships using this URL: http://swapi.co/api/spaceships. So let’s use that for the connector URL value. For the rest of the prompts, just accept the defaults.

swblog2

At this point, your datasources.json file should look something like this:

{
  "db": {
    "name": "db",
    "connector": "memory"
  },
  "swapi": {
    "name": "swapi",
    "baseURL": "http://swapi.co/api/starships",
    "crud": false,
    "connector": "rest"
  }
}

To work with the remote API, we have to define a set of operations that will be exposed to our local model. You can define as many operations as you need, but most likely you will only need one. An operation consists of a template and a set of functions. The template is like a ‘meta’ description for the remote URL. It allows for tokens so that it can be dynamically updated based on how you call the API. The functions aspect is where you define your own names for accessing the remote API. This all sounds a bit overwhelming, but let’s look at a complete example.

{
  "db": {
    "name": "db",
    "connector": "memory"
  },
  "swapi": {
    "name": "swapi",
    "baseURL": "http://swapi.co/api/starships",
    "crud": false,
    "connector": "rest",
    "operations": [
      {
        "functions": {
          "getships": []
        },
        "template": {
          "method": "GET",
          "url": "http://swapi.co/api/starships/",
          "headers": {
            "accepts": "application/json",
            "content-type": "application/json"
          },
          "responsePath": "$.results.*"
        }
      }
    ]
  }
}

The template aspect defines a lot of different parts of how we’ll call the API. It supports a method and headers, which should look familiar to folks. It obviously needs a URL, and finally, we can define a responsePath to ‘parse’ the result. responsePath is using JSONPath as a way to fetch a portion of a JSON response. In functions, we’ve defined a name for how we want to “address” the URL in the template. The empty array is where we define arguments to pass to the API. For now we’re not going to pass any arguments at all, so it is empty. There’s a lot we’re leaving out here, but for now, let’s move on.

To expose our remote API as a local API, we need to add a model. Since we’re working with the starship aspect of the Star Wars API, let’s create a new model called spaceship. When asked what connector to use, select swapi.

swblog3

Then be sure to use Model as the base class, not PersistedModel.

swblog4

Take the rest of the defaults and don’t create any properties.

Now fire up your LoopBack server (or restart it if was already running) and then open your API Explorer, you’ll see the new model:

swblog5

Woot! We’ve added an API that proxies to another API and our users are none the wiser. If you test it, you’ll see the results:

swblog6

Notice there is an array of ships. That isn’t exactly what the remote API returns. You can see for yourself by going here: http://swapi.co/api/starships/

But remember our resultPath argument? That was used to “trim down” and focus on the result we wanted.

Sweet! But let’s kick it up a notch. The Star Wars API returns data that we don’t want to expose to our end users. If we open up starship.js, we can modify the result by using the afterRemote method. Here’s an example of it in action:

'use strict';

module.exports = function(Spaceship) {

	Spaceship.afterRemote('getships', function(ctx, ship, next) {
		var ships = ctx.result;
		ships.map(function(ship) {
			delete ship.films;
			delete ship.created;
			delete ship.edited;
			return ship;	
		});
		ctx.result = ships;
		next();
	});
};

We’ve removed three keys from the ship data: film, created, and edited. This was obviously somewhat arbitrary, but you get the idea. If you restart the server and rerun your API test, you’ll see now that the results are slimmer. Along with removing keys, we could even rename them and add our own. Again, we’ve got complete control over the result.

Now let’s go a step further: Since a new Star Wars film is pretty rare (or it was, now it’s going to be once a year!), we can add our own caching layer. Here’s an example with simple RAM-based caching:

'use strict';

module.exports = function(Spaceship) {

	var cache = {};

	Spaceship.beforeRemote('getships', function(ctx, ship, next) {
		if(cache.ships) {
			ctx.res.send(cache.ships);
		} else {
			next();
		}
	});

	Spaceship.afterRemote('getships', function(ctx, ship, next) {
		var ships = ctx.result;
		ships.map(function(ship) {
			delete ship.films;
			delete ship.created;
			delete ship.edited;
			return ship;	
		});
		ctx.result = ships;
		cache.ships = ships;
		next();
	});
};

All I did was add an object, cache, and store in it the result from the remote API. Then the new beforeRemote function checks if the cache exists, and if it does, return it instead of calling out to the API. I could also store the time I created the cache if I wanted to programmatically expire it.

All in all a pretty powerful feature. There’s a lot more to it and I encourage you to check out the full documentation for examples: REST Connector