Coming Soon: LoopBack 3.0

Since July 2014, when we released LoopBack version 2.0, we have added over 90 new features while keeping full backwards compatibility. To achieve that, we had to make some compromises and elect suboptimal solutions over those that would introduce breaking changes. Additionally, as an increasing number of LoopBack users put the framework into a wider variety of uses, we have learned that some parts of LoopBack need a redesign to support new needs in an easy-to-use manner.

As a result, we feel that the time has come to apply what we learned by tidying up the code and public APIs, even at the expense of breaking backwards compatibility.  Read more

StrongLoop and sl-blip

Recently, some members within the the node.js community expressed concern about StrongLoop’s use of sl-blip to collect data. We’ve heard and understand these concerns and want to explain what data we collect, how we use it, and how we plan on addressing the concerns that were raised.

Read more

Announcing StrongLoop API Gateway Powered by Node.js

This morning, we announced the private beta of StrongLoop Gateway. We’ve never seen so much interest in a product that we haven’t even released or promoted. Of course, for any business, this is a “good problem” to have. It’s been rewarding to validate that the StrongLoop Gateway will meet the needs of so many people. It’s also rewarding to see that we have an advantage over competitors because the StrongLoop Gateway is tailored to the needs of API development and not – management.

Let’s start from the beginning and explain the overloaded term “gateway.”

Overview

An API gateway externalizes, secures, and manages APIs. It is an intermediary between API consumers (clients) and backend API providers (API servers).

Screen Shot 2015-07-29 at 2.13.14 PM

In this intermediary position, the API gateway performs several functions depending on the needs of the enterprise, as summarized in the table below.

Function API Gateway Role
Security Acts as both provider and delegator to authentication, authorization, and auditing (AAA) sources within the enterprise as the first intercept to establish identity.
Mediation and Transformation Mediates between protocols and transforms portions of the API payload (both header and body) for clients that have fixed and/or specific requirements for consumption.
Infrastructure QoS Performs infrastructure-level API consumption functions required by client such as pagination, throttling, caching, delivery guarantee, firewall,  and so on.
Monitoring and Reporting Instruments APIs to fulfill service-level agreements (SLAs) through the monitoring of APIs and also injects metadata to report on API usage, health, and other metrics.
Aggregation Composes coarse-grained APIs (mashups) from fine-grained micro-APIs to fulfill specific business case operations through dynamic invocation and construction.
Virtualization A layer of abstraction that virtualizes API endpoints and acts  as a reverse proxy to API server host instances for high availability, security and scale.

Read more

Getting Started with the StrongLoop Node.js API Gateway – Step by Step

The goal of this tutorial is to understand the design principles behind StrongLoop Node.js API Gateway and how you can configure it step by step to test your use cases.

The following diagram illustrates the API Gateway architecture.

APIGateway

Request/response pipeline

The gateway creates a “pipeline” of layers for API requests and responses. The layers of the pipeline correspond to different stages of processing API requests, and orchestrating and constructing API responses.

loopback-pipeline

The API Gateway “pipeline” has four layers:

  • Transport layer: Listen for incoming messages and un-marshal them into request objects. For example, LoopBack uses Express to accept HTTP requests and produce HTTP responses.
  • Remoting layer: Resolve the request objects to Node.js objects as receivers and methods to be invoked.
  • Model layer: Delegate data access requests to connector implementations
  • Connector layer: Dispatch CRUD and other model methods to database driver calls to the backend system.

Typical flow of an API request

  1. Client application requests an access token from the OAuth 2.0 authorization server.
  2. Client application sends an API request with the access token to the OAuth 2.0 resource server.
  3. Resource server validates the access token (including expiration, scope, client id and resource owner) to either allow or deny the access. If deny, an error response is sent to the client application and the flow terminates.
  4. Metrics middleware captures the API usage and updates the metrics.
  5. Rate limiting middleware introspects the API request and check the rate limits based on various keys. If it exceeds one of the limits, , an error response is sent to the client application and the flow terminates.
  6. Proxy middleware looks up the routing configuration to decide if the request should be handled locally or forwarded to a remote API server.
  7. API server processes the request and produces a response.
  8. Response is received by the proxy middleware, and then sent back to the original client application.

Here are the steps to get the Gateway up and running

First of all we assume you have StrongLoop installed. If not, do the following:

The StrongLoop Gateway is currently under private beta. Please contact sales@strongloop.com to have a license provisioned. After receiving confirmation that a license has been provisioned, run slc arc --licenses to retrieve the new license key. If you are provisioning the Gateway to a remote Linux server, please make sure you have StrongLoop Process Manager installed on that box. Please check: http://strong-pm.io. If you are testing on a local box, process manager is already installed when you did npm install -g strongloop.

Step 1: Setup a backend API/Resource server and a RESTful client

API Gateway is a configurable middleware module. We have built a sample application to configure and test the gateway. Clone the demonstration project as follows:

In this project, we have an API backend called notes-app-plain. This application is a standard loopback application. To set it up from scratch, please follow https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-plain. If you are an experienced Loopback user, you can skip this tutorial and just setup any loopback app with it’s API endpoints exposed over REST.

The below diagram depicts what we are setting up logically, the Resource Server (in OAuth) terms, being our API Server

You can simply run it by

Once the notes are retrieved, the web server displays it to the user:

Now that you are familiar with the Loopback backend, lets move to our main Gateway repository, called the “notes-app-gateway”

When you cloned the “strong-gateway-demo”, this repository was already included in that project, so you can just

We have some configuration scripts setup to make life easier. To setup the backend API app within the context of our gateway, learn how to build from scratch using https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-1—copy-files-from-notes-app-plain-to-notes-app-gateway or just run :

This script creates the Resource/API Server, resets StrongLoop Process Manager, builds and deploys the app and boots it up.

If you are really interested in what the setup script looks like, here is a sample

Step 2: Proxy requests through the auth server

In this step we will setup a proxy or an Authorization server, through which the requests & responses from the client to the Gateway will be handled.

To learn how to configure this from scratch, please check https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-2—proxy-requests-through-the-auth-server or simply run the following:

Browse to localhost:3001.You should be redirected to localhost:3101 and you should see:

After configuring the proxy (the setup script already does it for you), Browse to localhost:2001 and you should see:

While the results look identical to notes-app-plain, the client is now making requests to the auth server, which is acting as a proxy to the resource server. The client is now fully decoupled from the resource server.

Step 3: Enable security on the auth server

Now we will enable security (Authentication and ACL primarily) on the Auth Server. To learn how to configure this from scratch, please check https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-3—enable-security-on-the-auth-server or simply run the following:

After the config is setup (the install script does it for you) browse to localhost:2001. You should see:

The server is now denying requests from unauthorized users.

Step 4: Enable the OAuth2.0 Authorization Code flow on the client

The Gateway supports multiple Authorization flows. Namely:

Since the auth server is now blocking the client from retrieving notes, we need a way to authenticate it. To do this, We will use the OAuth 2.0 Authorization Code Grant flow to authenticate client requests. During this process we will also setup SSL certificates, routing and redirection.

To learn how to configure this from scratch, please check https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-4—enable-the-oauth-20-authorization-code-flow-on-the-client or simply run the following:

After the configuration (the script does it all for you) Browse to localhost:2001. You should see:

Click the “here” link. You should see:

Notice in the URL bar, you’ve been redirected to https://localhost:3202. Log in with the bob:secret credentials by clicking “Submit”. You should see:

Once you are authenticated as bob, the auth server asks if you want to allow demo-app access to your account.

Note : bob and demo-app are preregistered sample data values included with the auth server. StrongLoop API Gateway includes this data out of the box for demo purposes.

Click “Allow” to approve the permission request. You should see:

Step 5: Policy Management and enforcement

In addition to the basic gateway features, the StrongLoop API gateway also provides set of features known as policies.

In this step, we are primarily setting up rate-limiting policies. These policies can be based on user, application, user&application, IP Address, etc. Volumes are controlled by time buckets which are also a controlling parameter in the policies.

To learn how to configure this from scratch, please check https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-5—strongloop-api-gateway-policies or simply run the following:

To try it out :

Run the rate-limiting script:

You should see:

Notice the headers show the maximum number of requests (1000), the number of remaining requests (999, 998, …) and the time left (in ms) until that count resets (60000, 59982, …). See the official docs for more info.

Step 6: Check API Analytics

The StrongLoop API Gateway can gather analytics related to API usage. We will be using StrongLoop Arc (Arc) to view the collected data.

Start the auth server using PM:

Start Arc:

At this point, strong-arc will automatically open your browser.

Go to the “API Analytics” module

Arc_analytics

 

Enter the following info on the Process Manager tab:

pm_setup

Hostname: localhost
Port: 8701

Click the “Load” button on the left side of the API Analytics screen to load the graphs.

api_aggregation

Note: The graph does not update automatically, you will need to click the “Load” button to refresh the data. This will be addressed in a future update.

To create analytics to gather, we’ll reuse the rate-limiting script from the previous section:

This script will make a large number of requests and cause changes to the analytics graph data.

Go back to the Arc window and view the analytics again. Click “Load” again to refresh the data. You should many changes that correspond with the activity caused by the rate limiting script.

You can also view analytics broken down by time – hourly

hourly_breakdown

 

Once you drill down into the hourly breakup, you can see distribution of the API by response times and number of requests.

Analytics_graph2  Analytics_graph1

Step 7 (optional) – Use MongoDB for the auth server’s data source

We will not run this step. In case you are interested, please check the configuration as below. These step allow us to persist the role and api mapping metadata into an un-structured database like MongoDB or an objectDB which ships out of box with Loopback.

https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-6—use-mongodb-for-the-auth-servers-data-source

Step 8 (optional) – Use MySQL for the auth server’s data source

We will not run this step. In case you are interested, please check the configuration as below. These step allow us to persist the role and api mapping metadata into a structured database like MySQL, SQLServer or Oracle. Please note you will have to use the appropriate loopback connector for particular databases.

https://github.com/strongloop/strong-gateway-demo/tree/master/notes-app-gateway#step-7—use-mysql-for-the-auth-servers-data-source

Get Started Evaluating API Gateway

Send us an email to get your free evaluation started!

What’s next?

 

Announcing Transaction Tracing for Node.js Beta

At StrongLoop, we develop tools to support development and operations throughout the entire lifecycle of API development. Initially, we released Arc Profiler to help you understand performance characteristics of your Node application. Next, we added Arc Metrics to provide real-time visibility into your staging and production environments. Today we’re announcing the latest Arc module, Tracing, (currently in public Beta) that enables you to perform root cause analysis and triage incidents in production. (You can watch a short overview and demo of the Tracing module here.)

Those who have cut their teeth on the seemingly endless iterations in the dev lifecycle  will understand this satirical spin on Dorothy’s line from the Wizard of Oz:

“Toto, I don’t think we’re in staging anymore…  There’s no place like production… There’s no place like production…”

Simulated load, automated testing, and all the CI magic in the world won’t prepare you for the “gotchas” that can happen in production.  If you’re lucky, you’ll have a canary that keels over as soon as you enter the production mine.  But what then?

The answer is Tracing.  The Arc Tracing module provides the ability to call in the artillery when you need it.  If you see something of interest in Metrics, open up Tracing and you’ll be shown a timeline of memory and CPU usage.

Understanding the Timeline

tracing21

Locate the point of interest—more often than not a CPU or memory spike in the form of a peak in the line—and start to drill down. When you’ve located the incident, you’ll want to freeze the time slice by clicking on the chart drawing your black line denoting a time slice and starting your drill down.

Read more