Use React Native to post to secure AWS API Gateway endpoint

I am setting up a React Native application that will interface with an authenticated API hosted by AWS API Gateway. Here is how I set up my API to be secured through authentication. I am not sure that this will be used in production, but it is working well for testing.

This post will go over the following:

  1. Setting up a very simple React Native application
  2. Adding a simple button that will later be used to get data from an endpoint
  3. Using the react-native-dotenv module for environment set up
  4. Using the react-native-aws-signature module for authorization
  5. Debugging with react-native-aws-signature

Here is the code for this example on githhub

Setting up a very simple React Native application

Start with a brand new react-native application. To set one up, run:

You should get something in the simulator that looks like this:

Adding a simple button that will later be used to get data from an endpoint

In the index.ios.js file, add Button to the imports:

Replace the existing SampleProject Component with this:

Reloading in the simulator should give you something like this:

If you press the ‘API Request’ link, you should get this:

Using the react-native-dotenv module for environment set up

In a production mobile application, you don’t want to save secret API keys anywhere in the code because it can be reverse engineered. There is a SO post here about it.

That being said, if you are only installing the app on your phone during the testing phase, it is probably fine.

The official react-native-dotenv instructions are here, but this is what I did to set it up.

First, install the module

Add the react-native-dotenv preset to your .babelrc file at the project root.

Create a .env file in your project root directory with your AWS credentials and the host.

Now, let’s set up a really simple class that we will use to interface with our API. This should be at the same level as index.ios.js, and mine is called called SampleApi.js.

Then, somewhere near the top of index.ios.js, import the new class:

Replace the retrieveData function with:

Our full index.ios.js should now look like:

Note: if you change the .env file only, the simulator will not recognize the change and your changes will not take affect.

Using the react-native-aws-signature module for authorization

Now, we want to actually hit the API when the button is pressed. Start by installing the react-native-aws-signature module

In SampleApi.js, add the import for AWSSignature:

Remove the contents of the get() method in SampleApi.js and start by setting up some variables based on the .env file:

Next, set up the header and options. These will be used to generate the authorization details and they will be used in the request to the API.

Then, create a new AWSSignature object and call setParams. This will generate the authorization header, which we retrieve in the next bit of code:

Now, retrieve the authorization information and append it to our header.

Finally, make the request to the API using the header we just created. We are expecting json back, and I have included some basic error checking.

Here is what the SampleApi.js file should now look like:

Modify index.ios.js to set the state to include the return value of the request. Since we are getting a json array back, we have to loop through it to make a readable text block:

After you refresh the simulator, you should be able to press the button and receive a screen that looks something like this:

Debugging with react-native-aws-signature

This AWS troubleshooting guide is helpful, but react-native-aws-signature does most of the work for you, so it can be difficult to determine where your mistakes are.

I got this error when I was including the https:// at the beginning of the host parameter in the header. The full error includes what AWS was expecting for the ‘canonical string’ and the ‘string to sign’.

I figured out how to fix the issue by using the getCanonicalString() and getStringToSign() methods.

 

Create secure endpoints for AWS API Gateway

I am building an application that will rely on the AWS API Gateway for a REST API. I want to make sure that other people are not able to read or write data on the endpoints. I will be using IAM authentication, using the steps below:

  1. Set up an example API
  2. Test that the API works without authorization
  3. Enable authorization on your endpoints
  4. Set up a new User in IAM for API requests
  5. Configure your request to use your credentials

Set up an example API

If you already have an API set up, skip this part.

From API Gateway, select “Create API”.


On the next screen select “Example API” and click “Import”.

The UI will then prompt you to “Deploy API”, if it doesn’t, you can select the option from the “Actions” dropdown. You must provide a stage name for the deploy, I just used ‘test’.

After you have deployed the API, you should see a screen like this, that includes a link to the API.

Click on the link and it will bring you to an info page. We will test the endpoint in the next step.

Test that the API works without auth

Now, make sure you can get to your endpoints without authentication. You can test the GET endpoint by appending ‘/pets’ to your url, either in a browser or with an application like Postman.

The browser output will look something like this:

The Postman output will look something like this:

Enable authorization on your endpoints

Now, let’s lock down the API so only we can get to it. In API gateway, select the /pets GET resource:

Then go to the configs for the Method Request and select ‘AWS_IAM’ under the Authorization setting.

In order for the changes to take affect, you have to use the “Deploy API” action under the “Actions” dropdown. You can deploy over an existing stage, or create a new one.

Now when you try to hit the endpoint via the url, you should get this response

Set up new User in IAM for API requests

Go to your IAM setup and add a new group with the following permissions: AmazonAPIGatewayInvokeFullAccess

Set up a new user that is a part of the group you just created. You won’t need to log in as that user, so don’t set up a password.

On the last screen, the credentials will be provided, make sure you capture both the Access Key ID and the Secret access key, the secret key won’t be displayed again.

Configure your request to use your credentials

In order to get to our endpoint, we need to include authorization values in the header. These are calculated at the time of the request to make sure other people cannot just reuse your headers to gain access.

Here are the full instructions from Amazon, but Postman makes it pretty easy.

Under the “Authorization” tab, select “AWS Signature”

You will be taken to this screen where you can enter the configuration:

The configuration includes:

  • AccessKey – This is the Access Key ID that we copied in the previous step, it is the shorter of the keys
  • SecretKey – This is the “Secret Access Key” that was copied in the previous step, it is the longer of the keys. (It should never be shared)
  • AWS Region – This is ‘us-west-2’ for me, but may be different for you
  • Service Name – this should be ‘execute-api’

After you have entered the values, press “Update Request”. Now if you try to access the endpoint, you should get the data as before.

Check out the values that were included in the “Headers” tab. The ‘X-Amz-Date’ and ‘Authorization’ will change with each request and they are what Amazon verifies on their end to ensure you have up to date permissions.

Next Steps

In order to use the API from an application, you will need to systematically add the headers to your requests. I am using react-native-aws-signature for my React Native application.

Using Amazon Web Services to create a serverless web app

Amazon Web Services (AWS) provides resources that can take the place of a traditional webserver. A huge advantage to this approach is removing the need for the developer to maintain servers, and to allow for easy scaling depending upon use.

In my project, I want to set up mobile app that will use AWS as a backend. I plan on using the following services:

  • Lambda – allows you to run arbitrary code without setting up a server
  • Relational Database Service (RDS) – hosted database. I will be using postgresql
  • Step Functions – Connect services together using visual workflows
  • API Gateway – Create API endpoints to allow your backend to be used outside of AWS

Other technology I will be using

  • Node.js – I am new to JavaScript and want to learn more about it
  • PostgreSQL
  • Bookshelf.js/Knex – I want a well supported ORM that supports PostgreSQL. Bookshelf.js seems robust and since it is built on top of Knex, I can always fall back to Knex for unsupported functionality.
  • node-lambda – Make it easier to manage AWS Lambda deployment

Here is an outline of my steps

  • Set up PostgreSQL and Bookshelf.js
  • Set up hosted database using RDS
  • Create simple lambda that reads from RDS
  • Integrate API Gateway with Lambda
  • Connect 2 Lambdas together with Step Functions
  • Integrate API Gateway with Step Functions

Set up Bookshelf.js for Node.js

Intro

I am going to walk through setting up a brand new Node.js project using Bookshelf and PostgreSQL. At the end, we will have a simple Node.js project that can insert and retrieve data from a database.

Here is what this post will cover

  • Set up a new Node.js project
  • Install necessary libraries to interface with PostgreSQL database
  • Set up process to create a database table
  • Use Bookshelf to
    • insert data
    • retrieve data

The code for this post can be found here.

Why Bookshelf.js?

Bookshelf.js is an Object Relational Mapping (ORM) for Node.js that allows developers to more easily interface with a relational database. It allows the dev to focus on higher level functions, rather than low level queries. It takes care of some basic attribute mapping to make data models simpler. It also allows for the same code to be used against different relational databases in case you want to change the underlying database.

I chose Bookshelf.js because it is well maintained, well documented, and it is built on Knex.js, which is a SQL query building framework. I can use Knex for queries that are not easily supported in Bookshelf.

Install necessary software

Node.js

I am using Node.js 4.3.2 because 4.3 is what is supported in AWS Lambda. If you don’t have node installed, follow these instructions.

PostgreSQL

Instructions on how to install PostgreSQL. I use Postico to easily interface with the PostgreSQL.

Before we go on, start the database.

Set up a new Node.js project

Set up the project directory and initialize the project. When you run npm init, it will prompt you to answer some questions, you can use the default values or set your own, at this point they don’t matter too much.

Install the node modules that we will need, the -save option will add the modules to the package.json file.

Here is what each module does

  • bookshelf – the ORM we will be using
  • knex – required by bookshelf, but in the future it may not be included automatically
  • pg – the postgresql module
  • dotenv – a utility that makes dealing with environment variables easy

Here is our current package.json

The dotenv module reads from a .env file in the project directory. This will set the environment variables. Right now we only need to set which environment we are in.

Hello (node) world

Let’s create a simple javascript file named index.js that will show that we have node working.

Now, make sure that it works

Create a database

Let’s create our database, and verify that it exists.

Create table in database

Set up a knexfile.js config file for Knex. This tells Knex how to connect to the database, notice there are entries for development and production. Right now we care most about development.

Now create a migration that will define our table. The resulting XXXXX_create_table.js file is just a blank template that we will fill out. Yours will have a different date and time in the name.

Later, we will be using the test jsonAPI here. To keep it simple, let’s just create a table that replicates the post resource, which has data like this:

Edit the XXXXX_create_table.js file to include the columns we want our table to have. Mine looks like this

This will create a table called ‘articles’.  We have an id field that will automatically increment when we insert new records. The remote_id is the id from the test service, we want to be sure that we don’t have duplicates, so we use the .unique() method. The table.timestamps(); command will add a created_at and updated_at column that are automatically populated when a record is added or updated. I have excluded the user_id that appears in the test API to keep the table simple.

The exports.down function defines what should happen if we rollback the migration, in general this should do the reverse of what the exports.up function does.

Now, run the migration with knex migrate:latest, and make sure the table exists.

Set up Bookshelf

Start with a bookshelf.js file that will allow us to use bookshelf in other files.

Create a model class for the articles table that we can use to interface with the table. By extending the bookshelf.Model, we get functionality that allows us to access the attributes of the table as methods on the Article variable.

Insert data into the table

Edit index.js to create a new entry in the articles table

Run index.js and make sure that it adds data to the table.

Retrieve data from the database

Now, modify index.js to also query the database for the article we just inserted. Below we have added the getInsertedArticle function, commented out the code that just did the insert, and added some new code to insert and then retrieve the record.

Done

That is it. There are many more things we can do with a database, like create relationships, do complex joins, etc., but this is a good starting point.

Here is a link to the code from this post.

Working with servos and an Arduino Nano clone

Over the past few weeks I have been working on building the software to allow my RC Sailboat to sail autonomously. To keep the form factor as small and light as possible, I bought a cheap Arduino Nano clone to use in place of an Arduino Uno.

Nano with rudder and Sail
Nano microcontroller with servos for sail and rudder control

Obstacle 1: drivers

My first obstacle was getting my mac to communicate with the board. The basic issue is that the WCH340G USB interface chip on the clone is different than the FTDI FT232RL used on the authentic Arduino Nano. There are many sites with instructions and links to drivers, but this one had the most up to date driver that was signed. The older drivers were not signed and required me to bypass OSX’s security to allow unsigned drivers. direct link to driver download

Obstacle 2: consistent power

I had some semi-working software running on the Uno that would adjust a sail and rudder servo. When I swapped in the Nano the servos generally worked, but would sometimes move wildly. Since the Nano and Uno are very similar (chip, ram, clock speed, etc.) I figured the issue was going to be with the power or maybe the WCH340G USB chip.

To troubleshoot I used my voltmeter to see how the Nano voltage changed while adjusting the servos. When not moving the servos, it put out 4.8V on the 5V pin and while moving the servos simultaneously it would drop to ~4.2V. The servos expect 4.8-6V so this could contribute to the issue. As reference, the Uno put out 5V, I didn’t test it while moving servos.

In the past I have had issues with macs not putting out enough voltage through USB, and I have a cord that includes 2 plugs, one for power and one for power+data. The cord is similar to this one.

I swapped out the cords and the servos started behaving as expected. My guess is that a combination of data transfers (to signal the servo change) and keeping a consistent power supply were too much for the single cord.

Since I want to run the system without my computer, I think the USB cord is not the best solution. A couple of alternative options are a separate power supply for the servos, or using a capacitor to smooth out the power.

To keep down weight and complexity, I took the capacitor approach. I found many sources that said a minimum of 470uF was the correct capacity and that the voltage rating should be 3x what you have in the system. I added a single 1000uF/25V capacitor with the old cord and the servos behaved well. The servos also worked well with three 100uF/16V capacitors in parallel, but I prefer the single capacitor solution.

How to display Google Earth maps on website

When I hike, I track my routes with a GPS and import them into Google Earth. This works great on my personal computer, but it doesn’t allow me to easily embed the maps on a webpage. To allow others to view my maps, I import them into Google Maps and then embed these custom maps on my webpage.

Steps to embed Google Earth routes on webpage using Google Maps

  1. In Google Earth, right click on the route you want to save and select “save place as”save place as
  2. Save the file as a .kml
  3. Sign in to Google Maps and open the menu on the left side of the search bar by clicking the three line icon
    menu
  4. Within the menu, select “Your Places”, then the “Maps” tab.
    your places
    Maps tab
  5. This page will show all of your custom maps, click the “Create Map” link at the bottom of the page
  6. The map edit screen will open with one map layer created for you. Click “import” under the layer. This will let you select the .kml file that you created earlier.
    import
  7. If you want to add multiple routes to the same map (for instance a multi-day hike), add a new layer with the “Add layer” link, and import a separate .kml file.
  8. You can change the look of the map. These changes will be reflected on the embedded map if you change them in the future.
    • To change the color of the route, mouse over the name at the bottom of the layer section (“Enchantments Through hike” in the images below) and click the paint bucket icon.
      color
    • To add a label to the route, click on the “Individual styles” link and select the label you want to show.
      label
    • You can change the type of map (terrain, satellite, etc.) from the “Base Map” link under the layers section.
  9. In order to embed the map, you will have to make the map public through the “share” link.
    public
  10. Once the map is public, you can select the “Embed on my site” item from the menu.
    embed on my site
  11. Insert the provided code on your site, and you are done.
    copy code

 

Here is an example of an embedded map

Connecting the Shaft Encoder to Fio

After figuring out my voltages, I wanted to connect the shaft encoder to the Fio and make sure it worked.

I started by creating a diagram (Diagram 1) for the breadboard.

Breadboard Fio with Step Up and Encoder
Diagram 1: Fio with Encoder and Step Up Breakout.

I built out the circuit on a breadboard and added it to my mess of wires, which included the accelerometer I had already connected to the Fio.

fio with breadboards
Fio with accelerometer and shaft encoder

I created a simple program to display the values from the shaft encoder to a terminal on my computer. The most up to date code is available here.

The values displayed on my terminal vary between 0 and ~1023. Sometimes I get values greater than 1023 and the values change when the encoder isn’t moving. I am not sure what is contributing to this variance, but in the future I will try to eliminate this noise with software by averaging the values over time and limiting the max to 1023.

Preparing to Add a Shaft Encoder to Arduino Fio

The Arduino Fio is a 3.3V board, but the MA3 Shaft Encoder uses 5V. To compensate, I needed to step up the voltage for input, but then step down the output to the Fio input.

I didn’t know exactly how to do this, so I started simple and worked my way up.

Getting 3.3V out of a 5V input

First, I wanted to get 3.3V out of an input of 5V. This would be the part connecting the encoder output to the Fio input.

My Arduino Uno board can output 5V or 3.3V, so it served as my power source. In my images here I have just represented a generic power source to keep the diagrams simple.

Through research, I determined that I would need to build a voltage divider, which uses the formula: Vout = Vin * (R2 / (R1 + R2))
R1 is the resistance between the input and the output, and R2 is the resistance between the output and the ground. Since my desired Vout is 3.3 and my Vin is 5, I was looking for resistors where R1 was half the resistance of R2.

Since the ratio between the resistors is key, I could have used resistors with large or small resistance. In voltage dividers, the smaller the overall resistance the more accurate the output, the tradeoff is that more energy would be wasted. Since I am using battery power, but also care about accuracy, I wanted something in between. This post describes it pretty well.

Also, the formula assumes a perfect environment where the only resistance is on the resistors. In my circuit, the current drawn from the input pin would reduce the ratio between the resistors.

I am using PWM input, which is measuring the time between the peaks in voltage, so it isn’t super critical that I have exactly 3.3V. Based on this post I went with an 18k and 33k resistor, which should give me a peak around 3.23V.

I built the circuit shown in Schematic 1.

Using my multimeter I measured

  • 5V between the 5V input and ground
  • 3.3V between A and ground
5V to 3.3V Schematic
Schematic 1: 5V to 3.3V at “A”

Suppling 5V power to encoder, with 3.3V output

My next step was to add the encoder to the circuit to make sure it could receive 5V of power, but have the output reduced to 3.3V.

Schematic 2 shows the circuit, though I am not sure that the image used for the encoder is appropriate.

I measured

  • 5V between the 5V input and ground
  • Max 3.15V between A and ground (depending on position of encoder)
5V to encoder to 3.3V Schematic
Schematic 2: 5V Encoder input with 3.3V output at “A”

When I first measured at A, it was a lot less than 3.3V. I then turned the encoder and found that I could increase the voltage up to 3.15V. The encoder is using PWM and the voltage should be jumping between 0 and 3.15V with varying spaces between the highs and lows. My multimeter must average the voltage it is receiving, rather than show the peak.

I think the 3.15V is probably due to variations in individual resistors.

3.3V input, stepped up to 5V, with a 3.3V output

My final step before moving to the Fio was to use my 5V step up board to take a 3.3V input and output 5V to the encoder.

Schematic 3 shows the circuit I used. The part number on the step up should be NCP1402, not 1400.

I measured

  • 5V between the input to the encoder and the ground.
  • 5V between left side of R1 and ground.
  • Max 3.15V between A and ground (depending on position of encoder)

Step up and encoder schematic

Schematic 3: 3.3V input, stepped up to 5V for encoder, with output reduced to 3.3V at “A”.

Next Step:

Connecting the Shaft Encoder to Fio

Arduino Fio and XBee

I had several criteria for selecting the onboard Arduino.

  1. It should be as light as possible to keep boat weight down. This includes the battery.
  2. It should have low power consumption to reduce the number of times I have to access it in the boat hull.
  3. It should be as small as possible due to limited space in the boat.
  4. It should be able to gather information from several sensors.
  5. It should be able to transmit the data wirelessly through an XBee.

The Arduino Fio seems like an ideal choice because it fits the criteria and they are pretty cheap.

I had previously set up my XBees (Pro Series 1) to communicate with each other using an Arduino Uno (on shore) and Arduino Mega (on boat). I thought I would be able to seamlessly upload the sketch that was used on the Mega onto the Fio, but no such luck. I was getting only garbage characters coming through. After hours of rechecking XBee settings and stepping through iterations of XBee/Computer/Arduino setups, I found that the issue was how the Fio treats the XBee serial connection.

On the Mega and Uno, I use SoftwareSerial to communicate with the XBee. On the Fio, you can write and read directly from Serial, like the code below. The most up to date code is here.

This makes sense when you remember that the Fio has a restriction where you can use the UART connection or the XBee, but not both at the same time.

At first I was hesitant to tackle uploading the sketches over the XBee, but the tutorial here was great. A couple pieces of information I wish I knew when I started:

  • Use a baud rate of 57600 so you don’t have to reprogram the XBees.  The tips section mentions this, but the description was confusing to me.
  • You don’t have to reconfigure the XBees when you are done uploading your sketches. The XBees can still be used to communicate Arduino to Arduino when set up to bootload to the Fio.

Arduino Enabled RC Sailboat

While in the HCDE program at University of Washington, I took a Physical Computing course which introduced me to Arduino.

My current project is to incorporate an Arduino into my 30 inch radio controlled sailboat that I built a few years ago. My end goal is to integrate the Arduino to automate basic sailing controls like sail trim and rudder control.

boat_deck

boat_cockpitTo start, I want to collect information from the boat and transmit it to shore.

  • Wind Direction in relation to boat
    • Gathered with a wind vane connected to a rotary encoder.
    • Will show the apparent wind in relation to the boat. Apparent wind will change based on boat speed, this is fine because the sails are trimmed to the apparent wind.
  • Boat Heel and Pitch
    • A three axis accelerometer will detect boat heel as well as pitch.

The system consists of two Arduinos, one on shore and another on the boat.

  • The on-boat microprocessor gathers data from the sensors, does some data formatting, and then sends the data wirelessly to the on-shore Arduino.
  • The on-shore microprocessor receives the data and displays it on an LCD screen.

In the future, I may include additional data

  • Sail Position
  • Wind Speed
  • Boat Speed
  • Location of (or proximity to) other boats
  • Location of course markers