Lab 02b: Logging Walking

The questions below are due on Tuesday February 13, 2018; 08:25:00 PM.


Partners: You have not yet been assigned a partner for this lab.
You are not logged in.

If you are a current student, please Log In for full access to the web site.
Note that this link will take you to an external site (https://oidc.mit.edu) to authenticate, and then you will be redirected back to this page.

POSTing Music

 

Goals:

Today we're going to pick up from where we left off in the previous lab. At the end of Lab 02A, we had a device that could count steps. What we're going to do today is merge your working system with two state machines (if you designed your step counter as a state machine, that means by the end of lab you should have three of them running on your system): The first new one will report your number of steps to a website which then accumulates that into a database to keep track of your total number of reported steps. The second state machine for today will be responsible for controlling what displays on the OLED: in one state it should display your current batch of steps (similar to Lab 02A), and in the second state it will display your user's total number of steps ever reported along with the current step leader in all of 6.08.

We are working with a shared database today, which all students can see. Any attempt to or successful execution of posting anything offensive, inappropriate, or malevolent will have serious consequences. Please see the Student Responsibilities Page for more information. Do not ruin this for the class.

REALLY IMPORTANT: There is a bug in the I2C library (used to talk with the IMU) that comes out when working with WiFi. We think the solution is the following: Download the file FOUND HERE. Unzip that file and do the following:

  • Go into your \*/Documents/Arduino/hardware/espressif/esp32/libraries/Wire/src directory and replace the Wire.h and Wire.cpp with the versions found in the folder we provide.
  • Go into you \*/Documents/Arduino/hardware/espressif/esp32/cores/esp32 directory and replace the three files esp32-hal-i2c.h, esp32-hal-i2c.c, and esp32-hal-log.h with the versions in the folder we provide.
  • Restart Arduino
  • Life should be good.

While we'll give you an example file below in the lab, there is no final code skeleton, so you'll need to produce that by bringing together other pieces that you've been developing. In creating your GET and POST infrastructure today. Carefully pull pieces from other files you've already developed, and test as you build it up. It'll make it far easier to identify errors.

One common type of IoT device acts by acquiring sensor information from the world, performing some computation on the information (sensor conditioning), and sending information to a server (aka the Cloud). The server is nothing more than a computer sitting in a room somewhere. That server stores the data from the device in a database, along with data from many other connected devices, and performs computation on that data if desired. It may then send that information back to the device or share it with other devices (or the world).

We're going to implement a similar system with our step counter. Our "cloud" is our personal class server, iesc-s1.mit.edu, a computer in Building 34, which will store all data sent by the embedded systems in the class. In Exercise 03, you'll start to work on the server itself, but as we don't want to go too crazy too quickly we'll leave that alone for today. Instead, we'll focus on developing the mobile, embedded part of the IoT system.

1) HTTP POST

Last week we used the HTTP GET method to receive and partially display fascinating numerical facts from our server1. This week, we'll again use GET request, but we'll also introduce a new HTTP verb, POST. In web browsing you use the POST method a lot less than the GET method, but one example where a POST method is usually involved is when you fill out some sort of web form. Often these will pack up all the data in the form and send it to the form-processing script as a POST request.

The purpose of an HTTP POST request is to POST data to some sort of persistent storage mechanism (a database). POSTs can support large files. With this in mind, we can now reevaluate a bit of how we understood GET requests from last week. A GET request should generally be an action that "gets" information from a server, whereas a POST request should be a way to contribute or place information on the server. There are more complicated and/or official definitions, and, as we'll see later in the class, these rules aren't necessarily always enforced, but it is a good way to start working with (and thinking about) them.

Here is what a typical POST statement looks like (If you remember GET requests from Lab 01B last week you'll see it is very similar). Note internet.com is an actual website, but we're just using that here as a generic server name.:

POST /folder/spot HTTP/1.1
Host: internet.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 22

thing1=value&thing2=89

The POST request has two parts, the header and the body. The general format is: first comes the header, then a blank line, and then finally the body. This should seem familiar from studying the HTTP response format from last week (remember there it was also header, blank line, response body). The POST header will provide specific information about the POST in general as well as details regarding the POST body which the host/server uses to parse and make sense of the incoming data. Let's analyze it line by line.

First, after establishing a connection with the host of interest, which is generic server internet.com, we send the following, which contains the POST HTTP verb, the URI where we want to POST and the HTTP version number.

POST /folder/spot HTTP/1.1

Following this, the next line specifies the host, which is

Host: internet.com

So far, so similar, to the GET request format. Now new stuff will appear. Next is a new line specifying how the data being POSTed is formatted:

Content-Type: application/x-www-form-urlencoded

The host uses this line to determine how to interpret and parse the incoming data. We will usually use either this formatting, application/x-www-form-urlencoded, or application/json in this class, although there are many more formatting specifications that could be used.

Next we have a line telling the host how long of a POST body to expect. This will of course be determined by the body which you create down below. In our particular case, our POST body is 22 characters long (go ahead count it to convince yourself):

Content-Length: 22

Finally, after our blank line to indicate the end of the request headers (remember the blank line from last week), we'll then add our actual body, consisting of the data we want to post. In this formatting specification, we add the data in key-value pairs here: the value of thing1 is value and the value of thing2 is 89, separating out the pairs with ampersands (&)

thing1=value&thing2=89

And then we're done.

2) Step Logging

We are going to integrate our system with one of the class servers where we're serving an application for tracking everyone's steps located at http://iesc-s1.mit.edu/608dev/stepcounter". We'll use this application in several different ways:

  • If you just go and visit this page in the browser, http://iesc-s1.mit.edu/608dev/stepcounter, which is performaing a GET request via browser with no query arguments specified, it'll render a table listing the entire class step-count leaderboard.
  • If you perform a GET request at http://iesc-s1.mit.edu/608dev/stepcounter with a single query argument user (so .../stepcounter?user=jodalyst where jodalyst is my username, for example), it will return a string in the following format user:user_steps&max_user:max_users_steps listing the queried user's current total steps and the lead user's name and total steps (the user with the most steps).
  • If you perform a POST request to the application using the format discussed below, the application will take the POSTED number of steps, add them to the specified user's steps already recorded, store that in the database, and then return the total user's steps taken so far.

An example POST format would look like the following. Let's say your username is rober_donat and you just took 39 steps. You'd want to generate the following HTTP POST to http://iesc-s1.mit.edu.

POST /608dev/stepcounter HTTP/1.1
Host: iesc-s1.mit.edu
Content-Type: application/x-www-form-urlencoded
Content-Length: 26

user=robert_donat&steps=39

Download the example POST code for today, upload it and study what it is doing. You should notice that the code is nearly identical to the WiFiGetter.ino from Lab 01B from last week except that it is implementing a POST request. In particular you should see that it is sending a POST request to the stepcounting site which states that the user test_user has taken one step. By viewing the Serial monitor you can see that the total number of steps submitted for test_user is returned in the response body to the POST request (compare this by visiting the page in the browser). Also be aware that everyone else in the class is running identical code at first so everyone is reporting that test_user has taken one step.

With all this in mind, your goal in this first part of the lab is to build a system that does two things:

  • Upon powerup it automatically connects to the WiFi, and starts to count your steps (use your step-counting code from Lab02a).
  • Upon pressing your system's button, it POSTs your current number of steps to the step counting application for a username of your choice (again, do not put offensive material here). Because the site has a persistent storage mechanism in its database, it will take over the responsibility of tallying all reported steps, and after you've POSTed your steps, you should reset your step counting variable to zero. In effect you'll make your local step counter tally be a recent step counter, and the site will be the cumulative step counter.

A rough sketch of the state machine we want is shown below. Note that things like button clicks are left open for your interpretation—you can act upon the button press or upon the button release, for example. We are expecting you to fill in these kinds of gaps depending on what you want.

The step POSTing state machine.

This state machine should coexist with your previous code, and by making them both non-blocking implementations, it should be able to be responsive to button pushes while still keeping track of steps. Also make sure to use a unique username (like...I don't know...your kerberos) in order to make sure you and some random person aren't clashing in the database.

The POST functionality as written is inherently blocking, but with good network connection should be able to execute in a 100 milliseconds or so, so for today we don't need to worry about this time delay and we can assume a POST event is sufficiently non-blocking.

Checkoff 1:
Discuss HTTP POST requests with a staff member and how you will integrate them with your current step-counting code.

When you are ready, demonstrate your working system to a staff member via the checkoff below.

Checkoff 2:
Show your working system to and discuss its implementation with a staff member.

3) Comparing Ourselves to Others

Now, we want to be able to display different information on our system. We know we can obtain class-wide rankings by just going to the site (via a regular GET request). What we'd now like is a separate way of doing that on our device where it either displays your current stretch of steps that you're taking, OR the total amount of steps that you have taken since the beginning of time and the current leader in the class along with their total steps.

To start getting things integrated with your embedded system, grab a second button, hook it up to another digital input of your choice on the ESP32 (choose an unused one, we recommend IO2. IO19 can get weird with WiFi running), and build a second state machine in such a way that it runs in parallel to the first that you built today. This way, we'll be able to display two different bits of information on our OLED. Our augmented system should have the following (additional) properties:

  • Upon startup the OLED will display the current number of steps since the last resetting in real-time.
  • Upon pressing the second button, an HTTP GET (to the same URL as earlier) will be performed, including a query argument specifying a user. For example if Robert Donat wanted to do this, he'd do:
GET /608dev/stepcounter?user=robert_donat HTTP/1.1
Host: iesc-s1.mit.edu

A query argument is anything placed following a ? at the end of a URL. It follows the same key-value pair sort of formatting as our POST requests. This GET request will return a chunk of data with a response body in the following form:

user:user_steps&max_user:max_steps

So if Robert Donat did his above request at a time when Nancy Sinatra had the highest amount of steps in 6.08 (from having walked significantly in her boots, which were made for walking) he might get back:

robert_donat:39&nancy_sinatra:1966

The system should then display that information properly on the screen (don't just use the raw output...format it nicely). Then, when the second button is pressed again, the OLED should go back to displaying the real-time step counter.

Note again the decision of button press vs. button release is up to you. Additionally, you do not need to real-time update the max—it only needs to do a GET when switching modes from real-time step counter to max display.

The OLED Display State Machine.

What we want to have, when everything is running, is a system where three separate code modules are running simultaneously. Many of you implemented the step counter as a state machine on Tuesday. Assuming you implemented the two other pieces we want for lab today, you should have all functionality carried out by three state machines that effectively act on a number of variables shared between them. For example both your step counter and your POSTing state machine will have the ability to modify the variable containing the total amount of steps taken since the last POST. Your display SM will utilize the number of steps taken to update the display.

As stated above, for now consider the event of POSTing to be non-blocking. Yes it may take a noticeable amount of time, but we can return to breaking up its tasks later in the semester.

We'd like all three subsystems to be working together in parallel together with minimal blocking code.

Once this is working in parallel with your other code, show it to a staff member. Then you're done.

Checkoff 33:
Show your overall system to a staff member.

For extra bonus (and by bonus we mean external validation with no grading or numerical value), adjust your system so that when in displaying your user total and the max total, it updates the user total in real time with recent step additions.

Happy long weekend! Exercises for Week 2 are due one day later, due to the long weekend (so, Monday at 11:59pm instead of Sunday night). Office hours will shift to Monday as a result.


 
Footnotes

1 1, for example, is the loneliest number, and 12, it turns out, is the largest number. (click to return to text)