Lab 05b: We're Going Places

The questions below are due on Tuesday March 06, 2018; 08:25:00 PM.


Partners: You have not yet been assigned a partner for this lab.
You are not logged in.

If you are a current student, please Log In for full access to the web site.
Note that this link will take you to an external site (https://oidc.mit.edu) to authenticate, and then you will be redirected back to this page.

Goals:

In this lab we will break out the compass functionality from our IMU, couple it with our GPS, and finally, hook into the Google Places API to get our system telling us where to go and what to do with our lives.

1) Getting Started

Get today's files by downloading from here. There are three files in this zip folder. Download and put them into their correct spots. The two libraries: Compass, and TinyGPS need to be inserted into the libraries folder under Arduino and will require a full restart of Arduino. The third file is a script that actually implements the compass (and performs some HTTP GET requests, which we'll integrate in at the end of the lab).

2) Postman

The image below might now cause reactions of fear or tears upon seeing it, but we hope you'll use those emotions to motivate the downloading of Postman to avoid the annoyances of sign-clicking. DOWNLOAD HERE!. We'll be using this more today!

You can't move on with lab until you click all the squares containing a sign1

3) Overview

In today's lab, we'll create a system where we use the MPU9255 IMU's magnetometer as a compass in order to determine our direction and then search for interesting Places 1 km away from us in that direction using the Google Places API, which we can then display on the OLED for our viewing pleasure. This will give us experience with using additional featuers of the IMU as well as a neat Google API.

We could certainly have the ESP32 send GET requests to the API. We know how to do this. But we can take advantage of our IOT system to share the workload between the ESP32 and the server. So we'll first send our location and compass heading to the class server, specifically directed to a script which you will write today. Then your server-side code will determine the location 1-km away in the direction given by the heading field, formulate and send a GET request to the Google Places API before parsing the returned JSON data, and finally send a small distilled subset of information back to the ESP32 for display.

4) Using the Magnetometer as a Compass

So far we have used the accelerometer in our MPU9255 Inertial Measurement Unit. Today we'll use the magnetometer as well. As we mentioned in lecture, the magnetometer uses a Hall Effect Sensor, where positive and negative charges flowing through a semiconductor material are deflected in the presence of a magnetic field (via the Lorentz Force), generating a voltage whose magnitude and direction depend on the applied magnetic field. By placing these magnetic sensors at various angles, the MPU9255 can infer the direction and strength of the magnetic field.

The Earth's magnetic field is ~50 \muT, whereas a typical refrigerator magnet may have a magnetic field of ~0.01 T. Thus, local "stray" magnetic fields can overwhelm the Earth's magnetic field and make it difficult to find magnetic North. Good magnetic sensors, therefore, incorporate a calibration routine, where the sensor is placed under different orientations. This allows one to calibrate out biases due to nearby magnets (so-called "hard-iron" distortion) and magnetic materials ("soft-iron" distortion). Our calibration routine only compensates for hard-iron distortion. Note, after calibration if you move your device closer to a magnet, the calibration will become useless, since the hard-iron we've calibrated for will have changed. In general, keep your device away from your computer since there are a number of magnets in it (speakers, lid-latch, magnetic-power-plug thing, etc...) Also be aware the tables in 6.08 have metal legs, and that can affect signal.

Open up compass_fun.ino and let's take a look. This code contains a reference to the Compass class that has two methods, calibrate() and update(). The main part of the calibrate() method is similar to this:

void calibrate(uint32_t calibrationTime) {
  ...
  imu.magCalMPU9250(imu.magBias, imu.magScale);
  ...

This code runs for 15 seconds and continuously reads from the three axes of the magnetic sensor. By measuring those values across a large number of orientations, it can estimate the offsets and first-order variations in the three axes. While this occurs (you'll receive an OLED prompt), you want to rotate the embedded system around the x, y and z axes.

The update() method reads from the magnetometer and then computes a heading. It actually tries to accommodate tilt, but since we didn't calibrate the tilt, most of the code ends up not mattering for today. What does matter is:

  void update() {
    imu.readMagData(imu.magCount);  // Read the x/y/z adc values
    //...
    imu.mx = (float)imu.magCount[0] * imu.mRes * imu.factoryMagCalibration[0] - imu.magBias[0];
    imu.my = (float)imu.magCount[1] * imu.mRes * imu.factoryMagCalibration[1] - imu.magBias[1];
    imu.mz = (float)imu.magCount[2] * imu.mRes * imu.factoryMagCalibration[2] - imu.magBias[2];
    //... calculate northy and northx, which are my and mx for today...
    float northAngle = -1*atan2(northy, northx)*180/PI; //Heading is based on X,Y components of North
    northAngle += declination; //Account for declination
    heading = -northAngle;  // Find our heading, given northAngle
    heading -= 90; //Change axes: Now 0 degrees is straight ahead (+y-axis)
    heading = int(heading+360)%360 + (heading - int(heading)); //Hack-y mod operation for floats
  }

This code removes the bias, then calculates the heading using atan2. We have to do some housekeeping in terms of making sure the heading is referenced to true north.

The magnetic north pole of the earth actually isn't where Santa lives (Santa lives at true geographic North, as has been shown in countless scientific studies). The earth's magnetic north is a different and time-varying location also up in the north beyond the Wall with the White Walkers.

OK, let's run our code. Compile and upload the code to your ESP32.

Once running, the OLED will first display a note about calibration. Calibrate by rotating your embedded system around all three axes over 15 seconds. Rotate at a moderate speed, and make sure the ESP32 makes a few full rotations during the calibration phase. This phase starts shortly after the ESP32 is powered on, so don't miss it! (If you do miss it, just restart by pressing the restart button on the ESP32).

Try Now:

Rotate the ESP32 to find North. Does your reading make sense? Does the heading change as you rotate? If not ask for help

5) Find a place of interest

Our goal is to get Google to tell us some interesting places exist 1 km away from us in different directions. So we need to use our heading and current location to calculate the latitude and longitude of the location of interest.

A map of our fair city.

In the picture above, we are finding a point (lat,lon)_{place} that is a distance L (which will be 1 km in our case) away from our current location (lat,lon)_{current} in a direction \theta_h relative to True North. We then want to search for Places within a circle of radius rad meters around from the point (lat,lon)_{place}.

One issue is that magnetic north and true north (i.e., geodetic north, or going to the north pole) differ in angle, by an amount called the declination. This difference varies depending on location, and in the vicinity of MIT is -14.75 degrees, which means that we need to add -14.75 degrees to the compass reading to convert from magnetic north to true north. Online calculators are available to determine the declination anywhere on Earth.

You can see in the Compass update() method above that we added in the declination.

Now we want to calculate the latitude and longitude of a point 1 km away when we are pointed at a certain direction. To do this accurately, we would need to use spherical trigonometry, but since our locations are pretty close, we can use a simpler approach: assume the area around us is roughly flat; this will be accurate enough.

Assume the heading angle is defined counter-clockwise relative to North like so:

Heading Angle.

In this image, we have a slightly northwesterly heading of around +30 degrees.

We're going to determine a formula to calculate the latitude and longitude of the desired point, assuming the earth's radius is 6,371.009 km. As an intermediate step, let's figure out a conversion factor between kilometers and degrees. We can use the following diagram as a guide, which you can imagine as a side-view through the earth:

Even if you don't believe it, just assume that the earth is a sphere and that it has a radius of 6,371.009 km.

Here the angle \phi (in latitude or longitude) between the current location and place location is related to the distance L that we wish to project via the earth's radius R.

How many kilometers are there in one degree of latitude? Enter your answer as a float with at least one digit of precision after the decimal point.

Longitude is a bit more complicated. Because lines of longitude converge at the poles, the distance in a degree of longitude is not fixed. For example, when we get very close to the poles, a degree of longitude becomes arbitrarily small. If we're adjacent to the North pole and we hop from the eastern hemisphere to the western hemisphere, we've moved a small distance but many degrees of longitude. The distance in a degree of longitude depends on your latitude: it's up to you to figure out the relationship!

How many kilometers are there in one degree of longitude, assuming we're at a latitude of +42.361169 degrees? Enter your answer as a float with at least one digit of precision after the decimal point. Do not round on intermediate steps.

Now, figure out a formula that, given the coordinates of an initial point, a heading in degrees, and a distance in kilometers, will find the coordinates of a final point that is the appropriate distance and heading from the initial point. Your formula can assume that we're staying near our current latitude, so your kilometers-per-degree longitude can be a constant.

To check your work, find the coordinates of the point 1 kilometer away from us at a heading of +30 degrees. Enter your answer as a tuple of the form (lat, lon), with at least four digits of precision after the decimal point. Recall that we're at (42.361169, -71.092259).

In our system, degrees increase counterclockwise. Thus, 0 degrees is North, 90 degrees is West, etc.

Remember this formula, as you will need to implement it in Python later on in the lab.

OK now place your labkit down. There is some other functionality involved with the button, but we won't use that until we get our server-side stuff working, which is next.

6) The Google Places API

First, every student will need to get a Google Places API key. This key provides access to their system and allows Google to see who is making requests and to charge for requests. Don't worry, you get 2500 free requests each day, which is plenty.

To get a key, go to the Google Places API Web Service and select "GET A KEY". In the popup, select "Create a new project", and then accept the Terms of Service. On the following page, you can type in a name for your key, if desired. (Google doesn't seem to like names that start with numbers, so if you get the error "Couldn't create project", try a different name.) Leave the IP address field blank (if you see such a field at all). Hit Create. In a moment, you will get an API key. Copy-paste that key into a safe place (like a text file) and don't lose it!

Using the Google Places API is fairly straightforward and a good thing to learn. Go to the Google Places Guide to see more details on the GET request syntax for a Nearby Search.

Using the Google Places Guide, figure out how to conduct a Nearby Search for our current location, with a radius of 100 m. Make sure to get your response back in json.

We are not giving you detailed instructions here. This is by design. As we approach the final projects, we want you to get the confidence to find things out on your own.

Once you execute the search, you should get back a whole lot of text. This text format is json, which we've seen a few times already. Near the top of the JSON you should see "results", with a set of the list of returned locations. Each one has a "geometry" key and within that a "location" key containing the "lat" and "lon" of the location. We also find a set of "name" keys, some of which should be recognizable.

If you go to the Google Places Guide and scroll down to "Search Results", you will see definitions for all the possible parameters.

Use Postman to experiment, with the Google Places API and get a feel for what it returns. Using your knowledge of Boston or Cambridge how drastic is the impact of varying radius, opennow, or type?

Checkoff 1:
When you are ready, demonstrate to a staff member a couple example requests of your design from the Google Places API, and explain the JSON results. Make sure you are able to describe the different fields.

7) Tell Me What to Do

Now that we can interface with the Google Places API, we want to work towards integrating our labkit with it. In particular we want to be able to report our latitude, longitude, and compass heading to a script of our design that we place on the class server, which will then use those three values to look up an interesting set of things we can do 1 km away from where we're facing and return it to the querying client (ESP32) to display it on our OLED.

Just like on Tuesday, let's get the web-side stuff working first before bringing the ESP32 into the picture. Using Postman as a development tool, create a Python script which when run on iesc-s1.mit.edu will run as described above. Keep in mind we will want to display our interesting location/place on the OLED, so return a simle text string of the objects/points of interest rather than something loaded down with curly-braces or html or something.

You will find the Python requests library useful for this lab. Review week 4's exercises if you don't remember how to use that. You may also find the Python json library useful, although requests should have sufficient functionality built-in.

7.1) Building Strings

We are going to want to build up the Google Places request URL programmatically in Python. When constructing strings in Python from data it can sometimes get unwieldy to assemble them piece by piece. For example, let's say you needed to build a string of the format:

The best year in living memory was BLANK. The economy was BLANK, bands like BLANK and BLANK were at the top of the charts, gas cost BLANK a gallon and we expected BLANK to be even better.

where the term BLANK is based on variables specified programmatically. As a reminder, Python (and C/C++) has the ability to use string formatting to make this task easier and more flexible. We could do:

year = 1999
economy_state = 'great'
bands = ['Smash Mouth','Britney Spears']
gas_price = 1.3625
total_string = "The best year in living memory was {0}. The economy was {1}, bands like {2} and {3} were at the top of the charts, gasoline cost {4:.2f} a gallon and we expected {5} to be even better.".format(year,economy_state,bands[0],bands[1],gas_price,year+1)

What we used here was string formatting. In Python, there are actually two ways to do string formatting, and we're showing you the new way. The old way more closely resembles C-style string formatting. Here's a really informative page explaining both the new style and the old style.

Note that format() returns a new string; it does not modify the caller string (which has all the placeholders).

In the example above, "{0}" serves as a placeholder. When we call format(), the value of the 0th argument (year, in this case) gets swapped into the spot where "{0}" was. The same applies for all the other placeholders. You can also just use "{}" as a placeholder. See the link above if you want to learn more.

The "{4:.2f}" placeholder is pretty neat. We're saying that argument 4 is expected to be a float (hence the "f"). Furthermore, we're telling Python we want to display two digits of precision after the decimal point (hence the ".2"). See the link above if you want to learn more.

When fully functioning, in response to a GET request with the appopriate headers (lat, lon, and heading), your system should return something that will be easily displayed by on the ESP32. Let's say I'm in Building 38 and pointing towards Boston. I should get back a simple response such as "Boston Symphony Orchestra" or something similar (obviously depending on coordinates and radius of interest).

Checkoff 2:
Demonstrate your working Python script by sending it an appropriately formatted GET request using Postman or similar software.

8) Using the Google Places API with our ESP32

OK the final piece in this lab is to return to our ESP32 and use the remaining functionality in that script. The code is already doing pretty much everything you need, even though you might not know it. When you press a button connected to Pin 15, it sends a GET Request to a page that bounces back the request. When you press the button again, it will return to the Compass/Heading display.

The file is broken into two "pages" (they just get merged together upon build). You'll want to update the do_GET function on the support page of the ESP32 code so that it fires a request off to your script rather than the bounceback page.

Modify this code so that it will work with your existing Python script. The We want you to be able to pick a direction, get some things to do in that direction, then click again to continue on. When ready ask for checkoff 3. Be ready to show some cool things that you can do near campus.

A Python Error Occurred:

Error on line 2 of Python tag (line 239 of source):
    kerberos = cs_user_info['username']

KeyError: 'username'

Checkoff 3:

9) Going Further

There's lots of add-ons one could do with this lab. You could return events that are only open at the current time, or events that are only relevant for that current time (is it 9am? maybe suggest breakfast). Another cool thing you could do is have the radius you search for be based on how long you hold down the button, or have that same button-hold signal correspond to how far away you want to look from where you're at. That'd be cool. If you have time, try to add something else to your functioning system.


 
Footnotes

1 The joke is that this is a png file...so you'll never be able to click them all. (click to return to text)