Course project 2019¶
NOTE! This is updated every now and then, so check back frequently.
The future technology megatrends IoT and AI are becoming massive. In this course, we aim to equip students with such basic skills and in the course project, we implement a real-world wearable computing AI application with SensorTag.
(Don't worry the term AI just yet, please read further.. basically you need to just implement a simple analysis for some sensor data)
Your task in the course project is to identify hand gestures (See gesture recognition) from the data produced by the MPU9250 sensor of the SensorTag. The first gesture you must recognize is the high five (as it is commonly known) and another gesture which you can freely select by yourself. When your program has recognized a gesture, it sends a broadcast message to the other devices in range. The other users are supposed to reply to your message by a hand gesture (..and the following message).
Your SensorTag application must be able to detect the high five and any other-hand gesture in a way that it can be demonstrated. How to detect the gesture is up to you.
Your application must have the following features:
- Minimal user interface.
- Use the MPU9250 sensor for gesture data collection. You can of course use other sensors.
- The recognition algorithm must detect the high five-gesture (in a way its understood by common sense) and another gesture that you can define by youself.
- Gesture must include some movement and a hand posture.
- Static hand / posture is not a gesture.
You can also test, evaluate and use your application with battery, i.e. detached from the workstation. You can find the instructions here.
Nowadays the vast majority of AI research and applications focuses on large-scale Big Data analytics and machine learning based on statistics. However, as these are not the topics of our course, we have a much simpler idea.
Your application needs to calculate some simple data features from collected sensor data and then classify the actions (i.e. gestures) based on comparison of the collected features and precalculated gesture features.
IMPORTANT! As statistics and AI are not in focus of this course, we will NOT grade the course project in this regard, i.e. the quality of your gesture recognition algorithm is not graded. However, you must be able to demonstrate a functional application in the grading session.
You can think of the gesture recognition in three phases:
1. Collect a dataset for initial analysis by performing a set of gestures (that you want to detect). With the MPU9250 sensor, you get three axis data of both acceleration and gyroscope that you can use to analyze hand movement and hand posture/pose. Note that the MPU9250 software library used in the course gives you angular velocity.
Now, the collected data set gives you information on what happens during the gestures with respect to these features.
2. The gesture recognition can be based, in general, to comparison of simple data features such as:
- Tresholding: find out what are threshold for different modes
- Data features: average, variance, standard deviation, ...
- Variance can be calculated for example with the algorithm here.
As a result, you get a set of features that describe the gestures through acceleration and hand posture, for example:
- What is the starting posture of your hand
- How to detect movement in the data?
- What is the ending posture of your hand
- How much error (dritfing, etc) you allow during the gesture?
3. Next, you can perform the gesture recognition on-line on the SensorTag device by calculating the selected features from the data and by performing the classification through for example some comparison.
Data set for testing¶
Course staff ahs collected initial data set for testing of the the high five hand gesture with the MPU9250 sensors.
Below, the data format in the example files is:
[timestamp, acc_x, acc_y, acc_z, gyro_x, gyro_y, gyro_z].
1. Acceleration: In the figure below, you see the acceleration values (of one axis) during the gesture. X-axis shows the time. (IDLE just means the device was stationary.)
2. Gyroscope /angular velocity: In the figure below, you see the angular velocity during the gesture. X-axis shows the time. (IDLE just means the device was stationary.)
Test data as plain text:
Test data as C header file:
You could increase your grade by implementing some extra features for your SensorTag application
- 1 point features (Extra stuff in the UI)
- Menu in your user interface.
- The program sends several and different types of messages.
- Produce some sound with buzzer.
- User feedback through messages and leds, etc.
- 2 point features (using additional SensorTag components)
- Read data from other sensors, show it on the display, etc.
- Pixel graphics in your application.
- Animated pixel graphics in your application.
- The buzzer plays "music/musical piece".
- Your application detects movement in on-line during the measurement. E.g. somehow informs the user about the phase of the gesture.
- Your data analysis performs some kind of evaluation of the performed gesture.
- 3 point features (significant own programs features)
- Your application reconizes more than two gestures.
- Teach the user to perform the gesture.
Every new feature is counted only once, i.e. additional sensors don't add up the score.
When evaluating the features for grading, the teacher's extensive experience in embedded programming and common sense is used.
It could be a good idea to start working on the project with Top-down design. This way you can partition a bigger problem into smaller problems that are easy to tackle in turn.
- A good design significantly reduces the amount of implementation work
- Do not leave the work for last minute, you will not succeed. Years of experience tells us that beginners can't finish the project in time, even when they have previous experience with embedded systems, in a week or two.
- Check the features and APIs that TI RTOS:n provides for SensorTag
- It might be beneficial to code algorithms and such first in PC, before moving the working code into the device.
- Implement debugging for your project, its signicficantly reduces the time used for bug hunting
IMPORTANT! If you get stuck, do not delay your work but visit the exercise sessions.
This chapter will be updated as the work progresses..
- You can find an example FSM for embedded application in the lecture material (Finnish). Please note that it does not work in your program exactly like it is.
- Split the program functionality into tasks, which all have proper functionality, e.g. sensor data collection, updating the display, wireless communication, etc.
- Implement On/Off button for the sensor data collection
- Code example of using the MPU9250 is here.
- Perhaps its a good idea to collect sensor data into an array.
- Please note the size of the array, i.e. how much memory it uses. Yoi have maximum 20kB, where everything in your program must fit.
- Figure out the sample rate for your recognition algorithm. This is your own software design decision.
- Fet timstamps for your data by using the function
Clock_getTicks()from the Clock library.
- You could also clean-up / filter your sensor data with some common algorithm like moving average.
- Implement the initialization of some peripheral devices to the tasks, because you need to first start the BIOS, because the initialization needs that functionality.
- Of course, when using SensorTag with a battery, it may run out. Especially if you leave it running overnight. Just ask for new batteries during the exercise sessions.
This could a working order of how to implement your application
- In device Collect your dataset for the recognition algorithm.
- In workstation implement your recognition algorithm with C and using your data set.
- In device test your algorithm on-line.
- Implement the user interface and wireless messaging.
- Extra features.
Collecting test data¶
You could use a laptop to collect your own real-world data. Just implement a task for your SensorTag program that reads the sensor data, for example in an array, and outputs it to debug console in the end of the collection. You can then copy&paste the data to text editor and use Python (
matplotlib) or gnuplot or M$ Excel or to such application to visualize the data.
Note that when you start your program with the MPU9250 sensors, it performs internal calibration. So please put the device into solid flat surface in the beginning.
When collecting data, try to always perform the collection exatcly the same way. Otherwise your data may not actually be comparable with your previously collected data.
As an obligatory part of the course project, you need to provide short (max one page) design documentation of your ideas. This is your private document that helps you to understand what exactly is needed to do in the course project.
Your design can be high level, no need to dig deep into the details. Here is a simple example of a design document.
Contents of the document:
- Outline of what is shown in the device display and what UI components, will be used, e.g. buttons and leds.
- Short description of the functionality of your program,
- Tasks, own functions, ...
- Data structures, constants, global variables, ...
- Data size, periods, etc, ..
- Just a picture needed
Returning design documentation¶
The deadline for the design documentation is 6th October. If you miss the deadline without justified cause, your course project score is reduced three points (see below). It is enough that either student of the group returns the document, but it must have the names of all group members.
note! You can of course start working on your project, before returning the document.
Note! We may use the documentation as a source material for academic research. If you want to prevent using your documentation, mark this into the document.
Returning your project for grading¶
The course project deadline is Sunday 17th November 2019 at 23:59.
Upload your source code files to the box below before the deadline. This source code is the graded code. It is ok for one student per group to upload the code. (In any case, you must do the upload, as it is required by Finnish law to store your course work for at least six months.)
Upload the code by the following instructions:
- Only your own code
main.cand additional files
- Do not return RTOS code, or the code in sensors and wireless directories.
- Add your names and student numbers into a comment header.
- For your own additional features, add source material, test results, etc, that should be considered in grading.
- If you upload several files, pack them with zip first.
- Never just pack the whole project directory, as it contains large unnecessary debug files etc.
Course project grading¶
Your project will be graded in a grading session after the deadline. The calendar to reserve time for your project grading will appear later
- Grading takes about 15 minutes per group.
- All members of group MUST be present.
- Missing students don't get grades until they demonstrate their contribution to the project.
- You need to present your code to the teacher, who will ask questions ab out your solution.
- In the end, your score is presented.
The maximum score of your project is max 30 points that is 50% of the overall course grade. The other 50% comes from the C programming exercise. You must have points from both exercises and projects to pass the course.
- Design document: 0 / -3p
- Program follows requirements: 10p
- Bugs or missing feature: -2p / each
- If you explain how to fix the bug: -1p
- The program implementation follows a finite state machine: 3p
- the structure of code is modular and uses functions: 1p
- Variable and function names are reasonable, code has comments and proper indentation is used: 1p
- Each feature 1-3p
- Can I do the project alone?
- Yes, however your workload is then more than with a peer.
- Yes, but you have to implement additional extra features worth 5 points into your project.
- Yes of course, but you still have to implement your own code. If you copy the code, you may be guilty of plagiarism, which may lead you to very serious trouble. Therefore, please write into the code comments the names of students you co-operated with.
- It's not worth it, we will reduce your total grade by -1. This is the actual grade, not points!
- If you have a justified cause to return the project late, please inform the course teacher beforehand. Typically justified causes are well-known early.
- Sure, but you must inform the source (e.g. www page) in your code comments. Othwerise you may be guilty of plagiarism.
- The university has strict rules regarding plagiarism. We follow those rules, where students can find themselves in serious trouble, if found guilty of plagiarism.