Final project 2024. Secret Messages.¶
Note: This material will be updated during the course with clarifications and additions to the content. During the course, we will also add hints to help with common problems and other information. So, check this page from time to time!
Last updated: 16.11.2024
- Clarification related of a bug in the serial client which does not recognize 3 characters at the end of the message. Only single words are accepted by the application.
Previous version:
- 1.10.2024. Initial version.
- 13.10.2024. Clarified deliverables.
- 30.10.2024. Clarification on the concept or movement / position.
- 06.11.2024
- Added information the need of interpretation of symbols (See Requirements Section)
- Clarification on the expected format expected and coming from the Serial Client (See Workstation Program section)
- Updated requirements section. Although MPU must be used, it is not mandatory that dots and dashes are defined using movements. You can use the button.
- Added demo video.
- Added source code of the serial client.
Introduction¶
Students are working for a goverment's intelligence agency. Your task is to develop a communication device that allows sending message to other agent using hidden interactions with the device. In this the Sensor Tag is going to be our communication tool.
Learning goals¶
In the project, the students will learn to use the peripheral devices of the SensorTag-board with TI-RTOS real-time operating system. In the basic completion level, the students are able to recognize certain movements with the MPU-sensor of the device, and transmit a character based on different movements to an external device via UART. UART is also used to receive messages from an external source, which are handled in the device in some way, by for example using the buzzer of the device to play out sounds with different lengths to represent certain characters.
Project specifications¶
Secret messages¶
In the project work you and your team need to implement a messaging program based on Morse-code. Morse alphabet is a part of a messaging system developed by Samuel Morse in 1838, that was used to deliver messages via telegraph. Nowadays Morse alphabet and telegraphy are only used rarely, but for example the reconnaissance officers with radio training (also called "sissiradistit|Sissiradisti" in Finnish) of the Finnish Defense Forces learn and use them. Radio amateurs are also sometimes required to learn these skills to have permissions to broadcast at certain frequencies.
The program to be implemented as a project work will recognize the dots, dashes and spaces needed to form the Morse-coded messages by using movements captured from the sensor data of the device, or for example from button presses. Messages produced in this manner will then be transferred to a program running on a workstation computer, where they will be translated back to text. The program created for the device of the course should also be able to interpret a received message that consists of Morse-coded characters (dots, dashes and space) and present it to its user.
- NOTE: There is no need that your application is able to do the translation between Morse code and letters or viceversa.
If your system is able to do this translation would be considered extra work. Minimum is that the system must be able to encode/decode letters A - Z (no need ä,ö) and numbers 0-9. Other symbols are not required.
Message protocol¶
The communication happening between the device and the workstation abides to the international Morse code alphabet with some exceptions and application-specific clarifications. These exceptions are listed below.
- Every character encoded into Morse-alphabet is sent as a series of dashes "-" and dots "."
- Between every Morse-encoded character (e.g. "a" => ".-"), there is one space " "
- There is two spaces between words
- A message ends to three spaces
- Every individual symbol (dot, dash or space) ends in newline ("carriage return needed in UART \r\n")
IMPORTANT NOTE: The application (serial client) we are using for testing, does not recognize the three spaces at the end of the message, so you can send only one word. The message ends with two spaces, instead of 3 as suggested in the instructions
Example¶
To make the rules defined above more concrete, we have an example of the usage of our protocol below.
Word to send: aasi Encoding the word to a sequence of Morse-encoded characters, inserting one space in between of every character: aasi => .- .- ... .. We send the character through UART to the workstation: .\r\n -\r\n \r\n .\r\n -\r\n \r\n .\r\n .\r\n .\r\n \r\n .\r\n .\r\n \r\n \r\n \r\n
Notice how the message ends with 3 spaces. Notice also that there are spaces before \r\n that does not start with . or -
IMPORTANT NOTE: The application (serial client) we are using for testing, does not recognize the three spaces at the end of the message, so you can send only one word. Hence, the message ends with two spaces
Workstation program¶
The course's virtual machine already has a program installed that sends and receives messages in Morse code via UART. You can start it using the "Serial Client" shortcut on the desktop. If this doesn't work, you can use the virtual machine's terminal and first navigate to the program's folder with the command
cd serial-client
, then start the program with the command npm start
.When it starts, the program reads the
config.json
-file located in the same folder, which contains the necessary configurations for using different development boards. Currently, the configuration file already has settings for the devices used in this course (SensorTag and Pico W), but it is also possible to include other devices. If the settings are read successfully, the program will print Config loaded with 2 known devices
to the screen, after which it will wait for a known device to be connected.When you connect a known device to the workstation, it switches to messaging mode, where messages typed into the terminal (except for program commands) are sent to the connected device via UART in Morse code. Data received from the device in Morse code is printed to the terminal, and when the three spaces indicating the end of a message are detected, the message itself is converted to text and printed to the window. This way, you can see the sent message as well as the characters that make up the message, which can help with debugging the program.
Important Note about the format expected by the Serial Client:
- In order to make things clear, let's call symbol to the individual piece of information sent to the Serial Client application (including the new line characters).
- Each symbol should end with the \0, that is, the
NULL
character. So, the expected text for each symbol contains 4 characters - For dot:
".\r\n\0"
- For dash:
"-\r\n\0
- For space:
" \r\n\0"
- Every individual symbol (dot, dash or space) must be sent individually, so you cannot send, for instance
.-\r\n\0"
or.\r\n-\r\n\0
- When a symbol is received from the UART, it does not contain the \0 (
NULL
character. ). Spaces are not used either to separate letters, words or end communication
Commands¶
As mentioned before, all messages containing only allowed characters are sent to the device via UART in Morse-encoded format, with the exception being the program commands. These commands you can use to control the program.
.clear
clears the terminal screen.exit
stops the execution of the program
Installation in your own computer¶
Source code and instructions to install and run the program in students' computer can be found from serial-client gitlab project
Demo¶
Following video summarizes the expected behaviour. Vertical movement send a dash, and pressing the button send a space. Pressing 3 times the space shows presents the writen message. Writting content in the Serial Client sends the morse signal that is played in the SensorTag.
Requirements¶
- The program should have a minimal user interface. Implementation with for example buttons, LEDs and sounds.
- Control commands (dots, dashes and spaces) have to be captured either from predefined button presses, from the data collected from accelerometer and gyroscope (MPU9250) or from a combination of both
- The program needs to be able to recognize at least two different positions or movements using the MPU90250. You can also combine button presses to the movements, along with other state information, enabling the implementation of more complex commands. The change in position or movement can be used either to:
- Send one of the basic symbols: dot, dash or space
- Trigger sending a symbol or a whole message
- Send a predefined message such as SOS, Mayday or similar.
- NOTE What is a position and what is a movement?
- Position is letting the device static in certain position and orientation. Usually, it just need to take one single sample from the accelerometer.
- In a movement the device has an original position and end position: moving the device up, letting it drop ... The movements can be also implemented in 3-dimensional way to many directions. In this case, multiple values from acceleromter / gyroscope must be collected and processed.
- The program has to send a message to the workstation when a command has been recognized
- The program has to inform the user about a recognized command with a sound cue or a LED
- The program has to receive messages from the workstation and react to those using either LEDs or buzzer (Morse code must be shown to the user)
- The implementation of the program has to be based on task-based implementation with the real-time operating system of the course, and a state machine
- You can use additional sensor recognition, for instance to send specific messages (e.g
HOT
if temperature > threshold)
- NOTE: There is no need that your application is able to do the translation between Morse code and letters or viceversa.
If your system is able to do this translation would be considered extra work. In that case, your system must be able to encode/decode letters A - Z (no need ä,ö) and numbers 0-9. Other symbols are not required.
Towards a better grade¶
The score for the project can be increased by implementing additional functionalities. Below are some examples, and the list will be updated further:
- 1-point features: Additional elements in the user interface
- Some kind of menu structure in the user interface using buttons and sounds.
- Indicating changes in the program state with sounds or LEDs (e.g., incoming message).
- Custom features in the user interface (per implementation).
- Detecting more than 3 movements.
- Utilizing additonal sensor.
- Collecting data for additional movements
- Translation Morse code / latin alphabet
- ...
- 2-point/3-point features: Extensive use of device capabilities
- Various more complex movements using multiple directions, different sensors, buttons, state information, etc., to assist in motion detection (per implementation).
- The speaker plays music at some point of the application.
- Using error thresholds in motion detection.
- Complex detection algorithms.
- Exhaustive data collection and analysys.
- Use of advanced synchronization mechanisms such as mutex or semaphores
- Use of display
- Communication of two SensorTag via wireless
- ...
Each implemented functionality is counted only once. The scoring of functionalities will be based on common sense and the teaching staff's experience in embedded systems programming.
Sensor data in final project¶
Here are the instructions on how to implement command interpretation in a program based on sensor data collection and interpretation.
There are two options to detect data using the MPU:
- Detect postion/orientation: the device is static. This would give less points
- Detect movement: Device is hold with the hand and the hand make certain movement that is detected.
Implementation of control commands as movements of the device, while holding it in hand, is freely decided by the group. Possible alternative implementations include at least the following:
- Sliding the device along the table surface
- Tilting and/or moving the device
- Bouncing the device (i.e., it occasionally leaves the table surface)
- Moving the device in the air without support, "3D" in the air.
General guidelines for interpreting control commands:
- It is advisable to determine how "strong" and how "long" in time one command is.
- For example, is the movement a small jerk or a slow movement lasting one second on the table surface in the chosen direction?
- When collecting data, it is important the "sampling rate": i.e., how often data is requested from the sensor.
- Example: once per second might be too slow an interval for the device's movement, but what if data is collected 5 or 10 times per second?
- Implementing a very "sophisticated" algorithm for interpreting control commands is not necessary. It is sufficient to analyze, for example, "changes in acceleration values as a function of time." See example data below.
- The exercise does not require a deeper understanding of mathematics than high school level, so the "quality of the recognition algorithm" for control commands is not evaluated. Of course, you can make the implementation as sophisticated as you want.
- However, the program must recognize movement commands in real-world test situations.
- "Chaining" control commands is okay, meaning that from continuous/long-lasting movement, multiple commands can be interpreted.
Interpreting a command¶
1. It is advisable to start the work by collecting your own dataset with the MPU9250 sensor as a time series while manually moving the device. The collection should be done so that only one movement is made at a time and saved in its own file.
- It is important to include a timestamp in the measurement results to calculate the duration.
- The sensor provides six different measurement values per data query using our course library, which are:
- Acceleration on the x-, y-, and z-axes, measured in gravitational force "g"
- Gyroscope readings on the x-, y-, and z-axes, measured in "angular velocity"
When the data is collected in a table, the result is a 7-dimensional table. The data in the table might look something like this:
aika, acc_x, acc_y, acc_z, gyro_x, gyro_y, gyro_z
0, 0.0029, -0.0063, -1.0115, 1.1215, -0.4578, 0.3204
11, 0.0133, 0.0291, -0.9753, 0.0229, -0.0458, -0.2060
19, 0.0028, 0.0964, -1.0406, 5.2948, -14.09, 19.0735
30, -0.085, 0.2474, -1.8069, -1.833, 0.0992, 15.4953
41, -0.20, 0.3864, -2.2000, -6.180, 29.9924, 70.5643
Note that the figures are normalized with the gravity. Hence, 1 means 9.8 m/s2
2. The collected data can be visualized in relation to different axes to see what happened in the data during each movement/command execution, and thus understand how to implement recognition programmatically.
By visualizing, it is easy to see what the sensor data looks like at different stages of the command when the device moves:
- Is it enough to measure just the movement and its duration? How does the movement appear in the data?
- Does the command need to start and end in a specific position?
- How much "error" (e.g., movement in the direction of other axes) is allowed in the execution of the command?
- etc.
3. After collecting and visualizing the dataset, the control command can be recognized from the data quite simply:
- The duration of the movement in relation to different axes
- Exceeding/falling below a chosen threshold value in relation to an axis
- Various features can be calculated for the time series, such as the average, variance, standard deviation, etc., which can be used to distinguish commands from each other (and optionally fine-tune command recognition).
- This can also be useful if you want to chain commands or implement more complex commands (for those extra points?)
- Since sensor data is obtained on three axes (x, y, and z), it can also be treated as a 2- or 3-dimensional vector, which has, for example, direction and length.
- Anything you can find in a math book / online / taught in an AI course.
- Remember to mention sources in the code.
Ready-made test data¶
The staff has pre-collected a small test dataset of various movements of the device held in hand, which can be used for preliminary command recognition in the work. The methods mentioned here for moving the device are not necessarily the only, easiest, or best way to implement commands with the device.
The test data was collected using the SensorTag's MPU9250 accelerometer. In the files, the data format is
[timestamp, acc_x, acc_y, acc_z, gyro_x, gyro_y, gyro_z]
. The data was measured so that the device was stationary at the beginning of the measurement and then moved up in two different ways. The images visualize the data for the x, y, and z axes (in different colors) over time for both the accelerometer (g) and the gyroscope (angular velocity).1. Sensor data when the device is stationary:
2. Sensor data when the device is jerked fast from the table surface to the direction up two times:
3. Sensor data, when the device is lifted up from the table surface and moved to the direction up:
Own test data¶
Collecting test data independently is easiest by first implementing a separate data collection task in the device as a separate program or task that simply collects movement sensor data into a table at selected intervals. Finally, the program prints the table row by row to the development environment, from where it can be easily copied and pasted into an editor and/or file. The data can be visualized using a Python library (hint hint, matplotlib) or a program like gnuplot or MS Excel.
When collecting your own test data, it is important to perform the movement always in the same way, i.e., the user and the sensor in the same position, etc., and absolutely only one movement at a time. This is so that the visualization clearly shows what happened in the data. Additionally, the data collected in different measurements is then comparable between different measurements. It is also noted that different people's hand movements are usually so different that, for example, the above test data does not directly apply to your own command recognition. The sensor is so precise that even small differences can be detected.
Tips for the project work¶
Not so surprisingly, many things are already told and many questions answered in the lecture materials. So, it is beneficial to study the materials very carefully..
The implementation of the project work should be designed with the help of Modular and procedural programming extra material. This way, even a big problem can be divided into parts that are easily manageable.
- Well planned is 90% done!
- A major mistake you can make is leaving things to the last minute. A beginner surely won't finish the work in a week!
- It's worth checking out the functionalities provided by the C language standard libraries and various RTOS libraries. Ready-made libraries greatly simplify programming, especially for tasks like string handling.
- When coding, it can be useful to code the difficult parts on a PC first. Testing motion detection algorithms and similar tasks on a PC is much easier. Once the code works, you can transfer the implementation to the device and test it there.
- Use intermediate print statements in loops, conditionals, etc., to print variable values or other essential information to the console window (System_printf(..) + System_flush(..) or printf(..)). This significantly speeds up debugging.
- A typical problem that can be easily solved with intermediate print statements is loop structures where you print different variable values to the console window in each iteration. Even at work, your boss is unlikely to appreciate you spending a lot of time figuring things out when adding debug information to the code would solve it quickly.
- It's important to remember that if you get stuck, ask!! The course staff is there to help you.
Project-related tips¶
We will update this section with additional tips when the implementation of the project work progresses and we know more..
- The program should be divided into tasks, such as one for communication, another for playing sounds, and another for reading sensor data, etc.
- Implement an Enable/Disable Command Recognition function in the user interface.
- An example of using the MPU9250 sensor can be found in the Git repository.
- Sensor data should be collected in a table, making it easy to handle as a time series.
- Note the size of the table in bytes where you are collecting data to ensure it fits in the device's memory! You have about 20kB of memory available, which must accommodate everything else as well.
- Sensor data can be cleaned if necessary (e.g., removing noise) by calculating a moving average or using another method. This is part of the exercise.
- The measurement timestamp for test data can be obtained using the TI-RTOS Clock library function
Clock_getTicks()
- The initialization of the device components should not be placed in the
main
function but at the beginning of the tasks. This is because the implementation should be task-based, and some components require RTOS functionality that starts only at the end ofmain
. - The lecture material includes an example of a state machine, which should be applied in the exercise. A direct copy & paste to SensorTag will not work.
- If you use SensorTag with a battery (without a USB cable), the battery will eventually run out, especially if the device is accidentally left on. Implement a power button in the code. You can get a new battery from the exercises if needed.
Implementation order¶
Below is a suggested (staff-approved) implementation order for the project. First, implement the required functionality and then aim for a higher grade by adding additional features.
- Project plan, which doesn't have to be final, but helps the group understand what they are doing. (The plan can change, see below)
- Collecting and analyzing sensor data on the device for command interpretation.
- Implementing the recognition algorithm in C on a workstation based on collected data.
- Transferring the working algorithm to the device and testing functionalities in real life.
- Fine-tuning the user interface, sending/receiving messages via serial communication.
- Additional features.
Deliverables¶
- Project Plan. Delivered by: 2024-10-25 23:59
- Final Project. Delivered by: 2024-11-24 23:59
- Diary. At least one entry per week. Delivered by: 2024-11-24 23:59
- Final Quizz. Done on: 2024-11-25 20:59
- Evaluation meeting. Times available for week 48 - 51.
Project evaluation¶
The project evaluation will be conducted in groups during the review session after the deadline. The session will be held in the workstation classroom.
- The review takes about 30 minutes per group.
- The entire group must be present either in the classroom or remotely. Absent member(s) will not receive a grade until they have clarified their contribution to the work.
- The functionality and code of the program will be presented to the teacher. The teacher will ask questions about the implementation.
- Teachers will use the code submitted by students before the deadline. Teachers will use the same development environment as the one installed on virtual machines for compiling and running the code.
- Students must be able to answer questions related to the functionality of the implemented application. All students must be able to answer such questions. Students may also be asked theoretical questions related to the course.
- If a student cannot answer basic questions related to the structure of the program or various implementation decisions, THEY CANNOT PASS THIS COURSE (The project grade would be 0).
- It is essential that the implemented application compiles, can be loaded onto the course device, and works for the basic functionality. Otherwise, the project will be graded as 0 and the student(s) cannot pass the course.
Grading¶
The maximum score for the project is 30 points, which constitutes 50% of the course grade.
Project grading:
1. Plan: 2 points -> Same grade for all members of the team
2. Project: 20 points -> Same grade for all members of the team
3. Personal diary: 1.5 points
4. Questions during the project evaluation: 2.5 points
5. Contribution to the project: 6 points to be distributed among group members. Decision agreed with students during final meeting.
6. Quizz about theoretical/practical aspects of the implementation 2 points
1. Plan: 2 points -> Same grade for all members of the team
2. Project: 20 points -> Same grade for all members of the team
3. Personal diary: 1.5 points
4. Questions during the project evaluation: 2.5 points
5. Contribution to the project: 6 points to be distributed among group members. Decision agreed with students during final meeting.
6. Quizz about theoretical/practical aspects of the implementation 2 points
Detailed evaluation criteria can be found in the submission box for the plan, the program code and the diary. Check the evaluation criteria in the corresponding submission box.
Final Grade
The other 50% of the course grade comes from previous C language exercises (max 30 points), so the maximum total score is 60 points.
The other 50% of the course grade comes from previous C language exercises (max 30 points), so the maximum total score is 60 points.
Requirements to pass
- Project compiles and it is uploaded to the SensorTag
- Project is able to detect at least two different positions / movements and send data through the UART.
- Project is able to produce 3 different commands (. - space)
- State machine is implemented and there are at least two tasks.
- Student is able to answer basic questions about the project implementation and about the theoretical aspects utilized.
- At least 70% of correct answers in the Quizz
- Student has contributed to the project implementation. Not only design or testing.
- Get at least 16 points in the project part.
Special requirements fo those aiming for 1 in the course¶
If you want a 1, you should mention clearly at the beginning of your diary. If you opt for this option, you wont get points but get just PASS / FAIL if following criteria is met:
- Have at least 16 points in C exercises
- Project compiles and it is uploaded to the SensorTag
- Project is able to detect at least two different positions / movements and send data through the UART.
- Student is able to answer basic questions about the project implementation.
- Student has contributed to the project implementation. Not only design or testing.
- At least 60% of correct answers in the Quizz
- At the begining of the code file should include comment section including the following information:
- Number of tasks you are using, function implementing each task, main parameters of each task, and explanation of the functionalit of each one of the tasks you are using.
- Number of interruptions you are using. For each interruption, you should define: interruption source, interruption handler, and short description of each interruption
- Enumerate how many serial communication communication protocols you are using. Describe which are the devices involved in the communication and a short description on how data is transferred.
FAQ¶
- Can the work be done individually?
- If there are valid reasons for working alone, please contact the teacher.
- Can the work be done in a larger group than a pair?
- The work can be done in a group of 2-3 people. In previous years, groups of three had to do additional tasks, but this practice is not in effect in autumn 2024. Groups of four are not allowed.
- The code worked at home, but not during the project evaluation. Can I try it on my own computer? Can I show it remotely from home?
- No. Make sure the code works in the development environment installed on the course virtual machines before submitting.
- Can I discuss the work with a friend?
- Of course, you can discuss it during exercises, social gatherings, etc., but if two groups submit the same code, you may be guilty of plagiarism. For this reason, groups that have collaborated should be mentioned in the code comments or similar.
- The line is that you can discuss the task, but you must write the code yourself!
- Can the work be submitted late?
- No.
- Can I use someone else's (or code found online)?
- You cannot use the code of those participating in the same course, even if you cite the source. In other words, you cannot copy code from someone else's project into your own.
- Otherwise, you can, as long as you indicate where the code was obtained from (website, etc.), so you do not commit plagiarism.
- Can I use AI?
- Yes, as long as you specify in your code what AI you used, what prompts you used, and how you modified the code based on the AI's response. If during the meeting we suspect that you have used AI and you do not have reported it your project might be rejected.
- What about plagiarism?
- The university has guidelines on how to deal with plagiarism. We follow them.
Temporary code sharing space¶
Use this return box to share code with your team members. DO NOT USE THIS RETURN BOX TO UPLOAD THE FINAL VERSION OF YOUR CODE. CODE IN THIS RETURN BOX WILL NOT BE EVALUATED