Final project 2024. Secret Messages.¶
Note: This material will be updated during the course with clarifications and additions to the content. During the course, we will also add hints to help with common problems and other information. So, check this page from time to time!
Last updated: 1.10.2024. Initial version.
Introduction¶
Students are working for a goverment's intelligence agency. Your task is to develop a communication device that allows sending message to other agent using hidden interactions with the device. In this the Sensor Tag is going to be our communication tool.
Learning goals¶
In the project, the students will learn to use the peripheral devices of the SensorTag-board with TI-RTOS real-time operating system. In the basic completion level, the students are able to recognize certain movements with the MPU-sensor of the device, and transmit a character based on different movements to an external device via UART. UART is also used to receive messages from an external source, which are handled in the device in some way, by for example using the buzzer of the device to play out sounds with different lengths to represent certain characters.
Project specifications¶
Secret messages¶
In the project work you and your team need to implement a messaging program based on Morse-code. Morse alphabet is a part of a messaging system developed by |Samuel Morse in 1838, that was used to deliver messages via telegraph. Nowadays Morse alphabet and telegraphy are only used rarely, but for example the reconnaissance officers with radio training (also called "sissiradistit|Sissiradisti" in Finnish) of the Finnish Defense Forces learn and use them. Radio amateurs are also sometimes required to learn these skills to have permissions to broadcast at certain frequencies.
The program to be implemented as a project work will recognize the dots, dashes and spaces needed to form the Morse-coded messages by using movements captured from the sensor data of the device, or for example from button presses. Messages produced in this manner will then be transferred to a program running on a workstation computer, where they will be translated back to text. The program created for the device of the course should also be able to interpret a received message that consists of Morse-coded characters (dots, dashes and space) and present it to its user.
Message protocol¶
The communication happening between the device and the workstation abides to the international Morse code alphabet with some exceptions and application-specific clarifications. These exceptions are listed below.
- Every character encoded into Morse-alphabet is sent as a series of dashes "-" and dots "."
- Every dash and dot ends in newline ("carriage return needed in UART \r\n")
- Between every Morse-encoded character (e.g. "a" => ".-"), there is one space " "
- There is two spaces between words
- A message ends to three spaces
Example¶
To make the rules defined above more concrete, we have an example of the usage of our protocol below.
Word to send: aasi Encoding the word to a sequence of Morse-encoded characters, inserting one space in between of every character: aasi => .- .- ... .. We send the character through UART to the workstation: .\r\n -\r\n \r\n .\r\n -\r\n \r\n .\r\n .\r\n .\r\n \r\n .\r\n .\r\n \r\n \r\n \r\n
Workstation program¶
The course's virtual machine already has a program installed that sends and receives messages in Morse code via UART. You can start it using the "Serial Client" shortcut on the desktop. If this doesn't work, you can use the virtual machine's terminal and first navigate to the program's folder with the command
cd serial-client
, then start the program with the command npm start
.When it starts, the program reads the
config.json
-file located in the same folder, which contains the necessary configurations for using different development boards. Currently, the configuration file already has settings for the devices used in this course (SensorTag and Pico W), but it is also possible to include other devices. If the settings are read successfully, the program will print Config loaded with 2 known devices
to the screen, after which it will wait for a known device to be connected.When you connect a known device to the workstation, it switches to messaging mode, where messages typed into the terminal (except for program commands) are sent to the connected device via UART in Morse code. Data received from the device in Morse code is printed to the terminal, and when the three spaces indicating the end of a message are detected, the message itself is converted to text and printed to the window. This way, you can see the sent message as well as the characters that make up the message, which can help with debugging the program.
Commands¶
As mentioned before, all messages containing only allowed characters are sent to the device via UART in Morse-encoded format, with the exception being the program commands. These commands you can use to control the program.
.clear
clears the terminal screen.exit
stops the execution of the program
Requirements¶
- The program should have a minimal user interface. Implementation with for example buttons, LEDs and sounds.
- Control commands (dots, dashes and spaces) have to be captured initially from the data collected from accelerometer and gyroscope (MPU9250)
- At least two different positions or movements need to be recognized
- Position is letting the device static in certain position and orientation.
- In a movement the device has an original position and end position: moving the device up, letting it drop ... The movements can be also implemented in 3-dimensional way to many directions
- You can also attach button presses to the movements, along with other state information, enabling the implementation of more complex commands.
- The program has to send a message to the workstation when a command has been recognized
- The program has to inform the user about a recognized command with a sound cue or a LED
- The program has to receive messages from the workstation and react to those using either LEDs or buzzer (Morse code must be shown to the user)
- The implementation of the program has to be based on task-based implementation with the real-time operating system of the course, and a state machine
- You can use additional sensor recognition, for instance to send specific messages (e.g
HOT
if temperature > threshold)
Towards a better grade¶
The score for the project can be increased by implementing additional functionalities. Below are some examples, and the list will be updated further:
- 1-point features: Additional elements in the user interface
- Some kind of menu structure in the user interface using buttons and sounds.
- Indicating changes in the program state with sounds or LEDs (e.g., incoming message).
- Custom features in the user interface (per implementation).
- Detecting more than 3 movements.
- Utilizing additonal sensor.
- Collecting data for additional movements
- ...
- 2-point/3-point features: Extensive use of device capabilities
- Various more complex movements using multiple directions, different sensors, buttons, state information, etc., to assist in motion detection (per implementation).
- The speaker plays music at some point of the application.
- Using error thresholds in motion detection.
- Complex detection algorithms.
- Exhaustive data collection and analysys.
- Use of advanced synchronization mechanisms such as mutex or semaphores
- Use of display
- Communication of two SensorTag via wireless
- ...
Each implemented functionality is counted only once. The scoring of functionalities will be based on common sense and the teaching staff's experience in embedded systems programming.
Sensor data in final project¶
Here are the instructions on how to implement command interpretation in a program based on sensor data collection and interpretation.
There are two options to detect data using the MPU:
- Detect postion/orientation: the device is static. This would give less points
- Detect movement: Device is hold with the hand and the hand make certain movement that is detected.
Implementation of control commands as movements of the device, while holding it in hand, is freely decided by the group. Possible alternative implementations include at least the following:
- Sliding the device along the table surface
- Tilting and/or moving the device
- Bouncing the device (i.e., it occasionally leaves the table surface)
- Moving the device in the air without support, "3D" in the air.
General guidelines for interpreting control commands:
- It is advisable to determine how "strong" and how "long" in time one command is.
- For example, is the movement a small jerk or a slow movement lasting one second on the table surface in the chosen direction?
- When collecting data, it is important the "sampling rate": i.e., how often data is requested from the sensor.
- Example: once per second might be too slow an interval for the device's movement, but what if data is collected 5 or 10 times per second?
- Implementing a very "sophisticated" algorithm for interpreting control commands is not necessary. It is sufficient to analyze, for example, "changes in acceleration values as a function of time." See example data below.
- The exercise does not require a deeper understanding of mathematics than high school level, so the "quality of the recognition algorithm" for control commands is not evaluated. Of course, you can make the implementation as sophisticated as you want.
- However, the program must recognize movement commands in real-world test situations.
- "Chaining" control commands is okay, meaning that from continuous/long-lasting movement, multiple commands can be interpreted.
Interpreting a command¶
1. It is advisable to start the work by collecting your own dataset with the MPU9250 sensor as a time series while manually moving the device. The collection should be done so that only one movement is made at a time and saved in its own file.
- It is important to include a timestamp in the measurement results to calculate the duration.
- The sensor provides six different measurement values per data query using our course library, which are:
- Acceleration on the x-, y-, and z-axes, measured in gravitational force "g"
- Gyroscope readings on the x-, y-, and z-axes, measured in "angular velocity"
When the data is collected in a table, the result is a 7-dimensional table. The data in the table might look something like this:
aika, acc_x, acc_y, acc_z, gyro_x, gyro_y, gyro_z
0, 0.0029, -0.0063, -1.0115, 1.1215, -0.4578, 0.3204
11, 0.0133, 0.0291, -0.9753, 0.0229, -0.0458, -0.2060
19, 0.0028, 0.0964, -1.0406, 5.2948, -14.09, 19.0735
30, -0.085, 0.2474, -1.8069, -1.833, 0.0992, 15.4953
41, -0.20, 0.3864, -2.2000, -6.180, 29.9924, 70.5643
Note that the figures are normalized with the gravity. Hence, 1 means 9.8 m/s2
2. The collected data can be visualized in relation to different axes to see what happened in the data during each movement/command execution, and thus understand how to implement recognition programmatically.
By visualizing, it is easy to see what the sensor data looks like at different stages of the command when the device moves:
- Is it enough to measure just the movement and its duration? How does the movement appear in the data?
- Does the command need to start and end in a specific position?
- How much "error" (e.g., movement in the direction of other axes) is allowed in the execution of the command?
- etc.
3. After collecting and visualizing the dataset, the control command can be recognized from the data quite simply:
- The duration of the movement in relation to different axes
- Exceeding/falling below a chosen threshold value in relation to an axis
- Various features can be calculated for the time series, such as the average, variance, standard deviation, etc., which can be used to distinguish commands from each other (and optionally fine-tune command recognition).
- This can also be useful if you want to chain commands or implement more complex commands (for those extra points?)
- Since sensor data is obtained on three axes (x, y, and z), it can also be treated as a 2- or 3-dimensional vector, which has, for example, direction and length.
- Anything you can find in a math book / online / taught in an AI course.
- Remember to mention sources in the code.
Ready-made test data¶
The staff has pre-collected a small test dataset of various movements of the device held in hand, which can be used for preliminary command recognition in the work. The methods mentioned here for moving the device are not necessarily the only, easiest, or best way to implement commands with the device.
The test data was collected using the SensorTag's MPU9250 accelerometer. In the files, the data format is
[timestamp, acc_x, acc_y, acc_z, gyro_x, gyro_y, gyro_z]
. The data was measured so that the device was stationary at the beginning of the measurement and then moved up in two different ways. The images visualize the data for the x, y, and z axes (in different colors) over time for both the accelerometer (g) and the gyroscope (angular velocity).1. Sensor data when the device is stationary:
2. Sensor data when the device is jerked fast from the table surface to the direction up two times:
3. Sensor data, when the device is lifted up from the table surface and moved to the direction up:
Own test data¶
Collecting test data independently is easiest by first implementing a separate data collection task in the device as a separate program or task that simply collects movement sensor data into a table at selected intervals. Finally, the program prints the table row by row to the development environment, from where it can be easily copied and pasted into an editor and/or file. The data can be visualized using a Python library (hint hint, matplotlib) or a program like gnuplot or MS Excel.
When collecting your own test data, it is important to perform the movement always in the same way, i.e., the user and the sensor in the same position, etc., and absolutely only one movement at a time. This is so that the visualization clearly shows what happened in the data. Additionally, the data collected in different measurements is then comparable between different measurements. It is also noted that different people's hand movements are usually so different that, for example, the above test data does not directly apply to your own command recognition. The sensor is so precise that even small differences can be detected.
Tips for the project work¶
Not so surprisingly, many things are already told and many questions answered in the lecture materials. So, it is beneficial to study the materials very carefully..
The implementation of the project work should be designed with the help of Modular and procedural programming extra material. This way, even a big problem can be divided into parts that are easily manageable.
- Well planned is 90% done!
- A major mistake you can make is leaving things to the last minute. A beginner surely won't finish the work in a week!
- It's worth checking out the functionalities provided by the C language standard libraries and various RTOS libraries. Ready-made libraries greatly simplify programming, especially for tasks like string handling.
- When coding, it can be useful to code the difficult parts on a PC first. Testing motion detection algorithms and similar tasks on a PC is much easier. Once the code works, you can transfer the implementation to the device and test it there.
- Use intermediate print statements in loops, conditionals, etc., to print variable values or other essential information to the console window (System_printf(..) + System_flush(..) or printf(..)). This significantly speeds up debugging.
- A typical problem that can be easily solved with intermediate print statements is loop structures where you print different variable values to the console window in each iteration. Even at work, your boss is unlikely to appreciate you spending a lot of time figuring things out when adding debug information to the code would solve it quickly.
- It's important to remember that if you get stuck, ask!! The course staff is there to help you.
Project-related tips¶
We will update this section with additional tips when the implementation of the project work progresses and we know more..
- The program should be divided into tasks, such as one for communication, another for playing sounds, and another for reading sensor data, etc.
- Implement an Enable/Disable Command Recognition function in the user interface.
- An example of using the MPU9250 sensor can be found in the Git repository.
- Sensor data should be collected in a table, making it easy to handle as a time series.
- Note the size of the table in bytes where you are collecting data to ensure it fits in the device's memory! You have about 20kB of memory available, which must accommodate everything else as well.
- Sensor data can be cleaned if necessary (e.g., removing noise) by calculating a moving average or using another method. This is part of the exercise.
- The measurement timestamp for test data can be obtained using the TI-RTOS Clock library function
Clock_getTicks()
- The initialization of the device components should not be placed in the
main
function but at the beginning of the tasks. This is because the implementation should be task-based, and some components require RTOS functionality that starts only at the end ofmain
. - The lecture material includes an example of a state machine, which should be applied in the exercise. A direct copy & paste to SensorTag will not work.
- If you use SensorTag with a battery (without a USB cable), the battery will eventually run out, especially if the device is accidentally left on. Implement a power button in the code. You can get a new battery from the exercises if needed.
Implementation order¶
Below is a suggested (staff-approved) implementation order for the project. First, implement the required functionality and then aim for a higher grade by adding additional features.
- Project plan, which doesn't have to be final, but helps the group understand what they are doing. (The plan can change, see below)
- Collecting and analyzing sensor data on the device for command interpretation.
- Implementing the recognition algorithm in C on a workstation based on collected data.
- Transferring the working algorithm to the device and testing functionalities in real life.
- Fine-tuning the user interface, sending/receiving messages via serial communication.
- Additional features.
Deliverables¶
Project plan¶
As part of the course, a short project plan must be created for the project. These plans are not shown publicly to others; the purpose is to help yourselves in completing the final project. Creating a plan might seem unnecessary, but based on previous student feedback, it really helps to get your thoughts organized and to think about what actually needs to be done in the project! It also helps in scheduling the work.
The plan does not need to go into great detail; a high level of abstraction is sufficient. You can use bullet points to describe the items briefly. The plan should be a maximum of one A4 page. The plan is not binding, meaning it will likely evolve as the project progresses. An example plan is provided below.
Contents:
- A brief description of the functionalities in the program.
- Outline of the peripherals to be used (buttons, sensors, etc.) and their purpose in the program.
- Define the number of tasks and the role of each one
- State machine that implements the project. It is easiest to present this as a diagram.
- The state machine should cover the main tasks of the program; lower levels of abstraction are not necessary.
- Identify data structures, constants and global variable that you are using.
- Sketch a set of target movements to be used in the application
- Collection of sensor data: number of samples to collect, time interval, etc.
- Considered additional features.
- Preliminary schedule, including milestones and distribution of work.
The plan must be submitted by Friday 25.10 at 23.59. The plan is a mandatory part of the project. It is sufficient for one student from the project group to submit the plan to Lovelace. However, if you have questions, it is best to ask the assistants directly during the exercises.
Note! You are allowed to start the implementation of the project work before presenting a plan!
Note! The plans might be used as a reference material in academic research at a later time, in which case they will be anonymised. If you want to, you can decline the usage of your plan as research material by stating that in your plan.
Project code¶
The deadline for submissions is 2024-11-17 23:59
The code must be returned to the return box below before the deadline. This will be the code that is evaluated.
- The groups will make a common submission, which means that the points will be visible for every member of the group
- The project has to be uploaded to Lovelace because Finnish law requires us to store the course completions of each students for at least 6 months.
- You will need to submit only your own source code: for example
main.c
and possiblelibrary.c
andlibrary.h
- You don't need to submit project files, RTOS's files, sensors-folder or the wireless-folder
- In comments, mention the names of the group members. Tell, what each of you did (e.g. Aki did the design, Eva did the implementation and Veeti did testing)
- For additional points, include the reference material, results of data analysis etc.
- Combine multiple files into one zip-file
- Never submit the whole project folder without it being zipped!! It contains big debug files etc. that are not relevant for the grading but take up space in the server
Personal diary¶
Students need to keep a personal diary and return it at the end of the course. Each member of the team, need to return a text file or pdf file. Diary can be kept online, but it should move to a file before delivering. The diary is a personal task, not a team task.
The diary should contain at least one weekly entry with the following information:
1. Tasks that you have performed during the week and time taken for each task. Be specific: e.g. Implemented function [getActivity], collected up and down movement data for sensor
2. Which has been the main challenges?
3. What have you learnt this week?
4. What are your tasks for next week.
1. Tasks that you have performed during the week and time taken for each task. Be specific: e.g. Implemented function [getActivity], collected up and down movement data for sensor
2. Which has been the main challenges?
3. What have you learnt this week?
4. What are your tasks for next week.
If you are aiming for a 1 (see [|[below]]), indicate it also at the beginning of the dairy
Quizz¶
This quizz is similar to the one completed for the C part:
- Each student have 12 hours to complete the exercise. Date is 20.11.2024
- It is composed by around 10 multiple choice questions.
- Each student have 3 attempt to complete the exercise. After 3 attempts the exam will close. Your grade for the exercise is determined by the last attempt.
- Students can use any available material, including Lovelace material and your final project.
- Students need to have at least 70% of answers correctly. Otherwise, your grade for the whole project is 0.
Project evaluation¶
The project evaluation will be conducted in groups during the review session after the deadline. The session will be held in the workstation classroom.
- The review takes about 30 minutes per group.
- The entire group must be present either in the classroom or remotely. Absent member(s) will not receive a grade until they have clarified their contribution to the work.
- The functionality and code of the program will be presented to the teacher. The teacher will ask questions about the implementation.
- Teachers will use the code submitted by students before the deadline. Teachers will use the same development environment as the one installed on virtual machines for compiling and running the code.
- Students must be able to answer questions related to the functionality of the implemented application. All students must be able to answer such questions. Students may also be asked theoretical questions related to the course.
- If a student cannot answer basic questions related to the structure of the program or various implementation decisions, THEY CANNOT PASS THIS COURSE (The project grade would be 0).
- It is essential that the implemented application compiles, can be loaded onto the course device, and works for the basic functionality. Otherwise, the project will be graded as 0 and the student(s) cannot pass the course.
Grading¶
The maximum score for the project is 30 points, which constitutes 50% of the course grade.
Project grading:
1. Plan: 2 points -> Same grade for all members of the team
2. Project: 20 points -> Same grade for all members of the team
3. Personal diary: 1.5 points
4. Questions during the project evaluation: 2.5 points
5. Contribution to the project: 6 points to be distributed among group members. Decision agreed with students during final meeting.
6. Quizz about theoretical/practical aspects of the implementation 2 points
1. Plan: 2 points -> Same grade for all members of the team
2. Project: 20 points -> Same grade for all members of the team
3. Personal diary: 1.5 points
4. Questions during the project evaluation: 2.5 points
5. Contribution to the project: 6 points to be distributed among group members. Decision agreed with students during final meeting.
6. Quizz about theoretical/practical aspects of the implementation 2 points
Detailed evaluation criteria can be found in the submission box for the plan, the program code and the diary. Check the evaluation criteria in the corresponding submission box.
Final Grade
The other 50% of the course grade comes from previous C language exercises (max 30 points), so the maximum total score is 60 points.
The other 50% of the course grade comes from previous C language exercises (max 30 points), so the maximum total score is 60 points.
Requirements to pass
- Project compiles and it is uploaded to the SensorTag
- Project is able to detect at least two different positions / movements and send data through the UART.
- Project is able to produce 3 different commands (. - space)
- State machine is implemented and there are at least two tasks.
- Student is able to answer basic questions about the project implementation and about the theoretical aspects utilized.
- At least 70% of correct answers in the Quizz
- Student has contributed to the project implementation. Not only design or testing.
- Get at least 16 points in the project part.
Special requirements fo those aiming for 1 in the course¶
If you want a 1, you should mention clearly at the beginning of your diary. If you opt for this option, you wont get points but get just PASS / FAIL if following criteria is met:
- Have at least 18 points in C exercises
- Project compiles and it is uploaded to the SensorTag
- Project is able to detect at least two different positions / movements and send data through the UART.
- Student is able to answer basic questions about the project implementation.
- Student has contributed to the project implementation. Not only design or testing.
- At least 60% of correct answers in the Quizz
- At the begining of the code file should include comment section including the following information:
- Number of tasks you are using, function implementing each task, main parameters of each task, and explanation of the functionalit of each one of the tasks you are using.
- Number of interruptions you are using. For each interruption, you should define: interruption source, interruption handler, and short description of each interruption
- Enumerate how many serial communication communication protocols you are using. Describe which are the devices involved in the communication and a short description on how data is transferred.
FAQ¶
- Can the work be done individually?
- If there are valid reasons for working alone, please contact the teacher.
- Can the work be done in a larger group than a pair?
- The work can be done in a group of 2-3 people. In previous years, groups of three had to do additional tasks, but this practice is not in effect in autumn 2024. Groups of four are not allowed.
- The code worked at home, but not during the project evaluation. Can I try it on my own computer? Can I show it remotely from home?
- No. Make sure the code works in the development environment installed on the course virtual machines before submitting.
- Can I discuss the work with a friend?
- Of course, you can discuss it during exercises, social gatherings, etc., but if two groups submit the same code, you may be guilty of plagiarism. For this reason, groups that have collaborated should be mentioned in the code comments or similar.
- The line is that you can discuss the task, but you must write the code yourself!
- Can the work be submitted late?
- No.
- Can I use someone else's (or code found online)?
- You cannot use the code of those participating in the same course, even if you cite the source. In other words, you cannot copy code from someone else's project into your own.
- Otherwise, you can, as long as you indicate where the code was obtained from (website, etc.), so you do not commit plagiarism.
- Can I use AI?
- Yes, as long as you specify in your code what AI you used, what prompts you used, and how you modified the code based on the AI's response. If during the meeting we suspect that you have used AI and you do not have reported it your project might be rejected.
- What about plagiarism?
- The university has guidelines on how to deal with plagiarism. We follow them.
Temporary code sharing space¶
Use this return box to share code with your team members. DO NOT USE THIS RETURN BOX TO UPLOAD THE FINAL VERSION OF YOUR CODE. CODE IN THIS RETURN BOX WILL NOT BE EVALUATED