Deployment record

What 25 students reported, and what that does not prove.

Only M1, M2, and M3 have survey results. The classes were not randomized, and no class ran without the modules. The knowledge ratings are what students said about themselves. These counts are a deployment report. They are not a statistical result.

CohortTermSurveyModulesEnrolledResponses
C1Spring 2023InternalM1, M2Not recorded13
C2Spring 2024External evaluatorM1, M2, M3138 (62%)
C3Fall 2024External evaluatorM1, M2, M3Not recorded4

C1, C2, and C3 in this table are the three classes. They are not the design rules C1 through C5.

The survey questions changed between Spring 2024 and Fall 2024, so those two classes should not be compared item by item. A response rate is known only for Spring 2024: 8 of 13 students. Of those eight, five were men, three were women, and six identified as Hispanic or Latino. All four Fall 2024 respondents were men, and one identified as Asian.

Self-reported knowledge, Fall 2024

Four-point scale. One is no knowledge and four is strong knowledge. These are self-reported ratings. There is no concept inventory behind them.

BeforeAfter
Binary reverse engineering
Buffer overflow
Side channel

Read as 1.3 to 2.5 for binary reverse engineering, 1.8 to 2.5 for buffer overflow, and 1.0 to 2.0 for side channels. Interest in hardware security on a ten-point scale moved from 2.8 to 4.5 in this cohort. Spring 2024 reported 9.1 at the end and has no matching start value, so that figure is a terminal value, not a gain.

Spring 2023 usefulness ratings

Five-point scale. The midpoint is 3. Means sat below it. Students were not enthusiastic about the early handouts, which is part of why later versions added worked structure.

ItemMeanSDN
M1 usefulness for learning binary reverse engineering2.771.2513
M1 contribution to assembly and instruction encoding2.691.2013
M1 adequacy of materials and lecture2.831.1412
M2 usefulness for learning buffer overflow2.731.2111
M2 usefulness for a prior understanding of memory layout2.361.2311
M2 adequacy of materials and lecture2.551.5011

What students said the modules did

ItemSpring 2024Fall 2024
More hands-on work than in other courses87% yes100% yes
Modules helped with embedded-systems fundamentals87% yes25% yes
Changed interest in a graduate degree75% yes0% yes
Sought a hardware-security internship100% yesNot asked
Wants a career in hardware security50% yesNot asked
Wants more hardware-security modules50% yesNot asked
Needs more time to finish the assignments75% yes25%
Workload against similar courses63% more75% about the same
Overall experience, ten-point scale7.6Not asked

The split on fundamentals is the result to keep. Spring 2024 agreed that the modules helped them understand embedded systems. Fall 2024 mostly did not. Half of the Fall respondents reported that they did not attempt the assignments. The sample is small. The two cohorts should not be averaged into one success rate.

Barriers they named

Students chose from a fixed list: worked examples, lecture, study materials, time, and teaching-assistant support. Equipment was not on the list. Their answers are about teaching, not about missing instruments.

BarrierSpring 2023, M1, share of selectionsSpring 2023, M2, share of selectionsFall 2024, share of respondents
Worked examples33.3%28.6%0%
Class lecture29.2%33.3%25%
Study materials16.7%19.0%25%
More time12.5%14.3%25%
Teaching-assistant support8.3%4.8%50%

The Spring 2023 shares come from 24 selections on M1 and 21 on M2. Fall 2024 is four people. Spring 2024 was not asked this question.

Four lessons for the next offering

1

Students wrote most clearly about the repair.

One Spring 2023 student called the buffer-overflow assignment among the most interesting work in the program, and described finding where execution had to resume so that later secrets would return in R7. M1 has no countermeasure. Students asked what binary reverse engineering is for.

2

A desktop mitigation is not on the 8051.

The lecture names stack canaries and address-space layout randomization. Neither exists on the AT89C51. The repair has to be in the program, because the platform does not provide one.

3

Worked examples mattered more than new equipment.

The 2024 overflow handout adds partial credit by how far the leak gets, and it adds step-by-step hints. Both match the barrier the surveys recorded: examples, lecture, and time, not instruments.

4

The toolchain has to run on the machines students have.

Free responses include a student who could not run the vendor tools on a personal machine. The three deployed embedded modules share one 8-bit target, the AT89C51, and one vendor IDE, Keil. Whether the same labs carry to a 32-bit core is untested.

Support

This work was supported in part by the U.S. National Science Foundation (NSF) Secure and Trustworthy Cyberspace (SaTC) program under Award No. 2114157.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the U.S. National Science Foundation.

The project is a collaboration of the University of Kansas, the University of Florida, and Florida International University.