PolyCast: Assistive Marker for Classroom Capture

When writing gets in the way of learning

PolyCast digitizes a handwritten whiteboard lecture while it is being written.

The prototype is a custom 3D-printed attachment that clips onto an ordinary whiteboard marker. Two kinds of sensor sit inside it. A motion sensor (UWB) tracks how the marker tilts and accelerates, and a positioning sensor (IMU) works out where it sits on the board by measuring its distance from four small radio beacons (UWB Kit) attached to the corners. Both readings travel wirelessly to a small computer (Raspberry Pi 4) at the back of the board, which turns them into strokes and draws them on a web page.

Students open that page on a phone or laptop and watch the board fill in as the instructor writes. Every few moments the system saves a snapshot, so a whole lecture ends up as a browsable set of pages that anyone in the session can download before they leave.

Nothing about the room has to change. The whiteboard, the markers, and the way the instructor teaches all stay the same, and the entire system runs on the classroom’s own network with no internet connection.

Keeping the classroom exactly as it is

In engineering and math courses, the board fills up faster than anyone can copy it. Students end up splitting their attention three ways: listening to the explanation, following the derivation, and writing it all down before it disappears. Most give up on one of the three, and it is usually the listening.

Then the instructor erases the board to make room, and whatever was missed is gone. The students who kept up did so by acting as recorders instead of learners.

Existing solutions do not fit many traditional classrooms:

• Smartboards require replacing existing whiteboards, which can be expensive for budget-constrained schools.
• Digital pens often depend on proprietary paper or specialized surfaces instead of a standard whiteboard.
• Camera-based capture can be affected by glare, changing lighting conditions, and the instructor blocking the board while writing.

Our goal was to keep the existing whiteboard and teaching workflow unchanged by developing an attachment for a standard whiteboard marker. We provide a cost-effective solution by focusing on the essential features of the expensive existing solutions.

Technical Lead & Backend Data Engineer

I was the technical lead on a team of four, working across the hardware and the software. My main responsibility was the backend and data-processing layer, where I helped plan the architecture and workflow that connected the hardware to the software in collaboration with the rest of the team

My contributions included:

• Designed and built the error-state Kalman filter that merges motion and position readings into a single estimate of the pen tip, including the admission rules that decide how far each position reading is allowed to move it.

• Structured the processing chain as nine independent stages from serial port to finished stroke, each runnable and testable on its own against recorded sessions.

• Built the recording and replay tooling the team debugged with, which moved diagnosis off live hardware and made intermittent faults reproducible.

• Designed the marker firmware with a dual-core task split to protect high-rate motion sampling from slower radio ranging, and debugged the full data path from anchor addressing and packet structure to server-side.

• Connected the BU03 UWB module directly to an Arduino instead of using its bundled STM32 Devkit. Since the standalon module with the DevKit is not supported, I studied open-source reference firmware to determine the required wiring and command sequence, then recreated the communication process in C++ for the Arduino.

• Built the Flask web application for live handwriting capture, session management, and browsing or downloading saved notes.

I also coordinated the team across hardware assembly, firmware development, and system testing, while contributing to some architectural decisions throughout the project.

How the data gets from marker to browser

Why keep the bridge and the server as separate devices? The receiver does no math at all. It takes radio packets from the marker and pushes them down a USB cable, and that is its whole job. Every calculation happens on the Raspberry Pi in Python, which made iteration fast. Changing how the filter worked meant editing a Python file and restarting a program, instead of reflashing firmware for every experiment.

Why use MJPEG instead of raw coordinates or a standard video stream? MJPEG is simpler for the Raspberry Pi: it renders the canvas once and sends the same JPEG frame to every viewer, keeping devices synchronized and allowing reconnecting students to load the latest image immediately.

How does one Raspberry Pi run the pipeline and serve the class at the same time? A single Raspberry Pi can serve an entire class. Each stroke is drawn once onto a shared in-memory canvas, then streamed to every connected browser as MJPEG frames. Because students only need to display the image rather than render it themselves, the Pi does the work once instead of once per viewer. And because one multithreaded process owns the canvas, every browser is looking at exactly the same thing.

Letting each sensor update on its own clock

Neither sensor can do the job alone, and they fail in opposite ways.

The IMU updates about 200 times per second, providing responsive motion tracking but gradually accumulating drift. UWB updates 10–50 times per second, providing absolute position corrections but with more noise and occasional jumps.

Instead of waiting for both sensors, the fusion engine processes each reading independently using its timestamp. The IMU predicts movement, while UWB corrects positional drift.

This approach allowed the system to use:

• IMU for responsive stroke movement and shape.
• UWB for correcting accumulated positional drift.
• Efficient processing at each sensor’s natural rate.

The cost is real, and it is mostly bookkeeping. Two sensors arriving on independent clocks have to be sorted into one ordered stream, and the filter has to reason about corrections that refer to a moment slightly in the past. That complexity bought responsiveness we could not get any other way, so we took it.

Preventing the pen from “teleporting” during a stroke

Radio positioning assumes the signal travels in a straight line from beacon to marker. An instructor writing on a whiteboard breaks that assumption constantly, because their own arm keeps passing between the two. The signal then arrives late, having bounced, and the system reads that delay as extra distance.

On screen this looked like the pen teleporting. A letter would be halfway drawn and the line would lurch sideways.

What made it hard was that two completely different problems produced the same picture. Either the position reading was wrong and the filter should ignore it, or the reading was fine and the motion estimate had drifted, in which case ignoring it would make things worse. Watching it happen live told us nothing, because by the time the line jumped, the cause was already gone.

So we stopped debugging live. Every session was recorded to disk and replayed offline, as many times as needed, which let us step through corrections one at a time and see which beacon’s reading was behind each bad correction.

The solution we used two safeguards:

• A statistical gate rejects UWB readings that fall outside the filter’s expected uncertainty. Anchors that repeatedly produce unreliable readings are temporarily muted.
• A recovery mechanism resets the estimate toward a trilaterated position only when the filter remains inaccurate across multiple update cycles.

The recovery correction is applied only when the pen is lifted (post-process solution), preventing the position from visibly jumping in the middle of a stroke.

A sustained non-line-of-sight event can still affect a long stroke and appear as a kink in the reconstructed line. Improving how the system handles these conditions would be an area for further development.

What the system does

• Captures handwriting on an ordinary whiteboard, with no special surface, marker refill, or wall mount required.
• Streams the board live to any browser on the classroom network. Students join by opening a link (polycast.local).
• Works out on its own whether the marker is writing, hovering, or set down, so there is no start or stop button for the instructor to remember, just need to turn on the prototype.
• Saves snapshots as the lecture runs and files them into browsable sessions, so a class from three weeks ago is still there.
• Downloads a whole session as a ZIP, packaged in the browser itself.
• Runs entirely offline. No account, no cloud service, and no student data leaving the room.

Hardware choices and system stack

What the numbers say, and what is still unfinished

We measured the system on two things: how close the digital stroke lands to where the pen actually was, and how long it takes for writing to appear on screen.

• Average positional error: 15.13 mm. About the width of a fingertip. Handwriting stays legible at that error; a floor plan would not survive it.
• Average end-to-end latency: 211.09 ms. A fifth of a second from the board to the browser, which reads as keeping up rather than lagging.

The web application was separately assessed against the ISO/IEC 25010 software quality standard and scored well on usability and maintainability.

Three things I would take further:

• Give the motion sensor a confidence score. The position side already tracks how much to trust each reading and scales its influence to match. The motion side has no equivalent, so it is trusted evenly whether the filter has been running cleanly for a second or fighting bad geometry for thirty.
• Handle a long blockage properly, rather than handling brief ones well and accepting a bent stroke for the rest.
• Test the machine learning approach we did not have time for. Early experiments hinted it could help, but nothing in this system rests on that, and I would want real results before claiming otherwise.

What I take from the project is a changed instinct about sensor data. I came in assuming the work was in the filter mathematics. Most of it turned out to be in deciding what to do when two sensors disagree, in a pipeline that cannot pause to think about it.