Data structures under tight limits
Choose records, queues, and ring buffers by access pattern and memory cost so sensor history fits without unpredictable allocation.
- Select and size a fixed-capacity representation for sampled data, commands, and device state, including overflow behavior.
Related study — optional context
Understand
The working idea
A data structure is a promise about how information is organized and what operations will be cheap. A labeled record groups one sensor sample’s time, value, and status. A queue preserves arrival order for commands. A ring buffer stores only the newest fixed number of samples by wrapping back to its first slot, much like a security camera overwriting its oldest footage.
On a microcontroller, capacity is an engineering quantity. If each sample contains a 32-bit timestamp, a 16-bit reading, and a 16-bit status field, its payload is 8 bytes. N samples therefore require at least 8N bytes, possibly more because of alignment or language-object overhead. A fixed packed buffer can make this cost predictable; a list of dynamic objects may cost several times the payload and can fragment a small heap. Measure on the chosen runtime rather than assuming desktop costs.
Access patterns determine the build decision. A screen plot needs recent samples in chronological order, so a ring buffer is suitable. A command mailbox that must never silently discard an emergency stop needs a bounded queue with an explicit full policy, perhaps reserving one high-priority flag outside the queue. Index update is write_index = (write_index + 1) mod capacity. When full, overwriting is acceptable only if “newest history wins” is part of the requirement.
A frequent misconception is that arrays are primitive and therefore always fastest or safest. An array with ambiguous positions—item[3] means temperature today but flags tomorrow—invites unit and field mistakes. Conversely, nested general-purpose objects can hide memory cost. Name fields at boundaries, use compact representations where measurements justify them, and document units and overflow behavior. Silent overwrite, unbounded growth, and mixing raw counts with converted units are data-design failures, not storage trivia.
Follow the reasoning
Worked example
A controller has 12 KiB available for logging. Each packed sample uses 4 bytes for milliseconds, 2 bytes for ADC counts, and 2 bytes for flags. At 50 samples/s, how much history fits if 2 KiB is reserved?
- 1
Usable logging memory = 12 KiB - 2 KiB = 10 × 1024 = 10,240 bytes.
- 2
Sample size = 4 + 2 + 2 = 8 bytes, so capacity = floor(10,240 / 8) = 1,280 samples.
- 3
History duration = 1,280 samples / 50 samples/s = 25.6 s.
A 1,280-entry packed ring buffer retains 25.6 seconds of history before overwriting the oldest sample.
Sanity check50 samples/s × 25.6 s × 8 bytes/sample = 10,240 bytes, exactly the allocated capacity before runtime overhead outside the packed buffer.
Try it yourself
Independent check
Move to the bench
Build a fixed eight-sample ring buffer and prove its ordering and overflow behavior from a trace.
- Microcontroller or desktop Python environment
- serial console or terminal
- potentiometer or generated integer sequence
- notebook
Experiment
- Represent each sample with a sequence number, raw value, and validity flag; state the units of each field.
- Insert values 0 through 11 into capacity 8 while printing the write index after each insertion.
- Read the retained entries oldest-to-newest and verify that sequence numbers 4 through 11 remain.
- Introduce a deliberate fault by removing the modulo operation; capture the out-of-range error or corrupted ordering, then restore it.
- Repeat with a queue-full policy that rejects the ninth item instead of overwriting it and compare the contracts.
Observe
- Expected: the write index repeats 0 after 7, and overwrite mode retains the newest eight sequence numbers.
- Record actual printed indices and retained values; if the runtime reports object memory, record it separately from the 8-byte packed estimate.
- Expected fault signature: without modulo, the ninth write targets index 8 rather than wrapping to 0.
Choose a representation and full policy for storing the last 60 fan-speed estimates while guaranteeing that a stop request cannot be overwritten.
- If using a potentiometer, connect it only between the board’s documented logic supply and ground; its wiper must stay within the ADC input range.
Keep evidence
Bench note
Record what you expected, what you actually measured or observed, and one difference worth investigating. This is self-reported evidence, not a certification.