Tonbandgerät

Tonbandgerät /ˈtoːnbantɡərɛːt/ n.

1. German for audio tape recorder.
2. An electrical device for recording sound on magnetic tape for playback at a later time.

A small embedded systems tracer with support for bare-metal and FreeRTOS-based targets.

Overview

Tonbandgerät's core is a small, portable, trace event generator and handler written in C and designed for embedded systems. It requires minimal porting and configuration, and features multiple backends for gathering and transmitting traces.

It can be used both with an RTOS, or in bare-metal environments to instrument user code and track hardware events by tracing interrupts. Full tracing of FreeRTOS tasks and resources is also supported out-of-the-box.

Tonbandgerät is based on a simple custom binary trace format, designed to be fairly fast to encode and keep traces as small as possible. Recorded traces can be viewed in Google's in-browser perfetto trace viewer after conversion with the rust-based tband CLI tool or in-browser converter. The latter runs a WASM-version of the rust conversion tool directly in the browser.

Using Tonbandgerät

🚧 Note

Tonbandgerät is in early development and by no means considered stable. Everything - including the binary trace format - is subject to change. Please report any issues here.

This is a quick overview of the technical details that went into building Tonbandgerät. If you are interested in simply using it, take a look at the following resources:

Trace Flow

On the microcontroller, trace events are generated by manually calling tracing functions or by using the provided FreeRTOS tracing hooks. Each of these events is encoded into a custom binary format and then passed to one of the included tracing backends.

Snapshot Backend

The snapshot backend, once triggered, fills a RAM buffer with tracing events until it is full. The trace data can then be sent to the PC for analysis using a user-provided communication interface.

Streaming Backend

The streaming backend immediately passes every trace event to the user-provided communication interface to be sent to the PC in real time. Note that this requires a fast interface: RTT is recommended.

Postmortem Backend

🚧 Not yet implemented 🚧

The postmortem backend will continuously fill a ring buffer until stopped. The trace data of the last moments before the backend was stopped can then be sent to the PC for analysis using a user-provided communication interface.

Trace Converter

Next, the binary trace stream is decoded and interpreted using a rust-based conversion tool 🦀. A CLI is provided. To keep the overhead of trace event generation and handling as low as possible, only absolutely necessary information is included in each event. The conversion tool takes care of connecting the dots between the different events to generate a rich trace representation.

This trace representation is then converted into the native protobuffer-based input format of Google's in-browser perfetto trace viewer.

Web Frontend

As an alternative to the command line trace converted, a web tool is also provided. This uses the same rust-based trace converter compiled to WASM bytecode, allowing it to run in the browser. A simple Vue-based web frontend makes it simple to upload or paste-in trace data, and visualize it in perfetto with a single click.

Trace Format

To store trace events in a compact binary format, Tonbandgerät uses zero-delimited COBS frames each containing a single trace event. The set of trace event types (each identified by an 8-bit id) and their respective structure is fixed.

Frame Structure

Inside each COBS frame, each event type is structure as follows:

• An 8-bit ID, which identifies the type of trace event and hence internal structure of the fields to follow.
• Zero or more required known-length fields, which are always present in every event instance of this type.
• One of the following:
  • Zero or more optional known-length fields, which (under some restrictions) are not required to be present in every event instance of this type.
  • One variable-length field.
  • Nothing

Importantly, an instance of an event type that permits some $N$ optional fields, may only omit the last $0 \leq n \leq N$ optional fields. In other words, if an event type contains the optional fields $\{ A, B, C \}$ valid event instances could only take on one of the following layouts: $\emptyset$ , $\{A\}$ , $\{A, B\}$ , or $\{A, B, C\}$ .

This restrictive format stems from the fact that there are no field IDs or other metadata encoded in the frame, and the decoder relies on the field types and framing. Specifically, the first block of required known-length fields can directly be decoded because the length of each field is fixed or can be determined based on its varlen encoding. If an event type specifies optional fields, the decoder will continue decoding until the frame ends. if an event type ends with a variable-length field, all bytes beyond the last required field are attributed to it.

Field Encoding

Many trace events feature large fields that often contain only a very small value. For example, all interrupt-related events feature a 32-bit interrupt ID, but most microcontrollers will at most have a few hundred interrupts. This would mean that most bytes of most numeric fields are zero most of the time, wasting trace storage capacity or transfer bandwidth.

To combat this, Tonbandgerät uses the same specific form of variable-length (varlen) encoding that is also by UTF-8 for most numeric values:

Values are split into 7-bit septets, and are encoding starting with the least significant septet. Each septet is encoded as an 8-bit value, consisting of the septet in the lower bits, and a control bit in the most significant bit position that is set to 1 if there are more septets to follow, or 0 if this is the last septet and all following bits should be assumed to be zero.

For example, consider the 32-bit value 0x5. With the scheme above, it is encoded as a single byte:

    +---> First 7 bits
 ___|___
00000101
|
+--> No more bits to follow.

The value 0xFF requires more than seven bits and therefor is split into two bytes:

    +---> First 7 bits          +---> Next 7 bits
 ___|___                     ___|___
11111111                    00000001
|                           |
+--> More bits to follow.   +--> No more bits to follow.

This system trades a much improved average message length for a longer worst-case message size.

COBS Framing

The tracer uses COBS (Consistent Overhead Byte Stuffing, wikipedia) framing to separate binary trace events that have been stored or transmitted together. Specifically, the COBS algorithm removes all zeroes from a binary message in a reversible fashion, with only minimal overhead. Zeroes are then used to delimit individual trace messages.

Specifically, after COBS framing, an $N \neq 0$ byte message will be at most $1 + \left\lceil \frac{N}{254} \right\rceil + N$ bytes long, including a trailing zero for delimination. Please read the above article for a more precise specification, but roughly speaking this is done by replacing all zeroes with a pointer to the next zero:

Original values:          0x01  0x02  0x00  0x04  0x00  0x05

                      +-----> +3 -----+  +-> +2 --+  +-> +2 --+
                      |               |  |        |  |        |
COBS framed:       0x03   0x01  0x02  0x02  0x04  0x02  0x05  0x00

                   Start  Data  Data  Zero  Data  Zero  Data  Delim.

Special care has to be given to a run of 254 or more consecutive non-zero bytes, as an 8-bit pointer is not sufficient to point beyond such a run. In this case, an additional pointer byte is added that does not correspond to a zero in the original data:

Original values:          0x00  0x01  0x02 ... 0xFD  0xFE         0xFF

                      +->-+  +----------> +255 ------------+  +--> +2 --+
                      |   |  |                             |  |         |
COBS framed:       0x01   0xFF  0x01  0x02 ... 0xFD  0xFE  0x02   0xFF  0x00

                  Start   Zero  Data  Zero ... Data  Zero  Extra  Data  Delim.

When decoding, the value pointed at by a 0xFF pointer must therefor not be decoded to a zero but only be interpreted as a another pointer. Because trace events are usually very short, this means that most can be framed with only two bytes of overhead.

Code Generator

The set of possible tracing events and their fields is defined in a simple python code script. This, in turn, generates the c event encoder, an event decoder test file, the rust event decoder, and the event index documentation.

Consider the following isr_name event as an example:

Based on this, the following encoder and decoder are generated:

#define EVT_ISR_NAME_IS_METADATA (1)
#define EVT_ISR_NAME_MAXLEN (COBS_MAXLEN((6 + tband_configMAX_STR_LEN)))
static inline size_t encode_isr_name(uint8_t buf[EVT_ISR_NAME_MAXLEN], uint32_t isr_id, const char *name) {
  struct cobs_state cobs = cobs_start(buf);
  encode_u8(&cobs, 0x3);
  encode_u32(&cobs, isr_id);
  encode_str(&cobs, name);
  return cobs_finish(&cobs);
}

#[derive(Debug, Clone, Serialize)]
pub struct BaseIsrNameEvt {
    pub isr_id: u32,
    pub name: String,
}

impl BaseIsrNameEvt {
    fn decode(buf: &[u8], current_idx: &mut usize) -> anyhow::Result<RawEvt> {
        let isr_id = decode_u32(buf, current_idx).context("Failed to decode 'isr_id' u32 field.")?;
        let name = decode_string(buf, current_idx)?;
        if bytes_left(buf, *current_idx) {
            return Err(anyhow!("Loose bytes at end of 'IsrName' event."));
        }
        Ok(RawEvt::BaseMetadata(BaseMetadataEvt::IsrName(Self { isr_id, name })))
    }
}

Links

• 📁 Repo
• 📼 Online Converter
• 📚 Docs