High-Level Analyzer timestamp rules make it impossible to buffer & parse interleaved multi-packet messages

Extensions
1

Description

I’m writing a High-Level Analyzer for a CAN-based protocol (Varex) where a single logical message can be split across multiple packets, and packets from different messages can be interleaved on the bus.

The screenshot below (you can see it in the attachment) shows the situation:

  • There is a header + payload (Packet 1) of Message A

  • Then the header of Message B appears in the middle

  • Then the payload (Packet 2) of Message A continues after that

Some messages consist only of a header with no payload. Others have a payload with multiple fields, and some values are 4-byte floats spanning several bytes.

From a protocol-parsing perspective, the natural thing to do is:

  1. Accumulate all bytes for one logical message into an array of bytes (regardless of how many CAN packets it is split into, and even if those packets are interleaved with other messages).

  2. Once the full message is received, parse the byte array into values (including multi-byte fields, floats, etc.).

  3. Emit the corresponding AnalyzerFrames for those values after the message is fully known.

This is much simpler than trying to keep state per individual byte or packet and emitting frames as bytes arrive.

Problem: timestamp monotonicity & interleaving

The issue is that the High-Level Analyzer framework enforces strictly increasing timestamps per frame type. If I try to implement the “buffer then emit” approach, I hit the following error:

CanVarexAnalyzer error - Invalid begin time for frame type 'xyz', begin must start after the previous frame.

Here’s a concrete example of what happens:

  • While decoding, I emit a message frame for the header of Message B, which appears between packet 1 and packet 2 of Message A, because this message is a simple message with only the header, and thus can be treated right away, without waiting for the payload to come.

  • Later, once I have collected all bytes of Message A, I want to emit value frames for Message A using the original timestamps of the bytes, which are slightly earlier because those bytes came from packet 1 that was on the wire before the header of Message B.

  • When I emit these frames with earlier timestamps than the already-emitted header frame, Logic 2 raises the “Invalid begin time” error.

From the protocol’s point of view, this is legitimate: I’m simply adding more semantic information about earlier packets once I have seen the complete message. I am not overlapping times (the new frames still sit in gaps of time that are not already covered by other frames); I just need to go “back in time” slightly for ordering.

Why the current behavior is limiting

  • Treating bytes strictly “as they come” and forcing all AnalyzerFrames to be emitted in real-time order makes complex parsers much harder.

  • For messages with multi-byte values (e.g. 4-byte floats) split across interleaved packets, I need to track state across bytes and packets, and emit partial or placeholder frames before I actually have all data. That is error-prone and much harder to maintain.

  • The more natural approach—buffering bytes for one logical message, then parsing and emitting frames when it is complete—is effectively blocked by the timestamp monotonicity requirement.

  • I tried another workaround: buffer all AnalyzerFrames in a list, sort them by timestamp at the end, and then emit them in order. But the HighLevelAnalyzer API only provides __init__ and decode. There is no “end-of-capture” or “finalize” callback where I can know that all frames have been seen and safely flush/sort the buffered frames.

Because of this combination (strict per-type timestamp monotonicity + no finalize hook), some protocols with interleaved, multi-packet messages are very hard or impossible to implement cleanly.

Expected / Desired Behavior

It would be extremely helpful if at least one of the following were supported:

  1. Relaxed timestamp rule:
    Allow a frame’s start_time to go backward as long as it does not overlap with an existing frame of the same type. In other words, reject overlaps, not merely non-monotonicity. This would allow analyzers to emit semantic frames later, using earlier timestamps, without breaking other frames.

  2. End-of-capture / finalize hook:
    Add a method such as:

    def finalize(self):
    # called once after all frames have been passed to decode()
    ...

    so that:

    • I can buffer AnalyzerFrames during decode(),

    • Sort them by timestamp in finalize(),

    • And then return them all in correct chronological order.

  3. Official support for buffering + sorted emission:
    For example, allowing decode() to return an empty list while buffering internally, and then a dedicated way for Logic 2 to ask the analyzer for “any remaining frames” at the end.

Summary

Because my protocol allows interleaved packets from multiple messages, and because I need to parse multi-byte values across packets, the most robust pattern is:

  • Buffer bytes per logical message → parse whole message → emit semantic frames after the fact.

The current High-Level Analyzer behavior (strict start_time monotonicity and no finalize hook) makes this pattern very difficult and leads to errors like:

Invalid begin time for frame type 'xyz', begin must start after the previous frame.

Relaxing the timestamp constraint to focus on overlaps instead of strictly increasing order, or providing a finalize mechanism, would greatly improve the flexibility of the HLA API and make it much easier to support complex, interleaved protocols like this one.

Thanks for considering this — I think such a change would help a lot of analyzer authors dealing with real-world bus traffic where messages are not nicely isolated or strictly sequential.

Here is a photo showing what I have described:

image
image3253×338 38.7 KB

2

This has been a problem I wish Saleae solved back in the older Logic software versions prior to HLA and V2 frames even existing. They used to have a concept of “transactions” documented in their API but it was never implemented. I made a similar post years ago on my desire for them to support this functionality but have not heard much interest in them improving on this.

This is a feature they really should try to figure out how to support either at the LLA or HLA level as this is not a unique issue, many protocols use some concept of sessions/transactions to allow multiple data streams to happen at the same time (roughly speaking).

This feature could have been utilized in the AbccSpiAnalyzer LLA plugin I wrote years ago, it would be useful for many CAN-based protocols that I work with today, and probably other protocols that are escaping my memory.

That said, I can appreciate the challenges this may pose regarding how to display this graphically in an intuitive way to a user (in a scalable way).

3

I think this is a fundamental limitation of how the analyzer is being displayed – consecutive rectangles that one must finish before another can begin, as they are all on the same horizontal line above the channel they’re analyzing.

If you let multiple output frames overlap, how should it be displayed? Allow an arbitrary vertical nesting? How is it vertically ‘sorted’ for different scenarios?

Instead, can you just add a ‘message type’ or ‘message id’ field to your analyzer, and then use multiple instances on the same channel? Each would only output the message(s) that match the ‘type’ or ‘id’ provided. Alternatively, you could auto-detect the type/id and have it ‘skip’ to the N-th different message. Each instance would increment the ‘skip to N-th’ setting by one. First instance decodes immediately, and show only the first matching message while next instance skips first, and only shows all messages matching the second message in the capture.

These two methods will allow you to decide which messages appear along which analyzer line, rather than the UI logic trying to ‘auto-layout’ overlapping frames.

4

My suggestion for solving this problem is to allow multiple stacked tracks (channels) for the analyser display track. The analyser determines which track a specific piece of output is shown in so that each active session or transaction is assigned to its own track. That avoids the “timestamp monotonicity” issue and makes it easy for the user to disentangle information from multiple interleaved transactions.

1 Like
5

Yes! This is another great solution! I was thinking about this exact solution today :slight_smile:

6

@P.Jaquiery I like the idea of ‘stacked tracks’ (or channels) … and you can emulate it now with multiple instances of the same analyzer, with a ‘track’ setting (0..N-1), such that each instance only outputs for its designated ‘track’, in lieu of having all channels handled by a single instance. The UI update could just add a suffix to each channel (.0, .1, etc) and treat the channels as separate ‘virtual analyzer’ streams for other output displays (console, table, etc.)

7

For the interleaved messages the instances would become independent and not aware what is going on in their “colleagues”.
The multitrack feature would be gold.
Or at least possibility to change colors in one line/track on the go as HLA detects different massages (or in my case it would be different pars of the packet)