Improved text message HLA & data formatting question

I have extended the ‘text messages’ HLA by adding another column in the list of messages which displays messages as hex values.

I needed this because I am analyzing a protocol that sends messages consisting of multiple bytes and uses a frame timeout. I wanted to concatenate these bytes like this HLA does, but they are in binary format rather than readable text, so I added a hex column.

The extra column worked right away, but I noticed that there is a row for the HLA in the waveform display which still showed the text messages. I “fixed” this by changing the line that sets result_types.message.format, but ultimately it would be nice to let the user select the format. The context menu for the HLA already has four options (Binary, Decimal, Hexadecimal, ASCII) but whatever is selected there has no effect. Ideally, that selection should determine how the messages are formatted. I couldn’t find how to achieve this.

By the way: why does this HLA only define a single result type “message” and dumps things like errors and I2C addresses into that message as strings, just like received data? From the documentation it seems that it would be more appropriate to define two more result types “error” and “I2C address” so that “message” is really only for received messages. Or did I get this wrong?

My fork of the HLA is available here:

@dennis Thanks so much for making those additions! I could certainly see the usefulness of adding a hex column for non-text data.

I’m going to shoot your message over to Mark, the creator of the Text Messages HLA.

Nice work @dennis!

Right now the display base in the settings menu only affects the data table. There are a lot of improvements we need to make here, but we won’t be able to get back to extensions development for a while, we’re focusing on other areas of the application at the moment.

I suggest using the HLA settings API instead. You can add settings to the HLA settings dialog, the same dialog where you select the input LLA.

You can use the delimiter option as an example:

You could add a new setting to select Ascii or Hex (or others).

Given that this is a pretty big expansion to this HLA, I suggest publishing it as a new extension. The purpose of the “Text Messages” HLA was to make it easy to read strings sent over UART connections. This is makes it more of a generic packetizer.

If the protocol you’re working on is a standard, you could take this further and customize the HLA to decode information specific to the protocol.

You’re right about that. This HLA was created as a simple demonstrator, to provide some value but to also act as a simple example extension to help people understand how it works and provide another starting point for users to modify.

Thank you for the feedback!

The protocol I am working on is not a standard, it’s something I came up with myself. Background: a device periodically needs to send binary 16-bit messages over one pin. Having 16 data bits per packet seems a bad idea, since it would require super accurate clocks (besides being non-standard), so I’m sending another start bit in the middle, i.e. essentially two packets with 8 bits each. Now the receiver still needs to connect two bytes to a message and it needs to know where a message starts (otherwise it might forever interpret the second byte as the first). This can be easily achieved by leaving some minimum time between messages and working with a timeout. The “Text Messages” plugin provided all that functionality, that’s why I used it. I don’t need the delimiters, just the timeout. While this is not a standard protocol, I think it is a common use case and it would be useful to have a HLA for this.

I will turn this into a plugin of its own. However, I am not convinced that it’s good to have tons of different analyzers with little differences. I agree it’s good to have a simple example for demonstration purposes, but when it comes to a collection of analyzers for regular use, a generalized HLA with more settings appeals to me more than multiple specialized ones that are similar.

I submitted it under a new name: Concatenator

I added a setting for the format to choose between ASCII and Hex. I also added the delimiter option “None” and changed the time unit to micro-seconds.

The repository is here: GitHub - pastaclub/hla-concatenator

@dennis I can confirm that your HLA has been added. I just tried it out myself. Thanks for contributing this!

You’re right, one of the big missing pieces of the marketplace is organization and management. We don’t have any features for this at the moment.
Ideally, we would have tags, search, ratings and comments, and perhaps some top level categories. We’ll also want to focus on the most common needs that we discover through the marketplace, and make sure these needs are well covered by either Saleae written extensions, or community extensions. It’s still very early days at the moment. Looking forward to it!

Where is it better to report a bug?
Concatenator does not parse last capture message.

I use only one parameter of Concatenator - “Packet timeout [us]” and set its value to 1000, which is quite small value.
It works fine and makes combined messages for all the bytes packets, except of last bytes packets.
Looks like Concatenator waits for a next byte after the configured delay, but when capture ends, there always will be not concatenated message.

@Oleksii It doesn’t look like the original author of the Concatenator HLA (@dennis ) has enabled Issues to be posted on their GitHub repository for the Concatenator HLA, linked below.

Therefore, this forum post may be the best way to reach @dennis.

Having said that, I’m not able to reproduce the issue when using the Concatenator HLA on top of the I2C analyzer. I’ve attached some screenshots below, including a sample capture file where you can add the Concatentor HLA to test it.
i2c-capture.sal (6.1 KB)

1 Like

Thank you for paying attention on this !
I’m sorry that I did not specify right away that I meant Serial capture.
Here is example on speed 9600, 2 bytes ASCII A. 1st is parsed by concatenator, 2nd - not:

The file to check it quickly
serial-9600-capture.sal (5.3 KB)

I’m the author of the extension. Thank you for the bug report. I just enabled Github issues (not sure why it was not already enabled).

Your diagnosis @Oleksii sounds completely right. The way the code works is that it keeps adding new bytes to a local variable as they come in, and when a timeout event happens (or when a delimiter is received), it commits those concatenated bytes back to the API. In your case, the capture ends before the timeout, so there is nothing that makes the extension commit the current message.

It’s been 3 years, but I remember my application had very short timeouts so I never ran into the bug.

The timeout handling happens in concatenator.py in lines 194+
(I am not allowed to post a link to github)

In order to fix it, we would have to detect the end of input and after the last frame commit the current message if it’s not empty. This could happen at the very end of the python file. But I am not sure how we would know when decode() is called for the last time. @markgarrison ?

Also, some of my messages above were marked as SPAM by the forum software, presumably because they contain links to github. I received 3 e-mails urging me to remove the “advertisement which is overly promotional in nature instead of being useful or relevant”. Well, the contrary is true: if I remove the links, the thread will become less useful, so please unflag them.

2 Likes

@dennis Sorry about your posts getting take down! It looks like Discourse (our forum platform provider) is fairly aggressive with marking posts as spam when it contains links that have previously been posted.

I ended up easing that threshold, and also manually added github to the allow list. I also unflagged your previous posts.

I was able to reproduce the issue with @Oleksii 's capture file.

The points you brought up above seem correct, and to my best recollection, we don’t have a way of detecting when the HLA reaches the end of the capture. Unfortunately, I don’t think we have a workaround for this, but I’ll discuss this with the team here and will let you know otherwise.

1 Like