The binary format is nice and simple. It allows for the case where there are multiple waveforms in a single file. The CSV format can allow the same thing.
Why doesn’t the export data function do this?
Having to read multiple files and merge them myself is error prone and doesn’t make sense given the export formats already support merged files.
I note that this was a requested feature on earlier products as well.
Hi Ted,
Sorry for my delay, I was out for a while and I’m still catching up.
Good question. are you looking for an export option to produce a single binary file that contains data from all channels, analog and digital? Would you imagine just basically concatenating all the files, with some extra header data?
Just to make sure I understand your request, let me start by reviewing our existing export options for Logic MSO:
CSV export. When you’re recording both analog and digital signals with Logic MSO, and you export to CSV, you will get 2 CSV files, one with analog data and one with digital. If we were to combine these, for example have a timestamp column then a column for each channel, analog or digital, the end result would technically make sense but would be very hard to read manually. We wanted to make sure our CSV formats would be pretty easy to use both from code and use manually from excel.
binary export. We create 1 file per channel enabled in Logic MSO. Analog and Digital files each use their own file format. This kept the system pretty simple on our end, but does make it hard to just ask what the state of all inputs were a specific point.
Are you asking for all channels in a single binary file?
Are you asking for two files: all digital channels in one file, and all analog channels in another file?
What are you trying to do with the merged data? For example, if you wanted to know the state of all digital channels at a specific point in time? are you looking to cross reference the analog signal with the digital signal?
Have you worked with any file formats for data like this that you like, and would like to see us support?
(By the way, our *.sal files are actually just zip archives that contain 1 file per channel. We don’t really have any system that combines all data from all channels into a single file, internally or externally)
I have been using your binary output format for analog channels only. The virtue of your binary format is that I can read 50 million samples in a fraction of a second. Reading CSV data takes much longer. The binary format has provision for recording multiple channels so there wouldn’t be any need to change that format if I just had multiple channels in the same file. The big usability boost to having all the channels in a single file is that there is much less chance that traces will get separated.
Your point about separating digital and analog formats makes pretty fair sense, especially if you are doing run length compression on the digital format.
Now, to your specific questions:
Are you asking for all channels in a single binary file?
All analog channels in a single file, yes.
Are you asking for two files: all digital channels in one file, and all analog channels in another file?
That would be fine. All in one file would be lovely, but I get it that such unification might be tricky. Obviously, a merged file format is trivial since it can just have a header saying what is coming and then concatenate a (possibly empty) multichannel digital file followed by a (possibly empty) multichannel analog file.
What are you trying to do with the merged data?
I am doing phase coherent analog processing, typically spectral processing.
For example, if you wanted to know the state of all digital channels at a specific point in time? are you looking to cross reference the analog signal with the digital signal?
I haven’t tried mixed signal processing yet. That will come and the need will be to look at trigger channels or diagnostic signals relative to analog. For instance, my microcontroller is generating signals at different frequencies. It would be desirable to have a digital channel that I can use to signal when I change frequency (and give the frequency as well) and then look at the corresponding window of analog data. This would allow me to exactly correlate back to what the controller should have sampled for accuracy testing. Right now, that kind of correlation strictly depends on keeping all of the traces together which can be very tricky when I am trying to document test results.
Have you worked with any file formats for data like this that you like, and would like to see us support?
I have worked with gazillions of such formats and none stand out very much other than something like, say, protobuf and CSV. Since the formats are so simple, it is easy enough to write a custom interface (yours took me 10-20 minutes to code in Julia, for instance). Rather than supporting yet another format, it would be nicer if you simply used the format you already have to its potential. Regarding your documentation, there were a few points of ambiguity (the details escape me just now) but a tiny bit of experimentation solved that pretty quickly. Supplying sample code in, say, Julia, Go, C++ and Python would be lovely and would speed things up a bit. I would be happy to contribute to that in Go or Julia.