Hello there, long time user of your devices, very happy with them.
Recently, I’ve been trying without success to use the automation API to export raw binary data of some very large analog captures (the .sal file is around 13GB.)
While I can manually export the raw data just fine (each of the 9 device channel analog.bin files comes to about 14GB), using the following Python code results in 9 files, each only 135MB in size:
from saleae import automation
fname = "capture.sal"
m = automation.Manager.launch("/usr/local/bin/Logic", port=10430)
with m.load_capture(fname) as capture:
capture.export_raw_data_binary("/home/cad")
m.close()
I do see Logic (latest, v2.4.44) appear, and about a 15-minute wait for it to load and decompress the .sal file, the save process ends after just 1-2 seconds and Logic exits. By comparison, on the same machine, saving the exports manually takes about 3 minutes.
@wohali Sorry for the late reply! Can you try loading your capture and exporting your raw data within a with block like in the Getting Started example below? There may be undefined behavior when you don’t wrap your code within a with block like below.
with automation.Manager.connect(port=10430) as manager:
When you exit the with block, it will automatically call close() for you so you don’t need to manually close it.
My hunch is that there may be some kind of race condition occurring with your code.
Unfortunately, using a with closure doesn’t solve the problem. The root cause is that the export_raw_data_binary function is non-blocking.
In this example, using the Manager.launch() function where the Python code is responsible for launching and shutting down Logic, there needs to be a way to know that the save process has finished. Instead, the API call returns with success as soon as the save starts. There is no way for an API client to know when the save has finished.
By using Manager.connect() as in your example, you’re asking the user to separately launch Logic and only use the API to instruct it to start the save, side-stepping this issue.
Without any way for automation code to know when the save has finished, there is no reliable way to automate shutdown of Logic other than something like inotify watching the save directory to see when file sizes finish changing. Idling in a check-sleep loop within the with closure feels like a poor workaround to me.
If you wanted an async API it probably would make sense to have it built around Python asyncio (async/await) if that’s possible. But that’s best in addition to a blocking API.