Software Dev Status

Q2, Sprint 6 of 6

Alpha 6 should ship in < 2 weeks (before June 21). This will include Protocol Analyzers.

We’re behind schedule due to some complications migrating the protocol analyzers to the new system. It’s work we needed to do eventually anyway, but now needs to be done asap in order to ship analyzers.

Michael is working on back-end support for Export and Saving Captures to Disk, on advanced cross-platform logging, and getting the back-end CI to “all green”.

Mark is working on finishing the migration off of the legacy Logic codebase, over to something called “Graph Server”. This move needs to be done right away due to some unforeseen technical complications with our migration of the protocol analyzers.

Anthony is working on migrating Logic Device support over to the new back end.

Oz is working on Export, Saving Captures to Disk, Themes, and open issues.

Rani is working on Protocol Analyzers, Electron, C++ crash reporting, and open issues.

1 Like

Mark has completed the first version of graph_server.exe and graph_server.dll which will be used by the Electron front-end instead of the legacy-codebase-hybrid currently in use. Now he’s working on the last loose ends of getting the new Analyzer Node working in the back end.

Rani is working on getting device calibration support into the front end, and later today he’ll be working on integrating graph_server.dll with Electron.

The backend is called “the graph” because it is comprised of a configurable set of data-processing ‘Nodes’ of various types connected by ‘Pipes’. Similar to systems such as GNU Radio, Simulink, etc.

This image gives the idea, although it’s not exactly what we ended up building:

Quick picture of “The Board” as it happens to appear today.

No guarantees, these get reshuffled periodically, but I am happy to be able to report that what we planned on getting done for Q2 (this quarter) actually has mostly been accomplished (exceeded in some respects), and so there is some reason to be hopeful that this isn’t all that far fetched or far away. Btw, the vast majority of things people want (as indicated by https://ideas.saleae.com) is actually covered on the board here. With the exception of specific protocol analyzers, which is a discussion for another time but certainly also on the radar.

Today and yesterday Rani & Mark put in some late nights and got Electron working properly with the new backend.

The primary piece still remaining is the “Usb Devices” support – which Anthony is working on – and that’ll be the last open item to be 100% off the legacy codebase. That’s coming along well. Good chance of integrating the first partial implementation of that tomorrow.

Oz has the first revision of Export done, and Michael just completed support for serializing digital and analog data on the back end.

Joe

We try to have a board that clearly shows everyone in the company what’s happening with the software during each sprint.

Right now it’s super backlogged with items that all (or largely) depend on each other before they can be released. This is normally a terrible sign, since we want to be able to develop features independently and rapidly - shipping each as soon as they are done.

Right now we’re in the midst of the final push to ditch the legacy software, so there are a lot of loose ends creating a backlog. We’re actually quite happy with the progress though, and should be able to ship all these items by the end of the week or shortly thereafter.

Mark and Rani put in a lot of hours this week pushing to get everything ready - they are shooting to have a totally integrated build tonight.

Given the sheer amount of the front-end codebase that has been modified in the last 1-2 weeks, it’ll take a few more days to get something stable enough to release. But the good news is we should be totally off the legacy codebase by tonight.

The next release should be next week by Friday. This is later than we had hoped, but given the incredible step forward and the work it took to get here, I’m quite proud of the team.

This next release will pull together:

  • Data Export
  • Saving captures
  • Electron (no more Chrome!)
  • Robust error logging
  • Protocol Analyzers

Next priority is stability and “I can use this for real work”. Looking forward to seeing how quickly we can converge on that!

Happy July 1 - start of Q3!

Internally we have a working build, and are working a bunch of issues against it. We are behind relative to my last update sorry to say. The next ~4 weeks are dedicated to total stability / engineering loose ends. That will set us up nicely to work on features the remainder of the quarter.

Here are the planned software tasks for this quarter. The numbers along the top are the six two-week “sprints” of Q3. Each ‘lane’ is a different developer; top to bottom is Oz, Mark, Rani, Michael, and Anthony. By the end of Q3 you should no longer need to use the old software except in rare cases.

Finally, a release! Alpha Release 6

I think we’re over the hump now: We’re solidly on electron and 100% off the old codebase. Our CI and release procedure is reasonably solid. The release cadence should get much higher from here on out - should be once every two weeks at the minimum.

1 Like

We’ve shipped Alpha 7, which includes protocol analyzers for the first times. Plenty of issues we’re working through - should have a lot of fixes to roll out later in the week.

Rani and Michael have been working on protocol search - it has a good chance of release this week as well.

Oz has been working on application settings persistence (the app doesn’t remember much between runs right now) as well as support for presets.

All for now!

Things are bit behind where we’d thought they would be unfortunately, but progress continues. Here’s what we’re working on now and should ship by Friday next week:

  • persistence
  • settings presets
  • analyzer table view & search
  • UI/UX refresh for capture settings
  • collapsible and resizable sidebars
  • proper installation on Windows, including drivers
  • proper dependency management on Ubuntu

Q3, Sprint 5 of 6

I am sorry to report that this quarter has not gone nearly as well as we all hoped. There are a number of reasons for this, all of which are being worked on.

However the good news is that the choice of Electron, and HTML/Typescript/Javascript for the main application development was absolutely the right choice. This has been super fast to develop for, super easy to iterate, to debug, etc. So all we have to do is bring the other areas of the project up to the same level of developer productivity and we’ll be golden.

Development cadence on – and integration with – the C++ data-processing framework has been unexpectedly slow and problematic.

The main problem is that the C++ framework was not initially designed to be easy to diagnose issues on - i.e. if the application developer sends an invalid command in JSON, the back end will typically assert - by design - which is very difficult to work with without a complete C++ debug setup and intimate knowledge of the C++ codebase – which are exactly what we don’t want to require of the application developers.

A v2 of the C++ architecture should remedy this issue, by essentially never asserting, providing rich error logging as well as rich logging in general, and by making it impossible for there to be JSON conversion errors. It will also be simplified by a factor of x2-x3 since we’ve found that my v1 architecture was overdesigned. This is embarrassing for a number of reasons and not something (obviously) I would repeat today. All in all though, it was directionally quite good.

There are two other issues with the C++ back end - up to now our developers for it were remote - and the other issue is that C++ itself just isn’t a rapid application development language. Modern C++ is much better, but not enough. So I personally am seriously considering if we should move off C++ for everything except for, essentially, performance critical algorithms. C# (.NET Core) is attractive, as are potentially Rust and Go. In the case of Rust, there may not be a reason to use C++ at all, even for the inner loops of performance critical, SIMD code (!). The choice is really just for the framework itself anyway, since in v2 the data processing ‘Nodes’ themselves are intended to be able to be developed in any language (e.g. Python).

I’m super excited to get to the point where the back end data processing development can go just as fast as the front-end application development!