Software Development from Saleae - our commitment

Hi everyone,

Definition of a good software development team, imo:

  1. The core software product just works
  2. There are a small set of known issues at any one time, and those issues are quickly resolved.
  3. There is a rapid release cadence (e.g. release every 2 weeks)
  4. It’s obvious that that user feedback is getting turned into shipping software at a brisk and predictable cadence.

It’s our intention that items 1-3 to be true by the end of Q3 (October 1) and all 4 be true by the end of the year.

Do you agree with the list? Let me know what you really think :slight_smile:

  1. Yes, the ideal is that the core functionality works, always works, and nobody could dispute that it works like it should. Of course, that’s quite a hurdle to overcome much of the time. But, it’s certainly the proper goal.

  2. Well, you can’t entirely control the # of known issues but you can control how quickly you look into them. Of course, not releasing buggy software goes a long way toward controlling the number of issues that could pop up. But, I know all too well how interesting cross platform development can be. What works on one machine might not work on another, especially if the OS changes. It’s a bit tough to control how quickly an issue is resolved. Some issues are quite nasty and hard to pin down. But, it is a worthy goal to make every effort to quickly respond to an issue and determine what is the cause and what is the solution.

  3. Not that important really. Honestly, I’ve felt like your pace of development with the Logic software has been a bit slow at times. But, for quite a long time the software has done what I really needed it to do. Going back to point #1 - it has pretty much been there for me. I can fire it up and get digital captures with my ancient Logic and it works properly. Now, I like new shiny things as much as anyone. So, I routinely go and check for updates because “ohh… shiny!” and usually you don’t have any. I don’t sweat it, the program worked anyway and I use it fairly regularly. Would I like faster updates? Sure, there are VERY few projects where the release schedule is too fast for the end users. Mostly we’re all waiting around for a new version with bated breath. But, don’t feel like you have to release rapidly if there isn’t really anything new to say. If a release ends up being something like “we tweaked some backend code this week and you won’t notice any difference but we feel better” then I don’t give a crap. Don’t bother releasing something that isn’t really worth mentioning. Now, sure, if you are trying out some new GUI stuff and you want to know how people like it, sure, release that and see what people say. In that case rapid releases help you to internally steer development where your customers want it to go. But, don’t push yourself to release any faster than what makes sense at the current time. If that’s every week for a while, great. If no releases come for a month because you’re trying to get things just right, we’ll wait.

  4. People would like that. All too often user comments can end up as black holes where nobody knows if they ought even have bothered. I’m as guilty of that as anyone, black pots and kettles and all that. But, brisk and predictable can be tough to do. What if what the users want you to do is something really big, really tough. Maybe it becomes apparent that users want a GUI that’s just totally different from what you have? That’s a tough sell. Do you do it? Perhaps you do but that’s neither brisk nor likely predictable. How long will such a thing take? Software developers are notoriously bad at time projections. I think what you really want more than the promise of brisk and predictable development cadence is brisk and predictable FEEDBACK on the development. I know as much as anyone that writing software can be tough. So, I get it. Sometimes you’ll make great progress and sometimes you won’t. It happens. What people really want isn’t for you to kill yourself trying to make a deadline. All anyone really is asking for is development transparency. Tell us what you’re working on, tell us how it’s going. Tell us when you hope things might be done and update us if that isn’t going to happen. That goes a long way toward keeping your customers happy. I really do think the key is keeping people in the loop and then letting it take as long as it takes to do it right. Valve Software has this about half right. They only release games when they’re done. If it takes 2 years, it takes 2 years. If it takes 3 or 4 then that’s how long it takes. What they fail to do is let anyone know about this. I guess that helps to alleviate the million questions and accusations. Some people are stupid enough that if you tell them “I’m working on this new feature and it should be ready in a month” they will assume that Jesus came down from the clouds with stone tablets and personally gave them his solemn word that the feature will be delivered no later than one month. And, if you miss that date they will hate you forever and swear to avenge this slight out to the seventh generation. But, hopefully such people are rare.

Well, you asked what people really think. I think a lot of times people are pretty squeamish to actually do that. Some of us will do it anyway. :wink:

:+1:

Well said. Referring to the C++ back end, I think we’re in a much better position that in the past in terms of software quality. Our game has been upped dramatically, thanks to some amazing consultants such as Anthony Williams. Great separation of concerns and solid test coverage. Conforming to the C++ standard guidelines (Clang Tidy), Google Thread Sanitizer, Valgrind, etc, all run automatically in CI. We also have a great logging solution that gives really good graphical insight into the behavior of the system, which we never had before.

For me the primary concern is with the USB device interface because of the difficulty of testing it - that has not yet been modernized either. Our goal is to have someone manage a garage full of computers, each with different OSs and USB host controllers,- attached to real devices and USB-controlled “connection exercisers” to connect and disconnect them – and then to run an extensive test suite repeatedly every time new code is pushed to CI.

Well taken. It’s more fun to ship often. I think it forces good tradeoffs - forces breaking down features to figure out what’s really important to the user and what can be improved later. And you’re right, we don’t want to bother the user with every minor back-end change. That’s why we’ve got to have a totally painless auto-update system, like VS Code, Github Desktop, Chrome, etc.

Excellent point. What do you suggest the best way to do that is? I’ve been thinking about just making our sprint board public, something like that. Then people can see what’s happening now, what’s planned for next month, next quarter, the next few quarters.

Hey thanks for the help Collin! Look forward to hearing more from you in the months ahead.

I kind of like the general brevity of GIT commits. Usually you want to try to explain what you did in one or two lines. That’s a good size for status updates. One thing I’ve seen is where projects have a release scheduled and that release has a list of GIT issues that must be checked off before the release can ship. The system thus can show X/Y where X is the # of issues satisfied so far and Y is the total # needed before the release is finalized. Obviously you’d want to be able to add/subtract from the list if necessity demands but at least people could see “we did this” and “we still have to do this”. As that whole thing would be tied to your internal processes it is automatically sync’d and you don’t really have anything extra to do. So, not sure whether you’d be thinking along that route or not but it’s one possible way. It also helps to alleviate the “but… YOU PROMISED!” situation. You don’t promise anything. You just list what you have to do and what you already did. From there people can form their own conclusions about the timing. Perhaps you can write short blogs about the pace of development if you’d like but with the progress in black and white already I think you won’t need to provide much else to satisfy people’s curiosity.

1 Like

Thanks Colin, I’ll try doing something along those lines. Will take some iteration to get it dialed in I think.

I’ve posted a ‘status update’. Not super happy with it, but it’s a step in the right direction I think.