By Lance Eliot, the AI Trends Insider

Do you know how the word “debugging” originated? It is attributed to Grace Hopper, a pioneer in the computer field. When I was first starting out in computers, I won a programming contest and was lucky to have Grace Hopper hand me the trophy. I spoke with her that day, and indeed for the rest of my career it was my honor to keep in touch with her, along with heeding her sage advice along the way.

The story she told about the origins of the word “debugging” goes something like this. In the early days of computing, the rather massive-sized mainframe computers were using vacuum tubes as a means of memory.  Vacuum tubes tended to generate a lot of heat. Grace was working late one evening and some of the windows were opened to let the heat vent to the nighttime sky. All of a sudden, the entire system came to a halt. She was tasked with investigating to figure out what had caused the system to stop.

Upon carefully inspecting the numerous hardware components and especially the vacuum tubes, she came upon one vacuum tube that had shorted out. There was a dead moth that had fluttered in via the open window and had landed on the vacuum tube at the hardware connection point, shorting out the circuit. Similar to a string of Christmas lights that goes dark if one bulb is bad, the shorted out vacuum tube had caused the entire mainframe to come to a halt. Grace removed the moth.

When she was asked what had happened and what she did to fix it, she said that she had “debugged” the system. For many years, the word “debugging” was an insider term that was used mainly by computer people. Eventually, the notion of debugging became popularized and we today use it for any kind of situation in which you fix a problem by removing or repairing some kind of error or bug. Thanks goes to Grace Hopper for adding the term to our global vocabulary!

The glory of programming always shines on the new development of code and tends to downplay or even ignore the nature of debugging a system.

As a former professor, I can attest to the aspect that my colleagues considered debugging to be something that students learning to program should just figure out on their own. It was considered something rather trivial and not worthy of any direct attention. There was a backward kind of logic often used by these colleagues, namely that if someone doing programming does the job “right” there should not be any need for debugging. Only foul-ups need to debug.

With my many years as an executive in software development organizations, I used to try and explain to them that debugging is actually a significant part of the job of professional programmers and software engineers. This idea that somehow code will be perfect is nonsensical.

Many of my academic colleagues had never worked on large-scale applications that consisted of hundreds of thousands or millions of lines of code, and thus for them a “large” program of a few hundred lines they envisioned should be written correctly and without any kind of debugging needed. They imagined programming as writing a mathematical proof.

The general rule-of-thumb in industry is that about one-third to maybe one-half of the development time of programmers or software engineers is consumed by doing debugging. Think about that statement for a moment. It means that relatively expensive labor is spending a significant chunk of their time and attention on debugging. If the amount of time consumed was say 1% or even perhaps 10%, we might be willing to just consider it as a minor aspect of the job, but when it rises to 30% to 50% of your time, it implies that it is substantive. Accordingly, it should be given due attention to try and ensure that it is as well spent as possible, minimizing when feasible the effort and providing suitable means to enhance it.

Oddly enough, in spite of those percentages, the world of system development is still generally in the dark ages when it comes to debugging. It still to this day tends to be the shadowy part of the job. Unspoken. Unheralded.

Here’s then another perhaps startling figure for some, the total cost of a software development over its entire life cycle will usually come to about 50%-75% toward debugging. Once again, this highlights how important debugging is.

I realize that some of you will complain about the 50%-75% figure and point out that the bulk of that debugging comes after the system has already been fielded. The debugging at that juncture often occurs because something in the system environment has changed and so the code itself has to be changed too. Also, there are often new requirements that come up and the code written for those requirements can at times get lumped into “debugging,” which seems like an unfair categorization. I’ll grant you that all of that is the case, and so we might debate somewhat about what should be tossed into the “debugging” bucket, but nonetheless there’s really not much question that debugging is a hefty proportion of the total life cycle cost.

I’d wager that most developers learn how to do debugging the old-fashioned way, simply by trial-and-error and learning it by the seat of their pants. No one likely explained to them the various techniques of debugging. Any of the tricks of debugging they had to discover on their own. They might have had someone more senior that gave them some pointers or aided them when doing some rather knotty debugging, which helped a little bit to get up the debugging learning curve. Overall, debugging is seen as a kind of apprenticeship skill and one that is considered more art than science.

Companies that provide specialized tools for debugging have often struggled to get developers to actively use the tools. Often, a developer will use such a debugging tool in a very rudimentary manner and avoid going to the effort to get into any more advanced debugging features. This lack of exploring those other debugging features might be due to a lack of awareness of what advantages they provide. There is also the aspect that no one is necessarily urging them to dive more deeply into being skilled at debugging. Most development managers assume it is just something that ripens as you get longer in the tooth in terms of developing more and more kinds of systems. By osmosis, the more you code, you just somehow presumably will get better at debugging.

Of course, one Catch-22 for many developers is that they are so pressured to debug systems that they don’t have the time to focus on getting more effective and efficient at debugging. They just keep plowing away on fixing problems, and unfortunately over time there is not much advancement in their debugging other than becoming more familiar with a particular system that perhaps they are assigned to. Thus, the familiarity boosts their debugging rather than due to an increase in their actual debugging skillset.

Some will say that maybe I am glorifying the use of debugging tools and suggesting that debugging tools are a kind of silver bullet. No, I’m not in that camp. I know some that believe that we should be able to automate entirely the debugging process and, in a sense, take the developer out of the equation. There have been many attempts at applying AI to the debugging process. This involves infusing into the debugging tools the “mindset” of a developer.

I’d say we are about as far along on having AI that can generate new systems for us as we are on AI that can do debugging for us, which is to say that we aren’t very far along on that path. Researchers that look at programming as nothing more than assembling Lego blocks have tried to come up with a means to automatically produce code and automatically debug code. Most of those efforts have fallen short by quite a bit. For now, systems development still seems to be a cognitively complex effort requiring human intelligence and skill. Maybe someday it will be more routinized, but not for the foreseeable future, I’d say.

Speaking of the mindset of a developer, studies of debugging tend to suggest that debugging is often greatly shaped by the mental model of the developer with respect to the system they are developing.

For example, a developer that originally developed a system will often have various assumptions in their mind about what the code is supposed to do. When the code does not perform as such, the developer can at first be quite flummoxed. This is why it is often the case that another developer, one that is not so close to the same code, can at times spot the issue, rather than the original developer, because the other developer does not necessarily carry the same assumption in their mind about the code.

Another element about debugging involves the use of language for developing a system. For most programming languages, there are a myriad of intricacies about what the language proposes a line of code will do, including whether you are using version X or version Y of the language, and whether the compiler or interpreter was established to execute the code in the way that you think it should. And so on. As a result, the programmer might have a mental model that the particular programming language they are using should do something M, when in fact at execution it does something else N.

Initializing of Variables Can Exasperate

One of my favorite such examples involves something so simple that it is wild to think that it still to this day stymies many programmers and many programming languages, and many development efforts. The simple matter of the initialization of variables is something that continues to be the exasperating culprit in many debugging scenarios. I might have written a line of code that says J = G + 1 and suppose that I had neglected to beforehand have something that gave G a value, such as I had meant to beforehand say that G = 4.

At execution time, if I had mistakenly not initialized G, the attempt to read the value of G and add one to it can do all sorts of crazy things. The system might assume that G was intended to be zero, and so at the execution time it decides that J will be calculated as though it says J = 0 + 1. Or, it could be that G is considered to have an undefined value that can be whatever value the system wishes to use, and so maybe the system decides at that moment in time that G is the value 31,727. Or, it could be that the value of G is considered a null, and the system maybe raises an exception or generates an interrupt that the calculation is invalid because the G is non-numeric. Etc.

The point being that our programming languages are so complex and brittle that there is bound to be a gap between what the developer thought would happen versus what really does happen. When I mentioned earlier that perhaps another developer upon looking at code, such as looking at the J = G + 1, might ask how does G get a value, and therefore find the “bug” that the original developer did not see, I don’t want you to assume therefore that all we need to do is have a second developer look over the shoulder of the code of another developer. That’s not a silver bullet either.

Some say that if we focus on the topic of debugging that we are then assuming that we are allowing bugs to be introduced to begin with, and we ought to instead be finding ways to prevent bugs from getting introduced. Well, I certainly agree with the part about wanting to try and avoid introducing bugs. Yes, I’m all for that.

Realistically, you need to realize that bugs are going to get introduced, and thus we need to look at debugging too. Don’t fall into the simpleton argumentative trap of somehow thinking these are mutually exclusive aspects. You can be trying to find ways to prevent bugs from getting included, and you can also be trying to find ways to detect bugs that do get included.

One of my seminal articles on software engineering involved an analysis of the role of so-called software factories. There was a period of time that it was thought that development should be construed as an assembly line. Each member of the development team is to be considered on an assembly line and each has their portion of the assembly process assigned to them. The hope was that we could use the techniques found in factories and manufacturing to potentially improve software development, leading to faster development and less error prone systems.

In studying developers in the United States, I found that most developers tended to be mavericks, cowboys as it were (and/or cowgirls), and preferred to work independently, even when on an assigned team. The United States culture tended to consider development to be a solitary task. Meanwhile, in Japan, there was a movement toward the software factory under the belief that the team-oriented approach would be perhaps a better fit for a culture oriented more so to team kinds of efforts.

The twist I discovered, and which became a notable point at the time, was that even in the case of the software factory, human behavior still comes to play and needs to be considered. Allow me to explain.

The software factory approach assumed that if there was a bug found in the code, the bug should be traced back to the developer that originated it. Furthermore, it was thought that if the developer had allowed one bug to occur, the odds were that there were more bugs too. The developer was assumed to be a flawed process. The flawed process, i.e., the programmer, likely generated more bugs since inherently they are presumed to be flawed.

I realize we can all see the logic in that approach. Sure, if a programmer makes one error, maybe it would be the case that they have allowed or made other errors too. At the time, in the United States, once a bug was discovered and fixed, everyone went along on their merry way. In the case of the software factory approach, the developer that had introduced the bug was tasked with not only resolving the bug, but also had to spend a designated amount of time to relook at their overall code, doing so to find other potential hidden bugs. The thought was that there must be more bugs to be encountered, and so find them before they actually arise.

For some of the developers in the software factory, they did not like the idea of having to use time to search for other bugs that might be of their own causing. They would rather have shrugged off the one bug as a fluke and just kept going further on new development. But, the software factory approach required that they devote a set amount of time to find those other assumed bugs. Plus, if they could not find another bug that was originated by them, this was worrisome because the assumption was that at least one or more such bugs must exist. Presumably, it would be sitting out there, waiting to strike at an inopportune time.

As a result of this circumstance, some of the developers decided to play a little trick. They would purposely seed an innocuous “bug” into their code, doing so beforehand. When an unintended bug of theirs arose, they would first dutifully fix the unintended bug, and then rather than having to go on a witch hunt to find some new additional bug, they would wait a little bit and then announce they found a second bug (the innocuous one that they had purposely seeded). This would then satisfy everyone that the assumed other bug(s) had been found, and the developer would then be allowed to proceed on their other assigned tasks.

And so goes the nature of human behavior.

One of the key tenets of debugging that should be blatantly branded on the signboard of all developers is the idea that “first, do no harm.” I mention this aspect because there is often a tendency to fix a bug and then inadvertently introduce an additional bug at the same time as enacting the fix. The programmer giveth and the programmer taketh, as they say. This is why debugging can be highly dangerous, since it is possible that you might fix a rather benign bug and find yourself accidentally having introduced a new and perhaps worse and more volatile bug.

I’ve had many situations wherein I faced making a leadership decision as to whether to have the developers fix a bug that was perhaps annoying but not incapacitating or take a chance on fixing the bug and maybe end-up with a much worse tornado on my hands. I used to take my car for repairs to a small auto shop that I had the same problem with. I’d take the car in for an oil change, and after I drove away the oil was fine but then the brakes weren’t working well. It became a game of trying to decide how serious the existing problem was and what kind of other outcome they might produce. Yes, I eventually found a different auto shop.

Another key aspect of debugging involves being a kind of scientific detective. Here’s what I mean.

When you are trying to figure out in a scientific inquiry what is taking place in a complex system such as the human body, you normally generate a hypothesis (notably, hypothesis generation is a crucial part of the scientific method). With your hypothesis in-hand, you then try to find evidence to either support the hypothesis or to disconfirm it.

For novices that do debugging, they often just start looking anywhere and everywhere to find what the bug might be. A more seasoned debugger will first craft a hypothesis. It might not be written down and might only be in their head. It might be well formulated and detailed, or it might be a hazy sketch. In any case, based on whatever they so far know about the bug, they craft a hypothesis. They then search for clues that allow them to refine the hypothesis.

This is what proficient developers do in order to try and more efficiently and effectively perform debugging. They perhaps weren’t taught to do this, and just figured out this approach on their own, or watched someone else and opted to do the same approach as them. Over time, their ability to craft on-target hypotheses gets better and better. You might liken this to a medical doctor that over time is more likely to be able to diagnose patient reported sicknesses, doing so because as a medical expert they have matured in their hypothesis generation and hypothesis testing capability.

What does this discussion of debugging have to do with AI self-driving cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. As such, we undertake debugging when needed, and we also confer with and advise other AI developers doing similar work for auto makers and other tech firms about ways to enhance their debugging skills.

The amount of software in an AI self-driving car can be enormous. Some estimates suggest that there might be on the order of 100+ million lines of code (LOC’s) involved in the “typical” system for an AI self-driving car.

That’s a lot of code.

You might be thinking that it is too large a number and that the number of human developers you would need would have to be enormous to produce that kind of volume of code.

Well, I hope you won’t think it “cheating,” but the number of LOC’s includes not just hand-crafted code for various core aspects of the AI, but it also includes lots of other code too. For example, most AI self-driving cars are using various canned libraries of code and using open source provided code, etc. That LOC gets included in the overall count.

You also need to consider that the AI self-driving car has lots of sensors, including radar, sonic, cameras, LIDAR, and the rest. There is a lot of code needed for the internal sensor drivers that enact the sensor hardware and get those sensory devices to work. The count of the AI self-driving car code includes that code too.

There is also various Machine Learning (ML) elements, including artificial neural networks (ANN). In terms of counting that code, it is somewhat ambiguous about how to count that as the equivalent of conventional LOC’s. Some try to count the underlying ANN execution on the basis of the LOC’s used for that capability. Others consider the code used in the portion of the ML tools to do the training of the ANN. It’s a bit open-ended about what is counted or not counted.

We’ll also include in the count the rest of the automated systems for the car. There are systems devoted to the AI part of the self-driving car, but you also need to keep in mind that the car is still a car, and so it has a lot of other “traditional” kinds of systems for the ECU (Engine Control Unit) and all of the other various internal systems for the brakes, steering, etc.

One aspect you need to be aware of involves the aspect that there are varying levels of AI self-driving cars.

The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the auto makers are even removing the gas pedal, brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task:

•         Sensor data collection and interpretation
•         Sensor fusion
•         Virtual world model updating
•         AI action planning
•         Car controls command issuance

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a utopian world in which there are only AI self-driving cars on the public roads. Currently there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other. Period.

Returning to the debugging topic, trying to detect a bug in an AI self-driving car can be quite tricky. In addition, tracking down the bug is likely to be arduous. Plus, fixing the bug, and doing so without upsetting something else, well, it can be doubly tricky to do.

Why would it be hard to discover bugs in an AI self-driving car and be able to ferret them out and fix them?

Mulitple Subsystems in AI Self-Driving Cars Make Debugging a Challenge

You’ve got a myriad of subsystems that interact with each other. There’s a wide variety of those subsystems in that some of them your own team might have actually written, or it might be subsystems that come along with the sensory devices or that come with the libraries used for your programming languages, etc.

Where throughout that overlapping, highly intersecting and convoluted set of subsystems upon subsystems should you look to find a bug?

Also toss into this equation that you are dealing with a real-time system. A bug that seemingly arises might actually be based on something else that took place seconds ago, or maybe minutes ago, and only after some time has elapsed does the bug “surface” to the attention of the overall system.

Suppose there is a bug in the radar sensory data collector software. On some periodic basis, the interpretation of the radar data is going to be incorrect. This is being fed into the sensor fusion. The sensor fusion is likely based on the assumption that what it is receiving from the radar software is considered “correct” and can be relied upon. Let’s pretend that the radar is reporting an image of a structure up ahead in the roadway, but it is a bug in the radar subsystem and there is no structure there.

The sensor fusion might have been written to compare the radar with the vision processing subsystem that is using the cameras of the self-driving car. Let’s assume that the cameras are working correctly, and the software indicates that there is not a structure up ahead. The sensor fusion now has to decide which is correct, the vision processing subsystem or the radar subsystem?

Let’s pretend that the sensor fusion is written to assume that if either of the vision processing subsystem or the radar subsystem suspects a structure is ahead, it is “safest” to indicate to the virtual world model updating subsystem that there is indeed a structure ahead. The virtual world model subsystem is keeping track of what objects are around the self-driving car. So, it dutifully places a virtual marker of a structure into the virtual world model at the place that the radar says there is one.

The AI action planning subsystem examines the virtual world model and is trying to figure out what actions to have the AI self-driving car undertake. The AI now assumes that a structure is up ahead in the roadway, since it so noted in the latest updated virtual world model. As a result, the AI urgently issues car controls commands to have the self-driving car swerve around the structure.

Let’s back-up for just a second or two. Suppose a human passenger in the AI self-driving car is quietly drinking their coffee and headed to work for the day. All of sudden, the AI self-driving car makes a crazy swerve, doing so for no apparent reason at all (a kind of “Crazy Ivan,” which those of you Cold War buffs might know of). The human spills their cup of coffee. Maybe the human even gets a bit of whiplash from the rather sudden movement of the self-driving car.

Why did the AI instruct the self-driving car to make a swerving action when there was no apparent reason to do so?

Imagine that you were one of the developers of the AI self-driving car system and you got asked that question. I’ve obviously already told you that the issue arose with the radar subsystem having a bug in it but put that to the side for the moment. Suppose all that you knew was that a human occupant in your brand of AI self-driving car had reported to the auto maker or tech firm that the AI self-driving did a radical swerve for no apparent reason.

Where do you start to look to find whether there is a bug or not? As I say, the more seasoned developers will try to use any clues they can about the nature of the alleged bug to craft hypotheses and then use those hypotheses to try and dig into finding the bug.

What makes this harder too is that there’s not going to be one developer that somehow magically knows the nature of the entire AI self-driving car. Instead, you’ll have many, many developers that each knows some smaller part of the AI system and the rest of the self-driving car. Discovering the bug will require working with those other team members and collectively sharing and searching. Working in teams adds further complexity. Plus, you might have some rather acrimonious arguments about where to look and what subsystem might be the source of the bug (there is often a lot of finger pointing involved!).

You also need to consider the nature of the Operating System (OS) that is running the on-board hardware and whether it might have played a role in the nature of the bug, either being the source of the bug or perhaps either magnifying the bug or maybe even hiding the bug.

There are also going to be a quite a number of IoT devices included into an AI self-driving car, and those might play a role in either causing a bug or helping a bug to become exposed and activated. There’s too the OTA (Over The Air) electronic communication aspects and possibly an interplay too with the V2V (vehicle-to-vehicle) communications.

Right now, the more advanced versions of AI self-driving cars are being kept relatively confined so that the developers can presumably (hopefully) find bugs now, before those AI self-driving cars are fully allowed into the wild. Thus, one method of finding the bugs involves restricted road trials on public roads, another involves doing testing on specialized road tracks, and another involves using extensive simulations.

There are some pundits of AI self-driving cars that seem to believe in a magical world that involves utterly flawless AI systems that are driving utterly flawless self-driving cars.

This is nonsense.

There are going to be bugs in the AI systems of self-driving cars. Face up to it. There will be recalls of parts of the AI systems and parts of the self-driving cars. Things will wear out. Maybe a parameter passed to a crucial API gets distorted and produces some cascading problem. Plus, it could be that the bug is intermittent and not readily reproducible. On and on.

Some say that the OTA will save us because via a quick electronic download you’ll have a patch put in place for any bugs. This belies the reality that first the bug has to arise and be reported. Then, the bug has to be figured out and fully discovered. Then, a fix or patch has to be devised and tested, and presumably be tested such that it does not introduce some other adverse consequence. All of that will take time. Meanwhile, the AI self-driving car will presumably be driving along, and possibly have a bug that endangers the occupants, or jeopardizes humans in other cars or perhaps imperils nearby pedestrians.

Allow me to say that I am not an alarmist and proclaiming that the sky is falling. I’m simply trying to exhort all fellow AI self-driving car developers to realize the importance of debugging.

An AI self-driving car is a life-or-death real-time system. We need to realize that it is not going to be bug free. We need to be well-prepared for debugging. Lives will depend upon it. I would also predict that if we have too many bugs at the first foray of AI self-driving cars, it will spoil the barrel and we’ll all experience a downturn in the support for and belief in AI self-driving cars.

We all must of course do whatever can be done to prevent bugs, but we must also face reality and be optimized and efficient and effective to cope with after-the-fact bugs and do an astounding job in finding and fixing them. Please don’t let debugging be perceived as a last resort and something of little standing. Debugging is vital to the safety of humans that will be using AI self-driving cars, and crucial to the entire future of AI self-driving cars.

Debugging, make sure to put it on the top of your priority list.

Copyright 2018 Dr. Lance Eliot

This content is originally posted on AI Trends.