In the beginning, there was chaos and uncertainty. Systems were running raw byte code, and debug interfaces were scarce or even nonexistent.

Arguably, one of the first embedded systems was the Apollo Guidance Computer (AGC), which was released in 1966 and was used for multiple Apollo missions [1].

It featured a tightly integrated system with very capable hardware specifications for its time.

• 1 MHz CPU clock speed
• ~69 kB of ROM
• ~3,8 kB of RAM
• 32 kg total system weight

By today's metrics, the clock speed may appear very low, but surprisingly, the typical memory sizes of modern microcontrollers are sometimes still of a similar magnitude as the APC's memory was.

But we all know about the famous success of the Apollo missions, so while these resources appear microscopic these days, they were plentiful for their intended purpose.

Figure 1.1: Front view of the AGC

However, debugging the software that was running on the system was tedious due to the lack of proper debug features that were integrated into the chip's architecture. Typically, debugging was done indirectly via light indicators or simple print statements to a terminal.

Luckily, analyzing the assembly source code manually was relatively straightforward as the complete available memory was just 2,048 words. Even though the application featured a rudimentary RTOS and multiple algorithms for different calculation tasks, it could be analyzed quickly.

Over time, in the 1970s, system-specific software debuggers (monitor debuggers) started appearing that allowed inspecting memory locations, using breakpoints, and controlling the execution flow of the program.

However, these debuggers resided in the system's memory, and on resource-constrained systems, it was not ideal.

Next, in the 1980s and 1990s, we saw the birth of in-circuit emulators (ICEs). These huge boxes allowed interacting with a target system directly, making debugging tasks in real-time possible [2].

Debug interfaces such as JTAG additionally standardized approaches to debug environments for embedded systems, bringing costs down [3].

Initially, there was just machine code for computers. To make them more readable for humans, assembly language was introduced in the 1960s. It was predominantly used in the embedded domain in the beginning, due to the very resource-constrained systems and precise control over hardware.

Then C was invented in 1972. It allowed structural programming on embedded systems with a balance between high-level software abstraction and low-level hardware access. Until this day, C is still the most used programming language for embedded systems [4].

During the 1980s, the first object-oriented programming language, ADA, was introduced with the main goal of code safety and maintainability. That is why it was mostly utilized in avionics [5].

In 1985, C++ was released, another object-oriented approach, which initially did not have embedded real-time systems specifically in mind but gained much popularity over the years due to its flexibility and great language support [6].

In the 1990s, Java was introduced to the software world, promising excellent cross-compatibility over all available platforms. It took over the world of general computing quickly, and for embedded systems, it paved the way for modern smartphones and IoT devices [7].

At the same time, scripting languages such as Python started to gain more popularity. These days, Python is one of the most popular programming languages and can even be cross-compiled to embedded systems as well [8].

With its very simple and easy to learn syntax, Python has taken over many software engineering disciplines in a flash.

In the 2010s, Rust was introduced, and while it took some time to be established in the embedded domain, its innovative ownership model for memory management promised memory-safe applications, eliminating some of the most common bug types in the software development domain [9].

Most debugging was initially done via terminal prints and command-line interfaces.

Build systems were usually separate systems, making testing code iterations cumbersome.

To improve this situation in the 1980s, so-called Integrated Development Environments (IDEs) were introduced, making code writing, building, and debugging all happen inside the same, usually GUI-based tool.

This not only streamlined development setups but also gave tremendous advantages in the time-to-market race.

Figure 1.2: Turbo C IDE

Here, we can see one of the first C IDEs, called Turbo C, which was available for DOS systems. It featured a text editor, project management features, a debugger, a build environment, and much more.

However, specifically at the beginning, using IDEs was frowned upon as experienced users argued that IDEs would limit their creative freedom [10].

But as history shows us, IDEs came to stay and are available these days on many systems in many flavors, for all kinds of programming languages.

So far, we have learned about the past of embedded systems debugging. We came from rather rudimentary debug features and environments to rather sophisticated setups with plentiful features.

But what does a modern landscape in the 2020s look like, and what are the current challenges and trends?

One great development is that nearly all modern microcontrollers that are released these days have debug ports integrated, even with some advanced debug features that go along with it. This led to a great availability of all kinds of debug probes that offer solutions for a broad range of budgets and chip architectures.

There are even on-board variants available, so whole debug probes can be tightly integrated into your own board designs.

The broad availability of debug interfaces and probes with advanced debug features makes bug hunting as efficient as it has ever been before.

Features such as hardware and software tracing, live memory manipulation, various breakpoint types, and many more will be covered in this book in more detail, so you can make the most of these modern features that are available even on the cheapest microcontrollers these days.

However, we also see that target systems are becoming increasingly complex. Multi-core devices are becoming the norm, complex memory management schemes become necessary, and advanced safety features increase the learning curve for newcomers to the embedded domain. So, having advanced debug setups becomes a necessity at some point.

To counter the trend of more complex devices, we see an increasing trend of third-party hardware abstraction layers and code generators being used to reduce the initial complexity of a new system. The trade-off is, however, that third-party code blobs are added to your software, where the chain of trust may become watered down, which, in the worst case, may add unwanted security risks to your application.

We also see the rise of embedded frameworks such as Zephyr that take the abstractions even further. Whether eliminating hardware dependencies will be the cure for future complex embedded systems, only time will tell, but it is definitely an approach that you should use consciously, as program safety is also becoming a core factor of modern embedded systems.

Another popular trend is the rapid rise to fame of Visual Studio Code from Microsoft. Starting off as a feature-rich free text editor with a plugin interface and built-in community features, it quickly transformed into a one-size-fits-all platform for modern software development, even in the embedded domain, challenging market-leading IDEs.

A more classic approach to counter complexity that is gaining much ground in the embedded domain is automated unit testing on embedded systems. Typically, such tests are run in separate simulation environments, oftentimes in the form of digital twins. However, many have now painfully learned that nothing beats testing on the actual hardware. Due to short prototyping cycles, quick availability of PCBs is becoming the norm, thus allowing testing on hardware early in a development cycle.

Last but not least, there's artificial intelligence (AI) in the form of large language models (LLMs). It is here now, so we have to talk about the elephant in the room. It can already write mostly correct code. It can somewhat debug broken code, but what it excels at is being a valuable assistant to an experienced developer. Even in the embedded domain where the training data for AI models is quite scarce and not always easily machine-readable, many of the suggestions and help you can receive from AI can still boost your productivity, and most importantly, it saves precious time when digging through thousands of pages of documentation, because what LLMs do best is pattern recognition of language context.

But what current LLMs also do very well is hallucinate answers, since current AI companies tune their models to give the user answers that would make them happy to keep engagement with the platform high, instead of always providing accurate information.

That is, of course, not ideal if you are trying to get a correct answer in a mostly deterministic profession such as software engineering. But LLM companies promise that this will all be fixed soon, and we just have to wait a bit longer.

History seems to be on their side, as so far, anything that humans could think or dream of in the technical domain was at some point later realized, so the chances are good that this will also happen with AI. But whether it will happen during my lifetime, I do not know. I am still waiting for cold fusion and teleportation.

But let's focus on reality and on what we humans do best – teaching each other new things, which, hopefully, will be achieved by the end of this book.

In this chapter, we learned about the history of embedded systems debugging, from the early beginnings, the first actual embedded systems, to modern interfaces, programming languages, and debugging tools.

Next, let's try to find out exactly what debugging is, why it is important, which peculiarities appear in the embedded domain, and what kind of debug interfaces exist.

1. The Virtual AGC Project Spaceborne Computer Systems: https://www.ibiblio.org/apollo/#gsc.tab=0
2. Debugging stories: from printf, just Flash and beyond, Erol Simsek, 2022: https://www.embedded.com/debugging-stories-from-printf-just-flash-and-beyond
3. IEEE Standard for Test Access Port and Boundary-Scan Architecture, IEEE Std. 1149.1-2013, 2013. IEEE.
4. Ritchie, D. M., & Kernighan, B. W. (1988). The C programming language (2nd ed.). Prentice Hall.
5. Burns, A., & Wellings, A. J. (2007). Concurrent and Real-Time Programming in Ada. Cambridge University Press.
6. Stroustrup, B. (2013). The C++ programming language (4th ed.). Addison-Wesley.
7. Gosling, J., Holmes, B., & Steel, G. (1996). The Java Language Specification. Addison-Wesley.
8. Lutz, M. (1996). Programming Python (1st ed.). O'Reilly Media.
9. Klabnik, S., & Nichols, K. (2018). The Rust Programming Language. No Starch Press.
10. Raymond, E. (2003). The Art of Unix Programming

Get this book's PDF version and more

Scan the QR code (or go to packtpub.com/unlock). Search for this book by name, confirm the edition, and then follow the steps on the page.

Note: Keep your invoice handy. Purchases made directly from Packt don't require an invoice.