Tips for embedded real-time design

At a recently held conference on embedded software, a speaker fielded the following question from the audience: "When we build our systems, we limit the execution load of the processor to 80% so there's enough leeway for meeting timing requirements. How much execution load leeway do you recommend?"

The speaker's answer: The system should be built for an execution load of 100%. Limiting the load to 80% wastes 20% of the resources.

The speaker was David Stewart of Embedded Research Solutions LLC, a software consulting and contracting firm in Columbia, Md. "That question highlights a common practice — writing software with tolerances of 20% or more," he laments. "Imagine building the cylinders for a car engine with 20% tolerance in each dimension!"

Stewart knows a thing or two about developing code for small computers. Before joining ERS he headed up the real-time laboratory at the University of Maryland and designed robotics software at Carnegie Mellon University's well-known Robotics Institute. This work has given him expertise in fielding small systems. It also has given him insights into devising reliable real-time controllers built around small computers.

In this regard, precision is one of his pet peeves. "Industry doesn't design software precisely, so developers need high tolerances to guarantee the system meets timing specifications," he says. "But more precise design methods let you use 100% of the resources. Suppose that instead of executing with 20% tolerance, the exact same software is designed to use 100% of a processor that is 20% less powerful. That's analogous to the difference between getting by with a 1.3-GHz Pentium IV instead of a new 1.7-GHz chip, which costs $150 more. For an embedded processor, there might only be a $1 difference in the alternatives. But if you are talking about making 1 million units, that's a $1 million savings for the manufacturer."

Moreover, it takes no longer to develop precise software than to do things the traditional way, claims Stewart. In fact, he says, such efforts can be faster because they need dramatically shorter testing and debugging periods.

However, for developers accustomed to working with large, full-blown computer systems, there has to be a change in mindset when it comes to fielding small embedded machines. Take as an example the area of component-based software. Software components, self-contained sections of code, are today viewed as a way of ensuring that programs get developed in understandable and manageable hunks. There are numerous tools available to help coders generate software components for relatively large systems.

Trouble is, a lot of these tools go out the window for systems smaller than 32 bits. "Techniques such as code generation and object request brokers, commonly used on bigger systems, just aren't suitable for 8 and 16-bit processors," explains Stewart. "Code generation tools might produce code occupying 120 kbytes of memory when small embedded systems more typically carry on the order of 16 kbytes."

The approach Stewart preaches for work on real-time embedded platforms is to first set up what is called an application-independent framework. A framework is a piece of software that, among other things, may schedule software tasks to run, facilitate the transfer of data from one software task to another, and manage housekeeping associated with starting and stopping software routines that perform specific tasks.

Once they've defined a framework, developers then devise software modules that plug into the framework by means of specific interface specifications. The modules perform tasks specific to the application at hand, such as reading an encoder generating a motor speed command, or executing a PID control algorithm. Defining programs this way lets software developers write program modules flexible enough to be used in more than one project.

The beauty of the approach is that it will work on both large and small embedded systems. In 32-bit computer systems that manage a lot of resources, the framework may take the form of middleware that sits on top of a conventional real-time operating system. But frameworks are particularly helpful in small eight and 16-bit embedded systems, with just a few kilobytes of memory. Here the framework may essentially function as a realtime executive.

Small computers
No question use of software components and other timesaving techniques are important. Estimates are that software development can account for as much as half the total development time on embedded system projects. Moreover, software techniques can also dramatically impact the progression of hardware development and system integration. Together, these phases often account for as much as 80% of an embedded development effort.

One of the techniques that can help shave time off development is to tweak code for better handling of real-time tasks. This is true even for large, 32-bit systems that involve the use of numerous resources and which are almost exclusively run by commercial real-time operating systems.

Any processor has peculiarities in handling specific operations that developers should account for, says Stewart. Typically, some actions take place either more slowly or quicker than might be expected.

Floating-point instructions in systems employing the Motorola 68030 with a 68882 floating-point coprocessor serve as an example. "Floating-point addition takes the same amount of time as integer addition, which may be counterintuitive, because the chip contains floating-point hardware," says Stewart. Mixed numerical formats can also be a source of surprises. "Adding an integer to a floating-point number takes three times longer than just adding two floatingpoint numbers," he points out. "Most people think this operation should happen in less time than just adding two floating-point numbers. But the integer must first be converted to floating-point format before the addition can take place. There's another conversion if the result must be an integer. I have seen systems where just a few operations took place in floating-point format, and this behavior resulted in something slower than if the whole thing had been done in floating point."

Allowances for such idiosyncrasies should take place in the initial phases of design. So too should coarse grain optimizations: Noting specific features of the processor hardware and deciding how to take advantage of them in the ensuing software design. Of course, sections of code optimized for a specific processor won't work as well on another machine.

Component-based software techniques are particularly helpful in such cases. The idea, says Stewart, is to create both generic and optimized modules that provide a specific function, then use the one that is most appropriate for the hardware at hand.

Considerations such as the need for optimization make it important that software developers be included in the initial phases of any design. Otherwise, computer resources can be easily overdesigned. "In one case, adding a little additional complexity to the hardware reduced the processor execution time by 25%," relates Stewart.

He illustrates the typical trade-off discussion this way: Suppose the processor costs $10. Say one software operation uses 25% of it. That means the hardware cost of that operation is $2.50. Now consider using $1.50 worth of intelligent I/O instead. There's a net savings of $1 in hardware resources. There is also 25% more processor time available for something else. Alternatively, there is an opportunity to run the processor 25% slower to save power, or to use a slower, lessexpensive processor to cut costs.

Typical hardware/software trade-offs are in peripheral circuits as, for example, analog/digital converters. Simple a/d circuitry that requires software for multiplexing may initially appear to be less costly than more intelligent devices that may only require software that handles a simple data exchange. But the question can only be answered by looking at where the flexibility lies in the system, and deciding what resources are the most important.

To interrupt or not
Stewart's own work has spanned 8-bit 6805 to 32-bit 68k-based Motorola microcontrollers, as well as Z180 devices from Zilog and digital signal processors from Texas Instruments. Nevertheless, the techniques he advocates are useful on small systems with limited memory from any vendor.

One example of systems in this category are those from Z World Inc., Davis, Calif. The firm has devised a Z80-like microprocessor called Rabbit 2000. It is an 8-bit device with both internal and external 8-bit data buses. It uses an extended Z80-style instruction set which, according to Z World, provides performance as good as many 16-bit processors.

The device is typically deployed on a small board along with a megabyte or less of static RAM or Flash memory. Programming is via a variation of C language called Dynamic C. It is said to have enhancements and variations designed to facilitate realtime programming, with constructs for cooperative and preemptive multitasking, protecting writes to variables during power failures, and writing interrupt routines.

The Rabbit processor's handling of interrupts, through four levels of priority, points up a technique often used in systems handling numerous time-sensitive tasks. Developers of real-time software often take an eventdriven (or interruptdriven) organizational approach. The idea in such cases is to write the program essentially as a collection of routines that each executes on the receipt of their respective interrupt signals. These signals are typically generated by external devices such as position encoders, Halleffect speed sensors, and so forth. They synchronize their execution through use of mechanisms such as semaphores, registers containing values indicating the status of the various interrupt routines running on a system.

Interestingly, however, Embedded Research's Stewart points out that interrupts often do more harm than good in real-time systems. Most interrupt-driven systems, in fact, can be converted to one following a cyclic regime, where timers trigger routines that check I/O points systematically to see if they need service.

A point to note about interrupt-based schemes is that they generally signal the need for interrupt service, the completion of service, by putting a value in a register. But procedures in a predefined list can often check the appropriate registers periodically to accomplish the same thing, points out Stewart.

Moreover, the presence of interrupts and other nonpreemtable high-priority events can be a cause of priority inversion — the kind of problem that caused the Mars Pathfinder to fail. "Interrupt-driven systems are difficult if not impossible to verify analytically," he points out. "The more interrupts you have, the more difficult it becomes to guarantee that the processor will meet its deadlines."

To prove analytically that an interruptdriven system will work, developers must use real-time systems theory to estimate how often interrupt routines will run and how long each will be active. Measurements of execution time must then verify that the analysis is correct.

It is a relatively straightforward process to devise scheduling algorithms for small systems. Stewart estimates that a rudimentary real-time cyclic executive can consist of about 30 lines of code, yet still support a component-based software approach. The do-it-yourself approach to writing a multirate real-time executive works best for systems that lack graphic display monitors, hard disks, and other peripherals that transfer huge amounts of data. For systems that do have these devices, he recommends use of a commercial real-time operating system that provides all necessary drivers. Developers should then create a middleware layer to provide a component-based software interface, instead of relying on the processoriented interfaces provided by nearly every real-time operating system.

Some off-the-shelf embedded systems also provide special interfaces targeting such uses. An example is the StrongARM platform from ARM Ltd. in the U.K. The 206-MHz 32-bit reduced instruction set (Risc) CPU core is used on a single-board computer recently developed by Applied Data Systems Inc., Columbia, Md. The processor delivers 235 Mips and dissipates 400 mW, letting it support two LCDs simultaneously. Compatible displays range from quarter VGA (320 240 pixels) up to XGA (1,024 768 pixels). Applications include point-of-sale terminals and gasoline pumps.

A number of the ARM-based systems that Applied Data provides go into medical applications. Some of these are classified as Class 1, those that are invasive and where a malfunction can be potentially life threatening. Developers of such equipment frequently go with a commercial real-time operating system called VXworks because of its fast interrupt response. Along with the operating system, Applied Data supplies low-level software such as drivers. Recently, it made available networking routines that permit one embedded system to access files on another over a network.