Articles by Robert Cravotta

As a former Technical Editor covering Embedded Processing at EDN, Robert has been following and commenting on the embedded processing space since 2001 (see article index). His expertise includes software development and system design using microprocessors, microcontrollers, digital signal processors (DSPs), multiprocessor architectures, processor fabrics, coprocessors, and accelerators, plus embedded cores in FPGAs, SOCs, and ASICs. Robert's embedded engineering background includes 16 years as a Member of the Technical Staff at Boeing and Rockwell International working on path-finding avionics, power and laser control systems, autonomous vehicles, and vision sensing systems.

« Older Entries
Newer Entries »

What capability is the most important for touch sensing?

Wednesday, August 25th, 2010 by Robert Cravotta

I have been exploring user interfaces, most notably touch sensing interfaces, for a while. As part of this exploration effort, I am engaging in hands-on projects with touch development kits from each of over a dozen companies that offer some sort of touch solution. These kits range from simple button replacement to complex touch screen interfaces. I have noticed, as I work with each kit, that each company chose a different set of priorities to optimize and trade against in their touch solution. Different development kits offer various levels of maturity in how they simplify and abstract the complexity of making a touch interface act as more than just a glorified switch.

It appears there may be a different set of “must have” capabilities in a touch development kit depending on who is using it and for what type of application they are adding it. For button replacement kits, the relevant characteristics seem to focus around cost and robustness with ease of development becoming more important. A common theme among button replacement kits is supporting aggregate buttons, such as a slider bar or wheel, that can act as a single control even though it consists of multiple buttons.

From my perspective, an important capability of a button replacement solution is that it simplifies the initialization and setup of the buttons while still being able to support a wide range of operating configurations. A development kit that offers prebuilt constructs that aggregate the buttons into sliders and wheels are a plus as they greatly simplify the learning curve to use these compound button structures. Another valuable capability is driver software that allows the touch system to detect calibration drift and assists or automates in recalibration. This week’s question asks if these are sufficient leading edge capabilities, or have I missed any important capabilities for button replacement systems?

In contrast, I have noticed that many touch screen solutions focus on multi-touch capabilities. However, I am not convinced that multi-touch is the next greatest thing for touch screens. Rather, I think more high-level abstraction and robust gesture recognition is the killer capability for touch screen solutions. Part of my reasoning for this is the relative importance of “pinching” to zoom and rotate an object versus flicking and tracing to navigate and issue complex commands to the system. The challenges to correctly recognize a zooming and rotating command are somewhat constrained; whereas the challenges to correctly recognize the intended context of a flick or trace gesture are significantly more difficult because there are a wider set of conditions to which a user may apply flick or trace gesture.

As a result, I feel that an important and differentiating capability of a touch screen solution is that it offers prebuilt drivers and filters that are able to consistently identify when a touch gesture is real and intended. It should also be able to accurately differentiate between subtle nuances in a gesture so as to enable the user to communicate a richer set of intended commands to the system. Again, this week’s question seeks to determine if this is the appropriate set of leading edge capabilities, or have I missed any important capabilities for touch screen systems?

Your answers will help direct my hands-on project, and they will help with the database and interface design for the upcoming interactive embedded processing directory.

Tags:
Posted in Question of the Week, User Interfaces | No Comments »

Clarifying third generation touch sensing

Tuesday, August 24th, 2010 by Robert Cravotta

Eduardo’s response to first and second generation touch sensing provides a nice segue to clarifying third generation touch sensing capabilities. Eduardo said:

One other [classification] that I would say relates to “generations” is the single touch vs multitouch history; which I guess also relates [to] the evolution of algorithms and hardware to scan more electrodes and to interpolate the values between those electrodes. First generation: single touch and single touch matrixes; second generation: two touch, low resolution sliders; third generation: high resolution x-y sensing, multi touch detection.

While there is a “generational” shift between single- and multi-touch sensing, I am not sure the uses for multi-touch commands have reached a tipping point of adoption. My non-scientific survey of what types of multi-touch commands people know how to use yields only zoom and rotate commands. The MSDN library entry for touch provides an indication of the maturity of multi-touch interfaces; it identifies that Windows 7 supports new multi-touch gestures such as pan, zoom, rotate, two-finger tap, as well as press and tap. However, these multi-touch commands are more like manipulations where the “input corresponds directly to how the object would react naturally to the same action in the real world.”

I am excited about the possibilities of multi-touch interfaces, but I think standardizing gesture recognition for movements such as flicks, traces, and drags, which go beyond location and pen up/down data, is the relevant characteristic of third generation touch sensing. Nor is gesture recognition limited to touch interfaces. For example, there are initiatives and modules available that enable applications to recognize mouse gestures. The figures (from the referenced mouse gesture page) highlight a challenge of touch interfaces – how to provide feedback and a means for the user to visually see how to perform the touch command. The figure relies on an optional feature to display a “mouse trail” so that the reader can understand the motion of the gesture. The example figure illustrates a gesture command that combines tracing with a right-up-left gesture to signal to a browser application to open all hyperlinks that the trace crossed in separate tabs.

Open links in tabs (end with Right-Up-Left): Making any gesture ending with a straight Right-Up-Left movement opens all crossed links in tabs.

A common and useful mouse-based gesture that is not yet standard across touch sensing solutions is recognizing a hovering finger or pointer. Several capacitive touch solutions can technically sense a hovering finger, but the software to accomplish this type of detection is currently left to the device and application developer. An important component of detecting a hovering finger is detecting not just where the fingertip is but also what additional part of the display the rest of the finger or pointer is covering so that the application software can place the pop-up or context windows away from the user’s finger.

While some developers will invest the time and resources to add these types of capabilities to their designs today, gesture recognition will not reach a tipping-point until the software to recognize gestures, identify and filter out bad gestures, and abstract the gesture motion into a set of commands finds its way into IP libraries or operating system drivers.

Tags:
Posted in User Interfaces | No Comments »

Can we improve traffic safety and efficiency by eliminating traffic lights?

Wednesday, August 18th, 2010 by Robert Cravotta

I love uncovering situations where there is a mismatch between the expected results and the actual results of an experiment because it helps reinforce the importance of actually performing an experiment despite how much you think you “know” how it will turn out. System level integration of the software with the hardware is a perfect example.

It seems, with a frequency that defies pure probability, that if the integration team fails to check out an operational scenario during integration and testing, the system will behave in an unexpected manner when that scenario occurs. Take for example Apple’s recent antenna experience:

“…The electronics giant kept such a shroud of secrecy over the iPhone 4′s development that the device didn’t get the kind of real-world testing that would have exposed such problems in phones by other manufacturers, said people familiar with the matter.

The iPhones Apple sends to its carrier partners for testing are “stealth” phones that disguise a new device’s shape and some of its functions, people familiar with the matter said. Those test phones are specifically designed so the phone can’t be touched, which made it hard to catch the iPhone 4′s antenna problem. …”

The prototype units did not operate under the same conditions as they would in a production capacity, and that allowed an undesirable behavior to get through to the production version. The message here is never assume your system will work the way you expect it to – test it because the results may just surprise you.

Two recent video articles about removing traffic lights from intersections support this sentiment. In one of the video interviews a traffic specialist that suggests that turning off the traffic lights at intersections can actually improve the safety and efficiency of some intersections. The other video highlights what happened when a town turned off the traffic lights at a specific intersection. The results are anti-intuitive. This third video of an intersection is fun to watch, especially when you realize that there is no traffic control and there are all types of traffic, ranging from pedestrians, bikes, small cars, large cars, and buses all sharing the road. I am amazed watching the pedestrians and the near misses that do not appear to faze them.

I am not advocating that we turn off traffic lights, but I am advocating that we explore whether we are testing our assumptions sufficiently – whether in our own embedded designs or in other systems including traffic control. What is causing better traffic flow and safety in these test cases? Is it because the flow is low enough? Is it because the people using the intersection are using a better set of rules rather than “green means go?” Are there any parallel lessons learned that apply to integrating and testing embedded systems?

Tags: ,
Posted in Question of the Week | 12 Comments »

Software Ecosystem Sweet Spots

Monday, August 16th, 2010 by Robert Cravotta

I have been refining a sweet spot taxonomy for processors and multiprocessing designs for the past few years. This taxonomy highlights the different operating assumptions for each type of processor architecture including microcontrollers, microprocessors, digital signal processors, hardware accelerators, and processing fabrics.

I recently started to focus on developing a similar taxonomy to describe the embedded software development ecosystem that encompasses the different types of tools and work products that affect embedded software developers. I believe developing a taxonomy that identifies the sweet spot for each type of software component in the ecosystem will enable developers and tool providers to better describe the assumptions behind each type of software development tool and how to evolve them to compensate for the escalating complexity facing embedded software developers.

The growing complexity facing software developers manifests in several ways. One source of the growing complexity is the increase in the amount of code that exists in designs. The larger the amount of code within a system, the more there are opportunities for unintended resource dependencies that affect the performance and correct operation of the overall system. Using a modular design approach is a technique to manage some of this complexity, but modular design works to abstract the resource usage within a module and does not directly affect how to manage the memory and timing resources that the modules share with each other by virtue of executing on the same processor.

Another source of growing complexity is “path-finding” implementations of new functions and algorithms because many algorithms and approaches for solving a problem are first implemented as software. New algorithms undergo an evolution as the assumptions in the specification and coding implementation of the algorithms operate under wider range of operating conditions. It is not until the implementation of those algorithms matures, by being used across a wide enough range of operating conditions, that it makes sense to implement hardened and optimized versions using coprocessors, accelerators, and specialized logic blocks.

According to many conversations I have had over the years, the software in most embedded designs consumes more than half of the development budget; this ratio holds true even for “pure” hardware products such as microcontrollers and microprocessors. Consider that no company releases contemporary processor architectures anymore without also providing significant software assets that include tools, intellectual property, and bundled software. The bundled software is necessary to ease the learning curve for developers to use the new processors and to get their designs to market in a reasonable amount of time.

The software ecosystem taxonomy will map all types of software tools including assembler/compilers, debuggers, profilers, static and dynamic analyzers, as well as design exploration and optimization tools to a set of assumptions that may abstract to a small set of sweet spots. It is my hope that applying such a taxonomy will make it easier to understand how different software development tools overlap and complement each other, and how to evolve the capabilities of each tool to improve the productivity of developers. I think we are close to the point of diminishing returns of making compilation and debugging faster; rather, we need more tools that understand system level constructs and support more exploration – otherwise the continuously growing complexity of new designs will negatively impact the productivity of embedded software developers in an increasingly meaningful manner.

Please contact me if you would like to contribute any ideas on how to describe the assumptions and optimization goals behind each type of software development tool.

Tags: ,
Posted in Robust Design | No Comments »

What should compilers do better?

Wednesday, August 11th, 2010 by Robert Cravotta

Is assembly language programming a dead-end skill? So much marketing material for embedded development tools seems geared toward making it seem that developers no longer need to work at the level of assembly language for their designs. How true is this message? My observation of the embedded market suggests that the need to program in assembly language is quite alive and well.

I think I have an explanation as to why compilers have enabled the majority of software development to occur in a higher-level language, and yet, they are falling short of eliminating that “final mile” of assembly programming. One target goal for compilers is that it can generate code that is as good as hand-coded assembly, but requires less time, energy, and resources to produce than hand-coded assembly.

While faster compilation is a good thing, I think we have reached a point of diminishing returns for many types of embedded software projects. While automated project and configuration management enables developers to spend even more minutes on software, it does not actually make those extra minutes more productive. Increasing the productivity of saved minutes is essential for today’s software development tools because embedded systems are growing in complexity, and a significant portion of that additional complexity (more than 50% from my estimate) manifests in the software portion of the design.

The problem I see is that while compilers are strong at scheduling and sequencing the loading and execution of opcodes and operands, they are horrible at partitioning and allocating global resources, such as memory spaces beyond the register set and possibly tightly coupled memories. This limits the compiler’s ability to increase a developer’s productivity in precisely the area that contributes a significant portion of the additional complexity in new designs.

Processor architects perform at herculean levels to deliver memory architectures that minimize silicon cost, minimize energy dissipation, and the hide latency of data reads and writes in the memory system so that the instruction execution engines do not stall. A problem is that programming languages do not capture, beyond register files, how the memory subsystem experiences varying access times for each of the different types of limited memory resources. Currently, it is only through manual assembly language coding that the software can match the different types of data with the access times for each memory resource that the processor architect provided.

Matching data with the different memory resources is an involved skill set, and it may be a long while before compilers can perform that partitioning with any level of proficiency. However, what other things should compilers do better to improve the productivity of software developers and offset the increasing amount of complexity they must account for in their embedded designs?

Posted in Question of the Week | 2 Comments »

First and second generation touch sensing

Tuesday, August 10th, 2010 by Robert Cravotta

I recently proposed a tipping point for technology for the third generation of a technology or product, and I observed that touch technology seems to be going through a similar pattern as more touch solutions are integrating third generation capabilities. It is useful to understand the difference between the different generations of touch sensing to better understand the impact of the emerging third generation capabilities for developers.

First generation touch sensing relies on the embedded host processor to support, and the application software to understand how to configure, control, and read, the touch sensor. The application software is aware of and manages the details of the touch sensor drivers and analog to digital conversion of the sense circuits. A typical control flow to capture a touch event consists of the following steps:

1)  Activate the X driver(s)

2) Wait a predetermined amount of time for the X drivers to settle

3) Start the ADC measurement(s)

4) Wait for the ADC measurement to complete

5) Retrieve the ADC results

6) Activate the Y driver(s)

7) Wait a predetermined amount of time for the Y drivers to settle

8) Start the ADC measurement(s)

9) Wait for the ADC measurement to complete

10) Retrieve the ADC results

11) Decode and map the measured results to an X,Y coordinate

12) Apply any sensor specific filters

13) Apply calibration corrections

14) Use the result in the rest of the application code

Second generation touch sensing usually encapsulates this sequence of steps to activate the drivers, measure the sensing circuits, and applying the filters and calibration corrections into a touch event. Second generation solutions may also offload the sensor calibration function, although the application software may need to know when and how to initiate the calibrate function. A third generation solution may provide automatic calibration so that the application software does not need to know when or how to recalibrate the sensor because of changes in the operating environment (more in a later article).

A challenge for providing touch sensing solutions is striking a balance between meeting the needs of developers that want low- and high-levels of abstraction. For low-level design considerations, the developer needs an intimate knowledge of the hardware resources and access to the raw data to be able to build and use custom software functions that extend the capabilities of the touch sensor or even improve its signal to noise ratio. For developers using the touch sensor as a high-level device, the developer may be able to work through an API (application programming interface) to configure, as well as turn on and off, the touch sensor.

The second and third generation touch API typically includes high-level commands to enable and disable, calibrate, and read and write the configuration registers for the touch sensor as well as low-level commands to access the calibration information for the touch sensor. The details to configure the sensor and the driver for event reporting differ from device to device. Another important capability that second and third generation solutions may include is the ability to support various touch sensors and display shapes without requiring the developer to rework the application code. This is important because for many contemporary touch and display solutions, the developer must be separately aware of the display, touch sensing, and controller components because there are not many options for fully integrated touch and display systems. In short, we are still in the Wild West era of embedded touch sensing and display solutions.

Tags:
Posted in User Interfaces | 1 Comment »

Exploring multiprocessing extremes

Friday, August 6th, 2010 by Robert Cravotta

Extreme multiprocessing is an interesting topic because it can mean vastly different things to different people depending on what types of problems they are trying to solve.

At one end of the spectrum, there are multiprocessing designs that maximize the amount of processing work that the system performs within a unit of time while staying within an energy budget to perform that work. These types of designs, often high-compute, parallel processing, work station, or server systems, are able to deliver a higher processing throughput rate at lower power dissipation than if they used a hypothetical single core processor that ran at significantly faster clock rates. The multiple processor cores in these types of systems might operate in the GHz range.

While multiprocessing architectures are an approach to increase processing throughput while maintaining an energy budget, for the past few years, I have been unofficially hearing from high performance processor suppliers that some of their customers are asking for faster processors despite the higher energy budget. These designers understand how to build their software systems using a single instruction-stream model. The contemporary programming models and tools are falling short for enabling software developers to scale their code across multiple instruction streams. The increased software complexity and risks outweigh the complexity of managing the higher thermal and energy thresholds.

At the other end of the spectrum, there are multiprocessing designs that rely on multiple processor cores to partition the workload among independent resources to minimize resource dependencies and design complexity. These types of designs are the meat and potatoes of the embedded multiprocessing world. The multiple processor cores in these types of systems might operate in the 10’s to 100’s MHz range.

Let me clarify how I am using multiprocessing to avoid confusion. Multiprocessing designs use more than a single processing core, working together (even indirectly) to accomplish some system level function. I do not assume what type of cores the design uses, nor whether they are identical, similar, or dissimilar. I also do not assume that the cores are co-located in the same silicon die, chip package, board, or even chassis because a primary difference for each of these implementation options are energy dissipation and latency of the data flow. The design concepts are similar between each scale as long as the implementation meets the energy and latency thresholds. To further clarify, multicore is a subset of multiprocessing where the processing cores are co-located in the same silicon die.

I will to try to identify the size, speed, energy, and processing width limits for multiprocessing systems for each of these types of designers. In the next extreme processing article, I will explore how scaling multiprocessing upwards might change basic assumptions about processor architectures.

Posted in Extreme Processing, Multiprocessing | No Comments »

How much of an engineer’s job is writing?

Wednesday, August 4th, 2010 by Robert Cravotta

My experience suggests that engineers do a lot more writing than the rest of the world realizes. We had a saying where I worked that the engineering effort was not complete until the documentation outweighed the final product. At one time, I was writing (more like typing) so much material for system specifications, proposals, and trade-off studies, that I actually developed an unpleasant case of acute tendinitis. To be fair, the tendinitis was not so much a function of the amount of writing I did, but more a function of the layout of my workspace. Eighteen months of physical therapy and reading everything I could find about ergonomics taught me how to continue writing without hurting myself.

As a boy, I recall hearing how my father was in the top of his class for math and science, but that he was also in the bottom of his class for writing and language. His teachers passed him through the language courses because they figured he would get by fine on his technical skills and would not need to know how to write well. How wrong they were. I remember sensing his frustration that the skill he was most weakest in, writing, was the one thing he had to spend significant portions of his time doing to produce specifications, interface documents, and data analysis reports.

Jon Titus has been writing about how engineers can write better, and I suspect he will be posting about this topic a few more times. Jon offers a nice list of types of writing that engineers might need to engage in, such as status reports, technical articles, application notes, marketing material, manuals, instructions, proposals and justifications, as well as blogs or columns. I would like to expand on that list and include logs or lab journals, specifications, as well as design justification and review documents.

Ambiguity is unacceptable in most of these documents, and as a result, engineering writing can exhibit structures that language majors find humorous or frustrating. I know that when I transitioned to article writing, it took me some time to adopt an active voice rather than a passive voice. Even then, there are times when the passive voice just makes more sense. I found an article that discusses the passive engineer in a useful fashion. What I appreciate about the essay is that it avoids being dogmatic about never using the passive voice, such as when you want to emphasize results or the actor is unimportant to the concept you are communicating.

I would like to uncover whether my experience with writing as an engineer is niche to the aerospace market or if engineers in many or all other fields also engage in a significant amount of writing. What kind of writing do you do as an engineer, and what percentage of your time do you spend doing it? Do you consider yourself a fast writer or a slow writer?

Posted in Question of the Week | 7 Comments »

How does a “zerg-to-market” strategy affect your design cycle?

Wednesday, July 28th, 2010 by Robert Cravotta

The term “zerg” serves several purposes in this week’s question. It is: a nod toward the recent release of Blizzard’s Starcraft 2 real-time strategy game; a reference to a rush strategy to win a Starcraft match that has found its way into general usage among online gamers; and a way to blend the terms time-to-market and first-to-market into a single phrase in a serious question for embedded developers. Despite the connotation of a rush strategy, there is more than a ten year gap between the launch of the prior and current release of Starcraft.

Time-to-market is the length of time for the design and development process of transforming a concept from an idea to a deliverable product in the market. It applies to new concepts/products as well as incremental improvements to existing products. Being able to get an idea to market quickly is especially important when products undergo a rapid obsolescence cycle.

First-to-market is a special-case of time-to-market, and it is usually the implied message when marketers tout how using their embedded components will shrink your time-to-market. Adopting a first-to-market strategy relies on the assumption that time-to-market matters most for first-of-a-kind products because it provides a first mover advantage.

A time-to-market emphasis assumes that you can shrink your design cycle so that it is shorter than your competitors that chose a different way to complete their design effort. There are several strategies to shrink the development cycle. The easiest, and most risky, strategy is to skip “low-value” steps in the development process, but this can cause the development team to miss flaws earlier in the design cycle and cause them to become expensive, in terms of both resources and schedule, to fix.

A more robust, but also more costly approach is to have more developers working in parallel so that you do not skip any steps in your design process. This is an approach taken by many teams aiming for a first-to-market product release; however, it has its own unique set of risks. By virtue of trying to be the first to build something, the components you have access to have not yet been characterized to your project’s requirements. In essence, you become an early adopter working with preproduction components. Likewise, your team usually lacks access to mature, domain-targeted tools, documentation, and training because you are completing your first-to-market design while these are being built and certified. This means your team will need to build some of the components, such as drivers, from scratch because the production version of these may not exist until you are ready to ship out your product.

When marketing material touts that it shrinks your time-to-market, it usually is relying on a third approach – using pre-engineered and pre-certified components and tools. In essence, you can safely skip steps because you actually outsourced those steps to someone else. On the other hand, you are not going to be able to find production ready components for novel first-to-market ideas because building these pre-engineered solutions is costly, and the companies that provide them choose to build them because they believe they can recover the development cost across many customers and designs – your competitors.

How important is a first-to-market strategy to your projects? What trade-offs do you consider when trying to shrink your time-to-market? What types of risks are you willing to accept to get the product out the door sooner, and what steps do you take to mitigate the consequences of those accepted risks?

Email me with any questions you would like to suggest for future posts.

Tags:
Posted in Question of the Week | No Comments »

Robotics and autonomous systems

Tuesday, July 27th, 2010 by Robert Cravotta

Robotics is embedded design made visible. It is one of the few ways that users and designers can see and understand the rate of change in embedded technology. The various sensing and actuating subsystems are not the end-system, nor does the user much care how they are implemented, but both user and designer can recognize how each of the subsystems contribute, at a high level of abstraction, to the behavior of the end-system.

The state of the art for robotic systems keeps improving. Robots are not limited to military applications. Robots are entrenched in the consumer market in the form of toys and cleaning robots. Aquaproducts and iRobot are two companies that sell robots into the consumer market that clean pools, carpets, roof gutters, and hard floors.

A recent video from the GRASP (General Robotics Automation Sensing and Perception) Laboratory at the University of Pennsylvania demonstrates aggressive maneuvers for an autonomous, flying quadrotor (or quadrocopter). The quadrotor video demonstrates that it can autonomously sense and adjust for obstacles, as well as execute and recover from performing complex flight maneuvers.

An even more exciting capability is groups of autonomous robots that are able to work together toward a single goal. A recent video demonstrates multiple quadrotors flying together to carry a rigid structure. At this point, the demonstration only involves rigid structures, and I have not yet been able to confirm whether the cooperative control mechanism can work with carrying non-rigid structures.

Building robots that can autonomously work together in groups is a long-term goal. There are robot soccer competitions that groups such as FIRA and RoboCup sponsor throughout the year to promote interest and research into cooperative robots. However, building packs of cooperating robots is not limited to games. Six development teams were recently announced as finalists for the inaugural MAGIC (Multi Autonomous Ground-Robotic International Challenge) event.

Robotics relies on the integration of software, electronics, and mechanical systems. Robotics systems need to be able to coordinate sensing the external world with their own internal self-state to navigate through the real world and accomplish a task or goal. As robotic systems continue to mature, they are incorporating more context recognition of their surroundings, self-state, and goals, so that they can perform effective planning. Lastly, multiprocessing concepts are put to practical tests, not only within a single robot, but these concepts are tested within packs of robots. Understanding what does and does not work with robots may strongly influence the next round of innovations within embedded designs as they adopt and implement more multiprocessing concepts.

Posted in Multiprocessing, Robotics | 1 Comment »

« Older Entries
Newer Entries »