Sam Siewert

Dept. of Computer Science and Colorado Space Grant College

Research Overview

"Semi-Autonomous Systems Support Software for Agent Tasks Executing in Multiple Time Frames"

I am working on operating system kernel software (software directly interfacing to hardware) [1 & 2], which supports flexible, efficient and reliable semi-autonomous operation of devices in complex systems. My focus is semi-autonomous operation of devices in uncontrolled environments to achieve goals with minimal human monitoring and control. The goal is to elevate the level of control to support task-level operator interfaces [3 & 13]. This frees the operator to focus on mission objectives, requiring only occasional system supervision at the level of mission objective tasks [4]. In general, autonomous actions are most efficiently implemented with "on-board" computers, software, sensors and actuators providing localized control. For example, an earth-orbiting spacecraft might autonomously point an imaging instrument at the earth, take images based on coverage goals, and monitor its own health and status, allowing the operator to focus on earth imaging rather than device management.

Systems such as spacecraft, submarines, unoccupied air vehicles, power grid management systems [5], and related systems with complex, remotely distributed devices and computers, have traditionally included significant "continuous human monitoring and control" with a high cost paid for remote device communications and operations. However, only well understood (deterministic), time critical functions are normally automated on computers local to device interfaces, for "on-board" autonomy. For example, spacecraft attitude control is frequently automated on-board with hardware and hard-coded software control. However, more complex, less deterministic functions, such as fault detection [14] and handling, or recognition of unexpected opportunities to increase performance, are traditionally not automated.

Figure 1: Semi-Autonomous Systems Concept

For my dissertation research, I am building prototype semi-autonomous operating systems support software which provides flexible allocation of management functions so that operators can increase the autonomy of the devices they are managing as they gain experience. My system is a fusion of the extremes of continuous monitoring and control and an agent system providing fully autonomous real-time functionality [6]. The agent system can accommodate an on-board planning agent [7] and includes event driven reactive agent [8] capabilities to provide autonomous monitoring and control of complex functions such as fault detection and handling. This system framework makes no clear distinction between "planning" and "reactivity" other than the fact that any kernel-level functionality must be implemented with a state machine. Distinguishing the two and allocating them is left up to the agent application developer, especially since this is an open issue in Artificial Intelligence and automation theory [15]. The approach taken here is to provide the systems framework for supporting a range of agent applications from the extreme of Brook's behaviorally decomposed subsumption approach to more traditional functional decompositions common in robotics. Functionality can be allocated between time frames based on time criticality and determinism. This system is depicted in figure 1. This concept has been presented to the space operations automation research community [9] and robotics research community [10]. My specific focus is to provide an efficient and reliable implementation of the reactive agent functionality which specifically operates in a "soft real-time" frame and a level of determinism between hardware-based real-time control and "near-real-time" control [11] (figure 2).

Figure 2: Agent Functions "Closing the Loop" in Multiple Time Frames

In order to test and experiment with my software concepts, I have developed a simple "lighter-than-air" UAV (unoccupied air vehicle), shown in picture 1 (early prototype) and picture 2 (current flying test-bed undercarriage), which has been flown indoors, and serves as a test-bed for my work on semi-autonomous systems software. The current on-board computer uses a camera for optical navigation, and will be enhanced to include a digital compass [12] for simplified, low-cost autonomous navigation. The digital compass will provide a simple way to obtain UAV heading knowledge for on-board navigation which can also be derived from the optical navigation.

Picture 1: Early Prototype

Picture 2: View of Current Working UAV

Sensors to monitor the system "health and status" include motor voltage and current draw sensors, temperature sensors, and air pressure (Figure 3). The UAV communicates with a ground laptop computer using a pair of PCMCIA radio-frequency network adapters. The health and status sensors and the hobby servos are controlled by a Motorola 68HC11 which is commanded through a serial interface to the on-board Intel 486 computer (may be upgraded to Pentium). The planned digital compass interface will be through a second 68HC11 which may also be used to interface to an "ultrasonic ranger" for simple altitude estimation. A bus interface between the two 68HC11s provides commanding of either microcontroller through a common serial interface. The Quickcam used for optical navigation is interfaced to the on-board computer through a parallel port interface.

Figure 3: UAV Block Diagram

The UAV testbed has been flown once, and is currently being upgraded for additional flights to experiment with systems support for reliable, flexible, and efficient multi-agent applications. The major thrust of this research is the systems software support for agent applications, with emphasis on agent systems which can be semi-autonomously operated, and include autonomous functionality implemented in a range of time frames including: on-board hardware (microseconds), on-board kernel software (milliseconds), on-board application software (seconds to hours), and ground software and operator interaction (seconds to hours). This allocation is characteristic of space systems which normally include latency in uplink and downlink on the order of seconds (assuming geosynchronous tracking and data relay satellites in are used for global coverage), or at least on the order of a millisecond for direct groundstation passes, which however are usually intermittant. In general, space systems, and similar systems previously mentioned, will always include bandwidth and latency limitations due to power constraints, distances, signal attenuation, and line of sight occlusion, which all make constant high-bandwidth low-latency radio-frequency communication challenging and expensive. Likewise, should high-bandwidth, low-latency communication be available, it is still not necessarily desirable to teleoperate systems due to the amount of human attention required (and therefore operations cost), as well as the human factor of understanding large amounts of data, and providing continuous control ("information overload"). The final argument for this type of semi-autonomous system is scalability, since in general, less monitoring and control of a given subsystem means that a single operator can manage a larger number of subsystems.

References

[1] Wells, D., "A Trusted, Scaleable, Real-Time Operating System Environment", Open Software Foundation Research Institute, Cambridge, Massachusetts, 1994.

[2] Vahalia, U., Unix Internals: The New Frontiers, Prentice-Hall, Inc., Upper Saddle River, N.J., 1996.

[3] Fleischer, S., Rock, S., Lee, M., "Underwater Vehicle Control from a Virtual Environment Interface", 1995 Symposium on Interactive 3-D Graphics, Association for Computing Machinery Special Interest Group on Computer Graphics, Monterey, California, April, 1995.

[4] S. Siewert and L. McClure, "A System Architecture to Advance Small Satellite Mission Operations Autonomy", 9th Annual AIAA/Utah State University Conference on Small Satellites, Logan, Utah, September 1995.

[5] Wittig, Thies, ed., ARCHON: An Architecture for Multi-Agent Systems, Ellis Horwood Limited, Chichester, West Sussex, England, 1992.

[6] Maes, P., "Situated Agents Can Have Goals," Robotics and Autonomous Systems, Vol. 6, 1990.

[7] Hayes-Roth, B., "Architectural Foundations for Real-Time Performance in Intelligent Agents", Real-Time Systems, May 1990.

[8] Russel, S. J., and Norvig, P., Artificial Intelligence: A Modern Approach, Prentice Hall, Englewood Cliffs, New Jersey, 1995.

[9] Siewert, S., and Hansen, E., "A Distributed Operations Automation Testbed to Evaluate System Support for Autonomy and Operator Interaction Protocols", The 4th International Symposium on Space Mission Operations and Ground Data Systems, Forum der Technik, Munich, Germany, September, 1996.

[10] Siewert, S. and Nutt G., "A Space Systems Testbed for Situated Agent Observability and Interaction", The ASCE 2nd Conference, Exposition and Demonstration on Robotics for Challenging Environments, Albuquerque, N.M., 1996.

[11] Stankovic, J., "Misconceptions About Real-Time Computing: A Serious Problem for Next-Generation Systems", IEEE Computer, Vol. 21, No. 10, October, 1988.

[12] Unknown author, "C-100 Compass Engine Technical Manual", KVH Industries, Inc., Middletown, Rhode Island, 1992.

[13] Simmons, R., "Towards Reliable Autonomous Agents", AAAI Spring Symposium on Software Architectures, Stanford, CA, March 1995.

[14] Doyle, R., "Determining the Loci of Anomalies Using Minimal Causal Models," International Joint Conference on Artificial Intelligence, Montreal, Canada, August, 1995.

[15] Brooks, R., "Intelligence Without Reason", In Steels, L. and Brooks, R., eds., The Artificial Life Route to Artificial Intelligence: Building Embodied, Situated Agents, Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1995.