DynBench: A Dynamic Real-time Benchmark Suite and Environment
(manual for use of the benchmark applications and associated tools)
Laboratory for Parallel and Distributed Real-time Systems
Department of Computer Science and Engineering
University of Texas at Arlington
This work was sponsored in part by DARPA/NCCOSC contract N66001-97-C-8250, and by NSWC/NCEE contracts NCEE/A303/41E-96 and NCEE/A303/50A-98.
1 Introduction *
2 Installation *
3.2 Manual Startup *
3.3 The Filter Manager (FM) *
3.4 Evaluate & Decide Manager *
3.5 Filter *
3.6 Evaluate and Decide (ED) *
3.7 Sensor *
3.8 experiment generator *
3.9 Action Manager *
3.10 Action *
3.11 Actuator *
3.12 Monitor & Guide Manager *
3.13 Monitor & Guide *
3.14 The Radar Console *
3.15 Scenario File *
After careful study of two instances
of an air defense system (adynamic real-time system), a working prototype
of such a system has been constructed as a benchmark for evaluation of
various approaches to dynamic resource and QoS management. The prototype
uses simulated sensors and actuators, and employs real algorithms for filtering,
evaluating, acting, and guidance. It also has the ability to dynamically
adapt to changes in environmental conditions via the addition and removal
of load sharing replicas of application programs.
Figure 1: The DynBench Applications
The software benchmark subsystem exhibits the behavior of all three dynamic path types: continuous, transient, and quasi-continuous (see Appendix for definition of the dynamic real-time path paradigm). The (continuous) situation assessment path consists of the sensor (a simulator program), a filter manager program, one or more replicas of the filter program, an evaluate & decide manager (E & D Manager), and one or more replicas of an evaluate and decide program. The (continuous) console path includes all programs of the situation assessment path plus a display console program. The (transient) initiation path consists of three programs: action manager, action (possibly replicated), and an actuator simulator. The (quasi-continuous) monitor and guidance path includes all components of the situation assessment path plus an M&G manager program and a monitor&guide program (possibly replicated).
The evaluation copy of DynBench consists of all three paths (depicted in Figure 1). The situation assessment path continuously senses the presence of moving bodies and evaluates their motions and relative positions. The simulated sensor reads a scenario file and generates a set of data samples (referred to as tracks) based on equations of motion contained in the file. The data samples represent the current positions of the moving bodies with respect to time. Sensor has the ability to service requests from other processes to add, modify, or delete the equations of motion of simulated bodies. The sensor supplies a continuous data stream to filter data manager, which distributes the current workload among the currently active filters. Each filter uses a least mean square regression algorithm to correlate the point data into three equations describing the motion of the body. The equations of motion for each of the observed bodies are sent to the evaluate and decide data manger, which distributes the current workload among the currently active evaluate and decide worker programs. For human observation, the equations of motion are also sent to console, where they are displayed as moving radar tracks. The evaluate and decide data manager also keeps track of the current evaluation rules and updates all the evaluate and decide programs when these rules change. Evaluate and decide processes determine if the current position of an observed body is within a critical region based on the evaluation rules. If it is a previously unidentified body of interest, it is passed to the initiation path. If it had been identified previously, it is sent to the monitor and guide data manager.
The components in the evaluation copy of DynBench are explained below:
Desiderata
|_____bin
|_____docs
|_____include
|_____lib
Update the command line arguments for each shortcut in the "DynBenchX_X"
folder. You should only need to chanage the "name server host" to the actual
host name where you will run the "NameSever" program. In some cases you
might have to change the port as well, we have found that 7300 is usually
unused.
The readme files on the following pages explain how to start the benchmark
applications manually.
Invocation syntax:
FM <unique name> <path id_no.> <name_server_host>
<name_server_port> <QoS path identifier>
Example: FM FM 1 texas 7300 <D:B:sensing>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
EDM <unique name> <path id_no.> <name_server_host>
<name_server_port> <QoS path identifier>
Example: EDM EDM 1 texas 7300 <D:B:sensing>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
filter <path id_no.> <name_server_host> <name_server_port>
<QoS path identifier>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
ED <path id_no.> <name_server_host> <name_server_port>
<EDM unique name> <AM unique name>
<MGM unique name> <DM unique name>
<QoS path identifier>
Example: ED 1 texas 7300 EDM AM MGM DM <D:B:sensing>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
sensor <unique name> <path id_no.> <name_server_host>
<name_server_port> <FM unique name>
<actuator unique name> <QoS path identifier>
Example: sensor sensor 1 texas 7300 FM actuator <D:B:sensing>
OPERANDS:
<QoS path identifier> should be omitted.
It also has interface to NSWC load simulator user interface.
Invocation syntax:
EG <name_server_host> <name_server_port> [load_simulator_host] [load_simulator_port]
Example: EG texas 7300
OPERANDS:
This application can run in two modes - interactive and command file. In command file, user has to specify a file name.
Usage of the Experiment Generator is illustrated in the figures below.
Invocation syntax:
AM <unique name> <path id_no.> <name_server_host>
<name_server_port> <QoS path identifier>
Example: AM AM 1 texas 7300 <D:B:action>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
action <path id_no.> <name_server_host>
<name_server_port> <action manager> <actuator>
<QoS path identifier>
Example: action 1 texas 7300 AM actuator<D:B:Action>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
actuator <unique name> <path id_no.> <name_server_host>
<name_server_port> <QoS path identifier>
Example: actuator actuator 1 texas 7300 <D:B:action>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
MGM <unique name> <path id_no.> <name_server_host>
<name_server_port> <QoS path identifier>
Example: MGM MGM 1 texas 7300 <D:B:Guidance>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
MG <path id_no.> <name_server_host>
<name_server_port> <monitor & guide manager>
<sensor> <QoS path identifier>
Example: MG 1 texas 7300 MGM sensor <D:B:Guidance>
OPERANDS:
<QoS path identifier> should be omitted.
Invocation syntax:
DM <Unique Name> <path_identifier> <NameServer Host_name>
OPERANDS:
Invocation syntax:
DC <path_identifier> <NameServerhost_name> <NameServerportno>
OPERANDS:
Invocation:
Step 1:
Start the program, select File|Connect menu.
Step 2:
Input the name server's host name and port number, also the name of Evaluate Decide Manager. After pressing the OK button, the radar console will connect to EDM.
Step 3:
You can select the Zoom|Zoom In/Out/Center menu to zoom the screen, and you can also click the toolbar buttons (+/-) or just press +|- in keyboard to zoom in or out. The Figure below shows the effect of zooming in on the view captured in Figure 4.
Step 4:
The tracks will be shown on the screen as shown in the Figure below.
The gray circle is the red alert / critical zone, a cylinder defined in
the evaluation file. Each radr track is labeled with its height, type and
recent trail. If the track is inside the red alert zone, the trail is marked
by red, otherwise it is green.
Cylce=<cycle frequency> // second
Type="<track type>"
Start=<start cycle for this track>
X = <b0> + <b1>t + <b2>t2 ; // East-West Motion Equation
Y = <b0> + <b1>t + <b2>t2 ; // North-South Motion Equation
Z = <b0> + <b1>t + <b2>t2 ; // Altitude Motion Equation
<b0> : Intercept
<b1> : Slope
<b2> : Curve
t : time (current cycle)
t2 = t * t ;
An example scenario file is:
Cycle = 1.0 ;
Type = RANDOM ;
Start = 1 ;
X = 0.00t2 - 10.5t + 600 ;
Y = 0.00t2 - 6.5t + 300 ;
Z = 0.0t2 - 0.082t + 20 ;
Type= THREATS
start = 1 ;
X = 0.0t2 + 5.7t - 590 ;
Y = 0.0t2 - 6.7t + 400 ;
Z = 0.0t2 - 0.87t + 2 ;
Name Server, QoS Manager, QoS Manager HCI
This section presents the dynamic real-time path paradigm. A path-based real-time subsystem (see Figure 7) typically consists of a situation assessment path, an action initiation path and an action guidance path. The paths interact with the environment via evaluating streams of data from sensors, and by causing actuators to respond (in a timely manner) to events detected during evaluation of sensor data streams. The system operates in an environment that is either deterministic, stochastic or dynamic. A deterministic environment exhibits behavior that can be characterized by a constant value. A stochastic environment behaves in a manner that can be characterized by a statistical distribution. A dynamic environment (e.g., a war-fighting environment) depends on conditions which cannot be known in advance.
A (partial) air defense subsystem can be modeled using three dynamic paths: threat detection, engagement, and missile guidance. The threat detection path examines radar sensor data (radar tracks) and detects potential threats. The path consists of a radar sensor, a sensor data stream, a filtering program and an evaluation program. When a threat is detected and confirmed, the engagement path is activated, resulting in the firing of a missile to engage the threat. After a missile is in flight, the missile guidance path uses sensor data to track the threat, and issues guidance commands to the missile. The missile guidance path involves sensor hardware, software for filtering, software for evaluating & deciding, software for acting, and actuator hardware.
The remainder of this section describes the features of dynamic real-time paths. Recall that a path may be one of three types: situation assessment, action initiation, or action guidance. The first path type, situation assessment, continuously evaluates the elements of a sensor data stream to determine if environmental conditions are such that an action should be taken (if so, the action initiation path is notified). Thus, this type of path is called continuous. Typically, there is a timeliness objective associated with completion of one review cycle of a continuous path, i.e., on the time to review all of the elements of one instance of a data stream. (Note: A data stream is produced by sampling the environment. One set of samples is called a data stream instance.)
The threat detection path of the air defense system is an example of a continuous path. It is a sensor-data-stream-driven path, with desired end-to-end cycle latencies for evaluation of radar track data. If it fails to meet the desired timeliness Quality of Service in a particular cycle, the path must continue to process track data, even though desired end-to-end latencies cannot be achieved. Peak loads cannot be known in advance for the threat detection path, since the maximum number of radar tracks can never be known. Furthermore, average loading of the path is a meaningless notion, since the variability in the sensor data stream size is very large (it may consist of zero tracks, or it may consist of thousands of tracks).
The second path type, action initiation, is driven by a stream of events sent by a continuous (situation assessment) path. It uses inputs from sensors to determine which actions should be taken and how the actions should be performed, notifies actuators to start performing the actions, and informs the action guidance path that an action has been initiated. We call this type of path transient, since it performs work in response to events. Typically, a timing objective is associated with the completion of the initiation sequence. The importance of the timing objective for a transient path is often very high, since performance of an action may be mission-critical or safety-critical.
For example, the engagement path of the air defense example is a transient path. It is activated by an event from the threat detection path, and has a QoS objective of end-to-end timeliness. The real-time QoS of this path has a higher priority than the real-time QoS of the continuous threat detection path.
The third path type, action guidance, is activated by an action initiation event, and is deactivated upon completion of the action. Action guidance repeatedly uses sensor inputs to monitor the progress of an actuator, to plan corrective actions needed to guide the actuator to its goal, and to issue commands to the actuator. This type of path is called quasi-continuous, since it behaves like a continuous path when it is active. A quasi-continuous path has two timeliness objectives: (1) cycle completion time: the duration of one iteration of the "monitor, plan, command" loop, and (2) action completion time (or deactivation time): the time by which the action must complete in order for success. Note that it is more critical to perform the required processing before the action completion deadline than it is to meet the completion time requirement for every cycle (although the two deadlines are certainly related). Thus, it is acceptable for the completion times of some cycles to violate the cycle deadline requirement, as long as the desired actions are successfully completed by the deactivation deadline.
The missile guidance path of the air defense example is a quasi-continuous path. It is activated by the missile launch event. Once activated, the path continuously issues guidance commands to the missile until it detonates (the deactivation event). The required completion time for one iteration is dynamically determined, based on characteristics of the threat. If multiple threat engagements are active simultaneously, the threat engagement path is responsible for issuing guidance commands to all missiles that have been launched.