Design Document #1

Rule Engine for Integrator

Introduction

Integrator Architecture

Integrator Specific phython command set

• begin
• This commands demarks the begining of integrator rule set. Internally this initializes the rule engine, embeded with standard python engine. The usage of this command is mandatory otherwise, usage of any *custom*/*new* integrator commands emits error as UNINITIALIZED_RULE_ENGINE.
• Usage:    > begin
• end
• This command demarks the end of integrator rule set. Internally this command issues a cleaning initiative and drop of all temporary tables/files used internally for rule computations. This usage is mandatory otherwise,usage of any "begin" command emits error as UNCLOSED_RULE_ENGINE.
• If the python engine, the container for custom rule engine, is exited, the system automatically clean up the temporary file as described above.
• Usage :  > end
• errorno
• This commands displays the last error condition, experienced in the current rule execution. This is with the similar idea of UNIX error variable,errorno.  Whenever the rule engine experiences an error condition, this variable is set. The latest error always overwrites the previous error if any.
• Usage : >errorno
• describe <errorno>
• This command describes briefly about the error, its possible cause and possible way to rectify this error.
• Usage : > describe UNINITIALIZED_RULE_ENGINE
• load <file_path_name>
• This commands loads the input data file. Internally this file is parsed with input schema converter and converted into internal XML data format. In case of errors, this command emits following error:
• UNKNOWN_FILE_RULE_ENGINE -: if file_path_name is invalid
• UNKNOWN_DATA_FORMAT_RULE_ENGINE -: if data from input file does not match with the input schema convertor.
• PARSE_ERROR_RULE_ENGINE -: data parsing error.
• Usage: > load  /mnt/myserver/rdbms/oracle/hr/export/employee.dat
• In future release, we need to modify this command to execute SQL statements so that it can load the results data directly from the data source.
• spit <output stream>
• This command uses for debugging purpose. This command spits out the internal XML data {before/after rule transformation} on the output stream. By default, it spits out on the standard output i.e STDOUT. If error, this emits the following : OUTPUT_STREAM_ERROR_RULE_ENGINE. Internally the xml data is stored as DOM tree.
• Usage: > spit myStream
• dump <output stream > | <output file name>
• This command dumps the processed data directly into output stream or into the datafile given as an argument. Internally it uses the output schema converter to convert data from internal XML format to the target data format. If error, it emits the followings :
• OUTPUT_STREAM_ERROR_RULE_ENGINE : if there is an error in output stream
• OUTPUT_FILE_ERROR_RULE_ENGINE: if there is an error in output file
• OUTPUT_GENERATE_ERROR_RULE_ENGINE : if there is an error while generating target schema data from xml internal data.
• Usage : > dump STDOUT
• run
• This commands executes the custom rules on internal XML data and transforms into *processed* xml data. Since the xml data internally represents as a DOM tree. This commands actually modify the initial tree structure and regenerate a new processed tree. If error, it emits the following :
• PROCESSING_ERROR_RULE_ENGINE: if there is an error internally while processing.
• Usage : > run
• rule  [ add| sub| mul|div|mod   a, b ,c ]  [ concat a,a1,a2,.......,an] [ split a[],a,b] [ replace a,b,c]
• This command describes each custom rules.All a,b,c,a1,......,an, a[] are qualified variables. That means, the values are associated with the name of element (XML). The basic assumpsion is the element name is Unique in the DOM tree.
• add/sub/mul/div/mod a,b,c  => a = b op c where op = +,-,*,/,% and a,b,c = real numbers
• concat a,a1,a2,.....,an => a .= ai.a(i-1)  where i = 1....(n-1)
• split a[], a,b => split a with b (for each occurence of b in a ) and store them to a[]
• replace a,b,c => replace the substring b with c in a
• If  error, this command emits the following:
• ARITHMATIC_ERROR_RULE_ENGINE
• Usage : > rule add V(TOTAL), V(RUM), V(GIN)    { adding rum and gin price to total }

Example scripts