In this section, the use of tools to assemble C and Modula-3 modules, interfaces and libraries, into libraries and programs is illustrated. The example selected is not too complex while involving C and Modula-3 files and libraries and performing useful work. Interfacing between languages is not however something recommended for novices. The tools involved are m3build, m3ship, m3where and m3totex. Their reference manuals are in appendix.
Each library or program is a separate package with its own directory tree and m3makefile. The example detailed in this section consists in two packages:
Assuming that all the packages are stored in, for instance, the pkg subdirectory of the home directory, the following directory structure will exist:
The Database library is composed of an interface to the dbm C library, Dbm.i3, a C module Dbm.c, a module Dbase.m3 and an interface Dbase.i3. The dbm library, /usr/lib/libdbm.a, and the Modula-3 standard library, libm3, are used. Thus, the m3makefile for the database package is as follows:
import("libm3") import_lib("dbm","/usr/lib") c_source("Dbm") interface("Dbm") % non exported Interface("Dbase") % Dbase.i3 is exported implementation("Dbase") % non exported Library("Database") % libDatabase.a is exported
The interfaces to be available outside of the package are listed with Interface, and usually document the package, while the interfaces used internally are listed with interface.
The Dbm.i3 interface only redirects the calls to the appropriate C functions in dbm and provides Modula-3 declarations for the C structures and constants.
UNSAFE INTERFACE Dbm; (* The functions and types offered by the ndbm library are declared in Modula-3 terms to make them accessible to Modula-3 procedures. *) IMPORT Ctypes; TYPE Datum = RECORD dptr: Ctypes.char_star; dsize: Ctypes.int; END; T = UNTRACED ROOT; CONST DBM_INSERT = 0; DBM_REPLACE = 1; O_RDONLY = 0; O_WRONLY = 1; O_RDWR = 2; O_CREAT = 16_0200; <*EXTERNAL dbm_open*> PROCEDURE Open(file: Ctypes.char_star; flags, mode: INTEGER): T; <*EXTERNAL dbm_close*> PROCEDURE Close(db: T); <*EXTERNAL dbm2_fetch*> PROCEDURE Fetch(db: T; key: Datum): UNTRACED REF Datum; <*EXTERNAL dbm_store*> PROCEDURE Store(db: T; key, content: Datum; flags: INTEGER): INTEGER; <*EXTERNAL dbm_delete*> PROCEDURE Delete(db: T; key: Datum): INTEGER; <*EXTERNAL dbm2_firstkey*> PROCEDURE FirstKey(db: T): UNTRACED REF Datum; <*EXTERNAL dbm2_nextkey*> PROCEDURE NextKey(db: T): UNTRACED REF Datum; END Dbm.
A few lines in C are provided in Dbm.c to adapt the values returned by some functions.
#include <ndbm.h> /* Functions returning structures are often handled in non portable ways. Instead, a pointer to structure is returned here. */ datum tmp_datum; datum *dbm2_fetch(DBM *db, datum key) { tmp_datum = dbm_fetch(db,key); return &tmp_datum; } datum *dbm2_firstkey(DBM *db) { tmp_datum = dbm_firstkey(db); return &tmp_datum; } datum *dbm2_nextkey(DBM *db) { tmp_datum = dbm_nextkey(db); return &tmp_datum; }
The Dbase.i3 interface, with the corresponding implementation Dbase.m3, offers a cleaner access to the ndbm library.
INTERFACE Dbase; (* A Dbase.T is a handle on a dbm database. *) TYPE T <: REFANY; (* Create opens the named file as a new database. *) PROCEDURE Create(file: TEXT): T; (* Open opens the existing named file as a database. *) PROCEDURE Open(file: TEXT): T; (* Close releases the program resources associated with the database. *) PROCEDURE Close(db: T); (* Fetch retrieves an item in the database given its key. NIL is returned if not found. *) PROCEDURE Fetch(db: T; key: TEXT): TEXT; (* Store puts a content under the given key in the database. *) PROCEDURE Store(db: T; key, content: TEXT); (* Delete removes the specified key and its content from the database. *) PROCEDURE Delete(db: T; key: TEXT); (* FirstKey and NextKey may be used to iterate through all the keys of the database entries. NIL is returned when there are no more entries. *) PROCEDURE FirstKey(db: T): TEXT; PROCEDURE NextKey(db: T): TEXT; END Dbase.
The Dbase.i3 interface is the only visible part of the database package and contains the relevant documentation. This documentation may be extracted and formatted with m3totex.
cassis>m3totex Dbase.i3 Dbase.tex cassis>tex Dbase.tex This is TeX, Version 3.1415 (C version d) (Dbase.tex) *\end [1] Output written on Dbase.dvi (1 page, 1340 bytes). Transcript written on Dbase.log.
The implementation in Dbase.m3, for sake of simplicity, does not perform much error checking. Fortunately most errors will be caught by the Modula-3 run time checks.
The ndbm C library reuses the datum strings returned. Thus, they must be copied before calling again the dbm library. This behavior is not uncommon in C, in the absence of garbage collection, but could create problems in a multi-threaded environment.
UNSAFE MODULE Dbase; (* This module offers a cleaner interface to the ndbm functions. It converts arguments and results between Modula-3 TEXT and C character pointers. All the pointers to structures returned by C functions (DBM and datum) are UNTRACED REFs because they should not be managed by the Modula-3 garbage collector. *) IMPORT Text, Dbm, Ctypes, M3toC, Cstring, TextF; REVEAL T = BRANDED REF RECORD d: Dbm.T; END; CONST Mode = 8_77777; PROCEDURE Create(file: TEXT): T = VAR db := NEW(T); BEGIN db.d := Dbm.Open(M3toC.TtoS(file), Dbm.O_RDWR + Dbm.O_CREAT, Mode); IF db.d = NIL THEN RETURN NIL; END; RETURN db; END Create; PROCEDURE Open(file: TEXT): T = VAR db := NEW(T); BEGIN db.d := Dbm.Open(M3toC.TtoS(file), Dbm.O_RDWR, Mode); IF db.d = NIL THEN RETURN NIL; END; RETURN db; END Open; PROCEDURE Close(db: T) = BEGIN Dbm.Close(db.d); END Close; PROCEDURE Fetch(db: T; key: TEXT): TEXT = VAR in: Dbm.Datum; out: UNTRACED REF Dbm.Datum; BEGIN in.dptr := M3toC.TtoS(key); in.dsize := Text.Length(key); out := Dbm.Fetch(db.d,in); IF out.dptr = NIL THEN RETURN NIL; END; RETURN CopyStrNtoT(out.dptr,out.dsize); END Fetch; PROCEDURE Store(db: T; key, content: TEXT) = VAR k, c: Dbm.Datum; BEGIN k.dptr := M3toC.TtoS(key); k.dsize := Text.Length(key); c.dptr := M3toC.TtoS(content); c.dsize := Text.Length(content); EVAL Dbm.Store(db.d, k, c, Dbm.DBM_REPLACE); END Store; PROCEDURE Delete(db: T; key: TEXT) = VAR k: Dbm.Datum; BEGIN k.dptr := M3toC.TtoS(key); k.dsize := Text.Length(key); EVAL Dbm.Delete(db.d, k); END Delete; PROCEDURE FirstKey(db: T): TEXT = VAR out: UNTRACED REF Dbm.Datum; BEGIN out := Dbm.FirstKey(db.d); IF out.dptr = NIL THEN RETURN NIL; END; RETURN CopyStrNtoT(out.dptr,out.dsize); END FirstKey; PROCEDURE NextKey(db: T): TEXT = VAR out: UNTRACED REF Dbm.Datum; BEGIN out := Dbm.NextKey(db.d); IF out.dptr = NIL THEN RETURN NIL; END; RETURN CopyStrNtoT(out.dptr,out.dsize); END NextKey; (* This procedure relies on the internal representation of TEXT elements. *) PROCEDURE CopyStrNtoT (s: Ctypes.char_star; n: INTEGER): TEXT = VAR t := NEW (TEXT, n + 1); BEGIN EVAL Cstring.memcpy (ADR (t[0]), s, n); t[n] := '\000'; RETURN t; END CopyStrNtoT; BEGIN END Dbase.
The QueryDbase program consists in a single module which exercises the Database library. Its m3makefile and implementation, Query.m3, are as follows. It contains time and memory performance bugs which will be used later to illustrate how such bugs can be uncovered.
% database is located in our private package collection override("database","../..") import("database") implementation("Query") build_standalone() Program("QueryDbase")
MODULE Query EXPORTS Main; (* This program accepts commands to create or open ndbm databases and to store, fetch or delete elements. A sorted list of keys in the database may also be printed. *) IMPORT Stdio, Dbase, Wr, Rd, Lex, Text, Thread; (* Exceptions are not caught and will stop the program execution. *) <*FATAL Wr.Failure*> <*FATAL Rd.Failure*> <*FATAL Thread.Alerted*> (* Read all the keys in the database into an array. If the array is too small, double its size. *) PROCEDURE PrintDbase(db: Dbase.T) = VAR key, tmp: TEXT; cursor: CARDINAL := 0; BEGIN key := Dbase.FirstKey(db); WHILE key # NIL DO IF cursor > LAST(keyArray^) THEN oldKeyArray := keyArray; keyArray := NEW(REF ARRAY OF TEXT,2 * NUMBER(oldKeyArray^)); SUBARRAY(keyArray^,0,NUMBER(oldKeyArray^)) := oldKeyArray^; END; keyArray[cursor] := key; INC(cursor); key := Dbase.NextKey(db); END; (* Use an inefficient sorting algorithm *) FOR i := cursor - 1 TO 0 BY -1 DO FOR j := 0 TO i - 1 DO IF Text.Compare(keyArray[j], keyArray[i]) = 1 THEN tmp := keyArray[i]; keyArray[i] := keyArray[j]; keyArray[j] := tmp; END; END; END; (* Print all the keys *) FOR i := 0 TO cursor - 1 DO Wr.PutText(Stdio.stdout,keyArray[i] & "\n"); END; END PrintDbase; (* The array for printing the keys is kept from one invocation to the next instead of starting from scratch each time the PrintDbase procedure is called. *) VAR keyArray := NEW(REF ARRAY OF TEXT,8); oldKeyArray : REF ARRAY OF TEXT; db: Dbase.T; command, name, key, content: TEXT; BEGIN (* Commands are read. For each possible command, the appropriate few lines are executed. *) LOOP Wr.PutText(Stdio.stdout,"Enter command\n"); Wr.Flush(Stdio.stdout); Lex.Skip(Stdio.stdin); command := Lex.Scan(Stdio.stdin); IF Text.Equal(command,"exit") THEN EXIT; ELSIF Text.Equal(command,"create") THEN Lex.Skip(Stdio.stdin); name := Lex.Scan(Stdio.stdin); db := Dbase.Create(name); ELSIF Text.Equal(command,"open") THEN Lex.Skip(Stdio.stdin); name := Lex.Scan(Stdio.stdin); db := Dbase.Open(name); ELSIF Text.Equal(command,"close") THEN Dbase.Close(db); ELSIF Text.Equal(command,"fetch") THEN Lex.Skip(Stdio.stdin); key := Lex.Scan(Stdio.stdin); Wr.PutText(Stdio.stdout,Dbase.Fetch(db,key) & "\n"); ELSIF Text.Equal(command,"store") THEN Lex.Skip(Stdio.stdin); key := Lex.Scan(Stdio.stdin); Lex.Skip(Stdio.stdin); content := Lex.Scan(Stdio.stdin); Dbase.Store(db,key,content); ELSIF Text.Equal(command,"delete") THEN Lex.Skip(Stdio.stdin); key := Lex.Scan(Stdio.stdin); Dbase.Delete(db,key); ELSIF Text.Equal(command,"print") THEN PrintDbase(db); ELSE Wr.PutText(Stdio.stdout,"Incorrect command\n"); END; END; END Query.
Whenever the database source files (Dbase.i3, Dbase.m3, Dbm.i3 or Dbm.c) or one of the imported libraries (ndbm or libm3) are modified, the library may be rebuilt (recompiling only the needed modules) with m3build. Similarly, when query or its imported libraries are modified (Query.m3 or database), it may be rebuilt with m3build.
cassis>cd ~/pkg/database cassis>ls src cassis>m3build mkdir LINUX --- building in LINUX --- m3 -w1 -why -g -a libDatabase.a -F/usr/tmp/qk13734aaa new source -> compiling ../src/Dbm.c new source -> compiling ../src/Dbm.i3 new source -> compiling ../src/Dbase.i3 new source -> compiling ../src/Dbase.m3 -> archiving libDatabase.a cassis>cd .. cassis>ls database/ query/ cassis>cd query cassis>ls src/ cassis>m3build mkdir LINUX --- building in LINUX --- m3 -w1 -why -g -o QueryDbase -F/usr/tmp/qk13793aaa new source -> compiling ../src/Query.m3 -> linking QueryDbase
Often, a common repository is used to make the packages available for general use. The m3ship command is used to copy all the exported interfaces and binaries produced by a package to the common repository. Once a package is installed in the common repository, there is no need to use the override command in the m3makefile to specify its location.
cassis>cd ~/pkg/database cassis>m3ship m3mkdir /usr/local/lib/m3/pkg/database/src cp -p Dbase.i3 /usr/local/lib/m3/pkg/database/src m3mkdir /usr/local/lib/m3/pkg/database/LINUX cp -p libDatabase.a /usr/local/lib/m3/pkg/database/LINUX --- shipping from LINUX ---
To know exactly where all the included modules, interfaces and libraries were found, the m3where command may be used. This is useful, among other things, to specify the paths when using a debugger.
cassis>cd ~/pkg/query cassis>m3where -h --- searching in LINUX --- /usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/src/random/Common/RandomPerm.i3 /usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/src/formatter/Formatter.i3 /usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/LINUX/SortedTextRefTbl.i3 ... ../../database/src/Dbase.i3
The executable program is produced in the architecture specific directory, LINUX on an intel architecture running the Linux operating system.
cassis>LINUX/QueryDbase Enter command create example Enter command store key1 content1 Enter command store key3 content3 Enter command store key2 content2 Enter command print key1 key2 key3 Enter command close Enter command exit cassis>