Modules in Modula-3

In this section, the use of tools to assemble C and Modula-3 modules, interfaces and libraries, into libraries and programs is illustrated. The example selected is not too complex while involving C and Modula-3 files and libraries and performing useful work. Interfacing between languages is not however something recommended for novices. The tools involved are m3build, m3ship, m3where and m3totex. Their reference manuals are in appendix.

Each library or program is a separate package with its own directory tree and m3makefile. The example detailed in this section consists in two packages:

Assuming that all the packages are stored in, for instance, the pkg subdirectory of the home directory, the following directory structure will exist:

The Database library is composed of an interface to the dbm C library, Dbm.i3, a C module Dbm.c, a module Dbase.m3 and an interface Dbase.i3. The dbm library, /usr/lib/libdbm.a, and the Modula-3 standard library, libm3, are used. Thus, the m3makefile for the database package is as follows:



import("libm3")
import_lib("dbm","/usr/lib")

c_source("Dbm")

interface("Dbm")          % non exported

Interface("Dbase")        % Dbase.i3 is exported

implementation("Dbase")   % non exported

Library("Database")       % libDatabase.a is exported

The interfaces to be available outside of the package are listed with Interface, and usually document the package, while the interfaces used internally are listed with interface.

The Dbm.i3 interface only redirects the calls to the appropriate C functions in dbm and provides Modula-3 declarations for the C structures and constants.


UNSAFE INTERFACE Dbm;

(* The functions and types offered by the ndbm library are declared
   in Modula-3 terms to make them accessible to Modula-3 procedures. *)

IMPORT Ctypes;

TYPE
  Datum = RECORD
      dptr: Ctypes.char_star;
      dsize: Ctypes.int;
    END;

  T = UNTRACED ROOT;

CONST
  DBM_INSERT = 0;
  DBM_REPLACE = 1;
  O_RDONLY = 0;
  O_WRONLY = 1;
  O_RDWR = 2;
  O_CREAT = 16_0200;

<*EXTERNAL dbm_open*>
PROCEDURE Open(file: Ctypes.char_star; flags, mode: INTEGER): T;

<*EXTERNAL dbm_close*>
PROCEDURE Close(db: T);

<*EXTERNAL dbm2_fetch*>
PROCEDURE Fetch(db: T; key: Datum): UNTRACED REF Datum;

<*EXTERNAL dbm_store*>
PROCEDURE Store(db: T; key, content: Datum; flags: INTEGER): INTEGER;

<*EXTERNAL dbm_delete*>
PROCEDURE Delete(db: T; key: Datum): INTEGER;

<*EXTERNAL dbm2_firstkey*>
PROCEDURE FirstKey(db: T): UNTRACED REF Datum;

<*EXTERNAL dbm2_nextkey*>
PROCEDURE NextKey(db: T): UNTRACED REF Datum;

END Dbm.

A few lines in C are provided in Dbm.c to adapt the values returned by some functions.



#include <ndbm.h>

/* Functions returning structures are often handled in non portable
   ways. Instead, a pointer to structure is returned here. */

datum tmp_datum;

datum *dbm2_fetch(DBM *db, datum key)
{
  tmp_datum = dbm_fetch(db,key);
  return &tmp_datum;
}

datum *dbm2_firstkey(DBM *db)
{
  tmp_datum = dbm_firstkey(db);
  return &tmp_datum;
}

datum *dbm2_nextkey(DBM *db)
{
  tmp_datum = dbm_nextkey(db);
  return &tmp_datum;
}

The Dbase.i3 interface, with the corresponding implementation Dbase.m3, offers a cleaner access to the ndbm library.


INTERFACE Dbase;

(* A Dbase.T is a handle on a dbm database. *)

TYPE
  T <: REFANY;

(* Create opens the named file as a new database. *)

PROCEDURE Create(file: TEXT): T;

(* Open opens the existing named file as a database. *)

PROCEDURE Open(file: TEXT): T;

(* Close releases the program resources associated with the database. *)

PROCEDURE Close(db: T);

(* Fetch retrieves an item in the database given its key. NIL is returned
   if not found. *)

PROCEDURE Fetch(db: T; key: TEXT): TEXT;

(* Store puts a content under the given key in the database. *)

PROCEDURE Store(db: T; key, content: TEXT);

(* Delete removes the specified key and its content from the database. *)

PROCEDURE Delete(db: T; key: TEXT);

(* FirstKey and NextKey may be used to iterate through all the 
   keys of the database entries. NIL is returned when there are no more
   entries. *)

PROCEDURE FirstKey(db: T): TEXT;

PROCEDURE NextKey(db: T): TEXT;

END Dbase.

The Dbase.i3 interface is the only visible part of the database package and contains the relevant documentation. This documentation may be extracted and formatted with m3totex.


cassis>m3totex Dbase.i3 Dbase.tex
cassis>tex Dbase.tex
This is TeX, Version 3.1415 (C version d)
(Dbase.tex)
*\end
[1]
Output written on Dbase.dvi (1 page, 1340 bytes).
Transcript written on Dbase.log.

The implementation in Dbase.m3, for sake of simplicity, does not perform much error checking. Fortunately most errors will be caught by the Modula-3 run time checks.

The ndbm C library reuses the datum strings returned. Thus, they must be copied before calling again the dbm library. This behavior is not uncommon in C, in the absence of garbage collection, but could create problems in a multi-threaded environment.



UNSAFE MODULE Dbase;

(* This module offers a cleaner interface to the ndbm functions.
   It converts arguments and results between Modula-3 TEXT and 
   C character pointers. All the pointers to structures returned by
   C functions (DBM and datum) are UNTRACED REFs because they should
   not be managed by the Modula-3 garbage collector. *)

IMPORT Text, Dbm, Ctypes, M3toC, Cstring, TextF;

REVEAL
  T = BRANDED REF RECORD 
      d: Dbm.T; 
    END;

CONST
  Mode = 8_77777;

PROCEDURE Create(file: TEXT): T =
  VAR
    db := NEW(T);
  BEGIN
    db.d := Dbm.Open(M3toC.TtoS(file), Dbm.O_RDWR + Dbm.O_CREAT, Mode);
    IF db.d = NIL THEN RETURN NIL; END;
    RETURN db;
  END Create;

PROCEDURE Open(file: TEXT): T =
  VAR
    db := NEW(T);
  BEGIN
    db.d := Dbm.Open(M3toC.TtoS(file), Dbm.O_RDWR, Mode);
    IF db.d = NIL THEN RETURN NIL; END;
    RETURN db;
  END Open;

PROCEDURE Close(db: T) =
  BEGIN
    Dbm.Close(db.d);
  END Close;

PROCEDURE Fetch(db: T; key: TEXT): TEXT =
  VAR
    in: Dbm.Datum;
    out: UNTRACED REF Dbm.Datum;
  BEGIN
    in.dptr := M3toC.TtoS(key);
    in.dsize := Text.Length(key);
    out := Dbm.Fetch(db.d,in);
    IF out.dptr = NIL THEN RETURN NIL; END;
    RETURN CopyStrNtoT(out.dptr,out.dsize);
  END Fetch;

PROCEDURE Store(db: T; key, content: TEXT) =
  VAR
    k, c: Dbm.Datum;
  BEGIN
    k.dptr := M3toC.TtoS(key);
    k.dsize := Text.Length(key);
    c.dptr := M3toC.TtoS(content);
    c.dsize := Text.Length(content);
    EVAL Dbm.Store(db.d, k, c, Dbm.DBM_REPLACE);
  END Store;

PROCEDURE Delete(db: T; key: TEXT) =
  VAR
    k: Dbm.Datum;
  BEGIN
    k.dptr := M3toC.TtoS(key);
    k.dsize := Text.Length(key);
    EVAL Dbm.Delete(db.d, k);
  END Delete; 

PROCEDURE FirstKey(db: T): TEXT =
  VAR
    out: UNTRACED REF Dbm.Datum;
  BEGIN
    out := Dbm.FirstKey(db.d);
    IF out.dptr = NIL THEN RETURN NIL; END;
    RETURN CopyStrNtoT(out.dptr,out.dsize);
  END FirstKey;

PROCEDURE NextKey(db: T): TEXT =
  VAR
    out: UNTRACED REF Dbm.Datum;
  BEGIN
    out := Dbm.NextKey(db.d);
    IF out.dptr = NIL THEN RETURN NIL; END;
    RETURN CopyStrNtoT(out.dptr,out.dsize);
  END NextKey;

(* This procedure relies on the internal representation of TEXT
   elements. *)

PROCEDURE CopyStrNtoT (s: Ctypes.char_star; n: INTEGER): TEXT =
  VAR 
    t := NEW (TEXT, n + 1);
  BEGIN
    EVAL Cstring.memcpy (ADR (t[0]), s, n);
    t[n] := '\000';
    RETURN t;
  END CopyStrNtoT;

BEGIN
END Dbase.

The QueryDbase program consists in a single module which exercises the Database library. Its m3makefile and implementation, Query.m3, are as follows. It contains time and memory performance bugs which will be used later to illustrate how such bugs can be uncovered.


% database is located in our private package collection
override("database","../..")

import("database")

implementation("Query")
build_standalone()
Program("QueryDbase")


MODULE Query EXPORTS Main;

(* This program accepts commands to create or open ndbm databases
   and to store, fetch or delete elements. A sorted list of keys
   in the database may also be printed. *)

IMPORT Stdio, Dbase, Wr, Rd, Lex, Text, Thread;

(* Exceptions are not caught and will stop the program execution. *)

<*FATAL Wr.Failure*>
<*FATAL Rd.Failure*>
<*FATAL Thread.Alerted*>

(* Read all the keys in the database into an array. If the array
   is too small, double its size. *)

PROCEDURE PrintDbase(db: Dbase.T) =
  VAR
    key, tmp: TEXT;
    cursor: CARDINAL := 0;
  BEGIN
    key := Dbase.FirstKey(db);
    WHILE key # NIL DO
      IF cursor > LAST(keyArray^) THEN
        oldKeyArray := keyArray;
        keyArray := NEW(REF ARRAY OF TEXT,2 * NUMBER(oldKeyArray^));
        SUBARRAY(keyArray^,0,NUMBER(oldKeyArray^)) := oldKeyArray^;
      END;
      keyArray[cursor] := key;
      INC(cursor);
      key := Dbase.NextKey(db);
    END;

    (* Use an inefficient sorting algorithm *)

    FOR i := cursor - 1 TO 0 BY -1 DO
      FOR j := 0 TO i - 1 DO
        IF Text.Compare(keyArray[j], keyArray[i]) = 1 THEN
          tmp := keyArray[i];
          keyArray[i] := keyArray[j];
          keyArray[j] := tmp;
        END;
      END;
    END;

    (* Print all the keys *)

    FOR i := 0 TO cursor - 1 DO
      Wr.PutText(Stdio.stdout,keyArray[i] & "\n");
    END;
  END PrintDbase;

(* The array for printing the keys is kept from one invocation to the next
   instead of starting from scratch each time the PrintDbase procedure 
   is called. *)

VAR
  keyArray := NEW(REF ARRAY OF TEXT,8);
  oldKeyArray : REF ARRAY OF TEXT;
  db: Dbase.T;
  command, name, key, content: TEXT;

BEGIN

  (* Commands are read. For each possible command, the
     appropriate few lines are executed. *)

  LOOP
    Wr.PutText(Stdio.stdout,"Enter command\n");
    Wr.Flush(Stdio.stdout);

    Lex.Skip(Stdio.stdin);
    command := Lex.Scan(Stdio.stdin);

    IF Text.Equal(command,"exit") THEN EXIT;

    ELSIF Text.Equal(command,"create") THEN
      Lex.Skip(Stdio.stdin);
      name := Lex.Scan(Stdio.stdin);
      db := Dbase.Create(name);

    ELSIF Text.Equal(command,"open") THEN
      Lex.Skip(Stdio.stdin);
      name := Lex.Scan(Stdio.stdin);
      db := Dbase.Open(name);

    ELSIF Text.Equal(command,"close") THEN
      Dbase.Close(db);

    ELSIF Text.Equal(command,"fetch") THEN
      Lex.Skip(Stdio.stdin);
      key := Lex.Scan(Stdio.stdin);
      Wr.PutText(Stdio.stdout,Dbase.Fetch(db,key) & "\n");

    ELSIF Text.Equal(command,"store") THEN
      Lex.Skip(Stdio.stdin);
      key := Lex.Scan(Stdio.stdin);
      Lex.Skip(Stdio.stdin);
      content := Lex.Scan(Stdio.stdin);
      Dbase.Store(db,key,content);

    ELSIF Text.Equal(command,"delete") THEN
      Lex.Skip(Stdio.stdin);
      key := Lex.Scan(Stdio.stdin);
      Dbase.Delete(db,key);

    ELSIF Text.Equal(command,"print") THEN
      PrintDbase(db);

    ELSE
      Wr.PutText(Stdio.stdout,"Incorrect command\n");
    END;
  END;
END Query.

Whenever the database source files (Dbase.i3, Dbase.m3, Dbm.i3 or Dbm.c) or one of the imported libraries (ndbm or libm3) are modified, the library may be rebuilt (recompiling only the needed modules) with m3build. Similarly, when query or its imported libraries are modified (Query.m3 or database), it may be rebuilt with m3build.


cassis>cd ~/pkg/database
cassis>ls
src
cassis>m3build
mkdir LINUX
--- building in LINUX ---
m3 -w1 -why -g -a libDatabase.a -F/usr/tmp/qk13734aaa 
new source -> compiling ../src/Dbm.c
new source -> compiling ../src/Dbm.i3
new source -> compiling ../src/Dbase.i3
new source -> compiling ../src/Dbase.m3
 -> archiving libDatabase.a
cassis>cd ..
cassis>ls
database/  query/
cassis>cd query
cassis>ls
src/
cassis>m3build
mkdir LINUX
--- building in LINUX ---
m3 -w1 -why -g -o QueryDbase -F/usr/tmp/qk13793aaa 
new source -> compiling ../src/Query.m3
 -> linking QueryDbase

Often, a common repository is used to make the packages available for general use. The m3ship command is used to copy all the exported interfaces and binaries produced by a package to the common repository. Once a package is installed in the common repository, there is no need to use the override command in the m3makefile to specify its location.


cassis>cd ~/pkg/database
cassis>m3ship
m3mkdir /usr/local/lib/m3/pkg/database/src
cp -p Dbase.i3 /usr/local/lib/m3/pkg/database/src
m3mkdir /usr/local/lib/m3/pkg/database/LINUX
cp -p libDatabase.a /usr/local/lib/m3/pkg/database/LINUX
--- shipping from LINUX ---

To know exactly where all the included modules, interfaces and libraries were found, the m3where command may be used. This is useful, among other things, to specify the paths when using a debugger.


cassis>cd ~/pkg/query
cassis>m3where -h
--- searching in LINUX ---
/usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/src/random/Common/RandomPerm.i3
/usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/src/formatter/Formatter.i3
/usr/local/soft/modula3-3.3/lib/m3/pkg/libm3/LINUX/SortedTextRefTbl.i3

...

../../database/src/Dbase.i3

The executable program is produced in the architecture specific directory, LINUX on an intel architecture running the Linux operating system.


cassis>LINUX/QueryDbase
Enter command
create example
Enter command
store key1 content1
Enter command
store key3 content3
Enter command
store key2 content2
Enter command
print
key1
key2
key3
Enter command
close
Enter command
exit
cassis>


Copyright 1995 Michel Dagenais, dagenais@vlsi.polymtl.ca, Wed Mar 8 14:41:03 EST 1995