Distributed Persistent Objects in Modula-3

The Sharedboard diagram editor examined in earlier sections uses distributed and persistent objects. Network objects allow several users to simultanously edit diagrams while persistent objects insure that the up to date diagrams are always available on persistent storage even if the diagram server crashes.

Network objects

A distributed version of the diagram editor would allow several users to simultaneously see and interact with the same diagrams. This may be useful to complement a phone conversation with graphical explanations or to insure that everyone sees immediately any modifications to a diagram, to avoid the problems associated with reconciling the local changes made by several users when they take a local copy.

With network objects, it is possible to call the methods of objects which are in another process, possibly on a remote server. The stubgen tool and netobj library take care of relaying the methods and arguments between programs.

A board server maintains a table of currently opened boards. When a user requests a board already in use, he obtains a reference to that board. Thus, the board will receive editing commands from several users and must notify the other users.


INTERFACE BoardServer;

IMPORT NetObj, Thread, 
       Board;

TYPE T = NetObj.T OBJECT 
  METHODS
    create (boardName: TEXT): Board.T 
        RAISES {Failed, NetObj.Error, Thread.Alerted};
    open (boardName: TEXT): Board.T 
        RAISES {Failed, NetObj.Error, Thread.Alerted};
    save (boardName: TEXT) 
        RAISES {Failed, NetObj.Error, Thread.Alerted};
    close (boardName: TEXT) 
        RAISES {Failed, NetObj.Error, Thread.Alerted};
    remove (boardName: TEXT) 
        RAISES {Failed, NetObj.Error, Thread.Alerted};
  END;

EXCEPTION Failed (TEXT);

Once a board is obtained from the server, editing commands must be sent to it.


INTERFACE Board;

IMPORT NetObj, Thread,
       Callback, Item, ClientInfo, RectR;

TYPE T = NetObj.T OBJECT 
  METHODS
    register    (cb: Callback.T): ClientInfo.T 
        RAISES {NetObj.Error, Thread.Alerted};
    setScope    (ci: ClientInfo.T; scope: RectR.T) 
        RAISES {NetObj.Error, Thread.Alerted};
    createItems  (ci: ClientInfo.T; its: Item.TArray): Item.IDArray
        RAISES {NetObj.Error, Thread.Alerted};
    modifyItems  (ci: ClientInfo.T; its: Item.TArray; additive: BOOLEAN)
       RAISES {NetObj.Error, Thread.Alerted};
    deleteItems  (ci: ClientInfo.T; ids: Item.IDArray)
       RAISES {NetObj.Error, Thread.Alerted};
    unregister  (ci: ClientInfo.T)
       RAISES {NetObj.Error, Thread.Alerted};
  END;

Before editing a board, a new client must register with the board by providing a Callback object used to interact with the client. This Callback object will be used by the board to notify this client when other clients modify the board contents. The Callback object too is a network object.


INTERFACE Callback;

IMPORT NetObj, Thread,
       Item;

TYPE T = NetObj.T OBJECT 
    METHODS
      itemsCreated (it: Item.TArray)
          RAISES {NetObj.Error, Thread.Alerted};
      itemsModified (it: Item.TArray; additive: BOOLEAN) 
          RAISES {NetObj.Error, Thread.Alerted};
      itemsDeleted (id: Item.IDArray) 
          RAISES {NetObj.Error, Thread.Alerted};
    END;

When a client interacts with the board, it must identify itself. The client may also want to query the information stored about him in the board. Indeed, for each client, the board remembers the focus, such that the client is only notified about modified objects within its focus. When a client registers with the board, a ClientInfo network object is returned for this purpose.


INTERFACE ClientInfo;

IMPORT RectR, NetObj, Thread,
       Callback;

TYPE  T = NetObj.T OBJECT 
  METHODS
    getScope (): RectR.T RAISES {NetObj.Error, Thread.Alerted};
    setScope (scope: RectR.T) RAISES {NetObj.Error, Thread.Alerted};
    getCallback (): Callback.T RAISES {NetObj.Error, Thread.Alerted};
  END;

The simple diagram editor becomes a client editor and must connect to a BoardServer which maintains the shared boards and sends notification to clients. The procedure previously called in the diagram editor to add an item CreateItems is renamed to ItemsCreated and used when a notification to add an item is received from the board server. When a new item is added locally in the diagram editor, the add command is first sent to the board server. The CreateItems procedure thus becomes as follows.


PROCEDURE CreateItems (v: T; its: Item.TArray) =
  VAR ids: Item.IDArray;
  BEGIN
    IF its = NIL THEN RETURN END;
    TRY
      (* board is a remote object in the board server. The new item is
         thus first added in the shared board stored in the remote server *)
      ids := v.board.createItems (v.ci, its);
      LOCK v.mu DO
        FOR i := FIRST (its^) TO LAST (its^) DO
          its[i].id := ids[i];
          EVAL v.display.put (ids[i], its[i]);
          its[i].paint (v, v.focus);
        END;
        VBT.Sync (v); 
      END;
    EXCEPT
      NetObj.Error (atom) => Error (v, Atom.ToText (atom.head)); 
    | Thread.Alerted => Error (v, "Thread.Alerted");
    END;
  END CreateItems;

The Callback network object registered in the board server is used to call the ItemsCreated and similar procedures.


INTERFACE CallbackX;

IMPORT View, Callback;

TYPE T <: Public;
     Public = Callback.T OBJECT
     METHODS
       init (v: View.T): T;
     END;

END CallbackX.


MODULE CallbackX;

IMPORT View, Item;

REVEAL T = Public BRANDED OBJECT
    v: View.T;
  OVERRIDES
    init := Init;
    itemsCreated := ItemsCreated;
    itemsModified := ItemsModified;
    itemsDeleted := ItemsDeleted;
  END;

PROCEDURE Init (cb: T; v: View.T): T =
  BEGIN
    cb.v := v;
    RETURN cb;
  END Init;

PROCEDURE ItemsCreated (cb: T; its: Item.TArray) =
  BEGIN 
    View.ItemsCreated (cb.v, its);
  END ItemsCreated;

PROCEDURE ItemsModified (cb: T; its: Item.TArray; additive: BOOLEAN) =
  BEGIN
    View.ItemsModified (cb.v, its, additive);
  END ItemsModified;

PROCEDURE ItemsDeleted (cb: T; ids: Item.IDArray) =
  BEGIN
    View.ItemsDeleted (cb.v, ids);
  END ItemsDeleted;

BEGIN
END CallbackX.

Each board in the board server receives editing commands from clients and must send notification to the other clients. A separate thread is used to send the notifications in order to return the control faster to the client and to avoid deadlocks.


INTERFACE BoardX;

IMPORT Board, Item, Wr, Rd, Pickle, Thread;

TYPE T <: Public;
     Public = Board.T OBJECT
     METHODS
       init(): T;
       initT();
       initS();
       createItemsState(its: Item.TArray): Item.IDArray;
       modifyItemsState(its: Item.TArray);
       deleteItemsState(ids: Item.IDArray);
     END;

PROCEDURE Busy (board: T): BOOLEAN;


MODULE BoardX;

IMPORT Thread, NetObj,
       Item, ItemList, ItemTbl, AtomicItemTbl, 
       TextItem, RuleItem, <* NOWARN *> (* must be included *)
       Callback, RectR,
       ClientInfo, ClientInfoX, ClientInfoList, AtomicClientList, 
       NotifyRec, NotifyQueue,
       Wr, Rd, Pickle;

REVEAL T = Public BRANDED OBJECT
    items: AtomicItemTbl.T;
    clients: AtomicClientList.T;
    nq: NotifyQueue.T;
    state: AtomicItemTbl.State;
  OVERRIDES
    init             := Init;
    initS            := InitStable;
    initT            := InitTransient;
    register         := Register;
    unregister       := Unregister;
    setScope         := SetScope;
    createItems      := CreateItems;
    createItemsState := CreateItemsState;
    modifyItems      := ModifyItems;
    modifyItemsState := ModifyItemsState;
    deleteItems      := DeleteItems;
    deleteItemsState := DeleteItemsState;
  END;

CONST ExpectedItems = 1000;

(* Initialise the board *)

PROCEDURE Init (bd: T) : T =
  BEGIN
    InitStable(bd);
    InitTransient(bd);
    RETURN bd;
  END Init;

PROCEDURE InitStable(bd: T) =
  BEGIN 
    bd.state := NEW (AtomicItemTbl.State, 
                     tbl := NEW (ItemTbl.Default).init (ExpectedItems),
                     id := 0);
    bd.items := NEW (AtomicItemTbl.T, state := bd.state)
  END InitStable;

(* A thread is forked to manage the notification of clients *)

PROCEDURE InitTransient(bd: T) =
  VAR nc := NEW (NotifyClosure, bd := bd);
  BEGIN
    bd.clients := NEW (AtomicClientList.T);
    bd.nq := NEW (NotifyQueue.T).init ();
    EVAL Thread.Fork (nc);
  END InitTransient;

(* Register a new client. Store its Callback network object and
   return a ClientInfo network object to the client *)

PROCEDURE Register (bd: T; cb: Callback.T): ClientInfo.T =
  VAR ci := NEW (ClientInfoX.T).init (cb);
  BEGIN
    bd.clients.add (ci);
    RETURN ci;
  END Register;

(* Remove a client from the list *)

PROCEDURE Unregister (bd: T; ci: ClientInfo.T) =
  BEGIN
    bd.clients.remove (ci);
  END Unregister;

(* Change the scope of a client *)

PROCEDURE SetScope (bd: T; ci: ClientInfo.T; scope: RectR.T) =
  VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Scope, doer := ci, 
                 newScope := scope);
  BEGIN
    bd.nq.enq (nr);
  END SetScope;

(* A new item is created, add it and queue a notification message *)

PROCEDURE CreateItems (bd: T; ci: ClientInfo.T; 
                       its: Item.TArray): Item.IDArray =
  VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Create, doer := ci,
                 its := its);
      ids: Item.IDArray;
  BEGIN
    IF its = NIL THEN RETURN NIL END;
    LOCK bd.items DO
      ids := bd.createItemsState(its);
    END;
    bd.nq.enq (nr);
    RETURN ids;
  END CreateItems;
    
PROCEDURE CreateItemsState (bd: T; its: Item.TArray): Item.IDArray =
  VAR ids: Item.IDArray;
  BEGIN
    ids := NEW (Item.IDArray, NUMBER (its^));
    FOR i := FIRST (its^) TO LAST (its^) DO
      INC (bd.items.state.id);
      ids[i] := bd.items.state.id;
      its[i].id := ids[i];
      EVAL bd.items.state.tbl.put (its[i].id, its[i]);
    END;
    RETURN ids;
  END CreateItemsState;
    
(* An item is modified, do it and queue a notification message *)

PROCEDURE ModifyItems (bd: T; ci: ClientInfo.T; 
                       its: Item.TArray; additive: BOOLEAN) =
  VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Modify, doer := ci, 
                 its := its, additive := additive);
  BEGIN
    IF its = NIL THEN RETURN END;
    LOCK bd.items DO
      bd.modifyItemsState(its);
    END;
    bd.nq.enq (nr);
  END ModifyItems; 
    
PROCEDURE ModifyItemsState (bd: T; its: Item.TArray) = 
  VAR old: Item.T;
  BEGIN
    FOR i := FIRST (its^) TO LAST (its^) DO
      IF its[i] = NIL OR NOT bd.items.state.tbl.get (its[i].id, old) THEN
        its[i] := NIL;
      ELSE
        EVAL bd.items.state.tbl.put (its[i].id, its[i]);
      END;
    END;
  END ModifyItemsState;
    
(* An item is deleted, do it and queue a notification message *)

PROCEDURE DeleteItems (bd: T; ci: ClientInfo.T; ids: Item.IDArray) =
  VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Delete, doer := ci, 
                 ids := ids);
  BEGIN
    IF ids = NIL THEN RETURN END;
    LOCK bd.items DO
      bd.deleteItemsState(ids);
    END;
    bd.nq.enq (nr);
  END DeleteItems; 

PROCEDURE DeleteItemsState (bd: T; ids: Item.IDArray) =
  VAR old: Item.T;
  BEGIN
    FOR i := FIRST (ids^) TO LAST (ids^) DO
      EVAL bd.items.state.tbl.delete (ids[i], old);
    END;
  END DeleteItemsState; 

(* The notification thread awaits notification messages and
   sends the notification to clients *)

TYPE NotifyClosure = Thread.Closure OBJECT
    bd: T;
    OVERRIDES 
      apply := NotifyLoop;
    END;
  
PROCEDURE NotifyLoop (nc: NotifyClosure): REFANY =
  VAR nr: NotifyRec.T;
  BEGIN
    LOOP
      nr := nc.bd.nq.deq ();
      Notify (nc.bd, nr);
    END;
  END NotifyLoop;

PROCEDURE Notify (bd: T; nr: NotifyRec.T) =
  BEGIN
      CASE nr.code OF

        (* If a message to a client does not get through (Netobj.Error),
           the client is simply unregistered and looses the connection *)

        (* The scope of a client was changed, send to this client only
           all the items in the new scope. When a client first connects,
           its scope (focus) is empty and later set to something else,
           it then automatically receives all the items in its scope. *)
        NotifyRec.Code.Scope =>
        LOCK bd.items DO
          VAR ir := bd.items.state.tbl.iterate ();
              id: Item.ID;
              it: Item.T;
              il: ItemList.T := NIL;
              oldScope := nr.doer.getScope(); <*NOWARN*>
          BEGIN
            WHILE ir.next (id, it) DO
              IF RectR.Overlap (it.box, nr.newScope)
               THEN
                il := ItemList.Cons (it, il);
              END;
            END;
            VAR its := NEW (Item.TArray, ItemList.Length (il));
                i := 0;
            BEGIN
              WHILE il # NIL DO
                its [i] := il.head;
                il := il.tail;
                INC (i);
              END;
              TRY
                nr.doer.getCallback().itemsCreated (its);
                nr.doer.setScope (nr.newScope);
              EXCEPT 
                NetObj.Error, Thread.Alerted => Unregister (bd, nr.doer);
              END;
            END;
          END;
        END;

      (* All clients except the one from which the item creation originated
         are notified *)
      | NotifyRec.Code.Create =>
        VAR deadClients: ClientInfoList.T := NIL; BEGIN
          LOCK bd.clients DO
            VAR clients := bd.clients.list; BEGIN
              WHILE clients # NIL DO
                IF clients.head # nr.doer THEN
                  TRY
                    clients.head.getCallback().itemsCreated (nr.its);
                  EXCEPT 
                    NetObj.Error, Thread.Alerted => 
                    deadClients := ClientInfoList.Cons (clients.head, 
                                                        deadClients);
                  END;
                END;
                clients := clients.tail;
              END;
            END;
          END;
          (* must remove deadClients after unlocking bd.clients *)
          WHILE deadClients # NIL DO
            Unregister (bd, deadClients.head);
            deadClients := deadClients.tail;
          END;
        END;

      (* All clients except the one from which the item modification 
         originated are notified *)
      | NotifyRec.Code.Modify =>
        VAR deadClients: ClientInfoList.T := NIL; BEGIN
          LOCK bd.clients DO
            VAR clients := bd.clients.list; BEGIN
              WHILE clients # NIL DO
                IF clients.head # nr.doer THEN
                  TRY
                    clients.head.getCallback().itemsModified (nr.its, 
                                                              nr.additive);
                  EXCEPT 
                    NetObj.Error, Thread.Alerted => 
                    deadClients := ClientInfoList.Cons (clients.head, 
                                                        deadClients);
                  END;
                END;
                clients := clients.tail;
              END;
            END;
          END;
          (* must remove deadClients after unlocking bd.clients *)
          WHILE deadClients # NIL DO
            Unregister (bd, deadClients.head);
            deadClients := deadClients.tail;
          END;
        END;

      (* All clients except the one from which the item deletion
         originated are notified *)
      | NotifyRec.Code.Delete =>
        VAR deadClients: ClientInfoList.T := NIL; BEGIN
          LOCK bd.clients DO
            VAR clients := bd.clients.list;  BEGIN
              WHILE clients # NIL DO
                IF clients.head # nr.doer THEN
                  TRY
                    clients.head.getCallback().itemsDeleted (nr.ids);
                  EXCEPT 
                    NetObj.Error, Thread.Alerted => 
                    deadClients := ClientInfoList.Cons (clients.head, 
                                                        deadClients);
                  END;
                END;
                clients := clients.tail;
              END;
            END;
          END;
          (* must remove deadClients after unlocking bd.clients *)
          WHILE deadClients # NIL DO
            Unregister (bd, deadClients.head);
            deadClients := deadClients.tail;
          END;
        END;
      END;
  END Notify;

(* Any client remaining? *)

PROCEDURE Busy (bd: T): BOOLEAN =
  BEGIN
    RETURN bd.clients.list # NIL;
  END Busy;

BEGIN
END BoardX.

The notification records simply store the originating client, the operation and the associated information.


INTERFACE NotifyRec;

IMPORT RectR,
       Item, ClientInfo;

TYPE T = REF RECORD
    code: Code;
    doer: ClientInfo.T;
    ids: Item.IDArray := NIL;
    its: Item.TArray := NIL;
    newScope := RectR.Empty;
    additive: BOOLEAN;
  END;

  Code = {Scope, Create, Modify, Delete};

The notify queue must store notification records and synchronize queing and dequeing operations. Indeed, several clients may send editing commands simultaneously and the network objects library may allocate several threads to handle these requests. Proper locking insures that only one thread at a time queues a notification record. Only the notification thread removes notification records from the queue but locking is also required to guard against the other threads that queue records.


INTERFACE NotifyQueue;

IMPORT NotifyRec;

TYPE T <: Public;
 
     Public = Private OBJECT METHODS
       init (): T;
       enq (nr: NotifyRec.T);
       deq (): NotifyRec.T;
     END;   

     Private <: ROOT;


MODULE NotifyQueue;

IMPORT Thread,
       NotifyRec, NotifyRecList;

REVEAL Private = MUTEX BRANDED "NotifyQueueP" OBJECT END;
       T = Public BRANDED "NotifyQueue" OBJECT
         list: NotifyRecList.T := NIL;
         nonEmpty: Thread.Condition;
       OVERRIDES
         init := Init;
         enq := Enq;
         deq := Deq;
       END;    

PROCEDURE Init (nq: T): T =
  BEGIN
    nq.nonEmpty := NEW (Thread.Condition);
    RETURN nq;
  END Init;

(* Lock before adding and signal that the queue is not empty any more *)

PROCEDURE Enq (nq: T; nr: NotifyRec.T) =
  BEGIN
    LOCK nq DO
      nq.list := NotifyRecList.AppendD (nq.list, NotifyRecList.List1 (nr));
      Thread.Signal (nq.nonEmpty);
    END;
  END Enq;

(* If the queue is empty, the notification thread has nothing better to
   do than wait. *)

PROCEDURE Deq (nq: T): NotifyRec.T =
  VAR nr: NotifyRec.T;
  BEGIN
    LOCK nq DO
      WHILE nq.list = NIL DO Thread.Wait (nq, nq.nonEmpty) END;
      nr := nq.list.head;
      nq.list := nq.list.tail;
      RETURN nr;
    END;
  END Deq;

BEGIN
END NotifyQueue.

The board server maintains a table of the boards currently in use. Being a network object, it is accessed by clients to connect to boards.


MODULE BoardServerX;

IMPORT Board, BoardServer, StableBoardXTbl,
       BoardX, StableBoardX, StableError;

REVEAL T = Public BRANDED OBJECT
    mu: MUTEX;
    boards: StableBoardXTbl.Default;
  OVERRIDES
    init := Init;
    create := Create;
    open := Open;
    save := Save;
    close := Close;
    remove := Remove;
  END;

CONST ExpectedBoards = 10;

PROCEDURE Init (bs: T): T =
  BEGIN
    bs.mu := NEW (MUTEX);
    bs.boards := NEW (StableBoardXTbl.Default).init (ExpectedBoards);
    RETURN bs;
  END Init;

PROCEDURE Create (bs: T; boardName: TEXT): Board.T 
        RAISES {BoardServer.Failed} =
  BEGIN
    RETURN Open(bs, boardName);
  END Create;

(* Find a board in the table of currently used boards or add it in
   the table if not there *)

PROCEDURE Open (bs: T; boardName: TEXT): Board.T 
        RAISES {BoardServer.Failed} =
  VAR board: StableBoardX.T;
      recovered: BOOLEAN;
  BEGIN
    TRY
      LOCK bs.mu DO
        IF bs.boards.get (boardName, board) THEN (* in-memory *)
          RETURN board;
        ELSE (* load board *)
          board := NEW (StableBoardX.T).init (boardName, recovered);
          IF (NOT recovered) THEN
            board.initS();
          END;
          board.initT();
          EVAL bs.boards.put (boardName, board);
          RETURN board;
        END;
      END;
    EXCEPT
      StableError.E => 
      RAISE BoardServer.Failed ("Could not open " & boardName);
    END;
  END Open; 

(* Write the board to disk *)

PROCEDURE Save (bs: T; boardName: TEXT) 
        RAISES {BoardServer.Failed} =
  VAR board: StableBoardX.T;
  BEGIN
    TRY
      LOCK bs.mu DO
        IF bs.boards.get (boardName, board) THEN
          StableBoardX.Checkpoint(board);
        ELSE
          RAISE BoardServer.Failed ("Board not loaded");
        END;
      END;
    EXCEPT
      StableError.E => 
      RAISE BoardServer.Failed ("Error saving board " & boardName);
    END;
  END Save;

(* If no client remains for the closed board, save it and remove it
   from the table *)

PROCEDURE Close (bs: T; boardName: TEXT) 
        RAISES {BoardServer.Failed} =
  VAR board: StableBoardX.T;
  BEGIN
    TRY
      LOCK bs.mu DO
        IF bs.boards.get (boardName, board) THEN
          IF NOT BoardX.Busy (board) THEN
            EVAL bs.boards.delete (boardName, board);
            StableBoardX.Checkpoint(board);
            BoardX.Quit (board);
          END;
        ELSE
          RAISE BoardServer.Failed ("Board not loaded");
        END;
      END;
    EXCEPT
      StableError.E => 
      RAISE BoardServer.Failed ("Error saving board " & boardName);
    END;
  END Close;

(* If no client remains for that board, remove it altogether *)

PROCEDURE Remove (bs: T; boardName: TEXT) 
    RAISES {BoardServer.Failed} =
  VAR board: StableBoardX.T;
  BEGIN
    TRY
      LOCK bs.mu DO
        IF bs.boards.get (boardName, board) THEN
          IF BoardX.Busy (board) THEN 
            RAISE BoardServer.Failed ("Board is busy")
          END;
          EVAL bs.boards.delete (boardName, board);
          board.dispose();
        END
      END
    EXCEPT
      StableError.E => 
      RAISE BoardServer.Failed ("Problems deleting " & boardName);
    END;
  END Remove; 

BEGIN
END BoardServerX.

The main program of the board server simply exports the BoardServer network object to the Network Object Daemon. The clients will obtain the BoardServer object from the daemon. The BoardServer object will then be used to obtain board objects.


MODULE Server EXPORTS Main;

IMPORT NetObj, Thread, Err, 
       BoardServerX;

VAR bs := NEW (BoardServerX.T).init ();
BEGIN
  TRY
    NetObj.Export ("BoardServer", bs);
  EXCEPT 
    NetObj.Error => Err.Print ("Please start netobjd and retry.\n");
  END;

  (* Do nothing, the action takes place when methods of the BoardServer
     object are invoked by remote clients. *)
  LOOP Thread.Pause (10.0d0); END;
END Server.

Persistent objects

In the diagram editor example shown in earlier sections, little was said about reading or writing boards (diagrams) from disk. Since information about object types is available at run time, procedures exist to read or write graphs of objects. These procedures are found in the Pickle library. Such procedures obviate the need for defining a file format and writing input and output procedures.


PROCEDURE PutState(bd: T; wr: Wr.T)
  RAISES {Pickle.Error, Wr.Failure, Thread.Alerted} = 
  BEGIN
    Pickle.Write(wr, bd);
  END PutState; 

PROCEDURE GetState(bd: T; rd: Rd.T): T
  RAISES {Pickle.Error, Rd.Failure, Rd.EndOfFile, Thread.Alerted} = 
  BEGIN
    bd := Pickle.Read(rd); 
    RETURN bd;
  END GetState;

In practice it is often needed to split the object state into a persistent portion, which should be saved to disk, and a transient portion containing mutexes, list or currently registered clients...

Stable persistent objects

Migrating objects between the disk and memory, using a library such as Pickle, is a first step towards offering object oriented services similar to traditional databases. The next step is insuring that all modifications to the persistent objects are automatically propagated to disk even if the program or computer suddenly stops. Recovering from disk failures is another problem usually dealt with through disk mirroring or frequent backups to tape archives.

The SmallDB and Stable libraries, along with the StableGen tool may be used to provide persistent objects with little efforts. To add persistence to an object type, its declaration is passed to StableGen which generates a new derived type with the following methods and procedures added:

Furthermore, all the methods which affect the object state are overridden to first write the method name and arguments to a log, then call the original method. Thus, when a persistent object is initialized after a crash, its state is restored from the last checkpoint and all the method calls stored in the log are redone to get back to the exact state prior to the crash.

The BoardX.i3 interface describes the methods for accessing boards. Pragmas are added to identify the type for which a stable derived type is wanted and the methods that modify the object state, and thus which need to be recorded in the change log.


INTERFACE BoardX;

IMPORT Board, Item, Wr, Rd, Pickle, Thread;

TYPE T <: Public;
     Public = Board.T OBJECT
     METHODS
       init(): T;
       initT();
       initS();
       createItemsState(its: Item.TArray): Item.IDArray;
       modifyItemsState(its: Item.TArray);
       deleteItemsState(ids: Item.IDArray);
     END;
     <*PRAGMA STABLE *>
     <*STABLE UPDATE METHODS initS, createItemsState, modifyItemsState, 
                             deleteItemsState *>

The new stable derived type is created by stablegen and is named StableBoardX.T. It is used to initialise board objects from their persistent state, if it exists, and to periodically write a checkpoint of the state of the board. The frequency at which checkpoints are written is a compromise between the time taken to write the checkpoint, and the space occupied by the log of changes since the last checkpoint, as well as the recovery time which is proportional to the log length. In this case, the checkpointing criterion is simple. Every time the user issues a save command or closes a board, a new checkpoint is written and the change log emptied.

When a board is opened or created by the board server, the following actions are sufficient to create a board and recover its persistent state, if it exists.


    (* The board name identifies the checkpoint and log files on disk *)
    board := NEW (StableBoardX.T).init (boardName, recovered);

    (* This is a newly created board, there is no state for it yet *)
    IF (NOT recovered) THEN
      board.initS();
    END;

    board.initT();

When the board is saved or closed, a new checkpoint is written as follows.


  TRY
    StableBoardX.Checkpoint(board);
  EXCEPT
    StableError.E => 
    RAISE BoardServer.Failed ("Error saving board " & boardName);
  END;

If a board should no longer be persistent, when the remove command is issued by the user, the dispose method is called and the checkpoint and log files associated with the board are deleted.


  TRY
    board.dispose();
  EXCEPT
    StableError.E => 
    RAISE BoardServer.Failed ("Problems deleting " & boardName);
  END;

Indexing

For small databases that fit into memory, indexes are easily built using the Table (hash table) and SortedTable (sorted tree) interfaces. In the diagram example, indexes for the X and Y upper left coordinates of the items could be implemented using sorted tables that map the coordinate (REAL) to the item identifier (CARDINAL). These indexes are added to the Board object declaration.


REVEAL T = Public BRANDED OBJECT
    items: AtomicItemTbl.T;
    indexX: RealItemIDSortedTable.T;
    indexY: RealItemIDSortedTable.T;

Then, each time new items are created in CreateItemsState, they must be entered in the indexes.


  (* insert the item into the board items table *)
  WITH item = its[i] DO
    EVAL bd.items.state.tbl.put (item.id, item);
    EVAL bd.indexX.put(item.box.west,item.id);
    EVAL bd.indexY.put(item.box.north,item.id);
  END;

Similarly, when existing items are modified, the indexes must be updated in ModifyItemsState.


  WITH item = its[i] DO
    (* Retrieve the existing item and remove it from the indexes *)
    EVAL bd.items.state.tbl.get (item.id, oldItem)
    EVAL bd.indexX.delete(oldItem.box.west,it);
    EVAL bd.indexY.delete(oldItem.box.north,it);
    (* Install the new, modified, item in the table and indexes *)
    EVAL bd.items.state.tbl.put (item.id, item);
    EVAL bd.indexX.put(item.box.west,item.id);
    EVAL bd.indexY.put(item.box.north,item.id);
  END;

Finally, when an item is deleted, the index entries are removed in DeleteItemsState.


  WITH item = its[i] DO
    (* Retrieve the existing item and remove it from the indexes *)
    EVAL bd.items.state.tbl.get (item.id, oldItem)
    EVAL bd.indexX.delete(oldItem.box.west,it);
    EVAL bd.indexY.delete(oldItem.box.north,it);
  END;

The Queries on these indexes would return the sorted sequence of items in a speficied coordinate range.


PROCEDURE FindItems(tbl: RealItemIDSortedTable.T; min, max: REAL):
    IDSequence.T =
  VAR
    seq := NEW(IDSequence.T).init();
    iterator := tbl.iterateOrdered();
    cond := TRUE;
    key: REAL;
    val: Item.ID;
  BEGIN
    cond := iterator.seek(min);
    LOOP
      IF iterator.next(key,val) THEN
        IF key <= max THEN seq.addhi(val);
        ELSE EXIT;
        END;
      ELSE EXIT;
      END;
    END;
    SortIDSequence(seq);
    RETURN seq;
  END FindItems;

The intersection and union of such sequences may then be used to construct more complex queries.


PROCEDURE Intersect(seq1, seq2: IDSequence.T): IDSequence.T =
  VAR
    outSeq: NEW(IDSequence.T).init();
    i1 := 0;
    i2 := 0;
    end1 := seq1.size();
    end2 := seq2.size();
  BEGIN
    WHILE i1 < end1 AND i2 < end2 DO
      WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO
        IF id1 < id2 THEN
          INC(i1);
        ELSIF id2 < id1 THEN
          INC(i2);
        ELSE
          outSeq.addhi(id1);
          INC(i1);
          INC(i2);
        END;
      END;
    END;
  END Intersect;
  
PROCEDURE Union(seq1, seq2: IDSequence.T): IDSequence.T =
  VAR
    outSeq: NEW(IDSequence.T).init();
    i1 := 0;
    i2 := 0;
    end1 := seq1.size();
    end2 := seq2.size();
  BEGIN
    WHILE i1 < end1 AND i2 < end2 DO
      WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO
        IF id1 < id2 THEN
          outSeq.addhi(id1);
          INC(i1);
        ELSIF id2 < id1
          outSeq.addhi(id2);
          INC(i2);
        ELSE
          outSeq.addhi(id1);
          INC(i1);
          INC(i2);
        END;
      END;
    END;

    WHILE i1 < end1 DO
      outSeq.addhi(seq1.get(i1));
      INC(i1);
    END;

    WHILE i2 < end2 DO
      outSeq.addhi(seq2.get(i2));
      INC(i2);
    END;
  END Union;

Query Processing

Database systems often offer a specialized query language. Queries are parsed and optimized in order to translate them into efficient low level database operations. For example, when looking for all items with an X coordinate between 0.1 and 0.11 and a Y coordinate between 0 and 1, at least three possible paths may lead to the desired result.

If the database contains 1000 items uniformly distributed in the square defined by X and Y between 0 and 1, there will be approximately 10 items with X between 0.10 and 0.11 and 1000 items with Y between 0 and 1. Thus, the first solution finds a list of 10 and a list of 1000 and performs the intersection. The third solution is no better since it finds a lists of 1000 and checks each item for its X coordinate. The second solution in this case is clearly superior since it obtains a list of 10 items through the X index and checks the Y coordinate of each.

Transactions

Traditional sophisticated databases include support for atomic transactions. A transaction may be initiated, then a sequence of updating commands are issued and finally the transaction is ended with a commit or abort command. This way, the sequence of updating commands becomes atomic, they are all performed together if the commit is successfull or none of them is performed if the database crashes, the commit is not successfull or abort is issued.

In stable persistent objects, methods are atomic (logged atomically) but there is no explicit support for transactions. The same effect however may be obtained in different ways. For instance, all updating actions which need to be performed together may be grouped in a single atomic method.

Another possibility is to keep an undo list. The undo list starts recording when the begin_transaction method is called and is emptied after a successfull commit. However, when recovering from a crash or when the transaction is aborted the undo list would be used to recover the state prior to the begin_transaction.

Optimized loading and unloading

In the SmallDB and Stable libraries, no special mechanism is required to fit all the database in memory. The database must fit in virtual memory (less than 100MB or so) and the usual operating system swapping mechanisms are used to control which pages remain in main memory and which pages are left on disk. The performance is excellent since all the database is already converted to the in memory format, most queries are handled directly in main memory, and a single disk write is needed for each database update.

For larger databases (hundreds of megabytes to tens of gigabytes), however, special implementation tricks must be used. The loading, caching and unloading of database pages, from disk files to virtual memory, is managed directly by the database server. Objects are laid out on pages in such a way that little or no translation is required when they are loaded or unloaded (no pointers are used). The placement of objects on pages is optimized to reduce the number of pages to load for typical queries. For example, indexes are organized as balanced trees where each tree node fits exactly on a page.

In many cases, the database server accesses directly the raw disk device to avoid going through the operating system input-output buffer cache and to optimize the layout on disk. The result is some performance increase for very large databases at a huge cost in implementation complexity and non portability.


Copyright 1995 Michel Dagenais, dagenais@vlsi.polymtl.ca, Wed Mar 8 14:41:03 EST 1995