The Sharedboard diagram editor examined in earlier sections uses distributed and persistent objects. Network objects allow several users to simultanously edit diagrams while persistent objects insure that the up to date diagrams are always available on persistent storage even if the diagram server crashes.
A distributed version of the diagram editor would allow several users to simultaneously see and interact with the same diagrams. This may be useful to complement a phone conversation with graphical explanations or to insure that everyone sees immediately any modifications to a diagram, to avoid the problems associated with reconciling the local changes made by several users when they take a local copy.
With network objects, it is possible to call the methods of objects which are in another process, possibly on a remote server. The stubgen tool and netobj library take care of relaying the methods and arguments between programs.
A board server maintains a table of currently opened boards. When a user requests a board already in use, he obtains a reference to that board. Thus, the board will receive editing commands from several users and must notify the other users.
INTERFACE BoardServer; IMPORT NetObj, Thread, Board; TYPE T = NetObj.T OBJECT METHODS create (boardName: TEXT): Board.T RAISES {Failed, NetObj.Error, Thread.Alerted}; open (boardName: TEXT): Board.T RAISES {Failed, NetObj.Error, Thread.Alerted}; save (boardName: TEXT) RAISES {Failed, NetObj.Error, Thread.Alerted}; close (boardName: TEXT) RAISES {Failed, NetObj.Error, Thread.Alerted}; remove (boardName: TEXT) RAISES {Failed, NetObj.Error, Thread.Alerted}; END; EXCEPTION Failed (TEXT);
Once a board is obtained from the server, editing commands must be sent to it.
INTERFACE Board; IMPORT NetObj, Thread, Callback, Item, ClientInfo, RectR; TYPE T = NetObj.T OBJECT METHODS register (cb: Callback.T): ClientInfo.T RAISES {NetObj.Error, Thread.Alerted}; setScope (ci: ClientInfo.T; scope: RectR.T) RAISES {NetObj.Error, Thread.Alerted}; createItems (ci: ClientInfo.T; its: Item.TArray): Item.IDArray RAISES {NetObj.Error, Thread.Alerted}; modifyItems (ci: ClientInfo.T; its: Item.TArray; additive: BOOLEAN) RAISES {NetObj.Error, Thread.Alerted}; deleteItems (ci: ClientInfo.T; ids: Item.IDArray) RAISES {NetObj.Error, Thread.Alerted}; unregister (ci: ClientInfo.T) RAISES {NetObj.Error, Thread.Alerted}; END;
Before editing a board, a new client must register with the board by providing a Callback object used to interact with the client. This Callback object will be used by the board to notify this client when other clients modify the board contents. The Callback object too is a network object.
INTERFACE Callback; IMPORT NetObj, Thread, Item; TYPE T = NetObj.T OBJECT METHODS itemsCreated (it: Item.TArray) RAISES {NetObj.Error, Thread.Alerted}; itemsModified (it: Item.TArray; additive: BOOLEAN) RAISES {NetObj.Error, Thread.Alerted}; itemsDeleted (id: Item.IDArray) RAISES {NetObj.Error, Thread.Alerted}; END;
When a client interacts with the board, it must identify itself. The client may also want to query the information stored about him in the board. Indeed, for each client, the board remembers the focus, such that the client is only notified about modified objects within its focus. When a client registers with the board, a ClientInfo network object is returned for this purpose.
INTERFACE ClientInfo; IMPORT RectR, NetObj, Thread, Callback; TYPE T = NetObj.T OBJECT METHODS getScope (): RectR.T RAISES {NetObj.Error, Thread.Alerted}; setScope (scope: RectR.T) RAISES {NetObj.Error, Thread.Alerted}; getCallback (): Callback.T RAISES {NetObj.Error, Thread.Alerted}; END;
The simple diagram editor becomes a client editor and must connect to a BoardServer which maintains the shared boards and sends notification to clients. The procedure previously called in the diagram editor to add an item CreateItems is renamed to ItemsCreated and used when a notification to add an item is received from the board server. When a new item is added locally in the diagram editor, the add command is first sent to the board server. The CreateItems procedure thus becomes as follows.
PROCEDURE CreateItems (v: T; its: Item.TArray) = VAR ids: Item.IDArray; BEGIN IF its = NIL THEN RETURN END; TRY (* board is a remote object in the board server. The new item is thus first added in the shared board stored in the remote server *) ids := v.board.createItems (v.ci, its); LOCK v.mu DO FOR i := FIRST (its^) TO LAST (its^) DO its[i].id := ids[i]; EVAL v.display.put (ids[i], its[i]); its[i].paint (v, v.focus); END; VBT.Sync (v); END; EXCEPT NetObj.Error (atom) => Error (v, Atom.ToText (atom.head)); | Thread.Alerted => Error (v, "Thread.Alerted"); END; END CreateItems;
The Callback network object registered in the board server is used to call the ItemsCreated and similar procedures.
INTERFACE CallbackX; IMPORT View, Callback; TYPE T <: Public; Public = Callback.T OBJECT METHODS init (v: View.T): T; END; END CallbackX.
MODULE CallbackX; IMPORT View, Item; REVEAL T = Public BRANDED OBJECT v: View.T; OVERRIDES init := Init; itemsCreated := ItemsCreated; itemsModified := ItemsModified; itemsDeleted := ItemsDeleted; END; PROCEDURE Init (cb: T; v: View.T): T = BEGIN cb.v := v; RETURN cb; END Init; PROCEDURE ItemsCreated (cb: T; its: Item.TArray) = BEGIN View.ItemsCreated (cb.v, its); END ItemsCreated; PROCEDURE ItemsModified (cb: T; its: Item.TArray; additive: BOOLEAN) = BEGIN View.ItemsModified (cb.v, its, additive); END ItemsModified; PROCEDURE ItemsDeleted (cb: T; ids: Item.IDArray) = BEGIN View.ItemsDeleted (cb.v, ids); END ItemsDeleted; BEGIN END CallbackX.
Each board in the board server receives editing commands from clients and must send notification to the other clients. A separate thread is used to send the notifications in order to return the control faster to the client and to avoid deadlocks.
INTERFACE BoardX; IMPORT Board, Item, Wr, Rd, Pickle, Thread; TYPE T <: Public; Public = Board.T OBJECT METHODS init(): T; initT(); initS(); createItemsState(its: Item.TArray): Item.IDArray; modifyItemsState(its: Item.TArray); deleteItemsState(ids: Item.IDArray); END; PROCEDURE Busy (board: T): BOOLEAN;
MODULE BoardX; IMPORT Thread, NetObj, Item, ItemList, ItemTbl, AtomicItemTbl, TextItem, RuleItem, <* NOWARN *> (* must be included *) Callback, RectR, ClientInfo, ClientInfoX, ClientInfoList, AtomicClientList, NotifyRec, NotifyQueue, Wr, Rd, Pickle; REVEAL T = Public BRANDED OBJECT items: AtomicItemTbl.T; clients: AtomicClientList.T; nq: NotifyQueue.T; state: AtomicItemTbl.State; OVERRIDES init := Init; initS := InitStable; initT := InitTransient; register := Register; unregister := Unregister; setScope := SetScope; createItems := CreateItems; createItemsState := CreateItemsState; modifyItems := ModifyItems; modifyItemsState := ModifyItemsState; deleteItems := DeleteItems; deleteItemsState := DeleteItemsState; END; CONST ExpectedItems = 1000; (* Initialise the board *) PROCEDURE Init (bd: T) : T = BEGIN InitStable(bd); InitTransient(bd); RETURN bd; END Init; PROCEDURE InitStable(bd: T) = BEGIN bd.state := NEW (AtomicItemTbl.State, tbl := NEW (ItemTbl.Default).init (ExpectedItems), id := 0); bd.items := NEW (AtomicItemTbl.T, state := bd.state) END InitStable; (* A thread is forked to manage the notification of clients *) PROCEDURE InitTransient(bd: T) = VAR nc := NEW (NotifyClosure, bd := bd); BEGIN bd.clients := NEW (AtomicClientList.T); bd.nq := NEW (NotifyQueue.T).init (); EVAL Thread.Fork (nc); END InitTransient; (* Register a new client. Store its Callback network object and return a ClientInfo network object to the client *) PROCEDURE Register (bd: T; cb: Callback.T): ClientInfo.T = VAR ci := NEW (ClientInfoX.T).init (cb); BEGIN bd.clients.add (ci); RETURN ci; END Register; (* Remove a client from the list *) PROCEDURE Unregister (bd: T; ci: ClientInfo.T) = BEGIN bd.clients.remove (ci); END Unregister; (* Change the scope of a client *) PROCEDURE SetScope (bd: T; ci: ClientInfo.T; scope: RectR.T) = VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Scope, doer := ci, newScope := scope); BEGIN bd.nq.enq (nr); END SetScope; (* A new item is created, add it and queue a notification message *) PROCEDURE CreateItems (bd: T; ci: ClientInfo.T; its: Item.TArray): Item.IDArray = VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Create, doer := ci, its := its); ids: Item.IDArray; BEGIN IF its = NIL THEN RETURN NIL END; LOCK bd.items DO ids := bd.createItemsState(its); END; bd.nq.enq (nr); RETURN ids; END CreateItems; PROCEDURE CreateItemsState (bd: T; its: Item.TArray): Item.IDArray = VAR ids: Item.IDArray; BEGIN ids := NEW (Item.IDArray, NUMBER (its^)); FOR i := FIRST (its^) TO LAST (its^) DO INC (bd.items.state.id); ids[i] := bd.items.state.id; its[i].id := ids[i]; EVAL bd.items.state.tbl.put (its[i].id, its[i]); END; RETURN ids; END CreateItemsState; (* An item is modified, do it and queue a notification message *) PROCEDURE ModifyItems (bd: T; ci: ClientInfo.T; its: Item.TArray; additive: BOOLEAN) = VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Modify, doer := ci, its := its, additive := additive); BEGIN IF its = NIL THEN RETURN END; LOCK bd.items DO bd.modifyItemsState(its); END; bd.nq.enq (nr); END ModifyItems; PROCEDURE ModifyItemsState (bd: T; its: Item.TArray) = VAR old: Item.T; BEGIN FOR i := FIRST (its^) TO LAST (its^) DO IF its[i] = NIL OR NOT bd.items.state.tbl.get (its[i].id, old) THEN its[i] := NIL; ELSE EVAL bd.items.state.tbl.put (its[i].id, its[i]); END; END; END ModifyItemsState; (* An item is deleted, do it and queue a notification message *) PROCEDURE DeleteItems (bd: T; ci: ClientInfo.T; ids: Item.IDArray) = VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Delete, doer := ci, ids := ids); BEGIN IF ids = NIL THEN RETURN END; LOCK bd.items DO bd.deleteItemsState(ids); END; bd.nq.enq (nr); END DeleteItems; PROCEDURE DeleteItemsState (bd: T; ids: Item.IDArray) = VAR old: Item.T; BEGIN FOR i := FIRST (ids^) TO LAST (ids^) DO EVAL bd.items.state.tbl.delete (ids[i], old); END; END DeleteItemsState; (* The notification thread awaits notification messages and sends the notification to clients *) TYPE NotifyClosure = Thread.Closure OBJECT bd: T; OVERRIDES apply := NotifyLoop; END; PROCEDURE NotifyLoop (nc: NotifyClosure): REFANY = VAR nr: NotifyRec.T; BEGIN LOOP nr := nc.bd.nq.deq (); Notify (nc.bd, nr); END; END NotifyLoop; PROCEDURE Notify (bd: T; nr: NotifyRec.T) = BEGIN CASE nr.code OF (* If a message to a client does not get through (Netobj.Error), the client is simply unregistered and looses the connection *) (* The scope of a client was changed, send to this client only all the items in the new scope. When a client first connects, its scope (focus) is empty and later set to something else, it then automatically receives all the items in its scope. *) NotifyRec.Code.Scope => LOCK bd.items DO VAR ir := bd.items.state.tbl.iterate (); id: Item.ID; it: Item.T; il: ItemList.T := NIL; oldScope := nr.doer.getScope(); <*NOWARN*> BEGIN WHILE ir.next (id, it) DO IF RectR.Overlap (it.box, nr.newScope) THEN il := ItemList.Cons (it, il); END; END; VAR its := NEW (Item.TArray, ItemList.Length (il)); i := 0; BEGIN WHILE il # NIL DO its [i] := il.head; il := il.tail; INC (i); END; TRY nr.doer.getCallback().itemsCreated (its); nr.doer.setScope (nr.newScope); EXCEPT NetObj.Error, Thread.Alerted => Unregister (bd, nr.doer); END; END; END; END; (* All clients except the one from which the item creation originated are notified *) | NotifyRec.Code.Create => VAR deadClients: ClientInfoList.T := NIL; BEGIN LOCK bd.clients DO VAR clients := bd.clients.list; BEGIN WHILE clients # NIL DO IF clients.head # nr.doer THEN TRY clients.head.getCallback().itemsCreated (nr.its); EXCEPT NetObj.Error, Thread.Alerted => deadClients := ClientInfoList.Cons (clients.head, deadClients); END; END; clients := clients.tail; END; END; END; (* must remove deadClients after unlocking bd.clients *) WHILE deadClients # NIL DO Unregister (bd, deadClients.head); deadClients := deadClients.tail; END; END; (* All clients except the one from which the item modification originated are notified *) | NotifyRec.Code.Modify => VAR deadClients: ClientInfoList.T := NIL; BEGIN LOCK bd.clients DO VAR clients := bd.clients.list; BEGIN WHILE clients # NIL DO IF clients.head # nr.doer THEN TRY clients.head.getCallback().itemsModified (nr.its, nr.additive); EXCEPT NetObj.Error, Thread.Alerted => deadClients := ClientInfoList.Cons (clients.head, deadClients); END; END; clients := clients.tail; END; END; END; (* must remove deadClients after unlocking bd.clients *) WHILE deadClients # NIL DO Unregister (bd, deadClients.head); deadClients := deadClients.tail; END; END; (* All clients except the one from which the item deletion originated are notified *) | NotifyRec.Code.Delete => VAR deadClients: ClientInfoList.T := NIL; BEGIN LOCK bd.clients DO VAR clients := bd.clients.list; BEGIN WHILE clients # NIL DO IF clients.head # nr.doer THEN TRY clients.head.getCallback().itemsDeleted (nr.ids); EXCEPT NetObj.Error, Thread.Alerted => deadClients := ClientInfoList.Cons (clients.head, deadClients); END; END; clients := clients.tail; END; END; END; (* must remove deadClients after unlocking bd.clients *) WHILE deadClients # NIL DO Unregister (bd, deadClients.head); deadClients := deadClients.tail; END; END; END; END Notify; (* Any client remaining? *) PROCEDURE Busy (bd: T): BOOLEAN = BEGIN RETURN bd.clients.list # NIL; END Busy; BEGIN END BoardX.
The notification records simply store the originating client, the operation and the associated information.
INTERFACE NotifyRec; IMPORT RectR, Item, ClientInfo; TYPE T = REF RECORD code: Code; doer: ClientInfo.T; ids: Item.IDArray := NIL; its: Item.TArray := NIL; newScope := RectR.Empty; additive: BOOLEAN; END; Code = {Scope, Create, Modify, Delete};
The notify queue must store notification records and synchronize queing and dequeing operations. Indeed, several clients may send editing commands simultaneously and the network objects library may allocate several threads to handle these requests. Proper locking insures that only one thread at a time queues a notification record. Only the notification thread removes notification records from the queue but locking is also required to guard against the other threads that queue records.
INTERFACE NotifyQueue; IMPORT NotifyRec; TYPE T <: Public; Public = Private OBJECT METHODS init (): T; enq (nr: NotifyRec.T); deq (): NotifyRec.T; END; Private <: ROOT;
MODULE NotifyQueue; IMPORT Thread, NotifyRec, NotifyRecList; REVEAL Private = MUTEX BRANDED "NotifyQueueP" OBJECT END; T = Public BRANDED "NotifyQueue" OBJECT list: NotifyRecList.T := NIL; nonEmpty: Thread.Condition; OVERRIDES init := Init; enq := Enq; deq := Deq; END; PROCEDURE Init (nq: T): T = BEGIN nq.nonEmpty := NEW (Thread.Condition); RETURN nq; END Init; (* Lock before adding and signal that the queue is not empty any more *) PROCEDURE Enq (nq: T; nr: NotifyRec.T) = BEGIN LOCK nq DO nq.list := NotifyRecList.AppendD (nq.list, NotifyRecList.List1 (nr)); Thread.Signal (nq.nonEmpty); END; END Enq; (* If the queue is empty, the notification thread has nothing better to do than wait. *) PROCEDURE Deq (nq: T): NotifyRec.T = VAR nr: NotifyRec.T; BEGIN LOCK nq DO WHILE nq.list = NIL DO Thread.Wait (nq, nq.nonEmpty) END; nr := nq.list.head; nq.list := nq.list.tail; RETURN nr; END; END Deq; BEGIN END NotifyQueue.
The board server maintains a table of the boards currently in use. Being a network object, it is accessed by clients to connect to boards.
MODULE BoardServerX; IMPORT Board, BoardServer, StableBoardXTbl, BoardX, StableBoardX, StableError; REVEAL T = Public BRANDED OBJECT mu: MUTEX; boards: StableBoardXTbl.Default; OVERRIDES init := Init; create := Create; open := Open; save := Save; close := Close; remove := Remove; END; CONST ExpectedBoards = 10; PROCEDURE Init (bs: T): T = BEGIN bs.mu := NEW (MUTEX); bs.boards := NEW (StableBoardXTbl.Default).init (ExpectedBoards); RETURN bs; END Init; PROCEDURE Create (bs: T; boardName: TEXT): Board.T RAISES {BoardServer.Failed} = BEGIN RETURN Open(bs, boardName); END Create; (* Find a board in the table of currently used boards or add it in the table if not there *) PROCEDURE Open (bs: T; boardName: TEXT): Board.T RAISES {BoardServer.Failed} = VAR board: StableBoardX.T; recovered: BOOLEAN; BEGIN TRY LOCK bs.mu DO IF bs.boards.get (boardName, board) THEN (* in-memory *) RETURN board; ELSE (* load board *) board := NEW (StableBoardX.T).init (boardName, recovered); IF (NOT recovered) THEN board.initS(); END; board.initT(); EVAL bs.boards.put (boardName, board); RETURN board; END; END; EXCEPT StableError.E => RAISE BoardServer.Failed ("Could not open " & boardName); END; END Open; (* Write the board to disk *) PROCEDURE Save (bs: T; boardName: TEXT) RAISES {BoardServer.Failed} = VAR board: StableBoardX.T; BEGIN TRY LOCK bs.mu DO IF bs.boards.get (boardName, board) THEN StableBoardX.Checkpoint(board); ELSE RAISE BoardServer.Failed ("Board not loaded"); END; END; EXCEPT StableError.E => RAISE BoardServer.Failed ("Error saving board " & boardName); END; END Save; (* If no client remains for the closed board, save it and remove it from the table *) PROCEDURE Close (bs: T; boardName: TEXT) RAISES {BoardServer.Failed} = VAR board: StableBoardX.T; BEGIN TRY LOCK bs.mu DO IF bs.boards.get (boardName, board) THEN IF NOT BoardX.Busy (board) THEN EVAL bs.boards.delete (boardName, board); StableBoardX.Checkpoint(board); BoardX.Quit (board); END; ELSE RAISE BoardServer.Failed ("Board not loaded"); END; END; EXCEPT StableError.E => RAISE BoardServer.Failed ("Error saving board " & boardName); END; END Close; (* If no client remains for that board, remove it altogether *) PROCEDURE Remove (bs: T; boardName: TEXT) RAISES {BoardServer.Failed} = VAR board: StableBoardX.T; BEGIN TRY LOCK bs.mu DO IF bs.boards.get (boardName, board) THEN IF BoardX.Busy (board) THEN RAISE BoardServer.Failed ("Board is busy") END; EVAL bs.boards.delete (boardName, board); board.dispose(); END END EXCEPT StableError.E => RAISE BoardServer.Failed ("Problems deleting " & boardName); END; END Remove; BEGIN END BoardServerX.
The main program of the board server simply exports the BoardServer network object to the Network Object Daemon. The clients will obtain the BoardServer object from the daemon. The BoardServer object will then be used to obtain board objects.
MODULE Server EXPORTS Main; IMPORT NetObj, Thread, Err, BoardServerX; VAR bs := NEW (BoardServerX.T).init (); BEGIN TRY NetObj.Export ("BoardServer", bs); EXCEPT NetObj.Error => Err.Print ("Please start netobjd and retry.\n"); END; (* Do nothing, the action takes place when methods of the BoardServer object are invoked by remote clients. *) LOOP Thread.Pause (10.0d0); END; END Server.
In the diagram editor example shown in earlier sections, little was said about reading or writing boards (diagrams) from disk. Since information about object types is available at run time, procedures exist to read or write graphs of objects. These procedures are found in the Pickle library. Such procedures obviate the need for defining a file format and writing input and output procedures.
PROCEDURE PutState(bd: T; wr: Wr.T) RAISES {Pickle.Error, Wr.Failure, Thread.Alerted} = BEGIN Pickle.Write(wr, bd); END PutState; PROCEDURE GetState(bd: T; rd: Rd.T): T RAISES {Pickle.Error, Rd.Failure, Rd.EndOfFile, Thread.Alerted} = BEGIN bd := Pickle.Read(rd); RETURN bd; END GetState;
In practice it is often needed to split the object state into a persistent portion, which should be saved to disk, and a transient portion containing mutexes, list or currently registered clients...
Migrating objects between the disk and memory, using a library such as Pickle, is a first step towards offering object oriented services similar to traditional databases. The next step is insuring that all modifications to the persistent objects are automatically propagated to disk even if the program or computer suddenly stops. Recovering from disk failures is another problem usually dealt with through disk mirroring or frequent backups to tape archives.
The SmallDB and Stable libraries, along with the StableGen tool may be used to provide persistent objects with little efforts. To add persistence to an object type, its declaration is passed to StableGen which generates a new derived type with the following methods and procedures added:
Furthermore, all the methods which affect the object state are overridden to first write the method name and arguments to a log, then call the original method. Thus, when a persistent object is initialized after a crash, its state is restored from the last checkpoint and all the method calls stored in the log are redone to get back to the exact state prior to the crash.
The BoardX.i3 interface describes the methods for accessing boards. Pragmas are added to identify the type for which a stable derived type is wanted and the methods that modify the object state, and thus which need to be recorded in the change log.
INTERFACE BoardX; IMPORT Board, Item, Wr, Rd, Pickle, Thread; TYPE T <: Public; Public = Board.T OBJECT METHODS init(): T; initT(); initS(); createItemsState(its: Item.TArray): Item.IDArray; modifyItemsState(its: Item.TArray); deleteItemsState(ids: Item.IDArray); END; <*PRAGMA STABLE *> <*STABLE UPDATE METHODS initS, createItemsState, modifyItemsState, deleteItemsState *>
The new stable derived type is created by stablegen and is named StableBoardX.T. It is used to initialise board objects from their persistent state, if it exists, and to periodically write a checkpoint of the state of the board. The frequency at which checkpoints are written is a compromise between the time taken to write the checkpoint, and the space occupied by the log of changes since the last checkpoint, as well as the recovery time which is proportional to the log length. In this case, the checkpointing criterion is simple. Every time the user issues a save command or closes a board, a new checkpoint is written and the change log emptied.
When a board is opened or created by the board server, the following actions are sufficient to create a board and recover its persistent state, if it exists.
(* The board name identifies the checkpoint and log files on disk *) board := NEW (StableBoardX.T).init (boardName, recovered); (* This is a newly created board, there is no state for it yet *) IF (NOT recovered) THEN board.initS(); END; board.initT();
When the board is saved or closed, a new checkpoint is written as follows.
TRY StableBoardX.Checkpoint(board); EXCEPT StableError.E => RAISE BoardServer.Failed ("Error saving board " & boardName); END;
If a board should no longer be persistent, when the remove command is issued by the user, the dispose method is called and the checkpoint and log files associated with the board are deleted.
TRY board.dispose(); EXCEPT StableError.E => RAISE BoardServer.Failed ("Problems deleting " & boardName); END;
For small databases that fit into memory, indexes are easily built using the Table (hash table) and SortedTable (sorted tree) interfaces. In the diagram example, indexes for the X and Y upper left coordinates of the items could be implemented using sorted tables that map the coordinate (REAL) to the item identifier (CARDINAL). These indexes are added to the Board object declaration.
REVEAL T = Public BRANDED OBJECT items: AtomicItemTbl.T; indexX: RealItemIDSortedTable.T; indexY: RealItemIDSortedTable.T;
Then, each time new items are created in CreateItemsState, they must be entered in the indexes.
(* insert the item into the board items table *) WITH item = its[i] DO EVAL bd.items.state.tbl.put (item.id, item); EVAL bd.indexX.put(item.box.west,item.id); EVAL bd.indexY.put(item.box.north,item.id); END;
Similarly, when existing items are modified, the indexes must be updated in ModifyItemsState.
WITH item = its[i] DO (* Retrieve the existing item and remove it from the indexes *) EVAL bd.items.state.tbl.get (item.id, oldItem) EVAL bd.indexX.delete(oldItem.box.west,it); EVAL bd.indexY.delete(oldItem.box.north,it); (* Install the new, modified, item in the table and indexes *) EVAL bd.items.state.tbl.put (item.id, item); EVAL bd.indexX.put(item.box.west,item.id); EVAL bd.indexY.put(item.box.north,item.id); END;
Finally, when an item is deleted, the index entries are removed in DeleteItemsState.
WITH item = its[i] DO (* Retrieve the existing item and remove it from the indexes *) EVAL bd.items.state.tbl.get (item.id, oldItem) EVAL bd.indexX.delete(oldItem.box.west,it); EVAL bd.indexY.delete(oldItem.box.north,it); END;
The Queries on these indexes would return the sorted sequence of items in a speficied coordinate range.
PROCEDURE FindItems(tbl: RealItemIDSortedTable.T; min, max: REAL): IDSequence.T = VAR seq := NEW(IDSequence.T).init(); iterator := tbl.iterateOrdered(); cond := TRUE; key: REAL; val: Item.ID; BEGIN cond := iterator.seek(min); LOOP IF iterator.next(key,val) THEN IF key <= max THEN seq.addhi(val); ELSE EXIT; END; ELSE EXIT; END; END; SortIDSequence(seq); RETURN seq; END FindItems;
The intersection and union of such sequences may then be used to construct more complex queries.
PROCEDURE Intersect(seq1, seq2: IDSequence.T): IDSequence.T = VAR outSeq: NEW(IDSequence.T).init(); i1 := 0; i2 := 0; end1 := seq1.size(); end2 := seq2.size(); BEGIN WHILE i1 < end1 AND i2 < end2 DO WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO IF id1 < id2 THEN INC(i1); ELSIF id2 < id1 THEN INC(i2); ELSE outSeq.addhi(id1); INC(i1); INC(i2); END; END; END; END Intersect; PROCEDURE Union(seq1, seq2: IDSequence.T): IDSequence.T = VAR outSeq: NEW(IDSequence.T).init(); i1 := 0; i2 := 0; end1 := seq1.size(); end2 := seq2.size(); BEGIN WHILE i1 < end1 AND i2 < end2 DO WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO IF id1 < id2 THEN outSeq.addhi(id1); INC(i1); ELSIF id2 < id1 outSeq.addhi(id2); INC(i2); ELSE outSeq.addhi(id1); INC(i1); INC(i2); END; END; END; WHILE i1 < end1 DO outSeq.addhi(seq1.get(i1)); INC(i1); END; WHILE i2 < end2 DO outSeq.addhi(seq2.get(i2)); INC(i2); END; END Union;
Database systems often offer a specialized query language. Queries are parsed and optimized in order to translate them into efficient low level database operations. For example, when looking for all items with an X coordinate between 0.1 and 0.11 and a Y coordinate between 0 and 1, at least three possible paths may lead to the desired result.
If the database contains 1000 items uniformly distributed in the square defined by X and Y between 0 and 1, there will be approximately 10 items with X between 0.10 and 0.11 and 1000 items with Y between 0 and 1. Thus, the first solution finds a list of 10 and a list of 1000 and performs the intersection. The third solution is no better since it finds a lists of 1000 and checks each item for its X coordinate. The second solution in this case is clearly superior since it obtains a list of 10 items through the X index and checks the Y coordinate of each.
Traditional sophisticated databases include support for atomic transactions. A transaction may be initiated, then a sequence of updating commands are issued and finally the transaction is ended with a commit or abort command. This way, the sequence of updating commands becomes atomic, they are all performed together if the commit is successfull or none of them is performed if the database crashes, the commit is not successfull or abort is issued.
In stable persistent objects, methods are atomic (logged atomically) but there is no explicit support for transactions. The same effect however may be obtained in different ways. For instance, all updating actions which need to be performed together may be grouped in a single atomic method.
Another possibility is to keep an undo list. The undo list starts recording when the begin_transaction method is called and is emptied after a successfull commit. However, when recovering from a crash or when the transaction is aborted the undo list would be used to recover the state prior to the begin_transaction.
In the SmallDB and Stable libraries, no special mechanism is required to fit all the database in memory. The database must fit in virtual memory (less than 100MB or so) and the usual operating system swapping mechanisms are used to control which pages remain in main memory and which pages are left on disk. The performance is excellent since all the database is already converted to the in memory format, most queries are handled directly in main memory, and a single disk write is needed for each database update.
For larger databases (hundreds of megabytes to tens of gigabytes), however, special implementation tricks must be used. The loading, caching and unloading of database pages, from disk files to virtual memory, is managed directly by the database server. Objects are laid out on pages in such a way that little or no translation is required when they are loaded or unloaded (no pointers are used). The placement of objects on pages is optimized to reduce the number of pages to load for typical queries. For example, indexes are organized as balanced trees where each tree node fits exactly on a page.
In many cases, the database server accesses directly the raw disk device to avoid going through the operating system input-output buffer cache and to optimize the layout on disk. The result is some performance increase for very large databases at a huge cost in implementation complexity and non portability.