Package extension to Tcl by Cimarron Taylor Miro Systems, Inc. cimarron@miro.com 1. Introduction --------------- This directory contains the sources and documentation for the package extension to Tcl, an embeddable tool command language developed by John Ousterhout. The information here corresponds to package release 0.1 and requires Tcl release 7.0. 2. Audience ----------- This extension is intended for people experimenting with object and scope extensions to Tcl and Tcl gurus in general. It is expected that this extension should be useful to those building higher level object extensions to Tcl such as [incr tcl] and theObjects. 3. Background ------------- Tcl variable scoping rules permit a variable to be in the global scope or in the local scope of a proc invocation. Tcl procedure scoping rules define all procedures to be in a single global scope. In the following program #! /usr/local/bin/tclsh set x 10 proc foo {y} { set z [expr $y+1] } the variable x is in the global scope, and the variables y and z are local to the procedure foo. The procedure foo is in the global scope. Variable accesses from procedures are assumed to be local. The following program will fail, since there is no variable x in the scope of bad: #! /usr/local/bin/tclsh set x 10 proc bad {} { echo $x } To overcome this limitation, the Tcl "global" command may be used: #! /usr/local/bin/tclsh set x 10 proc ok {} { global x echo $x } 4. The Package Extension ------------------------ The package extension described here is designed to make development of complex Tcl applications easier as well as provide a framework for experimentation and development of more sophisticated Tcl enhancements, such as Tcl object and class systems, high level persistant data management, Tcl code browsers, etc. Packages represent collections of related variables and procedures. Packages extend Tcl variable and procedure scoping rules by providing a place for variables other than the single global scope and the current procedure scope; and also by providing a place for procedures to be defined other than the global scope. The syntax of several Tcl procedures has been extended to handle package and variable operations, and two new Tcl commands have been added: the "package" command and the "with" command. The "package" command provides a way to create, modify and delete new places for variables and procedures. The following example demonstrates one way in which a package is created and how variables in a package may be accessed: #! /usr/local/bin/tclsh package create #erewhon package set #erewhon a_ 10 b_ 20 echo "in #erewhon, a_ and b_ are [package get #erewhon a_ b_]" Note: package names must begin with a leading hash (#) character. A useful convention (from InterViews) to improve readability is that all package variables should have a trailing underscore (_). All of the examples presented here follow this convention. In the following program #! /usr/local/bin/tclsh set x 10 proc foo {y} { set z [expr $y+1] } package create #erewhon set #erewhon j_ 10 set #erewhon k_(1) 20 with #erewhon { proc baz {} {echo "j_ is $j_, k_(1) is $k_(1)"} } the variables "j_" and "k_" and the procedure "baz" are in the scope of the package named "#erewhon". "j_" is a scalar variable and "k_" is an array variable. Note also the extended syntax for the "set" command. Given the above definitions, the commands echo $j_ and baz will fail. The echo command fails because the variable "j_" is not in the global scope and invoking "baz" fails because it is not in the global scope. The "with" command provides a way to define and call procedures in a specified package. The "with" command functions the same way the "eval" command does, except that it modifies the current procedure scope. For example, to cause the "baz" command defined within the "#erewhon" package to be invoked, the "with" command is used. with #erewhon baz will produce j_ is 10, k_(1) is 20 Because "baz" is defined in the same package as the variables "j_" and "k_", it is able to access them. Note that the "with" command only provides a way to call procedures, not access variables. For instance, with #erewhon {echo $j_} will fail, because "with" only affects the procedure scope. 5. Basic Package Comands ------------------------ package create #shape creates new package called #shape package delete #shape deletes package #shape package exists #shape does package #shape exist? package get #shape var ?var..? return list of values of variables from #shape package list return list of name of all defined packages package set #shape var value ?var value ...? set values of variables in package #shape package procs #shape return names of all procedures in #shape package vars #shape return names of all variables in #shape with #shape ... invoke command with current package set to #shape 6. How it works --------------- The package extension accomplishes most of its functionality via a small set of macros. Most of the changes to Tcl simply replace the direct manipulation of iPtr->commandTable with one of the macros. For instance, in Tcl_Eval, the line hPtr = Tcl_FindHashEntry(&iPtr->commandTable, argv[0]); was changed to Tcl_GetCurrentProc(iPtr, argv[0], hPtr); A glance at the package0.1.patch file should give an idea of the nature of the changes. 7. Tcl Changes -------------- Packages extend the Tcl interpreter structure (Interp), the Tcl call frame structure (CallFrame) and Tcl procedure structure (Proc) to support the following semantics: 1. The interpreter maintains a notion of the "current" package. Whenever a procedure lookup is performed, the interpreter first checks the "current" package (if one exists) followed by the global command table. The "with" command temporarily substitutes the interpreter's current package with the specifed package, similar to the way "uplevel" substitutes the current call frame while executing its body. 2. Each Tcl procedure knows what package it belongs to. When a Tcl procedure is invoked, three things are done before the procedure's arguments are instanciated in the new call frame: * the interperter's current package is set to the procedure's package. * the name of the package is assigned to a variable "this". * variables from the package are upvar'ed into the procedure's call frame so that within the body of a procedure package variables are visible (note: a special syntax for "proc" allows visibility of only a subset of package variables). 3. Procedures created via "proc" are created in the interpreter's current package, if one exists, or in the global command table. C commands installed with Tcl_CreateCommand always go into the global command table by default. To install C commands in a package, the programmer is expected to use the TclSetPackage() macro. This is the mechanism used by the "proc" command. 4. By default, variable lookup is not affected by the interpreter's current package. Instead, as described above, procedure invocation causes upvar references to be installed into the newly created call frame. However, a new flag, TCL_USE_PACKAGE, is available to the Tcl_GetVar, Tcl_SetVar and related C routines. When TCL_USE_PACKAGE is specified, the variable table for the current package is used in place of the local or global variable tables. This is necessary to allow the Tcl "set", "unset" and related commands themselves to access variables in a package. 8. Tcl command changes ---------------------- The following Tcl commands now take an option ?#package? first argument: append, array, lappend, proc, set, unset, upvar The following Tcl commands now take new command options info, proc 9. Advanced Package Comands (not implemented in package 0.1) ---------------- These are described in the NOTES.PACKAGE file. These will be implemented in a future release of the package extension. package find findspec return packages matching findspec package linkvar #shape ?#pack? dv sv make dv in #shape a link to the variable sv package copy ?options? #shape #shape1 create shape1 ss a copy of shape package shared #shape z all packages copied from #shape see the same value of z package comm #shape #s2 procs|vars return differences between two packages package diff #shape #s2 procs|vars ?kvar? return list of variable values which differ package tokeylist #shape kvar put keylist representation of package in kvar package fromkeylist #shape kvar create package from a keylist representation package patch #shape kvar applies keylist previously returned by diff package sentinal #shape mgrproc ?mgrpack? specify a package trace function 10. Documentation ---------------- Unfortunately, there is not much documentation on using the package extension other than this file and the NOTES.PACKAGE files. I have not yet updated the man pages in the doc/ directory to reflect the package extension. Some examples of how the package routines work are in the tests/package.test file. 11. Compiling and installing the package extension -------------------------------------------------- This release has only been tested on a Sun ELC running SunOS 4.1.3. To install it, you will need Larry Wall's patch program and a tcl7.0 directory containing the Tcl 7.0 distribution. (0) If you do not already have a Tcl 7.0 distribution, obtain one and unpack it to create a tcl7.0 directory but do not compile Tcl yet. (1) Place the package0.1.patch file in the same directory which has the tcl7.0 subdirectory and type patch -p