OpenInteract Developer's Guide

Introduction

First, have you read the Illustrated Manager's Guide to OpenInteract and the OpenInteract Administrator's Guide?

We will take the following approach to explaining how things work in OpenInteract: first, we give you some overview information (you have read the stuff referred to in last paragraph, have you?), then we will explain things in a little more detail, and finally, along the way, we will show you how to build a simple package that displays data from a database (following tradition, this will be the famous fruit example database (table fruit, fields id, name, taste, price), so things don't get too complicated.

Packages

A package is all the code, SQL structures, initial data, configuration information and anything else necessary to implement functionality in OpenInteract.

In OpenInteract, packages implement the actual application functionality. OpenInteract handles the storage interface (e.g., putting your objects in a database), dispatches URL requests to your objects (this is called handling an action), security, authentication and authorization, and session management.

Applications need to define objects, which is how an application handles its state. It also needs to define how the objects are to be manipulated, which users can access them and how functionality is exposed to the user (by way of the URL-to-action mapping).

What goes into a package? In general, you will find:

(If you are experiencing problems with the terms, take a look at the glossary we provided. Even if you don't have problems with the terms we use, it might be a good idea to do so now.)

Example: The Start

Obviously, OpenInteract comes with tools to install, uninstall and query currently installed packages. This greatly simplifies the task of creating, testing and distributing your application.

We will now set up a skeleton package (we don't want to do all this from scratch):


$ oi_manage --base_dir /tmp/OpenInteract/ --package fruit create_skeleton
Creating package skeleton in current directory.
=========================

Package fruit          : ok

=========================
Finished with create_skeleton!

Let's see what it did for us:


$ find fruit/
fruit/
fruit/conf
fruit/conf/spops.perl
fruit/conf/action.perl
fruit/data
fruit/doc
fruit/doc/fruit.pod
fruit/doc/titles
fruit/struct
fruit/template
fruit/template/dummy.meta
fruit/template/dummy.tmpl
fruit/script
fruit/html
fruit/html/images
fruit/OpenInteract
fruit/OpenInteract/Handler
fruit/OpenInteract/SQLInstall
fruit/OpenInteract/SQLInstall/Fruit.pm
fruit/MANIFEST.SKIP
fruit/package.conf
fruit/Changes
fruit/MANIFEST

Now, what does all this stuff mean to you?

 
fruit                      # Main directory
fruit/conf                 # Configuration directory
fruit/conf/spops.perl      # Persistent object(s) configuration
fruit/conf/action.perl     # Action(s) configuration
fruit/data                 # Package data/security directory
fruit/doc                  # Documentation directory
fruit/doc/fruit.pod        # Starter documentation
fruit/doc/titles           # Map documentation name to subject
fruit/struct               # Package table definition directory
fruit/template             # Template directory
fruit/template/dummy.meta  # Starter template meta file
fruit/template/dummy.tmpl  # Starter template file
fruit/script               # Tools program directory
fruit/html                 # Static html directory
fruit/html/images          # Image directory
fruit/OpenInteract         # Object hierarchy directory
fruit/OpenInteract/Handler # Package handler directory
fruit/OpenInteract/SQLInstall # SQL installation handler directory
fruit/OpenInteract/SQLInstall/Fruit.pm # Skeleton for SQL setup handler
fruit/MANIFEST.SKIP        # Regexes to skip when creating MANIFEST
fruit/package.conf         # Basic package configuration (name, author, ...)
fruit/Changes              # Changelog of your package
fruit/MANIFEST             # List of files in package

You will normally need to edit/add the following:


fruit/package.conf         # Add name, version, author information
fruit/MANIFEST             # Add names of distribution files
fruit/Changes              # Let people know about what you've improved/fixed
fruit/conf/spops.perl      # Describe the objects your package uses
fruit/conf/action.perl     # Map URLs to handlers in your package
fruit/data                 # Specify the initial data and security
fruit/struct               # Describe the tables used to store your objects
fruit/template             # HTML to display and manipulate your objects
fruit/OpenInteract         # Optional Perl modules defining object behavior
fruit/OpenInteract/Handler # Manipulate objects for desired functionality
fruit/doc/mypackage.pod    # Last but not least, tell the world about it

By the way, the MANIFEST file can be created automatically. (Perl is great.) Here's how:

 
$ cd /path/to/mypackage
$ perl -MExtUtils::Manifest -e 'ExtUtils::Manifest::mkmanifest()'

That's it! If you have an old 'MANIFEST' file in the directory it will be copied to 'MANIFEST.bak'. Also note that files matching patterns in the 'MANIFEST.SKIP' file will not be included.

Now we'll step through what makes up a package and along the way edit the necessary files to make our own package.

How to organize your development process

A very important point in organizing your development process for working with OpenInteract is to establish discipline about where you will work -- that is, in which directory you should edit files in the iterative process of getting an application or tool to work.

For all your work, it is recommended that you edit the files in the directory created for you by oi_manage, not in the website directory, and not in the base directory of your OpenInteract installation.

In fact, editing files in your website directory or in the base directory is nearly guaranteed to give you headaches. Don't do that. The only exception to this is quick-and-dirty debugging of handler code (inserting print statements and the like).

The reason for this is that the process of installing a package to the base directory and then applying or upgrading it to a website uses the information you provide in your files from the package directory (the one created by oi_manage for you) to do all sort of smart things, and in that process the original files are changed, and some even "disappear" (e.g., code classes). (If you are interested, look at some diff output for the directories in question.)

So, save yourself the trouble and only work from the package directory, never from anywhere else. It is trivial to write a little script that automates the install/apply/upgrade/server restart cycle for your site, and you will never have to worry again.

If you work with designers (people working only on templates), they will probably work in the website database, not in the filesystem. Be sure to coordinate updates from the filesystem via "oi_manage install_template" with their changes. It is good practice to run a small job to call "oi_manage dump_template" to "rescue" any database-only changes in the templates during the install process for a package during development.

Package composition

Now that you've created a package already, you've seen most of its contents. (The ones you care about, anyway.) However, each package is also related to the OpenInteract::Package class and, once installed, can be referenced by a OpenInteract::PackageRepository object.

The PackageRepository object represents a registry of installed packages in a central location. OpenInteract maintains such a repository for the base installation as well as for each individual website. (To find the file, go to the conf/ directory of your base OpenInteract installation. The package_repository.perl holds the information.)

This registry includes meta information about all currently installed packages -- author, install date, version, etc. You can browse the information using a command-line tool (named oi_manage) to see what is currently installed, along with querying the information to find dependencies, authors, etc. (In times of emergency, you can also open up the file directly for editing since it's a simple Data::Dumper-serialized Perl data structure. But this is strongly discouraged otherwise.

package.conf: General package information

Now, we need to edit our package information. We does this to identify our package to OpenInteract.


$ cd fruit/
$ cat package.conf
name           fruit
version        0.01
author         Who AmI 
url            http://www.whereami.com/
sql_installer  OpenInteract::SQLInstall::Fruit
description
Description of your package goes here.

[ ... editing it ... ]

$ cat package.conf
name           fruit
version        0.01
author         wizards@openinteract.org
url            http://www.openinteract.org/
sql_installer  OpenInteract::SQLInstall::Fruit
description 
Demo package for showing how to build an OpenInteract package.
Traditional style, fruit flavor.

The only non-obvious thing here is the sql_installer stuff - it is the name of a class that contains code that tells the oi_manage tool how to setup the database structure and the initial data for your package. When you run the command create_skeleton, oi_manage creates a simple one for you to which you'll have to add names of SQL files and installer data files as needed.

conf/spops.perl: Describe package objects

Now we need to define the objects that we want to use to show our fruit data to the world. We do this by editing the spops.perl file. This file is in fact just input for a template processing mechanism somewhere deep in the OpenInteract framework that will generate Perl code for your object.


$ cat conf/spops.perl
# This is a sample spops.perl file. Its purpose is to define the
# objects that will be used in your package. Each of the keys is
# commented below.

# If you do not plan on defining any objects for your package, then
# you can skip this discussion and leave the file as-is.

# Note that you must edit this file by hand -- there is no web-based
# interface for editing a package's spops.perl (or other)
# configuration files.

# You can have any number of entries in this file, although they
# should all be members of the single hashref (any name is ok) in the
# file.

# The syntax for this file is checked when you do a 'check_package'
# with the 'oi_manage' tool -- this is a good idea to do.

# Finally, you can retrieve this information (some in a slightly
# different format) at anytime by doing:
#
#   my $hashref = $object_class->CONFIG;
# or
#   my $hashref = $R->object-alias->CONFIG;

# For more information about the SPOPS configuration process, see
# 'perldoc SPOPS::Configure' and 'perldoc SPOPS::Configure::DBI'

$spops = {

# 'object-alias' - Defines how you can refer to the object class
# within OpenInteract. For portability and a host of other reasons, OI
# sets up aliases for the SPOPS object classes so you can refer to
# them from $R. For instance, if you are in an application 'MyApp':
#
#  my $user_class = $R->user;
#  print ">> User Class: <<$user_class>
#
#  Output: '>> User Class: <<MyApp::User>>'
#
# This way, your application can do:
#
#  my $object = $R->myobjectalias->fetch( $object_id );
#
# and not care about the different application namespaces and such.
#
# Note that the 'alias' key allows you to setup additional aliases for
# this object class.

#            'object-alias' => {

# class - Defines the class this object will be known by. This
# should be set to your application's namespace (e.g. 'MyApp::User'
# instead of 'OpenInteract::User')

#              class        => 'OpenInteract::User',

# code_clas' - Perl module from which we read subroutines into the
# namespace of this class. This should *ALWAYS* be
# 'OpenInteract::MyClass'.

#              code_class   => 'OpenInteract::User',

# isa - Define the parents of this class. Every class should have at
# least 'OpenInteract::SPOPS' and some sort of SPOPS implementation,
# usually 'SPOPS::DBI'

#              isa          => [ qw/ OpenInteract::SPOPS  SPOPS::Secure  SPOPS::DBI::MySQL  SPOPS::DBI / ],

# field - List of fields/properties of this object

#              field        => [ qw/ user_id first_name last_name email language
#                                    title last_login num_logins login_name password 
#                                    removal_date notes theme_id / ],

# id_field - Name of primary key field

#              id_field     => 'user_id',

# no_insert - Fields for which we should not try to insert
# information, ever. If you're using a SPOPS implementation (e.g.,
# 'SPOPS::DBI::MySQL') which generates primary key values for you, be
# sure to put your 'id_field' value here.

#              no_insert    => [ qw/ user_id / ],

# no_update - Fields we should never update

#              no_update    => [ qw/ user_id / ],

# skip_undef - Values for these fields will not be inserted/updated at
# all if the value within the object is undefined. This, along with
# 'sql_defaults', allows you to specify default values. 

#              skip_undef   => [ qw/ last_login num_logins password language removal_date theme_id / ],

# sql_defaults - List fields for which a default is defined. Note that
# SPOPS::DBI will re-fetch the object after first creating it if you
# have fields listed here to ensure that the object always reflects
# what's in the database.

#              sql_defaults => [ qw/ language theme_id / ],

# base_table - Name of the table we store the object tinformation
# in. Note that if you have 'db_owner' defined in your application's
# 'server.perl' file (in the 'db_info' key), then SPOPS will prepend
# that (along with a period) to the table name here. For instance, if
# the db_owner is defined to 'dbo', we would use the table name
# 'dbo.sys_user'

#              base_table   => 'sys_user',

# alias - Additional aliases to use for referring to this object class

#              alias        => [],

# has_a - Define a 'has-a' relationship between objects from this
# class and any number of other objects. Each key in the hashref is an
# object class (which gets translated to your app's class when you
# apply the package to an application) and the value is an arrayref of
# field names. The field name determines the name of the routine
# created: if the field name matches up with the 'id_field' of that
# class, then we create a subroutine named for the object's
# 'object-alias' field. If the field name does not match, we append
# '_{object_alias}' to the end of the field. (See 'perldoc
# SPOPS::Configre' for more.)

#              has_a        => { 'OpenInteract::Theme' => [ 'theme_id' ], },

# links_to - Define a 'links-to' relationship between objects from
# this class and any number of other objects. This may be modified
# soon -- see 'perldoc SPOPS::Configure::DBI' for more.

#              links_to     => { 'OpenInteract::Group' => 'sys_group_user' },

# creation_security - Determine the security to apply to newly created
# objects from this class. (See 'SPOPS::Secure')

#              creation_security => {
#                 u   => 'WRITE',
#                 g   => { 3 => 'WRITE' },
#                 w   => 'READ',
#              },

# track - Which actions should we log? True value logs action, false
# value does not.

#              track => {
#                 create => 0, update => 1, remove => 1
#              },

# display - Allow the object to be able to generate a URL to display itself.

#              display => { url => '/User/show/', class => 'OpenInteract::Handler::User', method => 'show' },

# name - Either a field name or a coderef (first and only arg =
# object) to generate a name for a particular object.

#              name => sub { return $_[0]->full_name },

# object_name - Name of this class of objects

#              object_name => 'User',
#            },

};

Wow, that was quite a mouthful. Do we really need all this stuff? Yes. But it's actually easy to modify. And it has a lot of comments, as you might have noticed. Just follow the online instructions. BTW, you are expected to edit this stuff, and to uncomment all the lines that do apply to you.

Here's the final file with all the comments removed:


$ cat conf/spops.perl
$spops = {
            'fruit' => {
              class        => 'OpenInteract::Fruit',
              isa          => [ qw/ OpenInteract::SPOPS  SPOPS::Secure  
                                    SPOPS::DBI::MySQL  SPOPS::DBI / ],
              field        => [ qw/ id name taste price / ], 
              id_field     => 'id',

              no_update    => [ qw/ id / ],
              skip_undef   => [ qw/ name taste price / ],
              base_table   => 'fruit',
              creation_security => {
                 u   => 'WRITE',
                 g   => { 3 => 'WRITE' },
                 w   => 'READ',
              },
              display => { url => '/Fruit/show/' },
              name => 'name', 
              object_name => 'fruit',
            },
};

struct/ and data/: Describe SQL structures and data

Having defined our Fruit object, we will specify the table structure in the database that is used to store our Fruit objects. Additionally, we will specify some initial data to go into those tables.

All SQL data structures go into the struct/ directory, and you should have only one table per file.


[ ... creating the new file struct/create-table-fruit.sql ... ]

$ cat struct/create-table-fruit.sql
create table fruit (
    id %%INCREMENT%%,
    name varchar(255),
    taste varchar(255),
    price varchar(255),
    primary key (id)
)

That funny %%INCREMENT%% thing in there is an OpenInteract feature: it tells OpenInteract to automatically generate an id for every object we insert into the table, making for a database-independent "autoincrement id" command.

Now, some data. All data goes into the (you guessed it!) data/ directory in your package.


[ ... editing data/fruit-initial-data.perl ... ]

$ cat data/fruit-initial-data.perl
$data = [
    { 
        spops_class => 'OpenInteract::Fruit',
        field_order => [ qw( name taste price ) ],
    }, 
    [ "Apple", "slightly sour", "1.00" ],
    [ "Banana", "sweet", "1.00" ],
    [ "Cherry", "red", "0.05" ],
    [ "Tomato", "yes, this is a fruit - look it up in the dictionary", "1.00" ],
];

Yes, this is actually a Perl data structure, and not SQL. It's more database-independent that way.

We have now to create a special class that will do the setup for our data in the database. We just edit the template provided for us.

After editing, our %files data structure looks like:


my %files = (
 tables   => [ 'create-table-fruit.sql' ],
 data     => [ 'fruit-initial-data.perl' ],
 security => [],
);

Since we're just in development, we won't actually install these structures and data. We'll do that below once we're done editing templates and other information.

template/: display your objects

Now we have a nice little table in our database (at least in theory), and objects that will be able to fetch and store themselves there. But we still need to show this to the world. That's where templates come in.

Basically, a template is just a file of text that contains special directives that will be processed by a template processor (the Template Toolkit in this case). Our package will define a handler that gets the objects from the database, and hand their attribute values over to the template processor. The template processor in turn will replace the template directives in the file with the corresponding object attribute values. You will usually have more than one template.

Some template basics (you should take the time to read the OpenInteract template documentation - for more advanced uses of templates, you might want to have a look at the documentation for the Template Toolkit, available on your command line with perldoc Template or at the website -- see the Suggested Readings below).

 <p>Welcome back 
   <font color="red">[% login.full_name %]</font>!</p>

When run through the template processing engine with a normal user object in the 'login' key, this will result in:

 <p>Welcome back 
   <font color="red">Charlie Brown</font>!</p>

So the information between the '[%' and '%]' symbols ('login.full_name') was replaced by other text which was dependent on the user who was viewing the page. If another user viewed the page, she might have seen:

 <p>Welcome back 
   <font color="red">Peppermint Patty</font>!</p>

OpenInteract provides a number of information and behaviors for you in every template you write. However, you can also provide your templates access to query results from the various data stores that SPOPS provides.

Now, let's start building our own:


[ ... editing again ... ]

$ cat template/fruit-display.meta
name:    fruit-display
title:   Fruit Display
package: Fruit
Displays our modest fruit assembly.

$ cat template/fruit-display.tmpl
<h1>Fruits in store</h1>
<p>This is what is in store:

<table>
<td>Item</td>
<td>Description</td>
<td>Price (in local units)</td>
[% FOREACH fruit = fruits_in_store %]
<tr>
<td>[% fruit.name %]</td>
<td>[% fruit.taste %]</td>
<td>[% fruit.price %]</td>
</tr>
[% END %]
</table>

<p>Thank you for looking at our modest fruit assortment!
<p>The Management

OpenInteract/Handler/: Manipulate objects for desired functionality

Now, the fun part! We will write a handler that fetches our objects from the database, puts the data into our template, and returns the resulting HTML for handing it off to the browser.


[ ... editing again ... ]

$ cat OpenInteract/Handler/Fruit.pm
package OpenInteract::Handler::Fruit;

use strict;

$OpenInteract::Handler::Fruit::VERSION = '0.01';
$OpenInteract::Handler::Fruit::author = 'lemburg@aixonix.de';

sub handler {
  my ( $class, $p ) = @_;
  my $R = OpenInteract::Request->instance;
  my $fruits = $R->fruit->fetch_group( { 'order' => 'name' } ); 
  my $params = { fruits_in_store => $fruits };
  return $R->template->handler( {}, $params, 
                                { db      => 'fruit-display', 
                                  package => 'fruit' } );
}

1;

conf/action.perl: Map URLs to handlers in your package

Well, after having written this handler - how will it get called by the user?

This is what the action.perl file is for. Remember, the action table maps URLs to handlers. And the action table is formed by all the information in the action.perl files in a website.

So, we simply put in a URL (we just make one up - that's the nice thing with dynamic web pages), and map it to our handler routine. Then, we the user types in the URL, or clicks on a link that contains the URL, our handler will magically get called, returns its HTML, and the user can have a look at our Fruit objects.


$action = {
   'fruit' => {
     class   => 'OpenInteract::Handler::Fruit',
     method  => 'handler',
   },
};

doc/mypackage.pod - Last but not least, tell the world about it

Documentation of the Fruit package is left as an exercise for the reader. Go read perldoc perlpod. A starter documentation file is available in doc/package.pod when you run oi_manage to create a package skeleton. (Aren't we nice?)

MANIFEST - Add names of distribution files

Now we need to install our package. And for that, we need to package it up. So, which files do we need in there? Well, a first approximation would be just all the files in the current directory and its subdirectories. This can be done automatically. (Perl is great.) Here's how:

 
$ cd fruit
$ perl -MExtUtils::Manifest -e 'ExtUtils::Manifest::mkmanifest()'

That's it! If you have an old 'MANIFEST' file in the directory it will be copied to 'MANIFEST.bak'. Also note that files matching patterns in the 'MANIFEST.SKIP' file will not be included. Cool.

Installing our package

(See also the Administrator's Guide to OpenInteract.)

This is the sequence of commands that will install our package (together with the output):

Preparing the package for distribution:



$ perl -MExtUtils::Manifest -e 'ExtUtils::Manifest::mkmanifest()'
Added to MANIFEST: Changes
Added to MANIFEST: MANIFEST
Added to MANIFEST: OpenInteract/Handler/Fruit.pm
Added to MANIFEST: OpenInteract/SQLInstall/Fruit.pm
Added to MANIFEST: conf/action.perl
Added to MANIFEST: conf/spops.perl
Added to MANIFEST: data/fruit-initial-data.perl
Added to MANIFEST: doc/fruit.pod
Added to MANIFEST: doc/titles
Added to MANIFEST: package.conf
Added to MANIFEST: struct/create-table-fruit.sql
Added to MANIFEST: template/fruit-display.meta
Added to MANIFEST: template/fruit-display.tmpl

$ oi_manage check_package
Running check_package...
=========================

Status of the packages you requested to be checked:
fruit  OK: 
-- File (conf/action.perl) ok
-- File (conf/spops.perl) ok
-- File (OpenInteract/SQLInstall/Fruit.pm) ok
-- File (OpenInteract/Handler/Fruit.pm) ok
-- package.conf: ok
-- MANIFEST files exist: ok
-- MANIFEST extra file check: ok


=========================
Finished with check_package!

$ oi_manage export_package
Running export_package...
=========================

Package fruit: checking MANIFEST for discrepancies
Looks good

Package fruit: checking filesystem for files not in MANIFEST
Looks good



Status of the packages you requested to be exported:
fruit
  OK: Version 0.01 distribution to /home/oi_wizards/work/OpenInteract/doc/fruit/fruit-0.01.tar.gz

=========================
Finished export_package!

Installing the package into the local OpenInteract package registry:


$ oi_manage install_package --base_dir /tmp/OpenInteract \
                            --package_file fruit-0.01.tar.gz
Running install_package...
=========================

Installed package: fruit-0.01

=========================
Finished installing package!

Applying the package to our website:


$ oi_manage apply_package --base_dir /tmp/OpenInteract \
                          --website_dir /home/oi_wizards/work/myOI \
                          --package fruit
Running apply_package...
=========================

Status of the packages you requested to be applied:
fruit (0.01)
  OK

=========================
Finished applying package!

Setting up tables and data:


$ oi_manage --base_dir /tmp/OpenInteract --website_dir /home/oi_wizards/work/myOI --package fruit test_db
Running test_db...
=========================

Status of the database test:
OK: 
 -- Basic connect: ok
 -- Basic use database: ok

=========================
Finished test_db!

$ oi_manage --base_dir /tmp/OpenInteract --website_dir /home/oi_wizards/work/myOI --package fruit install_sql
Running install_sql...
=========================

Status of the packages requested for SQL install:
fruit (0.01)
  OK: 
Status: create_structure for mysql
  Name: create-table-fruit.sql -- ok


Status: install_data for mysql
  Name: fruit-initial-data.perl -- ok


Status: install_security for mysql


=========================
Finished install_sql!

Put the templates into the template database of our website:


$ oi_manage --base_dir /tmp/OpenInteract --website_dir /home/oi_wizards/work/myOI --package fruit install_template
Running install_template...
=========================

Status of the templates you requested for installation:
fruit
  OK: 
fruit-display being created as new... ok! Template saved and security set ok.


=========================
Finished install_template!

The package is installed now. After restarting the Apache server, typing in the URL we provided above ('fruit' from the action.perl we edited above) shows our modest fruit assembly. See how easy it is to prepare dynamic web pages with OpenInteract.

The next steps - Suggested Readings

For any real-world application, you will obviously have to do better than our little fruit example. But you are a developer. We will not insult your intelligence by providing you with hundreds of pages of little modifications to our fruit example. You now need to dive into the details. Here's how to.

Read more documentation (see the online system documentation in your installation - you can get it by clicking on the "System Documentation" tab in the Admin Tools section once you get OpenInteract running. If you don't, see the podfiles in the various packages.)

Study how other people did it:

Experiment! (Some things will just work ... hopefully!)

Good luck ... we hope you will have fun! And remember the old LISP motto (by Alan Perlis):

Invent and fit - have fits and reinvent!