"Why is it that we entertain the belief that for every purpose odd numbers are the most effectual?" - Pliny the Elder.
Karl Glazebrook, AAO, 4/10/1996. [kgb@aaoepp.aao.gov.au]
$a=$b+$c can
add two 2048x2048 images in only a fraction of a second.
It is hoped to eventually provide tons of useful functionality for scientific and numeric analysis.
$x,
which can hold numbers or strings, lists or arrays of scalars, e.g. @x,
and associative arrays/hashes of scalars, e.g. %x.
perl v5 introduces to perl data structures and objects. A simple
scalar variable $x now be a user-defined data type or full blown
object*.
The fundamental idea behind perlDL is to allow $x to hold a whole 1D
spectrum, or a 2D image, a 3D data cube, and so on up to large
N-dimensional data sets. These can be manipulated all at once, e.g.
$a = $b + 2 does a vector operation on each value in the
spectrum/image/etc.
You may well ask: "Why not just store a spectrum as a simple perl @x
style list with each pixel being a list item?" The two key answers to
this are MEMORY and SPEED. Because we know our spectrum consists of
pure numbers we can compactly store them in a single block of memory
corresponding to a C style numeric array. This takes up a LOT less
memory than the equivalent perl list. It is then easy to pass this
block of memory to a fast addition routine, or to any other C function
which deals with arrays. As a result perlDL is very fast --- for example
one can mulitiply a 2048*2048 image in exactly the same time as it
would take in C or FORTRAN (0.1 sec on my SPARC). A further advantage
of this is that for simple operations (e.g. $x += 2) one can manipulate
the whole array without caring about its dimensionality.
I find when using perlDL it is most useful to think of standard perl
@x variables as ``lists'' of generic ``things'' and PDL variables like
$x as ``arrays'' which can be contained in lists or hashes. Quite
often in my perlDL scripts I have @x contain a list of spectra, or a
list of images (or even a mix!). Or perhaps one could have a hash
(e.g. %x) of images... the only limit is memory!
perlDL variables support a range of data types - arrays can be bytes, short intgers (signed or unsigned), long integers, floats or double precision floats.
* It actually holds a reference (a smart ``pointer'') to this but that is not relevant for ordinary use of perlDL.
The default is to import all the function names from a module. If you only want certain names imported just say:
Also see below on ``Object-Orientation''.
The pdl() function is used to initialise a PDL variable from a scalar,
list, list reference or another PDL variable.
In addition all PDL functions automatically convert normal perl scalars to PDL variables on-the-fly.
(also see ``Type Conversion'' and ``Input/Output'' sections below)
(and other perl operators/functions)
'x' is hijacked as the matrix multiplication operator. e.g.
$c = $a x $b;
perlDL is row-major not column major so this is actually
c(i,j) = sum_k a(k,j) b(i,k) - but when matrices are printed the
results will look right. Just remember the indices are reversed.
e.g.:
Note: transpose() does what it says and is a convenient way
to turn row vectors into column vectors. It is bound to
the unary operator '~' for convenience.
If put in file dotproduct.pdl would be autoloaded (see below).
Also double(), short(), ushort().
These routines also automatically convert perl lists to allow the convenient shorthand:
etc.
Rules for automatic conversion during arithmetic:
perlDL betrays its perl/C heritage in that arrays are zero-offset.
Thus a 100x100 image has indices 0..99,0..99.
Further I adopt the convention that the center of the pixel (0,0)
IS at coordinate (0.0,0.0). Thus the above image ranges from
-0.5..99.5, -0.5..99.5 in real space. All perlDL graphics functions
conform to this defintion and hide away the unit-offsetness
of, for example, the PGPLOT FORTRAN library.
Again following the usual convention coordinate (0,0) is displayed
at the bottom left when displaying an image. It appears at the
top right when using ``print $a'' etc.
(Note: I hope to enable syntax like $$a{'0..200,3..200'} using
tie() but I am still thinking about the ramifications of this)
Read $1, $2, etc. pattern matches into $x, $y, etc.
PGPLOT provdes a simple library for line graphics and image display.
There is an easy interface to this in the interna;l module PDL::Graphics::PG. (This calls routines in the separately available PGPLOT top-level module.)
Current display commands:
Device manipulation commands:
Notes: $transform for image/cont etc. is used in the same way as the TR() array in the underlying PGPLOT FORTRAN routine but is, fortunately, zero-offset.
It is also hoped to use other graphic libraries to enable more sophisticated plots then is possible with PGPLOT. Some ideas:
Many astronomers like to use SAOimage and Ximtool (or there derivations/clones). These are useful free widgets for inspection and visualisation of images. (They are not provided with perlDL but can easily be obtained from their official sites off the Net.)
The PDL::Graphics::IIS package provides allows one to display images in these (``IIS'' is the name of an ancient item of image display hardware whose protocols these tools conform to.)
Commands are:
Variables are:
The frame buffer configuration is set by the variable $stdimage (analagous to iraf) whose default is ``imt1024''. System and user imtoolrc files are parsed so if you know about these you can do the same tricks as you can in with IRAF.
To come?
foo(), is currently undefined a file
``foo.pdl'' is searched for in the current directory, and any directories
in $PDLLIB, $PERL5LIB and $PERLLIB enviroment variables. (These are ``:''
seperated lists of directories.)
If you want to change the path within perldl simply change the lists @PDLLIB and @INC.
Note: ``foo.pdl'' is require'd so it must return a true value (see ``require'' perl documentation).
The syntax is:
@pdl_list is a list of pdl variables. Numbers get converted automatically. The file must be dynamically loadable object code - how the C compiler generates this will be different from system to system so see your man pages.
The C routine takes args (int nargs, pdl *args). The C type ``pdl'' is a
simple data structure representing the perl pdl variable. It is defined
in file ``pdl.h'' which is included in the perlDL distribution and has no
perl dependencies. It is trivial to cast the data array (pdl.data) to
(float), (double) etc. and pass to any other C routine.
This is all demonstrated in the files ``testcallext.*'' in the perlDL distribution.
Note: This is only intended as a quick and dirty prototyping interface for the scientist/hacker. perlDL developers should write a module along the lines of the example PDL::Examples.
perldl provides a simple command line - if the latest
Readlines/ReadKey modules have beeen installed perldl detects this
and enables command line recall and editing.
e.g.:
You can also run it from the perl debugger (perl -MPDL -d -e 1)
if you want.
Miscellaneous shell features:
perldl -oo starts perldl in Object-Oriented mode. It does
use PDL::OO instead of use PDL.
p to be a convenient short form of print, e.g.
~/.perldlrc and local.perldlrc (in the current
directory) are sourced if found. This allows the user to have global
and local PDL code for startup.
# character is treated as a shell
escape. This character is configurable by setting the perl variable
$PERLDL_ESCAPE. This could, for example, be set in ~/.perldlrc.
+ - * / > < >= <= << >> & | ^ == != <=> ** % ! ~ sin log abs atan2 sqrt cos exp
[All the unary functions (sin etc.) may be used with inplace() - see ``Memory'' below.]
pdl variables such as $x are implemented via Perl objects. However I have chosen to use an all-functional approach to perlDL syntax yo be more astronomer friendly.
However you can use perlDL in an OO fashion. In fact if you say:
It will load PDL functions as OO methods in the PDL class. This means you can say things like:
You can start the perldl shell in this mode with ``perldl -oo''.
Note: as you can see from the above all functions which create pdl variables are used with construct syntax in the OO mode. Finally you can even use both forms by simply saying ``use PDL; use PDL::OO''.
You can inherit from PDL methods (e.g. to a class Foo) by simply saying:
Then PDL methods will work on Foo objects as long as you simply build on the existing PDL data structure (see below) components.
So it would be possible to provide USER written modules to do really cool stuff for specific application areas, e.g. PDL::Spectrum might provide a $a which understands X-axes and error bars and +-/+ etc. might be overriden to do the Right Thing (tm). And writing the module would not be rocket science - just some cool perl hacking.
And you would not have to even use method syntax - if $a came out of my hypothetical PDL::Spectrum all the standard pdl functions (like hist() to give a concreate example) would work on it in the standard way provided they simply built on the existing PDL data structure (which means simply containing a $$a{Data} etc. PDL::Spectrum could even export it's own hist() function to override the built-in which might do something more sophisticated using the X-axis for example.
If you were feeling really ambitious you might do PDL::Ir::Spectrum which understood about the gaps between the J H and K bands!
NOTE: On some most systems it is better to configure perl (during the build options) to use the system malloc() function rather than perl's built-in one. This is because perl's one is optimised for speed rather than consumption of virtual memory - this can result in a factor of two improvement in the amount of memory storage you can use.
If $a is a big image (e.g. occupying 10MB) and I say:
then the total malloc()'d memory usage grows to 20MB. This is because the expression ``$a+1'' creates a temporary copy of $a to hold the result, then $a is assigned a reference to that. It is obviously done this way so ``$c=$a+1'' works as expected.
Also if one says:
Then $b and $a end up being different, as one naively expects, because a new reference is created and $a is assigned to it.
However if $a was a huge memory hog (e.g. a 3D volume) creating a copy of it may not be a good thing. One can avoid this memory overhead in the above example by saying:
The operations ++,+=,--,-=, etc. all call a special ``in-place''
version of the arithmetic subroutine. This means no more memory is
needed - the downside of this is that if $b=$a then $b is also
incremented. To force a copy explicitly:
Most functions, e.g. log(), return a result which is a transformation of their argument. This makes for good programming practice. However many operations can be done ``in-place'' and this may be required when large arrays are in use and memory is at a premium. For these circumstances the operator inplace() is provided which prevents the extra copy and allows the argument to be modified. e.g.:
The data structure for $a is implemented by a hash (associative array) which $a is a (blessed) reference too.
PDL reserves for it's own use:
Anything else stored in the structure will be copied to new objects (e.g. by $b = $a + 1) automatically as long as PDL knows how to copy it. [If it is a reference to another object PDL tries the ->copy method.]
If your perl routine manipulates the data structure guts directly, you don't want it to blow up in your face if you pass it a simple number rather than a PDL variable. Simply call the function topdl() first to make it safe. e.g.:
topdl() does NOT perform a copy if a pdl variable is passed - it just falls through - which is obviosuly the desired behaviour. The routine is not of course necessary in normal user defined functions which do not care about internals.
Finally there is no reason why the data structure should not contain another PDL variable!
Defined in PDL::Core
Defined in PDL::Examples
This contains examples of how to add C functiions via XS including use of the generic preprocessor (.g files are automatically converted to .c files with code automatically generated for each datatype).
Defined in PDL::Io
[See ``Io'' section above]
Defined in PDL::Graphics::*
[See ``Graphics'' section above]
Footnotes:
* = indicates inplace() safe & useful with this function