WebTools HELP File
Contents:
1.
Overview
2.
Web Tools
3.
GlobExport
4.
STDoutHandle
5.
Xreader
6.
Config
7.
Security and restrictions
8.
Various stuff
9.
Author
1.
Overview
WebTools
was projected to reduce maximum development work in web projects! It
decide most of common tasks, so you can think about more important things
at all. Note that, this package is not 10-15 kb and
you will need a time to learn everything about it. But don't afraid
to start. I am sure that it will be helpful for you :-)
This
document is not a comprehensive help. It is reference! For more information
and help please see examples into docs directory!
2.
Web Tools
This
module capsulate most of functionally that Web developer need for its
projects!
Mainly
is support: sessions, user accounts, html-perl code (php like), Database
support, templates and more..
Exported
Variables
*SESSIONSTDOUT
Standard
OUTput handler used by webtools to emulate PRINT-like
operators.
%SESREG
This
hash contain all registered variables with "this" session.
%SIGNALS
This
hash contained all catched signals from user's script.
%sess_cookies
Contain
all cookies received from input stream.
%uploaded_files
Contain
filenames of all uploaded files via multipart form!
They are names of files on local machine (i.e. your web server)
You can open and read these files from your server's tmp directory!
Note that you must to clean up by yourself (please delete these files
after
your job)
$cpg_priority
Show priority of cookies and get/post
global variables. This variable can
contain 'cookie' (cookies has higher priority)
or 'get/post' (get/post
variables rewrite cookies values)
$sentcontent
Show
whether Send_Content() where called! (i.e. data were flushed to browser)
$ip_restrict_mode
Set
'on' or 'off' restriction of session by IP
$apacheshtdocs
This
is apache`s htdocs directory.
$reg_buffer
Contain
registered session data!
$cookie_path_cgi
That
is PATH added to cookies (e.g: /cgi-bin/)
$cookie_domain_cgi
That
is DOMAIN added to cookies (e.g: .july.bg)
$cookie_exp_date_cgi
That
is DATE added to cookies
$secure_cookie_cgi
That
is SECURE atribute added to cookies (e.g: 1 or 0)
$session_started
Show
whether session_start were started!
$print_header_buffer
Here
is a place where script holds 'headers' (only!)
$print_flush_buffer
Here
is a place where script holds 'body'!
$sql_host
Variables
that contain SQL host from confing.pl
$sql_user
Variables
that contain user name from confing.pl, used
to access SQL server.
$sql_pass
Variables
that contain user pass from confing.pl, used
to access SQL server.
$sql_database_sessions
That
is database name of session's table.
$sql_sessions_table
That
is a name of session's table.
$cgi_script_timeout
That
is a expiration time of script i.e. maximum running time of script.
Default it's value is 120 seconds! If you need more time for your
script, please don't change that var directly, please use function
set_script_timeout() If
you thing you need more time for all your scripts at all,
then change this var (it is found into config.pl)
$cgi_lib_forbid_mulipart
If
you want to protect yourself from multipart spam turn
this to 'on' (Default it is set to 'off')
If you disable this variable you will not longer receive Multipart
forms via POST!
Exported
Functions
hideerror
Function
that will be executed when error occur in SQL server.
sql_query
SQL
query function. $result = sql_query ($query,$DBHandler);
sql_fetchrow
SQL
function that fetching rows from SQL server.
@arr
= sql_fetchrow ($result_handler);
sql_affected_rows
SQL
function that show a count of affected rows.
$number
= sql_affected_rows ($result_handler);
sql_inserted_id
SQL
function that show current inserted ID number.
$number
= sql_inserted_id ($result_handler);
sql_num_rows
SQL
function that show a count of affected rows.
$number
= sql_ num_rows ($result_handler);
sql_quote
Quote
one scalari.e. escape special chars and add at both begin and end parts
needed quotes!
$quoted_string
= sql_quote ($unquoted,$DBHandler);
sql_connect
Connect
SQL server. No params needed!
Return $DBHandler as result.
sql_connect
Connect
SQL server. Same as previous but it's take up to 6 parameters.
$DBHandler
= sql_connect ($host,$database,$user,$pass,[$port],[$full_path]);
$host
is needfull only for MySQL DB; $database is DNS for Access DB, normal DB
name for MySQL and file name (without path and without ext eg.: 'july');
$port
is proper only for MySQL DB; $full_path is a path to Flat DB file (eg.:
./db/)
NOTE:
You should specify all parameters except these in "[]" (if
they are not applyable)
sql_select_db
Select new database.
$new_DBHandler
= sql_select_db ($new_database,$DBHandler);
As
result of this function new database are now activated (selected) but
you must save (and use) the new one Database Handler instead of
previous one!
sql_disconnect
Disconnect
SQL server. No params needed!
Return
1 as result.
sql_errmsg
Return error message.
$err_mes
= sql_errmsg ($DBHandler);
sql_errno
Return error number.
$err_numb
= sql_errno ($DBHandler);
sql_data_seek
Move result handler to some row number.
$res
= sql_data_seek ($row_number,$result_handler);
load_database_driver
Load database driver (on runtime)
$res
= load_database_driver ($driver_name);
Where $driver_name can be 'mysql', 'flat', 'access' and 'sess_flat'!
DB_OnDestroy
Event
on destroy DB handler!
DB_OnExit
Event
on script exit!
________________________________________________
session_start
Start
SESSION. It can be inherited or created as is.
$SID
= session_start ($DBHandler,$NewSessionFLAG);
Where $NewSessionFLAG can be 1 or 0 (1 - will create absolutely
new session
(old
session is ignored), 0 - will inherit old session if any)
session_register
Register
one buffer (compressed vars) into Session table.
$result
= session_register ($Buffer,$DBHandler);
session_destroy
Destroy
current session
$SID
= session_destroy ($DBHandler);
session_ip_restrict
Set
mode of session restriction by IP. Valid modes are: 'on' and 'off'
session_ip_restrict
($mode);
session_id
Return
current $SID as result.
session_clear_expired
Clear
all expired records from database.
$SID
= session_clear_expired ($DBHandler);
session_id_adder
Add
SID ident to all links and forms in source!
$result_source
= session_id_adder ($source);
session_expire_update
Update
current session`s time up to now.
$result
= session_expire_update ($DBHandler);
action_sid_adder
Add
SID indents to all forms in source!
$result_source
= action_sid_adder ($source);
href_sid_adder
Add
SID ident to all links in source!
$result_source
= href_sid_adder ($source);
convert_ses_time
Convert
session time to "codded" integer.
$coded_string
= convert_ses_time ($source_time,$count_of_new_str);
new_session
Show
whether new session were started.
GetCurrentSID
Return
current Session ID.
register_var
Register
one variable (scalar , array or hash) into current session.
$coded_buffer
= register_var ($type,$name,@val);
where $type could be scalar, array or hash, $name can be
any permitted
variable name under whom module will save data.
@val
is a array that contain data for registration. It could be also just
scalar or even hash
unregister_var
UnRegister
one variable (scalar, array or hash)
$coded_buffer
= unregister_var ($name,$buffer);
read_scalar
Read
one scalar from DB (Already Registrated!!!)
$scalar
= read_scalar ($name);
read_array
Read
one array from DB (Already Registrated!!!)
@array
= read_array ($name);
read_hash
Read
one hash from DB (Already Registrated!!!)
%hash
= read_hash ($name);
update_var
Update
one variable from $reg_buffer.
$coded_buffer
= update_var ($type,$name,$buffer,@new_val);
exists_var
Check
whether one variable exists into $reg_var!
$bool_result
= exists_var ($type,$name,$buffer);
session_set_id_name
Set
name of session (labeled) (exmpl: 'sid')
session_set_id_name
($new_sid_label);
session_id_name
Get
current name of sesson (exmpl: 'pid','sid'…)
session_id_name
();
session_expiration
Get
current expiration date (exmpl: '+1h')
session_expiration
();
session_cookie_path
Get
current session cookie`s pat (exmpl: '/cgi-bin/')
session_cookie_path
();
________________________________________________
attach_var
Attach one variable(scalar) with output HTML
page.
Variable is 'moving' page to page via cookies
(or via links
and 'action' if cookies are not accepted)
attach_var($var_name,$var_value);
disattach_var
Disattach variable, attached with attach_var()
disattach_var($var_name);
________________________________________________
delete_cookie
High
level delete cookie function. It can get only 1 param!
delete_cookie
($name);
write_cookie
High
level write cookie function. It can get up to 6 params!
write_cookie
($name,$value,$expr_date,$path,$domain);
Params
from 3 to 6 are not required!
read_cookie
High
level read cookie function. It can get only 1 param!
read_cookie
($name);
GetCookies
Get
all cookies(or only wished) and put them all into %Cookies.
Don`t
call this function directrly! (Use: read_cookie()
instead!)
SetCookies
Set
cookie(or cookies) and return raw cookie string!
Don`t
call this function directrly! (Use: write_cookie()
instead!)
SetCookieExpDate
Set
cookie`s expiration for a cookie!
SetCookiePath
Set
cookie`s path for a cookie!
SetCookieDomain
Set
cookie`s domain for a cookie!
SetSecureCookie
Set
cookie`s secure attribute for a cookie!
GetCompressedCookies
Get
all compressed cookies (like common function)
SetCompressedCookies
Set
compressed cookies (like common function)
________________________________________________
Header
Write
one Header filed.
Header
(type=>'type' , val=>'value');
Type
can be: 'content',' cookie', 'raw',' modified',' MIME',' window',
'Pragma','
Expires',' Referer'.
And
respective example`s values for val are:
'text/html','name=July;Path=/','full
featured row field!', 'some DATE',
'1.0',
'some window target', 'nocache', 'some DATE'.
Most
wanted type is row!!! NOTE: You can`t access HTTP Header
anyway
except this Header function!
Example: Header (type=>'row'
, val=>"Content-type: text/html\n");
Parse_Form
Not implemented (only for back ware compatibility)!
b_print
Not implemented (only for back ware compatibility)!
ClearBuffer
Clear current printed body data! (use that way only!)
ClearHeader
Clear current printed header data! (use that way only!)
flush_print
At the end of script this function automatically flush all header's
and
body`s data!!! (automatically session, cookies and so on…)
If you call function with parameter "1" you will
use function in false mode! This mode is proper when your script use
downloader.pl (module for cgi downloads)
read_form
Read one var form input form (not need!!! All vars from forms and
cookies are global variables!)
read_form_array
Same like previous but reading is from array not from hash!
read_var
Read one scalar from browser (via cookie or just via link/form...
-
no mater :-))) (Well I sad…not need anymore :-)
________________________________________________
RunScript
Well that is main function at all! :-))) It eval (first compile html
Code into Perl code) all. That function is called from process.cgi
set_script_timeout
That function set script's timeout (in seconds), after which scripts
terminate automatically! This prevent from infinitive scripts!
$res = set_script_timeout
($time_out_in_sec);
r_str
rand_srand
Random string generator (and initial sub). Used from WebTools engine.
________________________________________________
SignUpUser
$res
= SignUpUser ($user,$pass,$data,$dbh);
$res
can be 1 - success and 0 - :-(
$data is user's data that you may whant to save into database.
You can save there up to 1MB!
SignInUser
($ID,$DATA)
= SignInUser ($user,$pass,$dbh);
$ID
is Id on user
$DATA
is custom data on same user fetched from DB!
________________________________________________
*
SIGNALS IN WEBTOOLS
That
module support base signals from Unix/Linux and couple customs.
So you can handle follow SIGs:
'OnFlush'
- That event will happened when script prepare to print all data (Header
and Body).
'OnError'
- That event will happened when some internal script error appear.
'OnExit'
- That event will happened when script exit.
'OnTerm'
- You can use this event when connection between server and browser
has been broken (used signals: TERM,STOP,PIPE)
'OnTimeOut'
- This event will happened when your scripts lifetime is down (Default
it is 120 seconds) (used signal is: ALRM)
To use
these events you must initialize global hash %SIGNALS whit proper reference
to sub.
Example:
$SIGNALS{'OnTimeOut'}
= \&MyDefaultTimeOutSubroutine;
sub MyDefaultTimeOutSubroutine { print "Timeout!!!"; }
Note:
All events are case-sensitive!
* What libraries are currently loaded?
That can be difficult question and some time you may need to reUSE or
to reREQUIRE some library. However I decide to introduce simple mechanism
solving that problem. Every library (or almost anyone) have unique number
that exist if given module were loaded.
Number
- Library:
1
- Database driver for MySQL (db_mysql.pl)
2 - Database driver for MS Access (db_access.pl)
4 - Database driver for Flat files (db_flat.pl)
8 - Xreader (xreader.pl)
16 - Flat support for Sessions (sess_flat.pl)
32 - Miscellaneous tools (tools.pl)
64 - Mail library (mail.pl)
128 - Simple functions for coding/encoding
(utl.pl)
256 - Download library (downloader.pl)
512 - Php simulation library (php.pl)
1024 - Simple fork library (fork.pl)
2048 - Sub set of JavaScript functions
(JavaScript.pl)
If you
want to find out whether some library is already loaded in WebTools
(this can speed up all execution process) you can check respective library
on that way:
if
(!($webtools::loaded_functions & 8)) {eval "require 'xreader.pl'";}
Follow
code will load "xreader.pl" library when it has not been loaded
yet.
Please use this format with require syntax to prevent double loading
of respective library!
3.
GlobExport
This
module is used indirect from scripts (included by webtools.pm). It parses
input data from form (POST), from GET and from Cookies. All input data
are exports to global variables!
There
aren't support functions for this module (constructor and destructor
are automatically called).
4.
STDoutHandle
This
module catch standard output (STDOUT) and printed data are buffered
on default.
5.
XReader
This
module is very cool its supply possibility of script to use extern templates
(html code and SQL inline)
That function read from file HTML data (with some
futures)
and substitute SQL queries and vars with
respective
values!
$scalar
= xreader($N_of_part,$filename);
For more
information see docs/examples/Lesson-8-xreader, but first read Templates
lesson in docs/templates !
6.
Config.pl
Typical
configuration for "Web Tools"
If you want to edit this file via Web just use
install.cgi !
#[Name_Of_Project]
$webtools::projectname = 'webtools'; # !!!
# This
is a name of project. Also this name is a base for other parameters
in this configuration file
#[SQL]
$webtools::db_support = 'db_flat'; # !!!
# Name
of database driver. Can be: db_mysql, db_access, db_flat
$webtools::sql_host = 'localhost'; # !!!
# host/ip
to DB server (MySQL)
$webtools::sql_port = '3306';
# Port
of SQL server
$webtools::sql_user = 'me'; # !!!
# User
name used to connect SQL server
$webtools::sql_pass = 'secret_password'; # !!!
# ...
and DB password
#[DataBase]
$webtools::sql_database_sessions = $webtools::projectname.'db'; #
!!!
# Database
name (name some like project!?!?!). If need change it!
$webtools::sql_sessions_table = $webtools::projectname.'_sessions'; #
!!!
# Name
of session table.
$webtools::sql_user_table = $webtools::projectname.'_users'; # !!!
# Table
that contain all users (and admin too)
#[CHECK]
$webtools::check_module_functions = 'on'; # !!!
# SCRIPTS
RUN IN CHECK MODE! After first check(test),
# please
turn this 'off'!
# YOU
CAN ALWAYS TURN THIS ON WHEN YOU WANT TO CHECK
# YOUR
CURRENT CONFIGURATION WITHOUT ANY OPPRESSION!!!
#[Secure]
$webtools::wait_attempt = '4';
# Count
of attempts when database is flocked
$webtools::wait_for_open = '2.0';
# Time
between two attempts (in sec)
$webtools::sess_time = '2'; # !!!
# Expire
time on session: 60 minutes
$webtools::sys_conf_d
= 'hour'; # !!!
# Type
of sess time dimension
$webtools::rand_sid_length
= '16';
# Length
of SID string!
$webtools::sess_cookie = 'sesstime';
# 'sesstime'
- cookie expire when session expired. Enter any other value to expire
cookie when user close browser.
$webtools::l_sid = 'sid';
# Session
label ID used by module
$webtools::cgi_lib_forbid_mulipart = 'off'; # !!!
# If
you want to protect yourself from multipart spam turn this 'on' (you
will be no longer able to use multipart forms)!
$webtools::cgi_lib_maxdata = 4194304; # !!!
# Set
maximum bytes to accept via POST (some king of multipart
# protection)
# Please
note that you can't set this to 0!!!
$webtools::cgi_script_timeout = 120; # !!!
# Expiration
time of script! (120 seconds default). After expiration script
# will
be terminated!
# Please
note that you can't set this to 0!!! You always must know how much time
you need!
$webtools::ip_restrict_mode = 'on';
# Set
'on' to restrict session by IP! If you have proxy problems with
# restricted
IPs, please set 'off' or use proper
# function
to set mode of this variable! (see: session_ip_restrict function)
# Allowed
values are: "on" and "off".
$webtools::run_restrict_mode = 'off';
# Set
'on' to restrict external web user to your scripts.
# If
IP's of user not exists in DB/ips.pl WebTools will
# close
script immediately!
# Allowed
values are: "on" and "off".
#[Debug]
$webtools::debugging
= 'on';
# Set
debugging mode
$webtools::debug_mail
= 'on'; # !!!
# Show
whether real mail must by sent or must by saved into mail
# directory!
#[Mail]
$webtools::sendmail
= '/usr/sbin/sendmail';
# Real
full path to sendmail
#[Other]
$webtools::charset
= 'n5b0xzlQWdgfNXytCMAwq1TYu7SVBmUIvcOPZ2aprER9kjh3sHJKL8e4oiDFG6';
# Please
mix well this chars to get higher security of your session ID :-)
$webtools::cpg_priority = 'cookie';
# Show
order of value fetching! There is 2 values: 'cookie' and 'get/post'.
# 'cookie'
means that cookie's variable has higher priority!
$webtools::sess_force_flat
= 'on'; # !!!
# Session
support via DB or via file! (possible values are: 'on' and 'off')
# When
you haven't DB support in your hand, you still can use sessions without
DataBase!
# To
begin using DB instead of flat files just set "this" variable
"off" (that is more secure and more compatible!)
# Most
session functions required "dbh" but you can leave it empty
or to supply real db handler: no matter!
# NOTE:
Use flat files when you haven't 500 opened sessions in "tmp"
directory or more!
$webtools::support_email
= 'support@your_host.com';
# Support
e-mail for unexpected errors! :)
$webtools::var_printing_mode = 'buffered';
# Default
output is buffered,
# leave
this variable empty if you need output
# of
your script to flush immediately!
# Other
else for non buffered mode.
# Actually
buffering depend of OS, Perl, Apache and socket!
@webtools::treat_htmls_ext
= (
# Order
of html files location: Default, module first look for:
# "html","htm","whtml"
and "cgihtml". If you specify in URL
# ...?file=env.html
script will ignore extension and will look for
# file
with extension ordered in @treat_htmls_ext array
# If
you leave this array empty then no lookups will be made!
# Order
of below array is really important!
# Example:
# If
your real file is: env.cgi and you want script to
# look
for env.html then with array below you will never success!
# That
is true because find operation will be broken when script found
# extension
you look for i.e. in our case (env.)html and after that it
# won't
continue! So if you want to run files that match our example
# file
then follow array should look like this:
# ('htm','whtml','cgihtml','cgi','html')
'html',
'htm',
'whtml',
'cgihtml',
'cgi',
);
#[PATHS]
$webtools::tmp
= '/tmp/'; # !!!
# Please
set full path to your temp directory
$webtools::driver_path
= './drivers/';
# Path
to your DB Drivers (please keep structure)
$webtools::library_path
= './libs/';
# Path
to your Libraries (please keep structure)
$webtools::db_path
= './db/';
# Path
to your DBs and tables (please keep structure)
$webtools::mailsender_path
= './mail/';
# Path
to your DBs and tables (please keep structure)
$webtools::xreader_path
= './jhtml/';
# Path
to your xreader files(jhtml-s) (please keep structure)
$webtools::perl_html_dir
= './htmls/';
# Path
to your perl's(webtools scripts) html files
$webtools::apacheshtdocs
= '/var/www/htdocs/';
# Path
to your apache htdocs directory('/usr/local/apache/htdocs/') (please
keep structure)
$webtools::cgi_home_path
= Get_CGI_Directory();
# Get
webtools cgi-bin directory (exmp: '/cgi-bin/webtools/')
# NOTE:
This path is not absolute and is not an HTTP!!! (actualy part of HTTP)
$webtools::http_home_path
= '/webtools/';
# Please
change this to your http path!
# NOTE:
This path is not absolute and is not an HTTP!!! (actually part of HTTP)
7.
Security and restrictions
If you have been completed read install.html document you probably
already mentioned most security problems in CGI world, so one of all
main targets are security and restrictions.
First
of all I want to insure you, that WebTools capture most of CGI security
holes.
*
Safely running of perl/html scripts:
Main
script "process.cgi" should run other scripts like "env.html"
, so only way to do it, is using of "file" variable in follow
format:
http://your_server/cgi-bin/your_webtools/process.cgi?file=some_script,
where
some_script can be located in your "htmls" directory! So it
is extremely important to set your default "htmls" directory
(in config.pl) with care (e.g.: /home/my_domain/my_web/cgi-bin/my_webtools/htmls/)
or (./htmls/ where your current directory is "webtools" directory).
WebTools
will care about security problems with script called like this:
http://your_server/cgi-bin/your_webtools/process.cgi?file=/etc/passwd
To prevent
security problems, you should follow next rules:
-
Never leave anyone directory variable empty! (like 'htmls', 'db', 'tmp'…)
-
Never set in these variable ROOT directory.
No body
can write in URL locater (in browser) query like that:
http://your_server/cgi-bin/your_webtools/process.cgi?file=../some_script
You are
protected for any kind of FILE crack actions like:
-
".." one directory down
-
"." Current directory
-
"/" or "\" Root directory
-
"~" Home directory
-
Also you have protected from any "illegal" characters in your
query!
However
you can use queries like:
…/process.cgi?file=/some_local_dir/some_script
where
/some_local_dir/ is directory in your "default htmls" directory.
For example that could be: /home/my_htmls/some_local_dir/
So I hope that you are understood how important is not keeping of "htmls"
directory empty or ROOT!!!
Also
via "file=" variable in query line you CAN'T call scripts
with whatever you want extension! Only acceptable are: whtml html htm
cgihtml and cgi
Please notice that all these files are WEBTOOLS files (HTML plus Perl
:-) )!
*
FLOOD protection:
Principle
WebTools haven't FLOOD protections but you should know that all multipart
data (I mean uploads) are saved in your TMP directory and if you don't
know about them, after end of your script WebTools will automatically
delete all of them!
Also
you have POST limit size. i.e. you can't accept more data that specified
in confg.pl file ($cgi_lib_maxdata default is 4 MB)
*
TIME protection:
If
your script stop to respond (for example stay in infinitive loop) or
wait forever for some event WebTools will "kill" your script
immediately after lifetime specified in config.pl ($cgi_script_timeout
default 120 seconds)
To change
that time please use set_script_timeout($timeout) function.
To capture event "OnTimeOut" please set follow handler:
$SIGNALS{'OnTimeOut'}
= \&your_sub;
*
IP restrictions:
If
you write web based application for small community you may want to
keep out external web visitors from your site?!? Now you can protect
all your WebTools based applications from external visitors based on
IP restriction. To switch this feature on, you need to fill out db/ips.pl
file with IPs that match your needs. Currently all IPs are allowed to
run your scripts! Please edit or delete lines that not match your protection
policy!
*
Session restriction based on IP
If
you use sessions you may want to protect your sessions by IP i.e. one
opened session can be used only by PC, created that session!
Note:
If your visitors use proxy to access your scripts you may need to switch
this feature off (Use follow syntax: session_ip_restrict('off'); )
Also
you should know that scripts are never restricted by IP if you use forced
flat file session support! (See config.pl and $sess_force_flat)
All
functions and WebTools itself are released to help and to protect all
of us, BUT you should realize that we are not GODs and you will take
all responsibility of WebTools actions on your server and on all world
wide! Please keep that in your mind.
PLEASE
SEND ALL SECURITY PROBLEMS THAT YOU MET IN YOUR PRACTICE WITH WebTools
at julian@proscrptum.com
Thank's
8.
Various stuff
That section regards various questions.
1. Highlighting for UltraEdit text
editor.
If
you want to have feature of highlighting with UltraEdit please open
your docs/WORDFILE.TXT
Now go
to your UltraEdit directory and open WORDFILE.TXT and add to this file
a contain of docs/WORDFILE.TXT file!
Edit follow line:
/L7"WebTools"
....
you may
need to change number of language supprt (default is 7: /L7....).
Save
changes of UltraEdit wordfile. Now UltraEdit will highlight files with
extensions: whtml and cgihtml :-)
9.
AUTHOR:
Julian
Lishev
email:
julian@proscriptum.com
Sofia,
Bulgaria