Ovid
There's more than one way to do things.
Perl Monks 

Re: Re: Re: Win32::MMF - Memory Mapped File Support for Perl

by xiper
 | log Roger out | Roger |  The Monastery Gates | Super Search | 
 | Seekers of Perl Wisdom | Meditations | PerlMonks Discussion | Q&A | Library | 
 | Obfuscation | Poetry | Cool Uses For Perl | Snippets | Code | Craft | 
 | Perl News | Reviews | Tutorials | Newest Nodes | Offering Plate | 

on Feb 11, 2004 at 14:31 GMT-10 print w/ replies, xml ) Need Help??

Reputation: 1

First of all, let me say this is an excellent module that is rapidly getting better. I have spent a couple of hours testing and thinking about this module and hopefully come up with some constructive comments on its current functionality. Please keep in mind these are only suggestions and very open to debate.

Comparing IPC::Shareable w/ Win32::MMF::Shareable

General interface

IPC::Shareable has the following basic interface:

use IPC::Shareable;

# simple method
tie( $scalar, 'IPC::Shareable', 'varid' );

# complex method
tie( $scalar, 'IPC::Shareable', 'varid', { ...options... } );  # OR
tie( $scalar, 'IPC::Shareable', { key => 'varid', ...options... } );
where you can do a simple 3-arg tie() and that's all, or optionally pass a hashref of options. Could i suggest a similar interface for Win32::MMF::Shareable:
use Win32::MMF::Shareable;

# simple method
tie( $scalar, 'Win32::MMF::Shareable', 'varid' );
# automatically creates a namespace using default options

# complex method
tie( $scalar, 'Win32::MMF::Shareable', 'varid', { ...options... } );  # OR
tie( $scalar, 'Win32::MMF::Shareable', { key => 'varid', ...options... } );
# for compatibility
Rather than having to call Win32::MMF::Shareable::Init to set things like namespace (see following section), size, autolock, etc, use defaults for everything and pass a hashref of option overrides to tie(). To avoid having to set the same options for each tie call every time, change Init to Defaults so it works something like this:
use Win32::MMF::Shareable;

# BAD

tie( $bigvar1, 'Win32::MMF::Shareable', 'bigvar1', { autolock => 0, size => 1024 ** 2 } );
tie( $bigvar2, 'Win32::MMF::Shareable', 'bigvar2', { autolock => 0, size => 1024 ** 2 } );
tie( $bigvar3, 'Win32::MMF::Shareable', 'bigvar3', { autolock => 0, size => 1024 ** 2 } );

# GOOD

Win32::MMF::Shareable::Defaults( { autolock => 0, size => 1024 ** 2 } );

tie( $bigvar1, 'Win32::MMF::Shareable', 'bigvar1' );
tie( $bigvar2, 'Win32::MMF::Shareable', 'bigvar2' );
tie( $bigvar3, 'Win32::MMF::Shareable', 'bigvar3' );
If you tie without any options you get the default values for each option. Optionally, you can call Win32::MMF::Shareable::Defaults to change the defaults for any future calls to tie. This way you have the best of simplicity and flexibility.

Thoughts on namespaces

IPC::Shareable uses a single identifier to reference a shared variable (refereed to as GLUE or key in the docs), where as Win32::MMF has two levels, a namespace and an identifier within the namespace. While this works well in Win32::MMF, as the namespace can be the object and you can have multiple items within the namespace, it becomes somewhat redundant when you tie a single variable with Win32::MMF::Shareable.

tie( $scalar, 'Win32::MMF::Shareable', { namespace => 'ns1', key => 'foo' } );
tie( $scalar, 'Win32::MMF::Shareable', { namespace => 'ns1', key => 'bar' } );
tie( $scalar, 'Win32::MMF::Shareable', { namespace => 'ns2', key => 'foo' } );
tie( $scalar, 'Win32::MMF::Shareable', { namespace => 'ns2', key => 'bar' } );
My suggestion would be to make the namespace option in Win32::MMF::Shareable optional, and have it transparently create and use the default namespace 'shareable'. This eliminates redundancy, still allows the user to override it if necessary, and allows users to mix-n-match the two modules:
# Process 1
use Win32::MMF::Shareable;

my @queue;
tie( @queue, 'Win32::MMF::Shareable', 'queue' );
@queue = qw( 8265 6201 7548 2165 7892 3546 3426 6246 );


# Process 2
use Win32::MMF;

my $ns = Win32::MMF->new( -namespace => "my_namespace" );
$ns->setvar( fredsname => 'fred' );
my $fredsname = $ns->getvar( 'fredsname' );
# etc... as normal

# get the current value of another processes tied variable!
my $server = Win32::MMF->new( -namespace => "shareable", reuse => 1 ); 
my @current_queue = $server->getvar( 'queue' );
Options

The two modules use different flags to control when a particular 'varid' is created or mapped to an existing one. IPC::Shareable uses create & exclusive flags, while Win32::MMF has a single reuse flag, and they behave as follows (bold is the default):

IPC::Shareable
create flag exclusive flag varid doesn't exist varid already exists
0 (unused) croaks maps to varid
1 0 creates new varid maps to varid
1 1 creates new varid croaks

Win32::MMF
reuse flag varid doesn't exist varid already exists
0 creates new varid maps to varid
1 fails silently maps to varid

Currently you can't specify for Win32::MMF to create exclusively – failing if the varid already exists. How about something like the following, which handles all 3 cases simply:

connect flag varid doesn't exist varid already exists
undef (or "auto"?) creates new varid maps to varid
1 fails silently maps to varid
0 creates new varid fails silently

tie( $var, 'Win32::MMF::Shareable', 'var' );  # OR
tie( $var, 'Win32::MMF::Shareable', 'var', { connect => 'auto' } );  # map if already available, otherwise create
tie( $var, 'Win32::MMF::Shareable', 'var', { connect => 1 } ); # must already exist
tie( $var, 'Win32::MMF::Shareable', 'var', { connect => 0 } ); # must not already exist
To maintain compatibility with IPC::Shareable you would still need to accept 'create' and 'exclusive' flags (but overridden by 'connect' if present) and convert them to the appropriate connect flag.

The other IPC::Shareable options are:

  • key – same as your 'varid', suggest you use this, or at least accept it as an alias
  • mode – redundant under win32, accept but ignore
  • destroy – removes shared data after the process exits, wether another process is using it or not. Not sure why one would use this, accept but ignore (unless you feel like implementing it)
  • size – same as your size
  • Tied object

    Now that we tie our shared variable, we no longer have access to an object which means we lose functionality of lock(), unlock(), debug(), and anything else you decide to implement in the future. Other tie() modules overcome this by returning an object from the call to tie(), which would allow things like:

    $data = tie( %data, 'Win32::MMF::Shareable', 'data', { timeout => 60 } );
    if( $data->{error} ) { die( “can't tie data: ” . $data->{errstr} ) }  # maybe? or just return undef
    
    $data{min} = 0;
    $data{max} = 500;
    $data{current} = 0;
    
    $data->debug() if $debug_flag;
    
    # each process must do blocks of 10 at a time
    {
      $data->lock();  # timeout 60 seconds
    
      for( $data{end} = $data{current} + 10; $data{current} < $data{end}; $data{current}++ )
      {
        # do job number $data{current}
      }
    
      $data->unlock();
    }
    

    Other thoughts

  • IPC::Shareable uses Storable to serialise its data – you may wish to consider switching from Data::Dumper to maintain consistancy. Not to mention Data::Dumper has some issues in what it can serialise. Maybe even provide an option for the user so they can choose.
  • The two modules Win32::MMF and Win32::MMF::Shareable, while implemented in a similar way, behave quiet differently to the user. It may be a good idea to document each separately to avoid confusion.
  • In the pod: "Because memory and variable management are managed internally by the Win32::MMF module, you do not need to specify how much memory is required by the variable." Is this correct? You don't have to specify a size for a tied variable? Will it automatically expand if the data grows? If so, why not have this for non-tied variables as well?
  • How does your module handle the issue of when to clear the data in shared memory? I assume that it's when the last 'connected' process exits or tied variable falls out of scope, correct? What about if the process is killed (via a SIGTERM)? Any possible memory leaks? (I assume you've already covered these issues, just interested to know).
  • *phew* I thinks that's about it... feedback welcome!

    - ><iper

    use japh; print;
    


    comment on Re: Re: Re: Win32::MMF - Memory Mapped File Support for Perl
    Download Code
    XP Nodelet
    You have 21 votes left today.
    Approval Nodelet
    Consider node: help
    Chatterbox
    xiper says np, hope it's helpful. Now i'd better get back to work... :)
    root says You've got a reply to Re: Re: Win32::MMF - Memory Mapped File Support for Perl from xiper at Re: Re: Re: Win32::MMF - Memory Mapped File Support for Perl
    PodMaster says http://crazyinsomn iac.perlmonk.org/p erl/ppm/README.htm l
      • Message Inbox

    <Roger> ybiC same here. :-)
    <ybiC> That's got to be among the mostest highest compliments I've ever received. But the others were from peeps who weren't smoking crack. {g}
    <Roger> :-)

    Whoa, Cowboy! You said that already. Don't 'reload'?

    How do I use this? | Other CB clients
    Other Users
    Others romping around the monastery: (27)
    demerphq
    rob_au
    PodMaster
    ybiC
    davido
    thatguy
    Roger
    atcroft
    thelenm
    barrd
    jdporter
    edan
    TVSET
    elwarren
    dug
    sporty
    markmoon
    dda
    xenchu
    SavannahLion
    chimni
    xiper
    3John
    scottj
    Koosemose
    eisforian
    sharmadi
    Node navigator
    Navigate all nodes:
    Disabled due to server load.
    Sections
    The Monastery Gates
    Seekers of Perl Wisdom
    Meditations
    PerlMonks Discussion
    Categorized Q&A
    Obfuscated Code
    Perl Poetry
    Cool Uses for Perl
    Snippets Section
    Code Catacombs
    Perl News
    Information
    PerlMonks FAQ
    Guide to the Monastery
    Voting/Experience System
    Tutorials
    Reviews
    Library
    Perl FAQs
    Outside Links
    Find Nodes
    Nodes You Wrote
    Super Search
    Perl Monks User Search
    Newest Nodes
    Best Nodes
    Worst Nodes
    Saints in our Book
    Leftovers
    Roger
    log Roger out
    The St. Larry Wall Shrine
    Offering Plate
    Awards
    Craft
    Quests
    Editor Requests
    Buy PerlMonks Gear
    Perl Monks Merchandise
    Random Node
    Voting Booth
    My interpersonal communication protocol/style is the most like...
    TCP
    UDP
    ICMP
    IPX
    SMB
    TFTP
    SMTP
    NNTP
    HTTP
    WAP
    SOAP
    [results]
    [423 votes] [past polls]