MCE 1.499_001

on Perl CPAN

• MCE 1.5 is underway. I have one method to add to MCE::Queue.
  In the meantime, the following have been completed code-wise.
  

Will work on documentation in the next 2 weeks and add additional
examples to highlight the various forms of parallel computing with MCE.

Regards,
;) Mario

• MCE.pm (8 data channels for up to 3x IPC handling)
  

e.g. 8 workers calling sendto in MCE 1.4 sends 80,000 per second.
That is now over 250,000 per second in MCE 1.5. That is just the
beginning. The following is not a good benchmark due to printf
formatting being done inside the preserve order code block run by
the manager. Foreach is able to do 40,000 and forseq 78,000 when
commenting out the printf statement. In any event, IPC is much
faster in MCE 1.5.

                     (Before)      (After)
 Parallel::Loops          600          600
 $mce->foreach         21,500       34,000  (+ 12,500)
 $mce->forseq          64,000       70,000  (+  6,000)
 $mce->forchunk       450,000      465,000  (+ 15,000)

• MCE::Flow, MCE::Grep, MCE::Loop, MCE::Map, MCE::Queue, MCE::Stream
  

How did I forget... The gather option... Take a look at the
MCE::Flow pod page when you have a chance.

• MCE 1.5 is backwards compatible with 1.4.x and lower.

• All public methods can be called directly using the package name
  and method e.g. MCE->wid, MCE->run.

  MCE->new(

  max_workers => 'auto',
  user_func => sub {
     my $wid = MCE->wid;
     MCE->sendto('STDOUT', "Hello from $wid\n");
  }
  

  );

  MCE->run;

• New example (interval.pl). More to come for the final 1.5 release.

• New options (gather, interval, task_name).

Changes to other options are described below.

The task_end option can now be specified at the top level.
See docs for use case.

Input_data can be specified inside the first task instead of
having to specify this at the top level. Note that input_data
is ignored if specified for tasks having task_id > 0.

 user_tasks => [{
    input_data => \@list,
    ...
 },{
    ...
 }

• New public methods
  chunk_id, gather, freeze, thaw, yield, task_name, print, printf, say

• Optimized the egrep.pl and wc.pl examples. These run much faster.
  Try these out against large log files. Both examples fly.

• Barrier synchronization update. Two sockets are utilized instead of
  two lock files. This, at last, now works wonderfully under the Cygwin
  environment. For threading, the removal of the 2 lock files allows
  one to have a greater number of threads within the limits of ulimit.
  Previously, only about 1/3rd of ulimit -n was possible. One can now
  spawn just under 1/2 of ulimit -- int(ulimit -n / 2 - 10).

• Removed the logic for determining MAX_OPEN_FILES and MAX_USER_PROCS.
  MCE no longer limits on max_workers allowed.

• Code re-factor work. Added a private method _validate_runstate called
  by various methods. Organized the code slightly such as placement of
  methods.

• Localize the input scalar $_ prior to calling userfunc. Folks can
  use $ for input_data and sequence of numbers. Added section to docs
  explaining DEFAULT INPUT SCALAR.