Reading files recursively in parallel in Perl -


i have 500 files read, reading recursively each file takes 2 minutes approximately. want operation in parallel using perl. how can that?

you're talking massive amount of reading if takes 2 minutes. you're spending time waiting hard drive. files on separate hard drives? if not, why think trying second file @ same time going faster? in fact, might make things slower increasing amount of seeking hard drive has make.

but if want try anyway,

use threads; use thread::queue qw( );  use constant num_workers => 4;  # twiddle  sub run {    ($qfn) = @_;    ...read file $qfn here... }  $q = thread::queue->new();  @threads; (1..num_workers) {    push @threads, async {       while (my $job = $q->dequeue()) {          run($job);       }    }; }  $q->enqueue($_) @qfns;  $q->enqueue(undef) @threads; $_->join() @threads; 

Comments

Popular posts from this blog

linux - xterm copying to CLIPBOARD using copy-selection causes automatic updating of CLIPBOARD upon mouse selection -

qt - Errors in generated MOC files for QT5 from cmake -