The Perl Short Questions Megathread: executable line noise

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > The Perl Short Questions Megathread: executable line noise

«‹›72 »

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

Triple Tech posted:

Maybe I'm being too idealistic (probably) but if I was developing a sort of Perl tool, a meta-program if you will, I would have concentrated on making it work on the language level and not use any OS-level stuff.

I'm sure it'll work fine on Windows once this one system call is figured out. It's not all going to come crashing down to support Windows.

Besides, this kind of library really has to be built into the interpreter, not on top of it, so it'll be sensitive to different OSes.

# ? Oct 8, 2008 03:28

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 20:31

Mithaldu: Sep 25, 2007; Let's cuddle.

Cross-platform stuff has its way of sneaking up on you. Aside from that, just from the experience of realizing my own first project at the moment, i'd say the first priority is making it work in the first place and then branch into other stuff once you know it works. Additionally, windows programmers tend to have a good bit of knowledge about linux simply because they're geeks. Heck, many probably know enough about linux to have decided that its flaws outweigh windows'. However linux programmers often don't even have windows around in the first place, nevermind any knowledge about it.

What system call? Mind giving a bit more information? I don't have the time to commit to a completely new project now, but i would still like to let it roll around in my brainpan a bit. =)

# ? Oct 8, 2008 03:29

Mario Incandenza: Aug 24, 2000; Tell me, small fry, have you ever heard of the golden Triumph Forks?

TiMBuS posted:

Maybe you can get it running, someone left a rating on the module telling you how to make it work:

Turns out it was actually this bug. Digests come out differently so Crypt::RIPEMD160 apparently installed, but Crypt::OpenPGP's t/07-digest.t blew up. Math::Pari's author fixed it for 5.10 in March, so as long as you build Pari from source it's relatively pain-free (for now).

# ? Oct 8, 2008 09:33

Squashy Nipples: Aug 18, 2007

Warning: I know no PERL whatsoever.

A continuation from my previous thread:
http://forums.somethingawful.com/showthread.php?threadid=2976738

Ok, I need to modify a PERL CGI script. It takes a bunch of data from an HTML form ($fields), and adds a line to a CSV output file. The problem is that the current script does NO data sanitizing what so ever. I need the CVS to be compliant to CSV standards:

1. Replace all literal double-quote marks with TWO double-quote marks
2. Then encapsulate all data strings with double quote marks

Googling around, I found a piece of code that does this:

code:

print join(',', map { s/"/""/g; qq("$_") } @fields), "\n";

And this is the relevant piece of code:

code:

   if ($fields{'outputfile1'} ne "") { 
      &get_the_lock; 
      open(OUT_FILE,">>$fields{'outputfile1'}"); 
   }

   foreach $to_print (@sortlist) {
      if ($fields{'outputfile1'} ne "")
       { print OUT_FILE "$fields{$to_print}\,"; }
      if ($fields{'submit_to'} ne "")
       { $msgtext .= "$to_print = $fields{$to_print}\n"; }
   }
   if ($fields{'outputfile1'} ne "") {
     print OUT_FILE "$SD\|$ST\|\n";
     close(OUT_FILE);
     &drop_the_lock;
   }

I really HATE to use code that I don't fully understand, but I'm rear end out on this one because I don't know PERL, and because I'm basically doing someone else's job for them.

Can someone please show me how to integrate this code?

# ? Oct 8, 2008 20:55

Mario Incandenza: Aug 24, 2000; Tell me, small fry, have you ever heard of the golden Triumph Forks?

If you can install Text::CSV_XS off CPAN (run su, then cpan Text::CSV_XS), it's fairly easy. I'm not sure about the $SD/$ST references so I've left them in there.

code:


use Text::CSV_XS;

if (my $filename = $fields{outputfile1}) { 
  my @output = ( @fields{@sortlist}, "$SD\|$ST\|" );
  append_line($filename, \@output);
  $msgtext .= join("\n", map { "$_ = $fields{$_}" } @sortlist);
}

sub append_line {
  my ($filename, $fields) = @_;

  my $csv = Text::CSV_XS->new({
    always_quote => 1,
    eol => "\r\n",
  });

  open my $filehandle, '>>', $filename or die "couldn't open $filename: $!";
  $csv->print($filehandle => $fields);
}

Mario Incandenza fucked around with this message at 01:17 on Oct 9, 2008

# ? Oct 9, 2008 01:14

Squashy Nipples: Aug 18, 2007

Nope, I can't install anything; this is a script running somewhere on my intranet, and all I can do is edit the script.

SD/ST is just a Date/Time Stamp.

Ok, I gave myself a two hour crash course in PERL last night. I'm still having trouble understanding how the variables are passed to functions, but I THINK I've got it figured out. This is the only line that I need to edit:

code:

{ print OUT_FILE "$fields{$to_print}\,"; }

Here are my candidates to replace that line:

code:

{  $_ = "$fields{$to_print}";
   s/"/""/g;
   print OUT_FILE qq("$_");  }


{  $_ = "$fields{$to_print}";
   print OUT_FILE "\"", s/"/""/g, "\"\,";  }


{  print OUT_FILE "\"", $fields{$to_print} s/"/""/g, "\"\,";  }

Think any of those will work?

# ? Oct 9, 2008 16:03

s139252: Jan 1, 1970; test

Dr.Khron posted:

code:
{ print OUT_FILE "$fields{$to_print}\,"; }
Think any of those will work?

Add this inside your foreach loop:

code:

my $data = $fields{$to_print};
$data =~ s/"/""/g;
$data = '"' . $data . '",';

Then replace 'print OUT_FILE "$fields{$to_print}\,";' with 'print OUT_FILE $data;'.

I think there is no single standard for CSV, so you'll want to make sure you're doing whatever else is required by whatever definition you adopt.

# ? Oct 9, 2008 18:33

Squashy Nipples: Aug 18, 2007

Thank you! Looks like that will work.

The 'My' just makes it a super-local variable?

EDIT:
Ah, its called a Lexical Varaible, and its scope is dependent on the nested blocks. Very cool! Thanks again.

Squashy Nipples fucked around with this message at 19:06 on Oct 9, 2008

# ? Oct 9, 2008 19:02

CanSpice: Jan 12, 2002; GO CANUCKS GO

Also, you should use lexical filehandles:

code:

open my $outfh, "outfile.whatever";
print $outfh "$fields{$to_print}\,";

Lexical filehandles help prevent bugs where the same filehandle could possibly get overwritten by a module or some other piece of code, and accidentally point to two files (which would probably just end up with your data being written to the wrong file).

Technically with lexical filehandles you don't need to close them, as once they go out of scope they automatically close, but it's good practice to always use open/close pairing.

# ? Oct 10, 2008 22:56

Ninja Rope: Oct 22, 2005; Wee.

CanSpice posted:

code:

open my $outfh, "outfile.whatever";
print $outfh "$fields{$to_print}\,";

You should also use parentheses around the open command (or any command that takes multiple parameters and returns something), to help avoid writing code like this:

code:

open my $h, '<', 'file' || die "Failed to open file!";

# ? Oct 10, 2008 23:35

Kraus: Jan 17, 2008

Hey perl dudes, I got a little question for you.

I'm writing a script that parses out some information from HTML code and reprints it in a readable fashion. Namely, this is me pulling apart the source code for an online dictionary and just printing the headword and its definitions.

My problem lies in that the line that prints the definitions looks like this:

print " $1, " if /regexp/;

The problem specifically is that if there is just one definition, then it has a comma after it. Or the last one in the list of definitions has a comma after it. Is there a way I could get rid of that comma if it's not needed? The spaces aren't a big deal since they're "invisible", but that comma's gotta go.

# ? Oct 12, 2008 02:17

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

Typically I build a list of all the items, then use join ", ", @items to make sure there are commas only between items.

If you have to process iteratively, then you can do something like:

code:

print get_item();
while (my $item = get_item()) {
    print ", ";
    print $item;
}

Alternatively, if you're building this up in a string..

code:

my $output = '';
while (my $item = get_item()) {
    $output .= $item;
    $output .= ", ";
}
chop $output; chop $output;

But really I recommend just join.

# ? Oct 12, 2008 02:41

Kraus: Jan 17, 2008

Thanks, Sartak. I'll give join a shot. I'll edit this post with the results of tests with join.

# ? Oct 12, 2008 02:55

syphon^2: Sep 22, 2004

Does anyone have any advice for making Perl talk to Active Directory?

My company is a Windows shop, and we use AD in both our corp workstations and test lab. The growing trend among my team is to interrogate DL's (which as I understand it, are the same thing as AD groups) to determine system authorization.

I've tried a couple how-tos, one on Net::LDAP and another that uses Win32::OLE, but I couldn't get either example working with my existing network. I think there's just too much about AD/LDAP I don't know about, so I don't even know where I'm going wrong.

Any advice would be greatly appreciated.

# ? Oct 12, 2008 22:21

Ninja Rope: Oct 22, 2005; Wee.

syphon^2 posted:

Any advice would be greatly appreciated.

I've used Net::LDAP many times to interact with Active Directory and it has worked quite well. Where exactly are things failing?

# ? Oct 13, 2008 00:11

dagard: Mar 31, 2005

syphon^2 posted:

Does anyone have any advice for making Perl talk to Active Directory?
(snip)
Any advice would be greatly appreciated.

It's been a while, but I seem to recall that, by default, you can't anonymously bind to the AD server, so you have to have a user even to do that. But otherwise, we use an AD server with a few unix hosts here (LDAP), and I know for a fact that TVA does too.

Edit: http://pastebin.com/m40d7df9c

Yes, it's ugly, but it's basically what I used when working with our AD administrators to make sure connectivity was working.

dagard fucked around with this message at 01:49 on Oct 13, 2008

# ? Oct 13, 2008 01:16

tef: May 30, 2004; -> some l-system crap ->

dagard posted:

It's been a while, but I seem to recall that, by default, you can't anonymously bind to the AD server

I recently got some python code to talk to AD, and yes I had to use an existing account to bind to AD first.

Bugzilla uses Net::LDAP, and I've used it to talk to ActiveDirectory. You might be able to fish a working example out of the code base.

# ? Oct 13, 2008 01:26

syphon^2: Sep 22, 2004

Ninja Rope posted:

I've used Net::LDAP many times to interact with Active Directory and it has worked quite well. Where exactly are things failing?

Here's the code I'm using... cut&paste from the Net::LDAP example I posted. The code first connects to the LDAP server, then binds to it (I'm not sure which of the two bind methods I should use, I've tried both individually). Then, it performs two searches based on the example given. It first tries to get all users, then it tries an individual user look-up.

(please forgive the fairly messy variable delcaration, I tried to sanitize my code before posting it here)

code:

#!perl

use strict;
use Net::LDAP;

my $adserver	= "AD-SERVER.MY.CORP.NETWORK.COM";
my $user	= "username";
my $pass	= "password";
my $name	= "User Name";
my $group	= "All Users";
my $dc		= "DC=MY,DC=CORP,DC=NETWORK,DC=COM";

my $ad = Net::LDAP->new('ldap://' . $adserver)
	or die("Could not connect to LDAP server. $!");
		  
$ad->bind($user . "\@" . $adserver, password=>$pass)
	or die "Error binding to LDAP Server: $!";
$ad->bind("CN=" . $name . ",OU=Policy 0, OU=" . $group . "," . $dc)
	or die "Error binding to LDAP Server: $!";

#------Search for all users----------	
# The search base?
my $base = 'OU=' . $group . ',' . $dc;
# What are we searching for?
my $filter = "memberof=CN=" . $group . "," . $dc;
# Which attributes should be returned?
my $attrs = "sn, givenname";
# Execute the search
my $results = $ad->search(base=>$base,filter=>$filter,attrs=>$attrs);
# How many entries returned?
my $count = $results->count;
print "Total entries returned: $count\n";
# Display entries
my $entry;
for (my $i=0; $i<$count; $i++) {
     $entry = $results->entry($i);
     print $entry->get_value('sn').", ".
           $entry->get_value('givenname')."\n";
}


#------Look up a specific user----------
use Time::Local;
# . . .
my $base = 'OU=All Users,' . $dc;
# What are we searching for?
my $filter = "CN=" . $name;
# Which attributes should be returned?
my $attrs = "sn, givenname, telephonenumber, mail, pwdLastSet";
# Execute the search
my $results = $ad->search(base=>$base,filter=>$filter,attrs=>$attrs);
# Display entries
my $entry;
$entry = $results->entry(0);
print $entry->get_value('sn').", ".$entry->get_value('givenname')."\n";
print "Tel: ".$entry->get_value('telephonenumber')."\n";
print "Email: ".$entry->get_value('mail')."\n";
print "Password last changed: ".
       localtime($entry->get_value('pwdLastSet'))."\n";

$ad->unbind;

This code outputs the following:

code:

C:\enlistment>perl ldap.pl
Total entries returned: 0
Can't call method "get_value" on an undefined value at ldap.pl line 54, <DATA> line 466.

As you can see, there are two problems. First, it's failing to retrieve any users from the generic search. Second, it's failing to look up a specific user (I hope that's causing the error it gave).

EDIT: I figured it out. I had a coworker sit with me (who'd done something similar in Ruby), and we were able to figure it out. I had my Search Base all wrong.

syphon^2 fucked around with this message at 13:44 on Oct 13, 2008

# ? Oct 13, 2008 12:04

Squashy Nipples: Aug 18, 2007

Once again, thanks for all the help guys.

CanSpice posted:

Also, you should use lexical filehandles:
code:
open my $outfh, "outfile.whatever";
print $outfh "$fields{$to_print}\,";
Lexical filehandles help prevent bugs where the same filehandle could possibly get overwritten by a module or some other piece of code, and accidentally point to two files (which would probably just end up with your data being written to the wrong file).

Technically with lexical filehandles you don't need to close them, as once they go out of scope they automatically close, but it's good practice to always use open/close pairing.

Ug, this is why I have no desire to learn PERL... All those limited scopes must be hard to keep track of. Also, the "Loosely Typed" thing drives me nuts: I'm sort of OCD with numbers, and it just makes me SO happy to plan out my variable use and properly declare both type and scope at the begining of each sub.

Explicit Variable Declaration: its just feels right!

# ? Oct 13, 2008 17:34

npe: Oct 15, 2004

Dr.Khron posted:

Ug, this is why I have no desire to learn PERL... All those limited scopes must be hard to keep track of. Also, the "Loosely Typed" thing drives me nuts: I'm sort of OCD with numbers, and it just makes me SO happy to plan out my variable use and properly declare both type and scope at the begining of each sub.

Explicit Variable Declaration: its just feels right!

Perl has "use strict", which forces you to declare all of your variables with either "my" or the somewhat rarely used "our".

Most perl programmers won't do anything without "use strict" for good reason.

# ? Oct 13, 2008 17:58

Mithaldu: Sep 25, 2007; Let's cuddle.

"Limited" scopes aren't hard to track at all. "use strict;" and you'll never have to spend a thought about scoping at all, it just comes naturally:

- variables are either:
-- passed as input at the beginning of a block, via @_
-- created at any point in a block via my

- variables are valid:
-- from start to end of a block
-- within all blocks created inside the current block

If i remember correctly, this works the same in c, doesn't it?

As for loose typing:

Yes, it is not very efficient. However that is the ONLY criticism you can level against it, that it uses more memory than it *needs* to. In practice it allows extremely fast prototyping. Example: I have implemented, solo, next to my usual 40 hours work per week, a basic, but fully functional 3d engine, that does some seriously weird stuff in the data processing part to boot. It took me less than 25 days and there's stuff in it like 4-dimensional arrays, mixed in being number-based and associative.

Being OCD about memory-use is all good and fine, but only if memory use is actually an important factor.

And declarations, scoping (and in part also type, either scalar, array or hash) also happen in Perl the same as in C, as long as you follow sane programming guide lines.

I also suggest reading this: http://www.perlmonks.org/?node_id=87628
It is a bit old, being from 2001 and talks about stuff that was relevant for 5.6, but the bulk of it is still relevant.

Mithaldu fucked around with this message at 18:06 on Oct 13, 2008

# ? Oct 13, 2008 18:01

Yeast Confection: Oct 7, 2005

I used LWP::Simple and HTML::LinkExtor to parse links off webpage and put them in to an array.

I tried to print the array and it gives me a list of references, like so

code:

print @parsedlinks;

ARRAY(0x9e5ebc4)
ARRAY(0x9e5ec00)
ARRAY(0x9e5ec3c)
ARRAY(0x9e5ed20)
ARRAY(0x9e5ecb4)
ARRAY(0x9e5ed50)

I'm really having trouble getting my head around de-referencing them because they're a bunch of smaller arrays stuck inside my @parsedlinks array.

I've tried this to just print the first element in the array dereferenced but it still gives me the same output as above.

code:

$linkref = \@parsedlinks;
print $linkref->[0];

I've also tried looking at each element individually like this

code:

foreach(@parsedlinks){
 my $ref=$_;
 print @$ref . "Unreferenced?\n";
}

output: 3 Unreferenced?

Any advice to help me understand this is a bit better?

Yeast Confection fucked around with this message at 23:14 on Oct 13, 2008

# ? Oct 13, 2008 22:45

syphon^2: Sep 22, 2004

Use Data:

umper to view the data structure. At the very least, you can paste it here and people can tell you how to de-reference it properly.

# ? Oct 13, 2008 23:54

Yeast Confection: Oct 7, 2005

After some fiddling around I managed to make it spit out the right data. Using dumper helped a lot to find out which element actually had the URLs I'm looking for in them.

Thanks for the tip

# ? Oct 14, 2008 00:23

syphon^2: Sep 22, 2004

Has anyone used Win32::OLE to access remote registry keys, instead of Win32::TieRegistry? I'm trying to compare the performance of the two, and the only example I could find of using Win32::OLE to grab remote registry data is... cumbersome, to say the least.

Here's the example: http://aspn.activestate.com/ASPN/Mail/Message/activeperl/2965351

Here's the code (adapted to my own needs):

code:

use Win32::OLE;
use Win32::OLE::Variant;

use constant HKEY_LOCAL_MACHINE	=> 0x80000002;

my $server = "SERVERNAME";

my $oReg = Win32::OLE-> GetObject( 'winmgmts:{impersonationLevel=impersonate}!\\\\'
 . $server . '\\root\\default:StdRegProv')
	or die "GetObject: " . Win32::OLE-> LastError();

my $path = "SOFTWARE\\CompanyName";
my @names = ('Key1', 'Key2', 'Key3');
my @types = (VT_BSTR, VT_BSTR, VT_I4); 
#Note that @types has to directly correlate to @names

my $strKeyPath = $path;
for (my $ii = 0; $ii < @names; ++$ii) {

	my $strValueName = $names[$ii];

	if ($types[$ii] == VT_BSTR) {
		my $strValue = Variant (VT_BSTR|VT_BYREF, '<Not Set> ');
		my $ret = $oReg-> GetStringValue(HKEY_LOCAL_MACHINE,$strKeyPath, $strValueName, $strValue);
		$strValueName = '(Default)' if not $strValueName;
		print "$strValueName: ", $strValue-> Value(), "\n";
	} else {
		my $Value = Variant (VT_I4|VT_BYREF, 0);
		my $ret = $oReg-> GetDWORDValue(HKEY_LOCAL_MACHINE,$strKeyPath, $strValueName, $Value);
		print "$strValueName: ", $Value-> Value(), "\n";
	}
}

# ? Oct 21, 2008 18:05

Ninja Rope: Oct 22, 2005; Wee.

I don't see why one would be any faster than the other, the big delays will still be network latency and the efficiency of whatever network protocol the remote registry service uses.

# ? Oct 21, 2008 19:13

syphon^2: Sep 22, 2004

The environment I'm running in is very geographically diverse (it's not uncommon for a server from Seattle to query one in Dublin or Sydney), so latency is quite high.

Also, Win32::TieRegistry operates via a tied-hash... it doesn't ever seem to store data in local memory. I have a web-page which queries registry information real-time, so it gathers about 8 reg-keys from a series of remote servers. This runs SLOW, so I'm looking to speed it up. Setting up some timestamps, I can see that Win32::TieRegistry seems to operate (roughly) in this manner...

1) Connect to remote server
2) Gather reg-key value
3) Disconnect from remote server

Now repeat that process serially for 8 reg-keys across 30 servers, and you see why I'm looking for performance gains.

I can save myself a lot of time if I can find a way to take a dump (heh heh) of a certain Registry Key and all its sub-keys, and then just let Perl navigate it internally as a hashref or something. I couldn't find any way to do this with Win32::TieRegistry, so I was trying other means.

# ? Oct 21, 2008 20:41

tef: May 30, 2004; -> some l-system crap ->

If speed is important why don't you do it in parallel ?

# ? Oct 22, 2008 01:23

npe: Oct 15, 2004

Back when I had to solve this problem, I did it by installing an "agent" script , either on the local machines, or on a machine close to them on the network, which would gather up the data and store it locally so that I could poll it in one big chunk. Sped things up tremendously, but I have no idea how applicable that solution is to you.

# ? Oct 22, 2008 02:24

JawnV6: Jul 4, 2004; So hot ...

I'm fairly new to socket programming in general. I'm using IO::Socket::INET to control UDP traffic. Is it possible to use the same socket to both send and receive?

I have the following:

code:

my $MyOutSocket=new IO::Socket::INET->new(PeerPort=>9035,
        Proto=>'udp',
        PeerAddr=>'255.255.255.255',
        Broadcast => 1,
        LocalPort =>58222);

my $MyInSocket =new IO::Socket::INET->new(LocalPort=>58222,
        Proto=>'udp') or die $!;

which fails printing 'Address already in use'. Trying ->send() then ->recv() just hung on the receive call. I send out the broadcast, then a non-broadcast UDP packet comes back to the same port, I guess this is breaking send/recv on the same socket. Do I need to close the 'Out' before I use the 'In'?

# ? Oct 22, 2008 08:13

Mario Incandenza: Aug 24, 2000; Tell me, small fry, have you ever heard of the golden Triumph Forks?

It's possible, though only under Linux/Unix. According to this Perlmonks thread, Windows will exhibit the blocking you describe, so you must use a two-socket solution.

# ? Oct 22, 2008 09:36

Ninja Rope: Oct 22, 2005; Wee.

SpeedFrog posted:

It's possible, though only under Linux/Unix. According to this Perlmonks thread, Windows will exhibit the blocking you describe, so you must use a two-socket solution.

That thread is 5 years old. I haven't tried it, but I would be surprised if you can't send and receive from the same socket on Windows. You sure can using winsock/BSD sockets.

Jawn, if it hangs on the recv() call then there's no data there to be received and it will block until there is some. If that's not what you want, you've got a few options, but your best bet is to use select() (or IO::Select) to test whether there is data waiting on the socket before calling recv(). If you have two sockets bound to the same interface and port, there is no guarantee which socket will receive the data, and it could be that the socket that received the data isn't the socket that you called recv() on.

# ? Oct 22, 2008 10:12

syphon^2: Sep 22, 2004

tef posted:

If speed is important why don't you do it in parallel ?

My Perl kung-fu is still relatively weak, so I'm not sure how best to tackle this. How would you recommend it? I was thinking of either setting up a fork or using Win32::Process::Create, but it looks like this is the route I'll end up going.

yaoi prophet posted:

Back when I had to solve this problem, I did it by installing an "agent" script , either on the local machines, or on a machine close to them on the network, which would gather up the data and store it locally so that I could poll it in one big chunk. Sped things up tremendously, but I have no idea how applicable that solution is to you.

I thought about this, but it's really not feasible in my situation. Our environment changes so fast (new servers added, old servers removed, servers renamed, servers re-imaged) that trying to maintain an agent system would be impossible (or at least, very very hard and prone to failure).

# ? Oct 22, 2008 10:38

Mithaldu: Sep 25, 2007; Let's cuddle.

You may wanna try using http://search.cpan.org/~mlehmann/Coro-4.8/Coro/Socket.pm and http://search.cpan.org/~mlehmann/Coro-4.8/Coro.pm to get parallelization. Coro allows such kinda stuff without having to gently caress around with multiple processes or Perl's horrible threads.

Otherwise just have one main script and one data getter script.

Data getter:
1. connects to server and gets registry data
2. uses Data::Dumper or Storable to serialize data
3. dumps data into time-stamp-tagged file in a certain directory

Constroller:
1. starts and spawns x getters with exec()
2. sleeps for y seconds
3. checks directory for new files and processes them if needed
4. if number of processed files not equal x -> 2.

# ? Oct 22, 2008 13:06

syphon^2: Sep 22, 2004

Mithaldu posted:

You may wanna try using http://search.cpan.org/~mlehmann/Coro-4.8/Coro/Socket.pm and http://search.cpan.org/~mlehmann/Coro-4.8/Coro.pm to get parallelization. Coro allows such kinda stuff without having to gently caress around with multiple processes or Perl's horrible threads.

Huh, threading/coroutines must be beyond me. I looked over this, and their example seems nothing more than an elaborate goto(), it doesn't run things asynchronously.

Take the following code... for example (ripped straight from http://search.cpan.org/~mlehmann/Coro-4.8/Coro.pm with a timer added in)

code:

use Coro;
use Time::HiRes qw(tv_interval gettimeofday);

my $timer = [gettimeofday];
async {
	# some asynchronous thread of execution
	print "2\n";
	sleep 5;
	cede; # yield back to main
	print "4\n";
};
print "1\n";
sleep 5;
cede; # yield to coroutine
print "3\n";
cede; # and again
print "Elapsed Time:\t" . int(tv_interval($timer) * 1000) . "ms\n";

The output is as follows:

code:

C:\>coro.pl
1
2
3
4
Elapsed Time:   10000ms

My understanding is that the code inside async should be running asynchronously, so the two sleeps should run in parallel. However, as you can see, the script is taking 10 seconds to fully execute.

What am I doing wrong?

# ? Oct 22, 2008 15:46

JawnV6: Jul 4, 2004; So hot ...

Ninja Rope posted:

That thread is 5 years old. I haven't tried it, but I would be surprised if you can't send and receive from the same socket on Windows. You sure can using winsock/BSD sockets.

Jawn, if it hangs on the recv() call then there's no data there to be received and it will block until there is some. If that's not what you want, you've got a few options, but your best bet is to use select() (or IO::Select) to test whether there is data waiting on the socket before calling recv(). If you have two sockets bound to the same interface and port, there is no guarantee which socket will receive the data, and it could be that the socket that received the data isn't the socket that you called recv() on.

I'm working on ubuntu 8.04. "Hang" was a terrible word, it's not picking up the data. On my network tracer I'm seeing the computer send out a ICMP Port Unreachable to the other code's UDP request, this doesn't happen if I just have the code do a recv() immediately. I think it has something to do with the Broadcast flag, I send out Broadcast then recv() non-Broadcast data, I don't think the same socket can do this? I'm going to try closing the socket (if that's a real thing) and initializing a listener for non-broadcast on the local port.

# ? Oct 22, 2008 16:15

Ninja Rope: Oct 22, 2005; Wee.

JawnV6 posted:

I'm working on ubuntu 8.04. "Hang" was a terrible word, it's not picking up the data. On my network tracer I'm seeing the computer send out a ICMP Port Unreachable to the other code's UDP request, this doesn't happen if I just have the code do a recv() immediately. I think it has something to do with the Broadcast flag, I send out Broadcast then recv() non-Broadcast data, I don't think the same socket can do this? I'm going to try closing the socket (if that's a real thing) and initializing a listener for non-broadcast on the local port.

I don't want to talk out my rear end without testing it, but I'm pretty sure that's not necessary. I've got some broadcast socket code somewhere I'll try and dig up later today and let you know how I got that working.

# ? Oct 22, 2008 16:36

Mithaldu: Sep 25, 2007; Let's cuddle.

syphon^2 posted:

What am I doing wrong?

His example is retarded. Sleep always locks no matter what you do. In fact, the creator of this is retarded. (Yelled at me for "use warnings;" in my code and, in his words, claimed only "lesser people" need warnings.) However his code is solid.

What this thing does is basically this:

It keeps a list of all coroutines waiting in the script, with the main routine being a coroutine in itself. Each coroutine has a timer and a ready state switch. When you create a coroutine it is added to the list, as soon as you switch its state to ready, the timer starts running. Now your program proceeds as normal until you either call cede() or schedule().

cede() puts the currently running coroutine into the list, switches it to ready, then selects the coroutine that has been waiting the longest and either starts running it, or continues running it after its last cede(). schedule() does something similar, only it doesn't switch the current coroutine to ready.

Yes, this isn't actual multi-threading, since it pretty much only runs one thread on one CPU. However, if you have proper code design, which means you have a sufficient amount of schedules in your loops, and a half-way decently designed idle loop controller that switches coroutines back to ready state when they're needed, you pretty much have actual parallel processing, only it runs on one core only, instead of 2.

The nice thing here for you is this: Sockets block because they keep waiting for a response. However, now take a socket implementation that calls schedule() after every loop in its response waiter. Suddenly your main program gets all the CPU it needs to keep running and launch more sockets and all it needs to do different is call schedule often enough to have the controller reactivate the sockets so they can check if there was an answer.

The main point here is: Sockets don't lock because they keep the CPU busy, but because they're idling until something happens. What you want is to do something during the time they spend idling. Coro::Socket makes that possible.

If you're interested, you can check here for a life example: http://dwarvis.googlecode.com/svn/trunk/lifevis/Lifevis/Viewer.pm

Mithaldu fucked around with this message at 16:55 on Oct 22, 2008

# ? Oct 22, 2008 16:51

Erasmus Darwin: Mar 6, 2001

syphon^2 posted:

Huh, threading/coroutines must be beyond me. I looked over this, and their example seems nothing more than an elaborate goto(), it doesn't run things asynchronously.

I think you're right. It does look like it handles some nifty things, but it's still at the mercy of your code providing a "cede" to switch contexts. So when you're blocking on a sleep or the registry lookup code, there's no way your code can do a cede to let your other code run. In short, I don't think this fits your needs.

I think you're probably better off using Win32::Process like you were thinking of before.

Mithaldu posted:

The main point here is: Sockets don't lock because they keep the CPU busy, but because they're idling until something happens. What you want is to do something during the time they spend idling. Coro::Socket makes that possible.

That obviously requires the sockets in use to be Coro::Sockets. However, since syphon^2 is relying on Win32::TieRegistry to abstract away the networking details, he doesn't have access to the sockets being created. Then again, I suppose he could go digging into Win32::TieRegistry's code and create his own Coro::Socket-based version. Seems like more trouble that just doing multiple processes, though.

# ? Oct 22, 2008 17:00

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 20:31

syphon^2: Sep 22, 2004

Erasmus Darwin posted:

I think you're probably better off using Win32::Process like you were thinking of before.

Doing it this way means I have to rip out a bunch of code an put it in a separate .pl file though.

(or can Win32::Process::Create execute Perl code within the same Perl executable?)

EDIT: I just thought of something else... Win32::Process::Create is like fork, in that I can't share variables, right?

# ? Oct 22, 2008 17:52

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > The Perl Short Questions Megathread: executable line noise

«‹›72 »