The Perl Short Questions Megathread: executable line noise

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > The Perl Short Questions Megathread: executable line noise

«‹›72 »

slipped: Jul 12, 2001

mit_senf posted:

What's the most efficient way to do multiple substitutions at once? There's lots of information online about doing multiple matches at once, but I can't find anything about substitutions.

For example, is it possible to do something like this:
code:
$text =~ s/string1/string2/g;
$text =~ s/string3/string4/g;
But do both substitutions at the same time, so that perl is not scanning the same text twice in a row?

21:54 < f00li5h> s/%$(@{[ join '|', sort keys %hash ]})$/$hash{$1}/g

# ? Apr 20, 2009 02:57

Adbot: ADBOT LOVES YOU

# ? May 21, 2024 16:15

dizzywhip: Dec 23, 2005

Thanks for the replies, seems to work well.

Triple Tech posted:

Does it really matter?

I would think so, I mean if you have a large number of substitutions to be made in a lengthy piece of text, it seems pretty inefficient and silly to go through the entire text for each individual substitution.

# ? Apr 20, 2009 04:27

Paradox: May 19, 2003

What are the benefits to using DBIx::Class over just straight DBI?

I've been using DBI to interact with databases for quite a while now and I've never had any complaints with it. It is fast and I can usually coerce it into returning the exact data structure that I want.

I've heard many people on the internet talk about how great DBIx::Class is and ORMs in general but from reading the documentation it just seems to replace a SQL string with a series of function calls. I admit that I haven't looked too deeply into it but it just seems to add another layer on top of DBI, slowing things down and limiting my ability to use interesting features of the database (for instance, the application I work on uses Oracle Spatial. I can't imagine that DBIx:Class has support for that.).

So what am I missing that makes this such a killer module that internet folk rave about?

# ? Apr 20, 2009 22:18

Mithaldu: Sep 25, 2007; Let's cuddle.

I'd be interested in hearing that too. Personally I've never seen any sort of benefit to the "let's do SQL in Perl" approach. I already know SQL syntax and it does what it does well, so there doesn't seem to be any sense in learning PerlSQL syntax.

The only thing I do miss in DBI is that there's no easy way to get an application to auto-initialize SQL tables when they don't exist yet, not to speak of automatically updating them in user environments if there were changes in development. However I haven't seen any indications that DBIx::Class can do that either.

# ? Apr 20, 2009 22:28

Triple Tech: Jul 28, 2006; So what, are you quitting to join Homo Explosion?

It's just an abstraction layer, albeit a really wonky one. If you're doing ORM and nothing complicated, it's probably the best the community has to offer, right? There was an article somewhere that was like... If you try to abstract everything that SQL can do, you will end up reinventing SQL. And I totally believe that the more I work with Perl and SQL. I guess the only benefit is (hoping) not having to redo your SQL layer in case you switch engines. And it also keeps the code relatively clean, I guess; Perl here, SQL there. It's kinda wonky. I haven't spent much time with it because I'm not really doing ORM, but ETL. I made my own Fey-esque SQL interface. My current, 5-second belief is that ORM layers are great for ORM, but for Perl-SQL, it should be Fey-ish.

# ? Apr 20, 2009 23:04

Mithaldu: Sep 25, 2007; Let's cuddle.

Do you have a short, down-to-earth explanation on ORM is? :shobon:

# ? Apr 20, 2009 23:10

Triple Tech: Jul 28, 2006; So what, are you quitting to join Homo Explosion?

Mithaldu posted:

Do you have a short, down-to-earth explanation on ORM is?

ORM = "I want to create, retrieve, update, and delete objects and not care how this process is implemented."

It's object-relational mapping, literally figuring out how an object-based universe is implemented atop a relational database. Pretty simple except maybe subclasses.

# ? Apr 21, 2009 00:12

leedo: Nov 28, 2000

I recently used DBIx::Class to define a fairly complex schema (~20 tables, each with 2-5 relationships), and it worked out really well. I wrote the entire schema in Perl first and used the create_ddl_dir method to export it out the SQL schema. This way I could export out very basic SQL for SQLite on my local machine, and SQL with the constraints supported by MySQL in production. It also handles diffs between schema revisions, so I could easily upgrade a DB without wiping everything out.

I also really enjoy not having to write out queries by hand, but that speaks more to my dislike of writing SQL than anything else.

# ? Apr 21, 2009 01:00

Mithaldu: Sep 25, 2007; Let's cuddle.

Triple Tech posted:

ORM = "I want to create, retrieve, update, and delete objects and not care how this process is implemented."

It's object-relational mapping, literally figuring out how an object-based universe is implemented atop a relational database. Pretty simple except maybe subclasses.

Ah, thanks. No wonder i never had any contact with it, given my stance on OOP.

# ? Apr 21, 2009 13:03

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

Someone doesn't like OO? Lemme at him..

In other news, gently caress yeah Devel::Declare:

code:

my $dispatcher = dispatcher {
    on foo {
        push @calls, 'foo';
    }
    on ['bar', 'baz'] {
        push @calls, 'bar baz';
    }
    on /help/dispatch {
        push @calls, 'help dispatch';
    }
};

# ? Apr 21, 2009 13:14

Mithaldu: Sep 25, 2007; Let's cuddle.

I don't dislike it, just think it's very often massively overused.

For example, it's absolutely fine how DBI uses it, but it's a bit retarded to make every string an object in itself.

(That and noone who uses it seems capable of talking in plain english, which makes me want to punch people.)

# ? Apr 21, 2009 13:50

Triple Tech: Jul 28, 2006; So what, are you quitting to join Homo Explosion?

Well, if there are a series of string-related functions whose first argument is a string... Then it sorta makes sense.

code:

$scalar.concat($beebop);
@list.push();
say for %entities.values;

# ? Apr 21, 2009 14:20

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

I would much rather have chomp, length, lc, etc. as methods on strings than unhidable keywords in every namespace.

lc on an array reference is not at all useful. That should throw an error (such as "method not found") instead of doing what it currently does. If lc were a method, someone could define a subclass of array reference with an lc method that, say, returns a new array reference whose elements have been lowercased. But no, since it's a keyword, it's basically immutable.

Making the builtins generic functions would also solve this, with the added benefit of not having to rewrite every Perl program to use method calls for string operations.

I wish there were a language that didn't suck.

# ? Apr 21, 2009 14:49

S133460: Mar 17, 2008; hello

Sartak posted:

I would much rather have chomp, length, lc, etc. as methods on strings than unhidable keywords in every namespace.

lc on an array reference is not at all useful. That should throw an error (such as "method not found") instead of doing what it currently does. If lc were a method, someone could define a subclass of array reference with an lc method that, say, returns a new array reference whose elements have been lowercased. But no, since it's a keyword, it's basically immutable.

What's to keep you from having an lc method for an array reference?

code:

package My::ListRef;
sub lc { [map(lc $_, @{$_[0]})] };

package main;
$a = bless [qw/A B C/], "My::ListRef";
print @{$a->lc()}

Or you can use something wacky like autobox::Core.

And if you really want to break a builtin, it can be done for some...

code:

BEGIN { *CORE::GLOBAL::lc = sub { die } }

I agree though. It would be nice if you had to say something like "use CORE qw/:string/" to get lc and chomp.

# ? Apr 21, 2009 18:46

Mario Incandenza: Aug 24, 2000; Tell me, small fry, have you ever heard of the golden Triumph Forks?

Am working with Unicode text files. To reliably get UTF16 I have to use the crazy open mode of:

code:

open my $fh, '<:raw:encoding(UTF-16LE):crlf:utf8', $filename;

But this then breaks ANSI and UTF8 text files so I need to close and re-open using a different mode, depending on whether the file starts with a BOM. Is there no open() mode string that will magically support any encoding without having to jump through hoops?

# ? Apr 21, 2009 19:28

S133460: Mar 17, 2008; hello

atomicstack posted:

Am working with Unicode text files. To reliably get UTF16 I have to use the crazy open mode of:
code:
open my $fh, '<:raw:encoding(UTF-16LE):crlf:utf8', $filename;
But this then breaks ANSI and UTF8 text files so I need to close and re-open using a different mode, depending on whether the file starts with a BOM. Is there no open() mode string that will magically support any encoding without having to jump through hoops?

You can change the perlio layers with binmode.

# ? Apr 21, 2009 20:32

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

satest3 posted:

What's to keep you from having an lc method for an array reference?

Good point.

A better example of the problem is subclassing strings. For others to use your string subclass, they would have to switch from functions to method calls. They'd have to remember if they were dealing with a builtin string or a custom string every time they wanted to manipulate it. Overloading wouldn't really work except for the simplest stuff like length.

It'd be less work if strings used methods to begin with. I could just pass my string subclass objects to some other code and it would just do the right thing because it's already using method calls.

satest3 posted:

Or you can use something wacky like autobox::Core.

autobox is great (I've certainly done my share of evil with it) but it is limited to a lexical scope. Once you're outside that scope, values are inert again. That's good for safety reasons (you don't have to rewrite any existing code to handle builtin types suddenly having methods).

If I'm writing code that expects a string subclass, then I could use autobox to handle regular strings too. But nobody does that.

What I think I want is to permanently autobox values that are created in some (possibly dynamic, probably lexical) extent. Actually, no. That sounds pretty fragile.

satest3 posted:

code:

BEGIN { *CORE::GLOBAL::lc = sub { die } }

Some builtins, but not all. You have to worry about getting this code to run early, because this redefinition won't affect code that's already compiled. It's also horrible in that it will silently override anyone else's modifications. Generic functions would solve this.

It'd just be nice if OO were more deeply-rooted in Perl. What I want is Perl 6 I guess.

# ? Apr 21, 2009 23:04

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

Matt Trout encourages every one of us to blog about Perl.

I love this guy sometimes.

# ? Apr 22, 2009 00:48

syphon: Jan 1, 2001

Heh, way back in the day when I started learning Perl with O'Reilly's Learning Perl, I heard a couple people tell me that PHP had largely superseded it, and that it was a dying language. Not knowing any better, I echo'd their comments on my livejournal. Apparenly, Randall Schwartz (one of the authors of Learning Perl, Programming Perl, and several other books) trolls LJ for certain keywords, because he commenting on my post calling me an ignorant douchebag (which was pretty much correct). I felt honored.

# ? Apr 22, 2009 23:38

Fenderbender: Oct 10, 2003; You have the right to remain silent.

Hahaha, when I started using Perl I got into an argument with merlin on PerlMonks about sorting algorithms, unbeknownst to me that merlin = Randall Schwartz. :v:

# ? Apr 22, 2009 23:42

father vivian: May 5, 2008; WE'VE OBTAINED ENERGY RECOVERY PILLS

perl n00b question for ya'll

finishing up my first class in perl and its def become my fav. There is an optional project that i want to do and am looking for some ideas. its pretty open ended, basically just need to write a nifty lil cgi/perl script that hasn't been covered in class.

im NOT trying to weasel my way into someone doing this for me! just lookin for a few ideas or a point in the right direction for something cool i can put together. we've done basic stuff like cookies, redirection, form validation, sendmail etc so maybe something along that experience level?

any thoughts would be much appreciated thx

# ? Apr 25, 2009 06:39

Rapportus: Oct 31, 2004; The 4th Blue Man

Triple Tech posted:

Does it really matter?

Try benchmarking a hash lookup with the /e switch. s/foo|bar/lookup($1)/ge (untested)

I saw this and applied it to a project at work this week, where I needed to perform many substitutions on a line, but also store the before/after result of that line and associate it to each pair of before/after tokens involved in the substitution.

code:

$re = '(' . (join '|', keys %{$hashref}) . ')';
$re = qr/$re/;
$line =~ s/$re/process($1,$before,\$line)/geo;
..
sub process
{
   my ($var, $before, $after) = @_;
   push( @{$changes->{$var}}, { $before => $after });
   return $hashref->{$var};
}

Using a hashref like this:

code:

$hashref->{'hello'} = "HELLO";
$hashref->{'world'} = "WORLD";

The output of the code above with the line "helloabcworld":

code:

Before: "helloabcworld"
After: "HELLOabcWORLD"
Data::Dumper output of $changes:
$VAR1 = {
          'hello' => [
                       {
                         'helloabcworld' => \'HELLOabcWORLD'
                       }
                     ],
          'world' => [
                       {
                         'helloabcworld' => $VAR1->{'hello'}[0]{'helloabcworld'}
                       }
                     ]
        };

The only catch is $line must be a new memory location on each iteration, so the reference is stored correctly. Also if you have duplicate replacements on the same line you'll get multiple entries in $changes (this project guaranteed at most one occurrence of any match on a line).

# ? Apr 25, 2009 07:58

Mithaldu: Sep 25, 2007; Let's cuddle.

Scentaur posted:

perl n00b question for ya'll

So basically you're looking for a problem to solve? As an off-the-wall thing, maybe do a very simplicistic open-access and anonymous forum? That should be doable and cover a lot of base. Alternatively, get ActivePerl, Wx and write a tetris clone.

Anyhow, question of my own: In making my apps more concise I'd like to move the sql queries into their own class, so i only need to do

code:

return $c->dbh->selectall_hashref(
	$query->{get_blueprint_types},
	'typeID',
	undef,
	$blueprintTypeID
);

in my relevant functions. Naively speaking a simple exported hash would be enough, but sometimes and especially in cases of inserts, it's better to create them programmatically. Are there any good or suggested modules for this kinda stuff? Or should i just go with a hash for simple cases and handle the complex ones case-by-case?

# ? Apr 25, 2009 11:31

Mithaldu: Sep 25, 2007; Let's cuddle.

Looked around a while and found nothing that would solve this problem, so I decided to create my own. Code would be a bit much to post here, so i did on perlmonks: http://perlmonks.org/?node_id=760006

If anyone's interested, please have a look and yell at me about it.

# ? Apr 25, 2009 14:30

Mario Incandenza: Aug 24, 2000; Tell me, small fry, have you ever heard of the golden Triumph Forks?

Did you take a look at SQL::Abstract? It underpins DBIx::Class.

# ? Apr 25, 2009 17:43

Mithaldu: Sep 25, 2007; Let's cuddle.

While not a bad idea, it goes down a road i don't care about.

I only realized i was miscommunicating my goal after talking to Sartak a bit: My main focus is not on the creation of queries. I'm good at that in pure SQL and only need to supplant it a bit with a minimum amount of dynamic parts.

My main focus is: How can i store this query somewhere that is NOT my main code in such a manner that i can simply call onto it by name. The storage and access part here is what i'm trying to solve.

In other words, how do i best go about making this:

code:

$query = "
            SELECT T2.typeID, T2.typeName, AMV.quantity, recycle
                FROM typeActivityMaterials AMV
                INNER JOIN invTypes T1 on T1.typeID = AMV.typeID
                INNER JOIN invTypes T2 on T2.typeID = AMV.requiredTypeID 
                INNER JOIN ramActivities A on A.activityID = AMV.activityID
                INNER JOIN invGroups G on G.groupID = T2.groupID
                INNER JOIN invCategories C on C.categoryID = G.categoryID
            WHERE
                AMV.typeID = ?
                AND AMV.activityID = 1
                AND C.categoryName != 'Skill'
                AND C.categoryName != 'Commodity'
                AND AMV.quantity != 0
";
return $c->dbh->selectall_hashref( $query, 'typeID', undef, $blueprintTypeID );

into this:

code:

return $c->dbh->selectall_hashref(
    get_query('get_reprocessing_list'), 'typeID', undef, $blueprintTypeID
);

# ? Apr 25, 2009 18:04

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

Mwa ha ha. Data::Phrasebook::SQL. You can use Data::Phrasebook directly if you don't want it to own your dbh.

# ? Apr 25, 2009 18:27

Mithaldu: Sep 25, 2007; Let's cuddle.

After talking in IRC, yes, D::B looks pretty good. It's templating is a tad barebone, but as the structure underneath is good, i can fix that if needed by slapping on an interface to a templating system.

Also thanks to Sartak for explaining subclassing in Perl. Pretty nifty.

# ? Apr 25, 2009 19:02

huge sesh: Jun 9, 2008

Not sure if it would be better to ask in the reading topic but do yall remember a book about writing more perl-ish perl? It's kind of short, available as a pdf free from the author's site. I wish I could remember the name, my perl is terrible.

# ? Apr 25, 2009 19:17

Mithaldu: Sep 25, 2007; Let's cuddle.

This maybe? http://hop.perl.plover.com/

# ? Apr 25, 2009 19:24

huge sesh: Jun 9, 2008

Mithaldu posted:

This maybe? http://hop.perl.plover.com/

Yes, thank you.

# ? Apr 25, 2009 19:30

leedo: Nov 28, 2000

Scentaur posted:

perl n00b question for ya'll

finishing up my first class in perl and its def become my fav. There is an optional project that i want to do and am looking for some ideas. its pretty open ended, basically just need to write a nifty lil cgi/perl script that hasn't been covered in class.

im NOT trying to weasel my way into someone doing this for me! just lookin for a few ideas or a point in the right direction for something cool i can put together. we've done basic stuff like cookies, redirection, form validation, sendmail etc so maybe something along that experience level?

any thoughts would be much appreciated thx

I'd suggest choosing something you are already interested in. Do you like music? If so take a look at the last.fm API and try doing something with that. Whatever you are interested in there is probably a CPAN module that deals with it, which you could then tie together in a CGI script.

# ? Apr 25, 2009 21:24

Mithaldu: Sep 25, 2007; Let's cuddle.

After two days of frustrating experimenting, I'm hoping that one of you can help me with this. Ahead: Please only post if you have a solution you know works, as any suggestion you may give I've likely tried already.

As such, here's the question: How would i go about downloading multiple files in parallel
- under Win32 AND Linux
- without having it crash due to too many leaked scalars
- without using a GB of ram
- without being incredibly slow when downloading

Edit: Got solution elsewhere. This seems to work fine as far as i can tell.

code:

    use IPC::Open2;

    for my $id  (@ids) {
        $wgets++;
        
        push @pids, open2(undef, undef, 'wget', $url.$id, '-q', '-O', $dir.$id);
        
        while ( @pids >= 10 ) {
            waitpid( shift @pids, 0 );
        }
    }

    while ( @pids ) {
        waitpid( shift @pids, 0 );
    }

Mithaldu fucked around with this message at 14:49 on Apr 29, 2009

# ? Apr 29, 2009 13:39

Polygynous: Dec 13, 2006; welp

I'm trying to parse command line arguments in a script and I seem to be missing something. I want to have an option -l that takes one or two arguments, like '-l 2009' or '-l 2008 foo', and this is what I have with some debugging output:

code:

$ok = 1;
GETARGS: while ($_ = shift) {
  print ":$_\n";
  ($cmd) = /^-([lmnop])$/;
  $_ = shift || do { $ok = 0; last GETARGS; };
  ($n) = /^(\d+)$/;
  if ($n) {
    print "#$n\n";
    for ($cmd) { # "switch"
      /l/ && do {
        $opt_l = 1;
        $name = shift; # get name or next opt
        print "$name\n";
        # if an opt, restart loop to process it
        if ($name =~ /^-/) { $_ = $name; redo GETARGS; }
      };
      # handle mnop here
    }
  } else {
    $ok = 0; last GETARGS;
  }
}

which gives me this baffling result with '-l 2009 -n 2':
:-l
#2009
-n
:2009 <- wtf
#2

The '-n' seems to just get lost somehow. It's possible I'm missing some scoping issue with my makeshift switch block (I swear I got it from the manpage but I can't find it now). I realize I could just require the arguments be put in quotes and use getopt, but I'd apprectiate if anyone could help me understand what's happening here.

# ? Apr 30, 2009 02:06

slipped: Jul 12, 2001

stop reinventing the wheel http://perldoc.perl.org/Getopt/Long.html

# ? Apr 30, 2009 03:03

Polygynous: Dec 13, 2006; welp

huh. I didn't notice that could handle multiple values when I looked at it earlier. I should be able to make that work. Thanks, I guess, though I kind of still would like to know what's going wrong with what I had.

vv Thanks, that was bugging the hell out of me. I found an old version of the manpage and it even mentions the assignment to $_ is temporary. :doh:

Polygynous fucked around with this message at 18:37 on Apr 30, 2009

# ? Apr 30, 2009 15:41

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

spoon0042 posted:

The '-n' seems to just get lost somehow. It's possible I'm missing some scoping issue with my makeshift switch block (I swear I got it from the manpage but I can't find it now). I realize I could just require the arguments be put in quotes and use getopt, but I'd apprectiate if anyone could help me understand what's happening here.

That's exactly what's happening; for implicitly localizes its iteration variable, which is this case is $_.

# ? Apr 30, 2009 18:03

Vanadium: Jan 8, 2005

So I am finally trying to learn proper perl after a couple of years of casually loving up any perl code I touched, and working through the interesting manpages.

In perlmod, there is a section "Making your module threadsafe", which seems to be about defining CLONE subroutines. But I am not sure whether I understand when I have to care about that, or even what the hell is going on with perl's threads/forking at all.

Do I need to figure out what to do for any module or write? Is that only for extension modules or something? Where do I get a more comprehensive idea about all the involved concepts?

# ? May 3, 2009 17:12

Filburt Shellbach: Nov 6, 2007; Apni tackat say tujay aaj mitta juu gaa!

I have never once cared about threading or CLONE. I think you're okay to skip it until you, or someone who pays you, begins caring about threads.

You don't need to do anything special to "handle" forks. Well, outside of the usual concerns, like open filehandles.

# ? May 3, 2009 17:29

Adbot: ADBOT LOVES YOU

# ? May 21, 2024 16:15

Mithaldu: Sep 25, 2007; Let's cuddle.

I don't know a lot about threads, but i know this much: The best way of handling threads is literally this: Fork/Create as early as loving possible, if possible in a BEGIN loop before anything else and have communication only happen through shared scalars. Reason for that being that Perl does those things by duplicating your entire application, which however does not work entirely perfect as far as creating actual seperate duplicates goes, nor does it bode well for the memory use of your application.

# ? May 3, 2009 17:44

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > The Perl Short Questions Megathread: executable line noise

«‹›72 »