Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Notorious b.s.d.
Jan 25, 2003

by Reene

fritz posted:

is there any reason to use awk instead of perl

i remember back when i started in the perl4 days one of its big selling points was that it was an awk- and sed-killer and thats still all i use it for, all " perl -pi.back -e 's/buttz/butts/g;' "

there is absolutely no reason to use awk instead of perl. none.

anyone who tells you to use awk either hasn't learned anything new in 20 years, doesn't know perl, or most probably, both. i'm going to give mononqc the benefit of the doubt and assume he's in the "doesn't know perl" category.

Adbot
ADBOT LOVES YOU

Nomnom Cookie
Aug 30, 2009



qntm posted:

a way back but: don't use double quotes, just backslash-escape every non-alphanumberic character including spaces

my solution is dont use bash

Notorious b.s.d.
Jan 25, 2003

by Reene

PrBacterio posted:

dont worry awk is terrible too, it's snobol where its really at so you fit right in!


I really do wonder though how regex based languages became so utterly dominant to the point that the "regexes" they parse arent even regular anymore but some sort of hybrid hellspawn of backtracking and exponential running time, when context free grammar parsing has been a solved problems for so long now. I mean obviously snobol and its successors are poo poo too, but why havent any modern, usable CFG based parsing languages been invented for whats literally decades now, when its quite obvious to anyone with a brain that regexes arent really what people actually want or need for parsing in most cases as, for one, what people call regexes nowadays actually arent

extended regex languages became popular because unix users were already obliged to know regex to use basic tools, so the extended languages were incremental and easy to learn

learning a brand new way express grammars or whatever is a massive barrier to entry. not gonna catch on with the oneliner crowd.

PrBacterio
Jul 19, 2000

Notorious b.s.d. posted:

extended regex languages became popular because unix users were already obliged to know regex to use basic tools, so the extended languages were incremental and easy to learn

learning a brand new way express grammars or whatever is a massive barrier to entry. not gonna catch on with the oneliner crowd.

so in short because peoplecomputerseverything is terrible, got it




:(

MononcQc
May 29, 2007

Notorious b.s.d. posted:

there is absolutely no reason to use awk instead of perl. none.

anyone who tells you to use awk either hasn't learned anything new in 20 years, doesn't know perl, or most probably, both. i'm going to give mononqc the benefit of the doubt and assume he's in the "doesn't know perl" category.

Don't know Perl, only used it in some homework to do conway's game of life and then never touched it again. Again Awk takes 15 minutes to learn (my post covers a huge part of the basics on its own). I'm also interested in comparing the speed of both, because when parsing 3-4GB files, a good speedup is worth the 15 minutes investment.

PrBacterio
Jul 19, 2000

MononcQc posted:

Don't know Perl, only used it in some homework to do conway's game of life and then never touched it again. Again Awk takes 15 minutes to learn (my post covers a huge part of the basics on its own). I'm also interested in comparing the speed of both, because when parsing 3-4GB files, a good speedup is worth the 15 minutes investment.

does awk use "extended" or regular (hah!) regexes?
because if it uses just plain old regular expressions, thats probably where the speedup comes from

MononcQc
May 29, 2007

PrBacterio posted:

does awk use "extended" or regular (hah!) regexes?
because if it uses just plain old regular expressions, thats probably where the speedup comes from

http://www.math.utah.edu/docs/info/gawk_5.html#SEC28 seems to say it supports both POSIX and extended 'unix stuff', so probably not PCRE. I guess that's where the speedup is, yeah.

Jonny 290
May 5, 2005



[ASK] me about OS/2 Warp
http://perldoc.perl.org/perlcheat.html is printed out on my cube wall and will never leave. I still glance at it several times a day

Mr SuperAwesome
Apr 6, 2011

im from the bad post police, and i'm afraid i have bad news

mn9 posted:

i think pycharm is pretty cute. they also released a free version somewhat recently that has most of the functionality that you would need unless you do a lot of web stuff.

cool so my options are pycharm, komodo and wing, ty dudes i shall ~investigate~

(and somehow figure out a parallels setup so i can do the dev on mac but just compile it on windows in a vm thing :getin: srsly i love deving on my mbp)

JewKiller 3000 posted:

what is a python ide going to do for you that vim doesn't? i guess you can have one window containing code, a directory tree, and a console, great. what about any of the actual features that make ides useful? you're limited by python to no more than a gui for grep

well replace vim for sublime text because i dont have autism but essentially an intellisense clone. all good the poo poo vs and xcode do that sublime doesnt

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder
there's a reason objective c/cocoa doesnt have a native regex library

Notorious b.s.d.
Jan 25, 2003

by Reene

MononcQc posted:

Don't know Perl, only used it in some homework to do conway's game of life and then never touched it again. Again Awk takes 15 minutes to learn (my post covers a huge part of the basics on its own). I'm also interested in comparing the speed of both, because when parsing 3-4GB files, a good speedup is worth the 15 minutes investment.

if parsing 4 GB of logs takes you more than 15 minutes, invest in some hardware from this decade first, worry about code later

seriously on any PC from after like 2003 you should have no problem churning through 100 MB/s with naive perl regex. what the gently caress is wrong with your setup?

Notorious b.s.d.
Jan 25, 2003

by Reene

USSMICHELLEBACHMAN posted:

there's a reason objective c/cocoa doesnt have a native regex library

it's penny ante crap developed at minimum cost to sell hardware?

Juul-Whip
Mar 10, 2008

yo is it just me or is git the absolute worst loving version control system

also is there a decent way to merge pbxproj yet because jesus christ

Deacon of Delicious
Aug 20, 2007

I bet the twist ending is Dracula's dick-babies

Notorious b.s.d. posted:

if parsing 4 GB of logs takes you more than 15 minutes, invest in some hardware from this decade first, worry about code later

seriously on any PC from after like 2003 you should have no problem churning through 100 MB/s with naive perl regex. what the gently caress is wrong with your setup?

for a discussion about parsing you aren't doing it very well

the 15 minute investment isn't for every single time you use awk, it's learning awk the one time and then you potentially save time with each file you parse with awk in the future

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

THC posted:

yo is it just me or is git the absolute worst loving version control system

it's better than subversion, and better than clearcase at many things

MononcQc
May 29, 2007

Deacon of Delicious posted:

for a discussion about parsing you aren't doing it very well

the 15 minute investment isn't for every single time you use awk, it's learning awk the one time and then you potentially save time with each file you parse with awk in the future

Exactly this.

JewKiller 3000
Nov 28, 2006

by Lowtax

Mr SuperAwesome posted:

well replace vim for sublime text because i dont have autism but essentially an intellisense clone. all good the poo poo vs and xcode do that sublime doesnt

that's what i'm saying though, it's not going to do that, because it can't. there is no static type information

a cyberpunk goose
May 21, 2007

THC posted:

yo is it just me or is git the absolute worst loving version control system

look at this guy

uG
Apr 23, 2003

by Ralp

http://search.cpan.org/~nwclark/perl-5.8.3/pod/perl.pod posted:

If you have a problem that would ordinarily use sed or awk or sh, but it exceeds their capabilities or must run a little faster, and you don't want to write the silly thing in C, then Perl may be for you.

hubris.height
Jan 6, 2005

Pork Pro
writing that thread parser, and i'm working on just getting a page to be pulled from the server, and search through for any href links containing the string 'youtube', then saving them to a txt file that i will use to make the playlist later. i can't seem to pull more than just the php head, it doesn't get the whole page. how do i work with this?

trigger warning: ugly loving code

code:
        static void Main(string[] args)
        {
            HtmlDocument doc = new HtmlDocument();
            HtmlAgilityPack.HtmlWeb docHFile = new HtmlWeb();
            var tubes = new List<string>();
            doc = docHFile.Load("http://forums.somethingawful.com/showthread.php?threadid=3540185&userid=0&perpage=40&pagenumber=2");
            using (System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\\Users\\Minerva\\testfolder\\site.txt"))
            {
                foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[@href]"))
                {
                    file.WriteLine(link.Attributes["href"].Value);
                }
            }
            foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[@href]"))
            {
                HtmlAttribute att = link.Attributes["href"];
                if (att.Value.Contains("youtube") == true){
                    tubes.Add(att.Value);
                }
            }

            using (System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\\Users\\Minerva\\testfolder\\tubes.txt"))
            {
                foreach (string line in tubes)
                {
                    file.WriteLine(line);
                }
            }
        }
results in the following text file:
http://pastebin.com/urbzxf2J

Bloody
Mar 3, 2013

hubris.height posted:

writing that thread parser, and i'm working on just getting a page to be pulled from the server, and search through for any href links containing the string 'youtube', then saving them to a txt file that i will use to make the playlist later. i can't seem to pull more than just the php head, it doesn't get the whole page. how do i work with this?

trigger warning: ugly loving code

code:
        static void Main(string[] args)
        {
            HtmlDocument doc = new HtmlDocument();
            HtmlAgilityPack.HtmlWeb docHFile = new HtmlWeb();
            var tubes = new List<string>();
            doc = docHFile.Load("http://forums.somethingawful.com/showthread.php?threadid=3540185&userid=0&perpage=40&pagenumber=2");
            using (System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\\Users\\Minerva\\testfolder\\site.txt"))
            {
                foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[@href]"))
                {
                    file.WriteLine(link.Attributes["href"].Value);
                }
            }
            foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[@href]"))
            {
                HtmlAttribute att = link.Attributes["href"];
                if (att.Value.Contains("youtube") == true){
                    tubes.Add(att.Value);
                }
            }

            using (System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\\Users\\Minerva\\testfolder\\tubes.txt"))
            {
                foreach (string line in tubes)
                {
                    file.WriteLine(line);
                }
            }
        }
results in the following text file:
http://pastebin.com/urbzxf2J

try logging in

Notorious b.s.d.
Jan 25, 2003

by Reene

Deacon of Delicious posted:

for a discussion about parsing you aren't doing it very well

the 15 minute investment isn't for every single time you use awk, it's learning awk the one time and then you potentially save time with each file you parse with awk in the future

i'm parsimonious in my parser selection

perl is useful for everything under the sun. if i write a quick dirty one liner i can re-use it elsewhere. awk is 1980s crap. even if it manages to be slightly faster at splitting on a tab, it's a less capable tool and no one else knows it.

MononcQc
May 29, 2007

yeah but perl is a p-lang

power botton
Nov 2, 2011

perl-lang.

p-erlang?

Bloody
Mar 3, 2013

MononcQc posted:

yeah but perl is a p-lang

yeah but using it for dumb little scripts is exactly the job of a p-lang

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

MononcQc posted:

yeah but perl is a p-lang

a powerful and productive language

Doc Block
Apr 15, 2003
Fun Shoe

USSMICHELLEBACHMAN posted:

there's a reason objective c/cocoa doesnt have a native regex library

autolayout has its own expression language and that's part of the reason autolayout can be such a drat pain.

Doc Block fucked around with this message at 23:51 on Oct 23, 2013

MononcQc
May 29, 2007

I'm just surprised Notorious b.s.d. isn't telling people to do it in Java or something

Stringent
Dec 22, 2004


image text goes here

MononcQc posted:

I'm just surprised Notorious b.s.d. isn't telling people to do it in Java or something

in lieu of that its p hilarious that someone got hubris to install vs to write a scraper in c#

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Stringent posted:

in lieu of that its p hilarious that someone got hubris to install vs to write a scraper in c#


using (System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\\Users\\Minerva\\testfolder\\site.txt"))

lmfao is that how you open a file for writing in c#

Posting Principle
Dec 10, 2011

by Ralp
you can actually just say

code:
using (var file = File.AppendText("c:\\myfile.bar"))
{
	file.WriteLine("im gay");
}

tef
May 30, 2004

-> some l-system crap ->

PrBacterio posted:

I really do wonder though how regex based languages became so utterly dominant to the point that the "regexes" they parse arent even regular anymore but some sort of hybrid hellspawn of backtracking and exponential running time, when context free grammar parsing has been a solved problems for so long now. I mean obviously snobol and its successors are poo poo too, but why havent any modern, usable CFG based parsing languages been invented for whats literally decades now, when its quite obvious to anyone with a brain that regexes arent really what people actually want or need for parsing in most cases as, for one, what people call regexes nowadays actually arent

because debugging lalr is painful, recursive descent is easy, and ll is really loving trivial and lr isn't. actual cfg parsing is still slow and clumsy.

PrBacterio
Jul 19, 2000

tef posted:

because debugging lalr is painful, recursive descent is easy, and ll is really loving trivial and lr isn't. actual cfg parsing is still slow and clumsy.
you mean slow and clumsy like PCRE's are?

Brain Candy
May 18, 2006

Notorious b.s.d. posted:

i'm parsimonious in my parser selection

perl is useful for everything under the sun. if i write a quick dirty one liner i can re-use it elsewhere. awk is 1980s crap. even if it manages to be slightly faster at splitting on a tab, it's a less capable tool and no one else knows it.

if you write a quick dirty one liner, you throw it the gently caress away. in the loving trash. you don't turn it inside out and use it again.

the plural of lovely hacks is not library

hubris.height
Jan 6, 2005

Pork Pro

Posting Principle posted:

you can actually just say

code:
using (var file = File.AppendText("c:\\myfile.bar"))
{
	file.WriteLine("im gay");
}

neat

Dessert Rose
May 17, 2004

awoken in control of a lucid deep dream...

Brain Candy posted:

the plural of lovely hacks is not library

Nomnom Cookie
Aug 30, 2009



on the other hand, if a codebase doesn't have any lovely hacks its practically guaranteed that it doesn't do anything interesting

SWIM should research optimal lovely hack density in kludges/KLOC and report back

coffeetable
Feb 5, 2006

TELL ME AGAIN HOW GREAT BRITAIN WOULD BE IF IT WAS RULED BY THE MERCILESS JACKBOOT OF PRINCE CHARLES

YES I DO TALK TO PLANTS ACTUALLY

Nomnom Cookie posted:

SWIM should research optimal lovely hack density in kludges/KLOC and report back
call a node in a dependency graph "dirty" if it contains a lovely hack. define a utility function that is positively related to the number of dirty nodes, but (very) negatively related to the size of the largest all-dirty connected subgraph

if you also posit that lovely hacks encourage lovely hacks in connected classes, i reckon there're some interesting questions to be asked about contagion in codebases

Brain Candy posted:

the plural of lovely hacks is not library

lol

PrBacterio
Jul 19, 2000

Brain Candy posted:

the plural of lovely hacks is not library
thats because its actually "software"

Adbot
ADBOT LOVES YOU

Nomnom Cookie
Aug 30, 2009



coffeetable posted:

call a node in a dependency graph "dirty" if it contains a lovely hack. define a utility function that is positively related to the number of dirty nodes, but (very) negatively related to the size of the largest all-dirty connected subgraph

if you also posit that lovely hacks encourage lovely hacks in connected classes, i reckon there're some interesting questions to be asked about contagion in codebases

i embedded my connected subgraph in your mam's spanning tree last night

  • Locked thread