Coding Horrors: You can gather all your technical debt into one easy framework!

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Coding Horrors: You can gather all your technical debt into one easy framework!

«‹›1503 »

Zombywuf: Mar 29, 2008

Internet Janitor posted:

-Put your dataset in a database and use a query language to find out what you actually want to know. It will consume fewer system resources, be less likely to lock up for minutes at a time and your results will be repeatable. Even interactively, if you can search and filter data more quickly, you can extract more information from it.

Sometimes this is easier, often it's harder. Yeah Excel sucks when you put a lot of data in it, but that doesn't make the person with a lot of data to bad party.

quote:

-Use a programming language meant for data processing. R, for example, can easily deal with millions of rows of data in a fraction the time and space Excel would require.

Here's the report boss, it comes with data in something that isn't Excel. What's that? I'm fired?

EDIT: if I wasn't on windows I'd use awk.

# ? Oct 29, 2011 14:42

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 07:10

fez2: Jan 24, 2002

PrBacterio posted:

I'm sorry, is it bad that I just can't look beyond the horror of actually using COBOL at all in the first place to whatever it is you're trying to show to us here?

Well a lot of this is COBOL's fault. When you call sub modules it doesn't validate that you pass the right layout or size of memory at all. I suspect that in this case we aren't even passing enough to communicate properly with the sub module.

Your best hope in COBOL when this happens is that you'll get a protection exception by accessing memory that's out of range of your application.

# ? Oct 29, 2011 15:02

npe: Oct 15, 2004

Zombywuf posted:

Here's the report boss, it comes with data in something that isn't Excel. What's that? I'm fired?

# ? Oct 29, 2011 15:34

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Internet Janitor posted:

-Put your dataset in a database and use a query language to find out what you actually want to know. It will consume fewer system resources, be less likely to lock up for minutes at a time and your results will be repeatable. Even interactively, if you can search and filter data more quickly, you can extract more information from it.

-Use a programming language meant for data processing. R, for example, can easily deal with millions of rows of data in a fraction the time and space Excel would require.

or I could just open it in excel and be done in less time than it would have taken to create the database to stick the data in (much less actually import the data and do stuff with it), and definitely way less time than it would take to learn a new language for something I spend 10 minutes a month on

# ? Oct 29, 2011 15:45

NotShadowStar: Sep 20, 2000

Zombywuf posted:

Name one and why I should use it.

http://www.minitab.com/en-US/products/minitab

All the GUI of Excel, none of the bullshit.

# ? Oct 29, 2011 16:38

MrMoo: Sep 14, 2000

Internet Janitor posted:

-Use a programming language meant for data processing. R, for example, can easily deal with millions of rows of data in a fraction the time and space Excel would require.

There are a few solutions that provide an Excel like interface around large scale data processing. FORA springs to mind:

http://broadstreetanalytics.com/spreadsheet.html

Thomson Reuters have one too but I don't know much about it's integration potential currently. I'm starting a new job on Monday which I think is about doing this, so :v:

# ? Oct 29, 2011 17:18

wwb: Aug 17, 2004

quote:

Here's the report boss, it comes with data in something that isn't Excel. What's that? I'm fired?

Last time I checked, most bosses don't want to wade through 65k++ lines of a spreadsheet, they want the relevent parts of that.

Now, our "stats" guy only uses excel. And he is trying to make sense of these huge assed government data sets. No, excel can't handle 20+ GB of relational data.

# ? Oct 29, 2011 17:22

Zhentar: Sep 28, 2003; Brilliant Master Genius

Internet Janitor posted:

-Put your dataset in a database and use a query language to find out what you actually want to know. It will consume fewer system resources, be less likely to lock up for minutes at a time and your results will be repeatable. Even interactively, if you can search and filter data more quickly, you can extract more information from it.

Have you ever actually used Excel? It takes two whole actions to load a couple hundred thousand lines of delimited text into a pivot table, and start analyzing. It would take me longer to type out even the most basic query.

# ? Oct 29, 2011 19:40

Scaramouche: Mar 26, 2001; SPACE FACE! SPACE FACE!

I think IJ's point is that if whatever the thing is, if it's evolved to the point where it's generating >65,535 rows of data regularly enough that it becomes a problem, that thing should probably have its own supporting infrastructure and reporting instead of using a jumped up office productivity tool.

# ? Oct 30, 2011 04:25

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

65k rows of data is not very much data at all and you only had to run into the limit once for it to be annoying.

I suspect it's more of that IJ's main exposure to Excel has been horrible things that should have been migrated to a special-purpose program long ago and so he does not realize that Excel is actually a pretty good program that's decent at a lot of things.

# ? Oct 30, 2011 05:57

Ursine Catastrophe: Nov 9, 2009; It's a lovely morning in the void and you are a horrible lady-in-waiting.

don't ask how i know; Dinosaur Gum

Plorkyeran posted:

65k rows of data is not very much data at all and you only had to run into the limit once for it to be annoying.

I suspect it's more of that IJ's main exposure to Excel has been horrible things that should have been migrated to a special-purpose program long ago and so he does not realize that Excel is actually a pretty good program that's decent at a lot of things.

I don't think there's anyone claiming Excel is bad per se, but even Microsoft basically says "Don't use Excel for databases", given that Access is shipped in MS Office by default as well.

It's just a matter of Excel is used as the catchall hammer of "I can't visualize RDBMSs properly, so just make it 2D for me, k?", and that leads to problems like this.

# ? Oct 30, 2011 06:02

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

I'm not even talking about using excel as a database. the most recent thing I used excel on a large dataset for was removing one column from a csv file because the format was slightly different from what the tool it was being read by wanted. the file had quoted commas inside fields so using sed (or just a text editor) would have been awkward. writing a script to parse the file correctly and remove the field would have taken about five minutes... but just opening the file in excel, selecting the column and hitting delete took about 30 seconds.

# ? Oct 30, 2011 06:14

Look Around You: Jan 19, 2009

clockwork automaton posted:

code:

    do{
                c = fgetc(file);
                if(31 < c && c < 127){
                        c = fgetc(file);
                        if(31 < c && c < 127){
                                c = fgetc(file);
                                if(31 < c && c < 127){
                                        c = fgetc(file);
                                        if(31 < c && c < 127){
                                                fseek(file, -4, SEEK_CUR);
                                                c = fgetc(file);
                                                i = 0;
                                                while(31 < c && c < 127){
                                                        string[i] = (char)c;
                                                        c = fgetc(file);
                                                        i++;
                                                }
                                                string[i] = NULL;
                                                printf("%s\n", string);
                                        }
                                }
                        }
                }
        }while((char)c != EOF);

I don't like grading (I don't teach this class, I promise).

CS0449?

# ? Oct 30, 2011 07:01

Zombywuf: Mar 29, 2008

OriginalPseudonym posted:

I don't think there's anyone claiming Excel is bad per se

I am.

NotShadowStar posted:

All the GUI of Excel, none of the bullshit.

That looks pretty cool. I work with unix like systems now though so I can just do stuff by hand now.

# ? Oct 30, 2011 09:36

Beef: Jul 26, 2004

A mathematician friend of mine has a job at a big insurance/consultancy firm. He's a big Excel proponent, but finds the file size limit annoying :raise:

. (can't remember how big it was, 4GB?)

# ? Oct 30, 2011 16:55

clockwork automaton: May 2, 2007; You've probably never heard of them.; Fun Shoe

Look Around You posted:

CS0449?

Yes...

# ? Oct 30, 2011 17:21

Ursine Catastrophe: Nov 9, 2009; It's a lovely morning in the void and you are a horrible lady-in-waiting.

don't ask how i know; Dinosaur Gum

Zombywuf posted:

I am.

I stand corrected. :v:

# ? Oct 31, 2011 08:30

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

A new feature request in our tracker. It has a pretty detailed description about the feature, what needs to be implemented in the backend, what changes in bahavior are needed.

I'm supposed to implement the UI changes for this feature. The only mention of this is "How about UI? Affected, isn't it?"

Great spec, guys. Real helpful.

# ? Oct 31, 2011 09:13

Optimus Prime Ribs: Jul 25, 2007

Why not just respond with "It sure is" and then call it a day.

# ? Oct 31, 2011 14:26

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Plorkyeran posted:

VB supported 16-bit through 4.0 and I doubt it got a full rewrite when it went 32-bit only so some lingering 16-bitisms isn't surprising. Excel being limited to 65536 rows until 2007 is quite a bit worse.

AFAIK Open Office still has this problem

# ? Oct 31, 2011 16:46

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Munkeymon posted:

AFAIK Open Office still has this problem

This is oddly reminiscent of Wine developers' complaints about having to faithfully reproduce bugs in the Win32 API, since lots of programs depended on or worked around those bugs :v:

I seriously doubt that this is the real explanation, but it wouldn't surprise me.

# ? Oct 31, 2011 18:10

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Lysidas posted:

This is oddly reminiscent of Wine developers' complaints about having to faithfully reproduce bugs in the Win32 API, since lots of programs depended on or worked around those bugs

I seriously doubt that this is the real explanation, but it wouldn't surprise me.

I should have edited out the part of the quote about VB since I was mostly talking about the row limit thing, but on the subject of 16 bit ints, it looks like they only got to 65k in 2004: http://sc.openoffice.org/row-limit.html

# ? Oct 31, 2011 18:28

Malloc Voidstar: May 7, 2007; Fuck the cowboys. Unf. Fuck em hard.

They got to 1,048,576 rows at the end of 2010. (and it took them six years after the bug was opened to do so)

# ? Oct 31, 2011 18:39

Toady: Jan 12, 2009

PrBacterio posted:

I'm sorry, is it bad that I just can't look beyond the horror of actually using COBOL at all in the first place to whatever it is you're trying to show to us here?

COBOL is not uncommon in the banking industry.

# ? Oct 31, 2011 18:40

Star Warrior X: Jul 14, 2004

Toady posted:

COBOL is not uncommon in the banking industry.

Are you trying to say that it's common for business-oriented applications?

# ? Oct 31, 2011 18:43

Toady: Jan 12, 2009

Star Warrior X posted:

Are you trying to say that it's common for business-oriented applications?

I mentioned banking because it was banking code that was quoted.

# ? Oct 31, 2011 18:55

Mr.NaviPacho: Aug 15, 2007; FTBRG!

quote:

A simple use of eval:

Sometimes you want to know both the value of a variable being passed to a function and its name. Passing by reference can get you there, but only through a lot of convolutions (as far as I can tell!)

But by using the stem of a variable's name, there are two methods to achieve this.

Say that

<?php
$emailName="me@example.com";
?>

You can pass this information thusly:

First, if you have the stem, "emailName', in a string var

<?php
$theVarStem="emailName";
?>

you can recover the value of the original variable like this:

<?php
$theVarValue=eval("return \$$theVarStem;");
?>

Using 'return' inside the eval is the trick that makes it work - something inherent in the documentation but not at all obvious to me.

Unless $emailName has global scope, this method won't work inside a function. For that purpose, you can do the following:

<?php
myFunction(compact("emailName"))

function myFunction($theInfo)
{
$theVarStem=key($theInfo);
$theVarValue=current($theInfo);
//...
}
?>

I apologize if all this seems too obvious: it took me a lotta sloggin' to figure it out.

I found this tacked to the wall at my work so I googled it and it seems to be from a comment in php's eval manual.
http://www.php.net/manual/en/function.eval.php#96407
Initially I thought this was intended as a joke, but I have my doubts.

Mr.NaviPacho fucked around with this message at 19:12 on Oct 31, 2011

# ? Oct 31, 2011 19:07

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Mr.NaviPacho posted:

I found this tacked to the wall at my work so I googled it and it seems to be from a comment in php's eval manual.
http://www.php.net/manual/en/function.eval.php#96407
Initially I thought this was intended as a joke, but I have my doubts.

Most of the user-supplied comments on that page are seriously :ughh:

so I would not be surprised either way. I'm surprised there isn't a big flashing warning in the top part of the page telling people not to use eval(). The PHP manual does at least do that for some other manual pages, like the ones for goto and magic quotes.

You should be more worried about whether your coworker put it there as a joke or as a helpful reminder.

# ? Oct 31, 2011 19:18

TasteMyHouse: Dec 21, 2006

what the gently caress posted:

Using 'return' inside the eval is the trick that makes it work - something inherent in the documentation but not at all obvious to me.

holy poo poo hahaha

# ? Oct 31, 2011 19:30

NotShadowStar: Sep 20, 2000

You don't need to eval actually if you want to get the name of a variable. Some trickery involving $name = $$thing or something, I don't remember because it made no sense. :php:

http://www.php.net/manual/en/language.variables.variable.php

Apparently it works because the $ token is some sort of macro that references the symbol in the local scope or some weird hokey crap.

NotShadowStar fucked around with this message at 20:17 on Oct 31, 2011

# ? Oct 31, 2011 20:15

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

NotShadowStar posted:

You don't need to eval actually if you want to get the name of a variable. Some trickery involving $name = $$thing or something, I don't remember because it made no sense. :php;

http://www.php.net/manual/en/language.variables.variable.php

Apparently it works because the $ token is some sort of macro that references the symbol in the local scope or some weird hokey crap.

That can be used to find out what's stored in the variable named $name. As in, if you have a variable $name and you want to know what's in the variable whose name is the value stored in $name then you can use $$name. It is ridiculous and I have no idea what you would use it for that isn't stupid.

As far as I know you can't find out what the name of a variable is by any similar means, it doesn't really make sense because a given value could be pointed to by more than one variable (or no variable at all). I just re-read the quoted post and realised I don't really understand what the guy is talking about. But I do know it's stupid.

# ? Oct 31, 2011 20:27

crazyfish: Sep 19, 2002

While not a specific piece of quotable code, this library (Kernel Mode Linux) horrified me: http://www.yl.is.s.u-tokyo.ac.jp/~tosh/kml/

# ? Oct 31, 2011 20:28

Optimus Prime Ribs: Jul 25, 2007

quote:

Sometimes you want to know both the value of a variable being passed to a function and its name.

Well I guess if you were doing something really esoteric, or just hacking...

php:

<?php
$theVarStem="emailName";

$theVarValue=eval("return \$$theVarStem;");
?>

Wait. What are you doing...

php:

<?php
myFunction(compact("emailName"))

function myFunction($theInfo)
{
$theVarStem=key($theInfo);
$theVarValue=current($theInfo);
//...
}
?>

Why...?

# ? Oct 31, 2011 21:07

Ursine Catastrophe: Nov 9, 2009; It's a lovely morning in the void and you are a horrible lady-in-waiting.

don't ask how i know; Dinosaur Gum

Hammerite posted:

That can be used to find out what's stored in the variable named $name. As in, if you have a variable $name and you want to know what's in the variable whose name is the value stored in $name then you can use $$name. It is ridiculous and I have no idea what you would use it for that isn't stupid.

Optimus Prime Ribs posted:

Why...?

It boils down to people coding and not knowing what a key => value hash looks like.

# ? Oct 31, 2011 21:28

Optimus Prime Ribs: Jul 25, 2007

OriginalPseudonym posted:

It boils down to people coding and not knowing what a key => value hash looks like.

These are probably the same people that will do something like this:

php:

<?
$myVar1 = 1;
$myVar2 = 2;
$myVar3 = 4; //;)
?>

Instead of using an array.
But I don't really want to think about coding like that.

e:

Boy do I wish I could look at the source code for my work's CMS.
One of our clients decided that they were going to write their own PHP/JavaScript for their pages and then give all that to us and we'd plug it into our system. It turns out if you escape a string when inserting into our SQL database it causes our reports to display incorrectly.
So whoever wrote that beast didn't bother with small things like preventing SQL injections.

Optimus Prime Ribs fucked around with this message at 23:06 on Oct 31, 2011

# ? Oct 31, 2011 23:03

Ursine Catastrophe: Nov 9, 2009; It's a lovely morning in the void and you are a horrible lady-in-waiting.

don't ask how i know; Dinosaur Gum

Optimus Prime Ribs posted:

These are probably the same people that will do something like this:
php:
<?
$myVar1 = 1;
$myVar2 = 2;
$myVar3 = 4; //;)
?>
Instead of using an array.
But I don't really want to think about coding like that.

e:

Boy do I wish I could look at the source code for my work's CMS.
One of our clients decided that they were going to write their own PHP/JavaScript for their pages and then give all that to us and we'd plug it into our system. It turns out if you escape a string when inserting into our SQL database it causes our reports to display incorrectly.
So whoever wrote that beast didn't bother with small things like preventing SQL injections.

Who uses PDO, anyways. Real PHP coders roll strings into mysql_query.

# ? Nov 1, 2011 00:20

Optimus Prime Ribs: Jul 25, 2007

OriginalPseudonym posted:

Who uses PDO, anyways. Real PHP coders roll strings into mysql_query.

The PHP version on our server doesn't even support PDOs. :downs:

# ? Nov 1, 2011 00:37

NotShadowStar: Sep 20, 2000

Optimus Prime Ribs posted:

The PHP version on our server doesn't even support PDOs.

Boy howdy it sure is awesome to make language extensions only available when you compile the binary. It's even more awesome when companies like RedHat provide their own forked version of the language that's ancient, and when you try and replace it with your own (painfully constructed) binary that actually has the poo poo you want, everything depending on the modified binary breaks horribly.

# ? Nov 1, 2011 02:24

McGlockenshire: Dec 16, 2005; GOLLOCKS!

I've had a good amount of luck using Remi's repo for modern PHP builds for CentOS/RHEL. He seems to be one of the maintainers of the PHP builds for Fedora and does a bunch of other worthwhile and trustworthy stuff.

# ? Nov 1, 2011 04:27

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 07:10

qntm: Jun 17, 2009

Star Warrior X posted:

Are you trying to say that it's common for business-oriented applications?

I just want you to know that I laughed at this. The other guy didn't get it, but I did. Well done.

# ? Nov 1, 2011 11:03

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Coding Horrors: You can gather all your technical debt into one easy framework!

«‹›1503 »