terrible programmers: my god, it's full of chars

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »

fart simpson: Jul 2, 2005; DEATH TO AMERICA

Valeyard posted:

ruby is real bad

# ? Apr 30, 2016 17:29

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 11:38

pokeyman: Nov 26, 2006; That elephant ate my entire platoon.

Blinkz0rz posted:

i'm just saying that in my many years of programming i've never had to parse a uri that wasn't a url

now igi

# ? Apr 30, 2016 18:40

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

weird posted:

i'm just saying that uri.parse should not throw an exception on a valid uri (it does throw on an invalid uri), and that calling it silent failure is objectively wrong

i get you and with some thought i agree.

the real dumb poo poo is that using uri.parse and then nil testing is the accepted way to parse urls in ruby

so what i'm saying is

Valeyard posted:

ruby is real bad

# ? Apr 30, 2016 19:01

distortion park: Apr 25, 2011

MALE SHOEGAZE posted:

no you're actually just extremely long

nice of your mum to give you trip reports on her dates

# ? Apr 30, 2016 19:55

HoboMan: Nov 4, 2010

This made me lol: https://stackoverflow.com/jobs/114426/actively-seeking-artisanal-javascript-engineer

# ? Apr 30, 2016 23:20

cowboy beepboop: Feb 24, 2001

HoboMan posted:

This made me lol: https://stackoverflow.com/jobs/114426/actively-seeking-artisanal-javascript-engineer

ipv6 is a new technology, lol

# ? May 1, 2016 10:36

Bloody: Mar 3, 2013

pulling really big (big enough to not want on disk/in ram) data sets from the internet
var fromWeb = new StreamReader(new GZipStream(HttpWebRequest.Create(url).GetResponse().GetResponseStream(), CompressionMode.Decompress));
while (!fromWeb.EndOfStream)
{
var x = fromWeb.ReadLine();
....
}

what a good stdlib

i love c#

# ? May 1, 2016 14:02

Bloody: Mar 3, 2013

had a truly terrible programmer moment where i was doing

foreach(var butt in collection.Select(a => a.fart).Distinct())
{
...
foreach (var x in collection.Where(a => a.fart == butt).GroupBy(a => a.whatever, b => b))

this is an idiot's reimplementation of a dictionary and it is very slow
using an actual dictionary, butt in collection.Keys, and collection[butt] netted a 99.5% speedup

# ? May 1, 2016 14:04

raminasi: Jan 25, 2005; a last drink with no ice

you could iterate over .Values if you just need the values or the collection itself if you need both

# ? May 1, 2016 17:19

JewKiller 3000: Nov 28, 2006; by Lowtax

ruby is really bad and postgresql is really good

# ? May 1, 2016 18:17

MeruFM: Jul 27, 2010

Bloody posted:

pulling really big (big enough to not want on disk/in ram) data sets from the internet
var fromWeb = new StreamReader(new GZipStream(HttpWebRequest.Create(url).GetResponse().GetResponseStream(), CompressionMode.Decompress));
while (!fromWeb.EndOfStream)
{
var x = fromWeb.ReadLine();
....
}

what a good stdlib

i love c#

looks like python if you squish it and squint your eyes

# ? May 1, 2016 19:45

Condiv: May 7, 2008; Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod

Bloody posted:

pulling really big (big enough to not want on disk/in ram) data sets from the internet
var fromWeb = new StreamReader(new GZipStream(HttpWebRequest.Create(url).GetResponse().GetResponseStream(), CompressionMode.Decompress));
while (!fromWeb.EndOfStream)
{
var x = fromWeb.ReadLine();
....
}

what a good stdlib

i love c#

i'm thinking about doing similar with a java/scala project for crunching big data with some algorithms my lab has developed. most of these algorithms are in C/C++, so i'm thinking about using named pipes instead of program arguments so I can stream 300MB+ pictures to the programs instead of loading them onto the disk somewhere. does anyone have experience using linux/mac named pipes for a program?

# ? May 1, 2016 23:57

FamDav: Mar 29, 2008

Condiv posted:

...crunching big data ... 300MB+...

what kind of computer are you using that your working set, even assuming it was 10x as big, doesn't fit into RAM?

EDIT: if you're just trying to do something to learn it that's fine, but if you need something to go faster you should probably say

1. where your data is
2. where your data needs to be
3. how parallel are your algorithms
4. what you are actually doing

FamDav fucked around with this message at 00:09 on May 2, 2016

# ? May 2, 2016 00:05

Condiv: May 7, 2008; Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod

FamDav posted:

what kind of computer are you using that your working set, even assuming it was 10x as big, doesn't fit into RAM?

EDIT: if you're just trying to do something to learn it that's fine, but if you need something to go faster you should probably say

1. where your data is
2. where your data needs to be
3. how parallel are your algorithms
4. what you are actually doing

a good portion of our datasets will fit in ram considering we have 128GB (not a ton, but more than your typical desktop at least). some of our algorithms have really nasty memory growth characteristics that mean that a 300mb image file input can balloon into 12GBs of ram usage easily, so trying to control ram usage is pretty important. also, the program i'm making needs to run uninterrupted for as long as possible.

1. data is on a 60TB server in our lab
2. data needs to be on a calculation server that has a much smaller harddrive
3. algorithms are parallel over the permutation of their inputs (if i input images 0-99 and a setting theta from 0.2 to 0.9 by 0.1, then i have a max of 800 threads for a single processing request) unless specifically disabled in someway by the algorithm. each processing request can therefore spawn a ton of threads
4. image analysis mainly

# ? May 2, 2016 00:20

CPColin: Sep 9, 2003; Big ol' smile.

Condiv posted:

does anyone have experience using linux/mac named pipes for a program?

I would guess that a named pipe would look the same as a normal file, from within Java. I don't know if NIO stuff would work (because I've still never used it!), but reading from a FileInputStream until it runs out should work, I'd think.

# ? May 2, 2016 01:04

Brain Candy: May 18, 2006

CPColin posted:

I would guess that a named pipe would look the same as a normal file, from within Java. I don't know if NIO stuff would work (because I've still never used it!), but reading from a FileInputStream until it runs out should work, I'd think.

you could try treating a named pipe as plane old file but running out of bytes doesn't mean that you've loaded the whole file, just that you've emptied the buffer of the pipe. a named pipe is just a named buffer. two readers of the same pipe means two readers with partial data.

you'll need a means to distinguish one image from the next image. typically you'd either rely on the particular image format at the cost of being fragile or add headers and/or footers.

# ? May 2, 2016 01:41

Brain Candy: May 18, 2006

Blinkz0rz posted:

i'm just saying that in my many years of programming i've never had to parse a uri that wasn't a url

your browser parses uris everytime you type something in the address bar because i doubt you type http or https

# ? May 2, 2016 01:58

FamDav: Mar 29, 2008

are you using an nfs? something like sshfs should handle streaming and caching for you without having to janitor pipes.

as for controlling ram usage, there's no substitute for fixing/optimizing the actual program but if you're looking to run multiple native applications in parallel without fighting each other for ram and cpu then you're going to want to look into using cgroups. the jvm won't do this for you by default for native code, but since its java i'm sure someone has made a library to do it.

# ? May 2, 2016 01:59

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Condiv posted:

i'm thinking about doing similar with a java/scala project for crunching big data with some algorithms my lab has developed. most of these algorithms are in C/C++, so i'm thinking about using named pipes instead of program arguments so I can stream 300MB+ pictures to the programs instead of loading them onto the disk somewhere. does anyone have experience using linux/mac named pipes for a program?

these should probably be memory-mapped read-only so the virtual memory system can handle paging everything in and dumping unneeded pages on demand

this will have the added benefit of allowing the memory to be shared automatically among multiple processes, since it's just a matter of mapping specific read-only pages into multiple tasks' address spaces

# ? May 2, 2016 03:08

Corla Plankun: May 8, 2007; improve the lives of everyone

i'm the ml algorithm that leverages 40x redundancy to analyze an image for reasons that were devel9oped IN A LAB

# ? May 2, 2016 05:03

Condiv: May 7, 2008; Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod

Brain Candy posted:

you could try treating a named pipe as plane old file but running out of bytes doesn't mean that you've loaded the whole file, just that you've emptied the buffer of the pipe. a named pipe is just a named buffer. two readers of the same pipe means two readers with partial data.

you'll need a means to distinguish one image from the next image. typically you'd either rely on the particular image format at the cost of being fragile or add headers and/or footers.

In general, the algorithms will get one or two images each, and each image will have its own variable/pipe. so distinguishing the images from each other shouldn't be an issue. if I understand correctly, if I write to the named pipe and then close the writer that should put an eof character in the pipe letting the client algorithm know that the stream is finished right?

FamDav posted:

are you using an nfs? something like sshfs should handle streaming and caching for you without having to janitor pipes.

as for controlling ram usage, there's no substitute for fixing/optimizing the actual program but if you're looking to run multiple native applications in parallel without fighting each other for ram and cpu then you're going to want to look into using cgroups. the jvm won't do this for you by default for native code, but since its java i'm sure someone has made a library to do it.

iRODS. its some weird file database thing. i can grab inputstreams to files on irods and pipe the file into an outputstream targeting a named pipe so that nothing touches the disk.

for the cgroups, yeah i've been looking into that and stuff like docker and rkt for running the applications inside. supposedly someday we're gonna run semi-trusted code on this server, so i'm hopeful i can use containers for isolation without incurring all the overhead of VMs.

eschaton posted:

these should probably be memory-mapped read-only so the virtual memory system can handle paging everything in and dumping unneeded pages on demand

this will have the added benefit of allowing the memory to be shared automatically among multiple processes, since it's just a matter of mapping specific read-only pages into multiple tasks' address spaces

the only problem i can see with this is that it looks like the processes themselves would have to be written to take advantage of the mapped memory. is that correct?

# ? May 2, 2016 07:12

Sagacity: May 2, 2003; Hopefully my epitaph will be funnier than my custom title.

fritz posted:

terrible programmer sighted: https://medium.com/@raymondchandler/down-and-out-in-the-magic-kingdom-92b0a715778f#.goh7nwgy3

oh my, this was a delicious read

# ? May 2, 2016 09:21

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Condiv posted:

the only problem i can see with this is that it looks like the processes themselves would have to be written to take advantage of the mapped memory. is that correct?

at least theoretically, it should still help if only some of the processes can mmap an image, because those that do a normal read will still get their "copy" of the data from the unified buffer cache

for reads of data already in memory for another process, the VM system could conceivably just map the pages into the reading process too and mark them copy-on-write

an mmap doesn't even have to do the latter, because by mapping the data read-only a write to a mapped page can just be treated as an error and no accommodation for copy-on-write needs to be made

# ? May 2, 2016 09:52

b0red: Apr 3, 2013

reading about null object pattern

really should start diving more into smart programmer things like this. would make my life so much easier

# ? May 2, 2016 14:08

Bloody: Mar 3, 2013

patterns are dogma

Bloody posted:

patterns are dogma

this post is also dogma

# ? May 2, 2016 14:12

BattleMaster: Aug 14, 2000

b0red posted:

reading about null object pattern

really should start diving more into smart programmer things like this. would make my life so much easier

https://en.wikipedia.org/wiki/Null_Object_pattern

oh so it's a formalized version of silent failure

# ? May 2, 2016 14:19

30 TO 50 FERAL HOG: Mar 2, 2005

how does a modern language (in this case c#) not have a math.percentage method?

int percentage = (int)((((double)top) / ((double)bottom)) * 100.0);

is some scrub tier bullshit, what am i writing here? loving lisp?

# ? May 2, 2016 14:37

Asymmetrikon: Oct 30, 2009; I believe you're a big dork!

BattleMaster posted:

oh so it's a formalized version of silent failure

no, it's just a really lovely version of the option monad

# ? May 2, 2016 14:50

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

because doing arithmetic in percentages is loving dumb when you have perfect accuracy decimal fractions?

the only reason to ever convert something to a percentage is when you want to display it to the user, and c# does in fact have a format specifier to do that.

# ? May 2, 2016 14:51

Bloody: Mar 3, 2013

the real question here is why are you doing that

# ? May 2, 2016 14:54

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

I never seen math.percentage before I mean what the gently caress it's just multiplying by 100 lmao

# ? May 2, 2016 14:57

HoboMan: Nov 4, 2010

BiohazrD posted:

how does a modern language (in this case c#) not have a math.percentage method?

int percentage = (int)((((double)top) / ((double)bottom)) * 100.0);

is some scrub tier bullshit, what am i writing here? loving lisp?

no, there's no polish notation :colbert:

the actual bullshit is that typecasting

# ? May 2, 2016 15:01

b0red: Apr 3, 2013

Asymmetrikon posted:

no, it's just a really lovely version of the option monad

well i have to use ruby so it's a lovely version i need sometimes

# ? May 2, 2016 15:02

tef: May 30, 2004; -> some l-system crap ->

BiohazrD posted:

how does a modern language (in this case c#) not have a math.percentage method?

int percentage = (int)((((double)top) / ((double)bottom)) * 100.0);

is some scrub tier bullshit, what am i writing here? loving lisp?

http://stackoverflow.com/questions/1790975/format-decimal-for-percentage-values

because percentages are locale dependent formatted strings?

# ? May 2, 2016 15:04

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

ladies and gentlemen, i present to you, "the userbase of leftpad():"

BiohazrD posted:

how does a modern language (in this case c#) not have a math.percentage method?

int percentage = (int)((((double)top) / ((double)bottom)) * 100.0);

is some scrub tier bullshit, what am i writing here? loving lisp?

# ? May 2, 2016 15:13

Captain Foo: May 11, 2004; we vibin'
we slidin'
we breathin'
we dyin'

Wheany posted:

ladies and gentlemen, i present to you, "the userbase of leftpad():"

lemao

# ? May 2, 2016 15:24

quiggy: Aug 7, 2010; [in Russian] Oof.

guys last week i had to convert from degrees to radians by hand, without using a built-in library function

truly i have suffered the most

# ? May 2, 2016 15:26

30 TO 50 FERAL HOG: Mar 2, 2005

idgaf, make my life easier not harder thats what libraries are for

# ? May 2, 2016 15:32

HoboMan: Nov 4, 2010

Wheany posted:

ladies and gentlemen, i present to you, "the userbase of leftpad():"

this is supposed to be a safe space for bad programming, let's not get too vicious

# ? May 2, 2016 15:34

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 11:38

BattleMaster: Aug 14, 2000

the appeal of what you want is limited enough that several people can't think of a reason why it needs to be standard

if it really does come up that much for you suck it up and write a function that does it and it would have taken you less time than you needed to bitch about it on the internet

# ? May 2, 2016 15:34

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »