terrible programmers: my god, it's full of chars

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

hackbunny posted:

C too. for example if a function returns -1 for failure, you don't check == -1, you check < 0. if it returns 0 for false and 1 for true, you check != 0 instead of == 1. also when a function returns both a success flag/error code and a value through a pointer, you don't trust the returned value if the function reports success, you explicitly check that it's valid/non-null/etc. which can turn a crash into a controlled failure

this kind of poo poo is why i don't like writing c.

# ? Jul 18, 2016 13:46

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 10:42

cinci zoo sniper: Mar 15, 2013

hackbunny posted:

C too. for example if a function returns -1 for failure, you don't check == -1, you check < 0. if it returns 0 for false and 1 for true, you check != 0 instead of == 1. also when a function returns both a success flag/error code and a value through a pointer, you don't trust the returned value if the function reports success, you explicitly check that it's valid/non-null/etc. which can turn a crash into a controlled failure

oh i see, i do that in python all over the place since i dont trust myself to understand what the hell im doing

# ? Jul 18, 2016 13:48

ynohtna: Feb 16, 2007; backwoods compatible; Illegal Hen

Luigi Thirty posted:

Uh oh open watcom's sampler doesn't support dos32a, just dos4g and phar lap

how am I going to profile my rasterizer now

old school analogue style

start of frame set bright colour in monitor overscan/border register. change colour upon entry of each rendering subsystem. set to black when finished awaiting vblank

relative size of each colour strip down the sides of the screen represent their duration relative to monitor refresh interval

e: if you can't see the first colour at the top border then your code is too slow!

ynohtna fucked around with this message at 13:56 on Jul 18, 2016

# ? Jul 18, 2016 13:53

Luigi Thirty: Apr 30, 2006; Emergency confection port.

Bloody posted:

time to make a profiler

oh god

# ? Jul 18, 2016 14:24

vodkat: Jun 30, 2012; cannot legally be sold as vodka

MALE SHOEGAZE posted:

how are you loading the data? you need to be loading the data in batches, processing a batch, outputting a batch, and loading the next batch.

it looks like you're loading everything into the data variable and then processing in one go, which won't for large datasets

also it could be something dumb in your code but i don't know Python well enough to tell you

Yes I am doing. I did not know this was something you were not meant to do, I'll try implementing batches and see how that works out. Thanks

# ? Jul 18, 2016 15:00

LordSaturn: Aug 12, 2007; sadly unfunny

vodkat posted:

Hello thread, can I post some bad code for you to tell me what I'm doing wrong/how to make it better?

Im currently trying to clean a list of names in a database. Since I'm just a stupid grad student (a social sciences one at that) something like the code bellow has always been good enough for what I want to do but now I'm trying to use it on a larger dataset its taking loving ages and its clear that I am probably doing this in a horrendously inefficient way.
code:
def clean_names2(name):
    if type(name) == str:
        name = name.upper()
        name_list = re.split(' ', name)
        name_list = set(name_list) - set(prefix)
        name_list = set(name_list) - set(postfix)
        name = ' '.join(name_list)
        return name

for n in data['col']:
        data.loc[n, 'col'] = clean_names2(data.loc[n, 'col')])
I'd really appreciate any advice you guys have since most of my programming knowledge comes from googling things and stack exchange so I really have no idea how to start about making something work better.

I don't think those set casts are free, and I also think they'll do unhappy things if someone's first and last names are the same. at the very least you could store prefix and postfix as sets natively instead of casting them every time. oh and what the gently caress are you doing with re.split, use name.split

EDIT: no idea if this is better, but I usually do it like this:

Python code:

for bad_name in prefix + suffix:
    while bad_name in name_list:
        name_list.remove( bad_name )

LordSaturn fucked around with this message at 15:17 on Jul 18, 2016

# ? Jul 18, 2016 15:11

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

vodkat posted:

Yes I am doing. I did not know this was something you were not meant to do, I'll try implementing batches and see how that works out. Thanks

just to be clear, it's possible this is not the issue. I'd check to see if the script works ok with a small data set. if it does, you probably just have a logic error

# ? Jul 18, 2016 15:35

vodkat: Jun 30, 2012; cannot legally be sold as vodka

MALE SHOEGAZE posted:

just to be clear, it's possible this is not the issue. I'd check to see if the script works ok with a small data set. if it does, you probably just have a logic error

Also I've just realised that pandas has a function exactly for what I'm trying to do using. But they were both really good tips and I appreciate the advice on how to suck slightly less.

code:

dataframe['col'] = dataframe['col'].apply(function)

This also seems to be much much faster.

vodkat fucked around with this message at 15:53 on Jul 18, 2016

# ? Jul 18, 2016 15:49

Luigi Thirty: Apr 30, 2006; Emergency confection port.

overheard at work

"now you know why (programmer on my team) is known as the Iceberg"
"why's that"
"because he has the ability to sink an entire release"

# ? Jul 18, 2016 16:59

triple sulk: Sep 17, 2014

Luigi Thirty posted:

overheard at work

"now you know why (programmer on my team) is known as the Iceberg"
"why's that"
"because he has the ability to sink an entire release"

# ? Jul 18, 2016 17:02

cinci zoo sniper: Mar 15, 2013

Luigi Thirty posted:

overheard at work

"now you know why (programmer on my team) is known as the Iceberg"
"why's that"
"because he has the ability to sink an entire release"

well gently caress me thinking about lettuce :eyepop:

# ? Jul 18, 2016 17:04

Lutha Mahtin: Oct 10, 2010; Your brokebrain sin is absolved...go and shitpost no more!

vodkat posted:

Also I've just realised that pandas has a function exactly for what I'm trying to do using.

yeah, pandas is built for speed. one of its dependencies is fellow fast library [url=http://www.numpy.org/], so that's something to keep in mind when you're working with pandas. if you're interested in learning more about pandas and data processing in python, check out the O'Reilly book "Python for Data Analysis". it's written by the guy who created pandas, and as someone who doesn't really work in python i found it very easy to read

# ? Jul 18, 2016 17:22

Deep Dish Fuckfest: Sep 6, 2006; Advanced
Computer Touching; Toilet Rascal

ynohtna posted:

old school analogue style

start of frame set bright colour in monitor overscan/border register. change colour upon entry of each rendering subsystem. set to black when finished awaiting vblank

relative size of each colour strip down the sides of the screen represent their duration relative to monitor refresh interval

e: if you can't see the first colour at the top border then your code is too slow!

this is kinda cool for something so simple, i'm surprised i've never heard about it before

# ? Jul 18, 2016 17:23

30 TO 50 FERAL HOG: Mar 2, 2005

just started digging into our CRM database

the users table does not have a primary key/id that is referenced in other tables as a foreign key. it literally has a username column in every table and text based matching to find rows associated with a particular user

# ? Jul 18, 2016 18:42

Captain Foo: May 11, 2004; we vibin'
we slidin'
we breathin'
we dyin'

Luigi Thirty posted:

overheard at work

"now you know why (programmer on my team) is known as the Iceberg"
"why's that"
"because he has the ability to sink an entire release"

super slam

# ? Jul 18, 2016 18:52

gonadic io: Feb 16, 2011; >>=

BiohazrD posted:

just started digging into our CRM database

the users table does not have a primary key/id that is referenced in other tables as a foreign key. it literally has a username column in every table and text based matching to find rows associated with a particular user

so you have an implicit string foreign key. i've seen it more than once

# ? Jul 18, 2016 19:16

30 TO 50 FERAL HOG: Mar 2, 2005

as far as i can tell there are no constraints on the database and nothing set as a foreign key, it is all done in software

# ? Jul 18, 2016 19:17

HoboMan: Nov 4, 2010

foreign key constraints are annoying

# ? Jul 18, 2016 19:26

Sapozhnik: Jan 2, 2005; Nap Ghost

HoboMan posted:

foreign key constraints are annoying

:wideniussay:

# ? Jul 18, 2016 19:38

Shaggar: Apr 26, 2006

HoboMan posted:

foreign key constraints are annoying

thanks, tbc's intern

# ? Jul 18, 2016 19:39

30 TO 50 FERAL HOG: Mar 2, 2005

HoboMan posted:

foreign key constraints are annoying

lol

foreign keys own, the problem is when some shitlord uses a natural key as the primary key instead of a surrogate key in the form of an GUID/UUID/whatever

# ? Jul 18, 2016 19:47

HoboMan: Nov 4, 2010

no you see i would much rather re-implement the database logic myself than risk getting a single SQL error

serious post: SQL error messages are pretty terrible though

# ? Jul 18, 2016 19:54

Sapozhnik: Jan 2, 2005; Nap Ghost

just cut to the chase and create a single FACTS(ID, KEY, VAL) table

who needs SQL anyway!

# ? Jul 18, 2016 22:01

Luigi Thirty: Apr 30, 2006; Emergency confection port.

who needs databases when you can store everything as CSV in a flat file on an SSD

# ? Jul 18, 2016 22:09

brap: Aug 23, 2004; Grimey Drawer

the natural end state of SQL is a table with a key column and a value column containing arbitrary XML

# ? Jul 18, 2016 22:15

Bloody: Mar 3, 2013

we call it infinite normal form

# ? Jul 18, 2016 22:38

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

BiohazrD posted:

lol

foreign keys own, the problem is when some shitlord uses a natural key as the primary key instead of a surrogate key in the form of an GUID/UUID/whatever

ON UPDATE CASCADE BITCH

DO YOU SPEAK IT?

# ? Jul 18, 2016 22:56

Ragg: Apr 27, 2003; <The Honorable Badgers>

fleshweasel posted:

the natural end state of SQL is a table with a key column and a value column containing arbitrary XML

Currently scraping a table that's this but HTML :cheers:

# ? Jul 18, 2016 23:43

Luigi Thirty: Apr 30, 2006; Emergency confection port.

fleshweasel posted:

the natural end state of SQL is a table with a key column and a value column containing arbitrary XML

hello this is the basis of one of our product's primary features

# ? Jul 18, 2016 23:45

30 TO 50 FERAL HOG: Mar 2, 2005

Janitor Prime posted:

ON UPDATE CASCADE BITCH

DO YOU SPEAK IT?

is there an easy way to do this on a constraint that has already been created

# ? Jul 18, 2016 23:50

30 TO 50 FERAL HOG: Mar 2, 2005

or like

update this value and all tables constrained by it also

# ? Jul 18, 2016 23:54

Deep Dish Fuckfest: Sep 6, 2006; Advanced
Computer Touching; Toilet Rascal

ain't nothing wrong with having xml or html or json stored in a relational db if it's treated as an opaque blob. just because you can normalize something or have it appear as part of your schema doesn't mean you should

i mean, in theory you could store images in a db in a table with (image_id, x_coord, y_coord, pixel_value) but that's probably a poor idea

# ? Jul 19, 2016 00:11

Shaggar: Apr 26, 2006

BiohazrD posted:

or like

update this value and all tables constrained by it also

if you are updating the tables with the same value as the value in the FK you are not doing foreign keys right.

# ? Jul 19, 2016 00:27

30 TO 50 FERAL HOG: Mar 2, 2005

Shaggar posted:

if you are updating the tables with the same value as the value in the FK you are not doing foreign keys right.

You don't have to tell me. I'm just trying to work on this piece of poo poo while changing as few things as possible

# ? Jul 19, 2016 00:31

Luigi Thirty: Apr 30, 2006; Emergency confection port.

huh turns out this is slow. who knew

code:

        for(int scanline = std::max(triangleBoundingBoxY[0], 0); scanline <= std::min(SCREEN_HEIGHT-1, triangleBoundingBoxY[1]); scanline++){
            for(int x=std::max(0, triangleBoundingBoxX[0]);x<=std::min(SCREEN_WIDTH-1, triangleBoundingBoxX[1]);x++){

                /* Is the point inside the triangle? */      
                Vector3f barycentric = barycentric_point(screenPoints[0], screenPoints[1], screenPoints[2], Point(x, scanline));
                if(barycentric.x < 0 || barycentric.y < 0 || barycentric.z < 0) continue; //If not, go to the next point

                /* Yes, so get the point's Z. */
                float z = 0;
                z += (worldCoordinates[0].z * barycentric.x);
                z += (worldCoordinates[1].z * barycentric.y);
                z += (worldCoordinates[2].z * barycentric.z);
            
                /* Check the Z-buffer. Should we draw this point? */
                unsigned int pixelIndex = x + VGA_Y_OFFSETS[scanline];
                if(z < zbuffer[pixelIndex]) {

                    /* Yes, draw the point. */
                    zbuffer[pixelIndex] = z;
                    pixels[pixelIndex] = color;

                }
            }
        }

# ? Jul 19, 2016 01:04

tef: May 30, 2004; -> some l-system crap ->

Mr Dog posted:

just cut to the chase and create a single FACTS(ID, KEY, VAL) table

who needs SQL anyway!

same

# ? Jul 19, 2016 01:17

tef: May 30, 2004; -> some l-system crap ->

Luigi Thirty posted:

who needs databases when you can store everything as CSV in a flat file on an SSD

well you could and use postgresraw :v:

# ? Jul 19, 2016 01:17

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

Mr Dog posted:

just cut to the chase and create a single FACTS(ID, KEY, VAL) table

who needs SQL anyway!

at my job, i frequently have to work with a database that has a table SYSTEM_FACTS(ID, KEY, VAL)

these (obviously, i guess) correspond to the system facts taken from the systems our software runs on. the facts are more or less arbitrary but a handful of them are really important and need to be returned with almost every query.

code:

> SELECT COUNT(DISTINCT(key)) FROM system_facts;
> 232937

i've spent the last month writing ~~performant~~ relatively performant software that needs to join on that table.

oh yeah and it's mysql.
it actually wasn't a big deal but i was worried at first

DONT THREAD ON ME fucked around with this message at 02:03 on Jul 19, 2016

# ? Jul 19, 2016 02:00

CPColin: Sep 9, 2003; Big ol' smile.

Luigi Thirty posted:

huh turns out this is slow. who knew

You think that's bad, try doing that tutorial in Ceylon!

# ? Jul 19, 2016 02:36

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 10:42

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

this probably belongs in this thread https://bugzilla.mozilla.org/show_bug.cgi?id=1154339

# ? Jul 19, 2016 03:28

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »