i actually have reasons to post here unlike half of the people with impostor syndrome

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > i actually have reasons to post here unlike half of the people with impostor syndrome

«‹›239 »

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

also if they're extremely good w/ sql they prolly actually know trivalent logic so you should disabuse them of that knowledge being correct in any other programming context

# ? Nov 12, 2017 15:25

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 21:41

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

actual python standard is to ask forgiveness not permission
aka, use try catch
you may find this idiotic. in which case, should prolly use the type annotation, make something like Findable which can be list or dict

i usually do type annotations in my own stuff, but since they are suggestive even if i will do Union(Dict, None) (to type it out quickly), the problem is the inputs, rather than the outputs. i guess i should have mentioned it before, that this specific problem is to assert if the correct output type has been achieved from processing variable input. for static checking in itself pycharm is fairly good with type annotations indeed (although i usually get lost in object hell when i end up with numpy arrays and matplotlib graphs every loving where)

try catch doesn't sound too great but i guess i could do, in second example

Python code:

def findthings(foo):
    #the double if
    try:
        assert type(baz) is dict
    except NameError:
        baz = None
    return baz

still havent gotten my rear end around checking performance on try/catch vs if/elif/else

# ? Nov 12, 2017 15:27

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

i usually do type annotations in my own stuff, but since they are suggestive even if i will do Union(Dict, None) (to type it out quickly), the problem is the inputs, rather than the outputs. i guess i should have mentioned it before, that this specific problem is to assert if the correct output type has been achieved from processing variable input. for static checking in itself pycharm is fairly good with type annotations indeed (although i usually get lost in object hell when i end up with numpy arrays and matplotlib graphs every loving where)

try catch doesn't sound too great but i guess i could do, in second example
Python code:
def findthings(foo):
    #the double if
    try:
        assert type(baz) is dict
    except NameError:
        baz = None
    return baz
still havent gotten my rear end around checking performance on try/catch vs if/elif/else

perf is loving miserable in try/catch
if you're 1000% concerned about perf don't use python lol
or use cython in the perf bits or use numpy or tf or pytorch for numerical poo poo

# ? Nov 12, 2017 15:30

cinci zoo sniper: Mar 15, 2013

c tp s: set up a local ubuntu vagrant box for dev, now to figure out the raw disk access mode on virtualbox, or other (preferably, less room to gently caress up) means to plug an ext4 drive into this poo poo

# ? Nov 12, 2017 15:30

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

c tp s: set up a local ubuntu vagrant box for dev, now to figure out the raw disk access mode on virtualbox, or other (preferably, less room to gently caress up) means to plug an ext4 drive into this poo poo

the real c tp s is using vagrant in 2017

(lol jk it's cool and good and better than docker until about mid-2016 lol)

# ? Nov 12, 2017 15:32

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

perf is loving miserable in try/catch
if you're 1000% concerned about perf don't use python lol
or use cython in the perf bits or use numpy or tf or pytorch for numerical poo poo

im not that concerned about perf. i just know that right now if/elif/else chains, even if i make poor mans case/switch statement with 15 elifs (parsing pretty printed salary range strings :v:

), are fast enough for my stuff. in other words, it doesn't sound like i'm winning any points there with try/catch, so i might not need to bother. might just test this back in office

# ? Nov 12, 2017 15:34

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

im not that concerned about perf. i just know that right now if/elif/else chains, even if i make poor mans case/switch statement with 15 elifs (parsing pretty printed salary range strings ), are fast enough for my stuff. in other words, it doesn't sound like i'm winning any points there with try/catch, so i might not need to bother. might just test this back in office

the win is by having lots shorter and more understandable code. sometimes there is not a win.

i would be pretty surprised if there was a perf gain from try catch blocks

# ? Nov 12, 2017 15:39

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

the real c tp s is using vagrant in 2017

(lol jk it's cool and good and better than docker until about mid-2016 lol)

im not sure how the docker vs vagrant goes down, but pycharm has neat vagrant integration. you can surely use docker with it too, but we're talking product-specific integration vs generic remote thing, afaik.

besides, vagrant runs on top of virtualbox which might be what i actually need (im not running a vm locally just because i can poorly do so), and it might actually be running better than docker would since tangential googling suggests that i have an actual vagrant use case, just setting my development environment right. im not deploying anything anywhere, this is for local high-level "io"/analysis tool to try to avoid having linux on dualboot due to data sitting on an external ext4 drive

# ? Nov 12, 2017 15:39

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

the win is by having lots shorter and more understandable code. sometimes there is not a win.

i would be pretty surprised if there was a perf gain from try catch blocks

im not even seeing that strong code readability with the in locals() or better the assert versions. compared to what ive coached my guy into, sure. that 15 elif example? impossible, the only alternative to bruteforcing

"less than 20,000 EUR"
"20,000 to 50,000 EUR"
"50,000 to 100,000 EUR"
"more than 100,000 EUR"

would've been a regex trick that would've taken more than the minute i spent on this

bob dobbs is dead posted:

why this is less idiotic than one may imagine at first glance:
1. you may stomp on the thing in threading so you'd have to handle error anyways
2. exception handling tends to be more durable. classic example is, what if you check existence of file but don't have permission, now you have to think of permission

why this still may loop back to being exactly as idiotic:
1. lol threading in python lol multiprocessing in python
2. better to annoy the hell out of yourself relatively early in development than get in try catch hell

in this specific thing, I would frankly do something for abstract iterator tho

missed this edit. the 2nd too is kinda important here since the point is to parse things as accurately as possible wrt whats expected since our middleman is really loving terrible about handling data. i guess i can also get behind the durability example, but im not sure i can come up with equivalent example to testing types

bob dobbs is dead posted:

also if they're extremely good w/ sql they prolly actually know trivalent logic so you should disabuse them of that knowledge being correct in any other programming context

im not holding my breath, 0-indexing is already proving to be a shocking discovery. not sure if he is good enough to make use of trivalent logic, but im not man enough to delve into thousands of lines long uncommented sql scripts

# ? Nov 12, 2017 15:49

NihilCredo: Jun 6, 2011; iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

tp q: is your development setup really so complicated and unwieldy that you need to learn yet another loving special snowflake configuration file format (vagrantfiles) instead of just saving the same install commands you originally used into a script and chucking that script into vc, and running it on a fresh blank vm when you need a new box?

and in a lot of cases, is your development setup so complicated *and* so often changing that you need to version control it at all instead of just downloading the .ova and maybe cloning it when you want to try an updated ide or whatever?

# ? Nov 12, 2017 15:49

cinci zoo sniper: Mar 15, 2013

NihilCredo posted:

tp q: is your development setup really so complicated and unwieldy that you need to learn yet another loving special snowflake configuration file format (vagrantfiles) instead of just saving the same install commands you originally used into a script and chucking that script into vc, and running it on a fresh blank vm when you need a new box?

and in a lot of cases, is your development setup so complicated *and* so often changing that you need to version control it at all instead of just downloading the .ova and maybe cloning it when you want to try an updated ide or whatever?

no, i want read access on ext4 drive from windows 10. that can allegedly be done with oracle virtualbox, and vagrant is fastest/laziest way to plug oracle virtualbox into pycharm. my interaction with vagrantfile has been limited to uncommenting a single line to enable private network interface, so i can set up project folder synchronisation between local environment and the vm via sftp (again, the laziest way, 1 click in pycharm)

# ? Nov 12, 2017 15:51

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

im not holding my breath, 0-indexing is already proving to be a shocking discovery. not sure if he is good enough to make use of trivalent logic, but im not man enough to delve into thousands of lines long uncommented sql scripts

this statement is incompatible with the person you're teaching actually being good at sql

# ? Nov 12, 2017 15:53

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

this statement is incompatible with the person you're teaching actually being good at sql

neither of us has compsci education or experience, we are just analysts whose sql interactions are limited to querying data. that's what he is really good at, albeit i have no illusions about the perf of his queries, since i remember how etl developer we had at the previous job was speeding up analysts' queries as much as weekend to 40 minutes

# ? Nov 12, 2017 15:56

cinci zoo sniper: Mar 15, 2013

and i mean, as always, i get to be the lucky loving exception, e.g. the designated database architect because this place has no one responsible/motivated enough, and probably no one competent enough either. thankfully, we have hired the etl developer i mentioned the post earlier so he will be taking this over from me before it gets from "here's table layout we want, and some indices" to the stage where my decisions start to actually gently caress things up

# ? Nov 12, 2017 15:57

mystes: May 31, 2006

cinci zoo sniper posted:

no, i want read access on ext4 drive from windows 10. that can allegedly be done with oracle virtualbox, and vagrant is fastest/laziest way to plug oracle virtualbox into pycharm. my interaction with vagrantfile has been limited to uncommenting a single line to enable private network interface, so i can set up project folder synchronisation between local environment and the vm via sftp (again, the laziest way, 1 click in pycharm)

It's surprising that there doesn't seem to be a good way to do this in 2017. I thought of a couple ways that you would think would work but don't seem to actually be possible:
Using dokan on windows: there doesn't seem to be a ext4 driver
Using a user mode program on WSL: no way to get raw access to a block device
Using a user mode program on Windows: there used to be a program called Ltools but it doesn't seem to have been updated in years (the last version only had ext3 support!)

# ? Nov 12, 2017 16:06

cinci zoo sniper: Mar 15, 2013

mystes posted:

It's surprising that there doesn't seem to be a good way to do this in 2017. I thought of a couple ways that you would think would work but don't seem to actually be possible:
Using dokan on windows: there doesn't seem to be a ext4 driver
Using a user mode program on WSL: no way to get raw access to a block device
Using a user mode program on Windows: there used to be a program called Ltools but it doesn't seem to have been updated in years (the last version only had ext3 support!)

ive seen a few more, like paragon ext4 tools etc, but basically it all seems shite. paragon worked i guess, but they provide their own interface to data, rather than mount it on system, which is pita since if i had this much space locally id long since copied it all over through a linux boot stick or something

# ? Nov 12, 2017 16:13

cinci zoo sniper: Mar 15, 2013

c tp mission report s: :patriot:

we are in, huston. that's one small step for the thread, one giant leap for a terrible programmer

# ? Nov 12, 2017 16:19

cinci zoo sniper: Mar 15, 2013

turned out it indeed was the matter of just going to vagrantbox settings in virtualbox and enabling usb 3.0 host controller. after that, my drive showed up as "seagate something" in there, and it worked like a charm. sudo mount /dev/sda1 /media, baby

# ? Nov 12, 2017 16:21

cinci zoo sniper: Mar 15, 2013

now the minor problem is that ive hamstrung myself out a fifth to a quarter of my ram this way, give or take, but, thankfully, ive written this project so it could run on a toaster if need be

# ? Nov 12, 2017 16:26

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

ctps on my liberal arts degree having rear end

rando from the 80s published way to reduce hidden markov model to linear neural net by loving around with the params. basically when you need a probabilistic (kolmogorov probabilistic) transition matrix, you normalize w/ softmax

fun times: this is not actually a linear op. softmax is not a linear op
fun times 2: this is not actually, then, a linear neural net
fun times 3: lol mnist isn't doing the standard linear 92% on hmm w/ ergodic ansatz
fun times 4: I NEED YOU TO BE LINEAR RIGHT NOW OKAY

# ? Nov 12, 2017 16:44

akadajet: Sep 14, 2003

OldAlias posted:

fake

possibly gay as well

# ? Nov 12, 2017 17:01

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

ctps on my liberal arts degree having rear end

rando from the 80s published way to reduce hidden markov model to linear neural net by loving around with the params. basically when you need a probabilistic (kolmogorov probabilistic) transition matrix, you normalize w/ softmax

fun times: this is not actually a linear op. softmax is not a linear op
fun times 2: this is not actually, then, a linear neural net
fun times 3: lol mnist isn't doing the standard linear 92% on hmm w/ ergodic ansatz
fun times 4: I NEED YOU TO BE LINEAR RIGHT NOW OKAY

are you doing some kind of nlp?

# ? Nov 12, 2017 17:02

gonadic io: Feb 16, 2011; >>=

terrible programmer update:

god it is the loving worst when a library has only 3 sources of documentation:
1) a single hello-world spinning square blog post
2) a single application that uses it, that you have to hope uses the feature you need
3) library source, which is a huge mess of interconnecting libraries and type synonyms

# ? Nov 12, 2017 17:12

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

are you doing some kind of nlp?

mnist is pictures of numbers. but it's also from the 90's so it's gotten the poo poo kicked out of it

no, i'm doing basic research for fun

# ? Nov 12, 2017 17:16

cinci zoo sniper: Mar 15, 2013

bob dobbs is dead posted:

mnist is pictures of numbers. but it's also from the 90's so it's gotten the poo poo kicked out of it

no, i'm doing basic research for fun

ah, i see

# ? Nov 12, 2017 17:20

gonadic io: Feb 16, 2011; >>=

To rant a little more: like seriously why is is SO much more complicated to render the text in the window than it is to choose the title bar text? Like I had to specify a glyph cache, an asset loading factory, a font asset file etc.
Also it turns out that you can just pass the unit type as the factory but nowhere is that specified in the list of implementations or in the docs.

If the author hadn't written a soduku program that I could copy paste from, I would not have been able to do it.

# ? Nov 12, 2017 17:28

MrMoo: Sep 14, 2000

Title bar text is a trivial API call, font rendering is a gigantic DIY sequence. Ask me about rendering fonts in JS manually onto a Canvas, using batched 2D calls obviously, to get onto WebGL because I want to use a specific OpenType feature and the typeface license says I cannot just bypass that crap and modify the font file to get it. The amount of tedium to then bring in relatively basic things like bold and italic text is horrendous.

# ? Nov 12, 2017 18:53

gonadic io: Feb 16, 2011; >>=

I mean I kind of aim to give this thing multiple front ends, one of which is capable of being rendered in a browser but gently caress text rendering, it may just put me off.

# ? Nov 12, 2017 19:01

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

cinci zoo sniper posted:

whats a good pattern to teach somewhat new programmer (some r scripting experience, extremely good with sql) for multi-case error handling in python

e.g. what ive shown him now, for intuition's sake, is
Python code:
def findthings(foo):
    foo = None
    found = False
    if isinstance(foo, list):
       baz = foo[0]
       found = True
    elif isinstance(foo, dict):
       baz = foo
       found = True
    else:
       pass
    return baz if found else None
which is a trainwreck and absolute code gore in terms of code quality, refactoring, etcetera, but he has grasped this reasonably well so i dont try to overcomplicate things for his first time seeing python

normally, what i would do in that example
Python code:
def findthings(foo):
    # + optional assertions that baz confirms my expectations
    if isinstance(foo, list):
       baz = foo[0]
    elif isinstance(foo, dict):
       baz = foo
    return baz if "baz" in locals() else None
but there likely is something smarter people than me have figured out to optimally handle this

probably not what you're talking about but I like early returns:

Python code:

def findthings(foo):
    if isinstance(foo, list):
       return foo[0]
    if isinstance(foo, dict):
       return foo
    return None

# ? Nov 12, 2017 23:26

cinci zoo sniper: Mar 15, 2013

Symbolic Butt posted:

probably not what you're talking about but I like early returns:
Python code:
def findthings(foo):
    if isinstance(foo, list):
       return foo[0]
    if isinstance(foo, dict):
       return foo
    return None

why, this would do the same primarily. i dont particularly like the coding style, but it'd work. trickier if i want to make a blanked assertion about the foo/[0], but that's a further part of the song

by the way, i wonder, whats the performance difference on if if vs if elif in this specific scenario

# ? Nov 12, 2017 23:41

Mr SuperAwesome: Apr 6, 2011; im from the bad post police, and i'm afraid i have bad news

early returns seems like the best way imo. better for abstraction. also more idiomatic

i would also never use locals(), you're diving into the python internals which just feels hacky

re: try/catch perf, as said before if you care about perf you're not using python, if you do care about perf you'd be measuring it, and your slowest code almost certainly isn't exception handling

Mr SuperAwesome fucked around with this message at 10:10 on Nov 13, 2017

# ? Nov 13, 2017 10:07

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

there's no perf difference here because the if doesn't fall through. if it didn't then yeah, you would get a slight win from "else if", but more importantly someone reading your code would understand immediately that you expect the cases to be independent

# ? Nov 13, 2017 10:23

cinci zoo sniper: Mar 15, 2013

Mr SuperAwesome posted:

early returns seems like the best way imo. better for abstraction

i would also never use locals(), you're diving into the python internals which just feels hacky

re: try/catch perf, as said before if you care about perf you're not using python, if you do care about perf you'd be measuring it, and your slowest code almost certainly isn't exception handling

early returns can be really annoying since it would force me to abstract verification, should i chose to do one

nothing wrong with locals(), an object will not exist in a function instance if i don't create it. sure, i might be flying close to the sun, but it works

getting really tired of this thread meme "le mao xddddddd why u goof are using werd "perf" in poopon". just because it isnt fastest thing on the planet does not invalidate my desire not to slow done code a 100 times or whatever because of some ~pythonic~ way for exception handling that yields identical result.

and yes, i fully expect exception handling to be one of the slowest parts by far, since everything else is bare pandas/numpy. it is blazing fast as is, and i would rapidly run out of things to optimise before my natural next step would become cython or numba or something.

# ? Nov 13, 2017 10:39

cinci zoo sniper: Mar 15, 2013

rjmccall posted:

there's no perf difference here because the if doesn't fall through. if it didn't then yeah, you would get a slight win from "else if", but more importantly someone reading your code would understand immediately that you expect the cases to be independent

i dont, which is why i wondered why symbolic butt put two ifs. its the same data, it can just be a list or a dict because our developers are bad coworkers and programmers alike

# ? Nov 13, 2017 10:41

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

cinci zoo sniper posted:

i dont, which is why i wondered why symbolic butt put two ifs. its the same data, it can just be a list or a dict because our developers are bad coworkers and programmers alike

in this case it compiles to the same bytecode, you can check it with dis.

I used 2 ifs instead of if-elif out of habit. I like to start functions with a bunch of "if condition: return" statements, functional programmers like to call these guards. they usually simplify a lot of the code in my experience, you try to deal with the "special" cases right at the beginning instead of leaving it to the middle of the logic of the function.

and oh I know it's kind of silly to think too much about this example but in a case like this I wouldn't write a function that works for both a list and a dictionary. I'd just write one function that works for list-likes and leave the responsibility of converting a dict to a list for the caller. :shrug:

and then maybe the caller should do the try-except if it's a relevant thing for their control flow (which is bad):

Python code:

try:
    findthings(myshit)
except gently caress:
    findthings(myshit.values())

# ? Nov 13, 2017 11:51

cinci zoo sniper: Mar 15, 2013

Symbolic Butt posted:

in this case it compiles to the same bytecode, you can check it with dis.

I used 2 ifs instead of if-elif out of habit. I like to start functions with a bunch of "if condition: return" statements, functional programmers like to call these guards. they usually simplify a lot of the code in my experience, you try to deal with the "special" cases right at the beginning instead of leaving it to the middle of the logic of the function.

sure, these guards is something i would normally do yeah. irl on that example i need to run a loop on each list element so you still end up with conditional logic, even if you abstract it to an external object. still a better practice for debugging, not going to deny that. in theory i could write a custom data structure or something and import my logic into it separately, but that sounds like way too serious effort for just janitoring up a beat-down sql

Symbolic Butt posted:

and oh I know it's kind of silly to think too much about this example but in a case like this I wouldn't write a function that works for both a list and a dictionary. I'd just write one function that works for list-likes and leave the responsibility of converting a dict to a list for the caller.

and then maybe the caller should do the try-except if it's a relevant thing for their control flow (which is bad):
Python code:
try:
    findthings(myshit)
except gently caress:
    findthings(myshit.values())

the problems is i have hundreds of thousands of multi-megabyte jsons as a result of broken xml-to-json conversion by our idiot it that ends up spitting lists for multiple arrays, dictionaries for single arrays, and empty or missing elements for when there is no data. i would very much like to do like you say, but that does not appear to be possible within reason

# ? Nov 13, 2017 11:59

Mr SuperAwesome: Apr 6, 2011; im from the bad post police, and i'm afraid i have bad news

cinci zoo sniper posted:

just because it isnt fastest thing on the planet does not invalidate my desire not to slow done code a 100 times or whatever because of some ~pythonic~ way for exception handling that yields identical result.

maintainability, readability and being idiomatic is often (IME, almost always) more important than pure performance

this is also the point of python: https://www.python.org/dev/peps/pep-0020/

# ? Nov 13, 2017 12:00

cinci zoo sniper: Mar 15, 2013

Mr SuperAwesome posted:

maintainability, readability and being idiomatic is often (IME, almost always) more important than pure performance

this is also the point of python: https://www.python.org/dev/peps/pep-0020/

ignoring know your audience etc and my opinion on opinionated programming things by people like van rossum, pike, wickham, etc, could you please on this example tell me how readability improves by substituting an if with a nested return?

Python code:

def fix_nest(x, y: str):
    """
    Flatten an arbitrary nested vector (e.g. <datasource>).

    Args:
        x: Element which may subset in a dictionary.
        y (str): Dictionary key.

    Returns:
        Flattened x.
    """
    if isinstance(x, dict):
        x = x[y]
    return x

my use case is not to publish it, or hell, not to even run it more than twice, or show anyone but one coworker. performance trumps it all for me, and i think i can find my way around 1-2 levels deep conditionals that i personally wrote, and have commented where necessary

# ? Nov 13, 2017 12:03

cinci zoo sniper: Mar 15, 2013

basically just forget i asked :effort:

# ? Nov 13, 2017 12:10

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 21:41

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

Python code:

def fix_nest(x, y: str):
    """
    Flatten an arbitrary nested vector (e.g. <datasource>).

    Args:
        x: Element which may subset in a dictionary.
        y (str): Dictionary key.

    Returns:
        Flattened x.
    """
    if isinstance(x, dict):
        return x[y]
    return x

this is slightly faster and it's pretty readable to me

you gotta float like a butterfly with returns man, don't let variables weigh on you (imo)

# ? Nov 13, 2017 12:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > i actually have reasons to post here unlike half of the people with impostor syndrome

«‹›239 »