Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Boris Galerkin posted:

Why does it seem like there�s suddenly a drive to static type everything now? I thought not having to do that was one of the core things with python.

cinci zoo sniper posted:

Then Guido worked on a real life Python project.

This is largely it. Guido got a job at Dropbox and all of a sudden he's dealing with stuff that's not small.

There's no actual need for static typing for many of the things Python does, but the larger your code base, the larger your team, and the larger your development timeframe, the more useful static typing becomes.

That being said, I use type annotations no matter the size of my project. The reason for this is because it serves as good documentation. I've always tried to write good comments, docstrings, and documentation. Type annotations can replace and/or support a lot of that even if you don't get any real benefit out of static typing.

# ? Feb 3, 2019 23:47

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 04:42

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

Boris Galerkin posted:

Why does it seem like there’s suddenly a drive to static type everything now? I thought not having to do that was one of the core things with python.

My impression was that the main motivation for the push on static typing is to provide better hints for bytecode optimization.

And yeah, I feel like this push is not really playing with python's strengths but that's like just my opinion. Language design is an ever changing thing.

# ? Feb 4, 2019 00:31

NinpoEspiritoSanto: Oct 22, 2013

Static typing makes code more predictable and easier to reason about. A Google for various PHP hilarities are mostly down to PHP occupying the complete opposite end of the typing spectrum to static typing where unlike Python it will go so far as coercing types of things to make lines of code work often with bizarre, surprising and even nonsensical results.

As the age old advice says: Take care of your data structures first and the logic almost writes itself. Unpredictable data is loving hard to write logic for not to mention debug.

# ? Feb 4, 2019 01:03

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Type annotations don't imply static typing. You can do whatever wierd type stuff you like, whenever you like. You can do stuff like dependant types (now with a sensible syntax) since annotations can be callable. You can use it just to call a bunch of assertions without using mypy or any static checks you want. It's just slow as poo poo.

# ? Feb 4, 2019 02:12

NinpoEspiritoSanto: Oct 22, 2013

Oh I wasn't suggesting any of it gives static typing (though I believe there are libs you can leverage to get that at runtime), just chiming in on why it's become "a thing".

# ? Feb 4, 2019 09:19

Dominoes: Sep 20, 2007

Just did a major refactoring on one of my projects. Made the root change, then followed the red squiggles. Would have been a major pain without type checking.

# ? Feb 4, 2019 15:13

fourwood: Sep 9, 2001; Damn I'll bring them to their knees.

Can anyone recommend a place to read up on some more modern-ish Python features like type hinting? I feel like I�m an okay basic Pythoner but am not very good at keeping up to date on this kind of stuff in general.

# ? Feb 4, 2019 19:37

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

fourwood posted:

Can anyone recommend a place to read up on some more modern-ish Python features like type hinting? I feel like I�m an okay basic Pythoner but am not very good at keeping up to date on this kind of stuff in general.

It doesn't cover type hinting, but the book Fluent Python will get you into a bunch of features of Python you probably don't know about or don't use.

Type hinting in python is easy. The hard part is understanding types in general...and I don't know of good resources for that.

Probably reading through PEP-484 will help you.

# ? Feb 5, 2019 02:24

NinpoEspiritoSanto: Oct 22, 2013

Idiomatic Python by Jeff Knupp is a great Python tricks style book too.

# ? Feb 5, 2019 02:56

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

Thermopyle posted:

It doesn't cover type hinting, but the book Fluent Python will get you into a bunch of features of Python you probably don't know about or don't use.

Type hinting in python is easy. The hard part is understanding types in general...and I don't know of good resources for that.

Probably reading through PEP-484 will help you.

Learn you a haskell to understand types way too loving well

# ? Feb 5, 2019 03:16

breaks: May 12, 2001

I�d actually recommend against learning some other typed language and trying to apply it directly to Python. It�s just a little weird especially around stuff like mypy and pycharm compatibiltity, when to use interfaces vs I guess traditional types and so on. The hard stuff to understand about typing is things like, you have a class that is often subclassed and has some factory method in the superclass that returns an instance of whichever subclass, how do you write that out, and the complicated situations are just a bit WIP at the moment, particularly as I mentioned wrt things that are acceptable across various tools. Long story short due to newness and the language itself you need to learn to accept and work with python�s relatively fuzzy static typing concept.

# ? Feb 5, 2019 03:35

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

"way too loving well" was not a joke, it will handicap you in other langs, lol

# ? Feb 5, 2019 03:41

Dominoes: Sep 20, 2007

Do y'all feel LYAH is too heavy on standard lib coverage? I got overwhelmed with the perpetual stream of new funcs being introduced. Like API docs in prose.

Dominoes fucked around with this message at 04:07 on Feb 5, 2019

# ? Feb 5, 2019 04:00

procgen: Feb 27, 2016

I just discovered the mypy and pylint plugins for PyCharm. They both provide a basic gui for running scans and sorting through any detected issues. It's nice being able to jump directly to affected lines, and the mypy plugin supports code highlighting with cursor inspection which I've found useful as I re-familiarize myself with a long-dormant codebase.

# ? Feb 5, 2019 04:28

KICK BAMA KICK: Mar 2, 2009

In the hobby project I'm working on I was using the default docstring format PyCharm generates when you type triple-quotes after a def and I hate those to begin with aesthetically and don't even document anything but type hints for parameters anyway so this conversation makes me wanna switch to the actual native type hints. Actually have question related to that:

code:

response = requests.get(url)
# Check for validity, etc...
stuff = response.content
numpy_array = np.fromstring(stuff, np.uint8)

Works exactly as it should but PyCharm inspection flags this because stuff is bytes but np.fromstring obviously expects str. Would you bother to do anything about that, if there is anything to do? You can just suppress the inspection but I don't like that cause it throws some PyCharm-specific comment above that line that wouldn't mean anything to anyone else, human or tool.

# ? Feb 5, 2019 05:08

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

KICK BAMA KICK posted:

In the hobby project I'm working on I was using the default docstring format PyCharm generates when you type triple-quotes after a def and I hate those to begin with aesthetically and don't even document anything but type hints for parameters anyway so this conversation makes me wanna switch to the actual native type hints. Actually have question related to that:
code:
response = requests.get(url)
# Check for validity, etc...
stuff = response.content
numpy_array = np.fromstring(stuff, np.uint8)
Works exactly as it should but PyCharm inspection flags this because stuff is bytes but np.fromstring obviously expects str. Would you bother to do anything about that, if there is anything to do? You can just suppress the inspection but I don't like that cause it throws some PyCharm-specific comment above that line that wouldn't mean anything to anyone else, human or tool.

Use response.text. Requests will decode in the correct format as long as its able to figure it out from headers or some poo poo.

# ? Feb 5, 2019 06:37

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

breaks posted:

I�d actually recommend against learning some other typed language and trying to apply it directly to Python. It�s just a little weird especially around stuff like mypy and pycharm compatibiltity, when to use interfaces vs I guess traditional types and so on. The hard stuff to understand about typing is things like, you have a class that is often subclassed and has some factory method in the superclass that returns an instance of whichever subclass, how do you write that out, and the complicated situations are just a bit WIP at the moment, particularly as I mentioned wrt things that are acceptable across various tools. Long story short due to newness and the language itself you need to learn to accept and work with python�s relatively fuzzy static typing concept.

So out of curiosity how would you do that?

code:

class Foo:
    def from_dict(cls, d) -> ???:
        return cls(parse(d))

If I put Foo in there then Pycharm either puts squiggles under it or tells me there's something wrong.

Also I've been annotating all the types in docstrings cause that's what Pycharm does for me,

code:

def foo(bar):
    """
    Does a foo.

    Args:
        bar (a_thing): the input to foo
    """

Does adding the new type annotations do/add anything that the docstring above doesn't?

# ? Feb 5, 2019 06:58

SirPablo: May 1, 2004; Pillbug

Not sure how handy anyone is with xarray and dask, but figure I'll ask...

I have about 13,000 ASCII files that contain raster floats. Each file is a 1405x621 array representing a given day's high temperature. Combined it is about 50 GB of data. xarray+dask seems like a very ideal for working with these data. My hang-up is, how can I efficiently load all these files in a dask array? It isn't a netCDF file, so I'm struggling with how to just do the initial load. I can use numpy.fromfile to load individual files, but that doesn't seem like it'll flow out into a parallel process.

Here's a snip of code that works, but it'll eat up my memory very quickly. Any ideas?

code:

data = [np.fromfile(x, dtype='float').reshape((1405,621)) for x in [list_of_files]]

# ? Feb 5, 2019 07:00

Foxfire_: Nov 8, 2010

Write it back out to disk as a big binary array, memmap it, and let the OS swap it in and out as needed?

# ? Feb 5, 2019 07:58

punished milkman: Dec 5, 2018; would have won

Anyone have any thoughts on Poetry as a dependency manager/packager? I was considering trying to get into pipenv again but I had a bad time on my last attempt and Poetry seems a lot snazzier. I've just been using pip (and sometimes conda, but making a new conda package is a real hassle) which hasn't been too bad in my experience, but it feels like we're moving away from pure pip so I should adapt.

# ? Feb 5, 2019 17:11

NinpoEspiritoSanto: Oct 22, 2013

Pipenv is terrible imo. What's wrong with setup.py?

# ? Feb 5, 2019 17:20

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

punished milkman posted:

Anyone have any thoughts on Poetry as a dependency manager/packager? I was considering trying to get into pipenv again but I had a bad time on my last attempt and Poetry seems a lot snazzier. I've just been using pip (and sometimes conda, but making a new conda package is a real hassle) which hasn't been too bad in my experience, but it feels like we're moving away from pure pip so I should adapt.

poetry is better than pipenv, but in i wouldn't be terribly surprised if pipenv "won" out. And while I'd be glad if poetry became the standard, I just hope one of them becomes the standard because python packaging is loving stupid. All other languages should be pointing at Python and laughing.

# ? Feb 5, 2019 17:33

cinci zoo sniper: Mar 15, 2013

Thermopyle posted:

poetry is better than pipenv, but in i wouldn't be terribly surprised if pipenv "won" out. And while I'd be glad if poetry became the standard, I just hope one of them becomes the standard because python packaging is loving stupid. All other languages should be pointing at Python and laughing.

Maybe except JavaScript.

# ? Feb 5, 2019 17:35

The Fool: Oct 16, 2003

I understand PHP has a pretty bad package management situation too.

# ? Feb 5, 2019 18:12

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Except at least they have a folder within a project they install modules to and I don't have to explain virtualenvs to people starting out. Can't wait for PEP-582 to be implemented.

# ? Feb 5, 2019 18:30

punished milkman: Dec 5, 2018; would have won

Master_Odin posted:

Except at least they have a folder within a project they install modules to and I don't have to explain virtualenvs to people starting out. Can't wait for PEP-582 to be implemented.

Wow this is great, first time I've seen this PEP

# ? Feb 5, 2019 18:33

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Sir, the tests just returned, I'm sorry to state that it is an instance of Haskell Brain. Unfortunately, we know nothing of an applicable cure. One it has been bound, it is very difficult to lift back out. We simply cannot erase Haskell Brain once it has been constructed. You have been reduced to two weeks to find some new job. Otherwise, this case of Haskell Brain is functionally terminating. My condolences, I will join you in prayer. I wish this were in error.

oh no ... oh GOD

dougdrums fucked around with this message at 21:57 on Feb 5, 2019

# ? Feb 5, 2019 21:54

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

dougdrums posted:

Sir, the tests just returned, I'm sorry to state that it is an instance of Haskell Brain. Unfortunately, we know nothing of an applicable cure. One it has been bound, it is very difficult to lift back out. We simply cannot erase Haskell Brain once it has been constructed. You have been reduced to two weeks to find some new job. Otherwise, this case of Haskell Brain is functionally terminating. My condolences, I will join you in prayer. I wish this were in error.

oh no ... oh GOD

This but unironically

# ? Feb 5, 2019 21:58

Dominoes: Sep 20, 2007

Master_Odin posted:

Except at least they have a folder within a project they install modules to and I don't have to explain virtualenvs to people starting out. Can't wait for PEP-582 to be implemented.

Great!

# ? Feb 6, 2019 02:43

Furism: Feb 21, 2006; Live long and headbang

I'm working on a class to interact with a REST API so I'm using Requests and some debugging tools. I'm using Python 3.7. My class starts like this:

code:

from requests.packages.urllib3.exceptions import InsecureRequestWarning
import requests
import json
import pickle
import logging
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

However VS Code (the Python extension for it, or more accurately, PyLint) complains that

quote:

Unable to import 'requests.packages.urllib3.exceptions'
Module 'requests.packages' has no 'urllib3' member

But at run time it all works as expected. What am I doing wrong?

Furism fucked around with this message at 10:53 on Feb 6, 2019

# ? Feb 6, 2019 10:20

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Furism posted:

I'm working on a class to interact with a REST API so I'm using Requests and some debugging tools. I'm using Python 3.7. My class starts like this:
code:
from requests.packages.urllib3.exceptions import InsecureRequestWarning
import requests
import json
import pickle
import logging
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
However VS Code (the Python extension for it, or more accurately, PyLint) complains that

But at run time it all works as expected. What am I doing wrong?

You're doing nothing wrong. The linter just can't "find" it because the requests.packages does some kind of magic to load that package or whatever.

code:

import sys

# This code exists for backwards compatibility reasons.
# I don't like it either. Just look the other way. :)

for package in ('urllib3', 'idna', 'chardet'):
    locals()[package] = __import__(package)
    # This traversal is apparently necessary such that the identities are
    # preserved (requests.packages.urllib3.* is urllib3.*)
    for mod in list(sys.modules):
        if mod == package or mod.startswith(package + '.'):
            sys.modules['requests.packages.' + mod] = sys.modules[mod]

# Kinda cool, though, right?

# ? Feb 6, 2019 13:38

SurgicalOntologist: Jun 17, 2004

You might be able to avoid the message by importing directly from urllib3.

# ? Feb 6, 2019 15:25

Furism: Feb 21, 2006; Live long and headbang

Thanks guys!

# ? Feb 6, 2019 18:55

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

I'm looking for a number that behaves like a float (all math methods implemented) but also has extra metadata and methods. The extra stuff is just extra/optional things. Would I want to (a) subclass from float for this, (b) use the numbers module (not really sure what this is for to be honest), or (c) not do this?

# ? Feb 7, 2019 10:59

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

Boris Galerkin posted:

I'm looking for a number that behaves like a float (all math methods implemented) but also has extra metadata and methods. The extra stuff is just extra/optional things. Would I want to (a) subclass from float for this, (b) use the numbers module (not really sure what this is for to be honest), or (c) not do this?

What's your ulterior goal

# ? Feb 7, 2019 22:08

NtotheTC: Dec 31, 2007

Master_Odin posted:

Except at least they have a folder within a project they install modules to and I don't have to explain virtualenvs to people starting out. Can't wait for PEP-582 to be implemented.

Is this entirely wise though? Not having to explain virtualenvs is great and the activate / deactivate schtick is terrible implementation but tools like poetry abstract that away behind a much nicer interface that functions similarly to this except you don't use the system python.

What happens with this pep if you accidently install a dependency into the system python, dont realise when importing the lib into your project that it isn't local, and then get baffling errors?

If the goal is to remove virtualenvs entirely then we need a way to replicate things likr pipx-app for exposing python command line apps neatly. And if the goal isnt to replace virtualenvs completely are you not just kicking the can down the road?

NtotheTC fucked around with this message at 22:35 on Feb 7, 2019

# ? Feb 7, 2019 22:29

NtotheTC: Dec 31, 2007

Related: https://www.jetbrains.com/research/python-developers-survey-2018/

21% of Python devs dont use any sort of isolation for developing :psyduck:

# ? Feb 7, 2019 22:37

wolrah: May 8, 2006; what?

NtotheTC posted:

Related: https://www.jetbrains.com/research/python-developers-survey-2018/

21% of Python devs dont use any sort of isolation for developing

I mean...should we want to?

I'm only a sysadmin who happens to like Python, definitely not a professional programmer, but while I totally understand why these things are sometimes necessary I also think it's something that should be avoided unless you need it, not used by default. I don't want to have to think about what environment an application might need, I just want to run it. If it really needs something that'd conflict with my distro's Python environment then so be it, but I'd rather not if I don't have to.

I like PEP 582 and hope it's accepted and rapidly adopted by the community for this reason.

# ? Feb 7, 2019 23:10

Presto: Nov 22, 2002; Keep calm and Harry on.

NtotheTC posted:

Related: https://www.jetbrains.com/research/python-developers-survey-2018/

21% of Python devs dont use any sort of isolation for developing

This Python dev doesn't even understand what you mean.

# ? Feb 7, 2019 23:40

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 04:42

NtotheTC: Dec 31, 2007

I mean thats two separate points really. You want to avoid using system python for installing an apps dependencies into because you may have multiple apps on the same system that have conflicting requirements. If you already isolate the python environment becsuse you use containers or whatever then absolutely you can install things into system python.

The PEP makes it a bit simpler to isolate by default which would be nice in production without needing extra tooling. Even in containers its a nice bit of redundancy, but I can't work out if their goal with that PEP is to replace virtualenvs entirely or just be "virtualenv lite" to aid with teaching beginners

NtotheTC fucked around with this message at 23:53 on Feb 7, 2019

# ? Feb 7, 2019 23:44

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »