Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Eela6: May 25, 2007; Shredded Hen

As far as NotImplemented vs NotImplementedError goes, read the docs at https://docs.python.org/3/library/constants.html or Ch. 13 of Fluent Python: Operator Overloading, or see this post I just made where I explained how the interpereter uses NotImplemented

Eela6 fucked around with this message at 22:59 on Jul 21, 2017

# ? Jul 21, 2017 22:54

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 13:45

onionradish: Jul 6, 2006; That's spicy.

Eela6 posted:

As far as NotImplemented vs NotImplementedError goes, read the docs at https://docs.python.org/3/library/constants.html or Ch. 13 of Fluent Python: Operator Overloading.

I'd never seen that before. I learned something new! Thanks again.

EDIT: a follow up... in your "go crazy" example, you omit the hasattr(other, "_public_attribs") test inside __eq__. Is it not necessary?

onionradish fucked around with this message at 23:24 on Jul 21, 2017

# ? Jul 21, 2017 23:00

Eela6: May 25, 2007; Shredded Hen

onionradish posted:

I'd never seen that before. I learned something new! Thanks again.

EDIT: a follow up... in your "go crazy" example, you omit the hasattr(other, "_public_attribs") test inside __eq__. Is it not necessary?

Check this out.

0. self.__eq__(other) -> self._public_attribs == other
1. self._public_attribs.__eq__(other) is NotImplemented.
2. Fallback to other.__eq__(self._public_attribs) -> other._public_attributes == self.public_attribs.
3. These are both dictionaries, so they compare 'normally'.

Python is cool.

Eela6 fucked around with this message at 23:40 on Jul 21, 2017

# ? Jul 21, 2017 23:36

onionradish: Jul 6, 2006; That's spicy.

Eela6 posted:

Check this out.

0. self.__eq__(other) -> self._public_attribs == other
1. self._public_attribs.__eq__(other) is NotImplemented.
2. Fallback to other.__eq__(self._public_attribs) -> other._public_attributes == self.public_attribs.
3. These are both dictionaries, so they compare 'normally'.

Python is cool.

My melon is officially twisted. I'm leaving the hasattr comparison in for explicitness, but I'm digging how slick that is.

Eela6 posted:

Python is cool.

onionradish fucked around with this message at 23:59 on Jul 21, 2017

# ? Jul 21, 2017 23:44

Eela6: May 25, 2007; Shredded Hen

onionradish posted:

My melon is officially twisted. I'm leaving the hasattr comparison in for explicitness, but I'm digging how slick that is.

I agree with your decision, actually. Explicit is better than implicit. I just wanted to show off :getin:

# ? Jul 21, 2017 23:55

Le0: Mar 18, 2009; Rotten investigator!

I'm currently using PyCharm to debug my stuff but I noticed that no matter what I do, the libs in Python\Python35\Lib\site-packages cannot be debugged.
Is there anything special I should be doing to be able to debug those files?

# ? Jul 24, 2017 15:42

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Is it ok to do this?

code:

class Foo:
    def __init__(self, parameters):
        self.parameters = parameters

    def __getattr__(self, key):
        try:
            return self.__dict__['_' + key]

        except KeyError:
            value = bar(self.parameters[key])
            setattr(self, '_' + key, value)
            return getattr(self, key)

parameters would be a dict, and bar() would do some calculations that only need to be done once and can be reused, but I might not always need said value. So my intention was to just evaluate and cache it iff I needed to.

I'm just not really sure if it's ok to use the __dict__ attribute like I am, and if I'm using __getattr__ properly.

# ? Jul 27, 2017 08:59

accipter: Sep 12, 2003

Boris Galerkin posted:

Is it ok to do this?
code:
class Foo:
    def __init__(self, parameters):
        self.parameters = parameters

    def __getattr__(self, key):
        try:
            return self.__dict__['_' + key]

        except KeyError:
            value = bar(self.parameters[key])
            setattr(self, '_' + key, value)
            return getattr(self, key)
parameters would be a dict, and bar() would do some calculations that only need to be done once and can be reused, but I might not always need said value. So my intention was to just evaluate and cache it iff I needed to.

I'm just not really sure if it's ok to use the __dict__ attribute like I am, and if I'm using __getattr__ properly.

What about this?

Python code:

class Foo:
    def __init__(self):
        self._bar = None

    @property
    def bar(self):
        if self._bar is None:
            self._bar = self._calculate_bar()
    
        return self._bar

Depending on what calculations you are trying to cache, you might consider https://pythonhosted.org/joblib/memory.html or http://code.activestate.com/recipes/52201/

# ? Jul 27, 2017 16:17

Loezi: Dec 18, 2012; Never buy the cheap stuff

Boris Galerkin posted:

Is it ok to do this?
code:
class Foo:
    def __init__(self, parameters):
        self.parameters = parameters

    def __getattr__(self, key):
        try:
            return self.__dict__['_' + key]

        except KeyError:
            value = bar(self.parameters[key])
            setattr(self, '_' + key, value)
            return getattr(self, key)
parameters would be a dict, and bar() would do some calculations that only need to be done once and can be reused, but I might not always need said value. So my intention was to just evaluate and cache it iff I needed to.

I'm just not really sure if it's ok to use the __dict__ attribute like I am, and if I'm using __getattr__ properly.

Is there some reason @lru_cache doesn't work?

# ? Jul 27, 2017 16:25

Hadlock: Nov 9, 2004

For those who have used VS Code in the last ~6 months (the vscode product has changed/improved a lot since early last year) What's the delta between vscode and pycharm these days once you plugin the top two or three vscode python plugins? I've been using vscode now for a little over a year writing Go, Powershell, Bash etc and trying to see if the jump to pycharm CE is worth it for Python.

# ? Jul 27, 2017 17:56

accipter: Sep 12, 2003

Loezi posted:

Is there some reason @lru_cache doesn't work?

Thanks, I never knew that existed.

# ? Jul 27, 2017 18:04

Dominoes: Sep 20, 2007

Hadlock posted:

For those who have used VS Code in the last ~6 months (the vscode product has changed/improved a lot since early last year) What's the delta between vscode and pycharm these days once you plugin the top two or three vscode python plugins? I've been using vscode now for a little over a year writing Go, Powershell, Bash etc and trying to see if the jump to pycharm CE is worth it for Python.

Pycharm for large, multi-file projects. VSCode for editing individual files.

# ? Jul 27, 2017 18:12

Loezi: Dec 18, 2012; Never buy the cheap stuff

Dominoes posted:

Pycharm for large, multi-file projects. VSCode for editing individual files.

This, but VSCode also when you are not running on a proper workstation. I'm simply unable to use PyCharm on my work laptop (SSD, 16Gb memory, i5-5300U) because it's so sluggish.

# ? Jul 27, 2017 18:32

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

Is it ok to do this?
code:
class Foo:
    def __init__(self, parameters):
        self.parameters = parameters

    def __getattr__(self, key):
        try:
            return self.__dict__['_' + key]

        except KeyError:
            value = bar(self.parameters[key])
            setattr(self, '_' + key, value)
            return getattr(self, key)
parameters would be a dict, and bar() would do some calculations that only need to be done once and can be reused, but I might not always need said value. So my intention was to just evaluate and cache it iff I needed to.

I'm just not really sure if it's ok to use the __dict__ attribute like I am, and if I'm using __getattr__ properly.

This seems like a take on lazy properties; i.e, calculate once, then store. I personally like the lazyproperty decorator described in the Python Cookbook by Beazley and Jones:

Python code:

class lazyproperty:
    def __init__(self, func):
        self.func = func

    def __get__(self, instance, cls):
        if instance is None:
            return self
        else:
            value = self.func(instance)
            setattr(instance, self.func.__name__, value)
            return value

Python code:

import math

class Circle:
    def __init__(self, radius):
        self.radius = radius

    @lazyproperty
    def area(self):
        print('Computing area')
        return math.pi * self.radius ** 2

    @lazyproperty
    def perimeter(self):
        print('Computing perimeter')
        return 2 * math.pi * self.radius

Python code:

>>> c = Circle(4.0)
>>> c.radius
4.0
>>> c.area
Computing area
50.26548245743669
>>> c.area
50.26548245743669
>>> c.perimeter
Computing perimeter
25.132741228718345
>>> c.perimeter
25.132741228718345
>>>

# ? Jul 27, 2017 18:36

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Hadlock posted:

For those who have used VS Code in the last ~6 months (the vscode product has changed/improved a lot since early last year) What's the delta between vscode and pycharm these days once you plugin the top two or three vscode python plugins? I've been using vscode now for a little over a year writing Go, Powershell, Bash etc and trying to see if the jump to pycharm CE is worth it for Python.

I find VS Code's python features acceptable, but definitely not as good as PyCharm. It's good for small projects, scripts, or making modifications to a larger project quickly. I just used VS 2017 Community for a project someone was working on in PyCharm and it worked almost as well as using VS for usual C# debugging. I was able to start a PyCharm project up and debug it with no problems.

VS2017 really stepped up the python support. You can run Flask projects on azure straight from it iirc.

Loezi posted:

This, but VSCode also when you are not running on a proper workstation. I'm simply unable to use PyCharm on my work laptop (SSD, 16Gb memory, i5-5300U) because it's so sluggish.

Yeah I prefer Code for laptop editing in pretty much any language.

# ? Jul 27, 2017 18:38

Hadlock: Nov 9, 2004

Thanks yeah I am on an aging 2013-era i5 laptop with 8gb memory, will stick with VSCode for now it sounds like. I'm at a medium sized rails shop and while some of the old timers are still using Ruby Mine, about 50% of developers are using vscode these days.

Seems that pycharm is worth at least exploring a bit so that I'm at least familiar with it. Thanks!

# ? Jul 27, 2017 18:49

Loezi: Dec 18, 2012; Never buy the cheap stuff

What's the easiest library to make a GUI with, given that I require hassle-free licensing (which rules out PyQt)?

Native elements are appreciated but not necessarily required.

Loezi fucked around with this message at 22:21 on Jul 29, 2017

# ? Jul 29, 2017 22:16

breaks: May 12, 2001

I think your options are wxPython (which they have finally gotten around to cleaning up a bit) if you care about native elements, kIvy if you don't, or just do a web app with something that can fairly easily pack it up into a distributable binary (web2py, not sure if there are others).

# ? Jul 29, 2017 23:13

QuarkJets: Sep 8, 2008

Loezi posted:

What's the easiest library to make a GUI with, given that I require hassle-free licensing (which rules out PyQt)?

Native elements are appreciated but not necessarily required.

PySide is LGPL which is hassle-free

# ? Jul 30, 2017 00:28

huhu: Feb 24, 2006

Loezi posted:

What's the easiest library to make a GUI with, given that I require hassle-free licensing (which rules out PyQt)?

Native elements are appreciated but not necessarily required.

Flask is also very nice.

# ? Jul 30, 2017 04:00

Loezi: Dec 18, 2012; Never buy the cheap stuff

QuarkJets posted:

PySide is LGPL which is hassle-free

Has the "LGPL allows import statements if you do not distribute the libraries together with your software" ever been commented on by the GNU lawyers, or better yet been tested in a court? I seem to recall that while it's a reasonable interpretation of the license terms, so is the opposite.

# ? Jul 30, 2017 09:14

QuarkJets: Sep 8, 2008

No, I don't think there's a definitive answer on whether importing a GPL module is legally distinct from linking to a GPL library. I don't think it matters so much for LGPL but it's a pretty important distinction for GPL, so I understand why people developing commercial software might want to avoid GPL code

# ? Jul 30, 2017 13:46

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Loezi posted:

What's the easiest library to make a GUI with, given that I require hassle-free licensing (which rules out PyQt)?

Native elements are appreciated but not necessarily required.

Keep in mind that most answers to these type of questions will come from people who have only seriously used one or two of the options.

I've used 4 different options on pretty large applications and I'd say you'll have the easiest time with HTML/CSS/JS + some python web framework and a browser.

However, every solution has its pros and cons and its hard to say which is actually easiest when there's so many variables from your experience level to the type of application you're developing to your ability to handle context switching between paradigms.

# ? Jul 30, 2017 16:28

QuarkJets: Sep 8, 2008

Thermopyle posted:

Keep in mind that most answers to these type of questions will come from people who have only seriously used one or two of the options.

I've used 4 different options on pretty large applications and I'd say you'll have the easiest time with HTML/CSS/JS + some python web framework and a browser.

However, every solution has its pros and cons and its hard to say which is actually easiest when there's so many variables from your experience level to the type of application you're developing to your ability to handle context switching between paradigms.

Yeah we're all going to have our biases

My thoughts are that the HTML/CSS/JS route is abhorrent, Qt (and therefore PySide) is love, wxPython is an acceptable alternative to Qt, and tKinter is a bit ugly and has a bad API

# ? Jul 31, 2017 02:57

Loezi: Dec 18, 2012; Never buy the cheap stuff

Thermopyle posted:

I've used 4 different options on pretty large applications and I'd say you'll have the easiest time with HTML/CSS/JS + some python web framework and a browser.

I've used the browser + bottle + HTML route before, the main problem is that I'd like this to be something that I can package into a single "thing" the users run and that quits when the user click the window away.

In other words, this needs to work for non-technical users, so a workflow of "start server, go to localhost:8080, quit server when done" is too complicated. And due to <reasons> (mainly: this things does long and heavy computations) dedicated hosting is not an option.

Is there something like Electron for python 3?

I guess I could work out some frankenscript for all the operating systems that starts the server, opens a page in $BROWSER and once $BROWSER quits, quits the server as well. But that sounds like something there should be library or a tool for.

# ? Jul 31, 2017 15:11

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Electron will work with python 3. Electron doesn't care what language you use on the "backend".

Also there is a browser starter-upper thing in the standard library, IIRC. I just can't remember what it's called and I'm on my phone right now and don't feel like going through the effort.

Thermopyle fucked around with this message at 16:41 on Jul 31, 2017

# ? Jul 31, 2017 16:39

Methanar: Sep 26, 2013; by the sex ghost

Loezi posted:

In other words, this needs to work for non-technical users, so a workflow of "start server, go to localhost:8080, quit server when done" is too complicated. And due to <reasons> (mainly: this things does long and heavy computations) dedicated hosting is not an option.

Can't you just roll these into a bash/powershell file to make it single click?

Or this

https://docs.python.org/3/library/webbrowser.html

# ? Jul 31, 2017 21:22

Data Graham: Dec 28, 2009; 📈📊🍪😋

So here's me refactoring my API crawling script for speed.

"I know, I'll make it multithreaded!"

*reads this article about the GIL*

"Surely the leopard won't eat MY face!"

Before threading on a 4-CPU box: 35 minutes on a single CPU

With 5 threads: 29 minutes, CPUs about 30% saturated, evenly

With 20 threads: 40 minutes, CPUs all saturated

"Uhh. Okay, let me just run four separate processes and slice up the data set four ways."

With 5 threads per process: 8 minutes, CPU usage evenly saturated

With 1 thread per process: 8 minutes, CPU usage evenly saturated

So um. Does this mean the GIL is really that much of a bastard and I should just have shot myself rather than try multithreading in Python in the first place?

Data Graham fucked around with this message at 13:26 on Aug 1, 2017

# ? Aug 1, 2017 13:21

OnceIWasAnOstrich: Jul 22, 2006

Data Graham posted:

So um. Does this mean the GIL is really that much of a bastard and I should just have shot myself rather than try multithreading in Python in the first place?

Yes

Longer answer: Yes, if you want to do something like multithreading and it uses any significant amount of CPU time in the interpreter itself, you simply cannot do it without more processes. If you are doing something that involves a lot of IO, theoretically you can use threads but it is still a huge pain in the rear end to actually work with the GIL and you may be better off using greenlets or eventlets or something where someone did the hard work of making sure the IO could actually run concurrently and the patterns can sometimes be easier to make sure you don't dump something blocking into the coroutine. If you are rate-limited by your parsing and not the IO itself none of those will help though.

Maybe some of the Python3 async stuff might be simpler for this purpose but I haven't gotten a chance to mess with that enough.

OnceIWasAnOstrich fucked around with this message at 13:37 on Aug 1, 2017

# ? Aug 1, 2017 13:28

Data Graham: Dec 28, 2009; 📈📊🍪😋

Cool. You may all point and laugh as you please

# ? Aug 1, 2017 13:29

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Depending upon what your crawler is actually doing, you should probably use asyncio or greenlet instead of threads/multiprocessing anyway. I/O bound tasks work great with async.

# ? Aug 1, 2017 15:40

Nippashish: Nov 2, 2005; Let me see you dance!

Thermopyle posted:

Depending upon what your crawler is actually doing, you should probably use asyncio or greenlet instead of threads/multiprocessing anyway. I/O bound tasks work great with async.

Is it possible to use async without making the code async "all the way down"? Maybe I'm just dense, but afaict anything that blocks in a non-async way (read: all legacy code ever) ends up blocking the world?

# ? Aug 1, 2017 18:10

Thern: Aug 12, 2006; Say Hello To My Little Friend

Data Graham posted:

So um. Does this mean the GIL is really that much of a bastard and I should just have shot myself rather than try multithreading in Python in the first place?

Yes because the GIL prevents any process from using more than a single core at a time. You basically have to split the work through multiple processes to get it to run concurrently.

# ? Aug 1, 2017 18:40

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Nippashish posted:

Is it possible to use async without making the code async "all the way down"? Maybe I'm just dense, but afaict anything that blocks in a non-async way (read: all legacy code ever) ends up blocking the world?

In general, there's two ways that async code gets blocked. 1. You use non-async-aware IO code or 2. you become CPU bound. In either case you're all blocked up.

If your code is substantially CPU bound, async is the wrong tool. On the other hand, if you're dealing with non-async-aware code you have several options.

The absolute easiest is to use eventlet or greenlet which will patch the standard library at runtime to make all the I/O functions (like the socket library and thus http, urllib, requests, etc) use yielding versions. This will make your legacy python code automatically async-friendly.

Another option is that if the right subset of your code is CPU bound or is not monkey-patch-able to make it async friendly is to spin up another process or thread to run that code in.

Either way, if your code is substantially bound up with IO operations async can make it faster than threads or processes.

# ? Aug 1, 2017 20:40

Nippashish: Nov 2, 2005; Let me see you dance!

Thermopyle posted:

The absolute easiest is to use eventlet or greenlet which will patch the standard library at runtime to make all the I/O functions (like the socket library and thus http, urllib, requests, etc) use yielding versions. This will make your legacy python code automatically async-friendly.

This is the thing I was missing. I understand what async is for, but last time I tried it I gave up because I convinced myself I'd need to rewrite all my existing non-async-aware code to see benefits.

# ? Aug 1, 2017 20:51

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

You have to be careful though. It seems like magic that you can make your code async-ready automatically, but its easy to have CPU-bound code hidden amongst your legacy code and then after using the monkeypatching abilities of greenlet you think "this poo poo don't work gently caress you".

# ? Aug 1, 2017 20:57

Newf: Feb 14, 2006; I appreciate hacky sack on a much deeper level than you.

Thermopyle posted:

Electron will work with python 3. Electron doesn't care what language you use on the "backend".

Do you know (or care to share) any os projects with a python / electron setup? I've only glanced at electron, and then only with JS/TS projects. Haven't seen used as a UI layer with other languages.

# ? Aug 1, 2017 22:54

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Newf posted:

Do you know (or care to share) any os projects with a python / electron setup? I've only glanced at electron, and then only with JS/TS projects. Haven't seen used as a UI layer with other languages.

https://github.com/fyears/electron-python-example

You have to have some boilerplate to spin up electron and get python to initially communicate, but after that, it's super easy to just use Python/Flask to design your entire app. It's also pretty easy to bundle this (though again requires some more boilerplate in the node layer).

# ? Aug 1, 2017 23:55

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Newf posted:

Do you know (or care to share) any os projects with a python / electron setup? I've only glanced at electron, and then only with JS/TS projects. Haven't seen used as a UI layer with other languages.

Theres a few around. I've done a fairly large project used internally by a company in their line of business so I can't share that. The one that comes to mind is called Rodeo. Google around for that.

I haven't used it, but an alternative to Electron is something called cefpython...someone might want to look in to that.

# ? Aug 2, 2017 00:17

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 13:45

SirPablo: May 1, 2004; Pillbug

I have several decades of weather data, daily highs and lows, loaded into a pandas Dataframe. Is there a simple way for me to resample based on the day of the year? Meaning I would end up with a Dataframe with 366 rows. Using df.groupby([df.index.month,df.index.day]) results in a weird Dataframe with an odd index.

I would think I could do something like df.resample('dayofyear') but I can never find that option.

# ? Aug 2, 2017 05:37

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »