Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Jose Cuervo: Aug 25, 2004

I have a small discrete event simulation which I wrote to solve this problem. I was wondering if anyone could give me some feedback on the way I wrote it, especially if there are bad practices in there regarding how to structure code, things you would do differently and why, etc.

The code can be found here.

Thanks.

# ? Feb 12, 2014 03:39

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 17:47

My Rhythmic Crotch: Jan 13, 2011

If you're writing a class, what is the preferred way to handle exceptions that may be thrown by other classes that you depend on?

For example:

Python code:

result = doohickey.complex_operation(42)

Inside doohickey, do I catch and then re-throw exceptions coming from other classes beneath me, or do I let the exceptions go right through? Or are there alternative ways to do it?

I'm leaning towards catching and re-throwing, because any application that utilizes doohickey does not need to import other stuff in order to catch exceptions from doohickey.

Edit: loving gevent is breaking our stack. Monkey patching breaks a bunch of stuff we want to use *sigh*

My Rhythmic Crotch fucked around with this message at 04:55 on Feb 12, 2014

# ? Feb 12, 2014 04:46

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

I don't normally catch and re-throw exceptions unless they happen to have really poorly written or really confusing (given the context) error messages. Explicit is better than implicit and so on. Also if there's an exception from one of my dependencies then it probably means I'm doing something wrong, not that my user is doing something wrong.

# ? Feb 12, 2014 05:35

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

My Rhythmic Crotch posted:

If you're writing a class, what is the preferred way to handle exceptions that may be thrown by other classes that you depend on?

For example:
Python code:
result = doohickey.complex_operation(42)
Inside doohickey, do I catch and then re-throw exceptions coming from other classes beneath me, or do I let the exceptions go right through? Or are there alternative ways to do it?

I'm leaning towards catching and re-throwing, because any application that utilizes doohickey does not need to import other stuff in order to catch exceptions from doohickey.

Edit: loving gevent is breaking our stack. Monkey patching breaks a bunch of stuff we want to use *sigh*

Context is important. If it's an implementation detail tied to a library you're using (say, a requests.exceptions.Timeout from requests) and it's something you expect a caller to deal with in normal code, it's better to re-throw as something that's not coupled to a library you happen to use. On the other hand, if it's basically an unchecked exception -- something you don't expect to hit besides in truly exceptional and unforeseen circumstances -- best to let it propagate up so the caller or the program's user can get a better handle on what's actually going wrong and why.

# ? Feb 12, 2014 05:44

QuarkJets: Sep 8, 2008

Is there a way to re-throw an exception in a way that references the original exception-raising line?

# ? Feb 13, 2014 00:09

Edison was a dick: Apr 3, 2010; direct current only

QuarkJets posted:

Is there a way to re-throw an exception in a way that references the original exception-raising line?

A bare raise is supposed to re-throw the exception without altering it I think.

If not, there's sys.exc_info(), which returns a tuple of (type, value, traceback), which you can re-raise directly with python's long-form raise statement

code:

try:
  raise Exception()
except Exception, e:
  etype, eval, tb = sys.exc_info()
  raise etype, eval, tb

Arguably the long form raise is of limited use if a bare raise will re-raise the current exception.

I have only once found it useful, and that was for this abomination.

# ? Feb 13, 2014 01:33

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

QuarkJets posted:

Is there a way to re-throw an exception in a way that references the original exception-raising line?

Python code:

>>> try:
...     print(a)
... except NameError as e:
...     raise Exception('something went wrong') from e
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
NameError: name 'a' is not defined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
Exception: something went wrong

# ? Feb 13, 2014 02:29

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

Lysidas posted:

Python code:

>>> try:
...     print(a)
... except NameError as e:
...     raise Exception('something went wrong') from e
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
NameError: name 'a' is not defined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
Exception: something went wrong

I did not know this syntax existed despite working with Python since ~2004, and it is an excellent thing. Thank you for posting it!

# ? Feb 13, 2014 04:22

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Misogynist posted:

I did not know this syntax existed despite working with Python since ~2004

Well that's understandable, since that syntax didn't exist in 2004. It was added in 3.0 and is not available in Python 2

# ? Feb 13, 2014 04:40

Luigi Thirty: Apr 30, 2006; Emergency confection port.

Aha, I asked in here a while ago about how to find the CPU temperature with Python and it was too difficult to wade through the bullshit of Windows to do it. Turns out there's a program called Open Hardware Monitor that fetches all the info for you and publishes it into a WMI namespace which you can easily parse with the WMI module. Now I can send it out over USB to my Arduino :toot:

Python code:

>>> import wmi
>>> hardwaremonitor = wmi.WMI(namespace="root\OpenHardwareMonitor")
>>> temps = [x.Value for x in hardwaremonitor.sensor() if "CPU Core #" in x.name and x.SensorType == "Temperature"]
>>> temps
[48.0, 39.0, 42.0, 40.0]
>>> gpu_temp = [x.Value for x in hardwaremonitor.sensor() if "GPU Core" in x.name and x.SensorType == "Temperature"]
>>> gpu_temp
[33.0]

# ? Feb 13, 2014 05:40

My Rhythmic Crotch: Jan 13, 2011

Python docs posted:

Note It is strongly advised that you do not add any handlers other than NullHandler to your library�s loggers. This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers �under the hood�, you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

This seems really dumb to me. So, basically, the official stance is to either add no handlers (so you get an error when a log message is thrown) or a NullHandler to suppress the log messages.

Is it not possible to specify a default logging config on a module level, and not have it interfere with the global logging configuration?

# ? Feb 14, 2014 00:39

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

stderr/stdout is a global resource. Libraries should never take over global resources.

# ? Feb 14, 2014 00:47

KICK BAMA KICK: Mar 2, 2009

e: nm, realized that this isn't right cause I wasn't doing MVP correctly anyway

KICK BAMA KICK fucked around with this message at 03:01 on Feb 15, 2014

# ? Feb 14, 2014 05:16

SurgicalOntologist: Jun 17, 2004

What's the preferred way to install a bash alias or system script alongside your package (presumably in setup.py)? I found myself running python -m mymodule a bunch of times so I made a bash alias. It would make things similar if setup.py could do this. Presumably there's an easy way as I think I've come across other packages doing this but I can't find the right terms to search for.

# ? Feb 14, 2014 06:09

evensevenone: May 12, 2001; Glass is a solid.

One of the args to setup() is a list of scripts that will be copied to /usr/local/bin (or somewhere similar). You can put whatever there; it doesn't have to be a python script.

# ? Feb 14, 2014 06:21

SurgicalOntologist: Jun 17, 2004

Are you referring to this? The documentation is a bit sparse here. It doesn't say that it's copied somewhere but this is probably it. Time to experiment...

E: It's much simpler. The option is called entry_points.

SurgicalOntologist fucked around with this message at 07:38 on Feb 14, 2014

# ? Feb 14, 2014 07:12

evensevenone: May 12, 2001; Glass is a solid.

The one I use is scripts, which seems to work fine. I don't know what entry_points does.

# ? Feb 14, 2014 09:10

Daynab: Aug 5, 2008

So I finally finished my first PySide application, but it occasionally likes to crash after exiting (as a Windows "python has stopped working" error), it's completely random, seems to happen more often when I open and close it quickly but it's otherwise impossible to reproduce.

How would I go about figuring out what the gently caress is going on? I've read this https://stackoverflow.com/questions/11945183/what-are-good-practices-for-avoiding-crashes-hangs-in-pyqt but I have no idea where to start. I don't use threading, but I do use a QTableView which I'm pretty sure is involved in a way.

# ? Feb 14, 2014 12:04

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

The faulthandler module is designed for exactly this purpose -- you can enable it and give a file handle as the file argument, and it'll dump a traceback if the interpreter crashes.

I've found it very useful in debugging extension modules written in C, since the traceback actually includes function calls in native code and it's far more helpful than "Segmentation fault (core dumped)" as the only output of the program.

# ? Feb 14, 2014 14:20

Dominoes: Sep 20, 2007

Daynab posted:

So I finally finished my first PySide application, but it occasionally likes to crash after exiting (as a Windows "python has stopped working" error), it's completely random, seems to happen more often when I open and close it quickly but it's otherwise impossible to reproduce.

How would I go about figuring out what the gently caress is going on? I've read this https://stackoverflow.com/questions/11945183/what-are-good-practices-for-avoiding-crashes-hangs-in-pyqt but I have no idea where to start. I don't use threading, but I do use a QTableView which I'm pretty sure is involved in a way.

I don't have a good answer for you, but you could try gradually removing bits of codes and testing for the crash. When I've had this, it's been related to threading, which you're not doing.

# ? Feb 14, 2014 15:57

Innocent Bystander: May 8, 2007; Born in the LOLbarn.

Daynab posted:

So I finally finished my first PySide application, but it occasionally likes to crash after exiting (as a Windows "python has stopped working" error), it's completely random, seems to happen more often when I open and close it quickly but it's otherwise impossible to reproduce.

How would I go about figuring out what the gently caress is going on? I've read this https://stackoverflow.com/questions/11945183/what-are-good-practices-for-avoiding-crashes-hangs-in-pyqt but I have no idea where to start. I don't use threading, but I do use a QTableView which I'm pretty sure is involved in a way.

Yeah I would definitely look into the logging module and begin using it. To solve the problem right now, you might consider wrapping your entry point of the code in a try, except.

code:

import logging

logging.basicConfig (filename='mylog.log')
try:
    start_your_app ()
except Exception, e:
    logging.exception (e)

While the global try catch is frowned upon, this will show you what is causing the problem, and you can fix it while adding more logging into your program.

# ? Feb 14, 2014 16:33

SurgicalOntologist: Jun 17, 2004

evensevenone posted:

The one I use is scripts, which seems to work fine. I don't know what entry_points does.

Seems like scripts is for installing scripts you've already written, entry_points is for automatically creating a script that points to some function in your package.

# ? Feb 14, 2014 16:37

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Innocent Bystander posted:

this will show you what is causing the problem

I disagree. Python exceptions don't cause Windows to tell you "python has stopped working" or segfault the interpreter on Unix systems. Crashing in native code precludes the raising of a Python exception, so the except clause that you propose would never be reached.

EDIT: except Exception, e: is invalid syntax too

# ? Feb 14, 2014 16:39

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Lysidas posted:

EDIT: except Exception, e: is invalid syntax too

It's merely deprecated in python 2.

# ? Feb 14, 2014 17:19

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Plorkyeran posted:

It's merely deprecated in python 2.

Exactly. Hence, it was valid and now it's invalid :v:

Unrelated:

I think I can count how many times I've been disappointed with the Python standard library on one hand. Most of those times are related to ancient modules whose interface can use some improvements, like tarfile. It's rare that I'm unhappy about something recent, like the addition of an exist_ok parameter to os.makedirs. This was added in 3.2 and seems like it would be nice to use, except that it still throws a FileExistsError (OSError in 3.2) if the directory exists and has different permissions than what makedirs would have created. There is no way to override this behavior.

There was a patch for this proposed after 3.2 was released, which attempted to add a strange on_wrong_mode parameter taking exactly one of two integer values: FAIL = 0, IGNORE = 1. I think it would make a lot more sense to add another boolean argument along the lines of ignore_mode or something like that, but this won't even make it into 3.4.

I wrote a simple makedirs_exist_ok wrapper function a long time ago, and I was looking forward to getting rid of it. Guess I'll have to keep using it for the indefinite future.

# ? Feb 14, 2014 18:39

Pie Colony: Dec 8, 2006; I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

i wrote a really stupid tail call optimizer in python this afternoon. in my defense, i had nothing else to do and also wanted to play around with the AST module a bit. i will combine this with my stupid-er currying function to finally make python the slow functional programming language it was never meant to be.

# ? Feb 14, 2014 22:47

Reformed Pissboy: Nov 6, 2003

Daynab posted:

So I finally finished my first PySide application, but it occasionally likes to crash after exiting (as a Windows "python has stopped working" error), it's completely random, seems to happen more often when I open and close it quickly but it's otherwise impossible to reproduce.

How would I go about figuring out what the gently caress is going on? I've read this https://stackoverflow.com/questions/11945183/what-are-good-practices-for-avoiding-crashes-hangs-in-pyqt but I have no idea where to start. I don't use threading, but I do use a QTableView which I'm pretty sure is involved in a way.

If it's Windows crashing, your best bet (unfortunately) is to install the Windows debugging tools (namely, WinDbg) and get a stack trace, and pray to god something jumps out at you. I had to do this for the first time a few weeks ago to track down a crash in 64-bit builds of my PyQt application, and it turned out to be a fault in the backup software my company uses (Druva inSync) injecting itself into filesystem queries and loving up the handle returned to Qt (both PyQt and PySide). That was fun.

# ? Feb 15, 2014 01:05

Nippashish: Nov 2, 2005; Let me see you dance!

Lysidas posted:

I think I can count how many times I've been disappointed with the Python standard library on one hand.

I wish they'd add a prod function to go with sum. As far as I can tell the rationale for not having one is that no one would have a use for it. I would use it :colbert:

# ? Feb 15, 2014 01:57

tef: May 30, 2004; -> some l-system crap ->

Lysidas posted:

Unrelated:

I think I can count how many times I've been disappointed with the Python standard library on one hand.

urllib2, urllib, httplib, socket, urlllib2

# ? Feb 15, 2014 02:58

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Tkinter is also a bit disappointing.

# ? Feb 15, 2014 05:46

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

concurrent.futures is dumb because it wasn't a well-tested, in-the-field library and everybody on the mailing list was talking what to name it.

# ? Feb 15, 2014 06:17

Lurchington: Jan 2, 2003; Forums Dragoon

pretty psyched that as a long time user of configobj for my configuration file needs, I was able to help get it python 3 after it had be lying untouched for the last 3 years. I'm now one of the new maintainers and we did the full 2014 python open source makeover:

source was on google docs, now on github
docs were on the original maintainers blog, now on readthedocs
I don't think there was any specific build process in place, but it's using tox and travis to guarantee single source compatibility with 2.6, 2.7, 3.2, and 3.3

next step, replace the thousands of doctests with something, anything else :gonk:

but we just release version 5.0.0. Feels good.

# ? Feb 15, 2014 07:22

Dominoes: Sep 20, 2007

Lysidas posted:

I think I can count how many times I've been disappointed with the Python standard library on one hand.

The datetime module needs timezone support, like tzinfo/dateutil.

# ? Feb 15, 2014 14:12

fritz: Jul 26, 2003

Nippashish posted:

I wish they'd add a prod function to go with sum. As far as I can tell the rationale for not having one is that no one would have a use for it. I would use it

reduce+operator.mul?

# ? Feb 15, 2014 17:26

Nippashish: Nov 2, 2005; Let me see you dance!

fritz posted:

reduce+operator.mul?

Good point, we should probably get rid of sum in that case; people can use reduce+operator.add and be grateful for it.

# ? Feb 15, 2014 17:32

tef: May 30, 2004; -> some l-system crap ->

Lurchington posted:

pretty psyched that as a long time user of configobj for my configuration file needs, I was able to help get it python 3 after it had be lying untouched for the last 3 years. I'm now one of the new maintainers and we did the full 2014 python open source makeover:

This is pretty cool and I'm glad you're stepping up to help out with stuff :toot:

# ? Feb 15, 2014 18:31

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Nippashish posted:

Good point, we should probably get rid of sum in that case; people can use reduce+operator.add and be grateful for it.

I don't particularly think a*b*c is that great of an API to be in the standard library by default. You have to fight over an API: what should prod("foo", 5) do? Is it as simple as a*b, so it would return "foofoofoofoofoo"[/fixed}? Or should it be restricted to numbers only, like [fixed]sum() is? What about numpy? Decimal and Fraction classes? SymPy? Will prod(["foo", 5]) work, or to multiply a list of numbers together, should you do prod(*my_list)? Both? If both, what does prod(["foo", 5], 5) do?

Somebody has to document it. You have to triple-check that adding a new name to the globals doesn't break any existing code. Once you decide on a set of edge cases, you have to test them. Adding another just global increases cognitive load, and discourages exploration and play: dir() and help() become longer.

There's bound to be a bug. Who will fix it? Who will make the test for every fix?

Just write a method to call a*b*c.

# ? Feb 15, 2014 19:25

Nippashish: Nov 2, 2005; Let me see you dance!

Suspicious Dish posted:

Just write a method to call a*b*c.

Do you think having sum in the standard library was a mistake?

# ? Feb 15, 2014 19:32

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

ffs just implement prod in terms of sum as math.exp(sum(math.log(value) for value in iterable))

# ? Feb 15, 2014 19:42

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 17:47

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Just write a library that spins up a hadoop cluster to do your prod operation via mapreduce, then make it part of the standard library.

# ? Feb 15, 2014 19:49

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »