Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

If the NumPy folks would get off their butts and release a Python 3 compatible version (even just an alpha), people might start caring more about it. I really want to release a Python 3 version of my package, but I can't because it needs NumPy.

Last I heard; the NumPy folks had recently announcing the py3 port would spin up soon - this was at pycon, and I don't know who the official source was, so ymmv.

# ¿ Mar 8, 2010 04:13

Adbot: ADBOT LOVES YOU

# ¿ May 11, 2024 06:13

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

MEAT TREAT posted:

These two statements are mutually exclusive to me and I imagine a large part of the python community.

Preemptively calling myself a big baby who can't write web applications without a framework.

I'm right there with you; I can't move to py3k until Django and a few other things I need move over. Until then, only small pet projects for py3k. Which sucks.

# ¿ Mar 9, 2010 15:01

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tripwire posted:

Which makes it a self-reinforcing problem. I know this is big talk for someone who's only written relatively tiny programs in python2.6 and not ported any of them over to 3.0, BUT, I don't think "not enough people use python3" is a valid excuse not to port over your old code- if everyone thought that way, we'd never advance in versions at all.

It will all come - it just takes time. Once some big dominoes fall - such as numpy - we will begin to see things shift. Until then, we're all consigned to the darkness which is the old print statement.

# ¿ Mar 10, 2010 04:18

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Depending on the nature of actions I think an event based application (using eventlet, or twisted or something) would be much preferable to threads, that only really works if your actions tend to be IO bound (network connections, writing to files, etc.).

I'd tend to agree with king_kilr; not that you couldn't do this with threads (you can) but event-based systems, such as gevent/eventlet/greenlet might be a better fit in the long haul. You can skip twisted and stackless though.

# ¿ Mar 13, 2010 03:50

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

# ¿ Apr 22, 2010 19:17

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Parker Lewis posted:

I figured Google was taking care of it.

Touche.

# ¿ Apr 22, 2010 20:46

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

Answer: because the only stuff I care about in Python core is stuff that would be relevant to Python 3 and I can't use Python 3 because NumPy isn't on Python 3.

What about the stdlib?

m0nk3yz fucked around with this message at 20:52 on Apr 22, 2010

# ¿ Apr 22, 2010 20:47

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Thanks for the feedback everyone; no real surprises, but it's good to have data from actual users versus what my gut said.

# ¿ Apr 23, 2010 15:23

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

Ok practical question: what's the most efficient way for me to go about getting something like pushd added to the stdlib?

More details

Actually, first - scope. If it's an addition to an existing stdlib module; then identifying the lead of that module is possible (and propose it to them). If there is no lead, and it's small, file a bug, with tests and docs to the tracker. +nosy me on it (so I can watch/advocate). If no one responds quickly, bring it up to python-dev (I'd give it 1 week).

If it's larger (all new module) you should write a PEP (I can help) and send it to stdlib-sig; where I can also help out, once the discussion is "done" (should be quick) we can send it to python-dev for acceptance. Then it's code, test, docs and done.

# ¿ Apr 24, 2010 05:34

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

Where is that information? (The module would be os.) pushd would basically be used with "with" statements and on entry you'd cd into some dir and on exit you'd cd back to the old one. I always find it really annoying that I have to do that manually, especially when there might be exceptions thrown.

http://svn.python.org/view/python/branches/py3k/Misc/maintainers.rst?revision=80328&view=markup

Yeah, I know - intuitive, right?

In any case, for this - file a bug with a patch+docs/tests. It's an enhancement, my gut says it makes sense in os.path.

# ¿ Apr 25, 2010 17:05

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

zelah posted:

Not sure if this is worth putting in the OP but I'm brand new to coding and this is helping a lot

Added.

# ¿ Apr 26, 2010 01:50

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Stabby McDamage posted:

Wow, I just lost a huge amount of respect for Python. Nothing you can do will ever allow parallelism with threads...that's nuts. Worse, all that work in that slide deck is just focused on making the overhead of threads closer to single-core performance! Nothing about actually taking advantage of multicore to achieve any kind of speedup!

Bullshit. Threads in python still work fine, despite David's tests for most (not all) I/O bound workloads. I use the crap out of them for parallelism all the time. Yes, they're "fundamentally" broken due to GIL contention, but most I/O bound apps using threads will see performance increases. I might be the maintainer for multiprocessing, but I still use threads in most of my code.

If you notice a problem in your heavily threaded app, go async/coroutine (eventlet, gevent, etc) or use multiprocessing (go me, "whoo"), or parallel python, or a bare fork() which is what multiprocessing uses.

# ¿ May 12, 2010 02:34

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

Here you go. I did it in a single statement ~~if you don't count the imports~~ (EDIT: who needs import when you have __import__) (EDITx2: wait I can make multiline lambdas a lot simpler):
code:
print("Heads came up {0} times.\nTails came up {1} times.".format(*__import__('functools').reduce(
lambda T, x: (print(['Heads!', 'Tails!'][x]), [T[0] + int(x==0), T[1] + int(x==1)])[-1],
(__import__('random').randint(0, 1) for _ in range(10)), [0, 0])))
Indented: http://pastebin.com/cfBXNSsZ

Wow, just wow. I need to find some way of using this in real code, just to be a dick.

# ¿ May 20, 2010 00:54

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

...stuff...

And Ctypes is perfectly good for the rest of us :smug:

# ¿ May 22, 2010 01:10

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Mido posted:

Django!!

Django.

# ¿ Jun 18, 2010 14:43

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

I'm sorry, was that a list of complaints, or a list of the best features of Django?

Seconding this. I do not like the routing stuff, and I despise templates (meaning, code in HTML) that evaluates real, unrestricted python. Everything you listed, modulo the ORM (which I am completely ambivalent about), seems like a feature.

# ¿ Jun 19, 2010 02:42

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Sneftel posted:

Yes. The manner in which multiprocessing funnels all output to the same place is OS-dependent; the short answer is, if you're going to do output from multiple processes, grab a lock first.

Or use the built in logging module support

# ¿ Aug 3, 2010 20:49

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

Thanks both of you*. This will be good for actual implementations I end up doing. I was mostly surprised that the lack of a "gotcha" on PyMOTW and the lack of a specific answer on google searches.

*m0nk3yz: I finally got around to seeing your PyCon 09 intro to multiprocessing talk, and I thought it was great

Thanks! I should do a follow up talk next pycon - sadly though, my own usage of multiprocessing in the past year is really low, as I'm working on single-core, lower memory boxes. I do use a lot of threads though.

# ¿ Aug 6, 2010 02:56

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

KuruMonkey posted:

I'm learning python (day 4!) and mostly its going well. I'm coming from PHP, and my transition project is to recreate my gallery web site I run on my HTPC in Python. (cherrypy)

Looking like about 3 days to re-implement something I originally wrote in an evening

Anyway:
Any thoughts on which image manipulation library is the one to go for if you, say, wanted to get to grips with just one rather than spend time learning 3 or 4 and then picking? Is there a best library? Any dogs to definitely avoid?

PIL

# ¿ Aug 10, 2010 16:55

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

Here's my interview whiteboard question from the yospos thread: link

questions that stood out, not exhaustive or anything:
...snip...

Nice! And I'm oddly flattered you cited my talks

Have you mentioned where you were interviewing - those are pretty good questions, although if it was a web shop, I'd have not asked the GIL/process questions. Also, I would have skipped the list implementation question, but that's personal taste.

m0nk3yz fucked around with this message at 03:55 on Aug 19, 2010

# ¿ Aug 19, 2010 03:35

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

PM sent in the actual company, but my from what I was told (interview conducted at a high level) was that it wasn't exclusively a web shop, and there whatever they were doing certainly leveraged threading/multiprocessing (and for that I'll thank your talk).

I talked a bit about the list implementation question on my previous post, but thinking about it more, maybe the job has a lot of writing in C extensions? And an understanding of the C implementation of a list would seem like the barrier to entry for going to that level.

If they're doing a lot of concurrency, yeah - I'd put money on them using a lot of C/C extensions as well.

# ¿ Aug 19, 2010 17:01

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

nbv4 posted:

yup thats probably the biggest drawback to using non-cpython implementations. App engine uses pypy, so it only works on 2.5.2, which really sucks.

Huh? I don't think appengine runs pypy, I think google's internal python version is just old as sin.

# ¿ Aug 25, 2010 20:54

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

I'm lucky in that I can use any Python version I can compile and make the dependencies run on for the most part - but I go along with the OS vendor's version because I'm lazy, and I don't need everything in 2.7 right now. I'll be really happy when my biggest dependency (Django) has a Python 3 version that's blessed

# ¿ Aug 27, 2010 03:47

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

defmacro posted:

It's CPU bound, ~78% of the time is spent in user. I'm extracting features from (lots of) pcap files to perform clustering on after. This seems to be a pretty reasonable candidate for threadpool-like worker concurrency but I keep hearing poo poo about the GIL in python. How should I got about doing this? Would I want to fork off worker threads (so they have a distinct GIL)?

Check out multiprocessing. It's built in, might work for you.

# ¿ Sep 14, 2010 19:20

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tripwire posted:

Is there any reason why python 3 didn't pick 1 and 3 in that list? (Did they, and I'm just not aware?)

We picked 1 - the initial patch(es) for free threading seriously harmed multithreaded performance; and Python has historically not been used in heavily multithreaded environments, the choice was made. After that point, no one picked the work back up, and no one volunteered to take it on for python 3.0.

I do think it will eventually be fixed; but it takes someone funding the work.

# ¿ Sep 18, 2010 01:17

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

If you don't care about backwards compatibility it's not such a hard problem (technically speaking), it's still a lot of work, but if you're a company who needs it I think it could be done in under 6 months of developer time. I suppose you could even compile time flag it up and make it backwards compatible, but that seems like a bad idea IMO.

Yup, when I make my millions, I'm going to pay someone to JFFI (Just loving Fix It)

# ¿ Sep 18, 2010 02:04

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

m0nk3yz is probably too humble to say so, but in this year's PyCon call for proposals it lists him as co-chair. Congrats and good luck.

More like too tired and spinning in too many directions

But yeah, what he said!

# ¿ Sep 30, 2010 02:57

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

If someone writes me a new / updated OP, I'll post it - sorry for my absence, I've been busy.

# ¿ Aug 19, 2011 18:36

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

OP updated.

# ¿ Aug 27, 2011 00:13

Adbot: ADBOT LOVES YOU

# ¿ May 11, 2024 06:13

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

fixed

# ¿ Aug 27, 2011 04:26

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »