Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Yup, as Bonus pointed out, 2.6 is live: http://www.python.org/download/releases/2.6/

I posted a bit of info about the changes, and the MP package too: http://jessenoller.com/2008/10/02/python-26-is-released/

# ¿ Oct 2, 2008 14:35

Adbot: ADBOT LOVES YOU

# ¿ May 11, 2024 06:35

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

bitprophet posted:

Congrats m0nk3yz (and the rest of the core team obviously)!

I wish I was more excited, though; I'm not even making use of everything new in 2.5, and it feels like only yesterday that 2.5 even became widely available in package managers, versus 2.4. So it's not like I'll realistically be able to make use of 2.6 in a "run on something that's not my personal workstation" capacity for goodness knows how long

Still! The road to 3.0...

I know, that's what sucks. One of the biggest reasons we dropped 3.0 on the floor was to get 2.6 done soon enough to be able to be included in things like Snow Leopard, and the rest of the OS ecosystem.

# ¿ Oct 2, 2008 15:53

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tef posted:

I added m0nk3ys blog to sn.printf.net, on the asumption he doesn't mind being associated with goons

It's all good

# ¿ Oct 3, 2008 13:38

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

outlier posted:

OK, I surrender. What's a "context manager"?

Well that's a loaded question

- it's funny, I wrote an article on context managers for Python Magazine, it was in the July issue. In a nutshell, Context Managers came about with PEP 343 (http://www.python.org/dev/peps/pep-0343/)- PEP 343 defines a new keyword "with", which allows you to refactor the normal:

code:

try:
    ...
finally:
    ...

Code blocks into something like this:

code:

with EXPRESSION [as VARIABLE]: 
    BLOCK OF CODE

The pep defined two new dunder methods - __enter__ and __exit__ for objects, this means what when you enter the BLOCK OF CODE as shown above, the __enter__ method is called, and when that block completes, the __exit__ is called. Additionally, __enter__ methods can return some variable which is where the "as VARIABLE" comes into play.

Here's one of my favorite examples:

code:

from Queue import Queue

class ThreadPool(object):
    def __init__(self, workers, workerClass):
        self.myq = Queue()
        self.workers = workers
        self.workerClass = workerClass
        self.pool = []

    def __enter__(self):
        # On entering, start all the workers, who will block trying to
        # get work off the queue
        for i in range(self.workers):
            self.pool.append(self.workerClass(self.myq))
        for i in self.pool:
            i.start()
        return self.myq

    def __exit__(self, type, value, traceback):
        # Now, shut down the pool once all work is done
        for i in self.pool:
            self.myq.put('STOP')
        for i in self.pool:
            i.join()

This is a simple context manager compatible class which can be used like this:

code:

from __future__ import with_statement
...

with ThreadPool(10, myThreadClass) as theQueue:
    for i in range(100):
        theQueue.put(i)

In this example, I'm constructing a new ThreadPool, and passing it in the number of threads to use, and the class (which would subclass threading.Thread) to call .run() on. The __enter__ method on the class will return a Queue.Queue object to the inner block, named theQueue which allows me to push things into the ThreadPool's internal queue.

When I am done putting things to be processed in the queue, and the inner function exits - the __exit__ method is called, and the queue is shut down gracefully.

In order to use context managers - you don't need to create your own classes, you can instead use contextlib to decorate functions which can yield control internally to achieve the same effect.

For more on contextlib, see: http://blog.doughellmann.com/2008/05/pymotw-contextlib.html

Hope that helps.

# ¿ Oct 3, 2008 13:57

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

I've been writing stuff in python pretty much every day for a year and it's only now coming to my attention how little of the language I'm actually using day-to-day. Context management seems exciting and I'm looking forward to getting better.

I think this was floated earlier in reference to podcasts, but what are the python RSS I should consider staying with. I think I'm interested in items similar to "here's a feature you might be under-utilizing, and here's why it's useful."

I've been writing python pretty much full time for about 6-7 years now, and I still don't do it as well, or take advantage of, some of the cooler features/language bits that I see others use. That's one of the big reason why I tend to read through the implementation of core modules in python SVN - they teach you a lot.

# ¿ Oct 6, 2008 14:25

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

My head hurts.

What the hell were apple thinking even CONTEMPLATING distributing python without loving database bindings. Without the db bindings, almost the entire spectrum of useful things to do with scripting languages is left a flibbering mess.

Lazy Apple. Lazy loving lazy.

Anyone know a good way of getting mysqldb of some variety working on the mac. This is making me too angry working it out for myself

Apple doesn't distribute mysql - why would they give you bindings, especially given those bindings compile against a particular version of MySQL that's local to the machine?

Note, I do have instructions, I just need to find them - I'll post them when I have them

Edit: Here you go: http://yousefourabi.com/apple/django-on-leopard

Note, that when he says edit _mysql.c - you need to change 2 parts, which he omits. Your diff should look like this:

code:

$ diff _mysql.c ../_mysql.c 
37,39d36
< #ifndef uint
< #define uint unsigned int
< #endif
484,485c481,482
< 	uint port = MYSQL_PORT;
< 	uint client_flag = 0;
---
>     unsigned int port = MYSQL_PORT;
>     unsigned int client_flag = 0;

Let me know if you need more - I can compile, import it and do all sorts of cool stuff with it. You need to install the 32 bit version of mysql though.

m0nk3yz fucked around with this message at 22:15 on Oct 7, 2008

# ¿ Oct 7, 2008 22:01

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

I'd argue however that Mysql is fundamental enough that apple ought be providing its own package of that as well, or at least providing bindings against the mysql downloadable from the mysql site.

Because could you imagine distributing a python app for , say cocoa, that had to do mysql or postgres work?

At least with windows, you have mysql.dll or whatever it is, and it all seems to magically kinda work.

Did the instructions I provided help you?

# ¿ Oct 8, 2008 17:41

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Also, do the following - open your ~/.bash_profile or ~/.bashrc in your favorite editor - this file controls your bash environment. For me, my ~/.bashrc file contains:

code:

. ~/.bash_profile

Because I can never remember which one OS/X uses.

Add the following line to your .bash_profile:

code:

export PATH=$PATH:/Library/Frameworks/Python.framework/Versions/2.5/bin/

And save the file. When you open a new terminal window or type ". ~/.bash_profile" - that new bin directory will be added to your PATH, and all you will need to do is type "epydoc" on the command line.

Epydoc is awesome - I just cut work over to using it for our python tests and code. I just can't get the UML charts to render for some odd reason.

# ¿ Oct 11, 2008 13:22

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

FWIW, I chucked another side project up on pypi today:

http://pypi.python.org/pypi/pyjavaproperties/

Here's the others:
http://pypi.python.org/pypi/nose-testconfig/
http://code.google.com/p/testbutler/

I've got two more pending nose plugin projects as well as a multiprocessing/MPI package coming as well, although the last one might turn into an actor-model thing, right now I am trying to pull dramatis (http://dramatis.mischance.net/) apart.

# ¿ Oct 12, 2008 02:54

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

woo testbutler looks nice.

It's still in the prototype stages, although we have it deployed for actual production use at work. I have a laundry list of things I want to add to it. Right now I am working on a plugin for nose to upload results to the system in a new testbutler.results app.

# ¿ Oct 12, 2008 15:05

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Have you tried out Eclipse+PyDev?

I just want textmate to have code completion. That's all I want.

# ¿ Oct 15, 2008 15:05

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Lurchington posted:

No, and I can honestly say I was scared off because I have irrational fear of anything related to Java. Emphasis on irrational.

I'd give it a try: It helps out quite a bit and is actually pretty nice - my biggest beef with it is that it is "AN IDE", and I'm more of a minimalist. That, and I can't figure out how to change the color that pydev uses when it highlights all occurrences of a symbol in the current file.

# ¿ Oct 15, 2008 16:25

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

GuyGizmo - here's something I used a long time ago - I haven't looked at this recipe since I uploaded it, and I know (knowing as much as I know now) it could be improved. Maybe it helps.

http://code.activestate.com/recipes/436873/

drat, that was 3 years and 3 months ago.

# ¿ Oct 15, 2008 21:59

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

outlier posted:

I wouldn't get too hung up on OOP - you can program procedurally in Python until you get the idea of objects. And then you will wonder how you ever did without them. But if you think of objects as just smart structs, records with functions attached, that will get you a long way.

A good place to start learning is Dive Into Python and/or How to think like a Computer Scientist. Both are free on the web. I'd suggest doing a few different tutorials, to get different points of view on things.

For a long time, I did procedural programming in Python - then one day, the sky opened up and I just "got oop" - just start with what you know and don't sweat getting everything down. You'll self-evolve and over time, your code will improve. Just always challenge yourself.

# ¿ Oct 24, 2008 13:41

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tef posted:

OOP is fun and games but functional style programming in python is where the real fun is at.

Generators and list comprehensions are both awesome and powerful.

But I use OOP and Generators/List Comps together. It's like a functionallyOOP bastard child!

# ¿ Oct 24, 2008 17:52

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

I used pyprocessing(aka multiprocessing) this weekend and it was really great to work with, I managed to offload all the simulations(aka computationally expensive work) to another process in hardly any work, just set up a pipe to communicate and set the process off, you can see the what change I made to made for this to happen here: http://github.com/alex/election-sim/commit/cc93df649d1295deab96f1f1ab1ff709e8b0f391

Yay! I'm glad it worked for you

# ¿ Oct 27, 2008 15:38

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Mr. Noller(forgot your SA name), your proposals for pycon sound very cool, I'd attend both of them.(As long as they don't interfere with mine of coure :P )

I proposed one on MultiProcessing, and one on Concurrency/Distributed systems. Dunno if they'll get approved.

Oh, and someone else proposed a HPC talk too, but it's tagged with IronPython too

m0nk3yz fucked around with this message at 18:49 on Oct 31, 2008

# ¿ Oct 31, 2008 18:36

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Oh no guys, I've picked up "Programming Erlang" - I'm going to the dark side :jihad:

Of course, this is all part of my clever plot to introduce Actors/Message passing pieces to python.

# ¿ Nov 2, 2008 21:41

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Guido said that he though Actors were one of the "right ways" to do concurency. It's somewhere in

http://moderator.appspot.com/#e%253Dagltb2RlcmF0b3JyDQsSBlNlcmllcxjJAQw%252Bt%253Dagltb2RlcmF0b3JyDAsSBVRvcGljGP8BDA%252Bv%253D15

Yup. He said it in response to the multi core question:

quote:

I think the panic is greatly exaggerated. We are *not* all going to have to write multi-threaded programs all the time. It would slow productivity to a grinding halt anyway, due to the extreme difficulty of writing correct concurrent code using the current crop of mechanisms. As we get more cores, the cost of accessing memory that's shared between multiple threads is only going to increase relative to the cost of accessing unshared (cache) memory, and message passing will become a much more appropriate paradigm than shared memory and critical sections.

Python 2.6 and 3.0 are including a new package, multiprocessing, which makes creation of and communication between multiple processes as simple as managing threads was in Python 2.5, and my expectation is that something not entirely unlike the Actor paradigm, but implemented on top of the multiprocessing package, will eventually become the recommended best practice for reaping the benefits of multiple cores. Each process will be self-contained and only communicate with others at specific times when it sends or receives a message.

And you can also see my intentions in this post to python-dev:
http://mail.python.org/pipermail/python-dev/2008-October/083244.html

So, while Erlang's syntax makes me about as excited as punching myself in the face, I'd rather know more about the Actor system as implemented in it.

Just to add: One of the big drawbacks to the multiprocessing package is IPC - the serialization/de-serialization costs can harm you if you're passing crap-tons of objects, not to mention the basic requirement of pickle-ability. Erlang's actors are not independent processes.

m0nk3yz fucked around with this message at 22:53 on Nov 2, 2008

# ¿ Nov 2, 2008 22:43

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

ATLbeer posted:

code:

1> X = 6.
6
2> X = 7.
** exception error: no match of right hand side value 7

Erlang is another planet

Yup, see above: I really don't like the syntax. I'm mainly interested in the concurrency primitives/etc. I've never been a fan of functional programming.

Also:

duck monster posted:

shave that beard son

I can't even grow one much less shave one. Although I have been considering suspenders.

# ¿ Nov 2, 2008 22:48

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tef posted:

use expect + ssh.

alternatively, use the subprocess module to launch ssh, and manipulate it

And if you really want to do it "right" - use Paramiko. But that's overkill if all you want is something silly. See also http://www.theether.org/pssh/

# ¿ Nov 5, 2008 20:02

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Need some help: I want to pick out a few "easy to grok" app examples to implement with single-threads, multi threads and then multiprocessing. I want the examples to be as approachable as possible. Ergo, no map-reduces or hadooping

# ¿ Nov 7, 2008 17:13

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Bouquet posted:

I have been working on a program that does almost exactly this as part of learning python and 3D graphics stuff. I'm using pyglet, which might be a dependency m0nk3yz would rather avoid for his examples, but I'd be happy to share if you'd like.

All in all I was thinking common tools/toys people work with - I want to showcase the simple case

# ¿ Nov 7, 2008 21:49

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Slightly on topic: anyone else going to PyWorks this week in atlanta?

# ¿ Nov 12, 2008 16:13

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

bitprophet posted:

I'm not! Hope it's a good con though.

Speaking of cons, I'm starting to think I may not be able to make it to PyCon next year either. Was hoping to attend, especially since I've got some pressure to help push the book, but I really doubt I can spare the money for the travel/board/fees (my wife is chronically unemployed due to the poor animation market, so that makes our financial situation not the greatest).

Almost makes me miss my last job; they paid for yearly training, including conferences and such.

When pycon gets closer, you could apply for PSF aid to go - I know they have a fund for just this.

# ¿ Nov 12, 2008 18:25

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Centipeed posted:

I'm thinking, since I'll start coding for my university project after Christmas, that just jumping straight into Python 3.0 is a better idea than 2.6. I've already learned some Python, but it's not ingrained in my head so much that 3.0 will be confusing, and I'd rather learn the new standard than the "old" one, especially since they're releasing it next month.

Is this a good idea? Also, is the Python.org tutorial going to be the only learning resource for 3.0 until other people start writing them? I'm not particularly fond of how the Python.org tutorial starts out, since I'm already familiar with the basics of programming, having done Java and some C++. It seems like a tutorial designed for newcomers to programming, since it starts with using Python as a calculator and whatnot.

No. Do not jump into python 3000 - Python 3000 is not going to be widely adopted for some time - you're much better off focusing on python 2.6 and using that to build up your skills. Python 3000 is not "that big" of a change from 2.6 that it will make all of your 2.6 knowledge and skills worthless.

# ¿ Nov 23, 2008 15:32

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

I'm super excited to say that my PyCon talk was accepted! It's a panel on ORM philosophies and design decisions with Jacob Kaplan-Moss, Ian Bicking, Mike Bayer, Guido van Rossum, and Massimo Di Pierro. I know we had a few other goons who submitted talk proposals, what were your results?

Both of mine - an intro to multiprocessing, and the "state of concurrency and distributed systems w/ python" were accepted.

# ¿ Dec 16, 2008 04:38

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Kire posted:

That big ben song is hilarious.

What on earth are you talking about

# ¿ Dec 22, 2008 18:57

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Jo posted:

At what point does SQLite not scale well? I have a webspider that throws off some 40,000+ (small) entries per page. I'd like to use SQLite, but if it chokes on that many entries...

It wouldn't be too hard to write a quick little test to see if it scales the way you want it too...

# ¿ Jan 2, 2009 23:37

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tayl0r posted:

Well of course after I posted to this thread I found what I was looking for. http://docs.python.org/library/markup.html

Edit: Hmm.. still trying to find something that *writes* XML

Edit 2: ahh ha! http://docs.python.org/library/xml.sax.utils.html

elementtree does writing as well

# ¿ Jan 3, 2009 18:37

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

tripwire posted:

Has anyone used the multiprocessing module before? I'm trying to use a pool to split up the work in my mona lisa triangle mover thing but it doesn't look like you can call it from an instance of a class and point to an instance method (it raises a pickle error, 'can't pickle instancemethod' or something like that. Do I have to make it use a global function? Is there some idiom or pattern for doing this?

Yes, it has to be a pickle-able function/class instance/etc. Make it a function, and you're golden. choo choo! I'm not good with the GUI stuff, so I will help where I can.

Also, I didn't write it, I just got it into python.

# ¿ Jan 13, 2009 03:25

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

thou shalt not prematurely optimize: http://bugs.python.org/issue5000

# ¿ Jan 19, 2009 17:59

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Heh, I just had some fun optimizing something myself: http://lazypython.blogspot.com/2009/01/optimizing-view.html

Heh, I closed like, 4 or 5 multiprocessing bugs this weekend - then I searched and found like 7 more unassigned :\

# ¿ Jan 19, 2009 23:42

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

bitprophet posted:

2.6. 3.0 is extremely new and almost nothing has been ported to it yet. 2.6 will include some of the newer libraries/ideas present in 3.0, so you'll be better off for learning 3.0 than someone who's used to older verisons of Python. 3.0 will not be mainstream for at least a year or two (someone like m0nk3yz or king_kilr will have a better estimate, probably)..

If you want cleanliness and purity; go with 3.0 (note that IO speeds suck) - if you want third party code, libraries and frameworks, go with 2.6.

Ultimately, to have strong python-fu, you'll need to know both.

# ¿ Jan 29, 2009 04:38

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

deimos posted:

I am gonna ask the boss for PyCon, and I am pretty sure I am going to go regardless of job paying for it, at the very least for the conference.

That being said, this is discouraging.

m0nk3yz do you think it'll be better this year?

Yes, I do - I think a lot of us, myself included, largely agreed with Bruce. I think things will be changed quite a bit, and I know that the pycon organizers are watching out for this pitfall this year.

Also, I have two talks. You should go. I did a post this morning which may be the first of many

# ¿ Jan 29, 2009 17:28

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Kire posted:

Is there an easy way to randomize the retrieval of key, value pairs from a dict?

I have a paired list of {English : German} words, and I want to randomly pull an English word and its German translation. I know about d.keys(), but I'm not sure how to randomly select one from the result.

why not choice = random.choice(d.keys())?

code:

>>> import random
>>> x = {1:2,3:4}
>>> random.choice(x.keys())
1

# ¿ Jan 30, 2009 20:45

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

I know a couple of developers who have decided to give 09 a miss due to the poo poo that went on with the last pycon. Spending $5K+ to fly from australia to go to a conference on the other side of the earth only to have it wrecked by blatant spamming by vendors doesn't endear folks to the organisation of it all.

And they're shortsighted if they don't think the organizers didn't listen to the feedback or attempt to change things. Everyone involved in 08 admitted to the mistakes, and steps were taken to rectify it.

# ¿ Jan 31, 2009 18:49

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Anyone looking for a giant read, I just reprinted an old article of mine on Python threads the and global interpreter lock. I'm going to be publishing all of my stuff through 2008 soonish. :frogsiren:

# ¿ Feb 1, 2009 23:23

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

deimos posted:

I realize now that single article is like 6500 words. I should make a cliff notes version.

# ¿ Feb 2, 2009 14:12

Adbot: ADBOT LOVES YOU

# ¿ May 11, 2024 06:35

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

I know there was some discussion about contextmanagers earlier in the thread, as an FYI I just put the article from pymag up which gives a lot more information. It's only 2000 words or so, so I shouldn't get yelled at for it being too long. Here.

# ¿ Feb 3, 2009 15:38

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »