|
duck monster posted:Ah. Nice. Much more elegant than the get_foo set_foo stuff. Speaking of decorators, this popped up on pypi this morning - simple threaded/threadpool decorators: http://pypi.python.org/pypi/threadec/
|
#
?
Nov 13, 2007 12:55
|
|
|
|
| # ? Apr 25, 2024 13:08 |
|
|
m0nk3yz posted:Speaking of decorators, this popped up on pypi this morning - simple threaded/threadpool decorators: I'm relatively python ignorant, I've only been learning it for a week or so ... What's the point of threading in python with the GIL, excluding heavy I/O situations? I'm ripping through project Euler and tested the multi threading of python/ironpython on one of the problems. Single threaded python: 24 seconds Dual threaded python: 35 seconds Single threaded ironpython: 16 seconds Dual threaded ironpython: 9 seconds Note:This was an example containing pure mathematical computation. I'm a multithreading junkie but it needs to be multi-core viable.
|
|
#
?
Nov 14, 2007 00:47
|
|
|
Yeah, the CPython implementation has the GIL. It was discussed in the beginning of the thread. Basically good solutions if you want real threading are to use either ironpython or jython or a module like parallel python (which I really like and you should try it).
|
|
#
?
Nov 14, 2007 01:05
|
|
|
Hammertime posted:I'm relatively python ignorant, I've only been learning it for a week or so ... I actually just got done writing a fairly lengthy article talking about this (the gil+threads, etc). Here's the short version. Python, in the cPython interpreter uses OS-level threads (pthreads for unix people) and due to this, is perfectly capable of leveraging multiple cores. However, this is blocked by the GIL, which prevents any single thread from being executed within the interpreter at once, which make the interpreter (and making extensions) simple as you don't have to manage threads/memory - the core interpreter and the GIL will do that for you. The GIL is there to protect the interpreter's memory - not to hinder users or programmers. Ultimately, it is a cPython interpreter developer feature, not something which benefits end users (unless you are writing c-extensions). Which neatly deltas to the next point - any python code which is threaded, and only calls python code - i.e: number crunching - will run as fast (slightly slower) than the same code written single-threaded. However, if the code you are writing uses any c-code (and this includes built-in modules written in c) which is thread-safe and uses the PyGILState_STATE/Py_BEGIN_ALLOW_THREADS macros your code will spread across processors and run faster than the single threaded implementation. For example, 2 threads running a fibonacci calculation will run slightly slower than running the same calculation single threaded - however 2 threads performing disk I/O or socket calls will run much faster than the single threaded example. That all being said - the GIL is a feature for the interpreter developers, and it's been stated by many people that removing the GIL will make life quite difficult for a lot of contributors and users. There has been work to remove it and I know of at least two people making a patch set for python 3000 to remove the GIL but keep the ease of use. There are however, other options than to use the basic threading module and adding a time.sleep(.00001) (that forces the GIL to be released) and using I/O. There's actually many modules that not only mimic the threading API, but allow applications to distribute load across processors and machines. Here's a "short list" of modules: http://code.google.com/p/python-distributed/wiki/PythonPackagesOfNote Check out the rest of the wiki for more information, of note is a quick run down of python GIL related articles: http://code.google.com/p/python-distributed/wiki/PythonGILRelatedArticles I tend to prefer the Processing module (http://pypi.python.org/pypi/processing/) given I can drop it in and quickly replace threading when I've run into scaling problems.
|
|
#
?
Nov 14, 2007 02:02
|
|
|
I still don't really understand why the GIL helps interpreter developers. They can't write threadsafe code?
|
|
#
?
Nov 14, 2007 02:19
|
|
|
fake posted:I still don't really understand why the GIL helps interpreter developers. They can't write threadsafe code? No, it means they don't have to worry about it being thread safe - no worries about memory/data corruption, deadlock issues, it keeps garbage collection simple and out of the way, etc. Yes, they can write thread safe code, but as people much smarter than I (Goetz and Eckel and others) threading is difficult to get dead-right and can lead to madness, especially when figuring out a testing matrix. Some could also argue that due to the simplicity of the C interface to python, there has been a much higher rate of third-party module developed for python. Even third party modules that bypass the GIL for concurrency.
|
|
#
?
Nov 14, 2007 03:17
|
|
|
Look, between that whitespace bullshit and this inability to handle threading natively, I don't see how anyone can consider Python anything but a toy language  ...on a more serious note, to anyone else who may have been using Python for a few years now and taking it for granted: go write nothing but PHP for a month or two. You'll stop taking it for granted real quick...  Also, the official Django book is done! Should be on shelves by mid/late December, if not earlier, I'm told 
|
|
#
?
Nov 14, 2007 03:47
|
|
|
bitprophet posted:Look, between that whitespace bullshit and this inability to handle threading natively, I don't see how anyone can consider Python anything but a toy language Dude, gently caress whitespace. http://timhatch.com/projects/pybraces/ Also, I pre-ordered the django book tonight.
|
|
#
?
Nov 14, 2007 04:10
|
|
|
m0nk3yz posted:Dude, gently caress whitespace. http://timhatch.com/projects/pybraces/ from __future__ import braces  I actually had someone argue the whole whitespace thing with me in college. I told him I didn't care and that I much prefer the way whitespace is significant in python. Same guy had some problem with his java code on one of our classes and asked me to check it... yep, he did an if, if, else and had it indented wrong and attributed the else to the wrong if. deimos fucked around with this message at 04:17 on Nov 14, 2007 |
|
#
?
Nov 14, 2007 04:14
|
|
|
m0nk3yz posted:Also, I pre-ordered the django book tonight. I pre-ordered it the day pre-orders were announced, which at this point must have been at least a year+ ago. Got like a half-dozen notices from Amazon since then about its release date being pushed back. But no more!
|
|
#
?
Nov 14, 2007 04:45
|
|
|
I have a system at work where I run about 10 processes in the background using pythonw.exe - this is a bit tricky if I want to kill a specific process from the task manager, since they are all listed as, well, "pythonw.exe" Anybody know if there's a way of "renaming" the python interpreter short of renaming the .exe/compiling with py2exe so that it's easier to kill? My current solution involves adding a XMLRPC interface to all of the processes, which kind-of solves the killing problem (unless it's stopped responding totally), but still makes it hard to see which of the processes is the one taking up 99% CPU 
|
|
#
?
Nov 16, 2007 13:22
|
|
|
What is the standard for documenting your code? I get a kick out of javadoc/phpdoc. Is there a pythondoc or some equivalent way of generating HTML documentation of your API?bitprophet posted:... Is this book you speak of The Definitive Guide to Django: Web Development Done Right or a different one? SmirkingJack fucked around with this message at 14:50 on Nov 16, 2007 |
|
#
?
Nov 16, 2007 14:35
|
|
|
SmirkingJack posted:What is the standard for documenting your code? I get a kick out of javadoc/phpdoc. Is there a pythondoc or some equivalent way of generating HTML documentation of your API? Docstings and pydoc. http://www.diveintopython.org/getting_to_know_python/documenting_functions.html http://docs.python.org/lib/module-pydoc.html
|
|
#
?
Nov 16, 2007 14:51
|
|
|
SmirkingJack posted:Is this book you speak of The Definitive Guide to Django: Web Development Done Right or a different one? Looks like it, based on the publication details (author and publisher). I was hoping they'd go with the same color scheme they use on the website though 
|
|
#
?
Nov 16, 2007 15:42
|
|
|
SmirkingJack posted:s this book you speak of The Definitive Guide to Django: Web Development Done Right or a different one? Yup, that's the one. Adrian and Jacob. I think the color scheme is because it's part of a technical series by the publisher, I recognize it from other books. A Django book with the Django website colors would rock, though.
|
|
#
?
Nov 16, 2007 16:50
|
|
|
tef posted:Docstings and pydoc. Also note EpyDoc - it's above and beyond pydoc http://epydoc.sourceforge.net/
|
|
#
?
Nov 16, 2007 21:33
|
|
|
Has anyone used pyglet yet? I've been playing around with it and it's awesome, I think I am going to see if I can make a Visual Hub wannabe with it.
|
|
#
?
Nov 16, 2007 23:25
|
|
|
Yes, I know I was the last reply to this thread, but I am currently debating a e/n post because I just got bit by one of python's more annoying "features": all directories are modules. Working on setting up my personal dev server, and goddamnit if I didn't have the hardest time setting up my virtual host with fastcgi. Who would've thought /home/lighttpd/vhosts/domain.tld/django was a bad loving idea for putting the django projects for my site. Yep. I just spent about half an hour wondering what the gently caress was wrong. deimos fucked around with this message at 01:44 on Nov 18, 2007 |
|
#
?
Nov 18, 2007 01:39
|
|
|
Call me stupid, but I don't get exactly what was wrong there? Never used FCGI, FWIW, always mod_python. vvv Gotcha. That's not really a directory-as-module issue (since it should only treat directories as modules if they contain a '__init__.py' file) but more a pathing issue - the local path overrides site-packages if there's a collision, as you discovered. bitprophet fucked around with this message at 18:27 on Nov 18, 2007 |
|
#
?
Nov 18, 2007 04:05
|
|
|
bitprophet posted:Call me stupid, but I don't get exactly what was wrong there? Never used FCGI, FWIW, always mod_python. Not a problem with fcgi. But it was trying to import stuff in my directory called django, not the actual django stuff in my site-packages.
|
|
#
?
Nov 18, 2007 04:48
|
|
|
I remember a year or two ago there was a set of videos that praised python as a language. I don't have the slightest clue as to where I saw this or who made it, but I do remember there were a few high school teachers talking about how they would teach it in their computer classes and how the students found it to be ridiculously simply to learn. If memory serves me right, the video also acted as a beginners tutorial to teach the basics of python. Does anyone recall anything like I described? It'd be an awesome addition to this thread. Other then that, I'd recommend skimming through the showmedo? videos regarding python. They're pretty straight forward and insightful if you can put up with the various accents the speakers have. http://showmedo.com/videos/python P.S. I hate the main guy Ian Ozvald. Not only does he ramble on and on in his videos, but he also tries to charge money for some of his poo poo. Which, in my mind, goes against the whole point of the open(free)-source'ness of pyton 
|
|
#
?
Nov 18, 2007 20:17
|
|
|
Yeah. Threading isn't really pythons strongest point. I think most people in the Python community pretty much acknowledge that. The reason isn't so much interpreter development, its to do with module development. Linking C libs into Python is pretty simple, but you do have to pay attention to the garbage collection. However, things get... messy... when you start threading (As they do in C). The GIL stops this messyness by putting some nice big locks in there to keep stuff sane re Garbage collection and its ilk. You certainly apreciate it when embedding python. But it is a problem, and one that really wants some thought put into fixing. For what its worth, I tend to use stackless python for multithreading stuff these days. That poo poo scales like crazy. I mean, insane-loving-lala crazy. Coroutines rock.
|
|
#
?
Nov 19, 2007 08:12
|
|
|
duck monster posted:For what its worth, I tend to use stackless python for multithreading stuff these days. That poo poo scales like crazy. I mean, insane-loving-lala crazy. Coroutines rock. I'm googling stackless python but I'm not turning up anything that useful. Do you have a good link or another sentence or two? Does it get around GIL? I *really* love python, but I also *really* love parallelism at the thread level. I'm buying the django book and I want to bring python in as a language I can leverage in case RoR pisses me off too much(full time software dev). Edit: I'm stupid, found some good links in another search, interesting stuff, downloading it now, then I'll check out the tutorials. Home Page (with download): http://www.stackless.com/ Tutorial/walkthrough: http://members.verizon.net/olsongt/stackless/why_stackless.html Edit2: Well, it removes much of python's threading overhead. Doesn't solve the GIL problem though, still bound to a single core. Neat idea, seems a big waste of effort though, unless I've missed something.
Hammertime fucked around with this message at 11:17 on Nov 19, 2007 |
|
#
?
Nov 19, 2007 09:49
|
|
|
I'm writing a wxPython application that deals with user-input regular expressions. Certain pathological regular expressions can take more than a few seconds (or forever) to terminate, and I want the UI to be responsive and allow the user to modify their regex during this computation. I have the regex in a separate thread, but it still seems to block the whole app while it's running. How can I prevent a computationally intensive thread from blocking my whole app? by the way, try matching this: a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa against this: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
|
#
?
Nov 19, 2007 11:22
|
|
|
Hammertime posted:Edit2: Well, it removes much of python's threading overhead. Doesn't solve the GIL problem though, still bound to a single core. Neat idea, seems a big waste of effort though, unless I've missed something. You're correct, stackless just removes the stack: Not the Global Interpreter Lock. As has been pointed out previously, if you're trying to sidestep the GIL, look at the module I (and others) have linked too. If you're worried about the GIL + Web applications ala Django: Don't just yet. When Django Apps are served up via mod_python within Apache, the model is wildly different (given apache does use multiple cpus/cores). Most of the time in web apps you're also i/o (not cpu) bound. m0nk3yz fucked around with this message at 13:18 on Nov 19, 2007 |
|
#
?
Nov 19, 2007 12:56
|
|
|
m0nk3yz posted:When Django Apps are served up via mod_python within Apache, the model is wildly different (given apache does use multiple cpus/cores). Same with FastCGI and WSGI.
|
|
#
?
Nov 19, 2007 14:17
|
|
|
I'm not sure if this deserves its own thread, but I'm not actually doing any work on this yet, so I thought I would post here to check feasibility. Any feedback people have for whether this would/should work, things to keep in mind, different ways to do things, ..., would be appreciated. If this goes anywhere, I'll start a thread about it. Ok. I'm not really a programmer, I'm a cognitive psychologist who does some coding. At the moment, I primarily use Matlab with Psychtoolbox to deploy behavioral experiments, and Matlab proper for writing neural network simulations. I'm interested in moving to Python for both of these purposes, probably using PyGame for deploying experiments and SciPy for vectorized computations for the modeling. I'd also like to be able to run statistical analyses in Python -- basically be able to never leave Python for the entire course of an entire study. Let me explain what is required in a typical experiment. Let's say I'm writing a simple Stroop task. I need to be able to do the following:
If I were doing something like an artificial language learning task, I might not care about reaction time, but would want to be able to play sounds, and so forth. None of this is very complicated, and it seems like a library optimized for writing games should be able to handle everything I need with ease. I guess my question is, how much work would it be to write something like the above from scratch, or at best, from snippets of reused code, using PyGame? I'm hoping that the answer is "not much", once the logic of the experiment has been determined, but I want to verify this with people who know the language and the libraries before I start sinking time into this project. My ulterior motive in all of this is to implement this all as a project for the One Laptop Per Child program. Given a few sample experiment programs and some tutorials, I really think that kids as young as 10 or 12 could learn to conduct some fairly sophisticated cognitive psychology. Plus, how interesting would it be to learn about psychology by actually running the experiments on your friends and classmates, crunching the numbers yourself, and seeing that the results are valid? I really believe that a hands-on understanding of the human mind would be enormously valuable for more people to have, and a constructivist learning tool like the OLPC XO provides a means for even younger children to grasp some very subtle concepts and results. Since Python is the language of choice for the project, that's the language I have to learn -- though of course I've heard good things about it anyway. So. Thoughts? Plausible? Waste of time? Monokeros deAstris fucked around with this message at 06:14 on Nov 20, 2007 |
|
#
?
Nov 20, 2007 05:48
|
|
|
Hammertime posted:I'm googling stackless python but I'm not turning up anything that useful. Stackless implements continuations and co-routines. Prepare for your head to explode. Basically a continuation is like a 'save point' in a game, although within the context of execution. Its sort of like saving the current state, then loving off and doing something else, so you can return later , refire execution and continue where you left off. co-routines are a species of continuation. I probably can't explain them well, so wikipedia them. What stackless lets you do is implement 'green threads', where essentially you supply your *own* scheduler and run potentially tens of thousands of microthreads within a single thread. They scale pretty much Big-O(1) and when a thread isnt doing anything they require zero cpu time. You really can go apeshit with threads and not worry too much about the performance overhead. A good example here would be a web server Picture this: code:It sets x to 10 then fires off a page that says "value = x" The page then saves its continuation and lets the server do other poo poo When the link (Next) is pressed, it wakes up the continuation, increments x by 10, fires off another page, then falls asleep again, and so on till it gets to the return, and it closes the thread. Now the fun part of continuations (And alas it doesnt work so well in the current stackless I *think*, is you should be able to hit the back button, and since the continuation identifier now points to the old page, it'll roll back the execution context, and re-run the code from that point in its history. Scary huh? Seaside (A smalltalk based system) uses that extensively. For multi-page forms, its erection inducing, because maintaining state gets *MUCH* easier. It also completely flies in the face of much commonly held belief on internet best pracice. gently caress em. The point being, is it allows you to implement cooperative multithreading in an extemely lightweight manner, that scales like a motherfucker. And no it doesnt handle multiple cores well. Yet. I guess they are working on that. No idea about the GIL. duck monster fucked around with this message at 08:22 on Nov 20, 2007 |
|
#
?
Nov 20, 2007 08:20
|
|
|
duck monster posted:Heaps of Stuff Thanks for the explanation. You made the threading model/scheduling and scalability really easy to digest. As others have suggested I'm looking into process level parallelism, parallelpython and such.
|
|
#
?
Nov 20, 2007 09:32
|
|
|
Alhireth-Hotep posted:So. Thoughts? Plausible? Waste of time? I just got done with a project in PyGame, and this sounds really, really easy. Saving the data and dumping to CSV will be super-trivial in Python, and PyGame makes throwing things on the screen easy. Sounds are pretty easy, too. I don't know how high of a resolution timer Python has, though, if you want to be super awesomely precise (as much as the processor/OS/whatever can handle) in your timing measurements.
|
|
#
?
Nov 20, 2007 18:53
|
|
|
Ugg boots posted:I just got done with a project in PyGame, and this sounds really, really easy. Saving the data and dumping to CSV will be super-trivial in Python, and PyGame makes throwing things on the screen easy. Sounds are pretty easy, too. I don't know how high of a resolution timer Python has, though, if you want to be super awesomely precise (as much as the processor/OS/whatever can handle) in your timing measurements. For my own experiments, I want to be as precise as possible, so I'll probably keep using Psychtoolbox. But as long as the timing errors don't correlate with any of the variables of interest, all they'll do is add noise to the outcome, which is a problem, but not a huge one. Thus, I think it'll probably be fine for educational purposes. Thanks, this is encouraging.
|
|
#
?
Nov 20, 2007 19:33
|
|
|
I've been playing around with python in terms with game-development and I am very impressed with what I can do with it. Using a dictionary to manage various states, memory management, abusing a static class to manage game variables, it's all good  .here is a paste bin of my current breakout style project. The ball bounces, use the arrow keys to move around, and F1 and F2 to switch the states. You also need pygame. http://rafb.net/p/DpRL0514.html
|
|
#
?
Nov 20, 2007 22:54
|
|
|
Yup. Thats the magic of python. Dictionarys, lists and tuples. So simple, and yet so expressive. Sure you can do it in other languages, but python makes it so elegant. Also, Py-game is a really fun library. Aeons ago I wrote a little game that I never finished based on a really spastic take on eve-online (Called something like "goonie lagsplot 2d" or something absurd). Although I never really bothered beyond about 4 or 5 pages of code it was loving fun to write what I wrote, and the prototype was actually kind of fun. I could see it being very easy to write a 'real' 2D game in it. What would be *real* cool would be a python version of Delphis old GLScene library. Open GL coding gets really drat cool when its turned into a first class OO library.
|
|
#
?
Nov 21, 2007 02:15
|
|
|
I would personally rewrite:code:code:duck monster posted:Yup. Thats the magic of python. Dictionarys, lists and tuples. So simple, and yet so expressive. Sure you can do it in other languages, but python makes it so elegant. Look into pyglet. OpenGL is pretty much a first class citizen. Not to mention a  of a positional sound library.
deimos fucked around with this message at 02:40 on Nov 21, 2007 |
|
#
?
Nov 21, 2007 02:37
|
|
|
Ok, I just don't get it.code:code:code:
|
|
#
?
Nov 24, 2007 17:55
|
|
|
Try "print test.var1" and "print test.var2". (It'll be more clear if you initialize var1 to "default" instead of "".) Here's what's happening: code:code:If you want to be sure an instance variable is initialized before calling add, do it in __init__: code:
|
|
#
?
Nov 24, 2007 18:14
|
|
|
I'm looking to get into Python a bit; since I learn best from paper books, any recommendations? And yes, yes, I know, you learn by doing not reading, but I like books for grokking languages. Something with some good depth and the spirit of the language would be great. The online Diving into Python is pretty good, and I was thinking of getting the dead-tree version. Is there better?
|
|
#
?
Nov 24, 2007 23:09
|
|
|
FuraxVZ posted:The online Diving into Python is pretty good, and I was thinking of getting the dead-tree version. Is there better? The only suggestion I can make is take one that uses version 2.5. DIP is written for 2.3 and as such has a kind of roundabout way of dealing with things that can be more elegantly written in 2.5.
|
|
#
?
Nov 25, 2007 00:15
|
|
|
FuraxVZ posted:I'm looking to get into Python a bit; since I learn best from paper books, any recommendations? And yes, yes, I know, you learn by doing not reading, but I like books for grokking languages. Something with some good depth and the spirit of the language would be great. The latest edition of Core Python Programming is really good. Covers 2.5 to boot.
|
|
#
?
Nov 25, 2007 01:27
|
|
|
|
| # ? Apr 25, 2024 13:08 |
|
|
m0nk3yz posted:The latest edition of Core Python Programming is really good. Covers 2.5 to boot. I second this, I particularly like Core Python's treatment of variables and memory.
|
|
#
?
Nov 25, 2007 18:47
|
|