Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

cowboy beepboop: Feb 24, 2001

Janin posted:

help

Thank you!

# ? Nov 19, 2008 23:48

Adbot: ADBOT LOVES YOU

# ? May 26, 2024 19:56

OneEightHundred: Feb 28, 2008; Soon, we will be unstoppable!

I'm obviously not reading the manual properly, so here comes my phenomenally dumb question: What determines the scope of a variable?

I'm used to Lua, where variables explicitly declared local, loop control variables, and parameters were limited in scope to where they were created, and everything else would be dumped in the global namespace. In Python, I have absolutely no idea where it puts variables, and what causes them them wind up in the global namespace.

Also, how would you import a module in another directory without breaking its imports? I wrote a module that needs to import a third-party module in the same directory, but importing it via the import API fails because that module's import statements fail to resolve (since it appears to use the CWD as the search path instead of the imported module's directory).

OneEightHundred fucked around with this message at 00:14 on Nov 20, 2008

# ? Nov 20, 2008 00:10

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

OneEightHundred posted:

I'm obviously not reading the manual properly, so here comes my phenomenally dumb question: What determines the scope of a variable?

I'm used Lua, where "local" variables, loop control variables, and parameters were limited in scope to where they were created, and everything else would be dumped in the global namespace. In Python, I have absolutely no idea where it puts variables, and what causes them them wind up in the global namespace.

Variables are global if they are assigned to in the module, rather than in a function. Global variables can be assigned to within a function only if the "global <varname>" statement is used first.

In Python, each function has its own set of local variables. Loop variables are leaked, generator expression intermediates are not, list comprehension intermediates are leaked in 2.x but are not in 3.x.

OneEightHundred posted:

Also, how would you import a module in another directory without breaking its imports? I wrote a module that needs to import a third-party module in the same directory, but importing it via the import API fails because that module's import statements fail to resolve (since it appears to use the CWD as the search path instead of the imported module's directory).

what

If they're in the same directory, everything should work fine. If it's in another directory, that directory should be added to sys.path.

# ? Nov 20, 2008 00:15

Pivo: Aug 20, 2004

Janin posted:

If they're in the same directory, everything should work fine. If it's in another directory, that directory should be added to sys.path.

I think he means the directories are set up like:

code:

- his_app.py
/ folder
  - foo.py
  - bar.py

His app imports foo, foo imports bar. I've been wondering this myself.

# ? Nov 20, 2008 01:27

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

OneEightHundred posted:

Also, how would you import a module in another directory without breaking its imports? I wrote a module that needs to import a third-party module in the same directory, but importing it via the import API fails because that module's import statements fail to resolve (since it appears to use the CWD as the search path instead of the imported module's directory).

Add the directory to sys.path if you can't install it. The PYTHONPATH environment variable will be read in when the interpreter starts and paths there added to the start of sys.path. The cwd isn't added to sys.path by default, though; only the directory of the script that you're running. Often times this is the cwd, but not always. So, if you have your script in a folder, the cwd is somewhere else, and the script tries to import a module in the same directory as itself, it'll work.

# ? Nov 20, 2008 01:35

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Pivo posted:

I think he means the directories are set up like:
code:
- his_app.py
/ folder
  - foo.py
  - bar.py
His app imports foo, foo imports bar. I've been wondering this myself.

If foo.py relies on bar.py, it should probably be in a package like this:

code:

- his_app.py
/ subpackage
  - __init__.py
  - foo.py
  - bar.py

then he can use from subpackage import foo

# ? Nov 20, 2008 02:21

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Janin posted:

If foo.py relies on bar.py, it should probably be in a package like this:
code:
- his_app.py
/ subpackage
  - __init__.py
  - foo.py
  - bar.py
then he can use from subpackage import foo

However, when absolute imports are turned on (by default in python 2.7+ and optionally in python 2.5+) this won't work, because that's an implicit relative import. The better solution is to have the directory in sys.path.

# ? Nov 20, 2008 04:19

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Habnabit posted:

However, when absolute imports are turned on (by default in python 2.7+ and optionally in python 2.5+) this won't work, because that's an implicit relative import. The better solution is to have the directory in sys.path.

Why wouldn't it work? As you mentioned, the directory containing the script being run will be added to sys.path. "import subpackage" and its variants would therefore work even in absolute-import mode.

e: just tested in 2.5, importing absolute_import from __future__, and it works as expected.

TOO SCSI FOR MY CAT fucked around with this message at 05:30 on Nov 20, 2008

# ? Nov 20, 2008 05:27

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Janin posted:

Why wouldn't it work? As you mentioned, the directory containing the script being run will be added to sys.path. "import subpackage" and its variants would therefore work even in absolute-import mode.

e: just tested in 2.5, importing absolute_import from __future__, and it works as expected.

If subpackage/foo.py imports subpackage/bar.py as 'import bar' then it won't work. __future__ imports also only affect the module that they're imported in. Try adding 'from __future__ import absolute_import' to foo.py and doing it.

# ? Nov 20, 2008 09:25

Lurchington: Jan 2, 2003; Forums Dragoon

what's the least hacky way to add things to sys.path? I've appended a directory in the past, but it didn't feel like the best way.

# ? Nov 20, 2008 14:52

tef: May 30, 2004; -> some l-system crap ->

Habnabit posted:

If subpackage/foo.py imports subpackage/bar.py as 'import bar' then it won't work. __future__ imports also only affect the module that they're imported in. Try adding 'from __future__ import absolute_import' to foo.py and doing it.

Didn't he say from subpackage import bar in subpackage/foo.py

Also you should be able to do from . import bar I think?

# ? Nov 20, 2008 15:04

Kire: Aug 25, 2006

The Remote Viewer posted:

Learning programming is such mindless loving drudgery. Coding is fun, learning how to code sucks, especially self-learning. I've never learned how to do anything worthwhile in any language because I can't get past the hump where enthusiasm starts to flag. At least outside of what I learned in high school.

When you first start out there's a little bit of excitement from embarking on a new venture, and then I assume you eventually get to the point where you can start working on your own projects which boosts enthusiasm again. Being in the middle where you're retyping examples of number-guessing games, sucks.

I suffer sometimes from boredom while I'm trying to learn Python, but I've found it helps if I ditch the introductory tutorials for a while and work on something slightly more complex and interesting, like Tkinter.

http://www.pythonware.com/library/tkinter/introduction/

Typing out the examples and making windows, buttons, status bars, is a fun diversion from trying to figure out classes and modules.

# ? Nov 20, 2008 15:08

tef: May 30, 2004; -> some l-system crap ->

Kire posted:

Typing out the examples and making windows, buttons, status bars, is a fun diversion from trying to figure out classes and modules.

The turtle module is fun too :3:

# ? Nov 20, 2008 15:35

Kire: Aug 25, 2006

I haven't gotten to Turtle mode yet, what's that? Part of Tkinter?

Also, this is a good resource for the upcoming changes when Python 3.0 is released (Dec 3 tentatively):
http://docs.python.org/dev/3.0/whatsnew/3.0.html

Can someone explain why they are changing the print function to print()? It looks annoying to use the new method. More typing.

Also, what are the new "as" and "with" keywords they're talking about? I don't understand their explanation.

# ? Nov 20, 2008 17:06

tef: May 30, 2004; -> some l-system crap ->

Kire posted:

I haven't gotten to Turtle mode yet, what's that? Part of Tkinter?

Here: http://www.python.org/doc/2.5.2/lib/module-turtle.html

The print change was documented in a pep:

http://www.python.org/dev/peps/pep-3105/

And so is the new with .. as statement:

http://www.python.org/dev/peps/pep-0343/

tef fucked around with this message at 17:27 on Nov 20, 2008

# ? Nov 20, 2008 17:18

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

tef posted:

Didn't he say from subpackage import bar in subpackage/foo.py

Also you should be able to do from . import bar I think?

Actually, rereading what he originally said, there was no indication which it was using. I suspect it's using 'import bar'. 'from . import bar' would work with absolute imports on, though.

# ? Nov 20, 2008 17:59

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Habnabit posted:

Actually, rereading what he originally said, there was no indication which it was using. I suspect it's using 'import bar'. 'from . import bar' would work with absolute imports on, though.

I was using from . import bar, since foo and bar are in a package.

# ? Nov 20, 2008 19:46

OneEightHundred: Feb 28, 2008; Soon, we will be unstoppable!

It's actually LOWER in the directory tree, so:

module1.py
module2.py
stuff/script.py

module1.py has:
import module2

... and script.py is what needs to be run.

# ? Nov 21, 2008 00:23

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

OneEightHundred posted:

It's actually LOWER in the directory tree, so:

module1.py
module2.py
stuff/script.py

module1.py has:
import module2

... and script.py is what needs to be run.

Temporary: ~$ PYTHONPATH=/my/app python /my/app/stuff/script.py

Permanent, modules in a fixed path (script may be moved anywhere):

code:

import sys
sys.path.append ('/my/app')

Permanent, modules must be in directory above script (cluster may be moved anywhere):

code:

import sys
from os.path import dirname
sys.path.append (dirname (dirname (__file__)))

# ? Nov 21, 2008 00:37

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

So, as you folks probably know, python has some characteristics of a functional programming language, like anonymous functions, 'map', 'filter', and 'reduce'. These are generally slower than their more pythonic counterparts (list comprehensions, generator expressions), and python code in general can start to look like a mess if you try to hack together more features of functional languages, such as monads.

However, none of this has ever deterred me from demonstrating the power of these features of the python language! As my latest example, I wrote a full-featured chat server in just one python expression. It binds to a TCP port and expects CRLF as a delimiter. It's a little long, so I posted it to a pastebin.

http://paste.pocoo.org/show/91848/

I guarantee this is safe to run! It doesn't execute or evaluate any arbitrary code at all; using 'eval' takes the fun out of this kind of thing. It also uses select, so it won't hog your CPU while waiting for IO. Not only that, but it handles SIGINT and SIGTERM nicely.

# ? Nov 21, 2008 07:40

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Habnabit posted:

http://paste.pocoo.org/show/91848/

It's possible to write functional-style Python without the resulting code look like ground meat. Your code looks more like an obfusticated code contest entry.

# ? Nov 21, 2008 08:38

fart simpson: Jul 2, 2005; DEATH TO AMERICA

I decided to finally try out stackless.

I'm in love
:swoon:

# ? Nov 21, 2008 09:52

Mashi: Aug 15, 2005; Just wanted you to know your dinner's cold and the children all agree you're a shitheel!

Habnabit posted:

So, as you folks probably know, python has some characteristics of a functional programming language, like anonymous functions, 'map', 'filter', and 'reduce'. These are generally slower than their more pythonic counterparts (list comprehensions, generator expressions), and python code in general can start to look like a mess if you try to hack together more features of functional languages, such as monads.

However, none of this has ever deterred me from demonstrating the power of these features of the python language! As my latest example, I wrote a full-featured chat server in just one python expression. It binds to a TCP port and expects CRLF as a delimiter. It's a little long, so I posted it to a pastebin.

http://paste.pocoo.org/show/91848/

I guarantee this is safe to run! It doesn't execute or evaluate any arbitrary code at all; using 'eval' takes the fun out of this kind of thing. It also uses select, so it won't hog your CPU while waiting for IO. Not only that, but it handles SIGINT and SIGTERM nicely.

Two questions:

1: What did you learn cramming a program onto one line?

2: Can you do it without lambdas?

# ? Nov 21, 2008 12:41

a slime: Apr 11, 2005

I want to subtract each item in a tuple from another, something like

code:

(15, 5) minus (5, 5) = (10, 0)

I thought maybe I could do this with list comprehensions

code:

(x-y for x in t1 and y in t2)

But that subtracts every element in t1 from every element in t2- I want (x[0]-y[0], x[1]-y[1], ...). Is this possible without reverting to for loops?

# ? Nov 21, 2008 14:50

computerality: May 6, 2006

not a dinosaur posted:

I want to subtract each item in a tuple from another, something like
code:
(15, 5) minus (5, 5) = (10, 0)
I thought maybe I could do this with list comprehensions
code:
(x-y for x in t1 and y in t2)
But that subtracts every element in t1 from every element in t2- I want (x[0]-y[0], x[1]-y[1], ...). Is this possible without reverting to for loops?

You might try one of these:

code:

imap(lambda x,y: x-y, (15,5),(10,0))

code:

(x-y for x,y in izip((15,5),(10,0)))

# ? Nov 21, 2008 15:46

bitprophet: Jul 22, 2004; Taco Defender

not a dinosaur posted:

But that subtracts every element in t1 from every element in t2- I want (x[0]-y[0], x[1]-y[1], ...). Is this possible without reverting to for loops?

Pedantry: what you posted is actually a generator expression, not a list comprehension

List comps have square brackets instead of parentheses. However, generator expressions are generally "better" as they have identical syntax but are usually more efficient -- so keep using them, just make sure you have the terminology right.

Anyway, think about your problem this way: you have two lists, and you want to do an operation on each pair of items, the pair being defined as "the items at the same index", right? I.e. subtract the pair of items at index 0, then index 1, ..., then index N.

You can't do this by iterating over one, then the other, which I think is what your expression does (honestly I've never seen that 'double loop' syntax before...curious). What you want to do is "zip" the two lists up, so you get a list or tuple of two-tuples: [(t1[0], t2[0]), (t1[1], t2[1]), ..., (t1[N], t2[N])].

This is, in fact, called zipping, and is a builtin method(s): http://docs.python.org/library/functions.html#zip (and also izip as in the previous post :argh:

)

Thus, I think you want: (x - y for x, y in izip(t1, t2)).

Ninja edit value-add: izip is better than just plain zip, because it itself uses an iterator and is thus more memory efficient if your input lists are large. Also: if len(t1) != len(t2), you'll run into problems unless you figure out what you want the 'padding' values to be, lest you get errors trying to do e.g. 1 - None.

bitprophet fucked around with this message at 15:49 on Nov 21, 2008

# ? Nov 21, 2008 15:47

tbradshaw: Jan 15, 2008; First one must nail at least two overdrive phrases and activate the tilt sensor to ROCK OUT!

Habnabit posted:

As my latest example, I wrote a full-featured chat server in just one python expression.

Hahaha! What a stunningly Perl-like thing to do. Good times.

# ? Nov 21, 2008 17:18

a slime: Apr 11, 2005

Thanks, both of you! izip did exactly what I needed.

# ? Nov 21, 2008 18:43

hey mom its 420: May 12, 2007

Or for even more code shortness and terseness, you can do imap(operator.sub, t1, t2) :v:

# ? Nov 21, 2008 20:12

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Mashi posted:

Two questions:

1: What did you learn cramming a program onto one line?

2: Can you do it without lambdas?

1. Not much. I do it for the challenge of it. This isn't the first time I've made a monolithic one-line program, though this is by far the longest.

2. Yes. You can turn all lambdas into infinite iterators and have a pseudo-call-stack at the root of it all. This would require a lot more effort, but it's definitely doable. In fact, now you've made me want to rewrite this without lambdas. It would certainly be more impressive to look at.

# ? Nov 21, 2008 22:11

duck monster: Dec 15, 2004

MeramJert posted:

I decided to finally try out stackless.

I'm in love

I wrote a really really insane web-spider for an old job using stackless. loving thing could max out a 20mbit fibrelink instantly. loving crazy-scaling.

# ? Nov 21, 2008 22:39

fart simpson: Jul 2, 2005; DEATH TO AMERICA

duck monster posted:

I wrote a really really insane web-spider for an old job using stackless. loving thing could max out a 20mbit fibrelink instantly. loving crazy-scaling.

The scaling and being able to pickle tasklets and resume them on another architecture just blows my mind.

# ? Nov 22, 2008 01:07

Mashi: Aug 15, 2005; Just wanted you to know your dinner's cold and the children all agree you're a shitheel!

Habnabit posted:

2. Yes. You can turn all lambdas into infinite iterators and have a pseudo-call-stack at the root of it all. This would require a lot more effort, but it's definitely doable. In fact, now you've made me want to rewrite this without lambdas. It would certainly be more impressive to look at.

I want to see what you mean by the fake callstack.

Perhaps 99 bottles doesn't present the same types of challenge but i just did it as an expression for fun:

code:

from __future__ import print_function
[(  print("%d bottle%s of beer on the wall. %d bottle%s of beer." % ((i, 's' if i>1 else '') *2)) or
    (
        print("Take one down and pass it around, %d bottle%s of beer on the wall.\n" % (i-1, 's' if i>2 else ''))
            if i>1 else 
        print("Go to the store and buy some more, 99 bottles of beer on the wall.")
    )
) for i in xrange(99, 0, -1)]

# ? Nov 22, 2008 06:40

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Mashi posted:

I want to see what you mean by the fake callstack.

I'm not going to work on it riight now, but I probably will when I have to take a three-hour bus ride in the next few days. Here's what I mean, though:

code:

    def dump(self, obj):
        callstack = [self.pack(obj)]
        while callstack:
            try:
                result = callstack[-1].next()
            except StopIteration:
                callstack.pop()
            else:
                if result is not None:
                    callstack.append(result)

    @packs(types.TupleType)
    def pack_tuple(self, t):
        e = -1
        for e, itm in enumerate(t):
            yield self.pack(itm)
        self.pack_int(e + 1)

This isn't condensed down into one expression, but it's an excerpt from a serializer that I wrote a while ago. Each function uses some limited recursion, but to allow (theoretically) infinitely nested structures, functions can yield generators which will be pushed to the call stack.

# ? Nov 22, 2008 06:47

tripwire: Nov 19, 2004; _{ghost flow}

Habnabit posted:

So, as you folks probably know, python has some characteristics of a functional programming language, like anonymous functions, 'map', 'filter', and 'reduce'. These are generally slower than their more pythonic counterparts (list comprehensions, generator expressions), and python code in general can start to look like a mess if you try to hack together more features of functional languages, such as monads.

However, none of this has ever deterred me from demonstrating the power of these features of the python language! As my latest example, I wrote a full-featured chat server in just one python expression. It binds to a TCP port and expects CRLF as a delimiter. It's a little long, so I posted it to a pastebin.

http://paste.pocoo.org/show/91848/

I guarantee this is safe to run! It doesn't execute or evaluate any arbitrary code at all; using 'eval' takes the fun out of this kind of thing. It also uses select, so it won't hog your CPU while waiting for IO. Not only that, but it handles SIGINT and SIGTERM nicely.

How do you use it? I can't even begin to understand what your code does.. are you supposed to open main with some specific arguments?
If I just call it from the command line with "python whatever.py" it instantly closes
Nevermind, figured it out. Wow, I'm impressed. I guess all of the text in it is compressed into that big chunk of gibberish.

tripwire fucked around with this message at 11:34 on Nov 22, 2008

# ? Nov 22, 2008 11:29

tef: May 30, 2004; -> some l-system crap ->

Mashi posted:

code:

from __future__ import print_function

Who needs imports when you can do __import__("sys").stdout.write("foo") :c00l:

# ? Nov 22, 2008 14:19

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

MeramJert posted:

The scaling and being able to pickle tasklets and resume them on another architecture just blows my mind.

Your last two posts have motivated me to get started with stackless. I've always wanted to learn erlang but it never clicked, so I will build yet another CouchDB clone in stackless. edit: clone building not so much to make a functional clone, but to play with the ideas that couch implements, reinventing the wheel to learn how a badly built wheel works.

deimos fucked around with this message at 04:31 on Nov 23, 2008

# ? Nov 23, 2008 04:11

Glee: Jul 22, 2006

EDIT! Nevermind, figured I need to check another part beforehand.

Glee fucked around with this message at 05:17 on Nov 23, 2008

# ? Nov 23, 2008 04:53

fart simpson: Jul 2, 2005; DEATH TO AMERICA

deimos posted:

Your last two posts have motivated me to get started with stackless. I've always wanted to learn erlang but it never clicked, so I will build yet another CouchDB clone in stackless. edit: clone building not so much to make a functional clone, but to play with the ideas that couch implements, reinventing the wheel to learn how a badly built wheel works.

Go for it! That's basically what I'm doing right now, only I'm making a really simple game to test it all out (not for the sake of making an actually playable game, but just to try out some new stuff from stackless and 2.6)

# ? Nov 23, 2008 05:15

Adbot: ADBOT LOVES YOU

# ? May 26, 2024 19:56

ahobday: Apr 19, 2007

I'm thinking, since I'll start coding for my university project after Christmas, that just jumping straight into Python 3.0 is a better idea than 2.6. I've already learned some Python, but it's not ingrained in my head so much that 3.0 will be confusing, and I'd rather learn the new standard than the "old" one, especially since they're releasing it next month.

Is this a good idea? Also, is the Python.org tutorial going to be the only learning resource for 3.0 until other people start writing them? I'm not particularly fond of how the Python.org tutorial starts out, since I'm already familiar with the basics of programming, having done Java and some C++. It seems like a tutorial designed for newcomers to programming, since it starts with using Python as a calculator and whatnot.

# ? Nov 23, 2008 13:06

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »