Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

king_kilr: May 25, 2007

Use virtualenv, it makes using whatever python you like eays.

# ? Oct 29, 2010 05:44

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 02:25

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

I'm just a hobby programmer right now, so I'm curious as to how you more experienced folk are dealing with this:

It's really frustrating that all the libraries I want to use don't support Python 3. Should I just stick to Python 2.x for now? I was thinking of learning Python 3 from the get go so I don't have to re-learn all the changes that are made. Advice?

# ? Oct 29, 2010 15:41

MaberMK: Feb 1, 2008; BFFs

king_kilr posted:

Use virtualenv, it makes using whatever python you like eays.

That doesn't solve the problem he was asking. You use virtualenv to isolate a set of modules from the set installed on the system.

You need to put the directory containing your custom Python build at the beginning of your PATH environment variable, or at the very least, ahead of any other directories containing Python binaries. Then, your shebangs should be:

code:

#!/usr/bin/env python

And that will do it.

# ? Oct 29, 2010 16:30

tef: May 30, 2004; -> some l-system crap ->

Bodhi Tea posted:

I'm just a hobby programmer right now, so I'm curious as to how you more experienced folk are dealing with this:

It's really frustrating that all the libraries I want to use don't support Python 3. Should I just stick to Python 2.x for now? I was thinking of learning Python 3 from the get go so I don't have to re-learn all the changes that are made. Advice?

which libraries are giving you a problem?

python3 is worth using if you're going to have to deal with unicode, beyond that 2.7 is pretty comparable to 3.2 so uh, learn whatever - the differences aren't big between them.

# ? Oct 29, 2010 17:50

Stabby McDamage: Dec 11, 2005; Doctor Rope

I have a minor curiosity. I sometimes use this class in various testing apps -- it basically walks an n-dimensional integer space. Put another way, it encodes n nested for loops in one. The list it produces can be indexed directly, so it makes it easy to say "re-run test 57", which is the test whose parameters are [1,4,7,5] or whatever. Code:

code:

# gives all possible integer vectors between two endpoints, inclusively.  For example:
# IntVectorRange([0,0,2], [1,1,3]) yields:
#   [0, 0, 2]
#   [1, 0, 2]
#   [0, 1, 2]
#   [1, 1, 2]
#   [0, 0, 3]
#   [1, 0, 3]
#   [0, 1, 3]
#   [1, 1, 3]
# if only one argument is given to __init__, then the range goes from [0,0,...,0] to the vector given
class IntVectorRange(object):
	def __init__(self,start,end=None):
		if end is None:
			end=start
			start=[0] * len(end)
 
		self.start=start
		self.end=end
		self.size = len(start)
		self.delta = [end[i]-start[i]+1  for  i in xrange(self.size)]
		self.len = reduce(lambda x,y: x*y, self.delta, 1)
 
	def __getitem__(self,index): 
		if index >= self.len: raise IndexError
		r = [0] * self.size
		for i in xrange(self.size):
			r[i] = index % self.delta[i] + self.start[i]
			index /= self.delta[i]
		return r
 
	def __len__(self): return self.len

This works perfectly fine and I have no problems with it, but I'm wondering if there's a really cool, jazzy way to make this simpler or more elegant.

# ? Oct 29, 2010 20:02

king_kilr: May 25, 2007

MaberMK posted:

That doesn't solve the problem he was asking. You use virtualenv to isolate a set of modules from the set installed on the system.

You need to put the directory containing your custom Python build at the beginning of your PATH environment variable, or at the very least, ahead of any other directories containing Python binaries. Then, your shebangs should be:
code:
#!/usr/bin/env python
And that will do it.

My point was you can use any Python to create a virtualenv, and then it's easy to ensure you're using the right Python (and assosciated modules) by simply ensuring you're in the right venv.

# ? Oct 29, 2010 21:11

tef: May 30, 2004; -> some l-system crap ->

Stabby McDamage posted:

I have a minor curiosity. I sometimes use this class in various testing apps -- it basically walks an n-dimensional integer spa

itertools is your friend

code:

>>> import itertools
>>> def get_range(start, stop):
...     return itertools.product(*(xrange(a,b+1) for a,b in zip(start,stop)))
... 
>>> for x in get_range((0,0,2),(1,1,3)):
...     print x
... 
(0, 0, 2)
(0, 0, 3)
(0, 1, 2)
(0, 1, 3)
(1, 0, 2)
(1, 0, 3)
(1, 1, 2)
(1, 1, 3)

# ? Oct 29, 2010 23:20

Stabby McDamage: Dec 11, 2005; Doctor Rope

tef posted:

itertools is your friend

code:

>>> import itertools
>>> def get_range(start, stop):
...     return itertools.product(*(xrange(a,b+1) for a,b in zip(start,stop)))
... 
>>> for x in get_range((0,0,2),(1,1,3)):
...     print x
... 
(0, 0, 2)
(0, 0, 3)
(0, 1, 2)
(0, 1, 3)
(1, 0, 2)
(1, 0, 3)
(1, 1, 2)
(1, 1, 3)

Wow, that's crazy. Very cool.

One thing though -- I need to be able to subscript the object without building the whole thing in memory. This generator would need to iterate up to the index. Any ideas on how to encode my math-based algorithm using similar magic?

# ? Oct 30, 2010 04:20

tripwire: Nov 19, 2004; _{ghost flow}

I don't get it, where in that code does anything get built up in memory entirely? Product gives you an iterator and xrange doesn't store the whole list in memory at once like range does.

# ? Oct 30, 2010 05:05

Stabby McDamage: Dec 11, 2005; Doctor Rope

Let me clarify: if I want to jump directly to an index, I need to walk through it. I can do that either by iterating it to a point and then stopping (large time cost per lookup) or building the whole thing in memory and accessing it like a list (initial time cost + large memory cost).

The class I posted can be used as an iterator or as a direct look-up without paying either of these overheads.

# ? Oct 30, 2010 13:14

Aafter: Apr 14, 2009; A is for After.

Another newbie!

Ive been trying to learn Python here and there for the last 3 years, on and off. I always get suck at dictionaries. Ill be in school for Computer Science soon, but gently caress, I want some programming action now.

Anyway, Im gonna try my hand at this again now that the urge to learn has come back again. I'll be back often to bug everyone. Remember me, Ill be the dumbass.

# ? Nov 1, 2010 10:49

Stabby McDamage: Dec 11, 2005; Doctor Rope

Three dumb questions:

1. What is the easiest way to turn a generator into a list? I was playing with itertools on the command line, and of course when I do "product('abc')", all I see is "<itertools.product object at 0x1e21140>". I've been wrapping these generators with "[x for x in <whatever>]", but I'm thinking there's got to be some shorthand for that.

2. Is there any way to ask for the old print statement in python 3? I'd never use it in code, but it might be nice on the command line. Unfortunately, it doesn't look like there's a "__past__" module.

3. I just noticed that I can chain comparators and it seems to do the correct (mathematical) thing. This conflicts with my expectation that operators of equal weight are evaluated left-to-right. For example:

pre:

a=2

I type:    3 < a < 6
I expect: (3 < a) < 6   →  (False) < 6     → True   # mathematically stupid, but matches behavior of C, Java, Perl, etc.
I get:     3<a and a<6  →  False and True  → False  # mathematically sound, useful syntax, but differs from programmer expectation.

Is this intentional? What caveats might I run into with this?

Stabby McDamage fucked around with this message at 18:35 on Nov 1, 2010

# ? Nov 1, 2010 18:25

Threep: Apr 1, 2006; It's kind of a long story.

Stabby McDamage posted:

1. What is the easiest way to turn a generator into a list? I was playing with itertools on the command line, and of course when I do "product('abc')", all I see is "<itertools.product object at 0x1e21140>". I've been wrapping these generators with "[x for x in <whatever>]", but I'm thinking there's got to be some shorthand for that.

You're probably going to slap yourself now: list(product('abc'))

# ? Nov 1, 2010 18:35

Stabby McDamage: Dec 11, 2005; Doctor Rope

Threep posted:

You're probably going to slap yourself now: list(product('abc'))

I knew it had to be something that simple. I should have thought of that.

# ? Nov 1, 2010 18:37

king_kilr: May 25, 2007

There's no __past__, or any other way to restore the print statement.

Chained comperators are quiet intentional, there aren't really any gotchas, unless you consider:

code:

a < foo() < b

foo is only called once there, not twice.

# ? Nov 1, 2010 19:14

Stabby McDamage: Dec 11, 2005; Doctor Rope

king_kilr posted:

There's no __past__, or any other way to restore the print statement.

Chained comperators are quiet intentional, there aren't really any gotchas, unless you consider:
code:
a < foo() < b
foo is only called once there, not twice.

Is there a document that explains this feature in greater depth? I'm curious what the precise algorithm for doing this is, since it violates my expectation that everything is always evaluated left-to-right.

# ? Nov 1, 2010 23:09

Lonely Wolf: Jan 20, 2003; Will hawk false idols for heaps and heaps of dough.

It is still evaluated left to right:

code:

def M(s):
     def f():
          print s
          return 0
     return f
one, two, three = [M(n) for n in "one two three".split()]

>> one() <= two() <= three()
one
two
three
>>

In a < b < c it evaluate a < b and returns false if a >= b but if it's true then it moves on and tries b < c, so it essentially expands it into a < b && b < c for you

# ? Nov 2, 2010 00:12

Rohaq: Aug 11, 2006

So I'm used to Perl, and being able to run my scripts under perl -d to throw it into a step-by-step debug mode where I can check and set values and such. Is there any way to do this in Python? IDLE is nice to test ideas out line by line, but running a script within it just seems to run python as normal, with no stepping through line/into functions etc, and I'm not even certain how to check/set values of variables.

Any clues?

# ? Nov 2, 2010 02:50

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Rohaq posted:

So I'm used to Perl, and being able to run my scripts under perl -d to throw it into a step-by-step debug mode where I can check and set values and such. Is there any way to do this in Python? IDLE is nice to test ideas out line by line, but running a script within it just seems to run python as normal, with no stepping through line/into functions etc, and I'm not even certain how to check/set values of variables.

Any clues?

http://docs.python.org/library/pdb.html

# ? Nov 2, 2010 03:06

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Huh, Sikuli looks pretty cool for automating GUI's with Python.

# ? Nov 2, 2010 04:24

Rohaq: Aug 11, 2006

BeefofAges posted:

http://docs.python.org/library/pdb.html

The -m pdb argument was exactly what I was looking for. I can't see much in setting conditional breakpoints though; I've only got breaking on lines or functions at the moment. Can you get pdb to break on a condition?

# ? Nov 2, 2010 13:47

Maccers: Mar 27, 2010

Could anyone tell me if there is a way to prevent python (2.6.5) appending an 'L' to long numbers?

For example I have a django view function that multitplies a unix timestamp by 1000 to give a javascript timestamp.

The page loads correctly via a webserver but python returns a 'long' number with an L on the end (for example 20991600000L) which breaks the javascript.

I havent been using python for long, so sorry if this is something obvious. I've had a quick read around the documentation, but cant really see an answer (other than maybe increase sys.maxint?).

Maccers fucked around with this message at 23:24 on Nov 2, 2010

# ? Nov 2, 2010 23:16

MaberMK: Feb 1, 2008; BFFs

Maccers posted:

Could anyone tell me if there is a way to prevent python (2.6.5) appending an 'L' to long numbers?

For example I have a django view function that multitplies a unix timestamp by 1000 to give a javascript timestamp.

The page loads correctly via a webserver but python returns a 'long' number with an L on the end (for example 20991600000L) which breaks the javascript.

I havent been using python for long, so sorry if this is something obvious. I've had a quick read around the documentation, but cant really see an answer (other than maybe increase sys.maxint?).

code:

>>> str(0L)
'0'

# ? Nov 2, 2010 23:34

Lurchington: Jan 2, 2003; Forums Dragoon

Maccers posted:

Could anyone tell me if there is a way to prevent python (2.6.5) appending an 'L' to long numbers?

For example I have a django view function that multitplies a unix timestamp by 1000 to give a javascript timestamp.

The page loads correctly via a webserver but python returns a 'long' number with an L on the end (for example 20991600000L) which breaks the javascript.

I havent been using python for long, so sorry if this is something obvious. I've had a quick read around the documentation, but cant really see an answer (other than maybe increase sys.maxint?).

code:

>>> 20991600000000000000
20991600000000000000L
>>> print '%d' % 20991600000000000000
20991600000000000000

I don't know if you were explicitly using string formatting, but I'd usually try that

# ? Nov 2, 2010 23:35

tripwire: Nov 19, 2004; _{ghost flow}

Rohaq posted:

The -m pdb argument was exactly what I was looking for. I can't see much in setting conditional breakpoints though; I've only got breaking on lines or functions at the moment. Can you get pdb to break on a condition?

Put this in your code:

code:

if condition:
    pdb.set_trace()

# ? Nov 3, 2010 03:01

tripwire: Nov 19, 2004; _{ghost flow}

Maccers posted:

Could anyone tell me if there is a way to prevent python (2.6.5) appending an 'L' to long numbers?

For example I have a django view function that multitplies a unix timestamp by 1000 to give a javascript timestamp.

The page loads correctly via a webserver but python returns a 'long' number with an L on the end (for example 20991600000L) which breaks the javascript.

I havent been using python for long, so sorry if this is something obvious. I've had a quick read around the documentation, but cant really see an answer (other than maybe increase sys.maxint?).

Your question has already been answered, but as a point of interest, that format you see is what is produced by the repr function. Repr is a builtin and also a method defined on all objects (invoking the builtin repr on an object will call the matching double underscore method defined on that class). Repr (which im guessing means reproduce) is mostly intended for representing objects in such a manner that they can be reconstructed, ideally by just throwing the result of repr into a call to eval. Compare this to the str function, which is used more for the purpose of representing objects in a compact and legible way; str doesn't care about making it reproduce the original object when you throw it into eval.

In python, integer values can either be of type int (which is really a c long) or long (which is arbitrary precision). To explicitly specify an integer as being a long in python code, you can suffix it with L. Accordingly, repr on longs will always have the L tacked on, because the expectation is that calling eval(repr(somelong)) will yield a value of the same type as "somelong".

I'm not sure how you are accessing this number from your application but if your data is ending up on a webpage then its a given that its getting converted to a string at some point. Its bizarre then that you see an L at the end of your number, since that would probably only result from calling repr on a long integer. Only I'm not sure why repr would be called on it if you didn't do it explicitly. As far as I'm aware, repr will get called if you surround an object in backticks (`) or if you are using python in interactive mode and just executing statements consisting of a variable name.

tripwire fucked around with this message at 03:37 on Nov 3, 2010

# ? Nov 3, 2010 03:33

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

tripwire posted:

I'm not sure how you are accessing this number from your application but if your data is ending up on a webpage then its a given that its getting converted to a string at some point. Its bizarre then that you see an L at the end of your number, since that would probably only result from calling repr on a long integer. Only I'm not sure why repr would be called on it if you didn't do it explicitly. As far as I'm aware, repr will get called if you surround an object in backticks (`) or if you are using python in interactive mode and just executing statements consisting of a variable name.

He might be using something like "a=%r; b=%r; c=%r" % (a, b, c) as a ghetto serializer; it would work for simple cases like ASCII strings or ints, but break on anything more complex (eg, longs).

Maccers: if you are using string formatting to build JavaScript variable definitions, stop that. Use JSON (via standard library module json).

# ? Nov 3, 2010 04:14

b0lt: Apr 29, 2005

tripwire posted:

Repr (which im guessing means reproduce) is mostly intended for representing objects in such a manner that they can be reconstructed

representation

# ? Nov 3, 2010 05:08

Maccers: Mar 27, 2010

Thanks for the assistance guys - got it working using a string rather than number type.

Janin: Yeah I was, I'll take a look at using JSON as a longterm solution.

# ? Nov 3, 2010 20:07

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Maccers posted:

Janin: Yeah I was, I'll take a look at using JSON as a longterm solution.

JSON is really not difficult to use:

code:

# python
import json
data = {"students": ["john", "jane", "bob"], "classes": ["CS 101", "CS 102"]}
print "<script type='application/javascript'>var data = %s;</script>" % json.dumps(data)

code:

// javascript
alert("first student = " + data.students[0]);

# ? Nov 3, 2010 20:54

Ularg: Mar 2, 2010; Just tell me I'm exotic.

I'm pretty brand spanking new to coding, and I'm trying to learn the logic of coding along with syntax of it (Python in particular). I was looking for videos for Python that could help me get a grasp on the language, but failing that I just started to read the documentation, word for word, going along and doing everything it does and I've gotten as far as "4. More Control Flow Tools". And I'm just wondering if this is the wrong way to try to learn it, because none of this seems to be sticking in my head longer than 5 minutes, and I have to go back and look to remember parts. I've just been trying to teach myself how to use it, and I'm not actually learning anything.

# ? Nov 4, 2010 02:35

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

Ularg posted:

I'm pretty brand spanking new to coding, and I'm trying to learn the logic of coding along with syntax of it (Python in particular). I was looking for videos for Python that could help me get a grasp on the language, but failing that I just started to read the documentation, word for word, going along and doing everything it does and I've gotten as far as "4. More Control Flow Tools". And I'm just wondering if this is the wrong way to try to learn it, because none of this seems to be sticking in my head longer than 5 minutes, and I have to go back and look to remember parts. I've just been trying to teach myself how to use it, and I'm not actually learning anything.

You will learn the syntax over time through repetition. I've been programming for a lot longer than you have and I still need to refer to the docs whenever I use some essoteric part of the syntax. The only benefit is that now I know what part of the docs to look at.

# ? Nov 4, 2010 02:44

Spime Wrangler: Feb 23, 2003; Because we can.

I'm hoping to use the matplotlib toolkit Basemap as a lightweight prototyping environment for some parts of an ArcGIS tool I'm putting together, but when I try

code:

from mpl_toolkits.basemap import Basemap

it insta-crashes python. Like, try/except does nothing, straight pythondeath.

I'm running XP64, Python 2.6.2. The python install is all 32bit.

"more information about this error" only gives:

code:

AppName: python.exe	 AppVer: 0.0.0.0	 ModName: unknown
ModVer: 0.0.0.0	 Offset: 00000000

In the error report I get:

code:

Module 1
python.exe
Image Base: 0x1d000000  Image Size: 0x00000000
Checksum: 0x00009c05    Time Stamp: 0x49e4f549

And the rest of the version information is all zeros.

I can't find any info online about anyone having the same issue. Any ideas?

# ? Nov 4, 2010 03:27

Spatial: Nov 15, 2007

Ularg posted:

I'm pretty brand spanking new to coding, and I'm trying to learn the logic of coding along with syntax of it (Python in particular). I was looking for videos for Python that could help me get a grasp on the language, but failing that I just started to read the documentation, word for word, going along and doing everything it does and I've gotten as far as "4. More Control Flow Tools". And I'm just wondering if this is the wrong way to try to learn it, because none of this seems to be sticking in my head longer than 5 minutes, and I have to go back and look to remember parts. I've just been trying to teach myself how to use it, and I'm not actually learning anything.

I think these MIT comp-sci lectures would be a good place to start. The class focuses on teaching the essentials of programming, using Python's interactive shell to demonstrate the concepts. Use the shell to play with the concepts as you go along, you'll learn a lot.

# ? Nov 4, 2010 05:58

Ularg: Mar 2, 2010; Just tell me I'm exotic.

Spatial posted:

I think these MIT comp-sci lectures would be a good place to start. The class focuses on teaching the essentials of programming, using Python's interactive shell to demonstrate the concepts. Use the shell to play with the concepts as you go along, you'll learn a lot.

This is a godsend, thank you.

# ? Nov 4, 2010 06:20

DirtyDiaperMask: Aug 11, 2003; by Ozmaugh

Spime Wrangler posted:

I can't find any info online about anyone having the same issue. Any ideas?

If Python is just crashing without even printing a stack trace, I'm guessing it might be an interpreter bug. Try with 2.6.6 maybe and see if there's the same issue?

# ? Nov 4, 2010 06:49

Spime Wrangler: Feb 23, 2003; Because we can.

Heh, guess I haven't updated for some time now. Its working fine at home under 2.6.6 so nvm I guess.

# ? Nov 4, 2010 07:35

Rohaq: Aug 11, 2006

tripwire posted:

Put this in your code:
code:
if condition:
    pdb.set_trace()

Is there no easier way to do this on the fly from within the Pdb console? In Perl, I could just type

code:

b $variableX > 6

and it would break when that condition was true. It seems daft to allow breakpoints of conditions if you need to actually put them into your code manually.

# ? Nov 4, 2010 13:29

Lamacq: Jun 15, 2001; Breezeblock RIP

Rohaq posted:

Is there no easier way to do this on the fly from within the Pdb console? In Perl, I could just type
code:
b $variableX > 6
and it would break when that condition was true. It seems daft to allow breakpoints of conditions if you need to actually put them into your code manually.

What tripwire is saying is that python will start an interactive debugging session when you get to that condition.

You can of course set conditional breakpoints inside pdb (or if you prefer, ipdb) using pretty much the same exact syntax as perl's debugger. pdb documentation lives here.

# ? Nov 4, 2010 15:15

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 02:25

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

This feels wrong to me:

code:

def parse_duration(value):
    t = re.search(r"((?P<hour>\d+) hour(s|))? ?((?P<min>\d+) min)? ?((?P<sec>\d+) sec)? ?((?P<ms>\d+) ms)?", value)
    if t:
        hour = 0 if not t.group('hour') else int(t.group('hour'))
        min = 0 if not t.group('min') else int(t.group('min'))
        sec = 0 if not t.group('sec') else int(t.group('sec'))
        ms = 0 if not t.group('ms') else int(t.group('ms'))
        return datetime.timedelta(hours = hour, minutes = min, seconds = sec, milliseconds = ms

Anyone have a better way to change strings that look like the following into timedeltas?

code:

2 hours 13 min 4 sec 312 ms
22 min 4 sec
1 hour 3 min 42 sec

The "ms" is not always present.
It can be "hours" or "hour"
If it doesn't reach hours or minutes then it will only contain up to minutes or seconds, respectively.

# ? Nov 4, 2010 23:52

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »