Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

tripwire: Nov 19, 2004; _{ghost flow}

I think I figured out the multiprocessing bit, although I had to move some methods around to global instead of instance.
The part I'm stuck on now before I get it working on windows is drawing triangles to an RGB image array, hopefully faster than cairo does it.

Since I'm doing most of the work with numpy arrays I figure it wouldn't be too much more to try and throw my own rasterizing code in there and compare it to cairo.

I know that if you assign to rectangular chunks of a 2d numpy array you end up saving a lot of time versus assigning to individual elements or using a lambda.

Lets say I wanted to make a function called draw_triangle which takes as parameters a source image to draw on, 3 point coordinates and 4 colour coordinates.
How would I go about doing this, if I'm looking to maximize performance?

I'm thinking there has to be some well researched algorithms that chop up a triangle into optimal rectangular chunks for fast assignment, can anyone help me out? Is there a library which does this already (in windows as well as linux?)?

# ? Jan 13, 2009 21:48

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 17:53

Not an Anthem: Apr 28, 2003; I'm a fucking pain machine and if you even touch my fucking car I WILL FUCKING DESTROY YOU.

I'm taking a mathematics/computer science class that is basically a rushed introduction to python. The teacher spent all of lecture and lab trying to teach the class a crash course in unix environments all the while showing how his linux laptop is superior in performance to people's random xp installs.

.. I just want to learn python dude :cry:

Before anyone replies I'm totally comfortable with your average idiot "navigate around, sftp your homework to the teacher, use vim/pico/etc" he's teaching, its just a waste of time I think.

# ? Jan 14, 2009 02:02

Lonely Wolf: Jan 20, 2003; Will hawk false idols for heaps and heaps of dough.

Are you asking how to bang him? . . . If not, do you have a question?

# ? Jan 14, 2009 02:12

Jo: Jan 24, 2005; Soiled Meat

Not an Anthem posted:

1) Type your teacher's name.
2) Hold shift.
3) Press the one key.
4) :downsrim:

# ? Jan 14, 2009 02:27

scanlonman: Feb 7, 2008; by R. Guyovich

code:

import os
modules={}
class Module():
	file=""
	vars={}
	lines=[]
	comments=[]
	type="?"
	def __init__(self,section,file_name):
		self.file=os.path.join(os.getcwd(),"data",section,file_name)
		load_file=open(self.file,"r")
		self.lines=load_file.readlines()
		load_file.close()
		new_lines=[]
		for a in self.lines:
			a=a.replace("\n","")
			if a[0]=="!":
				comments.append(a)
				continue
			new_lines.append(a)
		self.lines=new_lines
		self.type=self.lines[0]
		for a in self.lines:
			j=a.find("=")
			if j==-1:continue
			left=a[:j]
			right=a[j+1:]
			self.vars[left]=right
		i=self.vars["name"]
		modules[self]=self.vars
	def setVars(self,atom):
		for a in self.vars:
			setattr(atom,a,self.vars[a])
Module("tile","tile")
Module("tile","wall")
for a in modules:
	print a.vars["name"]
	for b in modules[a]:
		print b

and I have files in /data/tile which look like:

tile:

code:

TILE
name=tile
density=0
char=.

wall:

code:

TILE
name=wall
density=1
char=#

yet this loop:

code:

for a in modules:
	print a.vars["name"]
	for b in modules[a]:
		print b

outputs that each item in the modules dictionary is a wall when it loads up a tile and a wall, not just two walls. Any help? I really can't continue my project without getting this to work. :smithicide:

# ? Jan 15, 2009 02:30

Lonely Wolf: Jan 20, 2003; Will hawk false idols for heaps and heaps of dough.

that's because type, et al, are class variables not instance variables. Also, help(str.split).

e: et all? Am I retarded?

Lonely Wolf fucked around with this message at 02:47 on Jan 15, 2009

# ? Jan 15, 2009 02:34

scanlonman: Feb 7, 2008; by R. Guyovich

Lord Uffenham posted:

that's because type et all are class variables not instance variables

How would I fix that? Sorry, this is my second (third?) day actually using/learning Python so I don't know that much.

Lord Uffenham posted:

help(str.split)

Aha! I knew there had to be something simpler for what I was doing. Thanks.

# ? Jan 15, 2009 02:42

Lonely Wolf: Jan 20, 2003; Will hawk false idols for heaps and heaps of dough.

You need to set them in the body of init not the class body.

# ? Jan 15, 2009 02:47

scanlonman: Feb 7, 2008; by R. Guyovich

Lord Uffenham posted:

You need to set them in the body of init not the class body.

Aha! That got it to work. Thank you so much.

[edit]

How do I get a subclass to also call the parent's init?

code:

class Atom:
	def __init__(self):
		self.name="atom"
		self.style=0
		self.x=0
		self.y=0
		self.contents=[]
		self.density=0
		self.char=""
		self.desc="Nothing interesting"
class Tile(Atom):
	def __init__(self):
		self.name="tile"
		self.char="."

The Tile class doesn't inherit the name, style, x, y, etc. of the Atom class. Is there something I can do so it gets from that too?

scanlonman fucked around with this message at 02:56 on Jan 15, 2009

# ? Jan 15, 2009 02:51

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

scanlonman posted:

The Tile class doesn't inherit the name, style, x, y, etc. of the Atom class. Is there something I can do so it gets from that too?

Yes, you'll want to use super():

code:

class Tile(Atom):
	def __init__(self):
		super(Tile, self).__init__ ()
		self.name="tile"
		self.char="."

TOO SCSI FOR MY CAT fucked around with this message at 04:56 on Jan 15, 2009

# ? Jan 15, 2009 03:30

No Safe Word: Feb 26, 2005

scanlonman posted:

Aha! That got it to work. Thank you so much.

[edit]

How do I get a subclass to also call the parent's init?

super(parent_class, self).__init__()

# ? Jan 15, 2009 03:35

tripwire: Nov 19, 2004; _{ghost flow}

If anyone wants to fool around with that picture triangle thing I was struggling with earlier I have an exe packaged for windows that seems to work on my computer just fine. The code is still a mess but it was a learning experience at least. You can grab it here if you want. All the parameters are in a config file called 'configulation', edit that and away you go. I'm going to try rewriting it now so that opengl does most of the heavy lifting.

# ? Jan 15, 2009 16:47

scanlonman: Feb 7, 2008; by R. Guyovich

I get an error using super, I don't know why.

code:

class Tile(Atom):
	def __init__(self):
		super(Atom,self).__init__()    #this line
		self.name="tile"
		self.char=".

File "roguelike.py", line 68, in __init__
    super(Atom,self).__init__()
TypeError: super() arguement 1 must be type, not classobj

Changing it to super(Atom(),self).__init__() didn't work either. Does anyone know the issue?

# ? Jan 15, 2009 22:43

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

scanlonman posted:

I get an error using super, I don't know why.

Changing it to super(Atom(),self).__init__() didn't work either. Does anyone know the issue?

You've encountered one of the annoying gotchas in Python 2, the separation between old- and new-style classes. You must have your classes inherit from "object", or many features will break. Try code like this:

code:

class Atom(object):
	def __init__(self):
		super(Atom, self).__init__()
		...

class Tile(Atom):
	def __init__(self):
		super(Tile, self).__init__() # <-- NOTE: NOT "Atom"
		...

# ? Jan 15, 2009 22:55

scanlonman: Feb 7, 2008; by R. Guyovich

Ahhh, thanks! My game is running so much better it's fun to see how fast things are going and how well it's working. I'm really liking Python.

# ? Jan 16, 2009 01:34

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

thou shalt not prematurely optimize: http://bugs.python.org/issue5000

# ? Jan 19, 2009 17:59

king_kilr: May 25, 2007

m0nk3yz posted:

thou shalt not prematurely optimize: http://bugs.python.org/issue5000

Heh, I just had some fun optimizing something myself: http://lazypython.blogspot.com/2009/01/optimizing-view.html

# ? Jan 19, 2009 22:43

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Heh, I just had some fun optimizing something myself: http://lazypython.blogspot.com/2009/01/optimizing-view.html

Heh, I closed like, 4 or 5 multiprocessing bugs this weekend - then I searched and found like 7 more unassigned :\

# ? Jan 19, 2009 23:42

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I'm trying to write some function to upload files via FTP. My "problem" is that it feels like I'm kludging my status updates with percent done, KB's uploaded, and KB per second. The reason I've done it as I've done it is that I can't think of a better way to pass data back and forth between each time the callback is called.

In short, I hate screwing with global variables like this, is there a better way?

code:

import os
import ftplib
import time

def upload(ftp, f):
    #initialize some stuff since callbacks can't have arguments
    global blocksize
    blocksize = 8192
    global data_tran
    data_tran = 0
    global last_KBs_update
    last_KBs_update = 0
    global last_sec_tran
    last_sec_tran = 0
    
    ext = os.path.splitext(f)[1]

    if ext in (".txt", ".htm", ".html"):
        ftp.storlines("STOR " + f, open(f))
    else:
        file_start_time =time.time()
        ftp.storbinary("STOR " + f, open(f, "rb"), blocksize, print_callback)
        file_end_time = time.time()
        print

def print_callback(block):
    #variables that need to be persistent between each callback
    global data_tran
    global last_KBs_update
    global last_sec_tran
    global KBs
    
    data_tran += len(block) #Total data transferred
    KB_tran = data_tran / 1024 #In KB
    KB_filesize = filesize / 1024 #Filesize in KB
    per_done = (float(data_tran) / float(filesize)) * 100 #Percent transferred
    last_sec_tran += len(block)/1024 #Data transferred since updating KB/s

    if time.time() - last_KBs_update > 1: #Update KB/s every ~1 sec
        elap_time = time.time() - last_KBs_update #Exact time since update
        KBs = last_sec_tran/elap_time 
        last_KBs_update = time.time()
        last_sec_tran = 0
    
    print "%.2f%%  (%iKB of %iKB) (%.2f KB/s)" % (per_done, KB_tran,
                                                KB_filesize, KBs), '\r', 

filesize = os.path.getsize(YOUR_FILE)
f = ftplib.FTP('')
f.connect(IP, PORT)
f.login(USERNAME, PASSWORD)
f.cwd(FTP_DIR)
upload(f, YOURFILE)

# ? Jan 20, 2009 19:21

tripwire: Nov 19, 2004; _{ghost flow}

I am trying to refactor the triangle approximator I posted previously in this thread. It was using cairo to draw alpha blended triangles onto a canvas, and using numpy to read this canvas as an array and calculate the root mean square error between this array and an array made from the source image.

When I profiled the program, equal amounts of time were spent by cairo's fill method and by numpy's concatenate method (which I assume is called several times throughout the course of my RMSE scoring function.)

Because the source images are 8-bit unsigned integer arrays they must be upscaled to 64-bit float arrays before they can be stacked, diff'd, squared and summed up, so numpy is doing a shitload of unneccesary work which adds up quickly. Cairo is also profoundly slow, taking a significant amount of time to draw what amounts to 100 triangles on a 640x480 canvas. Both of these things are responsible for hogging all the CPU time and making this program slow as poo poo, so I want to fix them.

Rather than using cairo for drawing and numpy for calculating the root mean square error between two image buffers, my plan is to use the pyOpenGl bindings to do the drawing on hardware, and something else(?) to calculate the root mean square error, ditching numpy entirely if possible.

I've never used openGL before, but looking at PyOpenGL's function reference, it appears that it supports drawing to frame buffer objects and compiling shaders.

Can anyone give me any pointers as to what I have to do if I want to duplicate the functionality of my scoring function on the GPU?
Here it is:

code:

def rmse(first, second):
    """ Returns the root mean square error between two image arrays"""
    assert numpy.size(first) == numpy.size(second)
    difference = numpy.diff(numpy.dstack(
        (first.astype(numpy.float64),
        second.astype(numpy.float64))))
    return 1 - math.sqrt(numpy.mean(numpy.power(difference,2))) / 255.0

Do I want to use GLSL or do I have to write CUDA code and link it somehow?

# ? Jan 20, 2009 23:44

dorkface: Dec 27, 2005

I have a simple question about Parent/Child communication in wxpython in this paste:

http://paste.pocoo.org/show/100865/

Also, related to the paste, I've been told that I shouldn't create another class under the function for the button to create the child frame. So, in other words, I should just create the class outside of the function and just using the function to instanciate the class?

dorkface fucked around with this message at 05:14 on Jan 23, 2009

# ? Jan 23, 2009 05:11

ekalb: Jun 5, 2004

Anyone familiar with PAMIE (http://pamie.sourceforge.net/)? I'm running PAMIE 2.0 on Python 2.5 and having problems accessing a nested frame (which it says it supports in the module as frame1.frame2.frame3). The frames on the page are laid out like this:

|-name="nav"
|
|-name="content"
|---name="DirectoryContents"
|---name="DirectoryStructure"

so I've got something similar to this (already assuming I've navigated to the website with the frame layout above):

code:

import cPAMIE
ie = cPAMIE.PAMIE()
ie.frameName = "content.DirectoryContents"
print "page text: " + ie.pageGetText()

Obviously frameName is not getting set properly as it's saying the frame doesn't exist:

code:

File "C:\Python25\lib\site-packages\cPAMIE.py", line 1467, in pageGetText
    self._ie.Document.frames[self.frameName].document.body.outerHTML).

ekalb fucked around with this message at 06:59 on Jan 24, 2009

# ? Jan 24, 2009 05:43

Chin Strap: Nov 24, 2002; I failed my TFLC Toxx, but I no longer need a double chin strap; Pillbug

So I'm still pretty new to python, but this confuses me. If i do something like:

code:

class StrNew(str): pass

a = str()

type(StrNew(a))

I get it is a class '__main__.NewStr', as I would hope. If I do

code:

import numpy

class MatNew(numpy.matrix): pass

amat = numpy.matrix([[1,2],[2,1]])

type(MatNew(amat))

I get it is class 'numpy.core.defmatrix.matrix', which is the same class as amat was originally. I'm not necessarily asking a numpy question specifically, more like a general
one: why doesn't the class of amat get changed? Is there something simple I am missing?

# ? Jan 24, 2009 17:09

Allie: Jan 17, 2004

numpy.matrix has a __new__ method that customizes how instances are actually created. It looks like it checks the argument that's passed to it and if it's a matrix (or a subclass), it instantiates another matrix and returns that instead.

All that aside, what are you actually trying to do? If you're just trying to instantiate your subclass, do it directly:

code:

>>> import numpy
>>> class MatNew(numpy.matrix): pass
... 
>>> type(MatNew([[1, 2], [2, 1]]))
<class '__main__.MatNew'>

If you're trying to create MatNews from existing matrix objects, then you may have to override __new__. I don't know that that's a good idea, and I'm not too familiar with numpy, so you'll have to do more research there.

# ? Jan 24, 2009 20:03

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

Anyone worked with validating XML parsers, or any form of XML validation? Python's always been weak in this area and most of the (half-hearted) attempts have faded and gone away over the last few years.

# ? Jan 27, 2009 16:26

tef: May 30, 2004; -> some l-system crap ->

lxml does validation

# ? Jan 27, 2009 17:47

Kire: Aug 25, 2006

I was inspired by the vocab-flashcard script posted earlier, and I want to make an English : German, German : English flashcard script. I can't decide on the best way to store the vocab words that I add. I was thinking about 3 dicts, one for nouns, verbs, and adjectives, but if I stored the words as {English:German} then I wouldn't be able to do a key lookup by value (German : English), even though the values would be as unique as the keys.

So is a list of 2-tuples the better way to go, to preserve reversibility?

# ? Jan 27, 2009 17:54

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Kire posted:

I was inspired by the vocab-flashcard script posted earlier, and I want to make an English : German, German : English flashcard script. I can't decide on the best way to store the vocab words that I add. I was thinking about 3 dicts, one for nouns, verbs, and adjectives, but if I stored the words as {English:German} then I wouldn't be able to do a key lookup by value (German : English), even though the values would be as unique as the keys.

So is a list of 2-tuples the better way to go, to preserve reversibility?

No. Use one dict for English-German, and another for German-English. Whether you want to use multiple pairs of dicts for different pairs of speech is up to you.

# ? Jan 27, 2009 20:43

Sock on a Fish: Jul 17, 2004; What if that thing I said?

I'm using the sqlite3.Row row_factory and I've found that if I do a SQL join and have identically named columns in the join, the resulting row objects look like a dict with two of the same key. The value returned for row['key'] is the one for the first instance of the key.

Am I missing something obvious? This seems like a pretty big oversight. I've formatted my query to request everything as table.column, I would've figured that the result would return the keys in table.column format.

# ? Jan 28, 2009 20:27

Kire: Aug 25, 2006

I'm making progress, thanks for the tip.

Python is much easier to study when I have specific goal in mind that I want to work toward, instead of undirected "I just wanna learn about programming..."

I'm having trouble incorporating � � � � � � �. IDLE prompted me to put this line at the top of my file when I tried to save:

code:

 # -*- coding: cp1252 -*-

but when I try to call a flashcard of a word with an umlaut, I get

code:

{'House': 'H\xe4user'}

{'H\xe4user': 'House'}

Instead of H�user.

Kire fucked around with this message at 21:45 on Jan 28, 2009

# ? Jan 28, 2009 20:41

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Try # -*- coding: utf8 -*- , then make sure IDLE is saving in UTF-8.

# ? Jan 28, 2009 22:46

supster: Sep 26, 2003; I'M TOO FUCKING STUPID
TO READ A SIMPLE GRAPH

I'm sorry for asking probably previously-answered questions that no one likes to reanswer again... but:

(1) If I'm starting with Python, should I jump into 3.0 or 2.6? How common is it to not be able to find a 3.0 library for a common task that exists for 2.6?

(2) What development environments are recommended? Are different IDEs generally used for console applications vs windows applications vs django?

# ? Jan 28, 2009 23:26

bitprophet: Jul 22, 2004; Taco Defender

supster posted:

I'm sorry for asking probably previously-answered questions that no one likes to reanswer again... but:

(1) If I'm starting with Python, should I jump into 3.0 or 2.6? How common is it to not be able to find a 3.0 library for a common task that exists for 2.6?

2.6. 3.0 is extremely new and almost nothing has been ported to it yet. 2.6 will include some of the newer libraries/ideas present in 3.0, so you'll be better off for learning 3.0 than someone who's used to older verisons of Python. 3.0 will not be mainstream for at least a year or two (someone like m0nk3yz or king_kilr will have a better estimate, probably).

quote:

(2) What development environments are recommended? Are different IDEs generally used for console applications vs windows applications vs django?

This is totally up to you; many Pythonistas don't even use an IDE in the literal sense of the term (a big all inclusive whizbang thing with a built in browser and terminal and etc). Definitely don't think you'd need different editors for different "types" of Python coding, at any rate. Pick one and use that. I love vim (others emacs); others love TextMate (and it is quite good); some like IDLE or Eclipse-plus-some-Python-plugin; etc.

# ? Jan 29, 2009 02:45

wrok: Mar 24, 2006; by angerbotSD

supster posted:

I'm sorry for asking probably previously-answered questions that no one likes to reanswer again... but:

(1) If I'm starting with Python, should I jump into 3.0 or 2.6? How common is it to not be able to find a 3.0 library for a common task that exists for 2.6?

(2) What development environments are recommended? Are different IDEs generally used for console applications vs windows applications vs django?

1) 2.6 for all the reasons bitprophet said.

2) I really like Eclipse and PyDev. Outside of that I just use IDLE/vim/textpad/notepad. (For the record, I loathe Eclipse for C/C++/Java -- for some reason I just love it for Python, go figure.)

# ? Jan 29, 2009 03:01

king_kilr: May 25, 2007

bitprophet posted:

2.6. 3.0 is extremely new and almost nothing has been ported to it yet. 2.6 will include some of the newer libraries/ideas present in 3.0, so you'll be better off for learning 3.0 than someone who's used to older verisons of Python. 3.0 will not be mainstream for at least a year or two (someone like m0nk3yz or king_kilr will have a better estimate, probably).

I can't really speak intelligently for other libraries but Django's current trajectory is to start having a 3.0 branch in about 1.5-2 years. That being said Martin Loewis has a patch available that combined with 2to3 brings Django to a usable state on 3.0, so if you had a pressing need(and it doesn't sound like you do) you could probably make it work.

# ? Jan 29, 2009 04:19

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

bitprophet posted:

2.6. 3.0 is extremely new and almost nothing has been ported to it yet. 2.6 will include some of the newer libraries/ideas present in 3.0, so you'll be better off for learning 3.0 than someone who's used to older verisons of Python. 3.0 will not be mainstream for at least a year or two (someone like m0nk3yz or king_kilr will have a better estimate, probably)..

If you want cleanliness and purity; go with 3.0 (note that IO speeds suck) - if you want third party code, libraries and frameworks, go with 2.6.

Ultimately, to have strong python-fu, you'll need to know both.

# ? Jan 29, 2009 04:38

king_kilr: May 25, 2007

m0nk3yz posted:

If you want cleanliness and purity; go with 3.0 (note that IO speeds suck) - if you want third party code, libraries and frameworks, go with 2.6.

Ultimately, to have strong python-fu, you'll need to know both.

To add to this, the hope is to have 3.1 or a beta out around PyCon, this continues a lot of the cleanup that defined 3.0(plus io speedups and another 20% speed increase).

# ? Jan 29, 2009 04:59

tef: May 30, 2004; -> some l-system crap ->

supster posted:

I'm sorry for asking probably previously-answered questions that no one likes to reanswer again... but:

I think we need to put this in the first post.

quote:

(1) If I'm starting with Python, should I jump into 3.0 or 2.6? How common is it to not be able to find a 3.0 library for a common task that exists for 2.6?

Python 2.6 and Python 3.0 are not really that different, and learning one is not significantly harder than learning both. You can install both of them, too.

Still, it would be best to learn from 2.6 - there will be more documentation and libraries availiable.

Even so it would be best to do with an aim towards python 3 - avoiding deprecated features (use the -3 warning flag)

Python 3 is mainly a release to break backwards compatibility, many changes have made it into 2.6 also.

Python 3.1 will be out soon with the io library re-written in c

# ? Jan 29, 2009 10:12

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Kire posted:

I'm making progress, thanks for the tip.

Python is much easier to study when I have specific goal in mind that I want to work toward, instead of undirected "I just wanna learn about programming..."

I'm having trouble incorporating � � � � � � �. IDLE prompted me to put this line at the top of my file when I tried to save:
code:
 # -*- coding: cp1252 -*- 
but when I try to call a flashcard of a word with an umlaut, I get
code:
{'House': 'H\xe4user'}

{'H\xe4user': 'House'}
Instead of H�user.

When you print a dict, it prints the repr() of all of the keys and values within the dict. The repr() of a string in python 2.x will always escape non-printable and/or non-ASCII characters. You just need to print the string itself instead of the dict.

Also, you should be using unicode strings. Prefixing each string literal with a u makes a string literal into a unicode literal. So, you would write u'H�user'. This way, it'll work consistently across different terminals when you print the string. Of note: IDLE sucks at unicode. You might have to run your scripts in cmd to get the correct output.

# ? Jan 29, 2009 14:21

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 17:53

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

I am gonna ask the boss for PyCon, and I am pretty sure I am going to go regardless of job paying for it, at the very least for the conference.

That being said, this is discouraging.

m0nk3yz do you think it'll be better this year?

# ? Jan 29, 2009 16:29

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »