Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

tripwire: Nov 19, 2004; _{ghost flow}

unixbeard posted:

sometimes i use a global variable for a database connection

yet to suffer any material downsides to it

To clarify what I said originally, I don't even think theres anything at all wrong with that.

Theres a difference in python between using global variables like in your db connection example, and use of the global keyword. The use case for the second one is when you not only want to access a variable from outside the local scope (which happens automatically when you try to access a variable which doesn't happen to already be declared in local scope), but for when you want to change it and have those changes visible to things outside of the scope of what you're dealing with.

There are of course a few valid reasons I can think of for why you may want to use the global keyword, just like there are plenty of circumstances under which its obviously sensible to use a GOTO statement. However, a lot of people come to depend on it for sloppy hacks, and as program complexity grows over time the result can get tangled up and incomprehensible.

I think that sometimes this is just a lazy habit people get into, but relying on that style out of convenience or familiarity kind of curdles your brain and makes your code very unpleasant for others to follow.

# ? Jun 22, 2010 23:49

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 07:29

unixbeard: Dec 29, 2004

i agree you should minimize their use where feasible, and it can be a sign of sloppiness. But sometimes it's just not realistic to go through and refactor everything. For instance I have a large program that was reading its inputs from textfiles, then it grew and I needed it to read from a database as well. Yes in an ideal world it should've been refactored, but that would take a few weeks vs a few days to get it to support databases. It can be hard to justify that extra time if someone is paying and they needed it done yesterday.

It's a feature of the language, and people should be aware of arguments surrounding them and use them responsibly. Another time I've used them is for the mutex on an output stream in a threaded program.

I also hate when functions are passing around arguments they dont use, ie. if a() calls b(), which calls c() which calls d() and d() needs the database connection, you need to pass it all the way down even though a, b, and c aren't using it. You could argue that if it gets to that stage it's time to redesign, but again you need to look at what that will really get you vs the time spent on it.

I am also disappointed at my inability to get the phrase "willy nilly" into this post.

# ? Jun 23, 2010 10:33

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

This is the sort of thing that I always do in 15 lines and someone shows me how to do in 2...

Say I have a list of strings that are sorted the way I want them and that this isn't alphabetical...completely arbitrary sort order. Now I'm building another list of strings and I want to use the first list as the specification to sort the second list.

Bonus points for sorting strings in the second list that aren't in the first list to the end.

code:

master_string = ['this is a string', 'what up', 'dicks', 'bar', 'foo']
unsorted_string = ['dicks', 'what up', 'this is a string', 'foo', 'bar']

bonus = ['dicks', 'what up', 'word to your mother', 'this is a string', 'foo', 'bar', 'emo cutter']

code:

>>> magical_sort(unsorted_string, master_string)
['dicks', 'what up', 'this is a string', 'foo', 'bar']

>>> magical_sort(bonus, master_string)
['dicks', 'what up', 'this is a string', 'foo', 'bar', 'word to your mother', 'emo cutter']

# ? Jun 23, 2010 20:48

tripwire: Nov 19, 2004; _{ghost flow}

I was completely unaware you could do this in an interactive session until now (requires a unixy platform where python was built with readline)

code:


import readline
import rlcompleter
readline.parse_and_bind('tab: complete')

# ? Jun 23, 2010 21:06

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

For the first part,

code:

sorted(unsorted_string, key=master_string.index)

As for the bonus question, this works but there's probably a better way. You could probably do it with a cmp function passed to sorted().

code:

def magical_sort(items, order):
	out = []
	for i in order:
		try:
			out.append(items.pop(items.index(i)))
			if len(items) == 0: return out
		except ValueError: pass
	return out + items

the len(items) check could probably be dropped if efficiency isn't a concern.

Edit:

code:

def magic_key(l):
	master_string = ['this is a string', 'what up', 'dicks', 'bar', 'foo']
	try:
		return master_string.index(l)
	except ValueError:
		return len(master_string)

---

>>> sorted(bonus, key=magic_key)
['this is a string', 'what up', 'dicks', 'bar', 'foo', 'word to your mother', 'emo cutter']

This might work in Python 2.6 which I don't have access to right here:

code:

magic_key = lambda x: next((i for i in xrange(len(master_string)) if master_string[i] == x), len(master_string))

Carthag Tuek fucked around with this message at 22:06 on Jun 23, 2010

# ? Jun 23, 2010 21:19

nbv4: Aug 21, 2002; by Duchess Gummybuns

tripwire posted:

I was completely unaware you could do this in an interactive session until now (requires a unixy platform where python was built with readline)
code:
import readline
import rlcompleter
readline.parse_and_bind('tab: complete')

or just use ipython

# ? Jun 24, 2010 03:32

Fayk: Aug 2, 2006; Sorry, my brain doesn't work so good...

nbv4 posted:

or just use ipython

ipython seems to replicate some of my favorite features of irb, actually, but I'm kind of creeped out by the strange frankensteinian hybrid that it seems to be. Having 'rm', 'rmdir', and such alongside python is just...weird to me.

# ? Jun 24, 2010 06:15

king_kilr: May 25, 2007

Fayk posted:

ipython seems to replicate some of my favorite features of irb, actually, but I'm kind of creeped out by the strange frankensteinian hybrid that it seems to be. Having 'rm', 'rmdir', and such alongside python is just...weird to me.

Yeah, my first instinct to that was :confused:

, but then I realized it tab completed directories :3:

# ? Jun 24, 2010 14:42

ATLbeer: Sep 26, 2004; Über nerd

Has anyone every used pysage?
http://code.google.com/p/pysage/

# ? Jun 24, 2010 16:08

Fayk: Aug 2, 2006; Sorry, my brain doesn't work so good...

king_kilr posted:

Yeah, my first instinct to that was , but then I realized it tab completed directories

I think I'd be more comfortable if I understood the metaphors and methods used - like some commands don't work - it seems all special-cased, but I can't see how and where. I should look at it more, though...

# ? Jun 24, 2010 22:55

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Fayk posted:

I think I'd be more comfortable if I understood the metaphors and methods used - like some commands don't work - it seems all special-cased, but I can't see how and where. I should look at it more, though...

Yeah, this is why I installed ipython and then only used it once or twice.

# ? Jun 24, 2010 22:56

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Autocomplete and "?" for help is enough to make ipython awesome for me, I don't see why you'd want to switch back to the normal shell.

# ? Jun 24, 2010 23:31

devilmouse: Mar 26, 2004; It's just like real life.

Nevermind ipython's cross-session command history (in which you can actually type the first few letters of a command and then hit up to see previous commands with the same first letters), deep reloading, input and output caches, and the pdb integration! I get angsty without it.

# ? Jun 24, 2010 23:54

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

How the heck do you use virtualenv with ipython?

# ? Jun 25, 2010 00:23

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

devilmouse posted:

Nevermind ipython's cross-session command history (in which you can actually type the first few letters of a command and then hit up to see previous commands with the same first letters), deep reloading, input and output caches, and the pdb integration! I get angsty without it.

Oh yeah, the completion is amazing. But wait, deep reloading? How do you do that?

# ? Jun 25, 2010 00:29

ATLbeer: Sep 26, 2004; Über nerd

Thermopyle posted:

How the heck do you use virtualenv with ipython?

souce bin/activate
easy_install ipython

# ? Jun 25, 2010 00:42

king_kilr: May 25, 2007

Jonnty posted:

Autocomplete and "?" for help is enough to make ipython awesome for me, I don't see why you'd want to switch back to the normal shell.

?? for source code is better.

As for ipython in a virtualenv, I install it fresh into every venv (actually I have a postcreate hook with virtualenvwrapper that does it for me).

Also, wtf is easy_install, that's like a lovely dead pip right?

# ? Jun 25, 2010 00:49

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

king_kilr posted:

?? for source code is better.

Whoa. I never knew it did this, I must have only ever tried in on builtins where it doesn't work. Thanks, this'll save me a lot of effort!

# ? Jun 25, 2010 00:54

devilmouse: Mar 26, 2004; It's just like real life.

Jonnty posted:

Oh yeah, the completion is amazing. But wait, deep reloading? How do you do that?

dreload(<modulename>)

# ? Jun 25, 2010 01:21

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

devilmouse posted:

dreload(<modulename>)

Sweet, thanks. Any more magical features I'm missing?!

e: pretty sure I asked about deep reloading in ipython in #python once and they scoffed and told me that I shouldn't need it. twats.

# ? Jun 25, 2010 01:46

devilmouse: Mar 26, 2004; It's just like real life.

Jonnty posted:

Sweet, thanks. Any more magical features I'm missing?!

Take a look at this http://ipython.scipy.org/doc/manual/html/interactive/tutorial.html - It looks like it covers the high points.

# ? Jun 25, 2010 03:33

Stabby McDamage: Dec 11, 2005; Doctor Rope

I had to transpose a list of lists (AKA a 2D array) or python objects, and a bit of googling turned up this function, which works perfectly:

code:

def transposed(lists):
   if not lists: return []
   return map(lambda *row: list(row), *lists)

However, I don't fully understand what's going on here. I've used * in parameter lists before, but how is it being used in this context?

# ? Jun 25, 2010 20:09

Haystack: Jan 23, 2005

Stabby McDamage posted:

I've used * in parameter lists before, but how is it being used in this context?

It unpacks the list or tuple into positional arguments. See http://docs.python.org/tutorial/controlflow.html#tut-unpacking-arguments

# ? Jun 25, 2010 20:23

Stabby McDamage: Dec 11, 2005; Doctor Rope

Haystack posted:

It unpacks the list or tuple into positional arguments. See http://docs.python.org/tutorial/controlflow.html#tut-unpacking-arguments

Okay, that got me started on the right direction. The big thing to me is some magic in map() that I was previously unaware of:

http://docs.python.org/library/functions.html posted:

map(function, iterable, ...)�

Apply function to every item of iterable and return a list of the results. If additional iterable arguments are passed, function must take that many arguments and is applied to the items from all iterables in parallel. ...

So we unpack each list of lists, and the lambda walks each element of each list one at a time, packing them into the lambda's row. So the two nested loops are both hidden inside the call to map. That's awesome.

# ? Jun 25, 2010 20:54

qntm: Jun 17, 2009

Is there an easy standard way to do without the "do ... while" syntax in Python?

This is what I have:

code:

# randomly generate strings until an acceptable one appears
str = getRandomString()
while notAcceptable(str):
    str = getRandomString()
return(str)

What I'd like to do is obviously:

code:

do:
    str = getRandomString()
while notAcceptable(str)
return(str)

I can see that this doesn't mesh well with Python's indentation-as-block-delimiter philosophy but what I can't see is how I'm supposed to do without it. Ideally there would be only one getRandomString() call.

# ? Jun 25, 2010 23:06

hlfrk414: Dec 31, 2008

code:

while True:
    randstr = getRandomString()
    if notAcceptable(randstr):
        break
return randstr

# ? Jun 25, 2010 23:24

b0lt: Apr 29, 2005

hlfrk414 posted:

code:

while True:
    randstr = getRandomString()
    if notAcceptable(randstr):
        break
return randstr

# ? Jun 26, 2010 01:39

qntm: Jun 17, 2009

hlfrk414 posted:

code:

while True:
    str = getRandomString()
    if acceptable(str):
        break
return str

I guess this would work. Cheers

# ? Jun 26, 2010 11:26

tef: May 30, 2004; -> some l-system crap ->

nbv4 posted:

or just use ipython

oh my why did I never get around to this

# ? Jun 26, 2010 12:53

tef: May 30, 2004; -> some l-system crap ->

aside, I was looking up descriptors today and decided to make this horror

code:


class method(object):
    def __getattr__(self, name):
        class _method(object):
            def __init__(self, f):
                self.f = f

            def __get__(self, obj, type_=None):
                def wrapper(*args, **_kwargs):
                        kwargs = {}
                        kwargs.update(kwargs)
                        kwargs[name] = obj
                        return self.f(*args, **kwargs)
                return wrapper

        return _method

    def __call__(self, idx):
        class _method(object):
            def __init__(self, f):
                self.f = f

            def __get__(self, obj, type_=None):
                def wrapper(*args, **kwargs):
                        args = args[:idx] +  (obj,) + args[idx:]
                        return self.f(*args, **kwargs)
                return wrapper

        return _method

method = method()

so I can do things like this

code:

class Foo(object):
    def __init__(self, x,y):
        self.x = x
        self.y = y

    @method.self
    def foobar(foo,bar,self):
        print self.x, self.y
        print foo, bar

    @method(1)
    def barfoo(foo,self,bar):
        print self.x, self.y
        print foo, bar


f = Foo("x","y")

f.foobar("foo","bar")
f.barfoo("foo","bar")

I have no idea why I would ever want to do this

# ? Jun 26, 2010 14:50

polyfractal: Dec 20, 2004; Unwind my riddle.

Forgive me, I'm new to Python. I want to do this:

code:

while data = f.read(2):
    <stuff>

However Python does not seem to like that. Not a big deal, but I was curious if there is a more elegant method than the following:

code:

data = f.read(2)
while data:
    <stuff>
    data = f.read(2)

# ? Jun 26, 2010 18:14

king_kilr: May 25, 2007

code:

for data in iter(lambda: f.read(2), ""):

# ? Jun 26, 2010 18:25

polyfractal: Dec 20, 2004; Unwind my riddle.

king_kilr posted:

code:

for data in iter(lambda: f.read(2), ""):

Nifty, thanks!

# ? Jun 26, 2010 18:39

Chris Awful: Oct 2, 2005; Tell your friends they don't have to be scared or hungry anymore comrades.

I can't figure out how to solve this intermittent error, and I've failed to find a solution. I just ignored it; thinking I could solve it later, but now I've created another object that uses vertex arrays and so this error is occurring more often.

code:

Traceback (most recent call last):
  File "Z:\Goon\Goon.py", line 589, in <module>
    main()
  File "Z:\Goon\Goon.py", line 581, in main
    DrawGLScene()
  File "Z:\Goon\Goon.py", line 321, in DrawGLScene
    glDrawArrays(GL_QUADS, 4, 8)
WindowsError: exception: access violation reading 0x01AC5000

This happens most often when I run Goon.py from a batch file, but can happen when executing Goon.py normally. I'm thinking that the problem lies with the compiled python files that are created each time Goon.py is ran or executed. Sprites on the screen will often flash and glitch(blending) proceeding this error.

Here is heavily omitted code showing how I use Vertex Arrays. I'll post the vertex array lists if necessary.

code:

    glEnableClientState(GL_VERTEX_ARRAY)
    glEnableClientState(GL_NORMAL_ARRAY)
    glEnableClientState(GL_TEXTURE_COORD_ARRAY)
     
    glNormalPointer(GL_FLOAT, 0, normals)
    glTexCoordPointer(2, GL_FLOAT, 0, texCoords)
    glVertexPointer(3, GL_FLOAT, 0, vertices)
    
    glLightfv(GL_LIGHT1, GL_POSITION,LightPosition)
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
    
    for z in map
        for y map[z]
            for x map[z][y]
                glPushMatrix()
                glLoadIdentity()
                glTranslatef(x,y,z)

                if foo:
                   glBindTexture(GL_TEXTURE_2D, texture[t])
                   glDrawArrays(GL_QUADS, 4, 8)
                
                else:
                   glBindTexture(GL_TEXTURE_2D, texture[t])
                   glDrawArrays(GL_QUADS, 0, 4)

                glPopMatrix()

   glDisableClientState(GL_TEXTURE_COORD_ARRAY)
   glDisableClientState(GL_NORMAL_ARRAY)
   glDisableClientState(GL_VERTEX_ARRAY)

Chris Awful fucked around with this message at 03:00 on Jun 29, 2010

# ? Jun 28, 2010 23:23

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Jonnty posted:

e: pretty sure I asked about deep reloading in ipython in #python once and they scoffed and told me that I shouldn't need it. twats.

Typically when people want this, they should be writing unit tests instead.

# ? Jun 29, 2010 00:09

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Habnabit posted:

Typically when people want this, they should be writing unit tests instead.

Hello again!

Seriously though, I've not really needed to use it since, so he was probably right. I guess it's just my gut instinct to get angry at the line you see too often in open source forums and IRC channels: "it's not our fault for not having that feature, it's your fault for wanting it".

# ? Jun 29, 2010 00:34

Hughlander: May 11, 2005

I have a question about pyparsing... I'm parsing the output of save_object() from an LP style MUD which is a space separated key/value pair system where each kv is on a single line. It supports double quote delimited strings, numbers in either int or float format, a dictionary delimited by ([ ]) or a list delimited by ({ }) Based on the pyparsing JSON demo, I created:

code:

from pyparsing import *

lpcString = dblQuotedString.setParseAction( removeQuotes )
lpcNumber = Combine( Optional('-') + ( '0' | Word('123456789',nums) ) +
                    Optional( '.' + Word(nums) ) )

lpcMapping = Forward()
lpcValue = Forward()
lpcElements = delimitedList( lpcValue )
lpcArray = Group(Suppress('({') + Optional(lpcElements) + Optional(Suppress(',')) + Suppress('})') )
lpcValue << ( lpcString | lpcNumber | Group(lpcMapping) | lpcArray )
memberDef = Group( lpcString + Suppress(':') + lpcValue )
lpcMembers = delimitedList( memberDef ) + Suppress(Optional(','))
lpcMapping << Dict( Suppress('([') + Optional(lpcMembers) + Suppress('])') )


def convertNumbers(s,l,toks):
    n = toks[0]
    try:
        return int(n)
    except ValueError, ve:
        return float(n)

lpcNumber.setParseAction( convertNumbers )

lpcKeyvalue = Forward()
lpcKeyvalue = Group( Word( alphanums +  '_' ) + lpcValue )

lpcFile = Dict( ZeroOrMore( lpcKeyvalue ) )
lpcComment = pythonStyleComment
lpcFile.ignore( lpcComment )

if __name__ == "__main__":
    import pprint
    wc = open('file.o', 'r')
    bleh = lpcFile.parseString(wc.read())

Running it on a 100k file (Typical user data) looks like:

code:

$ time ./o.py

real    0m2.964s
user    0m2.908s
sys     0m0.040s

However, the goal of this was to replace a PHP script that was written a number of years ago. The php script can be found at http://pastebin.com/e36V9nGn

code:

$ time php ObjectData.php

real    0m0.872s
user    0m0.836s
sys     0m0.028s

The PHP version executes in 30% of the time of the pyparsing version! Is there anything wrong with my pyparsing code that would explain this? Is there anything I could be doing to speed it up other than write a C extension that will handle this? (For comparison the actual MUD implementation of this finishes in under 100ms.)

# ? Jun 29, 2010 03:54

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

Jonnty posted:

Seriously though, I've not really needed to use it since, so he was probably right. I guess it's just my gut instinct to get angry at the line you see too often in open source forums and IRC channels: "it's not our fault for not having that feature, it's your fault for wanting it".

Sometimes there's a reason features don't exist.

Hughlander posted:

pyparsing is horrendously slow, sadly. You might try pymeta or ply instead; both are parser generators.

# ? Jun 29, 2010 07:49

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Habnabit posted:

Sometimes there's a reason features don't exist.

It tends to be more when the feature is "linux sound actually working" though or something like that.

# ? Jun 29, 2010 11:35

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 07:29

Stabby McDamage: Dec 11, 2005; Doctor Rope

Habnabit posted:

Sometimes there's a reason features don't exist.

pyparsing is horrendously slow, sadly. You might try pymeta or ply instead; both are parser generators.

With a data format that simple, it may be easier just to read line-by-line and use split and/or a regex. If I was writing it, I never would even have considered using a parsing library for space-based key-value pairs.

# ? Jun 29, 2010 15:41

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »