Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Scaevolus: Apr 16, 2007

Khorne posted:

Note that varone[0], varone[1] are lists of numerical values.

Are they lists of floats?

Scaevolus fucked around with this message at 00:06 on Feb 26, 2010

# ? Feb 26, 2010 00:00

Adbot: ADBOT LOVES YOU

# ? May 28, 2024 16:37

Khorne: May 1, 2002

Bonus posted:

Can you post the exact code to reproduce that? My guess is that the items that are in those lists don't have proper __eq__ methods. Cause this works fine:

Edit: Nevermind. This has to do with floating point numbers and I shouldn't spend 14 hours per day working on things and 5 hours per day sleeping. This is the third time I have made a tremendously stupid mistake today, and the other two times weren't related to programming or a language I have been using for two days.

Khorne fucked around with this message at 07:31 on Feb 26, 2010

# ? Feb 26, 2010 00:06

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

BeefofAges posted:

From what I understand, nose is for testing Python projects. In this case, I'm testing a piece of hardware which is hooked up to my PC via a serial port. My utility doesn't have a command line interface. Instead it has a simple GUI for setting up the serial port and a simple terminal for interaction with the serial port. Could nose still fit my needs?

I've rarely used nose to test actual python products (mainly because previous jobs were mainly written in java). Nose is awesome, and can execute a test that's written in python that in turn tests, well, anything.

# ? Feb 26, 2010 01:31

tef: May 30, 2004; -> some l-system crap ->

Pycon stuff is online!

I recommend this talk by raymond hettinger on data structures:

http://python.mirocommunity.org/video/1530/mastering-team-play-four-power

# ? Feb 26, 2010 16:34

hey mom its 420: May 12, 2007

Lamacq posted:

Can you elaborate a little on this, and/or show me where in the python docs I can read up on this syntax? B/c I don't really understand what's going on here.

http://www.python.org/dev/peps/pep-0318/

They're decorators, all they do is take a function and return a function. So doing:

code:

@foo
def bar():
  pass

is equal to doing

code:

def bar():
  pass
bar = foo(bar)

# ? Feb 26, 2010 17:27

Stabby McDamage: Dec 11, 2005; Doctor Rope

Bonus posted:

http://www.python.org/dev/peps/pep-0318/

They're decorators, all they do is take a function and return a function. So doing:
code:
@foo
def bar():
  pass
is equal to doing
code:
def bar():
  pass
bar = foo(bar)

I've been looking at this for a while, and I get what basic decorators do, but I don't see what the do/undo decorators are for. If I apply the decorator transformation you showed to the do/undo example, I go from this:

code:

@do
def t1(self):
   # poo poo
   
@t1.undo
def t1(self):
   # undo poo poo

to this:

code:

def t1(self):
   # poo poo
t1 = do(t1)
   
def t1(self):
   # undo poo poo
t1 = t1.undo(t1)

What does that actually mean? How would these be called?

# ? Feb 27, 2010 05:41

hey mom its 420: May 12, 2007

A basic use of decorators is something like this:

code:

def tell(func):
  def new_func(*args, **kwargs):
    print "I'm being called!"
    ret = func(*args, **kwargs)
    print "I'm finished"
    return ret
  return new_func

@tell
def inc(x):
  return x+1

code:

>>> inc(3)
I'm being called!
I'm finished
4

Now with that in mind, you can use callable objects and some cleverness to do something like this:

code:

class Undoable(object):
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)

    def undo(self, func):
        self.undo_action = func
        return func

def do(func):
    return Undoable(func)

@do
def t1(x):
  print "haha"
  return x+1

@t1.undo
def t2():
  print "reversing haha"

It may be a bit confusing at first, but here's how it works: when you do do(func), it will return a callable object that acts exactly like that function, only that it has an extra attribute, namely undo. undo is itself a decorator and when it gets called, it sets the undo_action property of the function to which it belongs to the function that it decorated. So with the above, you can do:

code:

>>> t1(3)
haha
4
>>> t1.undo_action()
reversing haha

But that still doesn't really solve your original problem, which was basically implementing transactions in Python. And that's not a simple task at all, but it's probably doable.

# ? Feb 27, 2010 13:40

Stabby McDamage: Dec 11, 2005; Doctor Rope

Now I see. That's cool.

I was able to solve the transaction problem using the undo list I posted earlier. I just used inline defs to specify and enqueue the undo-action right after the do-action was complete.

# ? Feb 27, 2010 18:33

Dijkstracula: Mar 18, 2003; You can't spell 'vector field' without me, Professor!

edit is not reply god dammit

# ? Feb 28, 2010 10:13

Dijkstracula: Mar 18, 2003; You can't spell 'vector field' without me, Professor!

Hey Python people:

I'm a Python neophyte using it for a class project and I'm currently fighting with HTMLParser. In particular, I'm trying to parse a page that has Javascript with embedded HTML, and it's confusing the parser. In particular, this line:

code:

	else if (arrow == 3 && width != 0) a = "<div><img src=a2.gif style=position:absolute;left:"+((width / 2) - 5)+"px;top:0px;></div>";
                                                                                                  ^
                                                                                                  |

Because of the quotation mark, I end up getting a HTMLParseError: malformed start tag, at line 476, column 25 exception. Is there any sensible way of handling those quotation marks without trying to file a bug report upstream?

# ? Feb 28, 2010 10:13

Captain Capacitor: Jan 21, 2008; The code you say?

Dijkstracula posted:

Hey Python people:

I'm a Python neophyte using it for a class project and I'm currently fighting with HTMLParser. In particular, I'm trying to parse a page that has Javascript with embedded HTML, and it's confusing the parser. In particular, this line:
code:
	else if (arrow == 3 && width != 0) a = "<div><img src=a2.gif style=position:absolute;left:"+((width / 2) - 5)+"px;top:0px;></div>";
                                                                                                  ^
                                                                                                  |
Because of the quotation mark, I end up getting a HTMLParseError: malformed start tag, at line 476, column 25 exception. Is there any sensible way of handling those quotation marks without trying to file a bug report upstream?

Is this quotation mark that is in your code or the parser code?

# ? Feb 28, 2010 10:36

tef: May 30, 2004; -> some l-system crap ->

Dijkstracula posted:

Hey Python people:

I'm a Python neophyte using it for a class project and I'm currently fighting with HTMLParser. Is there any sensible way of handling those quotation marks without trying to file a bug report upstream?

Use another library? I would recommend lxml if you've been using element-tree style things or pyquery is a wrapper that uses css selectors.

# ? Feb 28, 2010 12:20

motorcyclesarejets: Apr 24, 2007

Dijkstracula posted:

Hey Python people:

I'm a Python neophyte using it for a class project and I'm currently fighting with HTMLParser. In particular, I'm trying to parse a page that has Javascript with embedded HTML, and it's confusing the parser. In particular, this line:
code:
	else if (arrow == 3 && width != 0) a = "<div><img src=a2.gif style=position:absolute;left:"+((width / 2) - 5)+"px;top:0px;></div>";
                                                                                                  ^
                                                                                                  |
Because of the quotation mark, I end up getting a HTMLParseError: malformed start tag, at line 476, column 25 exception. Is there any sensible way of handling those quotation marks without trying to file a bug report upstream?

Try:

code:

else if (arrow == 3 && width != 0) a = "<div><img src=a2.gif style=position:absolute;left:"+str((width / 2) - 5)+"px;top:0px;></div>"

# ? Feb 28, 2010 16:45

chips: Dec 25, 2004; Mein Führer! I can walk!

Maybe you need to wrap the javascript code in HTML comment tags, if that's allowed. Otherwise find a better parser.

# ? Feb 28, 2010 17:27

Dijkstracula: Mar 18, 2003; You can't spell 'vector field' without me, Professor!

Yeah, so this is in a random page's code that I can't change (yes, the sample I gave was in the page that the parser dies on), but still needs to work with. It really needs to be tolerant of any sort of broken HTML that I might try. (I mean, I assumed that the built-in parser would be, or at least more so than any third party alternatives)

So, lxml is the preferred library, or do people have other parser recommendations that won't choke at the first weird page I throw at it? :argh:

# ? Feb 28, 2010 18:55

tripwire: Nov 19, 2004; _{ghost flow}

Dijkstracula posted:

Yeah, so this is in a random page's code that I can't change (yes, the sample I gave was in the page that the parser dies on), but still needs to work with. It really needs to be tolerant of any sort of broken HTML that I might try. (I mean, I assumed that the built-in parser would be, or at least more so than any third party alternatives)

So, lxml is the preferred library, or do people have other parser recommendations that won't choke at the first weird page I throw at it?

Lxml is great. Pyquery is also great, and it uses lxml as a parser (among others).

# ? Feb 28, 2010 18:59

UberJumper: May 20, 2007; woop

Quick question bout numpy.

If i have a matrix

code:

matrix([[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]])

How can i easily zero the column at 2 and the row at 2? So the matrix becomes:

code:

matrix([[0, 1, 0, 3, 4],
        [0, 1, 0, 3, 4],
        [0, 0, 0, 0, 0],
        [0, 1, 0, 3, 4],
        [0, 1, 0, 3, 4]])

# ? Feb 28, 2010 19:36

tef: May 30, 2004; -> some l-system crap ->

Dijkstracula posted:

So, lxml is the preferred library, or do people have other parser recommendations that won't choke at the first weird page I throw at it?

I have used lxml in production environments, it is good at html soup.

# ? Feb 28, 2010 20:16

Sock on a Fish: Jul 17, 2004; What if that thing I said?

I'm encountering a very strange problem printing a simple text report. I've got some headers, and then some characters that I want inserted under the headers. Here's the code, with some extra stuff thrown in to help me debug:

code:

fmt_string = '%20s%20s%10s%5s%14s%11s'
headers = ('last_name','first_name','chapter','paid','meal_choice','checked_in')
header_underscores = []
for h in headers:
	header_underscores.append('-' * len(h))

print headers
print header_underscores

print len(headers)
print len(header_underscores)

print fmt_string % headers
print fmt_string % header_underscores

Here's the output:

code:

('last_name', 'first_name', 'chapter', 'paid', 'meal_choice', 'checked_in')
['---------', '----------', '-------', '----', '-----------', '----------']
6
6
           last_name          first_name   chapter paid   meal_choice checked_in
Traceback (most recent call last):
  File "poop.py", line 14, in <module>
    print fmt_string % header_underscores
TypeError: not enough arguments for format string

What the hell?

edit : never mind, the problem is that for these operations tuples and lists aren't interchangeable for some reason

Sock on a Fish fucked around with this message at 20:38 on Feb 28, 2010

# ? Feb 28, 2010 20:32

UberJumper: May 20, 2007; woop

Sock on a Fish posted:

I'm encountering a very strange problem printing a simple text report. I've got some headers, and then some characters that I want inserted under the headers. Here's the code, with some extra stuff thrown in to help me debug:
code:
fmt_string = '%20s%20s%10s%5s%14s%11s'
headers = ('last_name','first_name','chapter','paid','meal_choice','checked_in')
header_underscores = []
for h in headers:
	header_underscores.append('-' * len(h))

print headers
print header_underscores

print len(headers)
print len(header_underscores)

print fmt_string % headers
print fmt_string % header_underscores
Here's the output:
code:
('last_name', 'first_name', 'chapter', 'paid', 'meal_choice', 'checked_in')
['---------', '----------', '-------', '----', '-----------', '----------']
6
6
           last_name          first_name   chapter paid   meal_choice checked_in
Traceback (most recent call last):
  File "poop.py", line 14, in <module>
    print fmt_string % header_underscores
TypeError: not enough arguments for format string
What the hell?

edit : never mind, the problem is that for these operations tuples and lists aren't interchangeable for some reason

This fixes it.

code:

print fmt_string % tuple(header_underscores)

# ? Feb 28, 2010 21:21

tripwire: Nov 19, 2004; _{ghost flow}

UberJumper posted:

Quick question bout numpy.

If i have a matrix
code:
matrix([[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]])
How can i easily zero the column at 2 and the row at 2? So the matrix becomes:
code:
matrix([[0, 1, 0, 3, 4],
        [0, 1, 0, 3, 4],
        [0, 0, 0, 0, 0],
        [0, 1, 0, 3, 4],
        [0, 1, 0, 3, 4]])
?

To set the first row:
matrix[0, ] = 0

To set the first column:

matrix[ ,0] = 0

# ? Feb 28, 2010 21:59

UberJumper: May 20, 2007; woop

tripwire posted:

To set the first row:
matrix[0, ] = 0

To set the first column:

matrix[ ,0] = 0

Numpy. Is Pure. Awesome.

# ? Feb 28, 2010 23:10

hey mom its 420: May 12, 2007

tripwire posted:

To set the first row:
matrix[0, ] = 0

To set the first column:

matrix[ ,0] = 0

The first seems fine since 0, is (0,), but isn't the second assignment a syntax error?

# ? Mar 1, 2010 01:40

tripwire: Nov 19, 2004; _{ghost flow}

Bonus posted:

The first seems fine since 0, is (0,), but isn't the second assignment a syntax error?

Woops! Should have a colon in there:

code:

>>> matrix = numpy.zeros( (5,5), numpy.uint )
>>> matrix[3,] = 5
>>> matrix
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [5, 5, 5, 5, 5],
       [0, 0, 0, 0, 0]], dtype=uint64)
>>> matrix[:,2] = 3
>>> matrix
array([[0, 0, 3, 0, 0],
       [0, 0, 3, 0, 0],
       [0, 0, 3, 0, 0],
       [5, 5, 3, 5, 5],
       [0, 0, 3, 0, 0]], dtype=uint64)

# ? Mar 1, 2010 01:55

yippee cahier: Mar 28, 2005

I have some code that seems like it could be done concisely another way. Any weird lambda or generator tricks I'm missing out on?

code:

leap_second_lut = ( datetime.datetime(1981,7,1),
                    datetime.datetime(1982,7,1),
                    datetime.datetime(1983,7,1),
                    datetime.datetime(1985,7,1),
                    datetime.datetime(1988,1,1),
                    datetime.datetime(1990,1,1),
                    datetime.datetime(1991,1,1),
                    datetime.datetime(1992,7,1),
                    datetime.datetime(1993,7,1),
                    datetime.datetime(1994,7,1),
                    datetime.datetime(1996,1,1),
                    datetime.datetime(1997,7,1),
                    datetime.datetime(1999,1,1),
                    datetime.datetime(2006,1,1),
                    datetime.datetime(2009,1,1) )

time = datetime.utcnow()

leap_seconds = datetime.timedelta( seconds=len([x for x in leap_second_lut if x < time]) )

Code determines the number of leap seconds to add to UTC to get GPS time. Let's assume that 'time' could be a random datetime somewhere from 1980-present, and not datetime.utcnow() or else leap_seconds would be a static timedelta(seconds=15).

# ? Mar 1, 2010 20:27

tef: May 30, 2004; -> some l-system crap ->

you could do:

leap_second_lut = [datetime.datetime(*a) for a in [ (1987,7,1), (....) ] ]

# ? Mar 1, 2010 20:49

Lurchington: Jan 2, 2003; Forums Dragoon

Dijkstracula posted:

So, lxml is the preferred library, or do people have other parser recommendations that won't choke at the first weird page I throw at it?

I threw this completely boring page at lxml.html and it kept breaking on parse.

I have used BeautifulSoup before and it was ok-ish, although I've heard people throw around "unpythonic" and other similar epithets.

# ? Mar 1, 2010 22:18

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

It's unpythonic cuz it works. :laugh:

# ? Mar 1, 2010 22:21

tef: May 30, 2004; -> some l-system crap ->

Try parsing the html as html, rather than parsing as xml (the lxml default) :3:

code:

>>> import lxml.etree as et
>>> page =et.parse("http://www.usccb.org/nab/032110.shtml", et.HTMLParser())

>>> 
>>> for x in page.xpath("//a/@href"):
...     print x
... 
[url]http://www.usccb.org[/url]
[url]http://www.usccb.org[/url]
[url]http://www.usccb.org[/url]
...........

edit: you said you were using lxml.html? can you paste the error message?

tef fucked around with this message at 22:25 on Mar 1, 2010

# ? Mar 1, 2010 22:23

Scaevolus: Apr 16, 2007

Lurchington posted:

I threw this completely boring page at lxml.html and it kept breaking on parse.

I have used BeautifulSoup before and it was ok-ish, although I've heard people throw around "unpythonic" and other similar epithets.

It works fine for me.

code:

>>> from lxml import html
>>> h = html.parse("http://www.usccb.org/nab/032110.shtml")
>>> print html.tostring(h.getroot())
<html><head><title>USCCB - (NAB) - March 21, 2010</title>

[[REDACTED]]

<!-- comment out no need for this as months are all over the left margin
        <div class="menutitle" onclick="SwitchMenu('sub3')">Readings and Psalms for the Month</a></div>
        <span class="submenu" id="sub3">

# ? Mar 2, 2010 01:11

Dijkstracula: Mar 18, 2003; You can't spell 'vector field' without me, Professor!

Lurchington posted:

I have used BeautifulSoup before and it was ok-ish, although I've heard people throw around "unpythonic" and other similar epithets.

I tried Beautiful Soup this afternoon but it broke -- that is, threw some exception deep in its bowls -- on a reasonably trivial page.

lxml looks like a serious pain in that it has to be installed rather than my just dropping it in my working directory, and I'm working across OSs and need it to be basically self-contained...this is really preposterous, so I gather there's really no good solution for a reasonably simple thing like this? It's bullshit like this that causes people to just regex the poo poo out of HTML instead of using proper parsers.

edit: here's my team's torture test site. If you can find something that can make sense of it, let me know

Dijkstracula fucked around with this message at 07:18 on Mar 2, 2010

# ? Mar 2, 2010 07:00

king_kilr: May 25, 2007

pip install lxml

Ta.loving.da.

# ? Mar 2, 2010 07:19

the wizards beard: Apr 15, 2007; Reppin

4 LIFE 4 REAL

I'm fairly new to Python and have been mostly using it for simple scripting and writing file conversion tools.

As my first large-ish project I'd like to generate static callgraphs of a C style language I use at work. Functions are defined as

code:

Function func(params)
   ...
End

and called as

code:

func(params)

Is there an easy way to handle the parsing? I'd prefer not to start tokenizing if a module will do it for me. All I need is to pull out a list of functions and a list of functions called by each function. The graphing will probably be done to PDF using graphviz.

# ? Mar 2, 2010 16:51

MaberMK: Feb 1, 2008; BFFs

the wizards beard posted:

I'm fairly new to Python and have been mostly using it for simple scripting and writing file conversion tools.

As my first large-ish project I'd like to generate static callgraphs of a C style language I use at work. Functions are defined as
code:
Function func(params)
   ...
End
and called as
code:
func(params)
Is there an easy way to handle the parsing? I'd prefer not to start tokenizing if a module will do it for me. All I need is to pull out a list of functions and a list of functions called by each function. The graphing will probably be done to PDF using graphviz.

Check out ply

# ? Mar 2, 2010 17:06

the wizards beard: Apr 15, 2007; Reppin

4 LIFE 4 REAL

Thanks, I had thought of lex/yac/flex/bison but assumed they would be overkill, the articles linked on that page don't look too intimidating though. I'll look into this tonight, any other suggestions would be great.

# ? Mar 2, 2010 17:13

bitprophet: Jul 22, 2004; Taco Defender

the wizards beard posted:

Thanks, I had thought of lex/yac/flex/bison but assumed they would be overkill, the articles linked on that page don't look too intimidating though. I'll look into this tonight, any other suggestions would be great.

PyParsing is really nice too, though I've only used it in smallish setups so far. A talk was given at PyCon the other weekend that went over both it and PLY pretty well: slides (video might be on pycon.blip.tv but I can't find it quickly)

# ? Mar 2, 2010 19:22

the wizards beard: Apr 15, 2007; Reppin

4 LIFE 4 REAL

It looks like PyParsing is what I want, I've already got a script that can read a file and print all the functions declared and their arguments. Thanks guys.

# ? Mar 2, 2010 20:30

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Purely out of curiosity, is there a way to do something like

code:

someString += (aString = 'hello')

Basically, set aString to 'hello' and then append it to someString.

# ? Mar 2, 2010 21:27

king_kilr: May 25, 2007

BeefofAges posted:

Purely out of curiosity, is there a way to do something like
code:
someString += (aString = 'hello')
Basically, set aString to 'hello' and then append it to someString.

Yes

code:

aString = "hello"
someString += aString

# ? Mar 2, 2010 21:45

Adbot: ADBOT LOVES YOU

# ? May 28, 2024 16:37

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

king_kilr posted:

Yes

code:

aString = "hello"
someString += aString

Yeah, that's what I said to do, but the guy who asked me was like "I'm doing this in a lot of places, so it would be nice to make it compact". Heh.

# ? Mar 2, 2010 21:51

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »