Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

maskenfreiheit: Dec 30, 2004

Edit: doublepost

maskenfreiheit fucked around with this message at 01:23 on Mar 13, 2017

# ? Dec 27, 2010 02:14

Adbot: ADBOT LOVES YOU

# ? Jun 13, 2024 05:50

nbv4: Aug 21, 2002; by Duchess Gummybuns

GregNorc posted:

Ok thanks I guess I'm just a bit overwhelmed, they're both really big and I'm just trying to find this one method to grad table data, and everything seems to be more about creating html or css objects... like I have the script wget an html page, and I just need this data from the table, I don't need to render any changes I make or add colors and stuff, I just need the strings in the column, and everything seems really complex...

also, look into BeautifulSoup

# ? Dec 27, 2010 02:22

spankweasel: Jan 4, 2006

GregNorc posted:

Ok thanks I guess I'm just a bit overwhelmed, they're both really big and I'm just trying to find this one method to grad table data, and everything seems to be more about creating html or css objects... like I have the script wget an html page, and I just need this data from the table, I don't need to render any changes I make or add colors and stuff, I just need the strings in the column, and everything seems really complex...

Anything more complicated than basic data types should be thought of as objects anyway.

Also, lxml is really quite easy to use. Assuming a real simple HTML page with only one table, you can process table cells with something like this:

code:

from lxml import etree
tree = etree.parse("http://derp.com/foo.html", parser=etree.HTMLParser())
table_node = tree.find("table")
tr_node_list = table_node.findall("tr")
for tr_node in tr_node_list:
    td_node_list = tr_node.findall("td")
    for td_node in td_node_list:
        <process it>

# ? Dec 27, 2010 02:30

Scaevolus: Apr 16, 2007

uncleTomOfFinland posted:

I have this 3rd party ircbot/markov chain program thing written in Python and it's really thrashing the hard drive every time it generates replies to user messages. I have looked through the source code and can't figure out whats really causing it since it appears to me that the bot simply loads the database files into memory completely, marshall.loads() the huge string objects into dictionary objects and then goes on to do whatever with the dictionary objects. How should I proceed with trying to find out what is causing all this disk activity? Just pointing me at the right module/software would suffice I think.

The problem is that this is a terrible way to handle generating markov text chains. It takes an excessive amount of memory, and that's why you're getting hard drive thrashing.

The right way to do this is using a disk-backed key-value store. SQLite is an easy way. Hailo is a Perl implementation of a disk-backed markov text chain generator.

I wrote something like that in Python a while ago for my IRC bot, but I'm not sure where the code went.

# ? Dec 27, 2010 02:35

Scaevolus: Apr 16, 2007

edit: oops

# ? Dec 27, 2010 02:35

Deus Rex: Mar 5, 2005

Say I've got a dict like this:

code:

dick = dict()
dick['balls'] = 69
dick['nuts'] = 420
dick['nads'] = 666
...

and I want to put some of the values into single variables like this:

code:

balls = dick['balls']
nuts = dick['nuts']
nads = dick['nads']

Is there a more Pythonic way of doing this? I was hoping something like

code:

(balls, nuts, nads) = dick['balls', 'nuts', 'nads']

would work but that doesn't seem to be the case.

# ? Dec 27, 2010 14:47

ErIog: Jul 11, 2001

Deus Rex posted:

Is there a more Pythonic way of doing this? I was hoping something like
code:
(balls, nuts, nads) = dick['balls', 'nuts', 'nads']
would work but that doesn't seem to be the case.

code:

option_a, option_b, option_c = dick['option_a'], dick['option_b'], dick['option_c']

ErIog fucked around with this message at 15:53 on Dec 27, 2010

# ? Dec 27, 2010 15:51

Deus Rex: Mar 5, 2005

ErIog posted:

code:

option_a, option_b, option_c = dick['option_a'], dick['option_b'], dick['option_c']

This is actually what I've gone with, but it still seems ugly and I wish there were a way to index multiple dict values with a tuple of keys

Oh well, thanks!

# ? Dec 27, 2010 17:13

ErIog: Jul 11, 2001

Deus Rex posted:

This is actually what I've gone with, but it still seems ugly and I wish there were a way to index multiple dict values with a tuple of keys Oh well, thanks!

You could do it with a list comprehension, but that's just going to end up being more cutesy than need be.

code:

option_a, option_b, option_c = [dick[key] for key in ['option_a', 'option_b', 'option_c']]

Please don't do this.

# ? Dec 27, 2010 17:38

Lurchington: Jan 2, 2003; Forums Dragoon

honestly, seems like, depending on what's involved, this may be a good time to use a namedtuple

Your example of turning dictionary values (from keys) into separate objects is something I've mostly only done when I'm tired of using bracket syntax and putting quotes around the same attributes over and over.

code:

Dicks = namedtuple('Dicks', 'balls nuts nads')
my_dicks = Dicks(69, 420, 666)

assert my_dicks.balls == my_dicks[0] and my_dicks.balls == 69

this still has a namespace/scope of my_dicks which is nice, and you don't have to be stuck with a dictionary just to have meaningful names to access attributes with

# ? Dec 27, 2010 18:14

Stabby McDamage: Dec 11, 2005; Doctor Rope

ErIog posted:

You could do it with a list comprehension, but that's just going to end up being more cutesy than need be.
code:
option_a, option_b, option_c = [dick[key] for key in ['option_a', 'option_b', 'option_c']]
Please don't do this.

If you want to be truly unholy, you could do:

code:

globals().update(d)

This will plunk every key/value in d into the global variable namespace, so if d['a']=5, there's now a global variable a=5. This is purely for comedy value, do not actually do this.

# ? Dec 27, 2010 18:26

Deus Rex: Mar 5, 2005

Lurchington posted:

honestly, seems like, depending on what's involved, this may be a good time to use a namedtuple

Your example of turning dictionary values (from keys) into separate objects is something I've mostly only done when I'm tired of using bracket syntax and putting quotes around the same attributes over and over.
code:
Dicks = namedtuple('Dicks', 'balls nuts nads')
my_dicks = Dicks(69, 420, 666)

assert my_dicks.balls == my_dicks[0] and my_dicks.balls == 69
this still has a namespace/scope of my_dicks which is nice, and you don't have to be stuck with a dictionary just to have meaningful names to access attributes with

this works well for my situation, thanks!

# ? Dec 27, 2010 18:34

Yakattak: Dec 17, 2009; I am Grumpypuss
>:3

What's the best way to implement an enum in Python like in C?

# ? Dec 27, 2010 22:26

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Yakattak posted:

What's the best way to implement an enum in Python like in C?

there's an explanation here: http://norvig.com/python-iaq.html

# ? Dec 27, 2010 23:06

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

Reposting from the previous page, and then I'll let it rest:

Bodhi Tea posted:

I'm trying to do form based authentication with urllib2, I'm following the code given here: http://personalpages.tds.net/~kent37/kk/00010.html

So basically I have something that looks like this (taken from the site):
code:
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(opener)
params = urllib.urlencode(dict(username='joe', password='secret'))
f = opener.open('http://coolsite.example.com/login/', params)
The problem is that it seems that even if I provide bogus credentials, the login page is retrieved without a problem, so no error message is raised (that I'm aware of). How can I handle the case where I provide invalid username and password?

# ? Dec 28, 2010 05:18

tef: May 30, 2004; -> some l-system crap ->

look at the page returned for clues to the result of the form submission

# ? Dec 28, 2010 06:06

bc87: Dec 11, 2010

is def a function?

code:

def example()
     print "dick suck gently caress duck muck truck luck"

example()

Is it a function that is used to define other functions?

# ? Dec 28, 2010 10:57

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

def is a keyword

http://docs.python.org/reference/lexical_analysis.html#keywords

# ? Dec 28, 2010 12:04

bc87: Dec 11, 2010

Carthag posted:

def is a keyword

http://docs.python.org/reference/lexical_analysis.html#keywords

wow, all those keywords are nearly all the things I learned from this web tutorial.

http://www.ibiblio.org/swaroopch/byteofpython/read/

At first, I thought def was some kind of built-in function, like lens. It's a whole different class (pun not intended) altogether.

# ? Dec 28, 2010 12:34

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

tef posted:

look at the page returned for clues to the result of the form submission

Makes sense. Thanks.

# ? Dec 28, 2010 12:42

Lurchington: Jan 2, 2003; Forums Dragoon

Jonnty posted:

there's an explanation here: http://norvig.com/python-iaq.html

If you plan to use the code he provided, make it a new-style class:

class Enum(object):

# ? Dec 28, 2010 20:40

Yakattak: Dec 17, 2009; I am Grumpypuss
>:3

I have a Python class we'll call ClassA and another Python class which is supposed to import ClassA which we'll call ClassB. The directory structure is as follows:

code:

MainDir
../Dir
..../DirA/ClassA
..../DirB/ClassB

How the hell do I use ClassA from ClassB?

# ? Dec 28, 2010 20:45

spankweasel: Jan 4, 2006

from ..DirA import ClassA

# ? Dec 28, 2010 21:05

Yakattak: Dec 17, 2009; I am Grumpypuss
>:3

spankweasel posted:

from ..DirA import ClassA

Doing that just gives me this error:
ValueError: Attempted relative import in non-package

Yes, I do have __init__.py's in all the directories. Does it matter that ClassA is really just static with a bunch of methods and no actual class definition?

# ? Dec 28, 2010 21:17

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

What's the practical difference between these?

code:

class WhatUp:

code:

class WhatUp():

code:

class WhatUp(object):

# ? Dec 28, 2010 21:30

Yakattak: Dec 17, 2009; I am Grumpypuss
>:3

Thermopyle posted:

What's the practical difference between these?
code:
class WhatUp:
code:
class WhatUp():
code:
class WhatUp(object):

The first and second one don't inherit any classes, and the last one inherits from object.

# ? Dec 28, 2010 21:36

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Yakattak posted:

The first and second one don't inherit any classes, and the last one inherits from object.

Well...duh.

Why would a person care? Is there a difference between the first and second?

# ? Dec 28, 2010 21:39

Lurchington: Jan 2, 2003; Forums Dragoon

the last one is a new-style class, the first is a 'classic class' and I've never actually seen the middle one, although I suppose it's a new-style class that derives from nothing? :confused:

http://docs.python.org/reference/datamodel.html#newstyle

short version, use new-style classes, as they're the only syntax allowed in 3.0, but in using them now you get access to things like descriptors and although the docs like to say new-style classes aren't fully integrated yet, I've found the opposite. There's a lot of things ( like the semantics of overriding __getatrr__ and __getattribute__ ) that are assuming the reader already uses new-style classes.

# ? Dec 28, 2010 21:39

Lurchington: Jan 2, 2003; Forums Dragoon

Yakattak posted:

Doing that just gives me this error:
ValueError: Attempted relative import in non-package

Yes, I do have __init__.py's in all the directories. Does it matter that ClassA is really just static with a bunch of methods and no actual class definition?

for simplicity, I'd have wanted to do an explicit absolute import:

from dir.DirA import classB

I've only ever done a explicit relative import in a package I wanted to future proof for when I make it a plugin. For whatever reason, I find it harder to deal with relative.

Lurchington fucked around with this message at 21:50 on Dec 28, 2010

# ? Dec 28, 2010 21:43

Yakattak: Dec 17, 2009; I am Grumpypuss
>:3

Lurchington posted:

for simplicity, I'd have wanted to do an explicit absolute import:

from dir.DirA import classB

I've only ever done a explicit relative import in a package I wanted to future proof for when I make it a plugin. For whatever reason, I find it harder to deal with relative.

Does anything need to be in sys.path to be able to do this?

e: Somehow I got it to work. I just remade the __init__.py and added a print statement in Dir and now it works. I don't know why but whatever. v :shobon:

Yakattak fucked around with this message at 22:26 on Dec 28, 2010

# ? Dec 28, 2010 22:05

Lurchington: Jan 2, 2003; Forums Dragoon

if you were doing your testing inside of a python interpreter, there may have been an issue in of making sure you were in the right place or not seeing changes you made to already imported modules.

if you were doing python <module that has class A in it> then it's accurate to say that you really aren't in a package in the way python expects, whatever module you're running is going to have a name of "__main__" and you can't usually do a relative import up directory levels with that.

This stackoverflow answer gets at the same thing.

So, when you made your change, it's possible that you also re-ran your code from a different directory or with a different entrypoint. Of course, there's also a possibility that your change to add a print statement blew away some stale pyc's that somehow persisted with old stuff (this may be possible if you're using a full ide's interpreter like pycharm, eclipse, or pyscripter'

# ? Dec 28, 2010 22:54

ErIog: Jul 11, 2001

So I'm making a very small program for personal use in tkinter, and I've run into a really strange wall. I'm mixing tkinter with the pywin32 bindings because I really hate everything to do with the syntax and naming conventions of pywin32, and it feels like tkinter gets more done with far less code. The strangeness is happening in the transition between the pywin32 clipboard watching and my program's reaction to it in tkinter. I've put a piece of code up that demonstrates the crash here.

My window and all its controls are being handled in tkinter. The pywin32 bindings are doing clipboard watching and clipboard access when the clipboard changes. From what I've gathered about the way the clipboard watching pieces of pywin32 work, you can make it work with anything you want as long as you provide pywin32 with the hwnd value of your window. I'm doing that part, and it works when the program first starts. It just doesn't seem to work when the clipboard changes.

When the program launches, it grabs the clipboard and puts it into the search box and edit box just fine. When the clipboard is modified, the event I want to fire off is firing off...except that event that totally worked before when the program launched is now causing a weird hang instead of doing what it's supposed to do. I can print the clipboard contents to stdout all I want if the clipboard changes, but not put that same data into a tkinter widget. It only hangs like that if it starts to interact with any of my tkinter widgets after being fired off by a clipboard change notification.

It feels like there's some pywin32 etiquette I've missed in adapting the clipboard-watching sample code I was using over to my tkinter-using program. Tkinter apparently doesn't like to produce stack traces or error messages, and I can't really even begin to know what to look for trying to debug it with pdb.

ErIog fucked around with this message at 03:52 on Dec 29, 2010

# ? Dec 29, 2010 03:45

Steve Winwood: Sep 22, 2009; by Ozmaugh

I'm having some trouble importing the cx_Oracle module for use with Oracle databases.

If I want to connect to another server (Oracle 10g) from my machine (Oracle 9i client) then should I have the cx_Oracle module for 10g or 9i installed?

I'm running into this error and am not sure how to proceed

>>> import cx_Oracle

Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
import cx_Oracle
ImportError: DLL load failed: The specified procedure could not be found.

I'm fairly new to Python so I might be missing something obvious.

# ? Dec 29, 2010 22:47

MaberMK: Feb 1, 2008; BFFs

e: nm

# ? Dec 29, 2010 23:15

Lurchington: Jan 2, 2003; Forums Dragoon

figured out something kind of neat, so I figured I'd toss it here. Simulating a game of war with 4 players, and wanted to print the cards played on a certain round in the form of: 'K > J > 10 > 9' or 'K = K > Q > J' depending, in a relatively efficient way:

code:

played = <given and sorted list of the played cards/player who played them in descending order>
group_gen = (g for _, g in itertools.groupby(played))
print u' > '.join(u' = '.join(thing for thing in g)) for g in group_gen)

some sample output (real version uses logging module)

pre:

2010-12-31 23:01:12,073 - war - DEBUG - ComputerWar3-K♡ > ComputerWar0-7♡ > ComputerWar1-6♡ > ComputerWar2-4♢
2010-12-31 23:01:12,073 - war - DEBUG - ComputerWar2-A♢ > ComputerWar1-Q♠ = ComputerWar3-Q♡ > ComputerWar0-9♠
2010-12-31 23:01:12,074 - war - DEBUG - ComputerWar0-K♠ > ComputerWar1-Q♣ > ComputerWar2-10♡ > ComputerWar3-7♢
2010-12-31 23:01:12,074 - war - DEBUG - ComputerWar0-J♠ > ComputerWar2-7♣ > ComputerWar3-6♣ > ComputerWar1-2♡
2010-12-31 23:01:12,074 - war - DEBUG - ComputerWar2-K♣ > ComputerWar1-J♡ > ComputerWar0-5♢ = ComputerWar3-5♣
2010-12-31 23:01:12,074 - war - DEBUG - ComputerWar0-A♡ > ComputerWar1-9♣ > ComputerWar2-5♠ > ComputerWar3-3♡
2010-12-31 23:01:12,075 - war - DEBUG - ComputerWar0-A♣ > ComputerWar1-Q♢ > ComputerWar2-10♢ > ComputerWar3-3♣
2010-12-31 23:01:12,075 - war - DEBUG - ComputerWar2-K♢ > ComputerWar0-9♡ > ComputerWar3-8♣ > ComputerWar1-5♡
2010-12-31 23:01:12,075 - war - DEBUG - ComputerWar0-9♢ > ComputerWar1-8♠ > ComputerWar2-6♠ = ComputerWar3-6♢
2010-12-31 23:01:12,076 - war - DEBUG - ComputerWar3-A♠ > ComputerWar0-J♣ > ComputerWar2-10♠ > ComputerWar1-7♠
2010-12-31 23:01:12,076 - war - DEBUG - ComputerWar1-8♡ > ComputerWar0-4♡ = ComputerWar2-4♠ > ComputerWar3-3♢
2010-12-31 23:01:12,076 - war - DEBUG - ComputerWar0-10♣ > ComputerWar1-8♢ > ComputerWar2-4♣ > ComputerWar3-3♠
2010-12-31 23:01:12,077 - war - DEBUG - ComputerWar2-J♢ > ComputerWar0-2♣ = ComputerWar1-2♠ = ComputerWar3-2♢

side note: __repr__ does not like unicode characters at all. boo. I know there's a __unicode__ magic method somewhere which I'll probably check on

# ? Jan 1, 2011 08:06

Masked Pumpkin: May 10, 2008

I'm fiddling around with ftplib and have a problem - I'm uploading to an FTP server on a less than reliable connection and want to catch timeouts if they occur. I've looked into the signal library to help with this (it's a Xubuntu machine) but although the signal side of things works well with everything else, storbinary doesn't seem to pick anything up without terminating completely.

Some sample code:

code:

import os
import time
import signal
import ftplib

def receive_alarm(signum, stack):
     print 'Alarm :', time.ctime()

signal.signal(signal.SIGALRM, receive_alarm)
signal.alarm(90)
s = ftplib.FTP('myserver.co.za', 'myusername', 'mypass')
s.storbinary('STOR ' + 'filename.tar.gz', open('/home/user/filename.tar.gz'))

If I replace the storbinary line with a time.sleep(120) command, then the alarm triggers and all is well - but if I cut the connection on storbinary while it's busy uploading then the only way to kill the process is by running a kill -9 from the terminal (other kill codes also work, but nothing seems to just interrupt that one line and let it move on).

I'm guessing I'm missing something hideously obvious, but I just can't work it out - failing that, should I be using something different for FTP instead?

# ? Jan 1, 2011 16:29

Captain Capacitor: Jan 21, 2008; The code you say?

Masked Pumpkin posted:

I'm guessing I'm missing something hideously obvious, but I just can't work it out - failing that, should I be using something different for FTP instead?

I wouldn't call hideously obvious, but I imagine storbinary is a blocking call. According to the Python docs storbinary has an argument for a callback function after each block is transferred. You might be able to use that.

# ? Jan 1, 2011 17:41

Masked Pumpkin: May 10, 2008

Captain Capacitor posted:

I wouldn't call hideously obvious, but I imagine storbinary is a blocking call. According to the Python docs storbinary has an argument for a callback function after each block is transferred. You might be able to use that.

That works (thanks!), although putting together something to find out if the callback was timing out boggled my mind at this hour... I did find a solution though by using socket.setdefaulttimeout(20) and catching that exception - it's probably not as elegant a solution, though.

# ? Jan 1, 2011 19:00

awesomepanda: Dec 26, 2005; The good life, as i concieve it, is a happy life.

Python webbrowser module question. When i open a new page, it opens it through internet explorer. How do i get the module to open through firefox?

# ? Jan 3, 2011 01:38

Adbot: ADBOT LOVES YOU

# ? Jun 13, 2024 05:50

No Safe Word: Feb 26, 2005

CrazyPanda posted:

Python webbrowser module question. When i open a new page, it opens it through internet explorer. How do i get the module to open through firefox?

change your default browser

# ? Jan 3, 2011 01:53

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »