Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

RobotEmpire: Dec 8, 2007

Anyone fooling around in Flask yet? I can't quite figure out the flash() mechanism...

# ? Apr 15, 2010 00:30

Adbot: ADBOT LOVES YOU

# ? May 24, 2024 18:04

king_kilr: May 25, 2007

Other than discussing design points with Armin, no I haven't used it. I don't care for it's use of thread locals.

# ? Apr 15, 2010 07:02

Instigating Static: Mar 7, 2010; B)

Is there a regular expression to parse a D.N.S. packet?

I'm using the expression:

code:

"(?P<ID>..)(?P<FLAGS>..)(?P<QUESTIONS>..)(?P<ANSWERS>..)(?P<AUTHORITY>..)(?P<ADDITIONAL>..)((?:.)(?:[a-zA-Z0-9](?:.){1})*)*"

However on an example it produces:

code:

"[('1.', '2.', '..', '..', '..', '..', '.www.yahoo.com.1.2.')]"

And one with multiple queries:

code:

"[('1.', '2.', '..', '..', '..', '..', '.com.1.2.')]"

The result I'd like is...

code:

"[('1.', '2.', '..', '..', '..', '..', 'www.yahoo.com.', '1.', '2.', 'www.google.com', '1.', '2.')]"

The fields after the domain, ex: .www.google.com., should also be dynamic. As often responses contain 4-byte TTL's, and other 2-byte fields and primary names, such as: ".any-fp.wa1.b.yahoo.com." along with type and field classes.

# ? Apr 18, 2010 21:27

bitprophet: Jul 22, 2004; Taco Defender

A) It's "DNS", not "D.N.S.". Nobody puts periods in tech-related acronyms

Doing so will make you look kind of silly, no offense.

B) Is there any particular reason you're using regex for this instead of an actual DNS library? If you're having to examine post-facto network traffic and/or for some reason aren't able to use a tool designed for the job (e.g. Wireshark, tcpdump, etc) then you might also find raw protocol parsing easier with something like PyParsing or PLY.

C) What's your actual question? It's not very clear from your post. Is it that you're only getting a single result tuple when you expected two? You might want to paste your actual Python code for help with that

D) You're using a-zA-Z0-9 where you could be using \w (technically \w matches underscores too but based on the rest of your expression and my brief remembrance of the protocol itself, that shouldn't be a big deal) -- just a tip.

# ? Apr 19, 2010 00:13

Profane Obituary!: May 19, 2009; This Motherfucker is Dead

bitprophet posted:

B) Is there any particular reason you're using regex for this instead of an actual DNS library? If you're having to examine post-facto network traffic and/or for some reason aren't able to use a tool designed for the job (e.g. Wireshark, tcpdump, etc) then you might also find raw protocol parsing easier with something like PyParsing or PLY.

Just saying that pyparsing is absolutely fantastic, and my biggest issue with it is that i can't seem to find a way that works correctly with pypyarsing.CharsNotIn.

# ? Apr 19, 2010 01:34

Instigating Static: Mar 7, 2010; B)

bitprophet posted:

A) It's "DNS", not "D.N.S.". Nobody puts periods in tech-related acronyms Doing so will make you look kind of silly, no offense.

B) Is there any particular reason you're using regex for this instead of an actual DNS library? If you're having to examine post-facto network traffic and/or for some reason aren't able to use a tool designed for the job (e.g. Wireshark, tcpdump, etc) then you might also find raw protocol parsing easier with something like PyParsing or PLY.

C) What's your actual question? It's not very clear from your post. Is it that you're only getting a single result tuple when you expected two? You might want to paste your actual Python code for help with that

D) You're using a-zA-Z0-9 where you could be using \w (technically \w matches underscores too but based on the rest of your expression and my brief remembrance of the protocol itself, that shouldn't be a big deal) -- just a tip.

I'm sorry I couldn't be more clear... I simply want a pattern that can parse a DNS packet, based on fields, into a tuple.

# ? Apr 19, 2010 02:53

awillyc: Aug 13, 2006; I'm too goddamn lazy to buy my own account NYAHHH

I'm working on making a little simulator of sorts to emulate a piece of hardware and its serial protocol. The question is, what is a good way to let the protocol object know when an instance variable get modified so it can blast out the proper string on the serial port.

# ? Apr 19, 2010 15:16

bitprophet: Jul 22, 2004; Taco Defender

Instigating Static posted:

I'm sorry I couldn't be more clear... I simply want a pattern that can parse a DNS packet, based on fields, into a tuple.

Can you post some of the inputs you alluded to in your original post (i.e. the actual, literal string being fed to whatever regex method you're using)? Would make it a little easier to help that way. And again, the snippet itself where you call re.findall or similar, would also help a bit.

You probably do just need some tweaking of the regex to have some optional groups or whatnot, though again, I'd advise a step back because unless you're doing some sort of homework that says "USE REGEX AND NOTHING ELSE TO PARSE THIS DNS PACKET", this is probably not the best way to solve whatever your problem is

awillyc posted:

I'm working on making a little simulator of sorts to emulate a piece of hardware and its serial protocol. The question is, what is a good way to let the protocol object know when an instance variable get modified so it can blast out the proper string on the serial port.

If you're not aware of them, check out special method names like __setattr__. They're powerful and useful, though they can also sometimes lead to unexpected results (or harder-to-debug code) if used improperly.

Based solely on your problem description I'm not sure this sort of metaprogramming is really the path you want to go down; you may want something more explicit (i.e. a dedicated method call) instead. Depends on your comfort level, your exact use case and your intended audience (if any).

# ? Apr 19, 2010 16:11

awillyc: Aug 13, 2006; I'm too goddamn lazy to buy my own account NYAHHH

bitprophet posted:

If you're not aware of them, check out special method names like __setattr__. They're powerful and useful, though they can also sometimes lead to unexpected results (or harder-to-debug code) if used improperly.

Based solely on your problem description I'm not sure this sort of metaprogramming is really the path you want to go down; you may want something more explicit (i.e. a dedicated method call) instead. Depends on your comfort level, your exact use case and your intended audience (if any).

A few specifics, that maybe will help show where I'm at.
It's possible I've overcomplicated it, but I'm kinda new to OO and how the pieces should fall together I guess.

My protocol object knows how to build the string based on data fed to it, for example:

code:

#Method that should be called whenever led's change on any station
#Protocol Object

def led_changed(self, station):
    '''emits formated protocol string to serial port or socket'''

    bits = "".join([str(int(x)) for x in station.leds])
    self.emit('LEDU,%03d,%s\r' % (station.address, bits))



#Method that updates the led state inside Station instance
#Station Object

def set_led(self, index, state):
    '''Updates state of led in Station instance'''

    self._leds[index] = state
    #Do something here that will call the method in Protocol
    #Some form of Delegation or something?

    #I could store a reference to the protocol object and do something like:
    #self.protocol.led_update(self)

    #But that seems kinda clunky, ideas?

# ? Apr 19, 2010 16:40

Sock on a Fish: Jul 17, 2004; What if that thing I said?

Does anyone know of a library that will check a set of credentials against an htpasswd file and return whether or not the credentials are valid? There's a package on CPAN that does exactly this but I haven't been able to find anything for Python. Calling a Perl script from my Python is not ideal, the piece I'm writing will be running on a web cluster with a shared Python installation but there is no shared Perl installation (yet).

# ? Apr 19, 2010 22:30

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Can't you just roll your own? .htpasswd files are really simple.

# ? Apr 19, 2010 22:33

Sock on a Fish: Jul 17, 2004; What if that thing I said?

Avenging Dentist posted:

Can't you just roll your own? .htpasswd files are really simple.

Yeah, I'm doing that now, I'd just rather not have to. I've never done any crypto work before so there'll be some trial and error in figuring out the MD5 and SHA libraries.

# ? Apr 19, 2010 22:38

Sock on a Fish: Jul 17, 2004; What if that thing I said?

Sock on a Fish posted:

Yeah, I'm doing that now, I'd just rather not have to. I've never done any crypto work before so there'll be some trial and error in figuring out the MD5 and SHA libraries.

For instance, I'm already stuck because when I SHA1 digest something in hashlib the result is full of unprintable characters that are nothing like what I see in in an htpasswd. There's hexdigest, but the htpasswd obviously isn't hex because it has characters > f.

How do I use hashlib to digest a password for proper comparison against an htpasswd file?

# ? Apr 19, 2010 22:47

other people: Jun 27, 2004; Associate Christ

I am trying to use MySQLdb and it is not working

code:

sboxdb = MySQLdb.connect(host="localhost", user="huh", passwd="huh", db="slimserver")

cursor = sboxdb.cursor()

for track in tList:
	sqlc = "UPDATE what SET rating = " + str(track[1]) + ' WHERE url = "' + track[0] + '"'
	cursor.execute(sqlc)

This executes fine, but doesn't update anything. If I make it print sqlc and paste that command into command line mysql, it works just fine. What gives?

edit: The rating column is a tinyint(1), if that matters.

# ? Apr 19, 2010 22:55

Sock on a Fish: Jul 17, 2004; What if that thing I said?

Sock on a Fish posted:

For instance, I'm already stuck because when I SHA1 digest something in hashlib the result is full of unprintable characters that are nothing like what I see in in an htpasswd. There's hexdigest, but the htpasswd obviously isn't hex because it has characters > f.

How do I use hashlib to digest a password for proper comparison against an htpasswd file?

Whoops, looks like I should be encoding to base64 after digesting.

# ? Apr 19, 2010 22:59

king_kilr: May 25, 2007

Kaluza-Klein posted:

I am trying to use MySQLdb and it is not working
code:
sboxdb = MySQLdb.connect(host="localhost", user="huh", passwd="huh", db="slimserver")

cursor = sboxdb.cursor()

for track in tList:
	sqlc = "UPDATE what SET rating = " + str(track[1]) + ' WHERE url = "' + track[0] + '"'
	cursor.execute(sqlc)
This executes fine, but doesn't update anything. If I make it print sqlc and paste that command into command line mysql, it works just fine. What gives?

edit: The rating column is a tinyint(1), if that matters.

NEVER NEVER NEVER NEVER interpolate strings into SQL. This is a security hole waiting to happen, instead use the prepared query syntax provided for you by your driver:

code:

	sqlc = "UPDATE what SET rating = %s WHERE url = %s"
	cursor.execute(sqlc, str(track[1]), track[0])

# ? Apr 19, 2010 23:04

other people: Jun 27, 2004; Associate Christ

king_kilr posted:

NEVER NEVER NEVER NEVER interpolate strings into SQL. This is a security hole waiting to happen, instead use the prepared query syntax provided for you by your driver:
code:
	sqlc = "UPDATE what SET rating = %s WHERE url = %s"
	cursor.execute(sqlc, str(track[1]), track[0])

Thank you.

That gives me the error:

TypeError: execute() takes at most 3 arguments (4 given)

Not really worried about security, I am doing this just to insert the data one time for myself and then the script is gone. This is just something I want to do on my home computer. I wouldn't dare to let any of my horrible code run in the wild

# ? Apr 19, 2010 23:10

tripwire: Nov 19, 2004; _{ghost flow}

Kaluza-Klein posted:

Thank you.

That gives me the error:

TypeError: execute() takes at most 3 arguments (4 given)

Not really worried about security, I am doing this just to insert the data one time for myself and then the script is gone. This is just something I want to do on my home computer. I wouldn't dare to let any of my horrible code run in the wild .

The three parameters are the instance (implicit 1st argument), the template and a sequence of variables to safely interpolate into the template

so wrap the last parameter in another set of brackets.

# ? Apr 19, 2010 23:14

Sock on a Fish: Jul 17, 2004; What if that thing I said?

Sock on a Fish posted:

Whoops, looks like I should be encoding to base64 after digesting.

Here you go everyone, I only needed to implement SHA:

code:

import re
import hashlib
import base64

class HtPasswordChecker(object):
	def __init__(self, htpasswd_path):
		self.htpasswd_path = htpasswd_path

	def check_password(self,username,password):
		htpasswd_digested_password = self.get_htpasswd_digested_password(username)

		is_good = False
		if htpasswd_digested_password:
			if htpasswd_digested_password[0:5] == '{SHA}':
				is_good = self.check_password_sha(username,password)

		return is_good

	def check_password_sha(self,username,password):
		htpasswd_digested_password = self.get_htpasswd_digested_password(username)
		digested_input_password = base64.b64encode(hashlib.sha1(password).digest())

		is_good = False
		if htpasswd_digested_password[5:] == digested_input_password:
			is_good = True

		return is_good

	def get_htpasswd_digested_password(self,username):
		f = open(self.htpasswd_path,'rb')
		htpasswd_contents = f.readlines()
		f.close()

		htpasswd_digested_password = None
		for line in htpasswd_contents:
			match = re.search(r'%s:(.*)$' % username, line)
			if match:
				htpasswd_digested_password = match.group(1)

		return htpasswd_digested_password

Sock on a Fish fucked around with this message at 23:52 on Apr 19, 2010

# ? Apr 19, 2010 23:18

other people: Jun 27, 2004; Associate Christ

tripwire posted:

The three parameters are the instance (implicit 1st argument), the template and a sequence of variables to safely interpolate into the template

so wrap the last parameter in another set of brackets.

I added cursor.execute("COMMIT") to the end of the script and now it all works. I can't believe how much time I wasted on that.

Thank you guys!

# ? Apr 19, 2010 23:20

tripwire: Nov 19, 2004; _{ghost flow}

Sock on a Fish posted:

Here you go everyone, I only needed to implement SHA:
code:
.
.
.

Hey cool! I'm probably going to end up using this.

# ? Apr 19, 2010 23:30

Sock on a Fish: Jul 17, 2004; What if that thing I said?

tripwire posted:

Hey cool! I'm probably going to end up using this.

Glad someone else finds it useful. Just make sure to implement the other crypto standards acceptable in an htpasswd if you can't be sure that they'll all be SHA.

edit: Just made a change to the code in the above to account for cases in which the user doesn't exist, whoops on that. I think it's safe to forgo checking for length and assume that index errors won't be encountered as long as the htpasswd file is well-formed.

Sock on a Fish fucked around with this message at 23:53 on Apr 19, 2010

# ? Apr 19, 2010 23:38

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

# ? Apr 22, 2010 19:17

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Answer: because the only stuff I care about in Python core is stuff that would be relevant to Python 3 and I can't use Python 3 because NumPy isn't on Python 3.

# ? Apr 22, 2010 19:21

A A 2 3 5 8 K: Nov 24, 2003; Illiteracy... what does that word even mean?

Is there a tutorial anywhere for getting a build environment going? Actually I guess that's not the right question, building isn't difficult.

# ? Apr 22, 2010 20:09

wins32767: Mar 16, 2007

m0nk3yz posted:

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

The docs are solid, the language is fine, and I don't really have any itches to scratch. Why would I want to contribute when my needs are pretty well met, especially with respect to the core language?

# ? Apr 22, 2010 20:28

Parker Lewis: Jan 4, 2006; Can't Lose

m0nk3yz posted:

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

I figured Google was taking care of it.

# ? Apr 22, 2010 20:37

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Parker Lewis posted:

I figured Google was taking care of it.

Touche.

# ? Apr 22, 2010 20:46

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Avenging Dentist posted:

Answer: because the only stuff I care about in Python core is stuff that would be relevant to Python 3 and I can't use Python 3 because NumPy isn't on Python 3.

What about the stdlib?

m0nk3yz fucked around with this message at 20:52 on Apr 22, 2010

# ? Apr 22, 2010 20:47

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

wins32767 posted:

The docs are solid, the language is fine, and I don't really have any itches to scratch. Why would I want to contribute when my needs are pretty well met, especially with respect to the core language?

I have to agree with this. Python is pretty solid.

# ? Apr 22, 2010 21:00

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

m0nk3yz posted:

What about the stdlib?

The only things I've ever used more than once for work in the stdlib are re, os, sys, and unittest. About the only thing I could see myself caring about enough to make a stdlib module is a makefile parser since I had to write a limited version anyway.

Honestly, writing anything general-purpose in Python is fairly painful to me, so I try to avoid it.

EDIT: Oh actually I'd rewrite os.path if I had the opportunity, since I absolutely cannot stand working with paths in Python. os.path.join is just a massive pain in my rear end.

Avenging Dentist fucked around with this message at 21:09 on Apr 22, 2010

# ? Apr 22, 2010 21:03

tef: May 30, 2004; -> some l-system crap ->

MEAT TREAT posted:

I have to agree with this. Python is pretty solid.

give or take a handful of third party libraries.

a lot of the std library is fantastic, but things like asyncore or httpserver or mimehandling or email or html parsing or xml handling or soap...
(personally, I think urllib and it's kin should die in a fire).

So why haven't I done anything about it?

Excuse 1: So every so often i'm tempted to step up, but doing any of these involves getting large c libraries in core, so I'm not optimistic.

Excuse 2: I wrote a lot of python to deal with nightmarish scraping, but now it's no longer my job I don't feel like picking it up in my free time)

Excuse 3: my actual excuse for not contributing is that i'd rather write my own language :3:

# ? Apr 22, 2010 21:15

ctz: Feb 6, 2003

Is there a directory of cpython modules? (As opposed to pure python modules.) Pypi doesn't seem to make this classification

(Background: I'm looking for a corpus of existing modules to do static analysis on.)

# ? Apr 22, 2010 21:16

tef: May 30, 2004; -> some l-system crap ->

m0nk3yz posted:

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

The thing is, it isn't just about having simple or straight forward tasks available, it's about having fun tasks available. Lowering the bar doesn't necessarily mean there is more incentive. A lot of the contribution tasks you list are sundry and support roles.

You should also be asking how to make the python development process more fun, and as a result you'll get more people.

# ? Apr 22, 2010 21:23

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

m0nk3yz posted:

Since I'm casting my net far and wide:

http://jessenoller.com/2010/04/22/why-arent-you-contributing-to-python/

Because it's too hard.

It may be easy, but I don't know it and don't really feel like figuring it out.

Every once in awhile I'll come across something wrong in the docs, but I just move on because I don't have any clue what to do about it.

I need beat over the head with information about how to fix problems that I see.

For example, I'll correct things on Wikipedia every once in awhile because there's goddamn edit buttons all over the place. I've never noticed such a thing in the stdlib docs. There may be a link on each page about correcting errors, but I've never noticed it and that's the problem. (I'm not saying the docs need to be a wiki, but that Wikipedia makes it obvious how to fix mistakes.)

Someone made a comment on your site comparing Python to Ubuntu, and I think that's spot on. Most of the documentation and help available to Ubuntu users is all interactive, whereas Python.org seems pretty static.

Also, the reason I posted here instead of a comment on your site is among the reasons I haven't done any contributions to Python. The mental inertia involved with participating in something new (especially considering all the other things that require interaction on the internet) is fairly substantial which means that barriers to entry need to be as small as possible.

This applies to some ninja expert just as much as it does to a noob.

I realize I haven't said anything real concrete about what you could do yet, so here's a suggestion I've given up to 30 seconds of thought to:

Mimic the Wikipedia style of having an edit button for each part of the docs. I'm guessing you don't want it to be an edit button, but perhaps a "talk" or "suggestion" button would work. Don't require a login, but make it so you can easily put in an email address for notifications of replies to your suggestion. After someone in charge agrees with what's been ironed out in the talk section, make it part of the docs.

# ? Apr 22, 2010 21:36

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Here's another point: there doesn't appear to be an obvious place on the Python site that lists things that need work. Compare to clang: http://clang.llvm.org/OpenProjects.html

There's a reason I went through the effort of building clang and spending a good 2-3 weeks just figuring out their API in order to write some small patches (aside from the fact that I like clang a lot): they actually told me up front what they needed people to do, in specifics. As an example, my first project was on that page and was written up as something like "implement qualified ID parsing for C++ class members".

# ? Apr 22, 2010 21:53

Scaevolus: Apr 16, 2007

m0nk3yz posted:

Why aren't you contributing?

I submitted a test case for a known memory leak (with an uncommitted fix) that's existed for over 4 years last month and nothing happened.

# ? Apr 22, 2010 23:28

ThatNateGuy: Oct 15, 2004; "Is that right?"; Slippery Tilde

What's the reason Python hasn't become as big as say Java?

# ? Apr 23, 2010 01:15

A A 2 3 5 8 K: Nov 24, 2003; Illiteracy... what does that word even mean?

ThatNateGuy posted:

What's the reason Python hasn't become as big as say Java?

Java was marketed by a multi-billion dollar company and was associated with a lot of stupid hype.

# ? Apr 23, 2010 01:47

Adbot: ADBOT LOVES YOU

# ? May 24, 2024 18:04

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

And it is already cemented in every "Enterprise" system. Python missed that boat.

# ? Apr 23, 2010 01:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »