Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

MaberMK: Feb 1, 2008; BFFs

Sock on a Fish posted:

I'm using binary numbers to help me build out all the possible permutations of an arbitrary set of boolean variables. Is there a way to to manipulate binary numbers in Python without using string operations? I'd like to be able to ask for the value of a digit at some position in the number.

It seems both unnatural and unnecessary to treat binary numbers like strings.

Here's what I'm doing right now for an input s.
code:
	p = []
	s_len = len(s)
	poss_matrix = []
	for x in range(2**s_len):
		poss_list = [None for y in range(s_len)]
		bits = bin(x)
		bits = bits[2:]
		while len(bits) < s_len:
			bits = '0%s' % bits
		for r in range(len(poss_list)):
			if int(bits[r]):
				poss_list[r] = True
			else:
				poss_list[r] = False
		poss_matrix.append(poss_list)

So in other words, you want to print the binary representation for every number 0 ... 2^s.

Google turned up this:

code:

binary = lambda i, c = (lambda i, c: i and (c(i >> 1, c) + str(i & 1)) or ''): c(i, c)

In which case:

code:

s = <some number>
bin_strings = [binary(x) for x in xrange(2 ** s)]

... is a list of the binary representations of all numbers up to 2^s.

You're hung up on thinking binary numbers are somehow different from other numbers--they're not. Binary numbers, decimal numbers, hexadecimal numbers, whatever, are all the same thing with different representations. Do numeric work in numeric types and write separate display code to display the number in whatever representation it is you want.

MaberMK fucked around with this message at 23:40 on Jan 11, 2010

# ? Jan 11, 2010 23:33

Adbot: ADBOT LOVES YOU

# ? May 12, 2024 12:05

Sock on a Fish: Jul 17, 2004; What if that thing I said?

tripwire posted:

What kind of permutations are you trying to build up? Itertools might be more suitable here.
Depending on exactly what you want, you could use combinations with the product function, or permutations which is a little more specialized.

Here's the output I'm looking for if len(s) == 4, and which my code produces:

code:

[   [False, False, False, False],
    [False, False, False, True],
    [False, False, True, False],
    [False, False, True, True],
    [False, True, False, False],
    [False, True, False, True],
    [False, True, True, False],
    [False, True, True, True],
    [True, False, False, False],
    [True, False, False, True],
    [True, False, True, False],
    [True, False, True, True],
    [True, True, False, False],
    [True, True, False, True],
    [True, True, True, False],
    [True, True, True, True]]

A list of lists, with each list representing one possible combination of T/F values given a set of four boolean vars.

It's no big deal leaving it the way it is, it works just fine. It just sticks in my craw that I can't just do something like bin_digit(number,position) and figure out if I'm dealing with a one or a zero.

Practically, I'm using this to build all possible representations of a string in combined ASCII and hex so that I can filter out unwanted input no matter how a user constructs it. e.g., if True evaluate the string with an ascii representation of this char, if False then evaluate with a hex representation of this char.

# ? Jan 11, 2010 23:40

MaberMK: Feb 1, 2008; BFFs

Sock on a Fish posted:

Practically, I'm using this to build all possible representations of a string in combined ASCII and hex so that I can filter out unwanted input no matter how a user constructs it. e.g., if True evaluate the string with an ascii representation of this char, if False then evaluate with a hex representation of this char.

Why not have rules for valid input and invalidate any input that doesn't comply?

# ? Jan 11, 2010 23:44

kuffs: Mar 29, 2007; Projectile Dysfunction

Sock on a Fish posted:

Here's the output I'm looking for if len(s) == 4, and which my code produces:

code:

[   [False, False, False, False],
    [False, False, False, True],
    [False, False, True, False],
    [False, False, True, True],
    [False, True, False, False],
    [False, True, False, True],
    [False, True, True, False],
    [False, True, True, True],
    [True, False, False, False],
    [True, False, False, True],
    [True, False, True, False],
    [True, False, True, True],
    [True, True, False, False],
    [True, True, False, True],
    [True, True, True, False],
    [True, True, True, True]]

You've successfully constructed the list of the binary representation of the numbers 0-15?

Sock on a Fish posted:

Practically, I'm using this to build all possible representations of a string in combined ASCII and hex so that I can filter out unwanted input no matter how a user constructs it. e.g., if True evaluate the string with an ascii representation of this char, if False then evaluate with a hex representation of this char.

Regex replacement isn't good enough for you?

I can't figure out what you're trying to do, but I can tell you you're probably doing it wrong. Care to back up a few levels and tell us what you're really trying to accomplish?

# ? Jan 11, 2010 23:56

Sock on a Fish: Jul 17, 2004; What if that thing I said?

MaberMK posted:

Why not have rules for valid input and invalidate any input that doesn't comply?

Because I can't touch the actual application code, this is a hacky wrapper around an application designed without security in mind. At some points it actually echoes back verbatim the parameters passed in your querystring/post, so that you could actually pass &hack=<script>hax</script> and have it work.

# ? Jan 11, 2010 23:58

tripwire: Nov 19, 2004; _{ghost flow}

Sock on a Fish posted:

Here's the output I'm looking for if len(s) == 4, and which my code produces:
code:
[   [False, False, False, False],
    [False, False, False, True],
    [False, False, True, False],
    [False, False, True, True],
    [False, True, False, False],
    [False, True, False, True],
    [False, True, True, False],
    [False, True, True, True],
    [True, False, False, False],
    [True, False, False, True],
    [True, False, True, False],
    [True, False, True, True],
    [True, True, False, False],
    [True, True, False, True],
    [True, True, True, False],
    [True, True, True, True]]
A list of lists, with each list representing one possible combination of T/F values given a set of four boolean vars.

It's no big deal leaving it the way it is, it works just fine. It just sticks in my craw that I can't just do something like bin_digit(number,position) and figure out if I'm dealing with a one or a zero.

Practically, I'm using this to build all possible representations of a string in combined ASCII and hex so that I can filter out unwanted input no matter how a user constructs it. e.g., if True evaluate the string with an ascii representation of this char, if False then evaluate with a hex representation of this char.

If the position variable in bin_digit(number,position) is counting from the left side then i'd do it like this:

code:

def bin_digit(n,index):
    return bool( int( bin(n)[2+index] ) )

Otherwise if you meant from the right then this would also work:

code:

def bin_digit(n,index):
    return bool( int( bin(n)[::-1][2+index] ) )

It seems like the wrong solution to what you say is the problem though. Surely theres a more sensible way to validate user input.

# ? Jan 12, 2010 00:05

Sock on a Fish: Jul 17, 2004; What if that thing I said?

kuffs posted:

You've successfully constructed the list of the binary representation of the numbers 0-15?

Regex replacement isn't good enough for you?

I can't figure out what you're trying to do, but I can tell you you're probably doing it wrong. Care to back up a few levels and tell us what you're really trying to accomplish?

Regex replacement is what I use, I use the list of possible combinations of ASCII and hex chars as patterns. Regex doesn't have a flag to treat hex characters as ASCII, AFAIK.

Heres the whole function.

code:

def hex_possibilities(s):
	'''
	This function returns a list of all possible representations of s as a mixture of ASCII and HEX.
	'''
	p = []
	s_len = len(s)
	poss_matrix = []
	for x in range(2**s_len):
		poss_list = [None for y in range(s_len)]
		bits = bin(x)
		bits = bits[2:]
		while len(bits) < s_len:
			bits = '0%s' % bits
		#print bits
		for r in range(len(poss_list)):
			#print (x,r)
			if int(bits[r]):
				poss_list[r] = True
			else:
				poss_list[r] = False
		poss_matrix.append(poss_list)

	possibilities = []
	for l in poss_matrix:
		this_p = []
		for x in range(len(l)):
			if l[x]:
				this_p.append(s[x])
			else:
				this_p.append(s[x].encode('hex'))
		possibilities.append(''.join(this_p).upper())
	return possibilities

And then I'll do this for strings that I don't want passed on to the application:

code:

	for x in list_of_bad_things:
		possible = hex_possibilities(x)
		for p in possible:
			line = re.sub(p,'',line)

I wish there was a more sensible way, but I'm not allowed to touch the actual application code. Everything I do has to be done as a man-in-the-middle.

# ? Jan 12, 2010 00:21

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

code:

Python 3.1.1 (r311:74483, Aug 17 2009, 16:45:59) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import itertools
>>> values = (True, False)
>>> for x in itertools.product(values, repeat=4):
...   print(x)
...
(True, True, True, True)
(True, True, True, False)
(True, True, False, True)
(True, True, False, False)
(True, False, True, True)
(True, False, True, False)
(True, False, False, True)
(True, False, False, False)
(False, True, True, True)
(False, True, True, False)
(False, True, False, True)
(False, True, False, False)
(False, False, True, True)
(False, False, True, False)
(False, False, False, True)
(False, False, False, False)

# ? Jan 12, 2010 04:32

Contero: Mar 28, 2004

I'm almost completely new to python, what are the practical and syntactic differences between 2.6 and 3.1? Is it / will it be reasonably well supported?

I'm mostly curious about using python as an extension language, and less about using it to develop applications and using a lot of libraries. Would it be worth it for me to then try and use 3.1 or should I just not bother?

# ? Jan 12, 2010 07:39

LuckySevens: Feb 16, 2004; fear not failure, fear only the limitations of our dreams

if in doubt, just use 2.6 for now, its better supported.

# ? Jan 12, 2010 10:15

tef: May 30, 2004; -> some l-system crap ->

Use 2.6 with a mind to porting to 3 eventually.

Unless you have to use unicode a lot, in which case I might recommend 3 if the libraries you want are available.

# ? Jan 12, 2010 17:07

dirby: Sep 21, 2004; Helping goons with math

I haven't used Python in a long time, and I don't program often/particularly well, so if the solution to my problem is to not code in such a stupid manner (as opposed to me needing to be aware of some feature of Python), I apologize in advance.

code:

class Box:
    def __init__(self,things):
        self.things=things
class Thing:
    def __init__(self,name):
        self.name=name
    def __str__(self):
        return "A Thing called "+self.name
a=Thing("hat")
b=Thing("shoe")
print(a,b)
c=Box([a,b])
print(c.things)

Why is it that the second print command gives me "[<__main__.Thing object at 0x013E7530>..." even though I went to the trouble of giving Thing a "__str__" method? The behavior I'd like is something like "[A Thing called hat,A Thing called shoe]". Is there no better way than defining a "printthings" method for Box that uses "str" explicitly?

dirby fucked around with this message at 21:54 on Jan 12, 2010

# ? Jan 12, 2010 21:49

Hot Dog Day #42: Jun 17, 2001; Egbert B. Gebstadter

helopticor posted:

I haven't used Python in a long time, and I don't program often/particularly well, so if the solution to my problem is to not code in such a stupid manner (as opposed to me needing to be aware of some feature of Python), I apologize in advance.
code:
class Box:
    def __init__(self,things):
        self.things=things
class Thing:
    def __init__(self,name):
        self.name=name
    def __str__(self):
        return "A Thing called "+self.name
a=Thing("hat")
b=Thing("shoe")
print(a,b)
c=Box([a,b])
print(c.things)
Why is it that the second print command gives me "[<__main__.Thing object at 0x013E7530>..." even though I went to the trouble of giving Thing a "__str__" method? The behavior I'd like is something like "[A Thing called hat,A Thing called shoe]". Is there no better way than defining a "printthings" method for Box that uses "str" explicitly?

The second command is giving you what it does because there's a separate __repr__ method which returns an "official string representation" for the object (ideally, one which could be used to reconstruct the object), while __str_ produces an "informal string representation". When you print a list it prints the __repr__ strings of its elements, not the __str__ strings.

To do what you want, you should use something like

code:

 for x in c.things:
  print x

# ? Jan 12, 2010 22:01

dirby: Sep 21, 2004; Helping goons with math

Hot Dog Day #42 posted:

To do what you want, you should use something like
code:
 for x in c.things:
  print x

Thanks, why not just replace my "__str__"'s with "__repr__"? Will that break Python in some subtle way (even if no two "Thing"s have the same name)?

# ? Jan 12, 2010 22:03

Hot Dog Day #42: Jun 17, 2001; Egbert B. Gebstadter

helopticor posted:

Thanks, why not just replace my "__str__"'s with "__repr__"? Will that break Python in some subtle way (even if no two "Thing"s have the same name)?

I don't think it's the recommended use of __repr__, but I'm not aware of any way in which it'd break Python either.

# ? Jan 12, 2010 22:07

MaberMK: Feb 1, 2008; BFFs

Hot Dog Day #42 posted:

I don't think it's the recommended use of __repr__, but I'm not aware of any way in which it'd break Python either.

It won't break anything--the difference is semantic. __repr__ is intended to be used as the official string representation of an object, say, "<SomeObject>". __str__ is informal... "holy crap I'm some object" would be appropriate to put in __str__, but not __repr__.

# ? Jan 12, 2010 23:59

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Contero posted:

I'm almost completely new to python, what are the practical and syntactic differences between 2.6 and 3.1? Is it / will it be reasonably well supported?

I'm mostly curious about using python as an extension language, and less about using it to develop applications and using a lot of libraries. Would it be worth it for me to then try and use 3.1 or should I just not bother?

If you are new, and don't plan on building on a lot of extension libraries: Use 3.1, it's the future, and 2.7 will be the end of the 2.x line. It's a cleaner language. The problem is that there is a lack of ported 3rdparty libraries.

Use 2.6 if you want access to the huge pile of 3rdparty applications. Also note, if googling for recipes/examples, most, if not all, will be in python 2.x syntax and you will need to mentally port what you see to what you need.

Sorry I haven't added this to the op yet; I've actually been hacking on a site containing all the python3 info you could want.

edit: added some info to the OP

m0nk3yz fucked around with this message at 01:49 on Jan 13, 2010

# ? Jan 13, 2010 01:41

wins32767: Mar 16, 2007

I have an SQLAlchemy question. I'm attempting to take some data in a delimited file and make a guess as to what type of data each field is (int, date, etc) and craft a table based on that. I then want to present this table to someone (who knows sql) for review. Is there anyway to get a Table object to print what it's schema is?

t = Table('Foo', db.metadata,
Column('Bar', String(255)))
print t.__repr__()
print t.__str__()

yields:
Table('Foo', MetaData(Engine(/*db_url_snipped*/)), Column('Bar', String(length=255, convert_unicode=False, assert_unicode=None), table=<Foo>), schema=None)
Foo

I suppose that I could return the __repr__ of all the columns and the table name, but it seems like there should be some way to get:

create table Foo (
Bar varchar(255)
)

without actually creating the table query and showing the debug logs.

# ? Jan 14, 2010 17:10

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

I'm doing some work for a client with, shall we say, a sub-optimal hosting package. Python is available, but as far as I can tell, there is no module available for accessing a MySQL database. The Python version is 2.4.3, meaning sqlite is not include by default, and it's not available by separate installation either.

What are my options for persistent data storage? It's not a high traffic website, but it could conceivably have to deal with attempted concurrent writes.

Should I give up and just use Perl?

# ? Jan 16, 2010 19:05

king_kilr: May 25, 2007

Is there a reason you can't just install MySQLDb?

# ? Jan 16, 2010 20:16

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

king_kilr posted:

Is there a reason you can't just install MySQLDb?

No shell access. Is there a way to install it without having any access to the shell or compiler, or any permissions to speak of on the server?

# ? Jan 16, 2010 20:53

king_kilr: May 25, 2007

PT6A posted:

No shell access. Is there a way to install it without having any access to the shell or compiler, or any permissions to speak of on the server?

Nope. I was looking around to see if there way a pure python MySQL adapter and there doesn't seem to be one (postgres has http://pybrary.net/pg8000/). You could write a perl script that executes SQL and then have Python call it

(please don't do this).

# ? Jan 16, 2010 20:58

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

king_kilr posted:

Nope. I was looking around to see if there way a pure python MySQL adapter and there doesn't seem to be one (postgres has http://pybrary.net/pg8000/). You could write a perl script that executes SQL and then have Python call it (please don't do this).

Done it.

quote:

What are my options for persistent data storage?

Have you looked at shelve?

# ? Jan 16, 2010 21:12

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

Thermopyle posted:

Done it.

Have you looked at shelve?

I have, but I'm concerned about what might happen if two processes attempt to write to it at the same time, and it seems that Python doesn't have any sort of file-locking functionality without using a bizarre workaround.

# ? Jan 16, 2010 22:15

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

king_kilr posted:

Nope. I was looking around to see if there way a pure python MySQL adapter and there doesn't seem to be one (postgres has http://pybrary.net/pg8000/). You could write a perl script that executes SQL and then have Python call it (please don't do this).

Now, someone was working on a pure Python MySQL library. The name escapes me, but I read about it recently and it is out there.

# ? Jan 16, 2010 23:22

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

outlier posted:

Now, someone was working on a pure Python MySQL library. The name escapes me, but I read about it recently and it is out there.

myconnpy?

It's pre-alpha and they stress it should not be used in production.

EDIT: The more I consider my problem, the more I think shelve or something like it might be the best solution. How should I prevent two separate invocations of the thread from trying to modify the data, though?

EDIT 2: Some people seem to use os.mkdir("somelockname") to do this, since POSIX guarantees that it's atomic, but that seems rather kludgy. fcntl.flock doesn't work on whatever type is returned by shelve.open, or I'd use that. I've opened a support ticket with my client's hosting service, hopefully there's something obvious that I'm missing and I can just use MySQL or sqlite.

PT6A fucked around with this message at 03:39 on Jan 17, 2010

# ? Jan 16, 2010 23:29

BigRedDot: Mar 6, 2008

Is there a reason you can't just put the shelf access in a method and protect it with a mutex?

# ? Jan 17, 2010 04:12

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

BigRedDot posted:

Is there a reason you can't just put the shelf access in a method and protect it with a mutex?

I'm thinking the major risk would be two clients triggering the script which accesses the file at the same time, in which case they wouldn't have access to the same mutex. It's a small chance, but it's a chance I don't really want to risk if there's a better way to do it. Each instance of the script will access the file only once.

# ? Jan 17, 2010 04:21

Alcool Tue: Nov 6, 2009; by Ozma

Trying to learn some GIMP, but getting the popup dialogues to work at all is giving me loving fits.

the function and parameters in question:
---
pdb.gimp_palettes_popup(palette_callback, popup_title, initial_palette)
---

popup_title and initial_palette are self explanatory, figuring out what this thing wants as a palette_callback is ruining my mind. tried making a function that doesn't return anything and passing it that, tried making a function that returns something and passing it that, tried passing it an assortment of different poo poo from the pdb, but the best i've got out of the dumb thing is:
---
Traceback (most recent call last):
File "<input>", line 1, in <module>
RuntimeError: Procedure 'gimp-palettes-popup' has been called with value '(null)' for argument 'palette-callback' (#1, type gchararray). This value is out of range
---

any advice w/r/t popup callbacks in gimp?

# ? Jan 17, 2010 09:19

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Can you post the code you've got right now? Make sure you're treating your callback function as data and not making a function call.

# ? Jan 17, 2010 09:22

Alcool Tue: Nov 6, 2009; by Ozma

BeefofAges posted:

Can you post the code you've got right now? Make sure you're treating your callback function as data and not making a function call.

Instead of pretending I know what I'm doing here, I'm just going to ask "How do I use Python to popup a palette selection dialogue in GIMP?(or any dialogue, really)" Getting active values and such is pretty transparent, but the gtk interface is a bit of a clusterfuck and I have no idea what it wants as a callback.

Thanks!

Alcool Tue fucked around with this message at 10:49 on Jan 17, 2010

# ? Jan 17, 2010 09:45

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

PT6A posted:

EDIT 2: Some people seem to use os.mkdir("somelockname") to do this, since POSIX guarantees that it's atomic, but that seems rather kludgy. fcntl.flock doesn't work on whatever type is returned by shelve.open, or I'd use that. I've opened a support ticket with my client's hosting service, hopefully there's something obvious that I'm missing and I can just use MySQL or sqlite.

Check out http://pypi.python.org/pypi/lockfile/0.8 - I've used it somewhat for inter-process locking in a small web app that has to manage on-disk resources.

# ? Jan 17, 2010 14:29

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

m0nk3yz posted:

Check out http://pypi.python.org/pypi/lockfile/0.8 - I've used it somewhat for inter-process locking in a small web app that has to manage on-disk resources.

It seems that that's nothing more than a nice interface to the lock-directory/-link that I was talking about before; perhaps, even being less-than-elegant, it's the best solution.

# ? Jan 17, 2010 18:14

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

PT6A posted:

It seems that that's nothing more than a nice interface to the lock-directory/-link that I was talking about before; perhaps, even being less-than-elegant, it's the best solution.

It's not rocket science; I'll give you that - but it's useful sugar for the problem.

# ? Jan 17, 2010 19:09

Scaevolus: Apr 16, 2007

The GIL seems to be partially fixed in Python 3.2-- instead of multiple threads being slower on multicore systems, now they're about the same speed.

http://www.dabeaz.com/blog/2010/01/presentation-on-new-python-gil.html

# ? Jan 17, 2010 23:27

tripwire: Nov 19, 2004; _{ghost flow}

Scaevolus posted:

The GIL seems to be partially fixed in Python 3.2-- instead of multiple threads being slower on multicore systems, now they're about the same speed.

http://www.dabeaz.com/blog/2010/01/presentation-on-new-python-gil.html

Hurray!

# ? Jan 18, 2010 01:07

ductonius: Apr 9, 2007; I heard there's a cream for that...

edit: Forget it. I was trying to import Tkinter in python 3.1 instead of tkinter. :bang:

Thought of the day: "Letter case matters."

ductonius fucked around with this message at 03:51 on Jan 20, 2010

# ? Jan 20, 2010 03:46

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I get some text from one web service that just drops possessive apostrophe's. For example, for the sentence "one goon's banana" it would return:

code:

one goons banana

I then need to submit this text to another web service that sometimes doesn't work as expected if apostrophe's aren't included. The owner of the web service is aware of the issue, but I expect it to be many months before it's fixed.

My current solution is to submit the text twice. Once without the apostrophe's and then again with every word ending with a "s" changed to end with a "'s".

This screws up on words that end with an s but aren't a possessive noun.

Any ideas on how to address this?

# ? Jan 21, 2010 06:08

king_kilr: May 25, 2007

Trying all possibly combinations of possesives and "just ending in an s". It means possibly doing a ton of queries though.

# ? Jan 21, 2010 06:16

Adbot: ADBOT LOVES YOU

# ? May 12, 2024 12:05

UberJumper: May 20, 2007; woop

Does anyone know how exactly comtypes rips out all the CLSID information out from COM automon files?

# ? Jan 21, 2010 07:47

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »