Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Stabby McDamage posted:

Is this how people play with vectors in python? I figured there would be a dedicated vector class rippling with OO functions.

That would be awful, and not really mathematically idiomatic anyway. The norm of a vector isn't an innate property of the vector, it's a function that takes a vector. This is a pretty important difference that people seem to miss, probably due in part to the fact that languages like Java make it a pain in the rear end to have free functions. NumPy actually provides too many convenience functions on arrays as it is.

Besides, there are lots of different normal functions out there. Or do you expect that the vector class have a method for every possible normal function you'd need? (Ok, technically Python will let you extend a class with new methods in a kinda hacky way, but that's not very polite.)

# ? Mar 11, 2010 03:28

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 14:04

Captain Capacitor: Jan 21, 2008; The code you say?

tripwire posted:

Heres an example I found on the PIL homepage:
code:
i = Image.open('lena.jpg')
a = numpy.asarray(i) # a is readonly
i = Image.fromarray(a)

This is the way that I do it, except that I use the Numpy array returned by load.

code:

i = Image.open("emot-getout.gif")
pixels = i.load()

assert pixels[0,0] == (255, 255, 255, 0)

# ? Mar 11, 2010 04:08

tripwire: Nov 19, 2004; _{ghost flow}

Captain Capacitor posted:

This is the way that I do it, except that I use the Numpy array returned by load.
code:
i = Image.open("emot-getout.gif")
pixels = i.load()

assert pixels[0,0] == (255, 255, 255, 0)

Thats not exactly anumpy array though, is it? The fact that you can index into it is just some PIL fancyness, but I don't know if thats implemented with numpy at all. Looking at the source for Image.py, its actually a pixel_access object which is defined in the C extension module for image that PIL uses.

# ? Mar 11, 2010 04:18

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Avenging Dentist posted:

NumPy actually provides too many convenience functions on arrays as it is.

God forbid use of a class be convenient.

# ? Mar 11, 2010 21:05

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Jonnty posted:

God forbid use of a class be convenient.

Yeah, from numpy import some_useful_function sure is hard.

# ? Mar 11, 2010 22:30

hey mom its 420: May 12, 2007

Why some people want everything to be a method instead of a plain function I will never understand.

# ? Mar 11, 2010 22:35

Stabby McDamage: Dec 11, 2005; Doctor Rope

Bonus posted:

Why some people want everything to be a method instead of a plain function I will never understand.

I'd say namespace pollution is a big reason.

If I want 20 numpy functions, I either (a) request each of them by name, (b) preface each of them with numpy. (or numpy.linalg. or numpy.somethingelse.), or (c) import *, which fills my namespace with a bazillion things.

Now, plain functions make sense for 2+ variable operations like dot product (nobody wants to see javabortions like a.dotproduct(b)), but for single-input calculations, a.norm() makes sense.

# ? Mar 12, 2010 03:36

tripwire: Nov 19, 2004; _{ghost flow}

I dunno if I agree with that last part. I do agree that its a bit of a pain having to choose between cluttering up your lines with lots numpy.whatever or just importing EVERYTHING (and numpy has a shitload of functions!), but I don't think that conceptually the norm of a vector is an inherent property of the vector. Rather, its something you do to a vector; theres also many different kinds of norms, which means that having it in the base class necessitates making custom subclasses just to use a different norm (unless you cheat like Avenging Dentist points out and just reassign a new function as Vector.norm, which is a bit kludgey).

# ? Mar 12, 2010 04:01

Lonely Wolf: Jan 20, 2003; Will hawk false idols for heaps and heaps of dough.

Stabby McDamage posted:

I'd say namespace pollution is a big reason.

If I want 20 numpy functions, I either (a) request each of them by name, (b) preface each of them with numpy. (or numpy.linalg. or numpy.somethingelse.), or (c) import *, which fills my namespace with a bazillion things.

Now, plain functions make sense for 2+ variable operations like dot product (nobody wants to see javabortions like a.dotproduct(b)), but for single-input calculations, a.norm() makes sense.

The class is a namespace too :mad:

a.norm() is as nonsensical as a.dotproduct(b) :confused:

# ? Mar 12, 2010 04:10

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

tripwire posted:

I dunno if I agree with that last part. I do agree that its a bit of a pain having to choose between cluttering up your lines with lots numpy.whatever or just importing EVERYTHING (and numpy has a shitload of functions!),

Doesn't python have an equivalent of EXPORT_TAGS so you can import groups of functions from one library?

# ? Mar 12, 2010 13:47

tef: May 30, 2004; -> some l-system crap ->

Otto Skorzeny posted:

Doesn't python have an equivalent of EXPORT_TAGS so you can import groups of functions from one library?

from library import foo,bar,baz ?

# ? Mar 12, 2010 14:20

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

tef posted:

from library import foo,bar,baz ?

That's an equivalent of EXPORT_OK, not EXPORT_TAGS; I assume there is a way to do the latter, although it may not be built in or an idiom.

# ? Mar 12, 2010 14:24

Stabby McDamage: Dec 11, 2005; Doctor Rope

Lonely Wolf posted:

The class is a namespace too

a.norm() is as nonsensical as a.dotproduct(b)

The class is one item that can contain many related names, all of which are only accessed through the one class.

a.norm() makes sense because it is something that can be said of the vector. Every vector's got one. Other kinds of norms and one-input functions can have other method names. No additional namespace pollution needed.

a.dotproduct(b) doesn't work as well because it's hammering a operator into a method. Better to overload an operator, or if one doesn't cleanly exist (since there are multiple products and more binary functions than operators), then revert to plain functions, or if you're making a dedicated mathematical language, add operators ("a . b" is dot product in some languages).

Take mod:
100 % 3
For languages that don't have a mod operator (there are a few), you get:
mod(100,3)
Now imagine if that were a method:
100.mod(3)
Gross. What's worse, what if you're using some kind of special numbers in a neurotic language like Java:
BigInteger(100).mod(BigInteger(3))
Double gross.

# ? Mar 12, 2010 14:47

Captain Capacitor: Jan 21, 2008; The code you say?

tripwire posted:

Thats not exactly anumpy array though, is it? The fact that you can index into it is just some PIL fancyness, but I don't know if thats implemented with numpy at all. Looking at the source for Image.py, its actually a pixel_access object which is defined in the C extension module for image that PIL uses.

I thought it was NumPy simply because of the comma syntax. That's something I've not really seen outside of NumPy so I just made the association.

# ? Mar 12, 2010 15:11

ChiralCondensate: Nov 13, 2007; what is that man doing to his colour palette?; Grimey Drawer

Stabby McDamage posted:

Now imagine if that were a method:
100.mod(3)
Gross. What's worse, what if you're using some kind of special numbers in a neurotic language like Java:
BigInteger(100).mod(BigInteger(3))
Double gross.

code:

>>> 100
100
>>> _.__mod__(3)
1

# ? Mar 12, 2010 15:39

BigRedDot: Mar 6, 2008

Stabby McDamage posted:

The class is one item that can contain many related names, all of which are only accessed through the one class.

a.norm() makes sense because it is something that can be said of the vector. Every vector's got one. Other kinds of norms and one-input functions can have other method names. No additional namespace pollution needed.

a.dotproduct(b) doesn't work as well because it's hammering a operator into a method. Better to overload an operator, or if one doesn't cleanly exist (since there are multiple products and more binary functions than operators), then revert to plain functions, or if you're making a dedicated mathematical language, add operators ("a . b" is dot product in some languages).

Take mod:
100 % 3
For languages that don't have a mod operator (there are a few), you get:
mod(100,3)
Now imagine if that were a method:
100.mod(3)
Gross. What's worse, what if you're using some kind of special numbers in a neurotic language like Java:
BigInteger(100).mod(BigInteger(3))
Double gross.

I'm not sure why you spend to much time arguing that a.dot(b) is bad, since everyone already agrees that that is bad. You don't need to convince anyone.

As for a.norm() I don't really find your argument convincing. The context is mathematical programming, I think there is value in hewing to the abstractions that are already extant in applied math fields. A norm is something you do to a vector, not something a vector does to itself. That is, norm(a) (or preferably, euclidean_norm(a), manhattan_norm(a), sup_norm(a), etc.) looks like math, something a mathematician would recognize; a.norm() does not.

# ? Mar 12, 2010 15:41

Captain Capacitor: Jan 21, 2008; The code you say?

BigRedDot posted:

I'm not sure why you spend to much time arguing that a.dot(b) is bad, since everyone already agrees that that is bad. You don't need to convince anyone.

As for a.norm() I don't really find your argument convincing. The context is mathematical programming, I think there is value in hewing to the abstractions that are already extant in applied math fields. A norm is something you do to a vector, not something a vector does to itself. That is, norm(a) (or preferably, euclidean_norm(a), manhattan_norm(a), sup_norm(a), etc.) looks like math, something a mathematician would recognize; a.norm() does not.

Sounds like a possible use case for the factory design pattern to me! :v:

# ? Mar 12, 2010 16:35

Smugdog Millionaire: Sep 14, 2002; 8) Blame Icefrog

Stabby McDamage posted:

a.norm() makes sense because it is something that can be said of the vector. Every vector's got one.

That's not true at all. A vector norm is just a function that takes a vector and produces a real number, with a few qualities like the triangle inequality. There are many common norms. Feel free to invent your own. Try reading http://en.wikipedia.org/wiki/Vector_norm

euclidean_norm(a) vs. a.euclidean_norm() is syntactical bikeshed argument. Which would an actual mathematician right? Neither of course. They'd write ||x||.

EDIT: For your bikeshedding amusement, I submit a.norm(euclidean).

# ? Mar 12, 2010 18:07

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

I was about to ask why raising a negative number to a fractional power raises a ValueError. There's no reason it shouldn't return some kind of value; (-1)^1/3 = e^{ln(-1) * (1/3)} ~= 0.5 + 0.86602j. Maybe because you wouldn't expect an operation involving real numbers to return something complex?

On a hunch, I tried this in Python 3:

code:

Python 3.1.1+ (r311:74480, Nov  2 2009, 15:45:00)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> (-1) ** (1/3)
(0.5000000000000001+0.8660254037844386j)

Nevermind. Cheers, Python 3 :cheers:

# ? Mar 12, 2010 18:33

BigRedDot: Mar 6, 2008

Free Bees posted:

EDIT: For your bikeshedding amusement, I submit a.norm(euclidean).

I'm sure you meant a.norm_factory("euclidean").compute( )

Also let me goon-say myself and say that vectors also don't have to have any norm at all!

# ? Mar 12, 2010 18:38

ShoulderDaemon: Oct 9, 2003; support goon fund; Taco Defender

BigRedDot posted:

Also let me goon-say myself and say that vectors also don't have to have any norm at all!

That's true, although every vector space supports at least trivial seminorms. And on that note, numpy should include a .seminorm() method on every vector that just returns zero.

# ? Mar 12, 2010 18:54

ErIog: Jul 11, 2001

Free Bees posted:

euclidean_norm(a) vs. a.euclidean_norm() is syntactical bikeshed argument. Which would an actual mathematician right? Neither of course. They'd write ||x||.

EDIT: For your bikeshedding amusement, I submit a.norm(euclidean).

I agree with this, and I've never read an argument about one way versus another. I only care that it's loving consistent. It makes code look like garbage to mix them. If Python doesn't want to do across the board consistency, I would be fine with consistency within libraries.

I will admit that I do find a.whatever() easier to read.

# ? Mar 12, 2010 19:08

king_kilr: May 25, 2007

PyPy 1.2 was released! http://morepypy.blogspot.com/2010/03/introducing-pypy-12-release.html

# ? Mar 12, 2010 20:47

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Currently kicking around my head is an idea for a game server (Battlefield: Bad Company 2) administration program. I haven't really done much with threads, but I think I'll need to use them for this project.

This game server sends game events like "X killed Y", "X said Y", "X joined the server". These can be at the rate of 1 every few seconds to like maybe a dozen per second.

Depending on the event, my program will do different things. Log the event to MySQL, send a command to the server, email pictures of cats to random people, whatever. This "doing different things" part is what I think is going to require the use of threads so I don't miss events.

So, how I'm thinking about doing this is like this:

Each incoming event is appended to a list.

I have a loop in another thread that is popping event_list[0] and looking up what action to do with that event in a dictionary which contains {event: action}.

This loop then fires off another thread to do whatever the dict says to do and then goes back to the event_list for the next item.

Any general thoughts or advice?

# ? Mar 12, 2010 22:11

king_kilr: May 25, 2007

Depending on the nature of actions I think an event based application (using eventlet, or twisted or something) would be much preferable to threads, that only really works if your actions tend to be IO bound (network connections, writing to files, etc.).

# ? Mar 12, 2010 22:40

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

king_kilr posted:

Depending on the nature of actions I think an event based application (using eventlet, or twisted or something) would be much preferable to threads, that only really works if your actions tend to be IO bound (network connections, writing to files, etc.).

I'd tend to agree with king_kilr; not that you couldn't do this with threads (you can) but event-based systems, such as gevent/eventlet/greenlet might be a better fit in the long haul. You can skip twisted and stackless though.

# ? Mar 13, 2010 03:50

king_kilr: May 25, 2007

m0nk3yz posted:

I'd tend to agree with king_kilr; not that you couldn't do this with threads (you can) but event-based systems, such as gevent/eventlet/greenlet might be a better fit in the long haul. You can skip twisted and stackless though.

Say what you will about twisted (it has no docs, it has no docs, it has no docs), but it does have tests

# ? Mar 13, 2010 03:58

Stabby McDamage: Dec 11, 2005; Doctor Rope

BigRedDot posted:

I'm not sure why you spend to much time arguing that a.dot(b) is bad, since everyone already agrees that that is bad. You don't need to convince anyone.

Wow, I think I stumbled on the python equivalent of "0.999... = 1". I don't actually care that much. I was mainly responding to Lonely Wolf, who said "a.norm() is as nonsensical as a.dotproduct(b)", which struck me as odd.

# ? Mar 13, 2010 05:39

nbv4: Aug 21, 2002; by Duchess Gummybuns

code:

"{2007}".format({'2007': 'dfdf"})

why...why does this not work? Why can't the .format() function be smart enough to know what to do here?

# ? Mar 13, 2010 09:04

Allie: Jan 17, 2004

nbv4 posted:

code:
"{2007}".format({'2007': 'dfdf"})
why...why does this not work? Why can't the .format() function be smart enough to know what to do here?

I take it you're really trying to do this:

code:

"{0[2007]}".format({'2007': 'dfdf'})

Or maybe this:

code:

"{2007}".format(**{'2007': 'dfdf'})

It looks like numbers are always interpreted as int/long, and never as strings. That is kind of annoying, but on the other hand, something like this isn't even valid syntax:

code:

"{2007}".format(2007=dfdf)

# ? Mar 13, 2010 09:26

nbv4: Aug 21, 2002; by Duchess Gummybuns

Milde posted:

I take it you're really trying to do this:
code:
"{0[2007]}".format({'2007': 'dfdf'})
Or maybe this:
code:
"{2007}".format(**{'2007': 'dfdf'})
It looks like numbers are always interpreted as int/long, and never as strings. That is kind of annoying, but on the other hand, something like this isn't even valid syntax:
code:
"{2007}".format(2007=dfdf)

yeah, I just realized format only takes kwargs and args. It won't accept just a mapping like "%" does... One of the neat things you could do with the mod operator was feed it an object with a custom __getitem__, which allowed you to do all sorts of awesome stuff. If only there was a way to still do that with the new format() syntax...

# ? Mar 13, 2010 13:30

Scaevolus: Apr 16, 2007

Otto Skorzeny posted:

Doesn't python have an equivalent of EXPORT_TAGS so you can import groups of functions from one library?

__all__ is what you want.

http://docs.python.org/tutorial/modules.html#importing-from-a-package

# ? Mar 13, 2010 13:45

Scaevolus: Apr 16, 2007

Thermopyle posted:

Currently kicking around my head is an idea for a game server (Battlefield: Bad Company 2) administration program. I haven't really done much with threads, but I think I'll need to use them for this project.

This game server sends game events like "X killed Y", "X said Y", "X joined the server". These can be at the rate of 1 every few seconds to like maybe a dozen per second.

Depending on the event, my program will do different things. Log the event to MySQL, send a command to the server, email pictures of cats to random people, whatever. This "doing different things" part is what I think is going to require the use of threads so I don't miss events.

So, how I'm thinking about doing this is like this:

Each incoming event is appended to a list.

I have a loop in another thread that is popping event_list[0] and looking up what action to do with that event in a dictionary which contains {event: action}.

This loop then fires off another thread to do whatever the dict says to do and then goes back to the event_list for the next item.

Any general thoughts or advice?

Use Queue.queue instead of lists to pass information between threads.

Your idea sounds fine. Here's a rough sketch of how it might work (untested):

code:

import collections
import Queue
import sys
import thread

thread.stack_size(1024 * 512)  # reduce memory usage by restricting each thread's stack size to 512KB

handlers = collections.defaultdict(list)
event_queue = Queue.Queue()

# register event handlers
def kill_event(killer, victim):
    ...

handlers['kill'] += [kill_event]

# get events
def read_thread():
    # connect to server
    while True:
        # get information, parse into event type and a few values you want

        # event is a string, values is a tuple
        event_queue.put(event, values)

thread.start_new_thread(read_thread, ())

# main loop
while True:
    event, values = event_queue.get()
    for handler in handlers[event]:
        thread.start_new_thread(handler, values)

I use this general architecture in my IRC bot.

# ? Mar 13, 2010 13:59

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

Scaevolus posted:

__all__ is what you want.

http://docs.python.org/tutorial/modules.html#importing-from-a-package

That's a match for EXPORT, rather than EXPORT_TAGS. Meh, different idioms for different languages v :shobon:

# ? Mar 13, 2010 14:04

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

king_kilr posted:

Depending on the nature of actions I think an event based application (using eventlet, or twisted or something) would be much preferable to threads, that only really works if your actions tend to be IO bound (network connections, writing to files, etc.).

m0nk3yz posted:

I'd tend to agree with king_kilr; not that you couldn't do this with threads (you can) but event-based systems, such as gevent/eventlet/greenlet might be a better fit in the long haul. You can skip twisted and stackless though.

Thanks guys. After a cursory look at eventlet and gevent I want to make sure I understand what these things do.

If I understand correctly, these implement routines that while waiting on I/O return control back to let other routines that are also doing I/O to process and thus all routines get to work because all of them spend some portion of their time waiting?

Scaevolus posted:

Use Queue.queue instead of lists to pass information between threads.

Your idea sounds fine. Here's a rough sketch of how it might work (untested):

Thanks! That's another idea to consider...

# ? Mar 14, 2010 00:16

king_kilr: May 25, 2007

Thermopyle posted:

Thanks guys. After a cursory look at eventlet and gevent I want to make sure I understand what these things do.

If I understand correctly, these implement routines that while waiting on I/O return control back to let other routines that are also doing I/O to process and thus all routines get to work because all of them spend some portion of their time waiting?

Thanks! That's another idea to consider...

Yep, you got it!

# ? Mar 14, 2010 08:21

awesomepanda: Dec 26, 2005; The good life, as i concieve it, is a happy life.

Hi, this is just a mental exercise but if you have a function defined with a function lets say
def MathA:
print("2+2")
def MathB:
print("no")

how would you access the function within the first or nested functions?

# ? Mar 15, 2010 18:57

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Ok first off, use code tags, especially in a language where indentation matters:

code:

def MathA:
    print("2+2")
    def MathB:
        print("no")

CrazyPanda posted:

how would you access the function within the first or nested functions?

Second, you wouldn't, and I'm not sure why you think you would. MathB is just like any other local variable.

# ? Mar 15, 2010 18:59

Haystack: Jan 23, 2005

CrazyPanda posted:

how would you access the function within the first or nested functions?

It's not possible using the example you provided (as parsed by Avenging Dentist, anyway). MathB will be locally scoped to MathA, and will only exist while MathA is executing*. If you want to set up a function where MathB is usable outside of MathA, you can:

A) Make MathB the return value for MathA. E.g:

code:

>>> def bing():
...	def bang():
...		print "boom"
...	return bang

>>> a = bing()
>>> a
<function bang at 0x012EDB70>
>>> a()
boom
>>> bang()
Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    bang()
NameError: name 'bang' is not defined

or,

B) Have MathA set an external variable to MathB. This can be ok if MathA is a method and modifies it's object instance. E.g:

code:

class Wiz(Object):
    def bing(self):
   	def bang():
   		print "boom"
   	self.I_am_an_object_property = bang

>>> a = Wiz()
>>> a.bing()
>>> a.I_am_an_object_property()
boom

You can also modify globals if you're feeling brave and/or stupid.

-------
*Plus or minus some garbage collection, technically.

Haystack fucked around with this message at 03:15 on Mar 16, 2010

# ? Mar 16, 2010 03:08

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 14:04

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Haystack posted:

It's not possible using the example you provided (as parsed by Avenging Dentist, anyway).

I just quoted his post, which had the spacing in it, but not rendered because HTML.

# ? Mar 16, 2010 03:41

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »