Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Captain Capacitor: Jan 21, 2008; The code you say?

outlier posted:

OSX10.5, Python 2.5.2.

I've got a working solution, although the root problem is still unclear. I pointed the cgi script directly at the python interpreter in question (i.e. using the full path) and that means most of the other modules are being picked up. (Note: most. And I was certainly calling the same interpreter. Hmmm,)

Haven't found the common link as yet to the non-loading ones, although I have discovered that (logically) modules installed with "setup.py develop" don't load.

code:

>>> import sys
>>> sys.path

Always a good place to start. If you've been mixing setup.py develop and easy_install, check that your .pth files in /Library/Python/2.5/site-packages aren't all munged up.

# ? Jun 17, 2009 23:12

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 07:07

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

pankus posted:

Does anyone have some experience with numpy? I'm trying to copy a 2-dimensional array into a third dimension. I was wondering if there was a more efficient way to do what I am currently doing:
code:
calCCzExp = np.dstack( np.array( self.vinCount*[calCCz] ))
calCCz has dimensions [6,5]; calCCzExp has dimensions [6,5,25].

code:

>>> from numpy import *
>>> foo = array([ [1,2,3], [4,5,6] ])
>>> print foo
[[1 2 3]
 [4 5 6]]
>>> bar = foo.reshape(foo.shape+(1,)).repeat(5,-1)
>>> print bar
[[[1 1 1 1 1]
  [2 2 2 2 2]
  [3 3 3 3 3]]

 [[4 4 4 4 4]
  [5 5 5 5 5]
  [6 6 6 6 6]]]

EDIT: replaced 2nd arg of repeat with -1 for more generic-ness

Avenging Dentist fucked around with this message at 00:51 on Jun 18, 2009

# ? Jun 18, 2009 00:36

tripwire: Nov 19, 2004; _{ghost flow}

Avenging Dentist posted:

code:

>>> from numpy import *
>>> foo = array([ [1,2,3], [4,5,6] ])
>>> print foo
[[1 2 3]
 [4 5 6]]
>>> bar = foo.reshape(foo.shape+(1,)).repeat(5,-1)
>>> print bar
[[[1 1 1 1 1]
  [2 2 2 2 2]
  [3 3 3 3 3]]

 [[4 4 4 4 4]
  [5 5 5 5 5]
  [6 6 6 6 6]]]

EDIT: replaced 2nd arg of repeat with -1 for more generic-ness

Wow, I never knew about this... I wonder how it compares in performance to dstack?

tripwire fucked around with this message at 00:58 on Jun 18, 2009

# ? Jun 18, 2009 00:56

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

tripwire posted:

Wow, I never knew about this... I wonder how it compares in performance?

For the data used by the guy asking the question, it doesn't matter (seriously 6x5x25 is nothing). For larger datasets, you could get theoretically double the speed since the old-n-busted way allocates twice as much memory (one for each member of the Python list and then again for the new array).

# ? Jun 18, 2009 00:58

pankus: Jul 30, 2003

The reason I ask is because that operation runs about 20k times and was taking up 50% of my codes run time. It will eventually be expanded to 220x10x25, which I guess is still pretty small. Thanks for the help, I'll try it out tomorrow.

# ? Jun 18, 2009 02:09

Jo: Jan 24, 2005; Soiled Meat

Silly problem:

I have, in main.py, the following class:

code:

class MyMainMan:
  blahblah
  def doSomethingInMain( self ):
    self.makeFartNoises()

I would like to be able to do the following.
In another file, otherFile.py:

code:


import main

class OtherClass:
  def getFuncy( self ):
    main.MyMainMan.doSomethingInMain()

Is this silliness, or bad technique in Python? Does Python have something like a global singleton class from which I can extend others?

# ? Jun 18, 2009 04:58

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

If you want a static method like you're using, prefix the definition with the @staticmethod decorator.

# ? Jun 18, 2009 05:00

Pie Colony: Dec 8, 2006; I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

I've just started with Python a few days ago, so can anyone take a look at my code and see if my code's alright? The reason I ask is when I wrote my first Python program I was criticized for writing it too "c-style" and not using the features Python had to offer, so I want to know I'm doing everything efficiently. It's not that big, really.

http://pastebin.com/m790e96f4

e: for reference, IRC sends messages like so:

:origin COMMAND (maybe optional parameters) :text

So if there's an easier way to parse them...

Pie Colony fucked around with this message at 05:19 on Jun 18, 2009

# ? Jun 18, 2009 05:15

Jo: Jan 24, 2005; Soiled Meat

Avenging Dentist posted:

If you want a static method like you're using, prefix the definition with the @staticmethod decorator.

Exactly what I'm looking for. Thank you so much.

# ? Jun 18, 2009 05:16

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Pie Colony posted:

http://pastebin.com/m790e96f4

You could replace the whole loop in your recv_msg function with str.split. (Or splitlines if you don't mind playing fast and loose with line terminators.)

# ? Jun 18, 2009 05:33

Pie Colony: Dec 8, 2006; I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

Avenging Dentist posted:

You could replace the whole loop in your recv_msg function with str.split. (Or splitlines if you don't mind playing fast and loose with line terminators.)

I originally used str.split, but I ran into a little problem. Suppose I used '\r\n' as the delimiter, which is what all lines end with.

If I had a packet like

:server CMD :dothis\r\n :server CMD :dothat\r\n :serv

as a packet, and I split it, I don't know how I can verify if that last command (which didn't get transferred entirely through this particular packet) was complete or not.

# ? Jun 18, 2009 05:38

Avenging Dentist: Oct 1, 2005; oh my god is that a circular saw that does not go in my mouth aaaaagh

Pie Colony posted:

I don't know how I can verify if that last command (which didn't get transferred entirely through this particular packet) was complete or not.

code:

>>> foo = 'foo\r\nbar\r\nbaz\r\n'
>>> foo.split('\r\n')
['foo', 'bar', 'baz', '']
>>> bar = 'foo\r\nbar\r\nbaz'
>>> bar.split('\r\n')
['foo', 'bar', 'baz']

Pay particular attention to the last element of the returned lists.

# ? Jun 18, 2009 05:41

tripwire: Nov 19, 2004; _{ghost flow}

I've never made an irc client before, but wouldn't you want to avoid any while loops that don't use time.sleep? Unless you want to absolutely hammer any server you are attempting to connect to I guess.

Python is known for having short and sweet idioms for many things that require cumbersome boilerplate code in c or java. Whenever you catch yourself righting a c style (initialization, loop test, increment) loop you should try to think of how you can take advantage of python's for each loop. Builtin functions like zip and split are both useful helper functions, and you can use them with map and filter to get a lot done with one statement. Also try to take advantage of list comprehensions or generator statements as well. They have a few subtle differences but for many applications, which style you use is up to you. Certain people find certain things easier using list comprehensions, others might prefer to build up lists manually with while loops. If you just want to do a loop for X iterations, use "for index in xrange(X):".

My advice is try to keep things under 80 columns, because it makes it easier to read. You don't have to put brackets around a bunch of terms when assigning tuples, but it doesn't hurt. However, you can take advantage of list brackets and tuple parentheses to break up long statements on multiple lines. I do this because it makes it easier to read. I went through your code and tried to change things to be closer to how I might have done it.

code:

import socket

def main( server, port, nick, ident, name, owner, channel ):

    my_socket = socket.socket()
    my_socket.connect((server, port))
    my_socket.send(('nick ' + nick + '\r\n').encode())
    my_message = 'USER %s %s %s :%s\r\n' % (ident, server, server, name)
    my_socket.send(my_message.encode())

    print('Connecting...')
    
    partial_message = ''

    #this function will return True if a string contains the substring MODE
    contains_MODE = (lambda msg: msg.find('MODE') != -1)

    while True:
        packet = my_socket.recv(500)
        message_list = packet.split('\r\n')
        message_list[0] = partial_message + message_list[0]
        if message_list[-1] != '':
            partial_message = message_list[-1]
        else:
            partial_message = ''
        
        if len( filter(contains_MODE, msglist) ) > 0:
                break
                

    print('Connected')
    
    my_socket.send( ('JOIN %s\r\n' % channel).encode() )    

    while True:    
        pass

if __name__ == "__main__":
    arguments =  ('irc.synirc.org', 6667, 'testtt',
                 'testtt', 'testtt', 'rtab', '#dragoonica')
    main(*arguments) 
    # The * in front of arguments is for "unpacking" the tuple into a bunch of 
    #variables. This way you are making it explicit what resources the function
    # depends on, which makes things easier to maintain.

I also replaced the string concatenation with interpolation. This is also usually a personal preference thing, but when you find yourself doing a lot of + ' ' + varable + ', ' + etc etc, its usually much less of a hassle to just do it printf style.

tripwire fucked around with this message at 06:44 on Jun 18, 2009

# ? Jun 18, 2009 06:24

ahobday: Apr 19, 2007

This might be slightly off topic, but since it's for Python I figure it'd be OK. I want to set up a virtual Linux installation on this Windows PC so I can play around with pycurses development. I want it to have the necessary installations for Python stuff, but not be bogged down by anything else.

Does anyone else do this, and what distro would you recommend?

# ? Jun 18, 2009 11:45

deedee megadoodoo: Sep 28, 2000; Two roads diverged in a wood, and I, I took the one to Flavortown, and that has made all the difference.

CentOS has a server distribution that fits on a single CD. It doesn't include x windows so you should probably be comfortable with the command line before you download it.

# ? Jun 18, 2009 11:49

tef: May 30, 2004; -> some l-system crap ->

Centipeed posted:

This might be slightly off topic, but since it's for Python I figure it'd be OK. I want to set up a virtual Linux installation on this Windows PC so I can play around with pycurses development. I want it to have the necessary installations for Python stuff, but not be bogged down by anything else.

Does anyone else do this, and what distro would you recommend?

andLinux? you can run it within windows and it plays reasonably nicely

# ? Jun 18, 2009 13:39

Farrok: May 29, 2006

I've never messed with programming something that access a network and I wanted to try something like that out. All I really want to do is automatically fill in forms. However, I would like to have my program automatically log in to the site that these forms are on.

I've found a couple promising modules, such as Mechanize or twill, but I'm wondering if these are functionally equivalent or serve different purposes. Also, what do I need to know about security issues? Will these essentially use the server's SSL or other security measures during login or is it possible to open yourself to a giant security vulnerability if you don't know what your doing when your program sends such data as login info to a website?

# ? Jun 18, 2009 14:41

tef: May 30, 2004; -> some l-system crap ->

There are always things like coscripter as well, http://coscripter.research.ibm.com/coscripter/browse/

on the whole scraping is awful and I would avoid it.

# ? Jun 18, 2009 14:44

Scaevolus: Apr 16, 2007

Pie Colony posted:

I originally used str.split, but I ran into a little problem. Suppose I used '\r\n' as the delimiter, which is what all lines end with.

If I had a packet like

:server CMD :dothis\r\n :server CMD :dothat\r\n :serv

as a packet, and I split it, I don't know how I can verify if that last command (which didn't get transferred entirely through this particular packet) was complete or not.

code:

if '\r\n' in buffer:
  line, buffer = buffer.split('\r\n', 1)
out_queue.push(line)

here's (paraphrased) what I do to split the params:

code:

import re

irc_prefix_rem = re.compile(r'(.*?) (.*?) (.*)').match
irc_noprefix_rem = re.compile(r'()(.*?) (.*)').match
irc_netmask_rem = re.compile(r':?([^!@]*)!?([^@]*)@?(.*)').match
irc_param_ref = re.compile(r'(?:^|(?<= ))(:.*|[^ ]+)').findall

while True:
    msg = self.conn.iqueue.get()
    if msg.startswith(":"): #has a prefix
        prefix, command, params = irc_prefix_rem(msg).groups()
    else:
        prefix, command, params = irc_noprefix_rem(msg).groups()
    nick, user, host = irc_netmask_rem(prefix).groups()
    paramlist = irc_param_ref(params)
    lastparam = ""
    if paramlist and paramlist[-1].startswith(':'):
        lastparam = paramlist[-1][1:]
    self.out.put([msg, prefix, command, params, nick, user, host,
        paramlist, lastparam])

I could probably trim those regexes down a ton... hmmm...

# ? Jun 19, 2009 01:43

LuckySevens: Feb 16, 2004; fear not failure, fear only the limitations of our dreams

tef posted:

andLinux? you can run it within windows and it plays reasonably nicely

32bit only though.

# ? Jun 19, 2009 03:38

Jo: Jan 24, 2005; Soiled Meat

Any guesses why this is happening?

code:

(main.Game)
	@classmethod
	def getTimeDelta( this ):
		return this.timeDelta

(Entities.Actor)
	def update( self ):
		timeDelta = main.Game.getTimeDelta()
		print( "Time delta in update: %d" % (timeDelta) )
		self._updateImage_( timeDelta )
		self._updatePosition_( timeDelta, main.Game.getMap() )

Output:

pre:

Main: Time delta: 47 -- Now: 7759
Player: Time delta in update: 0
Guard: Time delta in update: 0
Main: Time delta: 21 -- Now: 7780
Player: Time delta in update: 0
Guard: Time delta in update: 0
Main: Time delta: 18 -- Now: 7798
Player: Time delta in update: 0
Guard: Time delta in update: 0
Main: Time delta: 43 -- Now: 7841

Edit:

code:

	@classmethod
	def getTimeDelta( this ):
		if this.clock == None:
			print( "Clock is null." )
			return 0
		else:
			print( this.clock.get_time() )
			return this.clock.get_time()

Output:

pre:

Clock is null.
Clock is null.
28
Clock is null.
Clock is null.
23
Clock is null.
Clock is null.
42
Clock is null.
Clock is null.
30
Clock is null.
Clock is null.

B-b-but why?

EDIT again: Solution found:
http://forums.somethingawful.com/showthread.php?threadid=3145796&userid=0&perpage=40&pagenumber=8#post362263110

Jo fucked around with this message at 05:49 on Jun 20, 2009

# ? Jun 20, 2009 00:26

eighty8: Jan 24, 2007

I have a bit of a stupid question but please bare with me as I am python newb...

How does one save a value from a generator object?

I am using twill to access a webpage and view all the forms on it, these forms are viewed by twill as a generator object. What I need to be able to do though is save some of the data from the forms to a spreadsheet, and to make some decisions based on other elements.

Any advice?

# ? Jun 20, 2009 01:07

hlfrk414: Dec 31, 2008

Have you tried appending your values to a list? That's the general way to save values.

# ? Jun 20, 2009 01:26

eighty8: Jan 24, 2007

Will that append the values individually or will it just be one big element in a list?

# ? Jun 20, 2009 01:30

hlfrk414: Dec 31, 2008

That depends on how you do it. I would suggest trying it and then coming back if you have problems. Here's some documentation if you need it.

# ? Jun 20, 2009 01:36

eighty8: Jan 24, 2007

hlfrk414 posted:

That depends on how you do it. I would suggest trying it and then coming back if you have problems. Here's some documentation if you need it.

Thanks for the link I will give it a go as soon as I get home.

# ? Jun 20, 2009 01:40

good jovi: Dec 11, 2000; 'm pro-dickgirl, and I VOTE!

eighty8 posted:

I have a bit of a stupid question but please bare with me as I am python newb...

How does one save a value from a generator object?

I am using twill to access a webpage and view all the forms on it, these forms are viewed by twill as a generator object. What I need to be able to do though is save some of the data from the forms to a spreadsheet, and to make some decisions based on other elements.

Any advice?

I presume you're using twill.commands.showforms(). Just call list() on the return value. I don't know why Titus used a generator there, just makes things harder to inspect. In twill trunk, and in the never-actually-going-to-be-released 0.9.2 release, showforms and some other similar function will return lists instead.

# ? Jun 20, 2009 17:11

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

I need to write an app that will interact over a network with a MySQL database. Problem is, I don't really know much about MySQL or how to use it. I've found a few Python modules for MySQL, but I have so little background in this sort of thing that I don't really know where to start. For example, can an app directly query a database over a network, or do I need to send requests to a second, serverside Python script that'll query the database and send the results back over the network?

If anyone can at least point me in the right direction, I'd be very grateful.

# ? Jun 20, 2009 19:42

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

BeefofAges posted:

I need to write an app that will interact over a network with a MySQL database. Problem is, I don't really know much about MySQL or how to use it. I've found a few Python modules for MySQL, but I have so little background in this sort of thing that I don't really know where to start. For example, can an app directly query a database over a network, or do I need to send requests to a second, serverside Python script that'll query the database and send the results back over the network?

If anyone can at least point me in the right direction, I'd be very grateful.

Use MySQLdb, let the server listen on a public network address, and then set up a user account that will accept connections from wherever the Python app is. Here's a good tutorial for connecting to MySQL from Python.

# ? Jun 20, 2009 19:55

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Janin posted:

Use MySQLdb, let the server listen on a public network address, and then set up a user account that will accept connections from wherever the Python app is. Here's a good tutorial for connecting to MySQL from Python.

This is great, thank you.

# ? Jun 20, 2009 20:53

Aeolius: Jul 16, 2003; Simon Templeman Fanclub

Does anyone here happen to have experience with the mechanize module? I'm finding the documentation to be a little lacking, which wouldn't be terrible in and of itself, but I also can't seem to find active discussion about it anywhere on the internet. Two questions for now (though more may follow!):

Is there a way to retrieve the cookies stored in an instance of Browser(), either individually or as a cookiejar?

How can one effectively use the select_form() method if the forms on a given page do not have 'name' attributes? I assume the 'predicate' argument plays a role, here, but I can't find a clear explanation of how it works or what it is or how to use it.

Aeolius fucked around with this message at 07:22 on Jun 21, 2009

# ? Jun 21, 2009 07:16

duck monster: Dec 15, 2004

Actually I'd strongly recomend not letting your mysql listen on a public port. Some dudes gunna scan it down and brutalize it.

If you must, set up some sort of SSH tunnel or something.

Or better still just put the server on the same box as the machine and only accept local connections.

# ? Jun 21, 2009 10:21

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

duck monster posted:

Actually I'd strongly recomend not letting your mysql listen on a public port. Some dudes gunna scan it down and brutalize it.

If you must, set up some sort of SSH tunnel or something.

Or better still just put the server on the same box as the machine and only accept local connections.

This is all going to be on an internal network. It won't be accessible on the internet. It needs to be networked because this app is going to run on several hundred machines, all talking to one database.

# ? Jun 21, 2009 11:04

Crazak P: Apr 11, 2003; PUNISHER > SPIDERMAN

Quick question: I'm on windows xp using python 2.5.1 and when I do an 'os.path.sep' it returns "\\". How do I fix this?

# ? Jun 21, 2009 12:42

hey mom its 420: May 12, 2007

That's normal behavior. In order for \ to be interpreted as an actual character instead of an escape sequence, it has to be escaped with, well, \. When you're doing Windows paths in Python, they always look like "C:\\blah\\blah", or you can prefix your string literals with r, then \ isn't treated as an escape sequence, so you can also do r"C:\blah\blah"

# ? Jun 21, 2009 13:18

Crazak P: Apr 11, 2003; PUNISHER > SPIDERMAN

Thanks. That makes sense. I was stupid and tried to just concatenate '\' to my path and realized that it's the character for line break in python.

Crazak P fucked around with this message at 13:30 on Jun 21, 2009

# ? Jun 21, 2009 13:25

duck monster: Dec 15, 2004

Im starting to fill with hate and despair about the lovely state of python on the mac compared with linux and windows (where module installation generally 'just works')

anyone got any idea how the gently caress to get igraph to install. loving things got me flumoxed. Theres an installer for it on the site, but I think it just goes off and has a beer instead of installing anything

:sigh:

# ? Jun 21, 2009 13:32

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

Im starting to fill with hate and despair about the lovely state of python on the mac compared with linux and windows (where module installation generally 'just works')

anyone got any idea how the gently caress to get igraph to install. loving things got me flumoxed. Theres an installer for it on the site, but I think it just goes off and has a beer instead of installing anything

poo poo works for me fine on my mac; I have no problems compiling stuff. I don't know about igraph - I looked at it, and the installers seems to be against the system installer, and not the framework builds python.org ships - however it is only python 2.5. Are you using the system python?

# ? Jun 21, 2009 14:25

duck monster: Dec 15, 2004

I think so.

Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

# ? Jun 21, 2009 14:38

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 07:07

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

I think so.

Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

You should see this:

Python 2.5.1 (r251:54863, Nov 12 2008, 17:08:51)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Are you not on leopard?

# ? Jun 21, 2009 17:23

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »