Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Scaevolus: Apr 16, 2007

Did you set your PYTHONDOCS environment variable?

# ? Sep 13, 2008 04:32

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 09:58

Kire: Aug 25, 2006

Scaevolus posted:

Did you set your PYTHONDOCS environment variable?

I thought I did that through the menu bar, "Configure IDLE", and then on the General tab I chose the index html file from \Doc.

# ? Sep 13, 2008 04:36

Pivo: Aug 20, 2004

Hi guys

For one of my CS classes at school we're working with MIPS assembly on a MIPS simulator.

I wrote my own assembler in Python; it's very simple...
basically many of these:

code:

	if (code.startswith("add")):
		d = int(nuggets[0])
		s = int(nuggets[1])
		t = int(nuggets[2])
		instr = 32 + (d << 11) + (t << 16) + (s << 21)

followed by:

code:

	sys.stdout.write(struct.pack(">i", instr))

Sometimes instr is a value like 0xffff000c. This fits in 32 bits. But Python interprets it as signed and makes it a long.

In Python 2.5.1, the assembler works, but I get this warning:
masm.py:109: DeprecationWarning: struct integer overflow masking is deprecated
sys.stdout.write(struct.pack(">i", instr))

In Python 2.4.2, the assembler crashes on the same line with:
OverflowError: long int too large to convert to int

How do I fix this...? (I should note that "write an assembler in Python" is not part of my coursework)

Pivo fucked around with this message at 19:59 on Sep 13, 2008

# ? Sep 13, 2008 19:55

Scaevolus: Apr 16, 2007

Pivo posted:

In Python 2.5.1, the assembler works, but I get this warning:
masm.py:109: DeprecationWarning: struct integer overflow masking is deprecated
sys.stdout.write(struct.pack(">i", instr))

code:

$ pydoc struct
       ...
    The remaining chars indicate types of args and must match exactly;
    these can be preceded by a decimal repeat count:
     x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;
     h:short; H:unsigned short; i:int; [u][b]I:unsigned int;[/b][/u]
     l:long; L:unsigned long; f:float; d:double.

# ? Sep 13, 2008 21:23

Pivo: Aug 20, 2004

Scaevolus posted:

code:

$ pydoc struct
       ...
    The remaining chars indicate types of args and must match exactly;
    these can be preceded by a decimal repeat count:
     x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;
     h:short; H:unsigned short; i:int; [u][b]I:unsigned int;[/b][/u]
     l:long; L:unsigned long; f:float; d:double.

Doh!

Thank you.

# ? Sep 13, 2008 21:40

ATLbeer: Sep 26, 2004; Über nerd

Treading into a territory I've never been in and am having a hard time Googling up what I need here.

I'm trying to 'monitor' the real-time quote information from Yahoo Finance and they have quite a clever system of getting to the browser.

So if you go here: http://finance.yahoo.com/q?s=wb

(If it's during US market hours) You'll notice the prices update in 'real-time' and not on a constant polling basis. This is because Yahoo is pushing the data to the page with a LONG open HTTP connection that is connected to a push (maybe Comet) server.

Here's the URL to the connection
long url

Watching Firebug it looks like Yahoo's push server occasionally sends down the updates and the JS on the page handles the rest. How do I hold open this HTTP connection and use the data coming down the pipe here?

ATLbeer fucked around with this message at 17:13 on Sep 18, 2008

# ? Sep 18, 2008 16:29

tef: May 30, 2004; -> some l-system crap ->

ATLbeer posted:

Treading into a territory I've never been in and am having a hard time Googling up what I need here.

You could use pycurl to open the connection, and it takes a call back

# ? Sep 18, 2008 17:48

Jo: Jan 24, 2005; Soiled Meat

I'm going image processing beyond the capabilities of the Python Imaging Library. I'd use OpenCV with Python wrappers, but I'm hoping to get more familiar with the techniques and algorithms by writing my own versions. Near as I can tell, NumPy is the only real 'quick' solution for dealing with 2D arrays in Python. Before I continue onward with my project, NumPy is the 'official' or most common way of dealing with large 2D arrays, correct?

# ? Sep 21, 2008 03:38

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

Jo posted:

I'm going image processing beyond the capabilities of the Python Imaging Library. I'd use OpenCV with Python wrappers, but I'm hoping to get more familiar with the techniques and algorithms by writing my own versions. Near as I can tell, NumPy is the only real 'quick' solution for dealing with 2D arrays in Python. Before I continue onward with my project, NumPy is the 'official' or most common way of dealing with large 2D arrays, correct?

Yes. The history of arrays in Python is tangled, but originally there was "array" in the standard library, but that was broken. Then there was Numeric Python, which dissolved into infighting after a while and forked numarray. Then they all made up and we ended up with Numpy. And that's what everyone uses.

The only problems you may encounter (and this depend on what you are trying to do):

* You have to compile Numpy for your platform. (The install does this, but the point is it's not pure python)

* To a large extent, Numpy is all about fixed-size arrays. You can reshape an array, but (say) adding a column or row is an rear end.

* The documentation is a bit skimpy, but you can refer to the Numeric python docs for a lot of pointers.

# ? Sep 21, 2008 09:00

Tomed2000: Jun 24, 2002

Just starting to learn Python recently and I have a quick newbie question about string formatting when it comes to left and right justifying.

Here is an example of the output that I'm having trouble with:

code:

Name	           Id     Hws     Quizzes     Exams     Total    Grade
----------------------------------------------------------------------
John Doe     7871
George Bush, Jr.     7766
Jim Bob     3333
Jane Doe     3689

It's obvious already but I just want to line up the numbers in the ID column. I've tried several variations of ljust, rjust and different values with string formatting but I can't get it to work. This is the line of code I'm working with:

code:

print '%-8s %8s' % (name, id)

What exactly do I have to use to get the id column aligned?

# ? Sep 21, 2008 16:00

Pivo: Aug 20, 2004

Use ljust to pad the 'name' field to the width of the 'name' column. Also your output needs to be displayed in fixed-width font but I'm sure you knew that.

# ? Sep 21, 2008 19:25

Tomed2000: Jun 24, 2002

Pivo posted:

Use ljust to pad the 'name' field to the width of the 'name' column. Also your output needs to be displayed in fixed-width font but I'm sure you knew that.

Wow, that was simple. Thanks.

# ? Sep 21, 2008 19:40

Moof Strydar: Jul 13, 2001

Here's a dumb one for y'all. Say I have a list of tuples as such:

code:

mylist = [(0x20, 40),
          (0x70, 6),
          (0x7B, 6),
          (0x04, 0x01, 3),
          (0x70, 10),
          ...]

Is there a way I can search through the list in place, trying to match a given value (such as 0x70) to the each tuple's first value, without making a copy (i.e. using slices)?

# ? Sep 26, 2008 12:44

hey mom its 420: May 12, 2007

Use generator expressions

code:

>>> mylist = [(0x20, 40), (0x70, 6), (0x7B, 6), (0x04, 0x01, 3), (0x70, 10)]
>>> matching_tuples = (a for a in mylist if a[0] == 0x70)
>>> for x in matching_tuples: print x
...
(112, 6)
(112, 10)

# ? Sep 26, 2008 13:24

Moof Strydar: Jul 13, 2001

Nice, I haven't used generators much. However, I didn't explain myself properly (or at all, really): I'm trying to make a "Find next" dialog, which searches from the currently selected item in a list control, so I only want to find the index of the next instance of the value, not all the values.

I think the way I've written it, the generator would end up searching from the start of the list and skipping everything generated until after the given starting index...

Anyway, I realized I can use something along the lines of boring old C-style 'for' loops. I'm a colossal dunderhead tonight. :shobon:

# ? Sep 26, 2008 13:44

bitprophet: Jul 22, 2004; Taco Defender

Moof Strydar posted:

Nice, I haven't used generators much. However, I didn't explain myself properly (or at all, really): I'm trying to make a "Find next" dialog, which searches from the currently selected item in a list control, so I only want to find the index of the next instance of the value, not all the values.

I think the way I've written it, the generator would end up searching from the start of the list and skipping everything generated until after the given starting index...

Anyway, I realized I can use something along the lines of boring old C-style 'for' loops. I'm a colossal dunderhead tonight.

Read up on Python iterators/iteration, specifically the next() method, or manual generators and the yield statement. In other words, you can get the same benefits as a generator expression, but without doing it all at once, instead calling next/yield when your code needs the next item.

# ? Sep 26, 2008 15:29

tef: May 30, 2004; -> some l-system crap ->

Python + LDAP + Active Directory is making me cry.

I'm trying to get it to authenticate, and I am either being told the credentials are wrong, or that a successful bind must be completed.

This is some of the crap code I have cobbled together trying to get python-ldap to work.

code:

class ActiveDirectory:
    def __init__(self, url,  binddn, password, basedn):
        self.url = url
        self.binddn = binddn
        self.password = password
        self.basedn  = basedn
        
    def auth (self, username, password):
        l = ldap.initialize(self.url,0) 
        #l.set_option(ldap.OPT_PROTOCOL_VERSION, 3)
        
        print "binding %s %s"%(self.binddn,self.password)
        print l.simple_bind_s(self.binddn,self.password)
        
        print "bound"
    
        
        print "searching for %s"%username
        
        print "basedn=%s"%self.basedn
        
        result = l.search_ext_s(self.basedn,ldap.SCOPE_SUBTREE,"sAMAccountName=%s" % username,['dn'])[0][1]
        
        print result
        l.unbind_s()
                
        l.simple_bind_s(result['dn'][0], password)
        
        result = l.search_ext_s(self.basedn,ldap.SCOPE_SUBTREE,"sAMAccountName=%s" % username,['mail','givenName','sn','sAMAccountName'])[0][1]
        
        print result
        
        l_client.unbind_s()

So far I have tried it on linux, windows, with a full DN, user@domain.co.uk, and I can't seem to authenticate *enough* to be able to perform a search.

Help!

Edit:
AHAHAHAHAHAHAAH

code:

l.set_option(ldap.OPT_REFERRALS, 0)

tef fucked around with this message at 17:25 on Sep 26, 2008

# ? Sep 26, 2008 16:00

hey mom its 420: May 12, 2007

Yeah, that's perfect for the next method of generators.

code:

>>> mylist = [(0x20, 40), (0x70, 6), (0x7B, 6), (0x04, 0x01, 3), (0x70, 10)]
>>> matching_tuples = (a for a in mylist if a[0] == 0x70)
>>> matching_tuples.next()
(112, 6)
>>> matching_tuples.next()
(112, 10)
>>> matching_tuples.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

# ? Sep 26, 2008 16:13

Moof Strydar: Jul 13, 2001

Bonus posted:

Yeah, that's perfect for the next method of generators.

But is there a way to start the generator from a specific point in mylist, without using slices? As the user will be searching from any point in the list (the currently selected item), and they can change the value they're searching for at any time, a generator seems inefficient for two reasons:

1) The generator will have to be created whenever the search value is changed (not really a problem), and
2) The generator will have to search through the entire list from the beginning, and just ignore the yields before the currently selected item.

# ? Sep 27, 2008 03:41

JoeNotCharles: Mar 3, 2005; Yet beyond each tree there are only more trees.

Moof Strydar posted:

But is there a way to start the generator from a specific point in mylist, without using slices?

code:

matching_tuples = (mylist[i] for i in range(current_index, len(mylist)) if mylist[i][0] == 0x70)

# ? Sep 27, 2008 05:16

Moof Strydar: Jul 13, 2001

Awesome, that looks like it will work, although I'll use xrange instead! Thanks JoeNotCharles, Bonus and bitprophet!

EDIT: vvv That looks funky too! vvv

Moof Strydar fucked around with this message at 13:09 on Sep 27, 2008

# ? Sep 27, 2008 10:29

hey mom its 420: May 12, 2007

Also a great way to do it would be to use lazy slicing from the itertools module.

code:

>>> import itertools
>>> mylist = [(0x20, 40), (0x70, 6), (0x7B, 6), (0x04, 0x01, 3), (0x70, 10)]
>>> index_to_search_from = 2
>>> sliced = itertools.islice(iter(mylist), index_to_search_from, None)
>>> matching_tuples = (a for a in sliced if a[0] == 0x70)
>>> matching_tuples.next()
(112, 10)
>>> matching_tuples.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

No copies of the list are made and everything is wonderfully lazy.

# ? Sep 27, 2008 12:03

Kire: Aug 25, 2006

I read "A byte of python" and "thinking like a computer scientist," two of the intro-to-python docs recommended on the main website. Does anybody have further reading recommendations for someone still getting into programming and Python? Some simple projects I should try to do?

# ? Sep 27, 2008 22:18

duck monster: Dec 15, 2004

Kire posted:

I read "A byte of python" and "thinking like a computer scientist," two of the intro-to-python docs recommended on the main website. Does anybody have further reading recommendations for someone still getting into programming and Python? Some simple projects I should try to do?

Grab a copy of Django and gently caress around with that.

# ? Sep 28, 2008 04:30

bitprophet: Jul 22, 2004; Taco Defender

duck monster posted:

Grab a copy of Django and gently caress around with that.

Not a bad idea (as long as you use the built-in "runserver" -- unless you're already a web dev, don't get bogged down with installing Apache or anything), doing the Django tutorial and then making a simple site will expose you to a lot of real-world Python very quickly

# ? Sep 28, 2008 05:27

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

bitprophet posted:

Not a bad idea (as long as you use the built-in "runserver" -- unless you're already a web dev, don't get bogged down with installing Apache or anything), doing the Django tutorial and then making a simple site will expose you to a lot of real-world Python very quickly

Apache deployment may cause your hair to turn grey or fall out. I'm almost positive my Dr. needs to put me on blood pressure meds after I fought with it on OS/X.

# ? Sep 28, 2008 23:01

Kire: Aug 25, 2006

I'll check that out.

At the bottom of last page/top of this page, I asked about getting IDLE's help docs to work. I installed python 3.0 and used IDLE, and the help() seemed to work right from the start. Very useful. I still don't understand why it has to be so difficult to get the program's own help() command to work, it should come ready to use when it's installed.

# ? Sep 28, 2008 23:08

duck monster: Dec 15, 2004

Getting psycopg2 postgresql library working on OS/X is an utter slut of a task. Apple really needs to provide canonical packages for fundamental stuff like this.

# ? Sep 28, 2008 23:08

bitprophet: Jul 22, 2004; Taco Defender

See, I just DO NOT GET why everyone has this huge stiffie for developing (sans runserver and the builtin Python, of course -- which is fine for a newbie) on a local OS X workstation. It's loving retarded -- you have to wrestle with 3 or 4 different potential methods of installing the various components and then you end up with 2 versions of just about everything (the Apple default and the new one you installed because the Apple default is too loving old or lovely).

I do all my dev on standalone Linux servers, or VMWare Fusion running the same. aptitude install libapache2-mod-python python-psycopg2 postgresql-server and those (and their automatically installed dependencies) get me 95% of the way there, no fuss, no muss. An svn checkout of Django and I'm pretty much all done.

Just kills me that people put so much effort into Mac-native dev; my office is largely a Rails shop and they do the same thing, drives me bonkers.

/derail

bitprophet fucked around with this message at 00:27 on Sep 29, 2008

# ? Sep 29, 2008 00:25

duck monster: Dec 15, 2004

Because I have a mac in front of me , and not a linux or windows box. v :shobon:

v

But yes, the state of python on the mac is shameful at the moment. The internet is full of people saying "Use MacPython its the unofficial standard". No, theres an official standard that apple supplies, and you break your computers head if you dont

Now I have Apple python, MacPython, ports python and Fink python all due to a 3 day long quest to get loving psycopg2 installed. End solution;- Reverted to the apple python (so xcode isnt broken) modify the poo poo out of the compile script and rebuild psycopg2 from a tarball.

Also the postgresql install shat itself so I had to manually install that too :smithicide:

duck monster fucked around with this message at 03:52 on Sep 29, 2008

# ? Sep 29, 2008 03:48

bitprophet: Jul 22, 2004; Taco Defender

duck monster posted:

Because I have a mac in front of me , and not a linux or windows box. vv

THAT'S NO EXCU--wait, maybe it is

My condolences. I do seriously recommend VMWare Fusion, though -- if you've got the RAM to do local web development, you've got the ram to drop 256-512 megs in a Linux VM. It's so much nicer...

Sorry to hear about your psycopg2 woes, I don't think I've ever actually set that up on OS X myself so I can't offer any advice. I tend to use Macports for everything that I do need locally, though (generally shell related stuff for non-Web Python scripting, vim, wireshark, etc) and it's always worked pretty well. Not sure why it didn't work out in your case.

# ? Sep 29, 2008 05:09

Git: Aug 21, 2004

Kire posted:

I read "A byte of python" and "thinking like a computer scientist," two of the intro-to-python docs recommended on the main website. Does anybody have further reading recommendations for someone still getting into programming and Python? Some simple projects I should try to do?

You might want to check out some of the stuff on iTunes U. I've had some people check out a couple of Stanford lectures on Python and come away happy, though I'm pretty sure they do require a fair amount of prior programming knowledge.

# ? Sep 29, 2008 09:59

tbradshaw: Jan 15, 2008; First one must nail at least two overdrive phrases and activate the tilt sensor to ROCK OUT!

duck monster posted:

But yes, the state of python on the mac is shameful at the moment. The internet is full of people saying "Use MacPython its the unofficial standard". No, theres an official standard that apple supplies, and you break your computers head if you dont

Well, part of being on OSX is that you have a full UNIX workstation at your finger tips. You should install MacPython, but do so in your home directory. I'd recommend creating a "local" directory in your home directory, and using that as the root for all of your personal software installations.

After installing python in your home directory, liberal use of virtualenv and easy_install should make things pretty trivial.

# ? Sep 29, 2008 15:23

duck monster: Dec 15, 2004

tbradshaw posted:

Well, part of being on OSX is that you have a full UNIX workstation at your finger tips. You should install MacPython, but do so in your home directory. I'd recommend creating a "local" directory in your home directory, and using that as the root for all of your personal software installations.

After installing python in your home directory, liberal use of virtualenv and easy_install should make things pretty trivial.

Except that literally every unix on the market except for the mac has a coherent policy on package management. I love the mac, but its package management has more in common with loving slackware than anything modern. Download tarball, compile and pray the gods will be benevolant. And no, easy_install will usually light on fire the minute its got to compile something and it becomes apparent that Apple puts things in different places to bsd's or SysV's. Reminder there are about 5 distros of python out there for the mac, none of which are compatible.

# ? Sep 29, 2008 15:32

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

duck monster posted:

Except that literally every unix on the market except for the mac has a coherent policy on package management. I love the mac, but its package management has more in common with loving slackware than anything modern. Download tarball, compile and pray the gods will be benevolant. And no, easy_install will usually light on fire the minute its got to compile something and it becomes apparent that Apple puts things in different places to bsd's or SysV's. Reminder there are about 5 distros of python out there for the mac, none of which are compatible.

Uh, my mac works great for me for installing python things - it's all I develop on. Also, if you're on leopard: You should not be installing any other python distribution other than the one it came with, unless it's to your home directory or some /opt location. Thou shalt not change the system python!

And just to add - virtualenv works great too for keeping things in local sandboxes and out of the main framework directories.

m0nk3yz fucked around with this message at 21:08 on Sep 29, 2008

# ? Sep 29, 2008 21:04

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

m0nk3yz posted:

Uh, my mac works great for me for installing python things - it's all I develop on. Also, if you're on leopard: You should not be installing any other python distribution other than the one it came with, unless it's to your home directory or some /opt location. Thou shalt not change the system python!

That's not true - there's no problem with compiling a framework build for OSX. I routinely maintain 3 different pythons on my Mac (2.3 for backwards, 2.4 for zope, 2.5 for work). And once you factor eggs in, installing packages is a snap. So your original point is true - python works great on the mac.

(With, admittedly, some exceptions. Over the years I've sometimes had problems with libraries like PIL and MySQLdb that have to be compiled, but always able to eventually hack around them. And I've had problems with these libraries under Redhat as well, so it's difficult to see OSX as deficit in this regard.)

# ? Sep 30, 2008 09:37

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

outlier posted:

That's not true - there's no problem with compiling a framework build for OSX. I routinely maintain 3 different pythons on my Mac (2.3 for backwards, 2.4 for zope, 2.5 for work). And once you factor eggs in, installing packages is a snap. So your original point is true - python works great on the mac.

(With, admittedly, some exceptions. Over the years I've sometimes had problems with libraries like PIL and MySQLdb that have to be compiled, but always able to eventually hack around them. And I've had problems with these libraries under Redhat as well, so it's difficult to see OSX as deficit in this regard.)

Let me correct myself: I don't compile additional framework builds unless I need to - which is almost never, simply due to the fact I can install a perfectly functional build in my home directory and keep it completely self-contained.

As for compiling packages - this is true pretty much everywhere. People get pretty hung up on having packages with pre-compiled everything, but what happens if you need version 2 and your upstream only provides version 1 (which happens to me with ubuntu all the time) - I'm going to need to compile the darn thing. I guess I just don't see what the big deal about compiling this stuff.

# ? Sep 30, 2008 14:04

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

m0nk3yz posted:

Let me correct myself: I don't compile additional framework builds unless I need to - which is almost never, simply due to the fact I can install a perfectly functional build in my home directory and keep it completely self-contained.

As for compiling packages - this is true pretty much everywhere. People get pretty hung up on having packages with pre-compiled everything, but what happens if you need version 2 and your upstream only provides version 1 (which happens to me with ubuntu all the time) - I'm going to need to compile the darn thing. I guess I just don't see what the big deal about compiling this stuff.

Point taken. I'd prefer to use a single build of Python myself, except for Zope's lunk-headed attitude to versions (i.e. only runs on Python 2.4 for obscure reasons, internal modules are incredibly picky about the versions of other modules they will work with). And granted, no package installer with help you with versioning in Python.

On a side note, when I started working with Python on Windows was when I was astonished how hard it was to develop with. What, I have to get a special installer for half the packages I want? What if the developer hasn't been arsed to compile one as yet? Now _that's_ difficult ...

nonathlon fucked around with this message at 18:55 on Sep 30, 2008

# ? Sep 30, 2008 14:15

duck monster: Dec 15, 2004

outlier posted:

Point taken. I'd prefer to use a single build of Python myself, except for Zope's lunk-headed attitude of versions (i.e. only runs on Python 2.4 for obscure reasons, internal modules are incredibly picky about the versions of other modules they will work with). And granted, no package installer with help you with versioning in Python.

On a side note, when I started working with Python on Windows was when I was astonished how hard it was to develop with. What, I have to get a special installer for half the packages I want? What if the developer hasn't been arsed to compile one as yet? Now _that's_ difficult ...

Oh god.... One of the angriest emails I ever wrote was something to the effect of "Why the gently caress are you compiling free software with an unfree compiler? USE GCC!!!!!!!!" because I was blocked from using so many libraries due to the need for Visual studio.

# ? Sep 30, 2008 14:28

Adbot: ADBOT LOVES YOU

# ? May 11, 2024 09:58

Bozart: Oct 28, 2006; Give me the finger.

duck monster posted:

Oh god.... One of the angriest emails I ever wrote was something to the effect of "Why the gently caress are you compiling free software with an unfree compiler? USE GCC!!!!!!!!" because I was blocked from using so many libraries due to the need for Visual studio.

This, 1000 times.

# ? Sep 30, 2008 14:32

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »