Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

YO MAMA HEAD: Sep 11, 2007

That's kind of interesting but I've already implemented my 4-letter base-26 solution (both decoding and encoding). I also don't know if I trust people to type eight letters more than I trust them to type four...

# ? Feb 10, 2012 08:02

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 08:08

fart simpson: Jul 2, 2005; DEATH TO AMERICA

Scaevolus posted:

Make a word list with 256 short, common, words. Now 169.254.13.24 gets converted into the 14th and 25th words on your list.

You can make your wordlist from RFC1751-- that has 2048 words, so just select about one word from each line that you like.

Hold on, how does 169.254.13.24 get converted to the 14th and 25th words? I don't understand

# ? Feb 10, 2012 16:46

TK422: Dec 22, 2007; I hate being right

Is there any way to avoid the need of doing a .strip() when reading lines from a file in a context where you do not want '\n' at the end of your lines.

code:

for line in file:
    line=line.strip()
    # work

# ? Feb 10, 2012 17:12

leterip: Aug 25, 2004

MeramJert posted:

Hold on, how does 169.254.13.24 get converted to the 14th and 25th words? I don't understand

Well, 0 is the 1st word, so 13 is the 14th word, and you get the picture.

# ? Feb 10, 2012 17:14

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

TK422 posted:

Is there any way to avoid the need of doing a .strip() when reading lines from a file in a context where you do not want '\n' at the end of your lines.
code:
for line in file:
    line=line.strip()
    # work

I don't know if there's anything built in - you could try:

code:

for line in [line.strip() for line in file]:
  #do stuff

but unless list comprehensions work lazily when using generators, which I'm not sure they do, that'll probably be quite inefficient for large files. Probably best just to keep that and be explicit about it, that line can hardly be cluttering up your code that much.

# ? Feb 10, 2012 17:53

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Jonnty posted:

I don't know if there's anything built in - you could try:
code:
for line in [line.strip() for line in file]:
  #do stuff
but unless list comprehensions work lazily when using generators, which I'm not sure they do, that'll probably be quite inefficient for large files. Probably best just to keep that and be explicit about it, that line can hardly be cluttering up your code that much.

You were so close to being efficient with that code:

code:

for line in (line.strip() for line in file):
  #do stuff

List comprehensions are never lazy; it doesn't matter what's providing the values. I suspect that in recent Pythons, list comprehensions are syntactic sugar for calling list() on a generator expression but I'm not too inclined to check the interpreter source code to find out.

I would use something instead of line in the generator expression -- reusing names like that makes me uneasy, but it's a minor nitpick.

Speaking of which: TK422, it's moderately bad form to overwrite a built-in name like file with one of your variables.

# ? Feb 10, 2012 18:20

TK422: Dec 22, 2007; I hate being right

Lysidas posted:

Speaking of which: TK422, it's moderately bad form to overwrite a built-in name like file with one of your variables.

Yes, I know, it read as pseudocode in my head. But thanks for the advice.

# ? Feb 10, 2012 18:38

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Lysidas posted:

You were so close to being efficient with that code:

Cheers, knew there was something.

# ? Feb 10, 2012 19:42

tef: May 30, 2004; -> some l-system crap ->

TK422 posted:

Is there any way to avoid the need of doing a .strip() when reading lines from a file in a context where you do not want '\n' at the end of your lines.

No. If you need to get rid of newlines, you need to strip them.

n.b The code you wrote originally is simpler and likely faster than the changes proposed here.

tef fucked around with this message at 22:55 on Feb 10, 2012

# ? Feb 10, 2012 22:52

vikingstrike: Sep 23, 2007; whats happening, captain

I usually just use strip() along with readlines() to store it right from the beginning.

input_lines = [line.strip() for line in input.readlines()]

I have no idea why I do this. Would it be better to just leave it alone and do what he is?

# ? Feb 11, 2012 02:51

tef: May 30, 2004; -> some l-system crap ->

well, you are reading the entire file into memory

his approach is simple, line at a time and easy to modify. using a generator comprehension is also line at a time, but it is a bit why are you putting in two iterations over when one will do ?

# ? Feb 11, 2012 03:43

vikingstrike: Sep 23, 2007; whats happening, captain

Well I guess you learn things everyday. Thanks for the feedback.

# ? Feb 11, 2012 07:32

Computer viking: May 30, 2011; Now with less breakage.

vikingstrike posted:

Well I guess you learn things everyday. Thanks for the feedback.

Of course, reading the entire file into memory is a perfectly fine choice if it's a small file (and the easiest choice if you want to access the lines out of order).

# ? Feb 11, 2012 13:04

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

To be honest, I like the two line version better than the one line one. Just looks a bit cleaner. Be sure to pay attention to PEP8, though. It's line = line.strip(), not line=line.strip().

# ? Feb 11, 2012 19:05

FoiledAgain: May 6, 2007

Computer viking posted:

Of course, reading the entire file into memory is a perfectly fine choice if it's a small file (and the easiest choice if you want to access the lines out of order).

Is there some kind of rule of thumb for deciding this? How big would a file have to be before you decided not to read it all in?

# ? Feb 11, 2012 19:49

Quebec Bagnet: Apr 28, 2009; mess with the honk
you get the bonk; Lipstick Apathy

YO MAMA HEAD posted:

Could you use an mDNS library like Avahi or Bonjour? That would save you the issue of even needing to discover IP addresses in the first place.

# ? Feb 11, 2012 19:53

Computer viking: May 30, 2011; Now with less breakage.

FoiledAgain posted:

Is there some kind of rule of thumb for deciding this? How big would a file have to be before you decided not to read it all in?

It's more a question of "how might the code be used in the future". If you're just doing a one-off, it's fine as long as you have the RAM. If you're writing something that could be used with files of an arbitrary size in the future, it's obviously not a good idea.

I'd probably limit myself to a handful of MB at most, and only really for things where it's elegant - such as if I can apply a list comprehension to the entire file to directly get something useful. I can also excuse using it if writing a per-line version would be more annoying than it would be worth; saving myself some minutes at the cost of using several GB of RAM is fine, if I'll only run the program twice.

# ? Feb 13, 2012 18:35

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

A (possibly dumb) question about WSGI:

So I'm rewriting a CGI script I wrote some years ago and want to do it The Right Way, so went for WSGI. Fine, but I've progressed to the point where I need to include static assets (css & js files). Here I hit a problem. I been able to test the script thus far by just using a toy server from wsgiref.simple_server. But to test something using static assets, am I going to have install the script into Apache or the like?

# ? Feb 14, 2012 14:04

Computer viking: May 30, 2011; Now with less breakage.

outlier posted:

A (possibly dumb) question about WSGI:

So I'm rewriting a CGI script I wrote some years ago and want to do it The Right Way, so went for WSGI. Fine, but I've progressed to the point where I need to include static assets (css & js files). Here I hit a problem. I been able to test the script thus far by just using a toy server from wsgiref.simple_server. But to test something using static assets, am I going to have install the script into Apache or the like?

That's the correct way to do it, yes - have a separate webserver/directory for serving static files. For a high-traffic site you can then use something faster like ngix or lighthttpd and some aggressive caching policies for those files - but that's probably overkill.

# ? Feb 14, 2012 16:23

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

Computer viking posted:

That's the correct way to do it, yes - have a separate webserver/directory for serving static files. For a high-traffic site you can then use something faster like ngix or lighthttpd and some aggressive caching policies for those files - but that's probably overkill.

Sure - but I was more interested in the development stages. To develop this script (further), will I have to deploy it to a "proper" webserver? It's a bit of a hassle if so, albeit understandable. Is there any stopgap solution while I'm hacking around with it?

# ? Feb 15, 2012 09:49

Computer viking: May 30, 2011; Now with less breakage.

There are ways to serve static files with wsgi, but I can't really help you beyond what google could tell us both. I'd personally recommend you just install apache (or another server, if you prefer). It depends on your distro/OS, but that's usually about 10 minutes of work: It's not like you're planning any configuration beyond the very basic "serve static files from this directory", and most installs will leave you with something where you just need to start httpd and find out where it's pre-configured to serve from.

# ? Feb 15, 2012 10:33

ynohtna: Feb 16, 2007; backwoods compatible; Illegal Hen

WAMP/MAMP make installing Apache a breeze, then install and configure mod_wsgi.

Alternately, you could use the Google AppEngine dev server.

# ? Feb 15, 2012 10:56

Computer viking: May 30, 2011; Now with less breakage.

ynohtna posted:

WAMP/MAMP make installing Apache a breeze, then install and configure mod_wsgi.

Alternately, you could use the Google AppEngine dev server.

Or run the dev server on another port and keep using that for the wsgi side for now.

# ? Feb 15, 2012 11:24

Jam2: Jan 15, 2008; With Energy For Mayhem

I'm trying to use Python to do stuff with xml from a URL which requires authentication.

When I access http://myplace.com:8086/stuff from a browser. I am prompted to enter a username and password. When I proceed, I see well-formed xml data in the body of a web page.

In python, I am using urllib2 to try to access this url. I am doing so because I believe that this is the first logical step in accessing this data.

The problem is, after using the following code to access the url, I receive an HTTPError 401 when I call urlopen.

code:

import urllib2

theurl = 'http://www.someserver.com/toplevelurl/somepage.htm'
username = 'johnny'
password = 'XXXXXX'
# a great password

passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
# this creates a password manager
passman.add_password(None, theurl, username, password)
# because we have put None at the start it will always
# use this username/password combination for  urls
# for which `theurl` is a super-url

authhandler = urllib2.HTTPBasicAuthHandler(passman)
# create the AuthHandler

opener = urllib2.build_opener(authhandler)

urllib2.install_opener(opener)
# All calls to urllib2.urlopen will now use our handler
# Make sure not to include the protocol in with the URL, or
# HTTPPasswordMgrWithDefaultRealm will be very confused.
# You must (of course) use it when fetching the page though.

pagehandle = urllib2.urlopen(theurl)
# authentication is now handled automatically for us

If I change my url to be one which does not require authentication, I am able to access the data on the page no problem.

Jam2 fucked around with this message at 06:49 on Feb 16, 2012

# ? Feb 16, 2012 06:38

vikingstrike: Sep 23, 2007; whats happening, captain

I'm not at my main computer to post code, but try using the cookie manager and build this into the opener. I'm not sure what properties are on that password manager but you probably just need cookie support.

# ? Feb 16, 2012 06:50

fart simpson: Jul 2, 2005; DEATH TO AMERICA

Also use requests. It's much better than urllib2.

# ? Feb 16, 2012 06:53

vikingstrike: Sep 23, 2007; whats happening, captain

What's wrong with urllib2? Between it and urllib I've never really run into issues. Does requests have special functionality that the other two don't offer?

# ? Feb 16, 2012 07:19

Quality_Guaranteed: Jan 23, 2006; by Y Kant Ozma Post

Total n00b question: what's the point of @classmethod decorator? Don't subclasses inherit methods from parent classes anyway?

# ? Feb 16, 2012 08:22

Jam2: Jan 15, 2008; With Energy For Mayhem

I tried this using requests. I still get 401

code:

from requests.auth import HTTPBasicAuth
import requests

r = requests.get('http://fms.mysite:8086/connectioncounts', auth=HTTPBasicAuth('user', 'pword'))

print r.cookies
print r.status_code

This returns

code:

{}
401

edit: It needed digest authentication. Works now!

Jam2 fucked around with this message at 08:36 on Feb 16, 2012

# ? Feb 16, 2012 08:28

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Quality_Guaranteed posted:

Total n00b question: what's the point of @classmethod decorator? Don't subclasses inherit methods from parent classes anyway?

Class methods are used in the context of the whole class, as in, it uses class variables rather than instance variables through the first argument (cls rather than self)

# ? Feb 16, 2012 08:44

Quality_Guaranteed: Jan 23, 2006; by Y Kant Ozma Post

Maluco Marinero posted:

Class methods are used in the context of the whole class, as in, it uses class variables rather than instance variables through the first argument (cls rather than self)

Does this mean that I don't have to instantiate an object to use the method? If I have a class, A, and a classmethod, f(cls,x,y,etc.), I can just type A.f(whatever) and it will work?

Because I just tested this and it seems to be the case. If I don't use classmethod and don't instantiate, I get "TypeError: unbound method f() must be called with A instance as first argument (got int instance instead)"

# ? Feb 16, 2012 08:56

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Here's the docs, it should be callable from a class or an instance, but in both styles will receive the class as the first argument. I haven't really used them at all in my (limited) work thus far.

http://docs.python.org/library/functions.html?highlight=classmethod#classmethod

Maluco Marinero fucked around with this message at 09:09 on Feb 16, 2012

# ? Feb 16, 2012 09:06

The Gripper: Sep 14, 2004; i am winner

Quality_Guaranteed posted:

Does this mean that I don't have to instantiate an object to use the method? If I have a class, A, and a classmethod, f(cls,x,y,etc.), I can just type A.f(whatever) and it will work?

Because I just tested this and it seems to be the case. If I don't use classmethod and don't instantiate, I get "TypeError: unbound method f() must be called with A instance as first argument (got int instance instead)"

It does, yes, but (if the method doesn't need to access the class itself) you can just add static methods without self as the first argument and get the same result. There aren't a <lot> of cases where you'd want to use class methods, the classic use is to add different ways to instantiate classes e.g. create a class from values in a text file.

I asked the same question as you a page or so back, and Suspicious Dish gave a good use-case for @classmethod with code Here

# ? Feb 16, 2012 09:15

Seaside Loafer: Feb 7, 2012; Waiting for a train, I needed a shit. You won't bee-lieve what happened next

Whats the best way to order a set/collection in terms of ascii/unicode value? I found some code to do it or could write my own but I cant help but think there is likely to be a built in function in the standard modules im missing without having to import some other stuff. Been looking through the documentation and cant see one I suspect im missing it. Python 2.7 btw.

I mean id like to be able to say in psudeocode:

code:

collectionofdata = new collection
collectionofdata.add ["zaslkjas", "asldkj", "fdsdf"]
collectionofdata.orderme

purely using the standard modules. Im pretty new to the language as you can tell.

Seaside Loafer fucked around with this message at 20:42 on Feb 18, 2012

# ? Feb 18, 2012 20:33

Nippashish: Nov 2, 2005; Let me see you dance!

Seaside Loafer posted:

I mean id like to be able to say in psudeocode:
code:
collectionofdata = new collection
collectionofdata.add ["zaslkjas", "asldkj", "fdsdf"]
collectionofdata.orderme

You can do this:

code:

collectionofdata = ["zaslkjas", "asldkj", "fdsdf"]
collectionofdata.sort()

There are also some options you can pass to sort if you want to change the order, see here.

# ? Feb 18, 2012 20:46

Seaside Loafer: Feb 7, 2012; Waiting for a train, I needed a shit. You won't bee-lieve what happened next

Nippashish posted:

You can do this:
code:
collectionofdata = ["zaslkjas", "asldkj", "fdsdf"]
collectionofdata.sort()
There are also some options you can pass to sort if you want to change the order, see here.

Thanks buddy.

# ? Feb 18, 2012 20:58

Sidpret: Jun 17, 2004

I have a really simple question about csv.writer. I have a big list of strings, and I want to write a csv where every string is a new row. This seems like it should be super easy but I'm doing something stupid. My list of strings is called trimmed, and the code I'm using is:

code:

writer=csv.writer(open('/filepath','wb'))
for row in trimmed:
        writer.write(row)

This code iterates over each character apparently, so it splits my list into strings but then splits each string into characters and makes each character a new row. Not at all what I want. How do I get it to just iterate over the list instead of over each string?

Sidpret fucked around with this message at 22:03 on Feb 18, 2012

# ? Feb 18, 2012 21:07

Scaevolus: Apr 16, 2007

Sidpret posted:

This code iterates over each character apparently, so it splits my list into strings but then splits each string into characters and makes each character a new row. Not at all what I want. How do I get it to just iterate over the list instead of over each string?

code:

writer=csv.writer(open('/Users/austinc/Desktop/sorted.csv','wb'))
for row in trimmed:
        writer.write([row])

# ? Feb 18, 2012 21:39

ufarn: May 30, 2009

I am trying to implement some automated minification for my static blog CMS, so are there any default Python packages to go for, when it comes to, say, images, CSS, and JS?

# ? Feb 18, 2012 22:15

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 08:08

Haystack: Jan 23, 2005

The only one I know off the top of my head is Webassets

# ? Feb 18, 2012 22:19

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »