Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Emacs Headroom: Aug 2, 2003

Is it just a stack dump or a memory dump? I think the bytes in memory on most (all?) machines is in reverse order; at least it is if you do a memory track in gdb or whatever.

# ? Aug 1, 2013 14:40

Adbot: ADBOT LOVES YOU

# ? Jun 11, 2024 11:27

evensevenone: May 12, 2001; Glass is a solid.

ahmeni posted:

Am I doing something wrong, and if not, what's the quickest way to perform the inversion on each byte in a string in Python?

Assuming that is what you want, how about this?

code:

''.join([chr(255-ord(c)) for c in a])

evensevenone fucked around with this message at 17:41 on Aug 1, 2013

# ? Aug 1, 2013 17:39

Telarra: Oct 9, 2012

Just a tip, the [ ] there aren't necessary (and since they force the generation of a list, they're also an unnecessary slowdown):

code:

''.join(chr(255-ord(c)) for c in a)

# ? Aug 1, 2013 17:56

evensevenone: May 12, 2001; Glass is a solid.

What happens then? Is that an iterator?

# ? Aug 1, 2013 20:03

Haystack: Jan 23, 2005

It's a generator expression inside of a method call, to be specific.

# ? Aug 1, 2013 20:07

Telemarchitect: Oct 1, 2009; TOUCH THE KNOB

I'm writing a PyQt4 Gui that has a bunch of duplicated controls. For example, I've got a bunch of table widgets that are identical and have controls to add, remove and rearrange the entries in their own table. Is there a way to make this scalable to any n number of tables/controls that doesn't involve lots of duplicate code? For example, some if my controls are named

code:

btn_tbl1_deleteentry
btn_tbl2_deleteentry
btn_tbl3_deleteentry
btn_tbl4_deleteentry

...meaning being hardcoded to 4 tables. Even if I call a function that sets an index and calls a function that handles all the common logic, the branch logic still is limited to four because of the names.

Is there a way to "vectorize" the controls so that I can call controls like

code:

btn_tbl[x]_deleteentry

...where x is 0 to n-1?

# ? Aug 1, 2013 21:37

Dren: Jan 5, 2001; Pillbug

DSA_Key have you heard of paramiko and have you considered using passwordless ssh keys?

# ? Aug 1, 2013 21:38

Dren: Jan 5, 2001; Pillbug

Telemarchitect posted:

I'm writing a PyQt4 Gui that has a bunch of duplicated controls. For example, I've got a bunch of table widgets that are identical and have controls to add, remove and rearrange the entries in their own table. Is there a way to make this scalable to any n number of tables/controls that doesn't involve lots of duplicate code? For example, some if my controls are named
code:
btn_tbl1_deleteentry
btn_tbl2_deleteentry
btn_tbl3_deleteentry
btn_tbl4_deleteentry
...meaning being hardcoded to 4 tables. Even if I call a function that sets an index and calls a function that handles all the common logic, the branch logic still is limited to four because of the names.

Is there a way to "vectorize" the controls so that I can call controls like
code:
btn_tbl[x]_deleteentry
...where x is 0 to n-1?

Have a look at getattr()?

# ? Aug 1, 2013 21:41

Telarra: Oct 9, 2012

Dren posted:

Have a look at getattr()?

To elaborate on this: getattr(self, 'btn_tbl%i_deleteentry' % x) would be equivalent to the hypothetical self.btn_tbl[x]_deleteentry

# ? Aug 1, 2013 21:51

ahmeni: May 1, 2005; It's one continuous form where hardware and software function in perfect unison, creating a new generation of iPhone that's better by any measure.; Grimey Drawer

Emacs Headroom posted:

Is it just a stack dump or a memory dump? I think the bytes in memory on most (all?) machines is in reverse order; at least it is if you do a memory track in gdb or whatever.

Nope, just network traffic but array.byteswap should give the proper result if it was just endian ordering.

evensevenone posted:

Assuming that is what you want, how about this?
code:
''.join([chr(255-ord(c)) for c in a])

Awesome, that does the trick. Ill give it a shot today with the generator expression as well. Thanks guys.

# ? Aug 1, 2013 21:54

evilentity: Jun 25, 2010

DSA_Key posted:

So I needed something that could execute commands on multiple Linux systems and bring the data back to me for parsing. So using Python and Pexpect I wrote the following and I just wanted to share as this is the very first project I've ever undertaken in Python let alone Pexpect. The only thing the script doesn't ask for is the file with the IP addresses or hostnames instead it's hard coded to read a targets.txt file in the directory it's executed from.

Eventually I'll update it so it has a help function and ask for a custom file with a list of hosts and maybe sanitize user input etc.... But I was stoked I got this far and I just want to share! Python's pretty cool.

edit* also it writes an error log with a best guess as to why it couldn't log into a server.

Looks reasonable for most part. Here are few tips:

When you use `with` you dont need to close the file.
Use logging module for logging.
In Python empty string is considered false, thus `if host != "":` is equal to `if host:`.
In the `if i == 0:` etc try not to using magic numbers. Just make a constant at the top like `E_TIMED_OUT = 0`
Also try using some more meaningful variable names, i's and j's arent very clear.

Python's pretty cool!

# ? Aug 1, 2013 21:56

Jose Cuervo: Aug 25, 2004

I am trying to write a small discrete event simulation and I run into a problem when there are several events that occur at the same time, but are 'out of order' because of the order in which they were placed on the event calendar (the event calendar gets sorted each time an event is placed on it but because the sort is a stable sort, events that occur at the same time remain in the order they were placed on the calendar).

When events occur at the same time, I have an ordering I would like to impose based upon the name of the function being called (that is stored in the event calendar as a string). Is there a general programming/ pythonic way to define an ordering given that I can define an ordering of the strings?

For example if the functions are 'update_value_x', 'update_value_y', 'seize_resource', 'release_resource', my order would be:

'release_resource'
'seize_resource'
'update_value_x', 'update_value_y'

where 'release_resource' is most important, and the two 'update' functions are equivalent (i.e. should be ordered the way they were placed on the calendar).

# ? Aug 2, 2013 00:01

Telemarchitect: Oct 1, 2009; TOUCH THE KNOB

Moddington posted:

To elaborate on this: getattr(self, 'btn_tbl%i_deleteentry' % x) would be equivalent to the hypothetical self.btn_tbl[x]_deleteentry

This really owns. Thanks. I'm not at home so I can't check, but I'm guessing this would work on the button's methods by just adding it onto that quoted string there?

# ? Aug 2, 2013 00:40

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Telemarchitect posted:

This really owns. Thanks. I'm not at home so I can't check, but I'm guessing this would work on the button's methods by just adding it onto that quoted string there?

Python code:

the_frickin_button =  getattr(self, 'btn_tbl%i_deleteentry' % x)
the_frickin_button.some_method_on_the_fricking_button()

# ? Aug 2, 2013 01:30

coaxmetal: Oct 21, 2010; I flamed me own dad

Telemarchitect posted:

This really owns. Thanks. I'm not at home so I can't check, but I'm guessing this would work on the button's methods by just adding it onto that quoted string there?

python's getattr is just a way to use a string to get an attribute. So, getattr(an_object, 'property_name') is the same as an_object.property_name
in your case it is useful because that way to can pass in a string with the name of the property you want dynamically.

also getattr takes an optional 3rd argument, which is a value to return if the attribute doesn't exist. If you don't pass that and it doesn't exist, you will get an AttributeError (which may be the correct behavior).

also also don't do string formatting like these other chucklefucks.

instead of 'btn_tbl%i_deleteentry' % x use
'btn_tbl{}_deleteentry'.format(x)

# ? Aug 2, 2013 01:43

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Ronald Raiden posted:

also also don't do string formatting like these other chucklefucks.

instead of 'btn_tbl%i_deleteentry' % x use
'btn_tbl{}_deleteentry'.format(x)

Yeah, this is right.

In my defense I only had it because I copy/pasted the previous example. :colbert:

# ? Aug 2, 2013 01:47

QuarkJets: Sep 8, 2008

What are the pros and cons of this alternative way to format strings? I always found the % technique to be really intuitive so I never learned the .format technique, but it looks easy too

# ? Aug 2, 2013 02:14

fart simpson: Jul 2, 2005; DEATH TO AMERICA

QuarkJets posted:

What are the pros and cons of this alternative way to format strings? I always found the % technique to be really intuitive so I never learned the .format technique, but it looks easy too

Cons: it's not immediately familiar if you're coming from another language

Pros: basically everything else

# ? Aug 2, 2013 02:52

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

QuarkJets posted:

What are the pros and cons of this alternative way to format strings? I always found the % technique to be really intuitive so I never learned the .format technique, but it looks easy too

There's no huge reason to change other than that it's kind of the way future versions of python are heading.

I think most people will find the .format technique to be clearer and easier to use, but I don't think there's anything you can do with one that you can't do with the other.

# ? Aug 2, 2013 03:08

coaxmetal: Oct 21, 2010; I flamed me own dad

QuarkJets posted:

What are the pros and cons of this alternative way to format strings? I always found the % technique to be really intuitive so I never learned the .format technique, but it looks easy too

One reason is that the % syntax isn't in python 3, so to write compatible code you have to use the .format method.

but that aside, it is more explicit, and doesn't' require any magic syntax (the % operator). It also is much easier to read if you are doing multiple substutions or named paramaters

# ? Aug 2, 2013 03:12

breaks: May 12, 2001

Jose Cuervo posted:

I am trying to write a small discrete event simulation and I run into a problem when there are several events that occur at the same time, but are 'out of order' because of the order in which they were placed on the event calendar (the event calendar gets sorted each time an event is placed on it but because the sort is a stable sort, events that occur at the same time remain in the order they were placed on the calendar).

When events occur at the same time, I have an ordering I would like to impose based upon the name of the function being called (that is stored in the event calendar as a string). Is there a general programming/ pythonic way to define an ordering given that I can define an ordering of the strings?

If you haven't already, you may want to look at the sched module and determine whether it meets your needs.

If it doesn't, you still probably don't want to sort the whole list of events every time you add one. There is a PriorityQueue class in the standard library that may be able to help you (but I've not checked it's behavior in any kind of detail).

Lastly, you may want to just pass the functions instead of strings with the function names. In any case having the scheduler portion of your program also translating the strings to functions is probably bad form, so try to avoid that if you haven't already.

All that said: without knowing anything else about what you are doing, a straightforward solution is to provide and store a string and an integer rather than only a string. The int represents the priority, so you can simply sort by that to get them in order. Basically instead of having a priority queue where each bucket is a list, you make a priority queue where each bucket is another priority queue.

# ? Aug 2, 2013 03:16

Gul Banana: Nov 28, 2003

MeramJert posted:

Cons: it's not immediately familiar if you're coming from another language

Pros: basically everything else

there's actually a bit of a cross language .format zeitgeist. it will be familiar to, say, .NET or Scala programmers.

# ? Aug 2, 2013 04:33

ahmeni: May 1, 2005; It's one continuous form where hardware and software function in perfect unison, creating a new generation of iPhone that's better by any measure.; Grimey Drawer

So it turns out that this takes a while on 3.5MB:

code:

''.join(chr(255-ord(c)) for c in a)

While this way is about 50% faster:

code:

a = bytearray(str)
for x in xrange(len(a)):
    a[x] = 255 - a[x]

For some reason it profiles terribly in cProfile, giving an execution time of ~40 seconds vs 1 seconds while in reality it's more like 4 vs 2 seconds.

pre:

import cProfile

def oldMethod(s):
        return ''.join(chr(255-ord(c)) for c in s)
def newMethod(s):
        a = bytearray(s)
        for x in xrange(len(a)):
                a[x] = 255 - a[x]
        return a
a = open("rawdump.data", "rb").read()
cProfile.run("oldMethod(a)")
cProfile.run("newMethod(a)")

~~~

         11244896 function calls in 40.414 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002   40.414   40.414 <string>:1(<module>)
        1    0.000    0.000   40.412   40.412 test.py:3(oldMethod)
  3748298   20.385    0.000   33.625    0.000 test.py:4(<genexpr>)
  3748297    6.862    0.000    6.862    0.000 {chr}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    6.788    6.788   40.412   40.412 {method 'join' of 'str' objects}
  3748297    6.378    0.000    6.378    0.000 {ord}


         4 function calls in 0.875 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.875    0.875 <string>:1(<module>)
        1    0.872    0.872    0.872    0.872 test.py:6(newMethod)
        1    0.000    0.000    0.000    0.000 {len}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

# ? Aug 2, 2013 05:10

Dren: Jan 5, 2001; Pillbug

Boost (c++ library) has very similar % syntax.

The whole getattr thing for those tables, the tables should be dynamically created and looked up in a dict or an array. The getattr thing will work but if there are 4 tables that are the same there shouldn't be 4 uniquely named tables, there should be a dynamic container for n tables.

# ? Aug 2, 2013 05:11

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

ahmeni posted:

So it turns out that this takes a while on 3.5MB:

code:


''.join(chr(255-ord(c)) for c in a)

While this way is about 50% faster:

code:


a = bytearray(str)
for x in xrange(len(a)):
    a[x] = 255 - a[x]

For some reason it profiles terribly in cProfile, giving an execution time of ~40 seconds vs 1 seconds while in reality it's more like 4 vs 2 seconds.

pre:

import cProfile

def oldMethod(s):
        return ''.join(chr(255-ord(c)) for c in s)
def newMethod(s):
        a = bytearray(s)
        for x in xrange(len(a)):
                a[x] = 255 - a[x]
        return a
a = open("rawdump.data", "rb").read()
cProfile.run("oldMethod(a)")
cProfile.run("newMethod(a)")

~~~

         11244896 function calls in 40.414 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002   40.414   40.414 <string>:1(<module>)
        1    0.000    0.000   40.412   40.412 test.py:3(oldMethod)
  3748298   20.385    0.000   33.625    0.000 test.py:4(<genexpr>)
  3748297    6.862    0.000    6.862    0.000 {chr}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    6.788    6.788   40.412   40.412 {method 'join' of 'str' objects}
  3748297    6.378    0.000    6.378    0.000 {ord}


         4 function calls in 0.875 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.875    0.875 <string>:1(<module>)
        1    0.872    0.872    0.872    0.872 test.py:6(newMethod)
        1    0.000    0.000    0.000    0.000 {len}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

I'm going to guess that it's entirely due to allocation and gc time since the join can't lookup the length of the iterable it's given, while the byte array is doing a highly vectorizable inner loop over an array it allocates once

# ? Aug 2, 2013 05:52

MeruFM: Jul 27, 2010

nevermind
figured it out

MeruFM fucked around with this message at 08:04 on Aug 2, 2013

# ? Aug 2, 2013 07:18

Dominoes: Sep 20, 2007

Ronald Raiden posted:

One reason is that the % syntax isn't in python 3, so to write compatible code you have to use the .format method.

% formatting works in Python 3 - it's just discouraged.

# ? Aug 2, 2013 08:05

coaxmetal: Oct 21, 2010; I flamed me own dad

Dominoes posted:

% formatting works in Python 3 - it's just discouraged.

really? I thought it had been deprecated entirely. That's almost disappointing, though good news for porting code more easily I suppose.

# ? Aug 2, 2013 16:28

DSA_Key: Dec 9, 2004; minimalist

Dren posted:

DSA_Key have you heard of paramiko and have you considered using passwordless ssh keys?

I had not heard of paramiko, reading about it right now. And ssh keys aren't an option at the moment but will be in the near future I just needed something quick and dirty and I remember a long time ago I wrote a TCL/TK script using Expect/TK that was very similar and thought it would be a good exercise for learning Python.

evilentity posted:

Looks reasonable for most part. Here are few tips:

When you use `with` you dont need to close the file.
Use logging module for logging.
In Python empty string is considered false, thus `if host != "":` is equal to `if host:`.
In the `if i == 0:` etc try not to using magic numbers. Just make a constant at the top like `E_TIMED_OUT = 0`
Also try using some more meaningful variable names, i's and j's arent very clear.

Python's pretty cool!

Excellent tips that will come in handy thanks!

Edit - for my next project I'm considering writing a python script to go out to ARIN, RIPE and APNIC to do whois lookups for me on IP addresses.

DSA_Key fucked around with this message at 17:01 on Aug 2, 2013

# ? Aug 2, 2013 16:59

xtal: Jan 9, 2011; by Fluffdaddy

Dominoes posted:

% formatting works in Python 3 - it's just discouraged.

Why is that? I've always preferred % to the format method, but it's honestly just because the double parentheses it involves really irritate me. Like method("something".format("something")) is uglier than method("something" % "something").

# ? Aug 2, 2013 17:09

digitalcamo: Jul 11, 2013

Something I've been thinking about while learning Python, is it possible to write iPhone apps using Python? That idea seemed pretty neat as I'm trying to figure out what kind of programming I'd like to pursue. Followed the advice of others and really enjoy Codecademy. Turns out it is not a total waste of time like some of the reviews I read before joining.

# ? Aug 2, 2013 18:21

coaxmetal: Oct 21, 2010; I flamed me own dad

digitalcamo posted:

Something I've been thinking about while learning Python, is it possible to write iPhone apps using Python? That idea seemed pretty neat as I'm trying to figure out what kind of programming I'd like to pursue. Followed the advice of others and really enjoy Codecademy. Turns out it is not a total waste of time like some of the reviews I read before joining.

I've looked into this a little, there is http://pythonhosted.org/pyobjc/ and some xcode plugins. So in theory, its possible, but I never ended up using it a lot and it seems inelegant. Not sure if its good.

# ? Aug 2, 2013 18:24

QuarkJets: Sep 8, 2008

xtal posted:

Why is that? I've always preferred % to the format method, but it's honestly just because the double parentheses it involves really irritate me. Like method("something".format("something")) is uglier than method("something" % "something").

You could always set up a variable to hold the string first, but I agree that the % method looks a little cleaner for single arguments. Format definitely has other advantages, though (for instance, if you need to populate the same variable into a string multiple times, and since format() is a function you can pass it around to things that take function arguments).

IIRC, % is also about twice as fast as .format

# ? Aug 2, 2013 18:56

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

digitalcamo posted:

Something I've been thinking about while learning Python, is it possible to write iPhone apps using Python?

Even if you could it'd be a bad idea. Performance (and battery usage) is consistently an issue with HTML/JS based apps, and Python would be even worse in that regard.

# ? Aug 2, 2013 19:04

xtal: Jan 9, 2011; by Fluffdaddy

QuarkJets posted:

You could always set up a variable to hold the string first, but I agree that the % method looks a little cleaner for single arguments. Format definitely has other advantages, though (for instance, if you need to populate the same variable into a string multiple times, and since format() is a function you can pass it around to things that take function arguments).

IIRC, % is also about twice as fast as .format

For those two use cases, you could use a dictionary argument ("%(repeated_key)s %(repeated_key)s" % { "repeated_key": "fart" }) or operator.mod.

# ? Aug 2, 2013 19:32

evilentity: Jun 25, 2010

digitalcamo posted:

Something I've been thinking about while learning Python, is it possible to write iPhone apps using Python? That idea seemed pretty neat as I'm trying to figure out what kind of programming I'd like to pursue. Followed the advice of others and really enjoy Codecademy. Turns out it is not a total waste of time like some of the reviews I read before joining.

I havent used it, but Kivy might interest you: http://kivy.org/
Its a multiplatform framework for various systems including ios.

# ? Aug 2, 2013 19:34

digitalcamo: Jul 11, 2013

Plorkyeran posted:

Even if you could it'd be a bad idea. Performance (and battery usage) is consistently an issue with HTML/JS based apps, and Python would be even worse in that regard.

Why would using Python for writing apps affect performance and battery usage?

# ? Aug 2, 2013 22:44

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

digitalcamo posted:

Why would using Python for writing apps affect performance and battery usage?

CPython is typically one to two orders of magnitude slower than Objective-C. In many situations this is not an issue. Running things on phones is not one of these situations.

# ? Aug 2, 2013 23:37

xtal: Jan 9, 2011; by Fluffdaddy

I don't know how it compares or if it's relevant, but PyPy recently released an ARM-compatible version that'd theoretically run on iOS.

# ? Aug 3, 2013 00:51

Adbot: ADBOT LOVES YOU

# ? Jun 11, 2024 11:27

minidracula: Dec 22, 2007; boo woo boo

thorf posted:

I've not been able to find an IDE for python that I like. The closest thing I've found so far is Sublime Text 2, but that doesn't work on the Red Hat 5 machines at work. I currently use gedit. What do you guys use?

QPZIL posted:

What is the best Python IDE for Linux? Should I stick with VIM? It does give me an excuse to be an elitist

I'm late to both of these solicitations for other IDE options, but since I hadn't yet seen it mentioned, allow me to suggest Wing IDE.

I've been using it for several years now on Python projects and I like it a great deal. I like it enough to recommend it and push people to try it that I think I'm directly and indirectly responsible for at least a dozen other people/teams/small companies buying licenses over the last six years or so. The developers have been working on Wing for quite a while (I remember getting a demo/trial CD-R back in the early 2000s for a much earlier version and first trying it then, though I was put off by the GUI back then), and they seem like they dogfood it non-stop (based both on what they say and how often they seem to roll out patches and updates, etc.).

Also handy is that when you purchase a license, you can sign a one page sensible NDA and fax it back to them to get access to the source code, and you're allowed to make your own changes, etc. (this may only apply if you buy the "Professional" license, I'm not sure; check the website).

The current stable release versions are built on GTK (using PyGTK), but version 5 (in beta now) is moving to Qt (via PySide).

All that said, I'm trying out PyCharm for the first time starting last week, and so far so good. I like and use a couple of JetBrains other tools (IntelliJ IDEA and ReSharper), so I'm seeing how PyCharm strikes me and compares to Wing.

I don't really work on huge Python projects that I write from the ground up, but I do usually have to read and understand someone else's libraries and substantial swaths of code, and I find my number one use of a decent Python IDE that understands the language (over plain text editors and command lines tools) is navigating around to see whats what with a minimal amount of clicks/file opens/greps.

# ? Aug 3, 2013 04:52

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »