Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

tef posted:

code:

import subprocess
proc = subprocess.Popen(['plink','username@server','-pw','password'],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE)

Fixed. You can't use shell=True if you're passing a list of parameters, and if 'plink' is on your %PATH%, it'll be able to find it. Otherwise, you'll have to pass the full path.

# ? Nov 6, 2008 05:07

Adbot: ADBOT LOVES YOU

# ? May 12, 2024 02:46

tef: May 30, 2004; -> some l-system crap ->

Habnabit posted:

You can't use shell=True if you're passing a list of parameters

Didn't realise that, ta.

# ? Nov 6, 2008 10:31

chemosh6969

Jul 3, 2004

code:
cat /dev/null > /etc/professionalism

_{_{I am in fact a massive asswagon.

Do not let me touch computer.}}

Habnabit posted:

You can't use shell=True if you're passing a list of parameters

It was working for me.

# ? Nov 6, 2008 16:59

chemosh6969

Jul 3, 2004

code:
cat /dev/null > /etc/professionalism

_{_{I am in fact a massive asswagon.

Do not let me touch computer.}}

Plink would get hung up.

I'm ending up writing an AutoIt script that python will call at the end. I managed to get past the menu with it, so it should work fine.

# ? Nov 6, 2008 19:36

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

chemosh6969 posted:

It was working for me.

It might behave differently on Windows because subprocess uses a different API on Windows. On POSIX systems, shell=True with an argument list will cause problems.

# ? Nov 6, 2008 23:19

chemosh6969

Jul 3, 2004

code:
cat /dev/null > /etc/professionalism

_{_{I am in fact a massive asswagon.

Do not let me touch computer.}}

Habnabit posted:

It might behave differently on Windows because subprocess uses a different API on Windows. On POSIX systems, shell=True with an argument list will cause problems.

FYI: It works on linux also.

Thanks for the help from everyone.

# ? Nov 7, 2008 00:02

tbradshaw: Jan 15, 2008; First one must nail at least two overdrive phrases and activate the tilt sensor to ROCK OUT!

chemosh6969 posted:

FYI: It works on linux also.

Thanks for the help from everyone.

It doesn't work on Linux at least.

A test directory:

code:

[ash@dexter tmp] $ ls
things  things2

[ash@dexter tmp] $ ls -l
total 0
-rw-r--r-- 1 ash ash 0 2008-11-06 18:53 things
-rw-r--r-- 1 ash ash 0 2008-11-06 18:54 things2

code:

[ash@dexter tmp] $ python
Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> p = subprocess.Popen(['ls'])
things	things2
>>> p = subprocess.Popen(['ls', '-l'])
>>> total 0
-rw-r--r-- 1 ash ash 0 2008-11-06 18:53 things
-rw-r--r-- 1 ash ash 0 2008-11-06 18:54 things2

Just as expected.

code:

>>> p = subprocess.Popen(['ls'], shell=True)
>>> things	things2

>>> p = subprocess.Popen(['ls', '-l'], shell=True)
>>> things	things2

Args were ignored.

# ? Nov 7, 2008 01:56

chemosh6969

Jul 3, 2004

code:
cat /dev/null > /etc/professionalism

_{_{I am in fact a massive asswagon.

Do not let me touch computer.}}

Try it without brackets:

p = subprocess.Popen('ls -l', shell=True)

or

subprocess.call('ls -l $HOME', shell=True)

That's how the examples I found worked.
http://blog.doughellmann.com/2007/07/pymotw-subprocess.html

chemosh6969 fucked around with this message at 02:28 on Nov 7, 2008

# ? Nov 7, 2008 02:25

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

chemosh6969 posted:

Try it without brackets:

p = subprocess.Popen('ls -l', shell=True)

or

subprocess.call('ls -l $HOME', shell=True)

That's how the examples I found worked.
http://blog.doughellmann.com/2007/07/pymotw-subprocess.html

Yes, there are two styles for passing parameters:

p = subprocess.Popen(['ls', '-l'])
p = subprocess.Popen('ls -l', shell=True)

The first is more secure, because there's no worries about the parameters being interpreted to contain shell metacharacters.

# ? Nov 7, 2008 02:49

tbradshaw: Jan 15, 2008; First one must nail at least two overdrive phrases and activate the tilt sensor to ROCK OUT!

chemosh6969 posted:

Try it without brackets:

p = subprocess.Popen('ls -l', shell=True)

or

subprocess.call('ls -l $HOME', shell=True)

That's how the examples I found worked.
http://blog.doughellmann.com/2007/07/pymotw-subprocess.html

You mean try it as a string rather than a list?

That would be equivalent to:

code:

p = subprocess.Popen(['ls -l'], shell=True)

Which would be the other direction to the same solution as what our friend Habnabit stated here. He removed the "shell=True", you instead removed the arguments (instead passing a single string as a single argument).

tbradshaw fucked around with this message at 03:00 on Nov 7, 2008

# ? Nov 7, 2008 02:58

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Need some help: I want to pick out a few "easy to grok" app examples to implement with single-threads, multi threads and then multiprocessing. I want the examples to be as approachable as possible. Ergo, no map-reduces or hadooping

# ? Nov 7, 2008 17:13

Pivo: Aug 20, 2004

Try writing some image filters. Seems like a really easy thing to split up and a lot of the algorithms are well known.

Try writing a little physics simulation maybe a few particles in one thread, then split the gfx and physics up.

# ? Nov 7, 2008 17:17

Bouquet: Jul 14, 2001

Pivo posted:

Try writing a little physics simulation maybe a few particles in one thread, then split the gfx and physics up.

I have been working on a program that does almost exactly this as part of learning python and 3D graphics stuff. I'm using pyglet, which might be a dependency m0nk3yz would rather avoid for his examples, but I'd be happy to share if you'd like.

# ? Nov 7, 2008 19:51

m0nk3yz: Mar 13, 2002; Behold the power of cheese!

Bouquet posted:

I have been working on a program that does almost exactly this as part of learning python and 3D graphics stuff. I'm using pyglet, which might be a dependency m0nk3yz would rather avoid for his examples, but I'd be happy to share if you'd like.

All in all I was thinking common tools/toys people work with - I want to showcase the simple case

# ? Nov 7, 2008 21:49

JoeNotCharles: Mar 3, 2005; Yet beyond each tree there are only more trees.

m0nk3yz posted:

All in all I was thinking common tools/toys people work with - I want to showcase the simple case

Recursive processing of an XML tree is pretty common and easy to parallelize.

# ? Nov 7, 2008 22:16

king_kilr: May 25, 2007

m0nk3yz posted:

Need some help: I want to pick out a few "easy to grok" app examples to implement with single-threads, multi threads and then multiprocessing. I want the examples to be as approachable as possible. Ergo, no map-reduces or hadooping

The app I just did uses one process for the GUI stuff, and a second for the heavy processing, something like that's a pretty good example IMO.

# ? Nov 7, 2008 23:10

Id4ever: Dec 2, 2004

Which of the following is the most pythonic way of checking that an object isn't None:

a) if x:

b) if not x is None:

c) if x is not None:

I think I've seen all of these in the wild. I prefer the first one, but when I see pros like Mark Summerfield use c) I wonder if there's a good reason to avoid a).

# ? Nov 7, 2008 23:51

bitprophet: Jul 22, 2004; Taco Defender

Id4ever posted:

Which of the following is the most pythonic way of checking that an object isn't None:

a) if x:

b) if not x is None:

c) if x is not None:

I think I've seen all of these in the wild. I prefer the first one, but when I see pros like Mark Summerfield use c) I wonder if there's a good reason to avoid a).

First off, anyone using b) is silly since c) says the same thing but is more readable

IMHO.

Secondly, a) is perfectly acceptable in most cases, but is simply less explicit / accurate than c). It's the same reason that databases have NULL as well as boolean False: sometimes False is an acceptable "good" value, whereas in the same scenario the "empty value" -- None -- would be "bad".

# ? Nov 8, 2008 00:18

No Safe Word: Feb 26, 2005

Id4ever posted:

I think I've seen all of these in the wild. I prefer the first one, but when I see pros like Mark Summerfield use c) I wonder if there's a good reason to avoid a).

The good reason is they do different things.

For example, using a) you would skip that branch for when x was "", [], or 0.

# ? Nov 8, 2008 01:05

Habnabit: Dec 30, 2007; lift your skinny fists like
antennas in germany.

No Safe Word posted:

The good reason is they do different things.

For example, using a) you would skip that branch for when x was "", [], or 0.

And you really need to be conscious of this. I've written code that did 'if x:' instead of 'if x is not None:' and it took me a while to figure out why I was getting these weird bugs when x, which was a dictionary, was empty.

# ? Nov 8, 2008 04:27

cowboy beepboop: Feb 24, 2001

m0nk3yz posted:

Need some help: I want to pick out a few "easy to grok" app examples to implement with single-threads, multi threads and then multiprocessing. I want the examples to be as approachable as possible. Ergo, no map-reduces or hadooping

Matrix multiplication?

# ? Nov 8, 2008 08:49

Pivo: Aug 20, 2004

I've been having a bit of a Python vs everything else debate with a CS prof at my school. He brought up an example that totally kills Python performance:

quote:

C code for computing all perfect numbers up to 10000:
code:
#include <stdio.h>

main() {
    int i = 0;
    int j = 0;
    for (i=1; i <= 10000; i++) {
        int acc = 0;
        for (j=1; j < i; j++) {
            if (i % j == 0) {
                acc += j;
            }
        }
        if (acc == i) {
            printf("%d is perfect\n", i);
        }
    }
}
.278 seconds on my MacBook Pro.

Scheme code, using v4 iterator syntax:
code:
#lang scheme

(for* ([i (in-range 1 10001)])
   (when (= i (for/fold ([sum 0])
                        ([j (in-range 1 i)])
                        (if (zero? (modulo i j)) (+ sum j) sum)))
     (printf "~a is perfect\n" i)))
2.4 seconds using the mzscheme interpreter.

Haskell code, using list comprehensions:
code:
divisors :: Int -> [Int]
divisors i = [j | j<-[1..i-1], i `rem` j == 0]
main = print [i | i<-[1..10000], i == sum (divisors i)]
.274 seconds on my MacBook Pro (impressive, considering GHC compiles Haskell to C!)

Python code, using list comprehensions:
code:
print [i for i in range(1,10000) if i == sum([j for j in range(1,i) if i%j == 0])]
13.2 seconds. (So, okay, only about fifty times as slow as Haskell.) List comprehensions are there, but they're just syntactic sugar. Because Python is imperative, it can't do much in the way of optimization.

I did it myself and yeah, I get 0.2 secs for the C code and 12.3 seconds for the Python list comprehension. Expanding out the Python example to be more or less a translation of the C code actually runs a bit slower, 12.7 seconds or so. How can this be?

# ? Nov 8, 2008 16:07

hey mom its 420: May 12, 2007

The Python example your CS prof provided runs in 10.6 seconds on my machine because it creates a lot of intermediate lists in memory only for the purpose of iterating over them. The example is hardly equivalent to the examples in other languages. You can get a nice speed up if you use this code:

code:

is_perfect = lambda x: sum(j for j in xrange(1,x) if x%j == 0) == x
perfect_numbers = (x for x in xrange(1,10000) if is_perfect(x))
for x in perfect_numbers: print x

runs in about 6 seconds for me. So almost a double improvement by using generators.

Anyway, C and Haskell are compiled, while Python is interpreted. You should ask your CS prof what his point is. If his point is that Python is slow or something, that's a pretty weaksauce claim because speed is just one out of many requirements and it has never been a measure of how good a language is on its own. And besides, I'm sure you could make this run quicker if you used Psyco.

# ? Nov 8, 2008 17:01

Pivo: Aug 20, 2004

I told him I'd like for it to be an option to do assignments in Python, and he said it's already an option to do them in Scheme, and why wouldn't you use Scheme? He's a bit of a Scheme fanatic.

We have two CS 'streams': one starts out in Java, then later introduces you to C. The other starts out in Scheme, and later introduces you to C.

Both of them end up in the same 2nd year course about formal language parsing, with the end goal of writing a compiler for a toy language. The options are to use Scheme, Java, and C. A discussion on the newsgroups about how Java doesn't have a built-in notion of tuples used as hash keys, and if you want that behaviour, you have to either write a class with a hashCode and equals function, or use a hashtable of hashtables.

The Scheme prof in question replied that you could just use Scheme, which understands how to hash pairs and how to compare them, and furthermore you can have completely heterogeneous mappings. I emailed him that Python can do exactly the same, but is a lot more approachable to people without the functional background.

And he's showing me that functional style in Python is half-hearted and slow.
Although in this case, the iterative example (directly translated from the C code) is slow too:

code:

dsafonov$ cat perfect2.py
i = 0
j = 0
o = []
for i in range(1, 10000):
 acc = 0
 for j in range(1, i):
   if i % j == 0: acc += j
 if (acc == i):
   o.append(acc)
print o

dsafonov$ time python perfect2.py
[6, 28, 496, 8128]

real    0m14.126s
user    0m13.668s
sys    0m0.271s

Your generator example does run in a little over half the time on my machine. But my prof has a valid point - Scheme is interpreted too, why does it do the same thing so much faster than Python?

Oh and I tried Psyco, has no effect on this example.

# ? Nov 8, 2008 17:48

tef: May 30, 2004; -> some l-system crap ->

Bonus posted:

So almost a double improvement by using generators.

Now with less lambda:

code:

for i in xrange(1,10000): 
     if i == sum(j for j in xrange(1,i) if i%j == 0):
             print i

Edit: We could always try running it in iron python.

# ? Nov 8, 2008 18:13

Pivo: Aug 20, 2004

I'll buy a custom title for anyone (edit: the first person I mean) who can solve this in Python and beat or at least match Scheme. I mean, I don't know how to do it. I'm thinking about learning Haskell heh.

Pivo fucked around with this message at 22:04 on Nov 8, 2008

# ? Nov 8, 2008 22:02

TOO SCSI FOR MY CAT: Oct 12, 2008; this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

Pivo posted:

I'll buy a custom title for anyone (edit: the first person I mean) who can solve this in Python and beat or at least match Scheme. I mean, I don't know how to do it. I'm thinking about learning Haskell heh.

Likely not possible, without dramatically changing the algorithm or using an experimental Python implementation. The same dearth of features that makes Scheme unsuitable for real projects also allows it to execute very quickly.

Also, if tef's solution is moved into a function it's about 2.5 seconds faster (7.6 vs 9.1 on my computer) because the variables will be optimized into registers. Type boxing still limits performance, though.

# ? Nov 8, 2008 22:30

Pivo: Aug 20, 2004

Janin posted:

Likely not possible, without dramatically changing the algorithm or using an experimental Python implementation.

I figured this would be the case but you never know.

# ? Nov 8, 2008 22:47

JoeNotCharles: Mar 3, 2005; Yet beyond each tree there are only more trees.

Janin posted:

Also, if tef's solution is moved into a function it's about 2.5 seconds faster (7.6 vs 9.1 on my computer) because the variables will be optimized into registers. Type boxing still limits performance, though.

Shouldn't psyco be taking care of type boxing? Why doesn't psyco provide more benefit with this code?

# ? Nov 9, 2008 00:22

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

Pivo posted:

I figured this would be the case but you never know.

Pyrex got me under half a second.

# ? Nov 9, 2008 01:09

BigRedDot: Mar 6, 2008

deimos posted:

Pyrex got me under half a second.

Right but Pyrex is generating and compiling an extension to solve the problem in C, and then merely calling it from python. Which is great, but I don't think I'd consider it interesting from the point of "how fast can python do this".

You might say that numpy is now an integral enough part of python to make using it a more acceptable comparison, but unfortunately I don't think the nature of the problem lends itself to being helped by numpy.

BigRedDot fucked around with this message at 01:17 on Nov 9, 2008

# ? Nov 9, 2008 01:12

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

BigRedDot posted:

Right but Pyrex is generating and compiling an extension to solve the problem in C, and then merely calling it from python. Which is great, but I don't think I'd consider it interesting from the point of "how fast can python do this".

You might say that numpy is now an integral enough part of python to make using it a more acceptable comparison, but unfortunately I don't think the nature of the problem lends itself to being helped by numpy.

The general point is that it's not python the language's fault but CPython's fault. In the same way Haskell is fast Python can be made fast.

The overall slowness in the python code is the generator calls inside the sum calls.

tef posted:

Edit: We could always try running it in iron python.

I suspected as much, but it runs slower because it's such a small amount of iterations, there's no chance of JIT optimizing.
It ran slightly slower on my machine (Q6600 running Windows Server 2008 x64).

deimos fucked around with this message at 02:22 on Nov 9, 2008

# ? Nov 9, 2008 02:05

Bunson_Honeydew: May 12, 2001

Greetings...
Would it be alright if my n00b-rear end chimed in with a couple questions from 'How To Think Like a Computer Scientist'? I've made it relatively smoothly to Chapter 7 about Strings. I've made it through most of the exercises, but I'm finding that I am super-stuck. Any input here would be appreciated.

Exercise:
Add body to the following function:

code:

def count(sub, s):
    """
      >>> count('is', 'Mississippi')
      2
      >>> count('an', 'banana')
      2
      >>> count('ana', 'banana')
      2
      >>> count('nana', 'banana')
      1
      >>> count('nanan', 'banana')
      0
    """

What I have gotten so far is:

code:

def count(sub, s, start=0):
	counter = 0
	while start < len(s):
		for letter in s:
			print string.find(s, sub, start)
			counter += 1
			start += 1
	return counter

The result is:

code:

>>> count("an", "banana")
1
1
3
3
-1
-1
6

I have all the essential elements functioning in at least a rudimentary manner:
-loop traversing the string
-string.find returning the location of the input value as it traverses the string
-counter is counting the iterations of the loop and returning its final count

Things it is not doing properly, that I cannot seem figure out (I realize these elements are not present; I have tried many things):
-string.find should update 'start' with the found position, and begin its next search at new 'start'
-loop should stop once string.find returns -1
-counter should only count twice, the two times it finds "an" in "banana", and not each iteration of the loop.

Can anyone throw me a bone?

# ? Nov 9, 2008 03:31

sw-itch: Apr 6, 2008

You can make things way easier:

code:

def count(sub, s):
    return s.count(sub)

# ? Nov 9, 2008 03:41

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

Bunson_Honeydew posted:

Things it is not doing properly, that I cannot seem figure out (I realize these elements are not present; I have tried many things):
-string.find should update 'start' with the found position, and begin its next search at new 'start'
-loop should stop once string.find returns -1
-counter should only count twice, the two times it finds "an" in "banana", and not each iteration of the loop.

Can anyone throw me a bone?

You should really try to do the exercise without using string.find, it's kind of cheating. That being said:

1. You're looping twice through the word, which is not what you want, get rid of the for and just use the while. This is why stuff is getting counted twice.
2. If you want start to have the value that string.find returns, you're gonna have to assign it to start not just add 1 to it!

Hate to fix your assignment for you instead giving you more hints, but it's not that easy to explain:

code:

def count(needle, haystack, start=0): 
	counter = 0 
	while start < len(haystack): 
		i = string.find(haystack, needle, start) 
		if i > -1:
			counter += 1 
			#make start be equal to where string.find found the needle 
			#plus the length of the needle
			start += i + len(needle) 
		else:
			break #exit the loop because string.find returned -1
	return counter

sw-itch posted:

You can make things way easier:
code:
def count(sub, s):
    return s.count(sub)

That's not very useful if he's trying to learn to code. (albeit my code is un-pythonic, but I think it's illustrating what Bunson is trying to learn.

deimos fucked around with this message at 03:54 on Nov 9, 2008

# ? Nov 9, 2008 03:48

Scaevolus: Apr 16, 2007

Pivo posted:

I'll buy a custom title for anyone (edit: the first person I mean) who can solve this in Python and beat or at least match Scheme. I mean, I don't know how to do it. I'm thinking about learning Haskell heh.

Here, this works for the first 3.3 billion positive integers:

code:

def is_perfect(x):
    return x in (6, 28, 496, 8128)

print [x for x in xrange(10000) if is_perfect(x)]

(Yes, Python isn't as fast at number crunching as Scheme)

# ? Nov 9, 2008 08:00

hey mom its 420: May 12, 2007

Here's an implementation that uses some recursion and a list comprehension. tails('hey'), for example, produces ['hey','ey','y','']. So it does that and then checks how many elements in that list start with the needle.

code:

def count(sub, s):
    def tails(xs):
        return [] if not xs else [xs] + tails(xs[1:])
    return len([x for x in tails(s) if x.startswith(sub)])

or you could do away with the intermediary tails function and just write it like this:

code:

def count(sub, s):
    starts = 1 if s.startswith(sub) else 0
    return 0 if not s else starts + count(sub, s[1:])

or if you're feeling like a code golfer you can write this out in one line::

code:

def count(sub, s):
    return 0 if not s else (1 if s.startswith(sub) else 0) + count(sub, s[1:])

hey mom its 420 fucked around with this message at 15:22 on Nov 9, 2008

# ? Nov 9, 2008 15:15

Scaevolus: Apr 16, 2007

Why do you insist on writing Python like you're writing Haskell

Bonus posted:

or if you're feeling like a code golfer you can write this out in one line::
code:
def count(sub, s):
    return 0 if not s else (1 if s.startswith(sub) else 0) + count(sub, s[1:])

code:

def count(sub, s): 
    return s != "" and s.startswith(sub) + count(sub, s[1:])

# ? Nov 9, 2008 20:53

hey mom its 420: May 12, 2007

Because God wants me to. :colbert:

Haha, but still, I don't like using the tricks with and, although I admit, doing s.startswith(sub) + count(sub, s[1:]) is nice, I've been doing Haskell so much that I totally forgot that you can add together booleans and numbers.

# ? Nov 9, 2008 22:51

Adbot: ADBOT LOVES YOU

# ? May 12, 2024 02:46

Thats so Brad!: Jul 5, 2008; by Ozma

After spending the past two weeks attempting to learn python which is supposedly the most newbie friendly language out there, my mind is blown by how people manage to program computer games and rocket ships with this.

# ? Nov 10, 2008 06:03

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »