Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Dren: Jan 5, 2001; Pillbug

keyframe posted:

Another day another question

I am trying to write a function that appends a user input number to the end of the list, this is what I have so far:
code:
numbers = [1,4,5]


def list_adder(n):
    n.append(int(input("enter number")))
    return n
  
    
    
print list_adder(numbers)
It works when you enter a number. I have been trying to write a if statement inside the function that checks if the input is numbers and give a "not a number" message if it is string. Did some search online and it seems the way to do a check for that is .isdigit() but it seems isdigit does not work with lists. What can I do here to accomplish this? Using python 2.7.5 by the way.

You need to use raw_input instead of input. In Python 2.7, input causes the user's input to be eval'd, which you don't want. raw_input gives you back a string.

Try separating the tasks of getting user input, validating user input, and appending to a list.

Python code:

def list_adder(n):
    def get_input():
        return raw_input("enter number: ")
    
    def validate_input(user_input):
        # write validation code here
        # return False for now b/c there is no validation code
        return False

    user_input = get_input()
    while not validate_input(user_input):
        print "ERROR: '%s' not a valid number" % user_input
        user_input = get_input()

    # now that the input has been validated as an integer, cast it to an int
    user_number = int(user_input)
    n.append(user_number)

numbers = [1,4,5]
list_adder(numbers)
print numbers

# ? Aug 27, 2013 18:46

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 14:56

the: Jul 18, 2004; by Cowcaster

Simple question, but googling can't find me an answer.

Can I use numpy.arange to make a 2-d array? I can't figure out how that's possible. Thanks.

# ? Aug 28, 2013 03:54

digitalcamo: Jul 11, 2013

I've tried everything with this piece code as I did not want to have to ask. But I've been jacking with it for almost an hour and it's getting rather annoying.

code:


students = [lloyd, tyler, josh]

def get_class_average(student):
	total = 0
		for student in students:
			total += average(student)
		return float(total) / len(students)

get_class_average(students)

I keep getting IndentationError: unexpected indent at for student in students:
I have checked all my spacing and everything has the correct spacing. Is there some small whitespace error I'm not seeing?

# ? Aug 28, 2013 03:57

the: Jul 18, 2004; by Cowcaster

Why do you indent at the for? Have you tried just moving everything back an indent and seeing if it works?

# ? Aug 28, 2013 03:58

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

What's the recommended python binary parsing library? Something like Data.Binary for Haskell is what I'd really like (declarative decoding/encoding)

# ? Aug 28, 2013 04:04

digitalcamo: Jul 11, 2013

the posted:

Why do you indent at the for? Have you tried just moving everything back an indent and seeing if it works?

I thought you had to start all for loops with an indention.

# ? Aug 28, 2013 04:18

the: Jul 18, 2004; by Cowcaster

digitalcamo posted:

I thought you had to start all for loops with an indention.

I'm no expert, but I'm pretty sure you don't (as my own code I'm looking at right now doesn't have it):

Python code:

div = 1000.
size = 1000
d = numpy.zeros(size)
def f(r):
    return numpy.sin(r)/((r)**0.5)
for t in range(1,size):
    dt = t/div
    x = numpy.arange(0,t,dt)
    integral = 0
    for i in range(1,x.size-1):                
        integral += (dt/2)*(f(x[i])+f(x[i+1]))
    d[t] = integral
    
pylab.plot(x,d)
pylab.show()

# ? Aug 28, 2013 04:27

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Malcolm XML posted:

What's the recommended python binary parsing library? Something like Data.Binary for Haskell is what I'd really like (declarative decoding/encoding)

Real-world formats are too messy to be entirely declarative. I'd just write custom code using struct.parse.

Though I've heard some people talk about how awesome Construct is. It's just not good enough for my use case (which is admittedly insane)

# ? Aug 28, 2013 04:34

Haystack: Jan 23, 2005

digitalcamo posted:

I thought you had to start all for loops with an indention.

Nope. An indented block of code generally means 'this block of code is executed within the logical context of the thing its indented under.' You only have to indent immediately after a statement that by it's nature encloses a block of code, such as is the case with for, if, while, and def statements. As a general rule of thumb, if a line ends with a colon, the next line should be indented. If not, you don't (and most often shouldn't) indent.

A corrected version of your code looks like this:

code:

students = [lloyd, tyler, josh]

def get_class_average(student):
	total = 0
	for student in students:
		total += average(student)
	return float(total) / len(students)

get_class_average(students)

# ? Aug 28, 2013 04:47

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

Suspicious Dish posted:

Real-world formats are too messy to be entirely declarative. I'd just write custom code using struct.parse.

Though I've heard some people talk about how awesome Construct is. It's just not good enough for my use case (which is admittedly insane)

I'll give that a look.

What kind of stupid format are you using that Data.Binary/construct can't decode/encode?

# ? Aug 28, 2013 04:55

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

SWF. Don't ask.

# ? Aug 28, 2013 04:57

Dominoes: Sep 20, 2007

the posted:

Simple question, but googling can't find me an answer.

Can I use numpy.arange to make a 2-d array? I can't figure out how that's possible. Thanks.

np.arange(15).reshape(3,5)

This is the first example in the official? Numpy tutorial.

# ? Aug 28, 2013 05:17

the: Jul 18, 2004; by Cowcaster

Dominoes posted:

np.arange(15).reshape(3,5)

This is the first example in the official? Numpy tutorial.

Well, hm..

So, the command

Python code:

numpy.arange(0,10,1)

Would give me a 1d array 0..10, right? but if I used that reshape command, wouldn't that just split the 0..10? I'm trying to make a 2d array that goes 0..10 in both dimensions.

# ? Aug 28, 2013 05:21

Dominoes: Sep 20, 2007

the posted:

Well, hm..

So, the command
Python code:
numpy.arange(0,10,1)
Would give me a 1d array 0..10, right? but if I used that reshape command, wouldn't that just split the 0..10? I'm trying to make a 2d array that goes 0..10 in both dimensions.

0-9. Yes. What do you mean by in both dimensions? Write out a truncated example of what you're looking for.

np.arange(100).reshape(10,10) will you give you 0-99, in 2 axes split in increments of 10.

Dominoes fucked around with this message at 05:31 on Aug 28, 2013

# ? Aug 28, 2013 05:28

the: Jul 18, 2004; by Cowcaster

Dominoes posted:

0-9. What do you mean by in both dimensions? Write out a truncated example of what you're looking for.

np.arange(100).reshape(10,10) will you give you 0-99, in 2 axes split in increments of 10.

I'm trying to make a coordinate grid of points that go from 0 to 10 in both "x and y" dimensions. So I want a 2d array that has (0,0, (0,1), (1,0), (1,1), etc...

thanks!

# ? Aug 28, 2013 05:31

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Suspicious Dish posted:

SWF. Don't ask.

Why do you need to develop your own parser instead of using one that someone else already wrote?

For example: https://github.com/timknip/pyswf

# ? Aug 28, 2013 06:30

Nippashish: Nov 2, 2005; Let me see you dance!

the posted:

I'm trying to make a coordinate grid of points that go from 0 to 10 in both "x and y" dimensions. So I want a 2d array that has (0,0, (0,1), (1,0), (1,1), etc...

thanks!

Try meshgrid.

# ? Aug 28, 2013 07:05

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Haystack posted:

code:

students = [lloyd, tyler, josh]

def get_class_average(student):
	total = 0
	for student in students:
		total += average(student)
	return float(total) / len(students)

get_class_average(students)

There is still an issue with variables in this code. (Although it will run and give the expected answer, thanks to Python's variable lookup rules.)

# ? Aug 28, 2013 08:49

QuarkJets: Sep 8, 2008

the posted:

I'm trying to make a coordinate grid of points that go from 0 to 10 in both "x and y" dimensions. So I want a 2d array that has (0,0, (0,1), (1,0), (1,1), etc...

thanks!

I still don't understand your request. It sounds like you want a 2D array of tuple coordinates or something. Is that what you want? I think that numpy only supports basic data types, so you wouldn't be able to use arange for something like this. If you want two 2D arrays, one with 0...10 in the X direction and one with 0...10 in the Y direction, then that's certainly feasible.

# ? Aug 28, 2013 09:39

tef: May 30, 2004; -> some l-system crap ->

Malcolm XML posted:

What's the recommended python binary parsing library? Something like Data.Binary for Haskell is what I'd really like (declarative decoding/encoding)

There isn't one. I don't know of any combinator parsers for python for binary data, and I would imagine they're horrifically slow on cpython.

In the meantime, there is always pack/unpack http://docs.python.org/2/library/struct.html

# ? Aug 28, 2013 10:30

Dominoes: Sep 20, 2007

the posted:

I'm trying to make a coordinate grid of points that go from 0 to 10 in both "x and y" dimensions. So I want a 2d array that has (0,0, (0,1), (1,0), (1,1), etc...

thanks!

You could just use cells of the array I posted (or np.zeros(100).reshape(10,10), and reference them as coordinates like so: a[0][1].

Like Quarkjets said, you can't use tuples in numpy arrays.

Dominoes fucked around with this message at 13:53 on Aug 28, 2013

# ? Aug 28, 2013 13:50

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

BeefofAges posted:

Why do you need to develop your own parser instead of using one that someone else already wrote?

For example: https://github.com/timknip/pyswf

Because I wrote it before that one (and any other Python one; I checked) existed:

https://github.com/magcius/fusion

# ? Aug 28, 2013 15:11

the: Jul 18, 2004; by Cowcaster

Thanks for the help on the arrays, I was able to figure it out.

Now, let's say I have a 10x10 array that has in each spot either a 0 or a 1. How would I go about plotting only the spots with 1, and in the coordinates that have a 1 in them? Like if array[5,3] has a 1, I'd put a dot at (5,3).

I was thinking something like:

Python code:

for i in range (0,lattice.size):
     for j in range (0,lattice.size):
          if lattice[i,j] = 1:
               pylab.plot(i,j)

edit: Hmm, tried it and I got a blank graph. How do I plot individual coordinate points with pylab?

# ? Aug 28, 2013 16:42

Nippashish: Nov 2, 2005; Let me see you dance!

the posted:

Now, let's say I have a 10x10 array that has in each spot either a 0 or a 1. How would I go about plotting only the spots with 1, and in the coordinates that have a 1 in them? Like if array[5,3] has a 1, I'd put a dot at (5,3).

You can use spy() from matplotlib.

# ? Aug 28, 2013 16:51

the: Jul 18, 2004; by Cowcaster

Nippashish posted:

You can use spy() from matplotlib.

Oh wow, That's friggin awesome! Thanks so much.

the fucked around with this message at 17:26 on Aug 28, 2013

# ? Aug 28, 2013 16:51

No Safe Word: Feb 26, 2005

Malcolm XML posted:

What's the recommended python binary parsing library? Something like Data.Binary for Haskell is what I'd really like (declarative decoding/encoding)

If it's your own data format you could use something like protobuf, which has first-tier support for Python.

# ? Aug 28, 2013 18:10

Dren: Jan 5, 2001; Pillbug

Suspicious Dish posted:

Real-world formats are too messy to be entirely declarative. I'd just write custom code using struct.parse.

Though I've heard some people talk about how awesome Construct is. It's just not good enough for my use case (which is admittedly insane)

When I've needed to parse binary data in python I've always rolled my own stuff with struct.pack/unpack that behaves a lot like Construct. I'll give Construct a shot next time.

Is your SWF so crazy that you can't employ something like Construct for pieces of it?

# ? Aug 28, 2013 19:24

madkapitolist: Feb 5, 2006

Alligator posted:

proxies is supposed to be a dict mapping a protocol to a proxy. if they're http proxies it'll look like this (untested):
Python code:
import json
import requests

with open('proxies.txt') as proxies:
    for line in proxies:
        proxy = json.loads(line)
        
        with open('urls.txt') as urls:
            for line in urls:
                url = line.rstrip()
                data = requests.get(url, proxies={'http': line})

I made some changes to this script and it seems like it should work but I'm still getting errors. Any ideas here? Only reason im printing the html is to give my self some sort of test output.

Python code:

import requests
import json
import urlparse
 
with open('proxies.txt') as proxies:
    for line in proxies:
        proxy=json.loads(line)
       
        with open('urls.txt') as urls:
            for line in urls:
               
                url=line.rstrip()
                if urlparse.urlparse(url).scheme not in proxy:
                    continue
                data=requests.get(url, proxies=proxy)
                data1=data.content
                print data1

This is the format of my proxies.txt file

{"https": "https://61.155.159.9:8989"}
{"http": "http://24.172.34.114:8181"}
{"http": "http://87.236.210.45:443"}

# ? Aug 28, 2013 19:43

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

I'm trying to be all fancy and clever and make a one-liner that takes

code:

ID, Product1, Product2, ...
0, [url]http://...,[/url] [url]http://...,[/url] ...

and turns it into

code:

ID, Product1, Address

Denormalize it, basically.

I came up with this:

Python code:

inpt = csv.DictReader(open('../inventory.csv', 'rb'))
splat = [[row['ID'], prod, row[prod]] for prod in inpt.fieldnames[1:] for row in inpt]

But splt only contains the first (well, second) field name for some reason. len(inpt.fieldnames[1:]) is 14, so it's finding the headers correctly.

Python code:

[[row['ID'], prod, row[prod]] for prod in inpt.fieldnames[1:] for row in [inpt.next()]]

Behaves the way I expect. I'm stumped.

I know how to do it with a couple of regular loops, but I'm annoyed that I can't get it to work this way, too.

# ? Aug 28, 2013 19:50

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Munkeymon posted:

I came up with this:
Python code:
inpt = csv.DictReader(open('../inventory.csv', 'rb'))
splat = [[row['ID'], prod, row[prod]] for prod in inpt.fieldnames[1:] for row in inpt]
But splt only contains the first (well, second) field name for some reason. len(inpt.fieldnames[1:]) is 14, so it's finding the headers correctly.

I take it you are aware that "for row in inpt" is the inner loop here? The result of that code will be that splat contains the "Product 1" line for each and every item in order, followed by the "Product 2" line for each and every item in order, etc.

quote:

Python code:
[[row['ID'], prod, row[prod]] for prod in inpt.fieldnames[1:] for row in [inpt.next()]]
Behaves the way I expect. I'm stumped.

On the other hand, I do not understand how this code can be giving you the result you expect. I would expect it to give you "Product 1" for the first item, then "Product 2" for the second item, and so on until it runs out of product numbers.

Also, use next(inpt) instead of inpt.next(), for future-proofing.

# ? Aug 28, 2013 20:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

madkapitolist posted:

I made some changes to this script and it seems like it should work but I'm still getting errors.

What errors?

# ? Aug 28, 2013 20:52

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Hammerite posted:

I take it you are aware that "for row in inpt" is the inner loop here?

Welp, that's the problem - I got the order of evaluation mixed up. Probably why I don't use nested (or even conditional) comprehensions more often and a reason I should: I don't parse them correctly at first glance. In my defense, nested comprehensions read rather awkwardly, unlike basically everything else in Python.

quote:

On the other hand, I do not understand how this code can be giving you the result you expect. I would expect it to give you "Product 1" for the first item, then "Product 2" for the second item, and so on until it runs out of product numbers.

It works as I expected it to work for one row, which probably should have tipped me off.

# ? Aug 28, 2013 21:06

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Munkeymon posted:

Welp, that's the problem - I got the order of evaluation mixed up. Probably why I don't use nested (or even conditional) comprehensions more often and a reason I should: I don't parse them correctly at first glance. In my defense, nested comprehensions read rather awkwardly, unlike basically everything else in Python.

The order of for clauses in comprehensions doesn't make sense to me either. Unfortunately it seems (based on my experiences) that is seen as sensible by a majority of Python users, so I guess we're out of luck there. It is not all that often that a comprehension with multiple for clauses is the best and most readable way of expressing something, anyway.

# ? Aug 28, 2013 21:47

Dominoes: Sep 20, 2007

Circular imports are bad.

I've been splitting my programs into multiple modules as they've increased in complexity. I've generally been able to do it where there's a primary module that calls the others. I'm currently trying to split off a module that needs to update the GUI in the primary module. Is there a clean way to do this without circular imports? Should I just go ahead, but do the import in a function? I need to send a QT signal.

# ? Aug 28, 2013 22:32

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Dren posted:

When I've needed to parse binary data in python I've always rolled my own stuff with struct.pack/unpack that behaves a lot like Construct. I'll give Construct a shot next time.

Is your SWF so crazy that you can't employ something like Construct for pieces of it?

SWF is an extremely crazy bit-packed format.

Instead of being byte-aligned, it simply strings together all bytes into one giant bitstream.

To read a rectangle, you first read 5 bits from the stream as an unsigned integer, call it "NBits", then read NBits bits for x, NBits bits for y, NBits bits for width, and NBits bits for height.

It's also very inconsistent; sometimes it wants to flush to the next byte boundary, sometimes it doesn't, and the rules for when are very ill-defined.

It also gets extremely messy when dealing with endianness. When reading 16 bits, sometimes it wants you to read them in little-endian order, but when reading a non-byte-aligned number of bits, it wants you to read them in big-endian order always.

And that's just about parsing the bitstream. There's other craziness layered on top.

# ? Aug 28, 2013 23:21

FoiledAgain: May 6, 2007

I hope this isn't a dumb question, but I'm still wrapping my head around generators...

I'm using all() to check a really large list (~5000 elements) and currently it's done this way:

if all([test(x) for x in X]): dostuff()

Python documentation says all() does this:

code:

def all(iterable):
    for element in iterable:
        if not element:
            return False
    return True

Supposing the first element fails the test, the program still has to access that list 5001 time before realizing that, right? It appends 5000 elements to the list first, passes that list to all(), checks 1 element and returns False. Worst case, I have to append 5000 times and then check 5000 times.

If I do this with a generator, I can avoid this problem. I think. Suppose the first item fails the test. all((test(x) for x in X)) would create one generator object, not a list of 5000 items, pass that generator to all(),and then return false after only checking 1 thing. Worst case it only checks 5000 things.

Am I understanding correctly?

# ? Aug 28, 2013 23:53

accipter: Sep 12, 2003

Dominoes posted:

Circular imports are bad.

I've been splitting my programs into multiple modules as they've increased in complexity. I've generally been able to do it where there's a primary module that calls the others. I'm currently trying to split off a module that needs to update the GUI in the primary module. Is there a clean way to do this without circular imports? Should I just go ahead, but do the import in a function? I need to send a QT signal.

Instead of telling the GUI to update, emit a signal from the model/data class. On the GUI side of things, connect to the signal of the model/data class and update the GUI as needed.

# ? Aug 28, 2013 23:53

accipter: Sep 12, 2003

FoiledAgain posted:

I hope this isn't a dumb question, but I'm still wrapping my head around generators...

I'm using all() to check a really large list (~5000 elements) and currently it's done this way:

if all([test(x) for x in X]): dostuff()

Python documentation says all() does this:
code:
def all(iterable):
    for element in iterable:
        if not element:
            return False
    return True
Supposing the first element fails the test, the program still has to access that list 5001 time before realizing that, right? It appends 5000 elements to the list first, passes that list to all(), checks 1 element and returns False. Worst case, I have to append 5000 times and then check 5000 times.

If I do this with a generator, I can avoid this problem. I think. Suppose the first item fails the test. all((test(x) for x in X)) would create one generator object, not a list of 5000 items, pass that generator to all(),and then return false after only checking 1 thing. Worst case it only checks 5000 things.

Am I understanding correctly?

I have not looked at the all() code, but if what you pasted is correct then the loop is exited at the first element that fails the test. The return statement exits the loop.

An example:

Python code:

def test(i):
    print(i)
    if i > 3:
        return False
    return True

all(test(i) for i in range(10))
0
1
2
3
4
Out[12]: False

accipter fucked around with this message at 00:01 on Aug 29, 2013

# ? Aug 28, 2013 23:56

Pollyanna: Mar 5, 2005; Milk's on them.

Say I have a list like [a, b, c, 0, d, e, f, 0, g, h, i]. How do I extract all elements between instances of 0?

# ? Aug 29, 2013 00:14

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 14:56

Lexical Unit: Sep 16, 2003

Pollyanna posted:

Say I have a list like [a, b, c, 0, d, e, f, 0, g, h, i]. How do I extract all elements between instances of 0?

Python code:

In [1]: a = ['a', 'b', 'c', 0, 'd', 'e', 'f', 0, 'g', 'h', 'i']

In [2]: b = [a for a in a if a != 0]

In [3]: b
Out[3]: ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']

Like that?

# ? Aug 29, 2013 00:18

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »