Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

the: Jul 18, 2004; by Cowcaster

Nah, I'd have to reload all the data again and somehow specify which race it was. They were all 5ks though (although I know that the terrain could have a bit of an effect on the times). Thanks for introducing me to Pandas and Seaborn though. I'd begun to think that Python didn't have any great data analysis/visualization tools. Seaborn looks beautiful.

# ? Jul 22, 2014 05:04

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 02:57

rock2much: Feb 6, 2004; Grimey Drawer

the posted:

Write a code that checks to see if a supplied string is a palindrome or not.

I started doing some more of the Euler ones when I got home so I got the number version of this done. Will try strings tomorrow if work is slow.

code:

top = 0
for a in range(100, 999):
    for b in range(100, 999):
        checkProd = a * b
        checkProd = str(checkProd)
        prodCheck = checkProd[::-1]
        if prodCheck == checkProd:
            if a * b > top:
                top = a * b
print(top)

There's probably an easier way but it's what I got. I wanted to store it as a string and then sort it and it kept coming out with 99999 as the top number. Then I tried to convert prodCheck back into int and that failed too. This works though!

EAT THE EGGS RICOLA posted:

Project Euler is alright, so is checkio.

This is cool too, thanks!

# ? Jul 22, 2014 05:09

the: Jul 18, 2004; by Cowcaster

Try this next:

Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?

# ? Jul 22, 2014 05:21

pmchem: Jan 22, 2010

the posted:

Try this next:

Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?

Bonus for solving in polynomial time?

# ? Jul 22, 2014 05:59

namaste friends: Sep 18, 2004; by Smythe

pmchem posted:

Bonus for solving in polynomial time?

I think you'd also get a Fields Medal for that.

# ? Jul 22, 2014 06:08

ohgodwhat: Aug 6, 2005

the posted:

Interesting, I don't know what you did there, but interesting:

I'm wondering about those 5 99 year olds; they make up a third of the 80-100 bracket. Could they be incorrectly recorded? (I'm actually kind of guessing that the ages are two digits maximum, or perhaps more likely, 99 indicates unknown age)

ohgodwhat fucked around with this message at 13:45 on Jul 22, 2014

# ? Jul 22, 2014 13:42

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

rock2much posted:

I started doing some more of the Euler ones when I got home so I got the number version of this done. Will try strings tomorrow if work is slow.
code:
top = 0
for a in range(100, 999):
    for b in range(100, 999):
        checkProd = a * b
        checkProd = str(checkProd)
        prodCheck = checkProd[::-1]
        if prodCheck == checkProd:
            if a * b > top:
                top = a * b
print(top)
There's probably an easier way but it's what I got. I wanted to store it as a string and then sort it and it kept coming out with 99999 as the top number. Then I tried to convert prodCheck back into int and that failed too. This works though!

This is cool too, thanks!

You're forgetting to check for 999! :spergin:

Remember how range (and slicing) works in python, it's doing 100 <= x < 999.

Try breaking your code into functions, like for example there's clearly some kind of function "is_palindromic" trying to come out of this. I don't know, I think it's good style and makes it easier to reason about.

code:

def is_palindromic(s):
    'Returns true if the string s is palindromic. Returns false otherwise.'
    pass # write your function here

top = 0
for a in range(100, 999):
    for b in range(100, 999):
        checkProd = a * b
        checkProd = str(checkProd)
        if is_palindromic(checkProd):
            if a * b > top:
                top = a * b
print(top)

Now if you want to make it better you can reason about 2 different parts of the problem separately. The first is how you check if the string is a palindrome and the other is how you should search the numbers.

Probably the biggest thing that will help speed up the code is to notice that multiplication is commutative (a*b == b*a) so you don't really need to loop over so many pairs like that, it's a waste of time to check for b*a if you already checked a*b. Also you can reduce the number of times you compute a*b by holding the value in a variable.

# ? Jul 22, 2014 14:00

rock2much: Feb 6, 2004; Grimey Drawer

Symbolic Butt posted:

You're forgetting to check for 999! Remember how range (and slicing) works in python, it's doing 100 <= x < 999.

Try breaking your code into functions, like for example there's clearly some kind of function "is_palindromic" trying to come out of this. I don't know, I think it's good style and makes it easier to reason about.
code:
def is_palindromic(s):
    'Returns true if the string s is palindromic. Returns false otherwise.'
    pass # write your function here

top = 0
for a in range(100, 999):
    for b in range(100, 999):
        checkProd = a * b
        checkProd = str(checkProd)
        if is_palindromic(checkProd):
            if a * b > top:
                top = a * b
print(top)
Now if you want to make it better you can reason about 2 different parts of the problem separately. The first is how you check if the string is a palindrome and the other is how you should search the numbers.

Probably the biggest thing that will help speed up the code is to notice that multiplication is commutative (a*b == b*a) so you don't really need to loop over so many pairs like that, it's a waste of time to check for b*a if you already checked a*b. Also you can reduce the number of times you compute a*b by holding the value in a variable.

You're right about the 100 <= x < 999 thing, forgot all about it.

I can't wrap my mind around how to make the a*b == b*a into less computations. The logic is sound, I just don't know the code to make it less than it is.

Also thanks!

# ? Jul 22, 2014 15:14

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Sweet graphs, the.

# ? Jul 22, 2014 15:21

BabyFur Denny: Mar 18, 2003

rock2much posted:

You're right about the 100 <= x < 999 thing, forgot all about it.

I can't wrap my mind around how to make the a*b == b*a into less computations. The logic is sound, I just don't know the code to make it less than it is.

Also thanks!

You only need to consider cases where b >= a. This is easily achieved by changing where the range for b starts.

You could actually loop over the product of a*b instead of b:

code:

for a in range(100, 1000):
    for b in range(a*a, a*1000,a):
        checkProd = str(b)
        if is_palindromic(checkProd) and b > top:
            top = b

# ? Jul 22, 2014 16:16

rock2much: Feb 6, 2004; Grimey Drawer

BabyFur Denny posted:

You only need to consider cases where b >= a. This is easily achieved by changing where the range for b starts.

You could actually loop over the product of a*b instead of b:
code:
for a in range(100, 1000):
    for b in range(a*a, a*1000,a):
        checkProd = str(b)
        if is_palindromic(checkProd) and b > top:
            top = b

In this example, a is 100
start at 100*100, end at 100*1000, count-up/step by 100

first output is 10000 (100*100)
second is 40000 (200*200), etc etc

When a is 101:
101*101 then 202*202, etc

102 is 102*102 then 204*204, etc

If I'm reading this right, and I'm probably not, does it ever get to do 101*102 and I guess the other numbers it's missing? Does it need to?

# ? Jul 22, 2014 19:20

QuarkJets: Sep 8, 2008

101*102 does get checked. If a is 101, then b starts at 101*101, but on the next iteration b becomes 101*101+101, which is equivalent to 101*102. And then b becomes 101*101+101+101, which is equivalent to 101*103, etc.

For completeness, BabyFur Denny alludes to another way to check every combination of numbers between 99 and 999 only once, which would be something this:

Python code:

for a in range(100, 1000):
    for b in range(a, 1000):
        prod = a*b
        checkProd = str(prod)
        if is_palindromic(checkProd) and prod > top:
            top = prod

First loop, you get to check 100*100, 100*101, 100*102, etc

Second loop, you get to check 101*101, 101*102, 101*103, etc

Eventually a is 998, at which point b only becomes 998 and then 999 (rather than looping through all values between 99 and 999 again)

The two methods are equivalent. What BabyFur Denny posted is a little more compact (by one line), but this one's a little easier to read

QuarkJets fucked around with this message at 19:49 on Jul 22, 2014

# ? Jul 22, 2014 19:41

QuarkJets: Sep 8, 2008

I've decided to solve your problem with a horrific one-liner. Never code like this:

Python code:

top = np.array([a*b for a in xrange(100,1000) for b in xrange(a,1000) if str(a*b) == str(a*b)[::-1]]).max()

# ? Jul 22, 2014 19:58

rock2much: Feb 6, 2004; Grimey Drawer

QuarkJets posted:

I've decided to solve your problem with a horrific one-liner. Never code like this:
Python code:
top = np.array([a*b for a in xrange(100,1000) for b in xrange(a,1000) if str(a*b) == str(a*b)[::-1]]).max()

Thanks for the explanation AND making my head explode with the one liner

. Because I'm a poo brain I saw the (a*a, a*100, a) and thought the a*a portion increased by a each time, so it's like a*a, then 2a*2a, then 3a*3a instead of what it actually does, just increase the total, b, by a .

# ? Jul 22, 2014 20:10

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

You can also use itertools.combinations_with_replacement :ssh:

itertools owns

# ? Jul 22, 2014 20:29

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

Is there a canonical way to get a column vector in numpy? The obvious a.T doesn't work on a normal vector. If I want to get a 1-5 multiplication table in Matlab I can do:

code:

a = 1:5;
a'*a

The similar numpy equivalent:

code:

a = np.arange(1,6)
a.T * a

Doesn't work because a.T is the same thing as a.

There's obviously a bunch of ways to do this:

code:

a = np.arange(1,6)
np.atleast_2d(a).T * a

a.reshape(len(a),1) * a

b = np.array([np.arange(1,6)]).T
a*b

All of those work, but they also seem rather crude.

# ? Jul 22, 2014 22:39

SurgicalOntologist: Jun 17, 2004

Still a bit crude but another way is

Python code:

a = np.arange(6)
a[:, np.newaxis] * a

# ? Jul 22, 2014 22:46

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

KernelSlanders posted:

Is there a canonical way to get a column vector in numpy? The obvious a.T doesn't work on a normal vector. If I want to get a 1-5 multiplication table in Matlab I can do:
code:
a = 1:5;
a'*a

You can use the outer product, that's what I use when I need something like that: np.outer(a, a)

Maybe someone better than me can answer you why working with column vectors aren't that great. :v:

# ? Jul 22, 2014 22:54

Jose Cuervo: Aug 25, 2004

I have the following function:

Python code:

def my_func(data_frame, filter=None):
    ...
    if filter is not None:
        ...

The parameter filter is optional, but if provided will be a string (e.g., 'agent'). I am not sure why (as far as I can tell) the if statement works, i.e., when filter is not specified the code following the if statement is not executed, but if filter is set to something other than None then the code following the if statement is executed.

Any clarification as to why this works would be appreciated.

# ? Jul 23, 2014 05:06

Nippashish: Nov 2, 2005; Let me see you dance!

Jose Cuervo posted:

The parameter filter is optional, but if provided will be a string (e.g., 'agent'). I am not sure why (as far as I can tell) the if statement works, i.e., when filter is not specified the code following the if statement is not executed, but if filter is set to something other than None then the code following the if statement is executed.

Any clarification as to why this works would be appreciated.

If you don't specify a filter then it gets set to None. If the filter is not None then the code inside the if is run.

# ? Jul 23, 2014 05:13

Jose Cuervo: Aug 25, 2004

Nippashish posted:

If you don't specify a filter then it gets set to None. If the filter is not None then the code inside the if is run.

I understand that part. What is unclear to my is why "None is not None" evaluates to False, but something like "'agent' is not None" evaluates to True.

# ? Jul 23, 2014 05:21

QuarkJets: Sep 8, 2008

Jose Cuervo posted:

I understand that part. What is unclear to my is why "None is not None" evaluates to False, but something like "'agent' is not None" evaluates to True.

Because None is not None is a false statement. That's why it evaluates to false. None is None.

Basically you asked Python to evaluate this:

code:

"agent" is not None

This is clearly a True statement; "agent" is not None, it's something else

# ? Jul 23, 2014 05:26

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

"is not" means "are the two things not the same thing"?

None and None are the same thing.

'agent' and None are not the same thing.

# ? Jul 23, 2014 05:33

Nippashish: Nov 2, 2005; Let me see you dance!

Jose Cuervo posted:

I understand that part. What is unclear to my is why "None is not None" evaluates to False, but something like "'agent' is not None" evaluates to True.

There is only one None in python. That's why None is None. On the other hand, is is sometimes pretty weird

code:

Python 2.7.6 |Continuum Analytics, Inc.| (default, Jan 10 2014, 11:23:15) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> None is None
True
>>> 123456789 is 123456789
True
>>> a = 123456789
>>> b = 123456789
>>> a is b
False
>>> a is 123456789
False
>>> a = 5
>>> b = 5
>>> a is b
True

# ? Jul 23, 2014 05:35

ShadowHawk: Jun 25, 2000; CERTIFIED PRE OWNED TESLA OWNER

rock2much posted:

I started doing some more of the Euler ones when I got home so I got the number version of this done. Will try strings tomorrow if work is slow.
Python code:
top = 0
for a in range(100, 999):
    for b in range(100, 999):
        checkProd = a * b
        checkProd = str(checkProd)
        prodCheck = checkProd[::-1]
        if prodCheck == checkProd:
            if a * b > top:
                top = a * b
print(top)
There's probably an easier way but it's what I got. I wanted to store it as a string and then sort it and it kept coming out with 99999 as the top number. Then I tried to convert prodCheck back into int and that failed too. This works though!

This is cool too, thanks!

As an aside to the above changes, there's a quick code smell here: you're calculating a*b 3 times, and I don't think Python is smart enough to cache it.

Python code:

top = 0
for a in range(100, 999):
    for b in range(100, 999):
        product = a * b
        checkProd = str(product)
        prodCheck = checkProd[::-1]
        if prodCheck == checkProd:
            if product > top:
                top = product
print(top)

This is also slightly more readable

# ? Jul 23, 2014 06:20

Lyon: Apr 17, 2003

Nippashish posted:

There is only one None in python. That's why None is None. On the other hand, is is sometimes pretty weird

code:

Python 2.7.6 |Continuum Analytics, Inc.| (default, Jan 10 2014, 11:23:15) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> None is None
True
>>> 123456789 is 123456789
True
>>> a = 123456789
>>> b = 123456789
>>> a is b
False
>>> a is 123456789
False
>>> a = 5
>>> b = 5
>>> a is b
True

is checks the identity of the underlying object. So if you did...

code:

Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> a = [1, 2, 3, 4]
>>> b = a
>>> a is b
True
>>> a = 123456789
>>> b = a
>>> a is b
True
>>> a = 123456789
>>> b = 123456789
>>> a == b
True
>>> a is b
False

So if you want to know if two variables point to the same object you use is, if you want to know if two values are equal you use ==. Integers up to 256 (I think) are kept on the heap so are not actual unique objects. Any Integer over 256 will be a unique object so is will fail to compare them as True.

code:

>>> a = 50
>>> b = 50
>>> a is b
True
>>> a = 300
>>> b = 300
>>> a is b
False
>>> a = 256
>>> b = 256
>>> a is b
True
>>> a = 257
>>> b = 257
>>> a is b
False

Lyon fucked around with this message at 07:08 on Jul 23, 2014

# ? Jul 23, 2014 07:01

Nippashish: Nov 2, 2005; Let me see you dance!

Lyon posted:

So if you want to know if two variables point to the same object you use is, if you want to know if two values are equal you use ==.

I was just trying to point out that when two things are actually the same object is not always intuitively obvious.

# ? Jul 23, 2014 07:05

Lyon: Apr 17, 2003

Oh whoops, misunderstood! Disregard this post.

# ? Jul 23, 2014 07:12

SurgicalOntologist: Jun 17, 2004

It depends on what the meaning of 'is' is.

# ? Jul 23, 2014 07:19

Jewel: May 2, 2009

Nippashish posted:

There is only one None in python. That's why None is None. On the other hand, is is sometimes pretty weird

code:

Python 2.7.6 |Continuum Analytics, Inc.| (default, Jan 10 2014, 11:23:15) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> None is None
True
>>> 123456789 is 123456789
True
>>> a = 123456789
>>> b = 123456789
>>> a is b
False
>>> a is 123456789
False
>>> a = 5
>>> b = 5
>>> a is b
True

The reason for this is because ints under 256 (and also 0 to -5 or something I think) are cache'd so it uses the same object to avoid creating lots of new objects with basic calculations people tend to do. None is also a cache'd object so all None would be the same as any other None.

I'm guessing "123456789 is 123456789" is True because it must be an optimisation to only create a single int object if they're both in the same statement.

# ? Jul 23, 2014 08:32

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

ShadowHawk posted:

As an aside to the above changes, there's a quick code smell here: you're calculating a*b 3 times, and I don't think Python is smart enough to cache it.

lol if you believe multiplies are expensive.

# ? Jul 23, 2014 12:18

ShadowHawk: Jun 25, 2000; CERTIFIED PRE OWNED TESLA OWNER

Suspicious Dish posted:

lol if you believe multiplies are expensive.

They're not (especially not at the range for the problem at hand), but avoiding redundant math in general does have other benefits (easier thread safety, for instance).

# ? Jul 23, 2014 14:31

ShadowHawk: Jun 25, 2000; CERTIFIED PRE OWNED TESLA OWNER

Style question:

Python code:

dudes = {'bob':'fedora', 'joe':'bowler'}
for (dude, hat) in dudes.items():
    eat(dude,hat)

or:

Python code:

dudes = {'bob':'fedora', 'joe':'bowler'}
for dude, hat in dudes.items():
    eat(dude,hat)

# ? Jul 23, 2014 14:43

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Neither a fedora nor bowler hat are going to be stylish. Try a baseball cap or a pork pie hat.

# ? Jul 23, 2014 15:12

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

ShadowHawk posted:

Style question:

Python code:

dudes = {'bob':'fedora', 'joe':'bowler'}
for (dude, hat) in dudes.items():
    eat(dude,hat)

or:

Python code:

dudes = {'bob':'fedora', 'joe':'bowler'}
for dude, hat in dudes.items():
    eat(dude,hat)

No parentheses 4 lyfe.

# ? Jul 23, 2014 15:31

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

Thermopyle posted:

No parentheses 4 lyfe.

Agree w/ this dude.

# ? Jul 23, 2014 16:06

the: Jul 18, 2004; by Cowcaster

Stupid question, I know, but I'm not sure why this isn't working:

code:

In [39]: query_opps[1]
Out[39]: 
{'Id': '0060000000ASckcAAD',
 'OpportunityContactRoles': [{'Id': '',
   'IsPrimary': True,
   'type': 'OpportunityContactRole'}],
 'type': 'Opportunity'}

code:

In [40]: query_opps[1]['OpportunityContactRoles']
Out[40]: [{'Id': '', 'IsPrimary': True, 'type': 'OpportunityContactRole'}]

code:

In [41]: query_opps[1]['OpportunityContactRoles']['IsPrimary']
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-41-07f7825d20d6> in <module>()
----> 1 query_opps[1]['OpportunityContactRoles']['IsPrimary']

/usr/local/lib/python2.7/dist-packages/beatboxxx-21.5-py2.7.egg/beatbox/python_client.pyc in __getitem__(self, n)
     45                 return getattr(self, n)
     46             except AttributeError, n:
---> 47                 raise KeyError
     48         else:
     49             return list.__getitem__(self, n)

KeyError:

# ? Jul 23, 2014 16:22

salisbury shake: Dec 27, 2011

query_opps[1]['OpportunityContactRoles'] is a list with a dictionary in it so do:

code:

query_opps[1]['OpportunityContactRoles'][0]['IsPrimary']

# ? Jul 23, 2014 16:26

the: Jul 18, 2004; by Cowcaster

I have a list of 176,876 Ids (strings), and I am trying to find an easy way to group them by occurrences (so I can plot it later). Is this something that Pandas can do?

I've loaded them into a DataFrame. Running something like df.pandas.groupby('ID').group() seems to sort of do what I want (printing out every string and the list locations where it occurs), but I'm really looking for something like:

code:

       ID : Occurrence
342000XBB : 37
200333XCC : 31
342203CBB : 17

edit: I think df.groupby('Id').size() solved the problem!

Second edit: Importing the list into a dataframe, I ended up having to write the list to a CSV file and then load it into the dataframe. I couldn't find a way to load a list into a dataframe directly even after reading the Pandas documentation. Did I miss something?

the fucked around with this message at 19:44 on Jul 23, 2014

# ? Jul 23, 2014 19:41

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 02:57

Jose Cuervo: Aug 25, 2004

the posted:

I have a list of 176,876 Ids (strings), and I am trying to find an easy way to group them by occurrences (so I can plot it later). Is this something that Pandas can do?

I've loaded them into a DataFrame. Running something like df.pandas.groupby('ID').group() seems to sort of do what I want (printing out every string and the list locations where it occurs), but I'm really looking for something like:
code:
       ID : Occurrence
342000XBB : 37
200333XCC : 31
342203CBB : 17
edit: I think df.groupby('Id').size() solved the problem!

Second edit: Importing the list into a dataframe, I ended up having to write the list to a CSV file and then load it into the dataframe. I couldn't find a way to load a list into a dataframe directly even after reading the Pandas documentation. Did I miss something?

You might also want to check out value_counts. For example if your column was named 'ID' then you would use the call:

Python code:

df['ID'].value_counts()

Jose Cuervo fucked around with this message at 20:35 on Jul 23, 2014

# ? Jul 23, 2014 20:32

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »