Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

emoji: Jun 4, 2004

suffix posted:

It depend on the framework and the server. IIRC CherryPy has a decent threaded server for development mode, while Flask and Bottle just uses a single-threaded server.

When you're using a separate WSGI server, it will usually spawn a number of worker processes or threads and delegate incoming requests to them, but I think most of them can be configured to thread/process-per-request.

The WSGI PEP specifies how a framework indicates support for multithreading or multiprocessing.

In bottle, it's as easy as

Python code:

bottle.run(server='paste')  # or cherrypy, or a number of others

# ? Jul 20, 2014 21:16

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 05:59

SurgicalOntologist: Jun 17, 2004

Carrier posted:

I'm basically trying to simulate an object responding to commands to move to a certain position and I want to simulate some sort of delay in this response so that sometimes, it might not make it to the position its supposed to reach before it gets given another command. Sorta simulating like a car being told to go to point A, then to point B, and so on but it can only travel at a certain speed to get there. In the long term I was hoping to plot this movement on a graph as I read a set of preset target positions from a file, but for now I'd settle for just numerically being able to do this kind of simulation. So visually, I'd have something like (using O for object, T for target position):
code:
---------O------------T--------- t = 0
------------O---------T--------- t = 1
---------------O------T--------- t = 2
------------------O---T--------- t = 3
---------------------O---------T t = 4
With a new target position being defined at t = 4 etc. and the potential for the new target position to be in the opposite direction.

Yeah I understand all that but I'm not sure where threading comes in. Is there a real-time component?

I think you're making it more complicated than you need to. Here's what I would do: create an object with attributes x and dx_dt (i.e. velocity). They can be scalars, or vectors for multidimensional position. To update the simulated position, do self.x += self.dx_dt * dt where dt is the time since the last update. To represent a new command being received, assign a new value to dx_dt. From there it's pretty easy to simulate things like a delay in command time or to calculate the commanded dx_dt based on some target position compared to the simulated position.

# ? Jul 20, 2014 21:22

Dominoes: Sep 20, 2007

HardDisk posted:

I need to parse ISO 8601 dates and I found the dateutil module (https://pypi.python.org/pypi/python-dateutil/1.5). However, the latest version for python 2.X is 1.5, which is 4 years old by now. Is there any recommendations against it?

You should try Arrow.

Python code:

>>> arrow.get('2013-09-30T15:34:00.000-07:00')
<Arrow [2013-09-30T15:34:00-07:00]>

# ? Jul 20, 2014 21:38

Space Kablooey: May 6, 2009

Looks very interesting. Will take a look as soon as possible. Thanks.

# ? Jul 20, 2014 21:56

Carrier: May 12, 2009; 420...69...9001...

SurgicalOntologist posted:

Yeah I understand all that but I'm not sure where threading comes in. Is there a real-time component?

I think you're making it more complicated than you need to. Here's what I would do: create an object with attributes x and dx_dt (i.e. velocity). They can be scalars, or vectors for multidimensional position. To update the simulated position, do self.x += self.dx_dt * dt where dt is the time since the last update. To represent a new command being received, assign a new value to dx_dt. From there it's pretty easy to simulate things like a delay in command time or to calculate the commanded dx_dt based on some target position compared to the simulated position.

Would this method work for more complicated velocities? I was eventually hoping to have the object move towards the target following a critically damped harmonic oscillator motion.

# ? Jul 20, 2014 22:12

fritz: Jul 26, 2003

Carrier posted:

Would this method work for more complicated velocities? I was eventually hoping to have the object move towards the target following a critically damped harmonic oscillator motion.

Look into Kalman filters.

# ? Jul 20, 2014 22:20

SurgicalOntologist: Jun 17, 2004

Carrier posted:

Would this method work for more complicated velocities? I was eventually hoping to have the object move towards the target following a critically damped harmonic oscillator motion.

It sounds like you might want to represent the object's behavior as a dynamical system (system of first-order ODEs) given some control parameters. Then you could use scipy.integrate to update its real position (what I suggested previously was basically this but with the dynamics being simply dx_dt = velocity with velocity as the only control parameter, and using an Euler solver to integrate).

Then you could implement an input function that updates the controller's perceived position of the object, perhaps with delay and noise, and an output function that updates the control parameters of the object, perhaps with delay and noise.

And finally the controller logic that takes the controller's perceived position of the object and decides what commands to send to the output function. This is where a Kalman filter could go if I'm understanding that suggestion correctly.

# ? Jul 20, 2014 22:30

Carrier: May 12, 2009; 420...69...9001...

Thanks for everyones help! Looks like this is going to be a little more involved than I had first hoped... Should be fun!

# ? Jul 20, 2014 22:37

the: Jul 18, 2004; by Cowcaster

Is there a way to use list.count() for a 2 dimensional list?

I have a list that's like:

mylist = [['dog',True],['cat',False],['horse',True]]

And mylist.count('dog') and mylist.count(True) both return 0. If the command doesn't work, is there another way? I was just going to do something like:

code:

for row in mylist:
     if row[0] == 'dog':
          dcount += 1
     if row[1] == True:
          tcount += 1

But I didn't know if there was a more "Pythonic" method. Thanks.

# ? Jul 20, 2014 22:41

SurgicalOntologist: Jun 17, 2004

Well you could either flatten the list first, then use count, or if you know you're only looking for dogs and cats in the first element (etc.) something like

code:

n_dogs = sum(row[0] == 'dog' for row in mylist)
n_true = sum(row[1] for row in mylist)  # No need to check if True == True, it's already True

That's taking advantage of the fact that True is 1 when summed, so maybe it's more explicit to do

code:

n_dogs = sum(1 if row[0] == 'dog' else 0 for row in mylist)
n_true = sum(1 if row[1] else 0 for row in mylist)

# ? Jul 20, 2014 23:58

Lyon: Apr 17, 2003

the posted:

Is there a way to use list.count() for a 2 dimensional list?

I have a list that's like:

mylist = [['dog',True],['cat',False],['horse',True]]

And mylist.count('dog') and mylist.count(True) both return 0. If the command doesn't work, is there another way? I was just going to do something like:
code:
for row in mylist:
     if row[0] == 'dog':
          dcount += 1
     if row[1] == True:
          tcount += 1
But I didn't know if there was a more "Pythonic" method. Thanks.

What are you trying to do? Without knowing any more details and just trying to provide a quick solution I'm going to suggest using a dictionary.

code:

counts = {}
for row in mylist:
    counts[row[0]] = counts.get(row[0], 0) + 1

This will create an empty dict and then for each row in your list it is going to then create a new key with value 1 if they key does not exist or it is going to increment an existing key's value by 1. Then you would just print counts['dog'] and that would print out the number of dog entries in the list etc etc.

# ? Jul 21, 2014 00:00

the: Jul 18, 2004; by Cowcaster

Lyon posted:

What are you trying to do?

I have a list that contains two columns: Column A is made up a mix of 4 different strings, and Column B is True or False.

I want to easily get a count of one of the strings in Column A and the amount of Trues.

# ? Jul 21, 2014 00:07

Nippashish: Nov 2, 2005; Let me see you dance!

the posted:

I have a list that contains two columns: Column A is made up a mix of 4 different strings, and Column B is True or False.

I want to easily get a count of one of the strings in Column A and the amount of Trues.

If you want to deal with tables of data in python you should very seriously consider using pandas.

# ? Jul 21, 2014 00:28

the: Jul 18, 2004; by Cowcaster

Nippashish posted:

If you want to deal with tables of data in python you should very seriously consider using pandas.

SurgicalOntologist has already been grumbling at me to use it for my 5k data experiment earlier in this thread, I'll give it a look eventually!

# ? Jul 21, 2014 00:41

SurgicalOntologist: Jun 17, 2004

the posted:

SurgicalOntologist has already been grumbling at me to use it for my 5k data experiment earlier in this thread, I'll give it a look eventually!

Although in truth you should be using pandas, I don't believe that's what I grumbled.

SurgicalOntologist posted:

You don't need pandas for that though (well you need it installed as a dependency), you can make the plots straight from your existing lists

# ? Jul 21, 2014 02:06

ohgodwhat: Aug 6, 2005

the posted:

I have a list that contains two columns: Column A is made up a mix of 4 different strings, and Column B is True or False.

I want to easily get a count of one of the strings in Column A and the amount of Trues.

Python code:

import pandas as pd

df = pd.DataFrame({
"A": ["abc", "xyz", "abc", "abc", "xyz", "ijk",...],
"B": [True, False, False, True, False, True,...]})

df.groupby('A').sum()

That should do it, I think - and it's really the simplest example of the power of groupby. Pandas makes everything better.

(Any word on wesm's next thing? I cannot remember what it's called for the life of me, but it's supposed to be a better pandas IIRC)

# ? Jul 21, 2014 02:14

BannedNewbie: Apr 22, 2003; HOW ARE YOU? -> YOSHI?
FINE, THANK YOU. -> YOSHI.

ohgodwhat posted:

(Any word on wesm's next thing? I cannot remember what it's called for the life of me, but it's supposed to be a better pandas IIRC)

It's an online data analytics system or something like that. He gave a talk about it at pydata:
https://www.youtube.com/watch?v=wPEmoT018s8

# ? Jul 21, 2014 02:38

Modern Pragmatist: Aug 20, 2008

the posted:

SurgicalOntologist has already been grumbling at me to use it for my 5k data experiment earlier in this thread, I'll give it a look eventually

Instead of coming in here all the time asking about the best way to do some convoluted task, you should probably start taking the advice of the people in this thread and use some of the tools they recommend. Have you noticed that the first response to your posts is typically "what are you trying to do?"? Maybe devote a little time to something like pandas and then look back at what you were trying to do before and have a good laugh.

On a somewhat related note, are you related to forums user Pollyanna by chance?

# ? Jul 21, 2014 03:01

the: Jul 18, 2004; by Cowcaster

I've been working through Pandas tonight

And No, why do you ask?

edit: \/\/\/ I was assuming that PollyAnna was "another forums user who asks way too many annoying questions."

the fucked around with this message at 03:48 on Jul 21, 2014

# ? Jul 21, 2014 03:38

emoji: Jun 4, 2004

Modern Pragmatist posted:

Instead of coming in here all the time asking about the best way to do some convoluted task, you should probably start taking the advice of the people in this thread and use some of the tools they recommend. Have you noticed that the first response to your posts is typically "what are you trying to do?"? Maybe devote a little time to something like pandas and then look back at what you were trying to do before and have a good laugh.

On a somewhat related note, are you related to forums user Pollyanna by chance?

This is a pretty rude response for the Python information and short questions megathread, and the has clearly learned how to do a lot of things coming from a non-programmer background.

# ? Jul 21, 2014 03:42

Modern Pragmatist: Aug 20, 2008

kraftwerk singles posted:

This is a pretty rude response for the Python information and short questions megathread, and the has clearly learned how to do a lot of things coming from a non-programmer background.

Sorry for the tone. I guess I'm turning into a crotchety old man. I realize everyone has to start somewhere and the really has made some good progress, but I get a little frustrated when the first instinct of some posters is to post in the forum rather than thinking through a problem for a bit or googling around. I think this only got to me because I read the past few pages of posts in rapid succession.

the, keep doing what you're doing.

# ? Jul 21, 2014 12:52

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

It seems like Pandas just making numpy behave like data frames from R or tables from Matlab. Is that the basic idea or is there more to it?

# ? Jul 21, 2014 16:50

SurgicalOntologist: Jun 17, 2004

That's the basic idea, yes. It's a relatively simple library when you realize it's fundamentally just providing a single data structure (really, three: Series, DataFrame, Panel).

# ? Jul 21, 2014 16:54

SurgicalOntologist: Jun 17, 2004

Weird question: is there a Python library that acts as an alternative to the Make utility? I.e. dispatching function calls based on missing or outdated links in a dependency graph?

# ? Jul 21, 2014 17:24

good jovi: Dec 11, 2000; 'm pro-dickgirl, and I VOTE!

SurgicalOntologist posted:

Weird question: is there a Python library that acts as an alternative to the Make utility? I.e. dispatching function calls based on missing or outdated links in a dependency graph?

Do you just want to write your Makefile in Python, or do you need an actual library that you can invoke programatically?

# ? Jul 21, 2014 18:50

SurgicalOntologist: Jun 17, 2004

good jovi posted:

Do you just want to write your Makefile in Python, or do you need an actual library that you can invoke programatically?

The latter I guess. Although I'm not really sure what you mean by the former.

Really I'm just curious what's out there. I have some ideas for a domain-specific build system I'd like to put together, and I don't want to start from scratch. It could work with Python scripts running from the Makefile but that would be less than ideal.

# ? Jul 21, 2014 19:09

Houston Rockets: Apr 15, 2006

https://wiki.python.org/moin/ConfigurationAndBuildTools

# ? Jul 21, 2014 19:17

SurgicalOntologist: Jun 17, 2004

Fantastic, thanks.

E: Any of these particularly well-respected or does anyone have a particular recommendation? After a quick browse (Python 3 support, Pythonic API, active development) my list is down to paver, doit, fabricate, and pynt.

SurgicalOntologist fucked around with this message at 19:29 on Jul 21, 2014

# ? Jul 21, 2014 19:22

rock2much: Feb 6, 2004; Grimey Drawer

the posted:

Find every prime number between 0 and 100.

edit: in 20 lines or less

I did this! :woop:

Any recommendations for more projects?

# ? Jul 21, 2014 20:28

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

rock2much posted:

I did this!
Any recommendations for more projects?

Just work through the problems on Project Euler.

# ? Jul 21, 2014 21:40

the: Jul 18, 2004; by Cowcaster

Something weird's going on in Pandas. So, I loaded up that race data. It's in a CSV with three columns:

code:

         Time  Age  Gender
0   17.433333   30    Male
1   17.600000   17    Male
2   18.050000   17    Male
3   18.116667   38    Male
4   18.233333   44    Male
5   18.333333   44    Male
6   18.716667   16    Male
7   18.750000   58    Male
8   18.900000   40    Male
9   19.133333   45    Male

And so I grab the mean sorted by gender:

code:

In [34]: df.groupby('Gender').aggregate(np.mean)
Out[34]: 
             Time        Age
Gender                      
Female  34.433310  36.413921
Male    29.315735  39.204721

[2 rows x 2 columns]

Looks good, now let's group by age:

code:

In [36]: df.groupby('Age').aggregate(np.mean)
Out[36]: 
          Time
Age           
3    51.633333
5    54.390476
6    38.733333
7    40.033333
8    35.588333
9    36.012057
10   31.702344
           ...

[84 rows x 1 columns]

How come this gets put into one column? I'm trying to find the mean for each age and then plot mean age vs time, but it ends up throwing errors afterwards because there's only one column now in the data.

# ? Jul 22, 2014 01:08

the: Jul 18, 2004; by Cowcaster

rock2much posted:

I did this!
Any recommendations for more projects?

Write a code that checks to see if a supplied string is a palindrome or not.

# ? Jul 22, 2014 01:10

Jose Cuervo: Aug 25, 2004

the posted:

Something weird's going on in Pandas. So, I loaded up that race data. It's in a CSV with three columns:
code:
         Time  Age  Gender
0   17.433333   30    Male
1   17.600000   17    Male
2   18.050000   17    Male
3   18.116667   38    Male
4   18.233333   44    Male
5   18.333333   44    Male
6   18.716667   16    Male
7   18.750000   58    Male
8   18.900000   40    Male
9   19.133333   45    Male
And so I grab the mean sorted by gender:
code:
In [34]: df.groupby('Gender').aggregate(np.mean)
Out[34]: 
             Time        Age
Gender                      
Female  34.433310  36.413921
Male    29.315735  39.204721

[2 rows x 2 columns]
Looks good, now let's group by age:
code:
In [36]: df.groupby('Age').aggregate(np.mean)
Out[36]: 
          Time
Age           
3    51.633333
5    54.390476
6    38.733333
7    40.033333
8    35.588333
9    36.012057
10   31.702344
           ...

[84 rows x 1 columns]
How come this gets put into one column? I'm trying to find the mean for each age and then plot mean age vs time, but it ends up throwing errors afterwards because there's only one column now in the data.

What does it mean to plot MEAN age vs time? Your last result gives you the mean time at each distinct age in the data frame. Do you want to plot the mean time for each distinct age?

# ? Jul 22, 2014 01:23

the: Jul 18, 2004; by Cowcaster

Jose Cuervo posted:

What does it mean to plot MEAN age vs time?

Excuse me, I miswrote that. I want to plot the Age vs. Mean Time (for that age)

# ? Jul 22, 2014 01:25

EAT THE EGGS RICOLA: May 29, 2008

rock2much posted:

I did this!
Any recommendations for more projects?

Project Euler is alright, so is checkio.

# ? Jul 22, 2014 01:26

SurgicalOntologist: Jun 17, 2004

the posted:

How come this gets put into one column? I'm trying to find the mean for each age and then plot mean age vs time, but it ends up throwing errors afterwards because there's only one column now in the data.

I'm not sure what else you expected. Those numbers are the mean times for each age. Of the three original columns, time is still there, gender is gone because you can't take the mean of strings, and age is in the index because it's what you grouped by.

You should be able to plot a single column no problem, it will use the index for the x axis. What happens when you do

Python code:

df.groupby('Age').mean().plot()

Also just make a jointplot it will be interesting.

code:

pip install seaborn

then

Python code:

import seaborn as sns

sns.jointplot('Age', 'Time', df)

It could hardly be easier. Many of the other Seaborn plots will be interesting too, easier than manipulating the dataframe yourself, and look better.

Python code:

sns.lmplot('Age', 'Time', df, hue='Gender')

# ? Jul 22, 2014 01:28

the: Jul 18, 2004; by Cowcaster

And the mean plot follows the path I was more or less expecting originally. Peaks at the really young and really old:

the fucked around with this message at 03:53 on Jul 22, 2014

# ? Jul 22, 2014 03:50

SurgicalOntologist: Jun 17, 2004

Cool! Now try it again without the outlier, and use the "advanced" syntax to customize a bit:

Python code:

from scipy import stats

grid = sns.JointGrid('Age', 'Time', df.drop(df['Time'].argmax()))
grid.plot(sns.regplot, sns.distplot, stats.pearsonr)

# ? Jul 22, 2014 04:29

the: Jul 18, 2004; by Cowcaster

Interesting, I don't know what you did there, but interesting:

# ? Jul 22, 2014 04:34

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 05:59

SurgicalOntologist: Jun 17, 2004

1. You removed the outlier with the methods drop and argmax (since you only wanted to get rid of one outlier it was straightforward).

2. Instead of using the "shortcut" function jointplot, you used the JointGrid object, allowing you to specify which plotting functions to use on each axes.

3. On the bivariate axes, you used regplot isntead of the default scatter, which added the best-fit line.

4. On the two univariate axes, you used distplot instead of the default histogram, which added the kernel density estimate.

In the end, it looks like you don't have much of a correlation. You might try the gender plot I suggested earlier, simply because you have that variable you may as well examine.

Also, do you have data as to which race each data point is from? That could be interesting.

# ? Jul 22, 2014 04:43

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »