Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

TheFluff: Dec 13, 2006; FRIENDS, LISTEN TO ME
I AM A SEAGULL
OF WEALTH AND TASTE

The default should be to always return a list, but in my experience it's nice if there is also an alternate method that explicitly unpacks single row results, that you can use for queries that are guaranteed to always return exactly one row, like aggregations (count/sum/min/max and friends).

# ? Jun 21, 2018 23:42

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 09:31

Hunt11: Jul 24, 2013; Grimey Drawer

So in pandas I am trying to combine data. As in my current formula can generate this information that does not actually appear on the csv file but is made from combining various data points.
Average Price of Purchase Total Number of Purchases Total Revenue
0 2.931192 780 2286.33

What I am trying to do is break it down to see how various demographics that are represented by the dataset so I end up with something like

Average Price of Purchase Total Number of Purchases Total Revenue
M 2.931192 780 2286.33
F X Y Z

So the code I have so far is.

code:

gender_group=player_file.groupby(["Gender"])
average_price = player_file["Price"].mean()
total = player_file["Price"].sum()
Total_Purchases = player_file["SN"].count()
summary1 = pd.DataFrame({ "Average Price of Purchase": [average_price],
                           "Total Number of Purchases": [Total_Purchases],
                           "Total Revenue": [total]})
summary1

# ? Jun 21, 2018 23:55

vikingstrike: Sep 23, 2007; whats happening, captain

Something like this? I haven't tested it but it should work.

code:

summary_frame = (player_file
                 .groupby('Gender')
                 .agg({'Price': 'mean', 'SN': 'count'})
                 .rename({'Price': 'AveragePrice', 'SN': 'TotalPurchases'})
                 .assign(TotalRevenue=lambda _: _['AveragePrice'] * _['TotalPurchases'])

# ? Jun 22, 2018 00:16

Hunt11: Jul 24, 2013; Grimey Drawer

Doing that gets me an invalid syntax error.

# ? Jun 22, 2018 01:35

vikingstrike: Sep 23, 2007; whats happening, captain

You aren�t really giving us much to go on here. Either way, something along those lines should do what I think you�re describing. Like I said, there could be some errors in my code since I didn�t run it against anything.

# ? Jun 22, 2018 02:09

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

If I'm not working with scientific applications or teaching, is there another reason I might want to use Jupyter for stuff?

# ? Jun 25, 2018 04:47

pubic void nullo: May 17, 2002

Rocko Bonaparte posted:

If I'm not working with scientific applications or teaching, is there another reason I might want to use Jupyter for stuff?

I use it for taking notes on courses for the LaTeX embedding. It's perfect for things like Project Euler and the Python Challenge. I also use it for doing one-off analyses that are bigger than what you would comfortably do right in the REPL, whether that be experimenting with map APIs or crunching some imported Excel.

# ? Jun 25, 2018 05:53

Daviclond: May 20, 2006; Bad post sighted! Firing.

I'd like to plot the trajectory of a vehicle in three dimensions. The position and velocity vector of the vehicle will change over the duration of a simulation and I'll be updating the plot as I go.

I reckon I can probably knock something out in matplotlib easily enough, with each position marked with a dot (producing a trajectory over time) and the current position of the vehicle represented by an arrow in the direction of its velocity vector.

Are there any alternatives I should consider? I don't want to reinvent the wheel, and I'd give bonus points to something which could render the vehicle as some kind of 3D shape with an arbitrary orientation that I pipe from the simulation.

# ? Jun 25, 2018 13:03

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Is there a good list of released python versions somewhere that is easy to grep? I'm trying to make a docker image that's based on stretch-slim but can be used for python 3.4+ (as one of the dependencies of the project will produce a different output depending on version so I can't use python:3.4-jessie-slim and python:3.5-stretch-slim). Alpine also isn't an option as it requires me to build several of my dependencies and include a third-party glibc layer which somewhat ruins the point of using alpine for me.

Master_Odin fucked around with this message at 20:36 on Jun 25, 2018

# ? Jun 25, 2018 20:25

bamhand: Apr 15, 2010

Rocko Bonaparte posted:

If I'm not working with scientific applications or teaching, is there another reason I might want to use Jupyter for stuff?

I find the display() in jupyter looks nicer than print(). Also this is probably specific to me only but when I login to a SAS session using saspy using a notebook my password is masked by ****** but just shows up as plain text if I do it from spyder.

# ? Jun 25, 2018 20:34

necrotic: Aug 2, 2005; I owe my brother big time for this!

Master_Odin posted:

Is there a good list of released python versions somewhere that is easy to grep? I'm trying to make a docker image that's based on stretch-slim but can be used for python 3.4+ (as one of the dependencies of the project will produce a different output depending on version so I can't use python:3.4-jessie-slim and python:3.5-stretch-slim). Alpine also isn't an option as it requires me to build several of my dependencies and include a third-party glibc layer which somewhat ruins the point of using alpine for me.

Not a single file but you could clone pyenv and look at the list of version files:

https://github.com/pyenv/pyenv/tree/master/plugins/python-build/share/python-build

# ? Jun 25, 2018 22:37

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

bamhand posted:

I find the display() in jupyter looks nicer than print(). Also this is probably specific to me only but when I login to a SAS session using saspy using a notebook my password is masked by ****** but just shows up as plain text if I do it from spyder.

Don't forget about pprint in the standard library!

# ? Jun 25, 2018 22:54

The Fool: Oct 16, 2003

Can someone explain the __future__ library to my like I'm an idiot.

I've done some googling, but I'm still not sure I get the point.

# ? Jun 25, 2018 22:56

necrotic: Aug 2, 2005; I owe my brother big time for this!

It has backports of new python features.

# ? Jun 25, 2018 23:41

The Fool: Oct 16, 2003

Like if I wanted a 3 feature in 2.7?

# ? Jun 25, 2018 23:48

necrotic: Aug 2, 2005; I owe my brother big time for this!

Sort of. Its so new behaviors can be introduced without breaking older releases.

https://docs.python.org/3/reference/simple_stmts.html#future-statements

https://docs.python.org/3/library/__future__.html

# ? Jun 26, 2018 00:02

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Yes it gives you a good basis for migrating your code to Python 3.

The first one of all a lot of people run into is using print as a function with parentheses versus as a statement.

I should note it has nothing to do with concurrent.futures

# ? Jun 26, 2018 00:29

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

I think the other big thing is you can import python3�s division operator. I think in 2.7, 1/2 will return 0 but python3 will always cast to float first unless you use the // operator.

# ? Jun 26, 2018 16:51

pmchem: Jan 22, 2010

I feel like this should be easily google-able but I'm missing the answer.

Anaconda Distribution 5.2.0 was released a little under a month ago. I am curious about the current development version of Anaconda Distribution, whatever will be released as either 5.2.1 or 5.3.0. I can't find a working link to a dev snapshot or repo anywhere. It's entirely opaque to me as to what the next version will be, or what's currently in it.

I can go here: https://anaconda.org/anaconda/anaconda and it links to https://github.com/ContinuumIO/anaconda but that 404's.

Has Anaconda totally hidden information about upcoming releases? Do I just need to conda update --all latest from 5.2.0 and see what happens? (lol)

# ? Jun 28, 2018 00:46

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Python 3.7 out.

Some of the new features :

http://w4t.pw/z7

Thermopyle fucked around with this message at 02:22 on Jun 28, 2018

# ? Jun 28, 2018 02:19

Jeesis: Mar 4, 2010; I am the second illegitimate son of gawd who resides in hoaven.

Hello nerds,

I have been doing some pythonning as of late and wanted to get a book to learn me how to do more than basic poo poo.

Should I get the giant gently caress you OReilly book or is there a better book? Kind of looking for something comprehensive but I wonder if it might be too outdated. Also thinking of learning python 3 but entirely have been learning 2.7, I assume it will not be too much of a culture shock.

Only just started doing some programming in the last few months and shat out a FTP script with a while loop interface but my brain is already exploding with ideas, but kind of getting sick of scrounging basic concepts from random sites.

# ? Jun 28, 2018 02:48

Dominoes: Sep 20, 2007

Thermopyle posted:

Python 3.7 out.

Some of the new features :

http://w4t.pw/z7

Dataclasses look fantastic And long overdue!

No boilerplate in class creation. Unlike typing.NamedTuple, is mutable, and you can add methods. Similar to Structs in Rust (and C?), but with builtin comparison behavior.

Dominoes fucked around with this message at 05:00 on Jun 28, 2018

# ? Jun 28, 2018 04:19

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

code:

import contextvars

name = contextvars.ContextVar("name")
contexts = list()

def greet():
    print(f"Hello {name.get()}")

# Construct contexts and set the context variable name
for first_name in ["Steve", "Dina", "Harry"]:
    ctx = contextvars.copy_context()
    ctx.run(name.set, first_name)
    contexts.append(ctx)

# Run greet function inside each context
for ctx in reversed(contexts):
    ctx.run(greet)

I'm not sure I understand this. This is the same thing as

code:

def greet(name):
    print(f"Hello {name}")

# Construct contexts and set the context variable name
contexts = []
for first_name in ["Steve", "Dina", "Harry"]:
    contexts.append(first_name)

# Run greet function inside each context
for ctx in reversed(contexts):
    greet(ctx)

isn't it? Why would i want to do whatever it is they showed first?

# ? Jun 28, 2018 07:58

Hunt11: Jul 24, 2013; Grimey Drawer

So I have a pandas question.

code:

ride_table = pd.merge(ride_file, city_file, on="city")
ride_table.head()
City_comp = ride_table.groupby(["city", "type"])
City_comp[["fare","ride_id"]].mean()
fare_mean = City_comp[["fare"]].mean()
ride_id = City_comp[["ride_id"]].count()
fare_mean["ride_id"] = ride_id
fare_mean.head()

fare	ride_id
city	type		
Amandaburgh	Urban	24.641667	18
Barajasview	Urban	25.332273	22
Barronchester	Suburban	36.422500	16
Bethanyland	Suburban	32.956111	18
Bradshawfurt	Rural	40.064000	10

This is the code I have gotten so far. I needed to combine two data sets and then filter it so that only the relevant information would be processed in a single dataframe. My problem is that I am trying to split up the single dataframe into three pieces so that I have individual dataframes for urban, suburban and rural.

# ? Jun 28, 2018 16:28

vikingstrike: Sep 23, 2007; whats happening, captain

Similar to last time, you aren't really describing what the desired output is. That said, if you are trying to index based on the values of rows, there are a couple of ways you can do this:

code:

frame = This is your DataFrame
urban_data = frame.loc[frame.type == 'Urban']

or you could do:

code:

frame = This is your DataFrame
urban_data = frame.query("type == 'Urban'")

# ? Jun 28, 2018 16:57

Dr Subterfuge: Aug 31, 2005; TIME TO ROC N' ROLL

Generally speaking it would seem like using query instead of loc doesn't give you a whole lot since you lose pretty much all the niceties of your editor by moving everything into a string that needs to be parsed eventually anyway. Am I missing something? F-strings maybe? Which might not be a good idea of you care about security?

# ? Jun 28, 2018 17:15

Hunt11: Jul 24, 2013; Grimey Drawer

vikingstrike posted:

code:

frame = This is your DataFrame
urban_data = frame.query("type == 'Urban'")

I really do need to start making a list of things to try if my original attempts aren't working. I was trying to muck around with the earlier part of the code but the second suggestion worked like a charm so thank you.

# ? Jun 28, 2018 17:55

vikingstrike: Sep 23, 2007; whats happening, captain

Sure. Glad it�s working!

Query can be easier to use when method chaining and I believe on larger frames it gets processed using pandas� eval() which speeds things up. I haven�t timed it recently though so I could be misremembering. I think this is actually discussed in the docs. I normally just use it when chaining methods. I would only worry about the input attack vector if this is public facing code. For personal use I don�t believe it matters. I�m also unsure what editor features you�re referring too but that could be an issue if you rely on something particular.

# ? Jun 28, 2018 18:02

Plasmafountain: Jun 17, 2008

BiVi9JFEtoEgUBVGOkeG
h3iSm4yMHvB2FLpp49ln
Umdj9fCRO6fmToeldEIk
v71euM4QS0UZHGvMht2Z
jLbAayJ4zDZQtIAYB3Pw
xrHrA3scU2gV399Afw7A
ihIwhTFa4t8VJlm0OG9d
2XsELOTUKbgtfPxEyzzV
SPhrmCcWB0e2hsDScEr4
CSOUW1BvEV2Q5Hx8Cpc5

Plasmafountain fucked around with this message at 23:58 on Feb 27, 2023

# ? Jun 28, 2018 18:04

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Zero Gravitas posted:

Probably a bit of a long shot but is there a ready made python library for interacting with Youtube as though the program was your average user?

To make sure this isn't an XY problem...what are you actually trying to do?

# ? Jun 28, 2018 18:14

Tigren: Oct 3, 2003

Thermopyle posted:

Python 3.7 out.

Some of the new features :

http://w4t.pw/z7

Love this part:

quote:

The Order of Dictionaries Is Guaranteed

In Python 3.6, this ordering was just a nice consequence of that implementation of dict. In Python 3.7, however, dictionaries preserving their insert order is part of the language specification. As such, it may now be relied on in projects that support only Python >= 3.7 (or CPython >= 3.6).

# ? Jun 28, 2018 18:17

Plasmafountain: Jun 17, 2008

e4auyNz03hvMRLmrcfjv
yVaXKhtm06DDRzqxoumh
uCMr5Sh61KH0rBKT2U45
SWmjHkWnVefyCEpyfrne
qZBi3CtuwTbFjxTIE6yu
VRb4dyTDHM83PuuHro8J
PscPT1nSnCEgJc7RFbQX
kX4fmPH9Lzan6bZeRsO5
8NYUDAxnwLJA16N1CeqD
pzPljIBdfUIqyg0lxQBM

Plasmafountain fucked around with this message at 23:58 on Feb 27, 2023

# ? Jun 28, 2018 18:21

Dr Subterfuge: Aug 31, 2005; TIME TO ROC N' ROLL

vikingstrike posted:

Sure. Glad it�s working!

Query can be easier to use when method chaining and I believe on larger frames it gets processed using pandas� eval() which speeds things up. I haven�t timed it recently though so I could be misremembering. I think this is actually discussed in the docs. I normally just use it when chaining methods. I would only worry about the input attack vector if this is public facing code. For personal use I don�t believe it matters. I�m also unsure what editor features you�re referring too but that could be an issue if you rely on something particular.

I'm thinking mostly about basic autocomplete and syntax checking. I make stupid typos way too often and sometimes get screwed over when I'm using strings because PyCharm doesn't know any better.

# ? Jun 28, 2018 18:22

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Zero Gravitas posted:

I'm trying to get an example of exactly how long it takes for the youtube algorithm to start pitching far right wing videos at a new user who views soft right or entryist political material.

You're probably just going to need to use pyppeteer or selenium.

Cool idea to test out though.

# ? Jun 28, 2018 18:42

Dr Subterfuge: Aug 31, 2005; TIME TO ROC N' ROLL

Zero Gravitas posted:

I'm trying to get an example of exactly how long it takes for the youtube algorithm to start pitching far right wing videos at a new user who views soft right or entryist political material.

This is an interesting problem. Something to keep in mind is YouTube keeps track of how much of a video their users watch, so an agent that jumps quickly between videos isn't going to register the same way that an attentive listener (someone who watches the whole thing) would. Same thing with liking videos, of course. Probably comments as well. Probably neither of those are things that you want to be doing with the videos of the ever more deranged parts of the political spectrum, though.

# ? Jun 28, 2018 20:09

jerry seinfel: Jun 25, 2007

Jeesis posted:

Hello nerds,

I have been doing some pythonning as of late and wanted to get a book to learn me how to do more than basic poo poo.

Should I get the giant gently caress you OReilly book or is there a better book? Kind of looking for something comprehensive but I wonder if it might be too outdated. Also thinking of learning python 3 but entirely have been learning 2.7, I assume it will not be too much of a culture shock.

Only just started doing some programming in the last few months and shat out a FTP script with a while loop interface but my brain is already exploding with ideas, but kind of getting sick of scrounging basic concepts from random sites.

I'd recommend going straight to python 3 and you should be fine. The biggest problem you're going to run into is with print statements. Switching over sooner rather than later is a good idea if only to start using some of the newer features like f-strings and pathlib.

https://realpython.com has a lot of really useful dives into standard library modules. Right now I'm doing their Flask tutorial for a refresher and it covers a lot of material pretty quickly.

As far as books, Python Standard Library by Example is exhaustive, but available more or less online under https://pymotw.com/3/. If you're looking to get some exposure on how to just get things done quickly, I started out with Automate the Boring Stuff which was really helpful just to get me to start working on stuff.

# ? Jun 28, 2018 20:22

wolrah: May 8, 2006; what?

Zero Gravitas posted:

I'm trying to get an example of exactly how long it takes for the youtube algorithm to start pitching far right wing videos at a new user who views soft right or entryist political material.

I agree with Thermopyle that you'll want something that looks and behaves pretty close to a browser being operated by a human for this experiment. To be honest I'd say the best answer might be to actually automate a standard non-headless browser instance. Keep in mind that Youtube is an ad-dependent business and thus puts a fair bit of effort in to trying to discriminate actual human views from any sort of "bot", so the closer your bot looks to a real user the better.

# ? Jun 28, 2018 22:01

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

As time goes on, I'm getting less and less happy with the pyplot API in matplotlib. I'd prefer to use the object-oriented API in new code, but I always find it hard to prioritize learning a new API over creating the plot that I need at any given moment.

I remember seeing a document (maybe part of the matplotlib documentation) that was something like a side-by-side description of how to perform common operations with the two APIs. For example, "this is how you instantiate a figure, and the axes, and select the artist" for the OO method, vs. "call pyplot.figure" for the stateful Matlab-like API.

I'm having a hard time finding this, though; does anyone know of any documentation like this?

# ? Jun 29, 2018 15:54

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I just wanted to say that the Markdown Navigator plugin for PyCharm (actually I guess any JetBrains product, but I'm doing lots of python lately so...) is the best and if you write any markdown it's a must-have!

I paid for the "enhanced edition" and it's pretty great.

# ? Jul 1, 2018 18:31

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 09:31

Hollow Talk: Feb 2, 2014

Thermopyle posted:

I just wanted to say that the Markdown Navigator plugin for PyCharm (actually I guess any JetBrains product, but I'm doing lots of python lately so...) is the best and if you write any markdown it's a must-have!

I paid for the "enhanced edition" and it's pretty great.

It pains me to say this as a long-term Emacs die-hard, but PyCharm in general is really rather amazing. It also has a really useful way of displaying pandas DataFrames when debugging. :pseudo:

# ? Jul 1, 2018 18:59

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »