Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

a witch: Jan 12, 2017

When I got a new laptop this is the only sticker I put on it https://www.stickermule.com/marketplace/8980-talk-about-something-other-than-programming

# ? Sep 1, 2017 19:21

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 07:36

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

There is something that bothers me about putting/seeing stickers on laptops, but I have no idea what it is.

# ? Sep 1, 2017 19:51

a witch: Jan 12, 2017

I started doing it because every conference and event I went to handed out tons of stickers. They can be decent conversation starters as well.

# ? Sep 1, 2017 19:55

Nippashish: Nov 2, 2005; Let me see you dance!

I put stickers on my laptop because I work in an office with 500 people and 99% of them use macbook pros. Gotta have some way to tell which one is mine.

# ? Sep 1, 2017 22:28

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

I lied actually, I have a label with my name and phone number on the bottom of it

# ? Sep 2, 2017 01:03

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

I use the ones I get from breweries

# ? Sep 2, 2017 01:46

Smugworth: Apr 18, 2003

I have a sticker that says Austin Texas on mine because I moved to goddamned Minnesota. please talk to me ex-Texans please :mrgw:

# ? Sep 2, 2017 03:37

Dominoes: Sep 20, 2007

Like, Intel Inside and Gateway2000 stickers?

# ? Sep 2, 2017 06:49

Linear Zoetrope: Nov 28, 2011; A hero must cook

Do you need to touch the reference counters or GIL if you want to spawn a bunch of threads from the C side, call Python functions that return PyObjects, and forward the PyObjects between the threads? To be clear, only one thread will touch a given PyObject * at a time.

# ? Sep 3, 2017 19:58

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Linear Zoetrope posted:

Do you need to touch the reference counters or GIL if you want to spawn a bunch of threads from the C side, call Python functions that return PyObjects, and forward the PyObjects between the threads? To be clear, only one thread will touch a given PyObject * at a time.

It's been awhile since I did this, but I'm fairly sure the answer is "no".

# ? Sep 3, 2017 20:39

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

Having spent the last six months working with data analysis in pandas and learning python as a minor part of my job, I have just moved to a new company where I will be dedicating most of my time to building subscriber forecasting models in Python. Although I can get around the pandas/dataviz side of things quite well, structuring a model like this from the ground up is something new to me. I could really use some direction.

I will be mostly converting a large excel workbook (think hundreds of MB) into Python and adding extra functionality. The model is based around forecasting subscribers on a monthly basis, to predict customer acquisition & cancellation on different products plus calculating net present value considering income, opex etc. I could build it as a huge chain of dataframes with a bunch of weird iteritems() and iterrows(); essentially replicating the lookups that are already there in Excel. But this seems very much like the excel way to do things, and we want to extend functionality with Python, not simply replace what is already there.

Are there any examples of this kind of model? There seem to be lots for pure financial forecasting (interest, stock prices etc) and for data analysis but this seems to fit between the two.

# ? Sep 4, 2017 19:14

SurgicalOntologist: Jun 17, 2004

If I were you, I would get it working however you can, then work on improving/extending it. You shouldn't have to use iterrows or iteritems very often if at all. If you are replacing lookups, you can think of indexing into a pandas object (e.g. prices[items]) as a vectorized lookup--so you don't need to do them one at a time.

If there's a specific thing you can't figure out how to do without iterating, post it (or a simplified example or whatever), those are fun challenges and I think transforming code from one style to another is a good way to learn.

# ? Sep 4, 2017 19:51

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

That's my general approach; I'm sure I will find some dead ends along the way but that's how I will learn the most about it.

For example of what I mean by excel versus python issue, there are calculations of customer losses in October that depend on the total customer numbers in September. Only then can you get total customers in October, which can be used to find losses in Nov etc. This kind of iterative approach makes sense to me in Excel, but in pandas I am used to calculating each series at a time, rather than two interdependently.

# ? Sep 4, 2017 20:01

Cingulate: Oct 23, 2012; by Fluffdaddy

Slimchandi posted:

That's my general approach; I'm sure I will find some dead ends along the way but that's how I will learn the most about it.

For example of what I mean by excel versus python issue, there are calculations of customer losses in October that depend on the total customer numbers in September. Only then can you get total customers in October, which can be used to find losses in Nov etc. This kind of iterative approach makes sense to me in Excel, but in pandas I am used to calculating each series at a time, rather than two interdependently.

Essentially your problem will be to concisely write up your problems and post them at the correct time ITT, so SurgicalOntologist can provide the required Pandas one-liner to solve all your problems :v:

# ? Sep 4, 2017 20:45

SurgicalOntologist: Jun 17, 2004

Slimchandi posted:

For example of what I mean by excel versus python issue, there are calculations of customer losses in October that depend on the total customer numbers in September. Only then can you get total customers in October, which can be used to find losses in Nov etc. This kind of iterative approach makes sense to me in Excel, but in pandas I am used to calculating each series at a time, rather than two interdependently.

For this kind of thing the trick is to "factor out" the inter-dependent aspects of the calculation. For example if you can first predict "percent customers lost", e.g. based on seasonality factors, then from there it's straightforward to build a series of actual customers lost or total customers or whatever you'd like. It will probably also be useful to index your data not by the actual month but by "months into the future", e.g. [0, 1, 2...]. You can always change the labels when you are visualizing/displaying the data.

# ? Sep 4, 2017 21:13

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

StackOverflow thinks python is the fastest-growing language. Started to outpace all other languages in 2012 in high-income countries and 2014 in lower-income countries.

# ? Sep 6, 2017 20:15

QuarkJets: Sep 8, 2008

now that python has overtaken javascript hopefully we can enjoy the rich bounty of coding horrors associated with novices using that language

# ? Sep 6, 2017 22:52

NtotheTC: Dec 31, 2007

So my new project is sufficiently out of the quagmires of legacy code that I get to start using the latest versions of everything- including python3.6 over python2.7. It's perhaps a bit shameful that I've done virtually no commercial coding with python3, but I'm wondering if there's a list of the biggest changes I'll need to wrap my head around? f_string_literals and type hinting are two that I've been made aware of but I'm sure there's more out there.

# ? Sep 7, 2017 12:01

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

NtotheTC posted:

So my new project is sufficiently out of the quagmires of legacy code that I get to start using the latest versions of everything- including python3.6 over python2.7. It's perhaps a bit shameful that I've done virtually no commercial coding with python3, but I'm wondering if there's a list of the biggest changes I'll need to wrap my head around? f_string_literals and type hinting are two that I've been made aware of but I'm sure there's more out there.

The biggest (to me) is the asyncio stuff. But here's a nice list of neat things in Python3 as compared to Python2: https://powerfulpython.com/blog/whats-really-new-in-python-3

Master_Odin fucked around with this message at 13:12 on Sep 7, 2017

# ? Sep 7, 2017 12:53

NtotheTC: Dec 31, 2007

That's perfect, thanks!

# ? Sep 7, 2017 13:13

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

NtotheTC posted:

So my new project is sufficiently out of the quagmires of legacy code that I get to start using the latest versions of everything- including python3.6 over python2.7. It's perhaps a bit shameful that I've done virtually no commercial coding with python3, but I'm wondering if there's a list of the biggest changes I'll need to wrap my head around? f_string_literals and type hinting are two that I've been made aware of but I'm sure there's more out there.

I just came across this article today:

http://www.b-list.org/weblog/2017/sep/05/how-python-does-unicode/

# ? Sep 7, 2017 18:10

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

I'm trying to add a subpackage to my package so that it isn't imported normally, but you can import it by doing import myproject.subpackage. I know SciPy does this for some of their packages (scipy.optimize comes to mind) so it should be possible but when I do this I get

code:

ImportError: No module named 'myproject.subpackage'

So I'm not really sure what's going on. The subpackage directory has a __init__.py that imports the relevant things in the subpackage, and an __all__ defined. The main package also has an __init__.py as it's needed, and imports all the classes/functions from the various modules that make up the main package.

In my setup.py I do have the directory added to the packages argument, like

code:

setup(..., packages=['myproject', 'myproject.subpackage'], ...)

# ? Sep 7, 2017 18:51

Seventh Arrow: Jan 26, 2005

So my attempts to find a tutor were not very fruitful. So maybe I could get a bit of direction instead.

I'm in a data science / data engineering course, but the level of python expertise required is higher than I initially thought.

I've read "Python Crash Course" and am working my way through "Automate the Boring Stuff with Python" so I know what lists and dictionaries are, etc., but I draw a blank when trying to come up with my own code and analyzing scripts is tricky - like here's an example of a script that I need to modify for a project. I eventually figured it out, but it was pretty challenging.

So anyways, books kinda take longer than I'd like, is there maybe an online course that's fairly decent? It doesn't have to be free. I think I just need to start doing more exercises, but I also still need training wheels somewhat.

Mod Edit: :iiam:

Somebody fucked around with this message at 22:21 on Sep 7, 2017

# ? Sep 7, 2017 20:29

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

* Seventh, sent pm *

dougdrums fucked around with this message at 20:49 on Sep 7, 2017

# ? Sep 7, 2017 20:45

onionradish: Jul 6, 2006; That's spicy.

Seventh Arrow posted:

So my attempts to find a tutor were not very fruitful. So maybe I could get a bit of direction instead.

I'm in a data science / data engineering course, but the level of python expertise required is higher than I initially thought.

I've read "Python Crash Course" and am working my way through "Automate the Boring Stuff with Python" so I know what lists and dictionaries are, etc., but I draw a blank when trying to come up with my own code and analyzing scripts is tricky - like here's an example of a script that I need to modify for a project. I eventually figured it out, but it was pretty challenging.

So anyways, books kinda take longer than I'd like, is there maybe an online course that's fairly decent? It doesn't have to be free. I think I just need to start doing more exercises, but I also still need training wheels somewhat.

I recently ran across PySlackers, a group of ~8500 Python users of various skils that seem to actively seek to help new Python users through Slack.

I'm not a member, so can't speak to how interactions go, but might be a consideration since you're likely to find others with a data science background who can answer questions, critique code, or point you toward relevant resources they found useful.

There are some learning resources on their Github page including some video tutorials.

Mod Edit: :iiam:

Somebody fucked around with this message at 22:21 on Sep 7, 2017

# ? Sep 7, 2017 20:53

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

I enjoyed http://www.diveintopython3.net/ as it jumps right in and then breaks things apart into smaller pieces. You get an explanation of almost everything but in the context of more real-world applications.

# ? Sep 7, 2017 22:11

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

Seventh Arrow posted:

So my attempts to find a tutor were not very fruitful. So maybe I could get a bit of direction instead.

I'm in a data science / data engineering course, but the level of python expertise required is higher than I initially thought.

I've read "Python Crash Course" and am working my way through "Automate the Boring Stuff with Python" so I know what lists and dictionaries are, etc., but I draw a blank when trying to come up with my own code and analyzing scripts is tricky - like here's an example of a script that I need to modify for a project. I eventually figured it out, but it was pretty challenging.

So anyways, books kinda take longer than I'd like, is there maybe an online course that's fairly decent? It doesn't have to be free. I think I just need to start doing more exercises, but I also still need training wheels somewhat.

Mod Edit:

It kind of sounds like you're confusing "I need to learn programming" with "I need to learn Python". Have you learned to program in any other languages previously? If not, there aren't really any shortcuts. You need to learn basic programming (you can do it in Python). Ideas like scope, abstraction, control flow, algorithmic complexity don't just show up in a python tutorial. You'll need to study a bit and practice a lot.

# ? Sep 8, 2017 02:03

Seventh Arrow: Jan 26, 2005

KernelSlanders posted:

It kind of sounds like you're confusing "I need to learn programming" with "I need to learn Python". Have you learned to program in any other languages previously? If not, there aren't really any shortcuts. You need to learn basic programming (you can do it in Python). Ideas like scope, abstraction, control flow, algorithmic complexity don't just show up in a python tutorial. You'll need to study a bit and practice a lot.

Yes, this is my first programming language (except for a bit of BASIC in the 80's, haha). I kind of suspected that the things you mentioned were lacking in my approach. I agree that a lot of practice is probably needed, so any suggestions are welcome. Thanks everyone for the recommendations so far!

# ? Sep 8, 2017 02:47

vikingstrike: Sep 23, 2007; whats happening, captain

I forget from your earlier posts, but have you posted any examples of code where you're getting stuck? This thread is quite willing to help people along if they know where to meet you.

# ? Sep 8, 2017 02:59

Seventh Arrow: Jan 26, 2005

There's no specific examples at the moment. I'm just in that awkward phase of having read about dictionaries, strings, lists, etc and now having to go out and do stuff with it. And what I need to do is specific to data science / data engineering. Since my initial post I've seen sites that have python exercises, so that might be the right direction to go in.

There actually is a project that I'm working on currently, but the main issues that I'm facing have to do mostly with the API that the script is interacting with and not python per se.

# ? Sep 8, 2017 05:13

Foxfire_: Nov 8, 2010

Linear Zoetrope posted:

Do you need to touch the reference counters or GIL if you want to spawn a bunch of threads from the C side, call Python functions that return PyObjects, and forward the PyObjects between the threads? To be clear, only one thread will touch a given PyObject * at a time.

You can pass the pointers themselves amongst your threads however you want. But at most one thread may be invoking Python code at any time and theoretically it should own the GIL. You can't be accessing multiple PyObject's simultaneously, even if any particular object is only accessed by one thread.

I think you avoid GIL stuff if you can guarantee:
- your threads will synchronize themselves to avoid simultaneously trying to run Python code
- no threads are ever created from Python

The interpreter would be running in its single-threaded mode where the GIL doesn't actually exist and there's only one possible value of it's global ThreadState structure in play. CPython doesn't use any OS thread-local variables, so it shouldn't care what OS thread is actually running its code, as long as there's only one at a time.

# ? Sep 8, 2017 05:28

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

Feel there's an obvious Python idiom that I'm missing here.

I'm adding a bunch of views to menus in a Flask application and rather than write this:

code:

appbuilder.add_view (AboutView, "About this site", category='Foo')
appbuilder.add_view (NewsModelView, "News", category='Foo')
...

appbuilder.add_view (AssociationModelView, "Associations", category='Bar')
appbuilder.add_view (TagModelView, "Tags", category='Bar')
...

i.e. I figure that there sound be a way to more neatly group items by menu, so I could move things between menus easily The best I came up with is:

code:

for cat in ['Foo']:
   appbuilder.add_view (AboutView, "About this site", category=cat)
   appbuilder.add_view (NewsModelView, "News", category=cat)
   ...

for cat in ['Bar']:
   appbuilder.add_view (AssociationModelView, "Associations", category=cat)
   appbuilder.add_view (TagModelView, "Tags", category=cat)
   ...

Ideas? It nearly seems like something that with could handle.

# ? Sep 8, 2017 13:04

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

If anything, it looks like you could create a dictionary where the keys are the categories and the values are dictionaries where the keys are the names of the views and values are view object. Then you'd just need a nested for loop to iterate through all this to make the repeated function call.

Moving stuff around the menus would be moving key/values around the inner dicts.

# ? Sep 8, 2017 16:09

Loezi: Dec 18, 2012; Never buy the cheap stuff

I would probably go with something like this, dunno how pythonic it is:

Python code:

menu = {
	'Foo': [
		('About this site', AboutView),
		('News', NewsModelView),
	],
	'Bar': [
		('Associations', AssociationModelView),
		('Tags', TagModelView)
	]
}
 
for category in menu:
	for label, view in menu[category]:
		appbuilder.add_view(view, label, category=category)

E: On a second thought, I don't really know why I did the inner structure as a list of tuples; it would work just as well (if not better) as another dict.

Loezi fucked around with this message at 19:30 on Sep 8, 2017

# ? Sep 8, 2017 19:16

Dominoes: Sep 20, 2007

Is there a package out there that simply requires the scipy stack? (numpy, scipy, pandas, jupyter, sympy, matplotlib) Like django-toolbelt. I didn't find anything.

edit: There is now.

Dominoes fucked around with this message at 21:22 on Sep 12, 2017

# ? Sep 11, 2017 01:37

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

SurgicalOntologist posted:

If there's a specific thing you can't figure out how to do without iterating, post it.

You asked for it...

Is there any way to create a pandas column multiindex where some of the columns headings 'span' down? In effect, some columns might have 2 levels of column but others have 3.

For example

Region: North, South
Widget Type: type 1, type 2
Rate: Day, Night (only applies to type 2 widgets)

So I could index a type 1 widget by ['North','type 1'] but type 2 would require ['North','type 2','Day'].

# ? Sep 11, 2017 17:30

pubic void nullo: May 17, 2002

Slimchandi posted:

You asked for it...

Is there any way to create a pandas column multiindex where some of the columns headings 'span' down? In effect, some columns might have 2 levels of column but others have 3.

For example

Region: North, South
Widget Type: type 1, type 2
Rate: Day, Night (only applies to type 2 widgets)

So I could index a type 1 widget by ['North','type 1'] but type 2 would require ['North','type 2','Day'].

Believe it or not, I actually have the same question right now.

I'm trying to do basic data analysis on a series of electricity bills. Most of the spreadsheet is indexed by the same row, but there are a couple sections that have a grouping header above that row, like this:

I have to use pd.read_excel to bring the sheets in, and I'm trying not to use a panel because apparently that's being deprecated as of pandas 0.20.

# ? Sep 13, 2017 21:41

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

pubic void nullo posted:

Believe it or not, I actually have the same question right now.

I'm trying to do basic data analysis on a series of electricity bills. Most of the spreadsheet is indexed by the same row, but there are a couple sections that have a grouping header above that row, like this:

I have to use pd.read_excel to bring the sheets in, and I'm trying not to use a panel because apparently that's being deprecated as of pandas 0.20.

I didn't find a solution and I'm lead to believe that pandas can't do a span like you and I would like.

Perhaps in your situation you could have a level 0 index above those last three columns called 'summary statistics'?

# ? Sep 13, 2017 22:09

vikingstrike: Sep 23, 2007; whats happening, captain

It can sure. You'd just have an empty value for that level, but to me that would be pretty weird to work with. You could also define the level you're struggling to map with an appropriate value. For example in your case it would be setting the value "Day" even though there is no variability for type 1. Other solutions would involve collapsing across levels through different naming or not using the extra level at all.

# ? Sep 13, 2017 23:50

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 07:36

creatine: Jan 27, 2012

Question: I have a CSV that has several columns of info where each row is a specific sample from a study. The last column has a coordinate to match with a position in a 9x9 box. With the X axis being A through I and the Y axis being 1-9 so it would look like:

��A B C ...
1
2
3
...

I would like to load this csv that is generated from audits and put the corresponding sample information each in a square based on its coordinates. I figured pandas would be best to work with the csv but I'm drawing a blank on the best way to format this to print. Anybody have thoughts/suggestions on directions I can investigate?

Edit: Managed to hack something together where I first made a template excel spreadsheet and then populated the cells correlating with the pandas dataframe

creatine fucked around with this message at 01:41 on Sep 14, 2017

# ? Sep 14, 2017 00:07

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »