Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Nippashish
Nov 2, 2005

Let me see you dance!

Thermopyle posted:

In other news I've been spending some time recently converting a bunch of bash scripts to python because gently caress bash.

I came across the plumbum library that's pretty neat for easing this type of work.

This is neat. I've used sh for this in the past, but plumbum looks a bit more featureful. Does it handle piping properly (i.e. hooking stdin/stdout of the processes together at the OS level) or does it shuffle everything through python? I've used a few shell replacement libraries that do the later and it can be a real show stopper if you need to move a large amount of data around.

A bit of fuckery is worth it though, just to avoid bash's insane string quoting rules.

Nippashish fucked around with this message at 12:48 on Feb 5, 2017

Adbot
ADBOT LOVES YOU

huhu
Feb 24, 2006
Any suggestions for naming variables in Python for loops? I tend to just go with each or index but it gets quite confusing if I'm nesting loops. I also sometimes use, if my list is named something like "pairs" I'll do for each pair in pairs but that also seems a bit chaotic.

Cingulate
Oct 23, 2012

by Fluffdaddy

huhu posted:

Any suggestions for naming variables in Python for loops? I tend to just go with each or index but it gets quite confusing if I'm nesting loops. I also sometimes use, if my list is named something like "pairs" I'll do for each pair in pairs but that also seems a bit chaotic.
I think the pythonic was is to assign variables based on what they refer to. If you're iterating over a list of names, call it
code:
for name in names:

Eela6
May 25, 2007
Shredded Hen
Python code:
for first, middle, last in names:


for prime in primes:

for real,  imaginary in compex_numbers:
# r,  i would work fine as well
for enemy in enemies:

for k, block in enumerate(blocks)

Eela6 fucked around with this message at 00:21 on Feb 6, 2017

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

QuarkJets posted:

:ughh:

Pylint recognizes other conventions as being okay, why won't you?

I'm not sure you understand what you highlighted in my quote means?

I mean...using your own style in a code base is bad and worse than using a less popular convention for an unused variable. I'm not sure what's controversial about that.


HardDiskD posted:

I mean, gently caress bash and that's a really sexy API, but I feel like this is just hiding the bash under a Python rug and calling it a day.

That's kind of what we do as programmers, no?


Nippashish posted:

This is neat. I've used sh for this in the past, but plumbum looks a bit more featureful. Does it handle piping properly (i.e. hooking stdin/stdout of the processes together at the OS level) or does it shuffle everything through python? I've used a few shell replacement libraries that do the later and it can be a real show stopper if you need to move a large amount of data around.

A bit of fuckery is worth it though, just to avoid bash's insane string quoting rules.

A quick perusal of the source makes it look like it uses OS piping via popen, but I'm not 100% confident in that. https://github.com/tomerfiliba/plumbum/blob/1441c5ae7dedc2f40aaf37207ff2585557c5746b/plumbum/commands/base.py#L266


Tigren posted:

That does seem fun, but why not learn the Standard Library os module and rely on subprocess when needed? They're already included for you.

I kind of feel like the answer to this is similar to the answer I gave HardDiskD. The only reason to use any library is if it makes your job easier...after all you can implement anything you want yourself. You could use C instead of Python. You could use assembly instead of C. It just might be easier to implement the functionality you need by using subprocess, or it might not. I've used subprocess extensively, but it's certainly not as concise as this particular library. This library seems to make it easier to express what you intend.

Eela6
May 25, 2007
Shredded Hen

HardDiskD posted:

I mean, gently caress bash and that's a really sexy API, but I feel like this is just hiding the bash under a Python rug and calling it a day.

That's good! It's called the adapter pattern, and it's often a mark of pythonic code. Raymond Hettinger's talk "Beyond PEP8" goes into this kind of thing in detail (his example hides a Java api under a python rug, but the thrust is the same)

QuarkJets
Sep 8, 2008

Thermopyle posted:

I'm not sure you understand what you highlighted in my quote means?

I mean...using your own style in a code base is bad and worse than using a less popular convention for an unused variable. I'm not sure what's controversial about that.

In the text that I highlighted you're subtly implying that using _ for unused variables is objectively better than other conventions. It's the "vim vs emacs" of variable naming; both options are fine and if you're starting a new codebase then it's fine to use whatever you want

Fusion Restaurant
May 20, 2015
What IDEs do people like for doing data stuff in Python? I've been just using Spyder or Jupyter notebooks pretty much because they were the default options w/ Anaconda.

Has anyone found that a different IDE has some particular benefits?

QuarkJets
Sep 8, 2008

Fusion Restaurant posted:

What IDEs do people like for doing data stuff in Python? I've been just using Spyder or Jupyter notebooks pretty much because they were the default options w/ Anaconda.

Has anyone found that a different IDE has some particular benefits?

Data scientist checking in

PyCharm is not really made with data analysis in mind but it is still a really great IDE. It's what I use when it's available and when I'm not being lazy/opening Vim for a quick thing.

Spyder is solid but debugging in it is a real chore compared to PyCharm. I think it also doesn't have native version control and that its code refactoring tools are per-file rather than per-project, but I could be wrong; I haven't used it in a long time. I think it's only as popular as it is because it comes with Anaconda and PyCharm doesn't, so it's sort of the path of least resistance.

Jupyter notebooks are cool but I don't really use them, so I can't comment on their effectiveness as an IDE. Creating a new notebook felt clunky and weird when I was playing around with it. However, I do know that PyCharm lets you open Jupyter notebooks, so it seems like you should be able to have the best of both worlds if a notebook suits your needs

Azuth0667
Sep 20, 2011

By the word of Zoroaster, no business decision is poor when it involves Ahura Mazda.
I've completed the code academy course and am working on the MIT open courseware course recommended in the OP. It's been a lot easier than I thought to learn to code but, one thing I've been wanting to know is how do I go about drawing geometric shapes? None of the lessons I've done so far cover any kind of drawing.

The Earl of ToeJam
Jan 22, 2012

Azuth0667 posted:

I've completed the code academy course and am working on the MIT open courseware course recommended in the OP. It's been a lot easier than I thought to learn to code but, one thing I've been wanting to know is how do I go about drawing geometric shapes? None of the lessons I've done so far cover any kind of drawing.

May be too basic, but how about the turtle module? http://greenteapress.com/thinkpython2/html/thinkpython2005.html

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord

Azuth0667 posted:

I've completed the code academy course and am working on the MIT open courseware course recommended in the OP. It's been a lot easier than I thought to learn to code but, one thing I've been wanting to know is how do I go about drawing geometric shapes? None of the lessons I've done so far cover any kind of drawing.

If you want to use python to draw geometry I'd say look at pillow https://python-pillow.org/ - then I'd say that pillow kind of sucks and that Python isn't a super great language for doing this kind of thing (visual work). What exactly is it that you want to use the shapes for?

Azuth0667
Sep 20, 2011

By the word of Zoroaster, no business decision is poor when it involves Ahura Mazda.

The March Hare posted:

If you want to use python to draw geometry I'd say look at pillow https://python-pillow.org/ - then I'd say that pillow kind of sucks and that Python isn't a super great language for doing this kind of thing (visual work). What exactly is it that you want to use the shapes for?

I want to be able to make a grid out of them and assign each one a different random number.

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord

Azuth0667 posted:

I want to be able to make a grid out of them and assign each one a different random number.

OK, yeah, I would use Pillow - should be fine.

baka kaba
Jul 19, 2003

PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Azuth0667 posted:

I want to be able to make a grid out of them and assign each one a different random number.

But what are you going to do with that?

If you have a bigger end goal in mind (like making an interactive sudoku grid or something) you might want to get learning a GUI library that will handle all the inputs and stuff

Azuth0667
Sep 20, 2011

By the word of Zoroaster, no business decision is poor when it involves Ahura Mazda.

baka kaba posted:

But what are you going to do with that?

If you have a bigger end goal in mind (like making an interactive sudoku grid or something) you might want to get learning a GUI library that will handle all the inputs and stuff

I'm trying to make an example of viral infection of cells for an intro class. Basically I want to be able to let the students choose a couple cells then have the infection spread based on what kind of virus it was. It will have to check if the cell is already infected, is it in division - what phase of division, is it senescent, does it have a lot of resources, does it have a lot of waste products, what kind of cell is it? If the conditions are true for that virus then it spreads to that cell. Most of those conditions seem like I could model them as booleans or integers.

The first step I have laid out is to take a square grid then fill it with closed shapes that are touching each other. I want to assign different random values representing the conditions I listed above and to visually indicate this by filling the respective shape with a color. Next I'm going to get a student volunteer to come up and pick the cell to be infected and do that until I've run out of viruses to use. Then watch the spread of the virus through the cells.

Jose Cuervo
Aug 25, 2004

Azuth0667 posted:

I'm trying to make an example of viral infection of cells for an intro class. Basically I want to be able to let the students choose a couple cells then have the infection spread based on what kind of virus it was. It will have to check if the cell is already infected, is it in division - what phase of division, is it senescent, does it have a lot of resources, does it have a lot of waste products, what kind of cell is it? If the conditions are true for that virus then it spreads to that cell. Most of those conditions seem like I could model them as booleans or integers.

The first step I have laid out is to take a square grid then fill it with closed shapes that are touching each other. I want to assign different random values representing the conditions I listed above and to visually indicate this by filling the respective shape with a color. Next I'm going to get a student volunteer to come up and pick the cell to be infected and do that until I've run out of viruses to use. Then watch the spread of the virus through the cells.

This sounds like something you might be able to accomplish using Agent Based Simulation. I have never used Mesa (https://pypi.python.org/pypi/Mesa/), but it should allow you to do what you want. You will have to write the rulesets for how viruses spread etc, but not have to worry about writing the graphics side of things.

Willzilla
Aug 16, 2006

Rawr
http://py.processing.org/tutorials/drawing/ in combination with some Processing videos from The Coding Train youtube channel guy could get you started with drawing stuff pretty easily

vikingstrike
Sep 23, 2007

whats happening, captain
I'm trying to make the transition to python 3.6, and am wondering if there are any helpful resources online that summarize the big changes in python 3? I found the What's New section of the official documentation, but am looking for more of a Cliff Notes version until I have time to finish reading through the docs.

Dominoes
Sep 20, 2007

A few highlights:
-Built-in functions tend to return iterators rather than lists. Ie range(), .items(), map() etc.
-Division doesn't auto-round.
-string encoding is handled differently
-print is a function
-Python 3 supports a number of new features.

I'd just dive in, and fix things as they break. You can try the new features once you're comfortable with the changes.

Dominoes fucked around with this message at 22:02 on Feb 7, 2017

Eela6
May 25, 2007
Shredded Hen

vikingstrike posted:

I'm trying to make the transition to python 3.6, and am wondering if there are any helpful resources online that summarize the big changes in python 3? I found the What's New section of the official documentation, but am looking for more of a Cliff Notes version until I have time to finish reading through the docs.

IN:
Python code:
pepper = 'jalapeño'
pepper_as_bytes = b'jalape\xc3\xb1o'

print("""STRINGS: Python 3 strings are python 2 unicode strings. """)
print('unicode', pepper)
print('bytes:', pepper_as_bytes)

print(f"""'jalapeño'.encode() == {pepper_as_bytes}: {'jalapeño'.encode() == pepper_as_bytes}""")

print('''
NUMBERS: Division does not auto-coerce. 
floordiv '//' produces rounds towards -&#8734 (-INF);. Produces INT if both args are ints, else FLOAT
realdiv '/' produces FLOAT and rounds using IEE double-precision arithmetic 
''')
print(f'5 // 2 = {5//2} and has type {type((5//2))}')
print(f'5/2 = {5/2} and has type {type((5/2))}')
print(f'-5.0 // 2 = {-5.//2} and has type {type((-5.//2))}')
print(f'-5.0 / -2. = {5./2.} and has type {type((5./2.))}')
OUT:
Python code:
STRINGS: Python 3 strings are python 2 unicode strings. 
unicode jalapeño
bytes: b'jalape\xc3\xb1o'
'jalapeño'.encode() == b'jalape\xc3\xb1o': True

NUMBERS: Division does not auto-coerce. 
floordiv '//' produces rounds towards -∞ (-INF). Produces INT if both args are ints, else FLOAT
realdiv '/' produces FLOAT and rounds using IEE double-precision arithmetic 

5 // 2 = 2 and has type <class 'int'>
5/2 = 2.5 and has type <class 'float'>
-5.0 // 2 = -3.0 and has type <class 'float'>
-5.0 / -2. = 2.5 and has type <class 'float'>
Also a ton of stuff with iterators, generators, new-style functions, async, etc. But strings and division are the things that will break your old code, so they're worth knowing.

Here's a somewhat contrived example to show why iterators and generators are such a big deal:

IN:
Python code:
import time
import sys

def fizzbuzz():
    print('Python 3: Iterator')
    start_time = time.time()
    total_range = range(500000000)
    print(f'memory usage of range: {sys.getsizeof(total_range)}')
    fizz_count = 0
    buzz_count = 0
    fizz_buzz_count = 0
    for i in total_range:
        if i % 15 == 0:
            fizz_buzz_count += 1
        elif i % 5 == 0:
            buzz_count += 1
        elif i % 3 == 0:
            fizz_count += 1
        
        if fizz_buzz_count == 1000000:
            print('we did it!')
            end_time = time.time()-start_time
            print(f'it took {end_time} seconds')
            return fizz_buzz_count, buzz_count, fizz_count


def fizzbuzz_python_2():
    print('Python 2: Explicit List')
    start_time = time.time()
    total_range = list(range(500000000))
    print(f'memory usage of range: {sys.getsizeof(total_range)}')
    fizz_count = 0
    buzz_count = 0
    fizz_buzz_count = 0
    for i in total_range:
        if i % 15 == 0:
            fizz_buzz_count += 1
        elif i % 5 == 0:
            buzz_count += 1
        elif i % 3 == 0:
            fizz_count += 1
        if fizz_buzz_count == 1000000:
            print('we did it!')
            end_time = time.time()-start_time
            print(f'it took {end_time} seconds')
            return fizz_buzz_count, buzz_count, fizz_count

if __name__ == '__main__':
    print('running fizzbuzz():')
    fizzbuzz()
    print('\nrunning fizzbuzz_python_2():')
    fizzbuzz_python_2()
OUT:
code:
running fizzbuzz():
Python 3: Iterator
memory usage of range: 48
we did it!
it took 2.1471869945526123 seconds

running fizzbuzz_python_2():
Python 2: Explicit List
memory usage of range: 4500000112
we did it!
it took 22.43296456336975 seconds

Eela6 fucked around with this message at 23:43 on Feb 7, 2017

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
I'm documenting some tutorials/examples with Sphinx and for each tutorial (it's a python module/script) I'm writing the documentation at the module level docstring and then using automodule in my docs/source directory to get it generated.

I want to include the actual code into the HTML page that gets generated. How do I do that? I tried with 'literalinclude' but this, well, literally includes also its docstring which I don't want. I guess I could just specify which lines to include but it would be better if it would automatically just include everything below the module level docstring.

Secondly is there a variable I can tell the docstring to include itself (with its path automatically expanded)? Right now I have to do "literalinclude: ../../tutorials/.." and so on. I'd like to just do something like "literalinclude: $SELF".

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
I think this is more of a Python question than a Django question...

I'm using django-after-response to invoke a long method after a request completes. I want to log that long running method in a separate file (as well as to the normal application log) so I do the following:
code:
                temp_deploy_log = NamedTemporaryFile()
                formatter = logging.Formatter("%(asctime)-15s %(levelname)-8s %(message)s")
                deploy_log_handler = logging.FileHandler(deployment.log.path)
                deploy_log_handler.setFormatter(formatter)
                deploy_log_handler.setLevel(logging.INFO)
                logging.getLogger('').addHandler(deploy_log_handler)
The problem I'm running into is that temp_deploy_log ends up having log entries from multiple threads if they are running concurrently. How do I prevent this from happening? I'm using uWSGI if that makes a difference.

Whybird
Aug 2, 2009

Phaiston have long avoided the tightly competetive defence sector, but the IRDA Act 2052 has given us the freedom we need to bring out something really special.

https://team-robostar.itch.io/robostar


Nap Ghost
What's tkinter like for working with accessibility things like a screen reader? All the articles I can find suggest the situation is pretty dire, but they're also something like three or four years old -- have things gotten any better, or am I better off switching to a different toolkit?

Space Kablooey
May 6, 2009


fletcher posted:

I think this is more of a Python question than a Django question...

I'm using django-after-response to invoke a long method after a request completes. I want to log that long running method in a separate file (as well as to the normal application log) so I do the following:
code:
                temp_deploy_log = NamedTemporaryFile()
                formatter = logging.Formatter("%(asctime)-15s %(levelname)-8s %(message)s")
                deploy_log_handler = logging.FileHandler(deployment.log.path)
                deploy_log_handler.setFormatter(formatter)
                deploy_log_handler.setLevel(logging.INFO)
                logging.getLogger('').addHandler(deploy_log_handler)
The problem I'm running into is that temp_deploy_log ends up having log entries from multiple threads if they are running concurrently. How do I prevent this from happening? I'm using uWSGI if that makes a difference.

Just spitballing here, but try to give the NamedTemporaryFile an unique prefix and/or suffix? Also, you might want to give it delete=False.

Dominoes
Sep 20, 2007

Hey dudes. Looking to see if there are pre-built solutions that would help for a project I'm going to undertake for work. I'm trying to build a scheduler app for a flying club. In early planning stages. Currently, we have a white board, and magnetic name tags we shift around. The schedule changes frequently, and there are different qualifications, training requirements etc each person maintains that place constraints on the scheduling. Subject to chaos, people getting sick, last-minute changes etc.

Thought 1: Scheduling? Aren't there a bunch of already made solutions, since it's such a common task?

Thought 2: This seems like a unique use case, with many non-standard variables; roll your own rather than fit a square peg in a round hole.

Thoughts?

vikingstrike
Sep 23, 2007

whats happening, captain



Thanks!

huhu
Feb 24, 2006
I'm building a webcam that pushes images to the internet after capture. I found a script to upload the images to Dropbox and it works great. However, I've found an issue that if the internet goes down mid upload, no error is returned and the upload just gets stuck at whatever percent its at. This wouldn't really be an issue however my hackspace's internet is utter garbage and I suspect the internet will go down pretty often. Is there some nifty way to detect a stuck upload and break?

Tigren
Oct 3, 2003

huhu posted:

I'm building a webcam that pushes images to the internet after capture. I found a script to upload the images to Dropbox and it works great. However, I've found an issue that if the internet goes down mid upload, no error is returned and the upload just gets stuck at whatever percent its at. This wouldn't really be an issue however my hackspace's internet is utter garbage and I suspect the internet will go down pretty often. Is there some nifty way to detect a stuck upload and break?

If your script uses the Dropbox v2 API, it looks like it raises a requests.exceptions.ConnectionError when it times out.

code:
ConnectionError: ('Connection aborted.', timeout('The write operation timed out',))

Tigren fucked around with this message at 03:34 on Feb 10, 2017

huhu
Feb 24, 2006

Tigren posted:

If your script uses the Dropbox v2 API, it looks like it raises a requests.exceptions.ConnectionError when it times out.

code:
ConnectionError: ('Connection aborted.', timeout('The write operation timed out',))
I think that error message is hidden away in the script I found on Github.

VikingofRock
Aug 24, 2008




Hey so what's the state of the art way of doing packaging? I want something where I can run some command(s) on the command line and it'll download all the right dependencies for my script, download the right version of python, and run the script. But it seems like there are a million different tools that each do a part of this and it's not really clear to me how they intersect.

Tigren
Oct 3, 2003

huhu posted:

I think that error message is hidden away in the script I found on Github.

Can you link to the script you're using? The Dropbox python API is very straight forward if you just want to write your own.

Python code:
In [1]: import dropbox
In [2]: import requests
In [3]: dropbox_client = dropbox.Dropbox(API_TOKEN)
In [4]: try:
            with open('test', 'rb') as f:
                upload = dropbox_client.files_upload(f.read(), '/path/to/file')
        except (requests.exceptions.ConnectionError) as err:
            print('Error uploading picture: ', err)
        else:
            print('Upload succeeded: ', upload)

quote:

Upload succeeded: FileMetadata(name='file', id='id:UmsjUZ9eogAAAAAAAAAACg', client_modified=datetime.datetime(2017, 2, 10, 7, 1, 10), server_modified=datetime.datetime(2017, 2, 10, 7, 1, 10), rev='8540e8f17', size=6, path_lower='/path/to/file', path_display='/path/to/file', parent_shared_folder_id=None, media_info=None, sharing_info=None, property_groups=None, has_explicit_shared_members=None)

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

VikingofRock posted:

Hey so what's the state of the art way of doing packaging? I want something where I can run some command(s) on the command line and it'll download all the right dependencies for my script, download the right version of python, and run the script. But it seems like there are a million different tools that each do a part of this and it's not really clear to me how they intersect.

I'm not an expert either but I think these two links should help:

https://packaging.python.org/distributing/
https://github.com/pypa/sampleproject

If you clone the sampleproject you should be able to customize it to your own package and then just run a pip install . from your project directory. You should be able to configure it to download Python dependencies as well.

I think if you want to manage non-Python dependencies then you'll have to use Conda/Anaconda, like "download the right version of python." Though I gotta wonder why do you think you need to manage the actual Python installation yourself? That just seems unnecessarily complicated and unless you specifically need to do for whatever reason that then wouldn't it be better off just testing your script against different Python versions? The sampleproject I linked has some scripts/tools that do just that for you, automatically.

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Dominoes posted:

Thoughts?

Sounds like you want a rules engine.

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord
If I need to edit some audio files (mp3 & wav) I should just use FFmpeg right? Is there some super secret audio editing package in Python that I don't know about? I'm just going to be selectively muting sections of audio based on timestamps, so nothing super fancy.

huhu
Feb 24, 2006

Tigren posted:

Can you link to the script you're using? The Dropbox python API is very straight forward if you just want to write your own.

Python code:
In [1]: import dropbox
In [2]: import requests
In [3]: dropbox_client = dropbox.Dropbox(API_TOKEN)
In [4]: try:
            with open('test', 'rb') as f:
                upload = dropbox_client.files_upload(f.read(), '/path/to/file')
        except (requests.exceptions.ConnectionError) as err:
            print('Error uploading picture: ', err)
        else:
            print('Upload succeeded: ', upload)

Oh that's much nicer than what I'm using, thanks.

Cingulate
Oct 23, 2012

by Fluffdaddy

The March Hare posted:

If I need to edit some audio files (mp3 & wav) I should just use FFmpeg right? Is there some super secret audio editing package in Python that I don't know about? I'm just going to be selectively muting sections of audio based on timestamps, so nothing super fancy.
You can do that in Scipy, with wavread. It's just one or two lists (arrays) of numbers; sample rate plus time stamp gives you the indices, then you set these to zero and write back the file.

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord

Cingulate posted:

You can do that in Scipy, with wavread. It's just one or two lists (arrays) of numbers; sample rate plus time stamp gives you the indices, then you set these to zero and write back the file.

Can it deal w/ MP3s though?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Anyone used uvloop as a replacment for the async event loop?

I've got an async-using application that is going to be approaching some limits in requests per second and I'm researching some of my alternatives and uvloop looks promising...

Adbot
ADBOT LOVES YOU

GrAviTy84
Nov 25, 2004

I have a data set that has a lot of features. Each row is an event but each event is part of a larger event (it's power outage data, I have bulk outage events, broken down into restoration steps, as well as geographical categories, device categories, etc.) I want to be able to "collapse" different categories so that, say I just have a total time elapsed for one outage event by summing over the time taken in each restoration step, but maintain dataframe structure so that if I wanted to break it down further into GIS or month, or whatever sub categories, it would be easy to do later on. As such, I'm trying to develop a function that will take a df, a column name to categorize rows into, and a column name that is summed in each category. I'd like to keep it as standalone as I can so I can collapse these indices in different ways depending on what features I want to study. right now I have:

code:
def sumByFeature(df,sumIndex,sumValue):
    featureIndex = df.groupby(by=sumIndex)[sumValue].sum()
    df = pd.concat([df,featureIndex], axis=1, join_axes=[featureIndex['sumIndex'])
    return df.drop_duplicates([sumIndex],keep='first')
but it really doesnt like the way I'm passing the column name into the function. Is there a correct way to pass a column name of a dataframe into a function?

semi-related: is there a data science thread? I suppose it could be either in here or in SAL.

edit: there might be a much better way to do this, too. I'm new to doing this stuff with python so I'm still learning a lot of the methods that I can use stock.

GrAviTy84 fucked around with this message at 21:36 on Feb 10, 2017

  • Locked thread