Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Dominoes
Sep 20, 2007

OnceIWasAnOstrich posted:

I taught a learning Python class once with Anaconda and I found everything to be too much work in Windows and unfortunately I never found a good way to make Windows act enough like Unix so I just skipped it and taught the entire class in the Spyder IDE that comes with Anaconda. Not my finest moment, but I wasn't getting paid and didn't want to do any more work than necessary.

Why do you need environments at all?
What in particular was too much work in Windows?

Adbot
ADBOT LOVES YOU

SurgicalOntologist
Jun 17, 2004

OnceIWasAnOstrich posted:

Why do you need environments at all?

To teach best practices for reproducibility, e.g. listing the precise dependencies of each project, which is greatly facilitated with putting each project in its own environment.

OnceIWasAnOstrich
Jul 22, 2006

Dominoes posted:

What in particular was too much work in Windows?

Anything that wasn't in Anaconda (Hey now you get to run executables off of some random dude's website instead of using pip), any time I wanted to use awk/sed/grep to explain/show a file format to them, differences in filesystems that kept popping up, all sorts of crazy problems that pop up when people who don't really understand computers try to run something like Anaconda on Windows instead of unix-like, the really lovely terminal (why is cmd.exe so much worse than Terminal.app or pick-a-term in a Linux? (I use PuTTY on my Windows machines to SSH to localhost just to get a nicer terminal), and probably other things I don't remember because I tried to block those days out of my memory.

SurgicalOntologist posted:

To teach best practices for reproducibility, e.g. listing the precise dependencies of each project, which is greatly facilitated with putting each project in its own environment.

Yeah, you can still do all of those without activating the environments into your shell, I honestly prefer using paths to activation scripts because then if you use IDEs or anything you don't have to figure out how to use your environment, you just use the same interpreter you always did, etc.

SurgicalOntologist
Jun 17, 2004

OnceIWasAnOstrich posted:

Yeah, you can still do all of those without activating the environments into your shell, I honestly prefer using paths to activation scripts because then if you use IDEs or anything you don't have to figure out how to use your environment, you just use the same interpreter you always did, etc.

I think I'm going to teach it your way, I gave up putting a wrapper together, it's a lot of effort for not much gain.

OnceIWasAnOstrich posted:

Anything that wasn't in Anaconda (Hey now you get to run executables off of some random dude's website instead of using pip), any time I wanted to use awk/sed/grep to explain/show a file format to them, differences in filesystems that kept popping up, all sorts of crazy problems that pop up when people who don't really understand computers try to run something like Anaconda on Windows instead of unix-like, the really lovely terminal (why is cmd.exe so much worse than Terminal.app or pick-a-term in a Linux? (I use PuTTY on my Windows machines to SSH to localhost just to get a nicer terminal), and probably other things I don't remember because I tried to block those days out of my memory.

I've been playing around on Windows all day, using msysgit as a way to get Unix-like on Windows is actually pretty nice, and there are console emulators to make cmd or msys easier to work with. Not even going to attempt anything with binaries that isn't on conda though.

QuarkJets
Sep 8, 2008

SurgicalOntologist posted:

To teach best practices for reproducibility, e.g. listing the precise dependencies of each project, which is greatly facilitated with putting each project in its own environment.

That's a pretty neat idea, but won't each Anaconda package come with a bunch of modules that aren't necessarily included in native Python anyway? I'd think that just having them keep track of what gets imported and identifying which modules are standard, which are 3rd party, which are theirs, etc would be good enough.

I tend to group my imports into 3 line-separated sections: one for native Python modules (sys, os, etc), one for 3rd party modules (pandas, numpy, etc), and one for local modules (things that I wrote myself or by a coworker, etc)

e: And I don't see what's so bad about teaching some introductory stuff in Spyder

QuarkJets fucked around with this message at 04:45 on May 24, 2014

SurgicalOntologist
Jun 17, 2004

QuarkJets posted:

That's a pretty neat idea, but won't each Anaconda package come with a bunch of modules that aren't necessarily included in native Python anyway? I'd think that just having them keep track of what gets imported and identifying which modules are standard, which are 3rd party, which are theirs, etc would be good enough.

I tend to group my imports into 3 line-separated sections: one for native Python modules (sys, os, etc), one for 3rd party modules (pandas, numpy, etc), and one for local modules (things that I wrote myself or by a coworker, etc)

e: And I don't see what's so bad about teaching some introductory stuff in Spyder

I've always just used pip freeze for this purpose and am only now exploring Anaconda, but I think conda list -e can serve the same purpose for keeping track of external dependencies, including non-Python dependencies (for conda). Also, if projects aren't isolated in environments you'll eventually run into a dependency version conflict. Or am I misunderstanding? I could very well be.

The only issue with Spyder seems to be that Python3 support is only in beta and therefore non-obvious to install (although the release seems to be imminent). I'm not going to be avoiding IDEs though, especially for the actual programming part. But the logistics (version control, package management, environments) I'll teach on the command-line. Maybe it's silly but I'm afraid of tying them down to any particular IDE. Once one understands the basics it's easy enough to pick up a new IDE/VC frontend and find the menu items for "install package", "switch virtualenv", "commit", etc.

SurgicalOntologist fucked around with this message at 05:31 on May 24, 2014

QuarkJets
Sep 8, 2008

SurgicalOntologist posted:

I've always just used pip freeze and am only now exploring Anaconda, but I think conda list -e can serve the same purpose for keeping track of external dependencies, including non-Python dependencies (for conda). Or am I misunderstanding? I could very well be.

Won't that just show you all of the packages that came with your current conda environment rather than just the packages that you need for your project? By default, anaconda usually comes with a lot of stuff

SurgicalOntologist
Jun 17, 2004

Oh, I wouldn't install the entirety of anaconda in every environment.

Turd Eater
May 11, 2003
http://conda.pydata.org/miniconda.html

Murodese
Mar 6, 2007

Think you've got what it takes?
We're looking for fine Men & Women to help Protect the Australian Way of Life.

Become part of the Legend. Defence Jobs.
Anyone know a decent replacement to Mechanize for 3.4? Thinking of writing an interface to Moodle 2.5 that isn't a huge piece of poo poo and lets us do things like upload feedback sheets to students by bulk without having to follow their arcane rules that don't allow the addition of any new files, and I have to do it without any server-side changes.

OnceIWasAnOstrich
Jul 22, 2006

RoboBrowser?

Murodese
Mar 6, 2007

Think you've got what it takes?
We're looking for fine Men & Women to help Protect the Australian Way of Life.

Become part of the Legend. Defence Jobs.
I'll have a look, cheers.

namaste friends
Sep 18, 2004

by Smythe
I'm having to do a bunch of repetitive stuff and I was wondering if any of you had any suggestions on how to make this more pythonic or just better.

I'm writing a method that does the following:

code:
def remote_api_call(condition):
	if condition 1:
		open_rest_session
		execute_rest_method(parameter1)
		close_rest_session
	elif condition2:
		open_rest_session
		execute_rest_method(parameter2)
		close_rest_session
	elif condition3:
		open_rest_session
		execute_rest_method(parameter3)
		close_rest_session
	else:
		pass
Is there a good way to write this without having to write out open_rest_session and close_rest_session multiple times? Is there some way to make this 'better'?

Edison was a dick
Apr 3, 2010

direct current :roboluv: only

Cultural Imperial posted:

I'm having to do a bunch of repetitive stuff and I was wondering if any of you had any suggestions on how to make this more pythonic or just better.

I'm writing a method that does the following:

code:
def remote_api_call(condition):
	if condition 1:
		open_rest_session
		execute_rest_method(parameter1)
		close_rest_session
	elif condition2:
		open_rest_session
		execute_rest_method(parameter2)
		close_rest_session
	elif condition3:
		open_rest_session
		execute_rest_method(parameter3)
		close_rest_session
	else:
		pass
Is there a good way to write this without having to write out open_rest_session and close_rest_session multiple times? Is there some way to make this 'better'?

You can consolidate the session logic by checking all the conditions first.
code:
def remote_api_call(condition):
    if not condition1 and not condition2 and not condition3:
        return
    open_rest_session
    if condition1:
        execute_rest_method(parameter1)
    elif condition2:
        execute_rest_method(parameter2)
    elif condition3:
        execute_rest_method(parameter3)
    close_rest_session
Though, it would probably worth while to turn the rest session handling into a context manager, which could be used like this instead:

code:
def remote_api_call(condition):
    if not condition1 and not condition2 and not condition3:
        return
    with rest_session:
        if condition1:
            execute_rest_method(parameter1)
        elif condition2:
            execute_rest_method(parameter2)
        elif condition3:
            execute_rest_method(parameter3)
Since you can have a context manager always clear up the session, even if your code raises an exception.

You could define one like this:
code:
import contextlib
@contextlib.contextmanager
def rest_session():
    s = open_rest_session()
    try:
        yield s
    except:
        abort_rest_session(s)
    else:
        close_rest_session(s)

namaste friends
Sep 18, 2004

by Smythe
That's awesome! Thanks Edison!

onionradish
Jul 6, 2006

That's spicy.
I'm frequently using a pattern to read from a text "config" file with values on a single line, and want to ignore blanks and comments (lines that start with #). Examples of the config files are RSS URLs to be scraped, folders to be indexed, etc.

Is there are more compact or better pattern than this generator?
Python code:
# Python 2.7, aiming for compatibility with Python 2 and 3
def get_stuff(filename):
    with open(filename, 'rb') as f:
        for line in f:
            line = line.decode('utf8').strip()
            if line and not line.startswith('#'):
                yield line

for item in get_stuff("foo.txt"):
	print(item)

Dominoes
Sep 20, 2007

Cultural Imperial posted:

That's awesome! Thanks Edison!
Seconding Edison's advice: 'With' is the ideal way to deal with this.

namaste friends
Sep 18, 2004

by Smythe
To take this another step, what do you guys think of eliminating all the conditional statements and using map?

I was reading this:

http://www.ibm.com/developerworks/library/l-prog/

Dominoes
Sep 20, 2007

Could you/someone else provide an example of how to use map? I haven't found a use for map that can't be done cleaner with a comprehension.

Depending on what the conditions are, something like this might work:
Python code:
def remote_api_call(condition):
    conditions = {condition1: parameter1,
                  condition2: parameter2,
                  condition3: parameter3}
    if condition not in list(conditions.keys()):
        return
    with rest_session:
        execute_rest_method(conditions[condition])
Another variant:
Python code:
def remote_api_call(condition):
    conditions = {condition1: parameter1,
                  condition2: parameter2,
                  condition3: parameter3}
    with rest_session:
        try:
            execute_rest_method(conditions[condition])
        except KeyError:
            pass

Dominoes fucked around with this message at 22:56 on May 28, 2014

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
That article concludes with saying that the "new" list comprehensions are the perfect way to do it (the article's from 2001).

FoiledAgain
May 6, 2007

onionradish posted:

I'm frequently using a pattern to read from a text "config" file with values on a single line, and want to ignore blanks and comments (lines that start with #). Examples of the config files are RSS URLs to be scraped, folders to be indexed, etc.

Is there are more compact or better pattern than this generator?
Python code:
# Python 2.7, aiming for compatibility with Python 2 and 3
def get_stuff(filename):
    with open(filename, 'rb') as f:
        for line in f:
            line = line.decode('utf8').strip()
            if line and not line.startswith('#'):
                yield line

for item in get_stuff("foo.txt"):
	print(item)

Depending on how your config files are set up, you might be interested in configparser. It handles some of this for you, including allowing for comments

namaste friends
Sep 18, 2004

by Smythe

Dominoes posted:

Could you/someone else provide an example of how to use map? I haven't found a use for map that can't be done cleaner with a comprehension.

Depending on what the conditions are, something like this might work:
Python code:
def remote_api_call(condition):
    conditions = {condition1: parameter1,
                  condition2: parameter2,
                  condition3: parameter3}
    if condition not in list(conditions.keys()):
        return
    with rest_session:
        execute_rest_method(conditions[condition])
Another variant:
Python code:
def remote_api_call(condition):
    conditions = {condition1: parameter1,
                  condition2: parameter2,
                  condition3: parameter3}
    with rest_session:
        try:
            execute_rest_method(conditions[condition])
        except KeyError:
            pass

I was barking up the wrong tree. What I really wanted was to use getattr to call a method from a library dynamically based upon input arguments, possibly presented in a list. So for example

code:
ARGS = ['GET', 'https://some.awesome.website', '/restful/api/path']

result = getattr(my.awesome.requests.library, ARGS[0])(ARGS[1:])
Thanks again to everyone replying to my questions. It's been a long time since I learned this much in one day.

SurgicalOntologist
Jun 17, 2004

Does anyone know how to install tkinter on Ubuntu into a virtualenv? Or where I can find the version installed with the apt package python3-tk in the system, so I can symlink it?

In short I'm getting this:
code:
>>> import tkinter
Traceback (most recent call last):
  File "/usr/lib/python3.3/tkinter/__init__.py", line 41, in <module>
    import _tkinter
ImportError: No module named '_tkinter'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/tkinter/__init__.py", line 43, in <module>
    raise ImportError(str(msg) + ', please install the python3-tk package')
ImportError: No module named '_tkinter', please install the python3-tk package
in python3.3 but not python3.4.

Maybe related, I seem to be having trouble install anything for 3.3, e.g.

code:
/tmp/pip_build_hen/cython/Cython/Plex/Scanners.c:8:22: fatal error: pyconfig.h: No such file or directory

 #include "pyconfig.h"

                      ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
thanks to (I think) this:
code:
$ pkg-config --libs --cflags python3
-I/usr/include/python3.4m -I/usr/include/x86_64-linux-gnu/python3.4m  -lpython3.4m  
Does pip not change the cflags?

SurgicalOntologist fucked around with this message at 01:48 on May 29, 2014

nonathlon
Jul 9, 2004
And yet, somehow, now it's my fault ...
Generic question with a Python bent:

I got sick of running my website through Wordpress and have decided to go retro and use a static site generator or something equivalent (so I can write stuff on my local computer and upload, rather than work through a web interface, etc.) So, what are my choices?

Pelican is the new up-and-comer, it works with restructured text. Great, huh? But my site is pretty much arranged around pages (for software, talks, etc.) and Pelican's really for a blog, despite assurance otherwise.

Alternatives? I've looked at Nikolai (too-blogish), Hyde (blogish), and several others (being blogish or needing you to write in HTML or YAML, etc.) So what I'd really like is something Pythonic, using restructured text or Markdown or the like, that handles pages not posts / blog entries.

Any pointers?

good jovi
Dec 11, 2000

'm pro-dickgirl, and I VOTE!

outlier posted:

Generic question with a Python bent:

I got sick of running my website through Wordpress and have decided to go retro and use a static site generator or something equivalent (so I can write stuff on my local computer and upload, rather than work through a web interface, etc.) So, what are my choices?

Pelican is the new up-and-comer, it works with restructured text. Great, huh? But my site is pretty much arranged around pages (for software, talks, etc.) and Pelican's really for a blog, despite assurance otherwise.

Alternatives? I've looked at Nikolai (too-blogish), Hyde (blogish), and several others (being blogish or needing you to write in HTML or YAML, etc.) So what I'd really like is something Pythonic, using restructured text or Markdown or the like, that handles pages not posts / blog entries.

Any pointers?

Jekyll is the classic option, and has the benefit of letting you just host it all on github easily. It's ruby, but whatever, and uses markdown (at least by default), which is nice because restructured text is hot garbage.

OnceIWasAnOstrich
Jul 22, 2006

good jovi posted:

Jekyll is the classic option, and has the benefit of letting you just host it all on github easily. It's ruby, but whatever, and uses markdown (at least by default), which is nice because restructured text is hot garbage.

It's not nearly as popular but I used Wok for a website. It is a lot like a less well maintained and advertised Jekyll but for Python. Who knows if it is dead now.

nonathlon
Jul 9, 2004
And yet, somehow, now it's my fault ...
I'll check both of those out.

There's a huge number of static site generators out there, but they're plagued by one of two things:
- actually being static blog generators, which would have though was the harder, more specialised task to just generating a consistent set of pages
- weird configuration details and files. Why should I have to explicitly define the create and modified dates and the slug to be used in some arcane YAML markup? I mean, just by looking at the file you should be able to work all that out.

good jovi
Dec 11, 2000

'm pro-dickgirl, and I VOTE!

outlier posted:

- weird configuration details and files. Why should I have to explicitly define the create and modified dates and the slug to be used in some arcane YAML markup? I mean, just by looking at the file you should be able to work all that out.

Basic key-value YAML is hardly arcane.

And the post date does kind of need to be explicit somewhere. You can't use the file's mtime/ctime, as that could easily just be whenever your repo was checked out on whatever host is actually compiling the html. And you should be able to make edits without possibly affecting the page's URL (ie blog posts with the date in the URL).

OnceIWasAnOstrich
Jul 22, 2006

outlier posted:


- actually being static blog generators, which would have though was the harder, more specialised task to just generating a consistent set of pages
.

I don't remember if Wok is for blogs primarily, but I used it entirely for a static-site that gets new pages about twice a year if that and is not in the least blog-like, and I remember that feeling pretty natural.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

It feels like you should be able to glue together some libraries to make your own task-specific static HTML generator in short order.

Of course, I've learned to be wary of any project where I think that as about 70% of the time it actually takes me forever.

tef
May 30, 2004

-> some l-system crap ->

outlier posted:

Alternatives? I've looked at Nikolai (too-blogish), Hyde (blogish), and several others (being blogish or needing you to write in HTML or YAML, etc.) So what I'd really like is something Pythonic, using restructured text or Markdown or the like, that handles pages not posts / blog entries.

I got quite far with Pandoc. The problems you'll face are that: markdown is clumsy for syntax extensions and every tool speaks a different dialect, rst is good for longer structured text, but crap for smaller things and a bit clunky in places, and every single one of the tools will probably break in mysterious ways. Enjoy!

salisbury shake
Dec 27, 2011
I have a regex with named groups and a string whose pattern to match can be repeated multiple times.

Example:
code:
01/01/2014 - 02/01/2014 "Some Info" Monday, Tuesday 12:00AM - 4:30AM City, Building, Apt 123
Which can also be
code:
01/01/2014 - 02/01/2014 "Some Info" Monday, Tuesday 12:00AM - 4:30AM City, Building, Apt 12302/02/2014 - 02/02/2014 "New Info" Wednesday, Thurs 11:00AM - 7:30AM New_City, New_Building, Apt 456
I'm able to match the pattern of the first, and hope to be able to go from this pseudo regex
code:
(?P<start_date> derp) - (?P<end_date> derp) "(?P<info> derp)" (?P<days> derp(, derp)*) (?P<start_time> derp) - (?P<end_time> derp) (?P<address> derp)
to something that can capture the repeating pattern and return a nice dictionary.

I could just take the match object, look at its 'end' method and compare it with the length of the string to match, and slice it from that index, and run the pattern match again on the substring; but I imagine there is a better way to accomplish this.

edit: did just that.

salisbury shake fucked around with this message at 22:18 on May 30, 2014

Alligator
Jun 10, 2009

LOCK AND LOAF

Thermopyle posted:

It feels like you should be able to glue together some libraries to make your own task-specific static HTML generator in short order.

Of course, I've learned to be wary of any project where I think that as about 70% of the time it actually takes me forever.
yeah I went down the static site generation road a while ago for a blog with with some extra stuff and most of them seemed much more complex than they needed to be.

ended up with a ~40 line script using python markdown that does exactly what I wanted. it's brittle as all hell but who cares it's just for me.

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.

Alligator posted:

yeah I went down the static site generation road a while ago for a blog with with some extra stuff and most of them seemed much more complex than they needed to be.

ended up with a ~40 line script using python markdown that does exactly what I wanted. it's brittle as all hell but who cares it's just for me.

I just started looking into this for one of my projects and found http://metalsmith.io. Node.js so fits in with the rest of my front end build processes without adding any system dependencies, and I sort of like the barebones approach to static site generators that doesn't try to tell you what you want.

Pollyanna
Mar 5, 2005

Milk's on them.


I'm having trouble understanding how to implement a workaround for Tkinter not having control over key holds vs. repeated key presses. As I understand it, the workaround is something like this:

code:

def key_hold():

   -if it already exists, check the timer.
        - if it's longer than a certain threshold (i.e. it's been a while since the last key press), then consider the button held.
        - if it's not, consider the button to be repeated, and release it.
Does that make sense? I really feel like I'm missing something.

SurgicalOntologist
Jun 17, 2004

I didn't get any responses on this the first time around, and this kind of design has come up again and I'm thinking about it.

Isn't this fairly common when you use composition, to "outsource" a protocol to an attribute? I find myself doing this a lot, and while I've never tried to do something tricky like the loops in this quote, I can't help but think there's some O-O concept I'm missing, or if not then there's an opportunity for syntactic sugar.

SurgicalOntologist posted:

Is there a way to define an attribute of a class, and say "when something tries to use an arithmetic operation on me, use this" or "when someone tries to use the sequence protocol on me, use this".

I guess it could work something like:

Python code:
class NumberedBag:
    def __init__(self, contents=None, number=1):
        self._contents = contents or []
        self._number = number

        for method in ('__iter__', '__next__', '__getitem__', '__setitem__',
                       '__delitem__', '__len__', '__bool__'):
            setattr(self, method, getattr(self._contents, method))

        for method in ('__add__', '__mul__'):  # etc.
            setattr(self, method, getattr(self._number, method))
Not sure if that would work but I think you get the idea. Is there syntactic sugar for this currently, or some sort of OO magic that I haven't figured out yet?

gently caress it... let's try a class decorator:

Python code:
def delegate(method_names, attribute_name):
    def wrap(class_):
        for method in method_names:
            setattr(class_, method, getattr(getattr(class_, attribute_name), method))

        return class_
    return wrap
Okay that wouldn't work because that's code for modifying a class instance, not a class. Any ideas?

SurgicalOntologist
Jun 17, 2004

Welp, made it into an afternoon project and even uploaded it to PyPi.

https://github.com/hsharrison/smartcompose

Good thing, bad thing?

EAT THE EGGS RICOLA
May 29, 2008

I need to validate the a ton of csv files before they get converted to JSON and passed to an API. I was going to use csvvalidator, but is there something better I should look at?

Edit: I guess that I'm going to also be receiving XML too if that changes anything.

EAT THE EGGS RICOLA fucked around with this message at 00:15 on Jun 4, 2014

tef
May 30, 2004

-> some l-system crap ->
Would you like to play a game?

code:
$ mkdir temp
$ cd temp
$ ls
$ virtualenv env
New python executable in env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
$ source env/bin/activate
code:
$ pip install wheel
Downloading/unpacking wheel
  Downloading wheel-0.23.0.tar.gz (50Kb): 50Kb downloaded
  Running setup.py egg_info for package wheel
    
    no previously-included directories found matching 'wheel/test/*/dist'
    no previously-included directories found matching 'wheel/test/*/build'
Installing collected packages: wheel
  Running setup.py install for wheel
    
    no previously-included directories found matching 'wheel/test/*/dist'
    no previously-included directories found matching 'wheel/test/*/build'
    Installing wheel script to .../temp/env/bin
    Installing wininst2wheel script to .../temp/env/bin
    Installing egg2wheel script to .../temp/env/bin
Successfully installed wheel
Cleaning up...

$ pip wheel
Usage: pip COMMAND [OPTIONS]

pip: error: No command by the name pip wheel
  (maybe you meant "pip help")
$
:argh:

Adbot
ADBOT LOVES YOU

good jovi
Dec 11, 2000

'm pro-dickgirl, and I VOTE!

Try upgrading pip

  • Locked thread