Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Rohaq: Aug 11, 2006

For anyone who's into Django, the Django Cheatsheet has been updated for 1.5

# ? Apr 17, 2013 16:18

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 23:54

Smarmy Coworker: May 10, 2008; by XyloJW

e: I think it might be throwing because of the sum(), actually. I need to trim my lists.

Smarmy Coworker fucked around with this message at 17:11 on Apr 17, 2013

# ? Apr 17, 2013 17:08

My Rhythmic Crotch: Jan 13, 2011

QuarkJets posted:

My girlfriend is interested in learning more web development skills. Maintaining a web site is a small facet of her job, so it actually does have some applicability to what she's doing. For instance, she knows how to use CSS and javascript. She knows that I've been using Python for a long time, and she has asked me if Python is a useful language to learn for web development purposes. I wasn't sure what to tell her; I've never done any web stuff.

Searching around on the web gives links to people talking about how awesome django is as a web framework, but I don't really understand what django does. Is it only useful for creating web applications with forms and the like or can it also just be used to make nice-looking web sites in a relatively easy way?

If she wants to get into it, Flask is the way to go for most people in my opinion. It's simple and expandable, with limitations that most 'casual' developers probably won't run into. It's certainly much easier to get into than Django.

The templating engine could make developing faster and easier, but I am sure there are javascript templating engines that do basically the same thing. The main reasons why I can imagine for someone to start using Flask would be if you want to implement stuff that's usually done in PHP, and as you might know, PHP is poo poo. The thing to watch out for is that when you go to production, you are probably not going to want to serve any Python-backed site with Apache because it's just too drat slow. People are moving to Nginx and that might not be possible in her environment. You are also not going to be using your cheapo $5/mo shared hosting - you need some kind of cloud/webapp host or a VPS.

Anyway I hope that gives you both some info if she decides to follow through with it. I would say go for it!

# ? Apr 17, 2013 20:15

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

My Rhythmic Crotch posted:

If she wants to get into it, Flask is the way to go for most people in my opinion. It's simple and expandable, with limitations that most 'casual' developers probably won't run into. It's certainly much easier to get into than Django.

The templating engine could make developing faster and easier, but I am sure there are javascript templating engines that do basically the same thing. The main reasons why I can imagine for someone to start using Flask would be if you want to implement stuff that's usually done in PHP, and as you might know, PHP is poo poo. The thing to watch out for is that when you go to production, you are probably not going to want to serve any Python-backed site with Apache because it's just too drat slow. People are moving to Nginx and that might not be possible in her environment. You are also not going to be using your cheapo $5/mo shared hosting - you need some kind of cloud/webapp host or a VPS.

Anyway I hope that gives you both some info if she decides to follow through with it. I would say go for it!

Biggest problem with Flask is it's insistence on context locals. Armin's a smart guy but he's against needing to pass certain arguments around to functions and I think it's completely wrong.

So Flask is great, but be aware that those context locals can and will bite you in the rear end sooner or later.

# ? Apr 18, 2013 14:16

Gaukler: Oct 9, 2012

Steampunk Hitler posted:

Biggest problem with Flask is it's insistence on context locals. Armin's a smart guy but he's against needing to pass certain arguments around to functions and I think it's completely wrong.

So Flask is great, but be aware that those context locals can and will bite you in the rear end sooner or later.

Can you expand on this? I'm curious on what the gotchas would be.

# ? Apr 18, 2013 15:57

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Debugging.

# ? Apr 18, 2013 16:24

Gaukler: Oct 9, 2012

Suspicious Dish posted:

Debugging.

I haven't had issues debugging Flask apps yet. I agree context locals for everything are a somewhat odd design decision but I haven't been bit by it.

# ? Apr 18, 2013 18:24

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

Gaukler posted:

Can you expand on this? I'm curious on what the gotchas would be.

Debugging is one, testing is another. Reusability is a third.

With "magic" context variables you never know what values any particular function requires. When you explicitly pass everything (and so does your function to it's internal functions etc) you know exactly what variables your function can use. This makes it really easy to reason about how a function is being called, and makes it easy to test, you just simply pass different values in.

However when you rely on context locals like Flask's request proxy you really have no way except by tracing the entire call graph of a function to know if it depends on being able to have the request or not. Gary Bernhardt did a good talk at PyCon 2013 that deals with the same sort of idea called "Boundaries". The basic idea being to have clear defined boundaries for the various "pieces" in your program and only pass along the things that are needed (Don't pass the entire request object if you only need the mimetype for instance).

The flipside of this (and Armin's rationale for them) is that it forces you to change the API of your functions when an internal one now wants to use a request. I disagree with him that this is a bad thing because as far as I'm concerned you've still changed your API because the function still requires that variable, just now it's a "magic" implicit variable that isn't obviously required by looking at your funcs.

Basically if you start with:

code:

    def inner():
        print "Inner!"

    def outer():
        inner()

The Flask method would change it to this:

code:

    def inner():
        print "Inner!", request.method

    def outer():
        inner()

and the api of outer() and inner() has not changed so you don't need to update other code. Except you do if you use the code anywhere that the magic request variable isn't setup.

Doing it explicitly would require you to change it to:

code:

    def inner(method):
        print "Inner!", method

    def outer(method):
        inner(method)

Requiring you to change anywhere in the code you used either outer() or inner() but making it way easier to maintain the code.

# ? Apr 18, 2013 18:38

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

Gaukler posted:

I haven't had issues debugging Flask apps yet. I agree context locals for everything are a somewhat odd design decision but I haven't been bit by it.

Then you're not doing anything interesting enough

Explicit boundaries are *great* for maintainability.

# ? Apr 18, 2013 18:49

Gaukler: Oct 9, 2012

Steampunk Hitler posted:

Thanks, this makes a lot of sense. What's your take on the Python web framework space? Is it pretty much Django or nothing? I'm putzing around with a personal project that has a fairly small core feature set but has a fairly long 'nice to have' list and I've been trying to test out various frameworks to see what they're like when programming actual features.

Steampunk Hitler posted:

Then you're not doing anything interesting enough Explicit boundaries are *great* for maintainability.

Agreed on both, I've done small projects in Flask because they're quick to have a result up and easy enough to maintain if its really just a bunch of API endpoints, but I do see the value in explicit boundaries.

# ? Apr 18, 2013 18:54

Opinion Haver: Apr 9, 2007

I've been doing a lot of Haskell work lately because it's my favorite language, and I've found that when I do Python I wind up passing parameters around explicitly more (Haskell doesn't really have mutable state in the same way most languages do, so you can't take the 'just shove it all into a hidden object' approach). It's a lot easier to debug.

# ? Apr 18, 2013 19:03

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I just found out something that excited me so I had to share.

You know how when you're debugging in (i)pdb you sometimes wish you could write more complex loops or something? No? Well, I do sometimes. Here's how you do it:

Python code:

ipdb> !import code; code.interact(local=vars())
Python 2.7.3 (default, Sep 26 2012, 21:51:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
In :

You have to hit Ctrl-D to get back to your (i)pdb session.

# ? Apr 18, 2013 19:07

QuarkJets: Sep 8, 2008

I feel like I'm typing 'self' a whole lot when I write classes. Using self.method() when I need to call internal methods, self.object to access internal variables, etc. This looks ugly to me; is there a way to prettify it?

# ? Apr 18, 2013 19:25

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

QuarkJets posted:

I feel like I'm typing 'self' a whole lot when I write classes. Using self.method() when I need to call internal methods, self.object to access internal variables, etc. This looks ugly to me; is there a way to prettify it?

Use a different name than self when writing your method signatures.

# ? Apr 18, 2013 19:28

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

Thermopyle posted:

Use a different name than self when writing your method signatures.

Don't actually do this. Just use self and suck it up. It's a de facto standard.

# ? Apr 18, 2013 19:55

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

Gaukler posted:

Thanks, this makes a lot of sense. What's your take on the Python web framework space? Is it pretty much Django or nothing? I'm putzing around with a personal project that has a fairly small core feature set but has a fairly long 'nice to have' list and I've been trying to test out various frameworks to see what they're like when programming actual features.

I've never used Pyramid so I can't speak to it.

Django is my go to framework it's not immune to the global state (*cough* django.conf.settings) but it's better than Flask and pytest-django takes care of the bulk of the issues for me there. I'm a core contributor to Django now so I may be biased.

Flask is great for small one off things but everytime I use it I start to get angry at the context locals and end up throwing it away and redoing it in Django.

Alex Gaynor is in love with using Werkzeug directly (see https://github.com/topazproject/topaz-site) which if you like the "micro" part of Flask it is not a bad option since Flask is mostly just a layer ontop of Werkzeug with some niceties and magic ontop of it.

If you're using Twisted there's a framework called klein which is supposed to be Flaskish but using Twisted's Async framework, it's pretty cool.

Someday I'd love to write my own but I find my free time is basically non existent anymore.

# ? Apr 18, 2013 20:00

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

Oh hey, Testing Framework.

Let me just recommend to anyone whose testing their software (and if you're not you should start!) to try out pytest. It's a really great library for writing tests that removes a lot of the boilerplate, I have a particular test that runs over a thousand actual test cases (that can each fail independently) from a thousand line data file (csv basically) and about 5 lines of code. It does a lot of great things.

* Use actual asserts without missing the "pretty printing". Yes this means ``assert 1 == 1`` instead of ``self.assertEquals(1, 1)`` without losing the great contextual data.
* Parametrized tests (See Below)
* Super Simple "Fixtures" that add powerful features that can be "scoped" to a test case, to a module, or globally. (See Below)
* Test discovery, no need to subclass a particular class (or use a class at all if you don't want!)

Here's an example of a parametrized test:

code:

import pytest


@pytest.mark.parametrize(("input", "expected"), [
    ("3+5", 8),
    ("2+4", 6),
    ("6*9", 42),
])
def test_eval(input, expected):
    assert eval(input) == expected

Which will run as if you wrote 3 separate test cases one for each entry in the list (shows as seperate test cases, fails or passes independently, etc).

Here's an example of a built in fixture:

code:

import os


# Before this test is ran a temporary directory will be created and passed into the
#    tmpdir variable. When the test is done executing pytest will clean it up automatically.
def test_create_file(tmpdir):
    p = tmpdir.mkdir("sub").join("hello.txt")
    p.write("content")
    assert p.read() == "content"
    assert len(tmpdir.listdir()) == 1

And here's an example of writing your own fixture!

code:

import pytest

@pytest.fixture
def smtp():
    import smtplib
    return smtplib.SMTP("merlinux.eu")


# The smtp variable will be equal to the return value of the smtp fixture function.
#    The smtp fixture will be called once for each test case that uses it (default "scope"
#    is function). However by passing the scope argument to pytest.fixture you can reuse
#    the same return value for the entire module, or the entire test run.
def test_ehlo(smtp):
    response, msg = smtp.ehlo()
    assert response == 250
    assert "merlinux" in msg

# ? Apr 18, 2013 20:14

Opinion Haver: Apr 9, 2007

QuarkJets posted:

I feel like I'm typing 'self' a whole lot when I write classes. Using self.method() when I need to call internal methods, self.object to access internal variables, etc. This looks ugly to me; is there a way to prettify it?

Python code:

import inspect

class Magic(object):
    def __init__(self, x):
        self.x = x

    def foo(self):
        exec ""
        magic_locals()
        print "foo says x is %d" % x

    def bar(self):
        exec ""
        magic_locals()
        print "calling foo!"
        foo()

def magic_locals():
    stack = inspect.stack()
    locals_ = stack[1][0].f_locals
    obj = locals_["self"]
    for attr in dir(obj):
        locals_[attr] = getattr(obj, attr)


magic = Magic(9)
magic.bar()

I don't think this will actually work in Python 3 since it apparently depends on exec "" forcing the locals storage to be in a state where inspect can modify it, which it might not on Python 3. Also if you use this in anything I hate you.

Opinion Haver fucked around with this message at 20:20 on Apr 18, 2013

# ? Apr 18, 2013 20:18

QuarkJets: Sep 8, 2008

yaoi prophet posted:

Python code:
import inspect

class Magic(object):
    def __init__(self, x):
        self.x = x

    def foo(self):
        exec ""
        magic_locals()
        print "foo says x is %d" % x

    def bar(self):
        exec ""
        magic_locals()
        print "calling foo!"
        foo()

def magic_locals():
    stack = inspect.stack()
    locals_ = stack[1][0].f_locals
    obj = locals_["self"]
    for attr in dir(obj):
        locals_[attr] = getattr(obj, attr)


magic = Magic(9)
magic.bar()
I don't think this will actually work in Python 3 since it apparently depends on exec "" forcing the locals storage to be in a state where inspect can modify it, which it might not on Python 3. Also if you use this in anything I hate you.

That's even uglier than just using 'self' everywhere

# ? Apr 18, 2013 21:00

Haystack: Jan 23, 2005

Gaukler posted:

Thanks, this makes a lot of sense. What's your take on the Python web framework space? Is it pretty much Django or nothing? I'm putzing around with a personal project that has a fairly small core feature set but has a fairly long 'nice to have' list and I've been trying to test out various frameworks to see what they're like when programming actual features.

My personal preference is Pyramid. It can be extremely similar to Flask, but entirely avoids using context locals (unless you really, really need them).

# ? Apr 18, 2013 21:03

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Steampunk Hitler posted:

Don't actually do this. Just use self and suck it up. It's a de facto standard.

Yeah. If you don't use self, you're a bad person.

# ? Apr 18, 2013 21:07

accipter: Sep 12, 2003

Thermopyle posted:

Yeah. If you don't use self, you're a bad person.

So why did you suggest it?

# ? Apr 18, 2013 21:12

Modern Pragmatist: Aug 20, 2008

accipter posted:

So why did you suggest it?

I'm guessing his most recent post is sarcasm.

Also, it's just as much a readability thing as anything. By using self, someone unfamiliar with your code can easily tell that you are accessing a property or method defined in your class rather than one defined elsewhere.

# ? Apr 18, 2013 21:17

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

accipter posted:

So why did you suggest it?

My append_smilies_if_sarcastic_intent_is_not_clear() method has a bug in it.

# ? Apr 18, 2013 21:44

accipter: Sep 12, 2003

Thermopyle posted:

My append_smilies_if_sarcastic_intent_is_not_clear() method has a bug in it.

You should consider unit testing.

# ? Apr 18, 2013 21:46

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

accipter posted:

You should consider unit testing.

unit testing is for the weak.

# ? Apr 18, 2013 21:57

Dominoes: Sep 20, 2007

I finished my first program! I'm having trouble making a binary using Python 3 and cx_freeze. The binary runs on my computer, but on another computer I'm testing it on, I get the message "Can't find MSCVR100.dll". I looked this up, and it appears to be a visual studio C++ file.

I located the DLLin my Windows/sysWOW64 directory, and moved it into my application directory on the computer with the error. (Both are Win 8 64-bit) I now receive an error 0xc000007b, which appears to be a 64-bit compatibility error. This website describes receieving these two errors in sequence, presumably using C++. He found the fix to be installing Visual Studio. Doesn't make sense.

Very little of the information available on either the DLL or the 07b error mentions python or cx_freeze, which is why I'm stumped. The program does use QT, which I feel may have something to do with it. Any ideas?

Dominoes fucked around with this message at 04:27 on Apr 19, 2013

# ? Apr 19, 2013 02:07

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Install the Visual Studio 2010 redistributable package.

# ? Apr 19, 2013 02:46

Dominoes: Sep 20, 2007

Plorkyeran posted:

Install the Visual Studio 2010 redistributable package.

I'm making a binary because I'd like to distribute this program to coworkers. I'd prefer the program to just work. There's got to be something I can add to my setup.py, or a way to get the proper .dll manually into the program's folder.

# ? Apr 19, 2013 02:58

QuarkJets: Sep 8, 2008

Dominoes posted:

I'm making a binary because I'd like to distribute this program to coworkers. I'd prefer the program to just work. There's got to be something I can add to my setup.py, or a way to get the proper .dll manually into the program's folder.

Do you have the Visual Studio 2010 redistributable package on the computer that you're using to create the binary?

# ? Apr 19, 2013 04:19

Dominoes: Sep 20, 2007

QuarkJets posted:

Do you have the Visual Studio 2010 redistributable package on the computer that you're using to create the binary?

No, but I have various 'Microsoft Visual C++ (year), x86 and x64 Redistrbutables' under the uninstall-a-program dialog.

# ? Apr 19, 2013 04:29

QuarkJets: Sep 8, 2008

Dominoes posted:

No, but I have various 'Microsoft Visual C++ (year), x86 and x64 Redistrbutables' under the uninstall-a-program dialog.

Check if there are any keys in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\VC\VCRedist\

And if you don't have it installed, then you might try installing it on the machine that you're using to create the executable; at some point the Visual Studio compiler is getting called (since you're on Windows). It's not totally clear to me how cx_freeze works in Windows

# ? Apr 19, 2013 11:07

evensevenone: May 12, 2001; Glass is a solid.

Thats the runtime library DLL. They're either going to have to download it themselves or you're going to have to use an installer. It should be a fairly painless download.

# ? Apr 19, 2013 11:54

Dominoes: Sep 20, 2007

The computer where the program's not working has an x86 visual c++ runtime, but not x64. It has mscrv100.dll in the syswow64 folder, but not the sys32. If I remove the .dll I added to the program folder, I get the missing .dll error. If I move it to sys32, I get the 000007b error.

This is a pretty simple program; there's got to be a better way. Whenever I'm thinking about installing a new program, and it makes me install some weird package, I get annoyed and consider not bothering. I don't want to program that way. I'm surprised more about this doesn't come up in searches about qt, cx_freeze etc; it seems like something lots of people would run into.

Does wx_widgets have this problem?

Dominoes fucked around with this message at 13:36 on Apr 19, 2013

# ? Apr 19, 2013 13:24

Modern Pragmatist: Aug 20, 2008

Dominoes posted:

The computer where the program's not working has an x86 visual c++ runtime, but not x64. It has mscrv100.dll in the syswow64 folder, but not the sys32. If I remove the .dll I added to the program folder, I get the missing .dll error. If I move it to sys32, I get the 000007b error.

This is a pretty simple program; there's got to be a better way. Whenever I'm thinking about installing a new program, and it makes me install some weird package, I get annoyed and consider not bothering. I don't want to program that way. I'm surprised more about this doesn't come up in searches about qt, cx_freeze etc; it seems like something lots of people would run into.

Does wx_widgets have this problem?

Any standalone executable you create for windows relies upon the c++ runtime regardless of which libraries you use. The runtime is required for the python interpreter that is included with the executable.

Have you read the page on this in the documentation?

Also you need to make sure that if you create a 64-bit executable then you need to have the 64-bit redistributable.

# ? Apr 19, 2013 15:44

accipter: Sep 12, 2003

Modern Pragmatist posted:

Any standalone executable you create for windows relies upon the c++ runtime regardless of which libraries you use. The runtime is required for the python interpreter that is included with the executable.

Have you read the page on this in the documentation?

Also you need to make sure that if you create a 64-bit executable then you need to have the 64-bit redistributable.

It seems like underscores break URL tags...

code:

[url]http://cx_freeze.readthedocs.org/en/latest/overview.html#microsoft-visual-c-2008-redistributable-package[/url]

# ? Apr 19, 2013 17:27

QuarkJets: Sep 8, 2008

I am using paramiko to sftp get a large file from a remote server.

code:

 
ssh = paramiko.SSHClient() 
ssh.load_system_host_keys() 
ssh.connect(ip_addr) 
sftp = ssh.open_sftp()
sftp.get('/tmp/test.txt', '/home/user/test.txt', callbackfunction)

The implementation works great, but keyboard interrupts cause weird behavior. One of two things happen:

A) Paramiko becomes a zombie and I have to kill the python process by hand

B) Paramiko catches the keyboard interrupt and raises a paramiko.SFTPError with the text "Garbage packet received."

Has anyone else had this experience? Can I somehow prevent A) from happening?

# ? Apr 19, 2013 20:58

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

I use fabric which wraps paramiko in a nicer interface imo

# ? Apr 19, 2013 23:19

Dominoes: Sep 20, 2007

Modern Pragmatist posted:

Any standalone executable you create for windows relies upon the c++ runtime regardless of which libraries you use. The runtime is required for the python interpreter that is included with the executable.

Have you read the page on this in the documentation?

Also you need to make sure that if you create a 64-bit executable then you need to have the 64-bit redistributable.

accipter posted:

It seems like underscores break URL tags...

code:

[url]http://cx_freeze.readthedocs.org/en/latest/overview.html#microsoft-visual-c-2008-redistributable-package[/url]

Thanks all! That page's explanation's what I was looking for. Installing C++ runtime 2010 x64 on the computer where the program was broken allows it to run. The page you linked has the file names for the 2008 version. Once I find the 2010 equivalents, this should work.

edit: Solved! The required files are msvcr100.dll and msvcp100.dll from the Sys32 directory.

Dominoes fucked around with this message at 17:02 on Apr 20, 2013

# ? Apr 20, 2013 00:23

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 23:54

the: Jul 18, 2004; by Cowcaster

I'm running a sci/numerical python program in Canopy right now It takes about 4 hours to run. Afterwards I need to manipulate the data in the console (like check specific values and whatnot). Just a bunch of 41x41x41 arrays and such.

Is there any way I can *save* all the computed data so I can just instantly load it up again to work with it? That would save me literal hours of work in having to run this program again if I have to shut down my computer for some reason.

the fucked around with this message at 22:13 on Apr 20, 2013

# ? Apr 20, 2013 22:09

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »