Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

opt: Oct 10, 2007

Suspicious Dish posted:

No, because you should always use absolute imports. You need to ensure that your PYTHONPATH is set correctly.

This was it, thanks.

Red Mike posted:

I suggest using something like nose for running your unit tests. It should set the PYTHONPATH itself, and find your tests itself without any issues when run from the base folder "rl".

I'm gonna check this out, thanks.

# ? Jan 5, 2013 19:01

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 12:58

Clanpot Shake: Aug 10, 2006; shake shake!

I found a code snippet online that I was hoping someone could explain.

code:

def dictinv(d):
    inv={}
    for k,v in d.items():		# iteritems() for Python 2.7
            keys=inv.setdefault(v,[])
            keys.append(k)
    return inv

Here's how it behaves:

code:

>>> a = {1:'a', 2:'a',3:'b',4:'c',5:'a'}
>>> dictinv(a)
{'a': [1, 2, 5], 'c': [4], 'b': [3]}

This does exactly what I was looking for, but I don't understand exactly how it works. The function declares 'inv', but never appears to assign it, and the variable 'keys' is never applied to anything. What exactly is happening here?

fake edit: Pressing tab inside code tags adds spaces, outside it focuses the submit button. That is awesome.

# ? Jan 8, 2013 21:45

Bunny Cuddlin: Dec 12, 2004

Clanpot Shake posted:

I found a code snippet online that I was hoping someone could explain.
code:
def dictinv(d):
    inv={}
    for k,v in d.items():		# iteritems() for Python 2.7
            keys=inv.setdefault(v,[])
            keys.append(k)
    return inv
Here's how it behaves:
code:
>>> a = {1:'a', 2:'a',3:'b',4:'c',5:'a'}
>>> dictinv(a)
{'a': [1, 2, 5], 'c': [4], 'b': [3]}
This does exactly what I was looking for, but I don't understand exactly how it works. The function declares 'inv', but never appears to assign it, and the variable 'keys' is never applied to anything. What exactly is happening here?

fake edit: Pressing tab inside code tags adds spaces, outside it focuses the submit button. That is awesome.

pre:

setdefault(key[, default])�
If key is in the dictionary, return its value. If not, insert key with a value of default and return default. default defaults to None.

from http://docs.python.org/2/library/stdtypes.html

so, for each key-value pair, get the list at the value's location. If it doesn't have a list, insert an empty list. Then, insert the key into that list and move on to the next one.

# ? Jan 8, 2013 21:52

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Clanpot Shake posted:

The function declares 'inv', but never appears to assign it

Python doesn't have declarations. It assigns it right at the top, there.

# ? Jan 9, 2013 02:57

The Gripper: Sep 14, 2004; i am winner

Suspicious Dish posted:

Python doesn't have declarations. It assigns it right at the top, there.

I think he means there's nothing writing values into inv, which is easy to assume because of how it's using setdefault and a list default:

code:

>>> x = {}
>>> k=x.setdefault("a",[])
>>> x
{'a': []}
>>> k.append("words")
>>> x
{'a': ['words']}
>>>

setdefault(k,d) returns the key, and since it's a list it's mutable so changing that returned value affects the original dict value. If it was an integer you wouldn't have the same result:

code:

>>> y = {}
>>> k = y.setdefault("a",1)
>>> y
{'a': 1}
>>> k=2
>>> y
{'a': 1}
>>>

The Gripper fucked around with this message at 03:27 on Jan 9, 2013

# ? Jan 9, 2013 03:21

Clanpot Shake: Aug 10, 2006; shake shake!

The Gripper posted:

I think he means there's nothing writing values into inv, which is easy to assume because of how it's using setdefault and a list default:
code:
>>> x = {}
>>> k=x.setdefault("a",[])
>>> x
{'a': []}
>>> k.append("words")
>>> x
{'a': ['words']}
>>>
setdefault(k,d) returns the key, and since it's a list it's mutable so changing that returned value affects the original dict value. If it was an integer you wouldn't have the same result:
code:
>>> y = {}
>>> k = y.setdefault("a",1)
>>> y
{'a': 1}
>>> k=2
>>> y
{'a': 1}
>>>

Thank you, this is very helpful. What is this kind of implicit modification called (appending to keys to modify inv)? And how do I know when a function returns something that can be modified in that way?

# ? Jan 9, 2013 14:58

The Gripper: Sep 14, 2004; i am winner

Clanpot Shake posted:

Thank you, this is very helpful. What is this kind of implicit modification called (appending to keys to modify inv)? And how do I know when a function returns something that can be modified in that way?

It's nothing special, just plain old mutability of types.

When you assign to a variable it doesn't make a copy of the value, it makes a reference to it. If you make a change to a mutable type it will change in-place (and affect every variable referencing it). If you make a change to an immutable type it will create a new object of that type with the new value and assign a reference to it, so it won't affect any old references.

I don't know the full list of what is mutable/immutable off the top of my head, but lists, dicts, and sets are the most common mutable ones you'll come across and most other things are immutable (strings, integers etc.)

This page can probably help more with it.

(this is the reason why using lists and dicts as default arguments in functions is a bad idea)

# ? Jan 9, 2013 15:30

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#other-languages-have-variables

This page has more information on names and binding.

# ? Jan 9, 2013 16:09

Posting Principle: Dec 10, 2011; by Ralp

Anyone have any experience with Kivy? Just wondering about speed, portability, and whether it gets in your face with Cython, or mostly just sticks to Python.

# ? Jan 9, 2013 23:50

duck monster: Dec 15, 2004

Jerry SanDisky posted:

Anyone have any experience with Kivy? Just wondering about speed, portability, and whether it gets in your face with Cython, or mostly just sticks to Python.

I've been using it a bit. It produces great apps, but its a little bit young. The cython thing you can safely ignore, I do.

That said, despite some of it being a bit young and rough around the edges, its supremely well written code and thus very easy to hack on when things dont quite work as expected, and the devs are quite happy to accept patches. I strongly recomend working out of its git repository rather than the prebuilts, just because its bit of a moving target at the moment, and in such cases ,it pays to keep up.

The documentation could definately use some work though. Its a giant framework, and not all of it is well documented, and requires a degree of browsing examples and source.

# ? Jan 10, 2013 15:05

The Gripper: Sep 14, 2004; i am winner

I played around with Kivy on iPad2 when it came out officially mid last year and had no problems, I assume it's only gone forward from there. Mind you these were very simple isolated data entry applications, so I wasn't pushing it to it's limits by any means.

It actually came in very handy as a prototyping tool for iPad apps: someone suggests a "useful" application and I'd just throw together something with Kivy, trial it on a few users and see how they respond. Even rushed, poorly thought out designs ran well.

# ? Jan 10, 2013 15:32

Victor: Jun 18, 2004

I'm looking into using Pyramid (formerly Pylons) and I'm looking for some code bases to look at that would give me a good idea of how it is meant to be used. (I know it's lower-level than Django and thus there is more than one way in which it is meant to be used.) Is there, for example, a good example of a simple set of forums, written with Pyramid? Something using backbone.js would be a bonus.

Edit: hmm, perhaps the SQLAlchemy + URL Dispatch Wiki Tutorial will suffice. Other suggestions are welcome, though.

Victor fucked around with this message at 00:32 on Jan 11, 2013

# ? Jan 11, 2013 00:09

duck monster: Dec 15, 2004

The Gripper posted:

I played around with Kivy on iPad2 when it came out officially mid last year and had no problems, I assume it's only gone forward from there. Mind you these were very simple isolated data entry applications, so I wasn't pushing it to it's limits by any means.

It actually came in very handy as a prototyping tool for iPad apps: someone suggests a "useful" application and I'd just throw together something with Kivy, trial it on a few users and see how they respond. Even rushed, poorly thought out designs ran well.

The .kv stuff is very very nice to prototype with. It sort of reminds me a bit of QTs declarative stuff, but a little less faux-css and a bit more actual widgets.

Its very important to pay close attention to the widget lifecycle I've found, as some of the more advanced operations require manipulating it at the canvas level which has impacts on resize/redraws, however the mechanism for dealing with this is quite straight forward so its no big deal.

I like it a lot, I just think its documentation could improve a bit. But then again I also think Djangos documentation is poo poo too, and almost nobody agrees with me on that, so take that as you will. e: Oh and someone really needs to go through and update all the info on coordinates with stuff about whether shits working in local or global coordinate spaces, I still havent fully divined the rules for that.

v :shobon:

v

Incidently, looks like Boa Constructor has been forked and is being worked on again, finally.

duck monster fucked around with this message at 09:23 on Jan 11, 2013

# ? Jan 11, 2013 01:31

Haystack: Jan 23, 2005

Victor posted:

I'm looking into using Pyramid (formerly Pylons) and I'm looking for some code bases to look at that would give me a good idea of how it is meant to be used. (I know it's lower-level than Django and thus there is more than one way in which it is meant to be used.) Is there, for example, a good example of a simple set of forums, written with Pyramid? Something using backbone.js would be a bonus.

Edit: hmm, perhaps the SQLAlchemy + URL Dispatch Wiki Tutorial will suffice. Other suggestions are welcome, though.

Pyramid's documentation lists a few sample applications, and has an easy to overlook tutorial that is somewhat more comprehensive than the wiki tutorials. The example projects are a touch old, but Shootout at least should be informative.

# ? Jan 11, 2013 06:01

Pythagoras a trois: Feb 19, 2004; I have a lot of points to make and I will make them later.

I'm using webapp2 to build, well, a web app, and I want to start abstracting a bunch of handlers into separate files to clean up the code and make it more manageable. But I'm at a loss of how to do this well, as I don't understand the python import and inheritance rules terribly well. For instance:

code:

import webapp2

class BaseHandler(webapp2.RequestHandler):
    def render(self, tempalte, **kw):
        //bunch of quality of life improvements over the standard webapp2.requestHandler)



class MyFrontPageCode(BaseHandler):
    def get(self):
        //all my page specfic code
        self.render(template)

class MyLoginPageCode(BaseHandler):
    def get(self):
        //all my page specfic code
        self.render(template)

class 3(BaseHandler):

Is it best operating procedure to make the main file call 'import MyFrontPageCode \n import MyLoginPageCode' and put those in separate .py files? To inherit the BaseHandler class, I need to start all of those files with 'from webapp2 import BaseHandler', which is already feeling like trading one overly crowded source file for multiple redundancies spread across a number of files.

I think there's a larger point to be made about using python as my first programming language, because of how easy it is to make working programs that are both powerful and poorly written.... but I can't seem to phrase it correctly.

# ? Jan 13, 2013 20:46

geera: May 20, 2003

I've been toying with Flask a little bit to try and branch out from Django, especially for smaller projects. One thing I haven't found a very good article or resource on is how to handle making changes to the database schema during development (or even in production). I'm probably hoping for too much to find a South-like project for Flask, but the alternative seems to be to use SQLAlchemy-migrate, which probably works great but is a bit much for me to learn while I'm just getting started. What do you all do in this situation?

# ? Jan 14, 2013 17:56

Comrade Gritty: Sep 19, 2011; This Machine Kills Fascists

geera posted:

I've been toying with Flask a little bit to try and branch out from Django, especially for smaller projects. One thing I haven't found a very good article or resource on is how to handle making changes to the database schema during development (or even in production). I'm probably hoping for too much to find a South-like project for Flask, but the alternative seems to be to use SQLAlchemy-migrate, which probably works great but is a bit much for me to learn while I'm just getting started. What do you all do in this situation?

I don't know if there's a Flask app for it, but Alembic is by the author of SQLAlchemy and i've liked it in the past.

# ? Jan 14, 2013 23:40

raminasi: Jan 25, 2005; a last drink with no ice

Is there a clever way to convert a list of 1d numpy arrays into a 2d numpy array? I thought that just using the array constructor would work but it creates a 1d numpy array of 1d numpy arrays.

e: The problem appears to be caused by the array lengths not matching when I thought they did.

raminasi fucked around with this message at 21:35 on Jan 16, 2013

# ? Jan 16, 2013 21:21

accipter: Sep 12, 2003

GrumpyDoctor posted:

Is there a clever way to convert a list of 1d numpy arrays into a 2d numpy array? I thought that just using the array constructor would work but it creates a 1d numpy array of 1d numpy arrays.

e: The problem appears to be caused by the array lengths not matching when I thought they did.

Sounds like you figured it out, but there is numpy._r[..] and numpy._c[..] for concatenation by row and by column.

# ? Jan 16, 2013 22:18

Ashex: Jun 25, 2007; These pipes are cleeeean!!!

I'm developing a script to automate software installs and decided to use an ini for storing settings and stuff.

Is there a better way to get the settings then what I'm doing? I'm using ConfigParser in python 2.7

code:

    dbhost = parser.get('tssmssql', 'dbhost')
    dbadminuser = parser.get('tssmssql', 'dbadminpass')
    dbadminpass = parser.get('tssmssql', 'dbadminuser')
    dbname = parser.get('tssmssql', 'dbname')
    dbuser = parser.get('tssmssql', 'dbuser')
    dbpass = parser.get('tssmssql', 'dbuser')
    dbdemodata = parser.get('tssmssql', 'dbdemodata')
    dbdemoname = parser.get('tssmssql', 'dbdemoname')
    dbdemouser = parser.get('tssmssql', 'dbdemouser')
    dbdemopass = parser.get('tssmssql', 'dbdemopass')

There's going to be multiple sections where I do almost exactly that. In the section I pulled the snippet from I'm running sqlcmd with those above values passed to it as args.

# ? Jan 22, 2013 21:37

The Gripper: Sep 14, 2004; i am winner

Ashex posted:

I'm developing a script to automate software installs and decided to use an ini for storing settings and stuff.

Is there a better way to get the settings then what I'm doing? I'm using ConfigParser in python 2.7

There's going to be multiple sections where I do almost exactly that. In the section I pulled the snippet from I'm running sqlcmd with those above values passed to it as args.

You could do tssmssql = dict(parser.items('tssmssql')) and then just read tssmssql['dbhost'] etc. to cut out a lot of the repetition.

# ? Jan 22, 2013 21:42

FoiledAgain: May 6, 2007

Not sure if this is the right thread for my question, but I'm working in Python so I'll start here.
I'm having trouble printing unicode in the command prompt window. I first got this UnicodeEncodeError: 'charmap' codec can't encode character u'\u0294' in position 15: character maps to <undefined>. Some googling suggested I type chcp 65001 in the prompt, which appeared to work since I got a confirmation message, but then when I ran the script I got LookupError: unknown encoding cp65001. Suggestions?

edit: to clarify, everything works fine when I print to the interactive window in PyScripter, and the same stuff print to file just fine. The problem is printing to screen in the command prompt.

FoiledAgain fucked around with this message at 23:02 on Jan 22, 2013

# ? Jan 22, 2013 22:22

Ashex: Jun 25, 2007; These pipes are cleeeean!!!

The Gripper posted:

You could do tssmssql = dict(parser.items('tssmssql')) and then just read tssmssql['dbhost'] etc. to cut out a lot of the repetition.

Thanks! That's exactly what I was looking for. I found an example for python 3.2 which didn't work, dict is what I was missing.

# ? Jan 22, 2013 22:57

The Gripper: Sep 14, 2004; i am winner

Has anyone managed to get Kivy working properly with pyinstaller on Windows? It's pretty much the only thing stopping me from moving some important cross-platform products to it at the moment, since I need an easy-to-distribute package for it.

The documentation for it (pyinstaller packaging) is absolute trash, the maintainer uses OS X exclusively and seems to have no idea what he's doing with regards to Windows despite committing things with messages that say otherwise.

I've spent a few hours dicking around with it and it never seems to package up everything it needs, at least in the final stage. The intermediary collection stage where it puts all dependencies and binaries in one place *seems* to work but I can't tell if it's touching things outside that directory as well.

Seems to be PIL causing problems, and I can't figure out why (other people have reported it with no solution).

# ? Jan 23, 2013 17:15

Chosen: Jul 11, 2002; Vibrates when provoked.

The Gripper posted:

Has anyone managed to get Kivy working properly with pyinstaller on Windows? It's pretty much the only thing stopping me from moving some important cross-platform products to it at the moment, since I need an easy-to-distribute package for it.

The documentation for it (pyinstaller packaging) is absolute trash, the maintainer uses OS X exclusively and seems to have no idea what he's doing with regards to Windows despite committing things with messages that say otherwise.

I've spent a few hours dicking around with it and it never seems to package up everything it needs, at least in the final stage. The intermediary collection stage where it puts all dependencies and binaries in one place *seems* to work but I can't tell if it's touching things outside that directory as well.

Seems to be PIL causing problems, and I can't figure out why (other people have reported it with no solution).

God, I've been having the same issues. One thing I've been meaning to try (but haven't yet) is using a nice portable kivy distribution for packaging, kept separate from my development python environment. This thread gives me hope, though. The group is pretty active.

# ? Jan 23, 2013 18:29

The Gripper: Sep 14, 2004; i am winner

Chosen posted:

God, I've been having the same issues. One thing I've been meaning to try (but haven't yet) is using a nice portable kivy distribution for packaging, kept separate from my development python environment. This thread gives me hope, though. The group is pretty active.

I was tossing up on doing that too, but the portable build looks to only be for Windows and ideally I'd like one solution that can target Win/Linux/OSX without change.

I was holding out hope that I'd crack the pyinstaller case today, but just now had an issue where pyinstaller generated the right directory and file structure but I could only run the result from the MinGW kivy session. As soon as I tried it on it's own it threw errors about missing DLLs with no stack trace.

Annoyingly it happened to only one of 3 test projects, so I'm not even sure where to start troubleshoooting now!

# ? Jan 23, 2013 18:48

Ashex: Jun 25, 2007; These pipes are cleeeean!!!

Another beginner question for you guys.

I'm attempting to use variables inside my config file but keep getting an interpolation error. This is what I have in the config:

code:

[global]
tss_host=banana
tss_port=8090
tss_url=http://%(tss_host):%(tss_port)

Is this not valid? I'm using SafeConfigParser and according to the documentation it's supported.

# ? Jan 24, 2013 00:50

accipter: Sep 12, 2003

Ashex posted:

Another beginner question for you guys.

I'm attempting to use variables inside my config file but keep getting an interpolation error. This is what I have in the config:
code:
[global]
tss_host=banana
tss_port=8090
tss_url=http://%(tss_host):%(tss_port)
Is this not valid? I'm using SafeConfigParser and according to the documentation it's supported.

I think you need to be using

code:

%()s

# ? Jan 24, 2013 01:05

Ashex: Jun 25, 2007; These pipes are cleeeean!!!

accipter posted:

I think you need to be using
code:
%()s

Well, I'm an idiot. Thanks!

# ? Jan 24, 2013 08:05

Ireland Sucks: May 16, 2004

Im trying to draw a png with PIL (the 3.2 version here http://www.lfd.uci.edu/~gohlke/pythonlibs/ since its the only version I can find for 3) and it uses stupid amounts of memory. Drawing a 37kb image with putpixel sends usage from 32mb to 700+ and the 5400x1200 one im drawing makes python kill itself at 2GB memory before it gets to 20% complete

What else can I use?

# ? Jan 26, 2013 08:47

BigRedDot: Mar 6, 2008

Probably better off creating an RGBA image with numpy, then PIL has Image.frombuffer for converting an array in one go. Image.putpixel is probably the slowest, least efficient method you could use. Alternatively you could try using the pure python PyPNG library to convert the array, but I have no personal experience with it.
.

# ? Jan 26, 2013 09:13

Ireland Sucks: May 16, 2004

BigRedDot posted:

Probably better off creating an RGBA image with numpy, then PIL has Image.frombuffer for converting an array in one go. Image.putpixel is probably the slowest, least efficient method you could use.

That works so much better, thankyou!

# ? Jan 26, 2013 10:32

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Does anyone have a recommendation for an IRC library that works in Python 3? I really like this library for how simple it is to get something working, but it doesn't support Python 3. I found that it is possible to get it to work in Python 3 by modifying it, but something similar that is supported might be nice.

# ? Jan 27, 2013 00:34

The Gripper: Sep 14, 2004; i am winner

Hammerite posted:

Does anyone have a recommendation for an IRC library that works in Python 3? I really like this library for how simple it is to get something working, but it doesn't support Python 3. I found that it is possible to get it to work in Python 3 by modifying it, but something similar that is supported might be nice.

I've always just used irclib, which claims to be Python 3 supported (though I've only used it with 2.7). Installs properly from pip (it's called "irc" rather than irclib though).

https://bitbucket.org/jaraco/irc

# ? Jan 27, 2013 00:45

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

The Gripper posted:

I've always just used irclib, which claims to be Python 3 supported (though I've only used it with 2.7). Installs properly from pip (it's called "irc" rather than irclib though).

https://bitbucket.org/jaraco/irc

I recall looking at this, and being turned off by how complex the example bot's code looks. It seems extremely involved just for making a simple robot that responds to messages in a channel, though maybe if you need to do a lot of advanced stuff that complexity is needed. In comparison with the one I linked, you can get going just by writing methods called on_channel_message and on_private_message, which makes things exceedingly simple.

# ? Jan 27, 2013 02:46

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

I've used Twisted for IRC bots before, but it doesn't yet support Python 3.

I guess I don't see why you can't just install Python 2.

# ? Jan 27, 2013 03:08

The Gripper: Sep 14, 2004; i am winner

Hammerite posted:

I recall looking at this, and being turned off by how complex the example bot's code looks. It seems extremely involved just for making a simple robot that responds to messages in a channel, though maybe if you need to do a lot of advanced stuff that complexity is needed. In comparison with the one I linked, you can get going just by writing methods called on_channel_message and on_private_message, which makes things exceedingly simple.

I can't vouch for the stability of this or whether it covers all bases, but I quickly threw together a Python3-fixed version of ircutils here. It's using UTF-8 for most things with latin1 fallback when necessary (testing on synirc with just UTF-8 throws errors because some poo poo in their MOTD uses continuation bytes or some other garbage, I don't know because I'm bad at Unicode).

It seems to work perfectly on this echobot example with varying non-ascii character sets.

quote:

[13:37:59] <face> 博客- 玉龙的技术博客- 开源中国社区
[13:38:01] <boneroo> 博客- 玉龙的技术博客- 开源中国社区

Honestly though, irclib isn't much more verbose and if you create a base bot that does the hard work from the example (handling server joins, nick collision) you can just subclass that and implement the same as ircutils with almost identical code.

# ? Jan 27, 2013 03:48

how!!: Nov 19, 2011; by angerbot

Hammerite posted:

I recall looking at this, and being turned off by how complex the example bot's code looks. It seems extremely involved just for making a simple robot that responds to messages in a channel, though maybe if you need to do a lot of advanced stuff that complexity is needed. In comparison with the one I linked, you can get going just by writing methods called on_channel_message and on_private_message, which makes things exceedingly simple.

You can use the irc controller from this: https://github.com/priestc/giotto

It uses irclib (someone in this thread wrote the code), and it supports python3.

Write your function, plug it into the manifest, configure the controller file with your irc server name/channel/etc, then execute the controller. Viola, instant irc bot.

# ? Jan 27, 2013 03:58

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

how!! posted:

You can use the irc controller from this: https://github.com/priestc/giotto

Trap sprung.

# ? Jan 27, 2013 04:31

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 12:58

tef: May 30, 2004; -> some l-system crap ->

Hammerite posted:

Does anyone have a recommendation for an IRC library that works in Python 3? I really like this library for how simple it is to get something working, but it doesn't support Python 3. I found that it is possible to get it to work in Python 3 by modifying it, but something similar that is supported might be nice.

Comedy option: Port ircutils to python3; 2to3 might get you most of the way too.

# ? Jan 27, 2013 09:20

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »