Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
m0nk3yz
Mar 13, 2002

Behold the power of cheese!

Avenging Dentist posted:

Here you go. I did it in a single statement if you don't count the imports (EDIT: who needs import when you have __import__) (EDITx2: wait I can make multiline lambdas a lot simpler):

code:
print("Heads came up {0} times.\nTails came up {1} times.".format(*__import__('functools').reduce(
lambda T, x: (print(['Heads!', 'Tails!'][x]), [T[0] + int(x==0), T[1] + int(x==1)])[-1],
(__import__('random').randint(0, 1) for _ in range(10)), [0, 0])))
Indented: http://pastebin.com/cfBXNSsZ

Wow, just wow. I need to find some way of using this in real code, just to be a dick.

Adbot
ADBOT LOVES YOU

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

m0nk3yz posted:

Wow, just wow. I need to find some way of using this in real code, just to be a dick.

That would be awesome. Of course, if I read it in some real code, I would track you down and stab you in the eyeball.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

m0nk3yz posted:

Wow, just wow. I need to find some way of using this in real code, just to be a dick.

I could actually see someone use multi-expression lambdas in some ill-considered plan to be "concise". The syntax is almost sensible.

Yay
Aug 4, 2007
I totally hadn't noticed the counting at the end; talk about glossing over an import part :)

That is some insanely clever code, mr. Avenging Dentist.

Myself, I was thinking of something along the line of (now with counting?):

code:
from random import randint
result = '%s came up %d times' # cos I ended up using it twice.
rolls = [] # because I need a counter :S
for _ in xrange(0, 100):
    # suppose I should print heads or tails here ... le sigh.
    roll = randint(0, 1)
    print 'Heads' if roll == 1 else 'Tails'
    rolls.append(roll)

print result % ('heads', sum(rolls))
print result % ('tails', len(rolls) - sum(rolls))

Yay fucked around with this message at 11:58 on May 20, 2010

Mr. Clark2
Sep 17, 2003

Rocco sez: Oh man, what a bummer. Woof.

AzraelNewtype posted:

The added comments above are important. Yay asked you before, but you left that out of what you replied to. I'm honestly curious what you think is going on here, and it's not really worth explaining what is happening until you answer.


I can see now that those two lines did absolutely nothing. At the time, I wrongly thought that I needed them in there to 'flip' the coin again. After doing some more reading and experimenting, I see that they are not necessary, and that 'flip' at the beginning of the loop takes care of that. I throw myself upon the mercy of the python gods... :D

Benji the Blade
Jun 22, 2004
Plate of shrimp.

Mr. Clark2 posted:

I throw myself upon the mercy of the python gods... :D

I know that it can sometimes be difficult to determine intent on the internet, but people are not pressing this issue to make you feel bad or to make fun of you or revel in our superiority. We're pressing you on this to try to figure out what misconception led you to believe that that line would do something so that we can help you learn what's actually going on. There's nothing wrong with not understanding.

Mr. Clark2
Sep 17, 2003

Rocco sez: Oh man, what a bummer. Woof.

Benji the Blade posted:

I know that it can sometimes be difficult to determine intent on the internet, but people are not pressing this issue to make you feel bad or to make fun of you or revel in our superiority. We're pressing you on this to try to figure out what misconception led you to believe that that line would do something so that we can help you learn what's actually going on. There's nothing wrong with not understanding.

Point taken. I welcome and and all constructive criticisms :D

Unboxing Day
Nov 4, 2003

I have come across a library written in C, and I want to use Python's ctypes to access it. However, I'm currently stuck at attempting to pass an array of structs to one of the functions inside the library, and to paste the entire codebase would be a little much, so I constructed a short little example:

My test library:

code:
    #include <stdio.h>

    #define MAXADR 22

    typedef struct {
      int gmflag;
      char address[MAXADR + 1];
    } ServerData;

    void test(ServerData *svi[], int svnumb);

    void test(ServerData *svi[], int svnumb) {
      int i = 0;
      for (i = 0;i < svnumb;i++) {
        printf("gmflag: %d\naddress: %s\n", svi[i]->gmflag, svi[i]->address);
      }
    }
Compiled it with `gcc -Wall -g -fPIC -shared -o test.so test.c`

My test python program:

code:
    import ctypes

    MAXADR = 22

    class ServerData(ctypes.Structure):
        _fields_ = [
            ('gmflag', ctypes.c_int),
            ('address', ctypes.c_char * (MAXADR + 1)),
            ]

    testlib = ctypes.CDLL("./test.so")

    sd_array_type = ServerData * 2
    sd_array = sd_array_type()

    sd_array[0].gmflag = 0
    sd_array[0].address = "123.123.123.4:1000"
    sd_array[1].gmflag = 1
    sd_array[1].address = "123.123.123.4:2000"

    testlib.test(sd_array, len(sd_array))
I then ran this program using `python test.py`, and I got a segmentation fault. I then used gdb to try and figure out what was going on:

code:
    (gdb) run
    Starting program: /usr/bin/python test.py
    [Thread debugging using libthread_db enabled]

    Program received signal SIGSEGV, Segmentation fault.
    0x00007ffff659a63f in test (svi=0x946fb0, svnumb=2) at test.c:15
    15          printf("gmflag: %d\naddress: %s\n", svi[i]->gmflag, svi[i]->address
    );
    (gdb) print i
    $1 = 0
    (gdb) print svi
    $2 = (ServerData **) 0x946fb0
    (gdb) print svi[0]
    $3 = (ServerData *) 0x2e33323100000000
    (gdb) print svi[0]->gmflag
    Cannot access memory at address 0x2e33323100000000
So my array of structs is being passed...just not any data. How can I fix this?

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Emo Businessman posted:

:words:

Well I'm going to (again!) suggest cython instead of ctypes; it's much less effort for much higher-quality bindings.

code:
# test.pyx

cdef extern from "test.h":
    int MAXADR
    ctypedef struct ServerData:
        int gmflag
        char address[1]
    
    void test(ServerData **, int)

cdef extern from "stdlib.h":
    ctypedef int size_t
    void *calloc(size_t, size_t)
    void free(void *)

cdef extern from "string.h":
    char *strncpy(char *, char *, size_t)

def do_test(items):
    cdef Py_ssize_t n_items = len(items)
    cdef ServerData *data = <ServerData *>calloc(n_items, sizeof(ServerData))
    cdef ServerData **array = <ServerData **>calloc(n_items, sizeof(ServerData *))
    cdef int i, gmflag
    for i, (address, gmflag) in enumerate(items):
        strncpy(data[i].address, address, MAXADR)
        data[i].gmflag = gmflag
        array[i] = &data[i]
    test(array, n_items)
code:
>>> import test
>>> test.do_test([
...     ('123.123.123.4:1000', 0),
...     ('123.123.123.4:2000', 1),
... ])
gmflag: 0
address: 123.123.123.4:1000
gmflag: 1
address: 123.123.123.4:2000
(If you really hate yourself and want to use ctypes: count the number of dereferences your C code is doing vs. the types you have declared in ctypes.)

e: whoops, I realized I didn't check the return value of the callocs. Proper error handling is left as an exercise to the reader.

Habnabit fucked around with this message at 18:36 on May 24, 2010

Scaevolus
Apr 16, 2007

Avenging Dentist posted:

Here you go. I did it in a single statement if you don't count the imports (EDIT: who needs import when you have __import__) (EDITx2: wait I can make multiline lambdas a lot simpler):

code:
print("Heads came up {0} times.\nTails came up {1} times.".format(*__import__('functools').reduce(
lambda T, x: (print(['Heads!', 'Tails!'][x]), [T[0] + int(x==0), T[1] + int(x==1)])[-1],
(__import__('random').randint(0, 1) for _ in range(10)), [0, 0])))
Indented: http://pastebin.com/cfBXNSsZ

code:
print "Heads came up {0} times.\nTails came up {1} times.".format(*(lambda x: (x, 100-x))
(sum(__import__('random').randint(0,1) for _ in range(100))))
edit: I misread your post :downs:

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

code:
(test) C:\Users\Therms> pip install -e hg+[url]http://bitbucket.org/alberanid/imdbpy#egg=imdbpy[/url]

...wall of text with no obvious errors...
code:
(test) C:\Users\Therms>search_movie.py "the matrix"
Traceback (most recent call last):
  File "C:\Users\Therms\test\Scripts\search_movie.py", line 4, in <module>
    from pkg_resources import require; require('IMDbPY==4.6dev-20100521')
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 2603, in <module>
    working_set.require(__requires__)
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 666, in require
    needed = self.resolve(parse_requirements(requirements))
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 565, in resolve
    raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: IMDbPY==4.6dev-20100521
All of the command line scripts that come with the IMDbPY package return this error. Actually using IMDbPY in my own scripts works fine. For example, the following functions correctly:

code:
>>> import imdb
>>> imdb_access = imdb.IMDb()
>>> imdb_access.search_movie("the matrix")
[<Movie id:0133093[http] title:_The Matrix (1999)_>, <Movie id:0234215[http] tit
le:_The Matrix Reloaded (2003)_>, <Movie id:0242653[http] title:_The Matrix Revo
lutions (2003)_>, <Movie id:0092106[http] title:_The Transformers: The Movie (19
86)_>, <Movie id:1392983[http] title:_The Father (2010)_>, <Movie id:0295432[htt
p] title:_The Matrix Revisited (2001) (V)_>, <Movie id:0277828[http] title:_Ente
r the Matrix (2003) (VG)_>, <Movie id:0274085[http] title:_Sex and the Matrix (2
000) (TV)_>, <Movie id:0451118[http] title:_The Matrix: Path of Neo (2005) (VG)_
>, <Movie id:0117021[http] title:_Menno's Mind (1997)_>, <Movie id:0365467[http]
 title:_Making 'The Matrix' (1999) (TV)_>, <Movie id:0390244[http] title:_The Ma
trix Online (2005) (VG)_>, <Movie id:0437137[http] title:_Crash Course (2003) (V
)_>, <Movie id:0391319[http] title:_Making 'Enter the Matrix' (2003) (V)_>, <Mov
ie id:0439783[http] title:_Return to Source: Philosophy & 'The Matrix' (2004) (V
)_>, <Movie id:0410519[http] title:_The Matrix Recalibrated (2004) (V)_>, <Movie
 id:1074193[http] title:_Decoded: The Making of 'The Matrix Reloaded' (2003) (TV
)_>, <Movie id:1499960[http] title:_The Living Matrix (2009)_>, <Movie id:038915
0[http] title:_The Matrix Defence (2003) (TV)_>, <Movie id:0438231[http] title:_
The Matrix: The Movie Special (1999) (TV)_>]
So this has something to do with how pip is installing this package. Since pip and easy_install are a big mystery to me, I was hoping someone can point me in the right direction for fixing this. It's seems obvious that it has something to do with what the install thinks is the required version for the scripts that are packaged with the package, but I don't know where it's getting that required version from, nor what it thinks the version I installed actually is.

As an example of my cluelessness, in the install command above, I don't really know if it matters what I put as the package name in the "#egg=imdbpy" part. I just thought it seemed like a good idea, and it did install...

edit: As per usual, 5 minutes after I posted I figured it out. I guess since .py files are associated with the system-wide installation of python, you have to actually type "python script.py" when using pip to be sure and use the virtualenv...

Thermopyle fucked around with this message at 18:27 on May 21, 2010

Captain Capacitor
Jan 21, 2008

The code you say?

Habnabit posted:

Well I'm going to (again!) suggest cython instead of ctypes; it's much less effort for much higher-quality bindings.

[cython stuff]

(If you really hate yourself and want to use ctypes: count the number of dereferences your C code is doing vs. the types you have declared in ctypes.)

e: whoops, I realized I didn't check the return value of the callocs. Proper error handling is left as an exercise to the reader.

I'm just going to add something to this (possibly because I just gave a talk on it for my local python group), that writing extensions in C isn't all that difficult. The API provides a nice set of functions for building Python objects.

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Captain Capacitor posted:

I'm just going to add something to this (possibly because I just gave a talk on it for my local python group), that writing extensions in C isn't all that difficult. The API provides a nice set of functions for building Python objects.

Sure. It's not hard; cython just removes all of the stupid boilerplate that writing C extensions requires. There is basically no reason to not use cython at this point. There is some stuff that cython can't do (easily), and for these things I just write some extra .c files that I link into the same extension module.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

There is basically no reason to not use cython at this point.

There's tons of reasons, and one of the big ones is "Cython just provides the illusion of easiness for people who are uncomfortable with C". Writing C extension modules for Python is really loving easy. In many places, I find it easier than writing the equivalent code in Python. Cython isn't actually good for everything.

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

There's tons of reasons, and one of the big ones is "Cython just provides the illusion of easiness for people who are uncomfortable with C". Writing C extension modules for Python is really loving easy. In many places, I find it easier than writing the equivalent code in Python. Cython isn't actually good for everything.

I meant to say when making python bindings to C libraries.

Writing C extensions for python still involves a lot of boilerplate if you're doing anything involving argument parsing, refcounting, or defining functions/classes/methods. Cython removes all of this.

Not only that, it by default produces C extensions that are compatible with python 2.3-3.1. That would, again, take a bunch of boilerplate to get right in just C.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

I meant to say when making python bindings to C libraries.

This is exactly the sort of thing I wouldn't use Cython for. What I would use it for is to optimize existing Python code in cases where I don't need to deal with anything C-specific.

Habnabit posted:

Writing C extensions for python still involves a lot of boilerplate if you're doing anything involving argument parsing, refcounting, or defining functions/classes/methods. Cython removes all of this.

Arg parsing is two lines of code in C (including error handling) and is considerably more flexible than in Python. It's also one of the main reasons I prefer writing stuff in C, since it actually allows you to do reasonable things like type-checking arguments and supplying poor-man's overloading without having ludicrous amounts of type-checking in your Python code. Granted, Cython can do limited typechecking (as I recall), but having access to more advanced arg parsing features (e.g. converter functions) is a godsend.

Refcounting is trivially easy and if you have to manage refcounts often when you're just binding C functions, you're doing something wrong. I think the only time I even touch refcounts (with one or two exceptions) is when converting stuff to NumPy arrays.

Defining functions/classes/methods are all easy and trivial to macro if you're doing it a lot and have some specific stuff in common. For instance, I have all my docstrings in an external header file and macro it in based on the function name.

You seem to be under the strange impression that boilerplate is something that you should strive to avoid at all costs, when in reality, much of the boilerplate in a C extension module is a one-time thing that doesn't actually disrupt development in any way once you've done it that one time. The benefit you get for this small amount of effort is a massive improvement in clarity by maintaining a logical separation between Python and C, as well as easy access to the Python C API's more useful features that are strangely absent in Python.

The fact is, unless you are simply binding C to Python without doing anything special, the majority of your code is going to deal with talking to C stuff. And if you are simply binding C to Python without doing anything special, you should be using SWIG. In writing a C extension module, I found that the overwhelming majority of my code was stuff that Cython would do absolutely nothing to help with, e.g. simple conditionals and actual calls to the underlying C library.

Habnabit posted:

Not only that, it by default produces C extensions that are compatible with python 2.3-3.1. That would, again, take a bunch of boilerplate to get right in just C.

Doing that in C consists of a handful of tiny changes in the module init func for 3.x compatibility. Provided you pay a tiny amount of attention to the Python C API docs, you'll have zero problems until you hit the 3.x bump.

EDIT: Granted, the specifics will vary depending on the specific C code you're interfacing with, but I'd have probably killed myself if I had to deal with it in my project.

Avenging Dentist fucked around with this message at 22:46 on May 21, 2010

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

Avenging Dentist posted:

...stuff...

And Ctypes is perfectly good for the rest of us :smug:

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Being pretty inexperienced I often have a hard time judging the quality/reliability/pitfalls of modules that don't have a lot of online ink written about them.

So.... I ask, has anyone here used y_serial?

On the surface, it seems like a pretty cool module for data persistence. I have a soft spot for useful modules that use nothing but standard library and are pure python. Thoughts?

(As a side note it's some of the most usefully documented code I've ever read...very educational to an amateur like me)

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

Arg parsing is two lines of code in C (including error handling) and is considerably more flexible than in Python. It's also one of the main reasons I prefer writing stuff in C, since it actually allows you to do reasonable things like type-checking arguments and supplying poor-man's overloading without having ludicrous amounts of type-checking in your Python code. Granted, Cython can do limited typechecking (as I recall), but having access to more advanced arg parsing features (e.g. converter functions) is a godsend.
I was never comparing arg parsing in C to arg parsing in python. Either way, overloading is a silly thing to try to do in python. And while cython doesn't support converter functions, as far as I understand their utility, they're not really useful with cython's other features. Cython also allows you to optionally specify the types of arguments in the function signature and will do all the necessary typechecking for you.

code:
cdef class Foo:
    cdef int my_int
    def __init__(self, int my_int):
        self.my_int = my_int

def use_a_foo(Foo x not None, int y):
    return x.my_int * y

def use_some_foos(seq, int y):
    cdef Foo x
    ret = []
    for obj in seq:
        x = <Foo ?>obj
        ret.append(x.my_int * y)
    return ret
This will check everything for ints or int-compatible things properly, as well as check for Foo instances or subclasses properly. The not None part is optional; when giving the name of a cdef class instance as a type, it defaults to allowing instances of that class or None. The <Foo ?> cast checks to see if obj is a Foo instance before trying to pull the my_int out of its struct.

Avenging Dentist posted:

You seem to be under the strange impression that boilerplate is something that you should strive to avoid at all costs, when in reality, much of the boilerplate in a C extension module is a one-time thing that doesn't actually disrupt development in any way once you've done it that one time. The benefit you get for this small amount of effort is a massive improvement in clarity by maintaining a logical separation between Python and C, as well as easy access to the Python C API's more useful features that are strangely absent in Python.
It really seems like you're confused about how cython works, between the last few lines here and your earlier comments about typechecking and argument parsing in python. You can call anything within the python C API directly from cython basically without any effort.

Avenging Dentist posted:

The fact is, unless you are simply binding C to Python without doing anything special, the majority of your code is going to deal with talking to C stuff. And if you are simply binding C to Python without doing anything special, you should be using SWIG. In writing a C extension module, I found that the overwhelming majority of my code was stuff that Cython would do absolutely nothing to help with, e.g. simple conditionals and actual calls to the underlying C library.
See, I mentioned before that cython should be used for quality python bindings for C code. SWIG is just going to expose the C library's API directly. So, either you force your users to use that directly-exposed API, or you build another layer on top of the SWIG bindings. Cython lets you do both in one module.

Avenging Dentist posted:

Refcounting is trivially easy and if you have to manage refcounts often when you're just binding C functions, you're doing something wrong. I think the only time I even touch refcounts (with one or two exceptions) is when converting stuff to NumPy arrays.
Again, I'm talking about quality bindings. Yes, you rarely have to do any refcounting stuff if you're just trying to provide an API that directly mirrors the C library's API. But if you're doing any extensive work with python objects, you'll have to do a fair amount of it.

Also, refcounting is only superficially trivially easy because it's easy to forget which things can implicitly execute arbitrary code. The extending/embedding tutorial points out a bug that was in python when explaining the potential perils of refcounting.. Cython does a lot of work and testing to ensure that its generated increfs/decrefs are correct.

Avenging Dentist posted:

Doing that in C consists of a handful of tiny changes in the module init func for 3.x compatibility. Provided you pay a tiny amount of attention to the Python C API docs, you'll have zero problems until you hit the 3.x bump.
It's just another thing that's taken care of for you.

e: Oh, yes, I forgot a few things.

Exception handling is another thing that cython makes amazingly trivial. When making python C API calls, it gets rather tedious rather quickly to check every return value to see if an exception was raised. Trying to emulate try-except-else-finally blocks also starts getting tedious quickly; just look at this trivial example in the docs.. That example also illustrates another thing cython takes care of for you: decreffing everything properly when you have to do an error return from a function. You still have to remember to clean up resources allocated from your C library, but putting that code in a try-finally block is easy enough.

Also sequence unpacking. It's a very minor thing, but it can definitely clean up the API you expose to python to support it. You could require a tuple and then use PyTuple_GET_ITEM or whatever, but why bother when you can support arbitrary sequence types?

m0nk3yz posted:

And Ctypes is perfectly good for the rest of us :smug:
Until you need to access any sort of macro defined in a header file or portably/sanely deal with {u,}int_{16,32,64}.

Habnabit fucked around with this message at 03:30 on May 22, 2010

nbv4
Aug 21, 2002

by Duchess Gummybuns

Thermopyle posted:

code:
(test) C:\Users\Therms> pip install -e hg+[url]http://bitbucket.org/alberanid/imdbpy#egg=imdbpy[/url]

...wall of text with no obvious errors...
code:
(test) C:\Users\Therms>search_movie.py "the matrix"
Traceback (most recent call last):
  File "C:\Users\Therms\test\Scripts\search_movie.py", line 4, in <module>
    from pkg_resources import require; require('IMDbPY==4.6dev-20100521')
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 2603, in <module>
    working_set.require(__requires__)
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 666, in require
    needed = self.resolve(parse_requirements(requirements))
  File "C:\Python26\lib\site-packages\pkg_resources.py", line 565, in resolve
    raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: IMDbPY==4.6dev-20100521
All of the command line scripts that come with the IMDbPY package return this error. Actually using IMDbPY in my own scripts works fine. For example, the following functions correctly:

code:
>>> import imdb
>>> imdb_access = imdb.IMDb()
>>> imdb_access.search_movie("the matrix")
[<Movie id:0133093[http] title:_The Matrix (1999)_>, <Movie id:0234215[http] tit
le:_The Matrix Reloaded (2003)_>, <Movie id:0242653[http] title:_The Matrix Revo
lutions (2003)_>, <Movie id:0092106[http] title:_The Transformers: The Movie (19
86)_>, <Movie id:1392983[http] title:_The Father (2010)_>, <Movie id:0295432[htt
p] title:_The Matrix Revisited (2001) (V)_>, <Movie id:0277828[http] title:_Ente
r the Matrix (2003) (VG)_>, <Movie id:0274085[http] title:_Sex and the Matrix (2
000) (TV)_>, <Movie id:0451118[http] title:_The Matrix: Path of Neo (2005) (VG)_
>, <Movie id:0117021[http] title:_Menno's Mind (1997)_>, <Movie id:0365467[http]
 title:_Making 'The Matrix' (1999) (TV)_>, <Movie id:0390244[http] title:_The Ma
trix Online (2005) (VG)_>, <Movie id:0437137[http] title:_Crash Course (2003) (V
)_>, <Movie id:0391319[http] title:_Making 'Enter the Matrix' (2003) (V)_>, <Mov
ie id:0439783[http] title:_Return to Source: Philosophy & 'The Matrix' (2004) (V
)_>, <Movie id:0410519[http] title:_The Matrix Recalibrated (2004) (V)_>, <Movie
 id:1074193[http] title:_Decoded: The Making of 'The Matrix Reloaded' (2003) (TV
)_>, <Movie id:1499960[http] title:_The Living Matrix (2009)_>, <Movie id:038915
0[http] title:_The Matrix Defence (2003) (TV)_>, <Movie id:0438231[http] title:_
The Matrix: The Movie Special (1999) (TV)_>]
So this has something to do with how pip is installing this package. Since pip and easy_install are a big mystery to me, I was hoping someone can point me in the right direction for fixing this. It's seems obvious that it has something to do with what the install thinks is the required version for the scripts that are packaged with the package, but I don't know where it's getting that required version from, nor what it thinks the version I installed actually is.

As an example of my cluelessness, in the install command above, I don't really know if it matters what I put as the package name in the "#egg=imdbpy" part. I just thought it seemed like a good idea, and it did install...

edit: As per usual, 5 minutes after I posted I figured it out. I guess since .py files are associated with the system-wide installation of python, you have to actually type "python script.py" when using pip to be sure and use the virtualenv...

it seems your search_movie.py script wants a specific version of IMDBpy (4.6dev-20100521), and the version you have installed (from mercurial) is different. In the python prompt if you import IMDBpy and try to get it to print out it's version, I bet it will say something other than 4.6dev-20100521.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

Either way, overloading is a silly thing to try to do in python.

Yeah, because using magic functions like __len__ is clearly a smarter solution. (This is one of the things I absolutely cannot stand about Python, though I guess it lets them avoid stuff like ADL.) Incidentally, the type of overloading I'm talking about is the type used by NumPy, e.g. overloading for arrays and array scalars.

Habnabit posted:

And while cython doesn't support converter functions, as far as I understand their utility, they're not really useful with cython's other features.

Until Python decides to add something standard-ish that approximates (type-safe) enumerations, I'll stick with converter functions.

Habnabit posted:

It really seems like you're confused about how cython works, between the last few lines here and your earlier comments about typechecking and argument parsing in python. You can call anything within the python C API directly from cython basically without any effort.

And I mentioned that I was aware of this (though I'm not aware of all of its semantics, e.g. when working with NumPy array scalars).

Habnabit posted:

Cython lets you do both in one module.

So does C, and for a person comfortable with C, the benefit of Cython essentially boils down to "reducing a small amount of tedium", which isn't really a problem for a fast typist with a decent editor. What I'm trying to get at here is that, if you're actually fluent in C, the benefit to using a Frankensteinian language to hide "real" C code from you is vastly overblown.

I'm not trying to be down on Cython. If someone prefers it, then fine. There are even a couple places that Cython would simplify my code. But those places are pretty rare and I honestly prefer the C API to Python proper (or Cython) in "library" development.

To be honest though, one of the main reasons I'm glad I chose the C API is that I now have a very clear picture of the CPython and NumPy internals, which has helped me enormously, especially when trying to do non-trivial things with NumPy.

Captain Capacitor
Jan 21, 2008

The code you say?
Oh god, I'm siding with Avenging Dentist...what the hell is wrong with me?

:v:

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

:words:

I can't help but notice that you didn't refute my points about reference counting and exception handling. Both of these things are very, very easy to screw up with how many times you have to account for them in C code, and cython handles both automatically. I would definitely not call this a small amount of tedium, especially when the python C API tutorial/reference points out very clearly that these are easy things to screw up.

I would call myself fluent in C, but I still would prefer to reduce the number of potential failure points in my code.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

I can't help but notice that you didn't refute my points about reference counting and exception handling.

If you'd really rather me say "manually handling exceptions and reference counts is not hard" again, I can do it, but I fail to see how it adds to the discussion in any way.

You're welcome to use Cython, but saying, "There is basically no reason to not use cython at this point," is pretty ignorant and basically just a childish "my way or the highway" attitude. It would be like me saying, "There is basically no reason to not use C++ at this point."

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

If you'd really rather me say "manually handling exceptions and reference counts is not hard" again, I can do it, but I fail to see how it adds to the discussion in any way.
See uh I just don't know what imaginary metric you're basing that on, when it the opposite seems to be agreed on enough to warrant mentions in the python documentation.

Avenging Dentist posted:

You're welcome to use Cython, but saying, "There is basically no reason to not use cython at this point," is pretty ignorant and basically just a childish "my way or the highway" attitude. It would be like me saying, "There is basically no reason to not use C++ at this point."
Again, I was talking about making python bindings for C libraries. It's a mature, well-tested project which eliminates a lot of common problems. While there are some things it can't do (easily), yes, these are solved by including some .c files along with the .pyx files.

And if it's good enough for lxml, ... :v:

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

See uh I just don't know what imaginary metric you're basing that on, when it the opposite seems to be agreed on enough to warrant mentions in the python documentation.

If you actually think properly handling errors is hard, as opposed to merely something you need to pay a modicum of attention to, then you are not very good at programming. Sorry, but this just isn't even remotely challenging, unless "paying attention" is something you consider challenging.

This is, of course, massively dependent on how much interaction with Python you need for your project, and that's really my point in all this: if you spend more time interacting with C stuff than with Python stuff, you might as well just use C because the benefits you get from Cython will be spread so thin that you're not gaining much. This was my experience, but this is probably in no small part because the code I was binding was already fairly "Pythonic", especially for something written in C and designed to work with Fortran.

Habnabit posted:

While there are some things it can't do (easily), yes, these are solved by including some .c files along with the .pyx files.

I certainly hope you're not doing that, since that would likely mean you have non-static names in your extension module (in order to access them across translation units), and Python explicitly recommends against this:

quote:

... all symbols in extension modules should be declared static, except for the module’s initialization function, in order to avoid name clashes with other extension modules ...

At first, I thought Cython might be smart enough to handle something like this, but it seems that, far from it, it actually generates .c files on its own, essentially requiring at least a handful of non-static names to appear in the C source so that things can be properly initialized in the module init function.

ShoulderDaemon
Oct 9, 2003
support goon fund
Taco Defender

Avenging Dentist posted:

I certainly hope you're not doing that, since that would likely mean you have non-static names in your extension module (in order to access them across translation units), and Python explicitly recommends against this

After doing all the linking to create the final object file for the module, you can go back and strip any symbols that shouldn't be visible to the outside world. This is fairly normal practice for shared libraries anyway, although that doesn't mean the python community does it when they should.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

ShoulderDaemon posted:

After doing all the linking to create the final object file for the module, you can go back and strip any symbols that shouldn't be visible to the outside world. This is fairly normal practice for shared libraries anyway, although that doesn't mean the python community does it when they should.

Cython definitely doesn't, and last I checked, distutils and friends don't either. Of course, in practice, this probably won't come up unless you're using really common names, but then again, neither will errors related to memory allocation (especially on Linux where malloc() essentially always succeeds and will only fail later when you actually try to use it).

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

If you actually think properly handling errors is hard, as opposed to merely something you need to pay a modicum of attention to, then you are not very good at programming. Sorry, but this just isn't even remotely challenging, unless "paying attention" is something you consider challenging.
No, I didn't say it's hard, or even remotely challenging. It's just another thing you have to remember, and thus, another potential point of failure. You have to remember which python C API functions steal references, which borrow references, which return new references, which return borrowed references, which use NULL as an error return, which use -1 as an error return, which don't have an error return but set an exception, ...

I agree that handling errors is not hard, but the python C API isn't exactly small. I wouldn't exactly say it's easy to try to memorize all of the different semantics of every API call.

(And then, like I said before, you have to deal with all of your temporary python objects and being able to clean them all up regardless of which part of your code threw the exception.)

Trying to avoid this whole mess is basically the reason people use python in the first place. Since, unfortunately, not all software is written in python, there needs to be something to bridge the gap.

Avenging Dentist posted:

I certainly hope you're not doing that, since that would likely mean you have non-static names in your extension module (in order to access them across translation units), and Python explicitly recommends against this:

At first, I thought Cython might be smart enough to handle something like this, but it seems that, far from it, it actually generates .c files on its own, essentially requiring at least a handful of non-static names to appear in the C source so that things can be properly initialized in the module init function.
You can have cython #include your .c file into its own generated .c file, or add prefixes to all of your non-static functions, or as mentioned above, strip those symbols.

Avenging Dentist posted:

Cython definitely doesn't, and last I checked, distutils and friends don't either.
Of course cython doesn't; it's a code generator. It's trivial to add this as a distutils command to run post-bdist_ext.

Habnabit fucked around with this message at 22:59 on May 22, 2010

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

Trying to avoid this whole mess is basically the reason people use python in the first place. Since, unfortunately, not all software is written in python, there needs to be something to bridge the gap.

:lol: I think we're done here.

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip
No, not having an awesome enough moustache to use Perl is the reason people use Python :bustem:

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

:lol: I think we're done here.

I'm just curious what you think the point of using python over C is, then.

Avenging Dentist
Oct 1, 2005

oh my god is that a circular saw that does not go in my mouth aaaaagh

Habnabit posted:

I'm just curious what you think the point of using python over C is, then.

Fear of curly braces?

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

Avenging Dentist posted:

Fear of curly braces?

Right, fear of redundant cruft. :v:

Scaevolus
Apr 16, 2007

Otto Skorzeny posted:

No, not having an awesome enough moustache to use Perl is the reason people use Python :bustem:

Avenging Dentist posted:

Fear of curly braces?

:{)

ATLbeer
Sep 26, 2004
Über nerd

Avenging Dentist posted:

Here you go. I did it in a single statement if you don't count the imports (EDIT: who needs import when you have __import__) (EDITx2: wait I can make multiline lambdas a lot simpler):

code:
print("Heads came up {0} times.\nTails came up {1} times.".format(*__import__('functools').reduce(
lambda T, x: (print(['Heads!', 'Tails!'][x]), [T[0] + int(x==0), T[1] + int(x==1)])[-1],
(__import__('random').randint(0, 1) for _ in range(10)), [0, 0])))
Indented: http://pastebin.com/cfBXNSsZ

hahah.. I hope that's code you'd only write for fun and never for a project. I would boot that so fast :)

Insanely clever though.

passionate dongs
May 23, 2001

Snitchin' is Bitchin'
I am translating some python into javascript, and I am confused by the following bit of code:
code:
sqrt(sum((v1-v2)**2))
What is **2? I am not a python developer and am failing at finding anything with google.

yatagan
Aug 31, 2009

by Ozma

passionate dongs posted:

I am translating some python into javascript, and I am confused by the following bit of code:
code:
sqrt(sum((v1-v2)**2))
What is **2? I am not a python developer and am failing at finding anything with google.

http://www.devshed.com/c/a/Python/Python-Operators/1/

Googling "python operator" was hard.

passionate dongs
May 23, 2001

Snitchin' is Bitchin'

yatagan posted:

http://www.devshed.com/c/a/Python/Python-Operators/1/

Googling "python operator" was hard.

:doh: I don't know why that didn't cross my mind. I feel like an idiot. thanks.

Adbot
ADBOT LOVES YOU

tehk
Mar 10, 2006

[-4] Flaw: Heart Broken - Tehk is extremely lonely. The Gay Empire's ultimate weapon finds it hard to have time for love.
Is there a popular extension framework for python? Something like MEF/MonoAddins/JPF/Boost-Extension.

Currently I am using my own plug-in manager(__import__+ a lot of fluff), but I would much rather use something else if a good plug-in framework exist. This is mostly because I want to have extension information(author, version, and title info) files stored along side the module and I do not want to write a parser. Currently I am storing that information in the module and hoping that plugs keep their modifications inside the activate and deactivate methods.

  • Locked thread