Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Kilometers Davis
Jul 9, 2007

They begin again

Well, i'll be learning it out of hobby-like interest. I won't be doing anything professional for a long while (I haven't even gone to school, which I plan on). I'm just thinking that 3.0 is my best bet since I won't be really working with it professionally for a long while. Sorry for being vague in the first post, hah.

Adbot
ADBOT LOVES YOU

Cryolite
Oct 2, 2006
sodium aluminum fluoride

jstirrell posted:

The problem is that it doesn't compile currently which I think is due to some syntax error at the end of the code which I can't figure out, but what worries me more is that I still feel like I have no clue if I even have a decent structure for the program I'm trying to write, or where I'm really heading with this. Anyone have any suggestions? What am I doing right? What is wrong? Any general tips for struggling self-teaching noobs like myself? Also, I'm trying to get the general structure of the code down so I realize some of the non-critical details (ie elifs) are missing.

Sorry for the wall of code, should I not be posting huge general questions here? I'm just kind of really lost.

There is one syntax error. def newSubject(name): should have self as the first argument. It runs for me after fixing that.

If you didn't catch that then it sounds like you're not using an IDE that inspects your code and points out syntax errors, which can be incredibly helpful if you're just starting out. Are you just typing things into IDLE? If you plan to be doing python for a while (or any programming) you should set up an IDE and use it. Everyone has their favorite but a pretty well-established python IDE is Eclipse using the pydev plugin. Go to eclipse.org and download the Classic Eclipse package down at the bottom then google pydev and follow the instructions to get the python plugin installed. The pydev site has a howto for creating projects and things. It might take a while to set up and get the hang of but the code completion and inspection will probably make it worth it.

tripwire
Nov 19, 2004

        ghost flow
Can someone help me get this stupid mona lisa approximator thing working? I've given up on trying anything remotely fancy and just mutate a list of floats which are the values used to draw triangles on a cairo canvas.

I'm using numpy to add up the squared difference between color values on each pixel and square rooting that to get the root mean square error.

For some reason my picture ALWAYS converges to some ridiculous garbage in a short amount of time which is frustrating, because it could mean either that my genetic algorithm is hosed up somehow (maybe I made a mistake with passing by reference vs passing by value somewhere?) or that the logic for my root mean square error code is hosed up.

Can anyone help me out?
code:
import random, numpy, math
import cairo
import cProfile

class Triangle_genome(object):
    _id = 0
    def __init__(self, parent_id):
        """
        A triangle genome encodes a set of 100 triangles.
        Each Triangle is defined 
        corresponding to the x1,y1,x2,y2,x3,y3,r,g, b,a """
        self._id = self.__get_new_id()
        self.fitness = None
        self._genes = [] #should be a list of 1000 floats
        self.parent_id = parent_id  #tracking the parents id should help with genealogy

    id    = property(lambda self: self._id)
    genes = property(lambda self: self._genes)

    @classmethod
    def __get_new_id(cls):
        cls._id += 1
        return cls._id

    @classmethod
    def create_new_genome(cls):
        """ Factory method for new triangle genomes """
        c = cls(0)
        for _ in xrange(1000):
            c._genes.append( random.random() )
        assert len(c._genes) == 1000
        return c
        
    def crossover(self,other):
        parent1 = self
        parent2 = other        
        child = self.__class__(self.id)
        if self.id == other.id:
            child._genes.extend(parent1.genes[::])
        else:
            child._inherit_genes((parent1,parent2))
        return child
        
    def _inherit_genes(child, parents):
        if len(parents) > 1: #perform crossover
            for i in xrange(100):
                child._genes.extend(random.choice((parent1,parent2))._genes[i*10:i*10+10])
        else: #just a clone
            child._genes.extend(parents[0]._genes)
    
    def mutate(self):
        """ Mutates the genome """
        child = self.__class__(self.id)
        child._inherit_genes((self,))
        r = random.random
        mutations = 0
        for gene in child._genes:
            if r() < 0.003:
                gene += random.gauss(0,1) * 0.4
                if gene > 1.0: gene = 1.0
                elif gene < 0.0: gene = 0.0
                mutations += 1
        print "genome %d has %d mutations" % (child.id,mutations)
        return child 

    def __cmp__(self, other):
        """ Compares genomes by fitness """
        return cmp(self.fitness,other.fitness)

    def __str__(self):
        s = "Genes:"
        for gene in self._genes:
            s += '\n\t' + str(gene)
        return s


class Population(object):
    """ Manages a collection of individuals """
    evaluate = None
    
    def __init__(self):
        self.__pop_size = 100
        self.__create_population()
        self.__generation = -1
        self.__elitism = 0.6 #the percent of the population which survives each gen
    
    def __create_population(self):
        self.__population = []
        for _ in xrange(self.__pop_size):
            g = Triangle_genome.create_new_genome()
            self.__population.append(g)

    def __repr__(self):
        s = "Population Size: %d" % self.__pop_size
        return s

    def __len__(self):
        return len(self.__population)

    def __iter__(self):
        return iter(self.__population)

    def __getitem__(self, key):
        return self.__population[key]

    def average_fitness(self):
        """ Returns the average raw fitness of population """
        sum = 0.0
        for c in self:
            sum += c.fitness

        return sum/len(self)

    def stdeviation(self):
        """ Returns the population standard deviation """
        # first compute the average
        u = self.average_fitness()
        error = 0.0        
        try:    # now compute the distance from average
            for c in self:
                error += (u - c.fitness)**2                 
        except OverflowError:
            #TODO: catch OverflowError: (34, 'Numerical result out of range')
            print "Overflow - printing population status"
            print "error = %f \t average = %f" %(error, u)
            print "Population fitness:"
            print [c.fitness for c in self]
        
        return math.sqrt(error/len(self))

    def reproduce(self):
        self.__population.sort()
        self.__population.reverse()
        survivors = self.__population[:int(round(self.__pop_size * self.__elitism))]
        spawn_amount = self.__pop_size - len(survivors)
        offspring = []
        offspring.extend([s.mutate() for s in survivors])
        while spawn_amount > 0:
            offspring.append( (random.choice(survivors)).mutate() )
        #    parent1,parent2 = random.choice(survivors),random.choice(survivors)
        #    if parent1.id != parent2.id:
        #        child = parent1.crossover(parent2)
        #    else:
        #        child = parent1.crossover(parent1) 
        #        
        #    child.mutate()               
        #    offspring.append(child)
            spawn_amount -= 1
#        print offspring, len(offspring)
        return offspring
    
    def evolve(self, n, termination_fitness):
        """ Runs a naive genetic algorithm for n generations,
        or until the termination fitness is reached, whichever comes first. """
        source_filename = 'Mona.png'
        assert source_filename[-4:] == '.png', 'sorry only using png for the moment'
        source_surf = cairo.ImageSurface.create_from_png(source_filename)
        source_buffer = source_surf.get_data()
        source_array = numpy.frombuffer(source_buffer, numpy.uint8)
        #this is to flip the red and blue channels because apparently
        #numpy uses BGRA color or something in its "frombuffer" function
        source_array.shape = (256,256,4)
        source_array = numpy.take(source_array,(2,1,0,3),2) 
        source_array.shape = (256*256*4)
        #not sure if this is really necessary or not, it should still evolve properly, just in a different color
        
        for _ in xrange(n):
            self.__generation += 1
            print '\n ******* Running generation %d ******* \n' % self.__generation
            self.evaluate(source_array)
            print 'Population\'s average fitness: %3.5f stdev: %3.5f' %(self.average_fitness(), self.stdeviation())
            #self.print_stats()
            best = max(self.__population)
            if best.fitness > termination_fitness: break
            new_population = self.reproduce()
            assert len(self) == len(new_population)
            self.__population = new_population  

    def evaluate(self,source_buffer):
        for individual in self.__population:
            if individual.fitness == None:
                individual.fitness = self.eval_genome(individual,source_buffer)      

    def eval_genome(self, individual, source_buffer):        
        WIDTH, HEIGHT = 256, 256
        surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, WIDTH, HEIGHT)
        ctx = cairo.Context (surface)
        ctx.translate(0,0)
        ctx.scale(256,256)        
        #ctx.set_operator(cairo.OPERATOR_SOURCE)        
        ctx.set_source_rgba(0,0,0,1)
        ctx.paint()
        for i in xrange(100):
            r,g,b,a = tuple( individual.genes[ i * 10 : i * 10 + 4 ] )
            x1,y1,x2,y2,x3,y3 = map(lambda q: q*3 - 1 ,tuple( individual.genes[ i * 10 + 4 : i * 10 + 10 ] ))
            ctx.set_source_rgba(r,g,b,a)
            ctx.move_to(x1,y1)
            ctx.line_to(x2,y2)
            ctx.line_to(x3,y3)
            ctx.close_path()
            ctx.fill()        
        if individual.id % 250 == 0: surface.write_to_png ('example' + str(individual.id) +'.png') # Output to PNG
        buf1 = surface.get_data()

        surface_array = numpy.frombuffer(buf1, numpy.uint8)
        assert numpy.size(surface_array) == numpy.size(source_buffer), 'Source image and Target image are different size!'
        rmse = (255 - math.sqrt(map(lambda x: x**2,numpy.diff(( numpy.dstack((surface_array,source_buffer)) ).astype(numpy.float64)))[0].sum())/255.0)
        print 'genome %d has fitness of %f' % (individual.id,rmse)
        return rmse

if __name__ == "__main__":
    pop = Population()
    #pop.evolve(20000,0.9) #going to try profiling for time being
    cProfile.run('pop.evolve(20000,0.9*255)')

Scaevolus
Apr 16, 2007

tripwire posted:

Can someone help me get this stupid mona lisa approximator thing working? I've given up on trying anything remotely fancy and just mutate a list of floats which are the values used to draw triangles on a cairo canvas.

I'm using numpy to add up the squared difference between color values on each pixel and square rooting that to get the root mean square error.

For some reason my picture ALWAYS converges to some ridiculous garbage in a short amount of time which is frustrating, because it could mean either that my genetic algorithm is hosed up somehow (maybe I made a mistake with passing by reference vs passing by value somewhere?) or that the logic for my root mean square error code is hosed up.

Can anyone help me out?

I'm way too tired to try to read this, but the original didn't use RMS, it used a simple sum of differences. Perhaps that causes garbage convergence?

Also it seems like after a few generations every genome has the exact same fitness, which seems to be an error.

I doubt the effectiveness of crossover between genomes in this problem domain-- stair-climbing seems better.

Scaevolus fucked around with this message at 09:16 on Jan 7, 2009

tripwire
Nov 19, 2004

        ghost flow

Scaevolus posted:

I'm way too tired to try to read this, but the original didn't use RMS, it used a simple sum of differences. Perhaps that causes garbage convergence?

Also it seems like after a few generations every genome has the exact same fitness, which seems to be an error.

I doubt the effectiveness of crossover between genomes in this problem domain-- stair-climbing seems better.

Yeah I commented out crossover and now each generation is produced by mutating the best 60 percent of the previous generation and then randomly choosing among those 60 percent other genomes to mutate and add into the genepool.

The fact that the fitness gets identical after a few generations is whats wrong but I don't know why, it has to be because of some stupid mistake I made somewhere.

There was some discussion in a previous thread here where someone was using particle swarm optimization to try accomplishing the same thing. People in that thread were saying that squaring the differences is likely to give better results because it makes larger differences count more than small differences which makes sense in the context of evolving a better approximation of a picture.

Swanson Broth
Apr 8, 2004

I am new to Python and Pygame (and coding). In what module/function should I be looking in order to draw a line at exactly a 45 degree angle from a point?

king_kilr
May 25, 2007

tripwire posted:

Can someone help me get this stupid mona lisa approximator thing working? I've given up on trying anything remotely fancy and just mutate a list of floats which are the values used to draw triangles on a cairo canvas.

I'm using numpy to add up the squared difference between color values on each pixel and square rooting that to get the root mean square error.

For some reason my picture ALWAYS converges to some ridiculous garbage in a short amount of time which is frustrating, because it could mean either that my genetic algorithm is hosed up somehow (maybe I made a mistake with passing by reference vs passing by value somewhere?) or that the logic for my root mean square error code is hosed up.

Can anyone help me out?

I too tried to implement it(using Pyglet+OpenGL) and I had the same problem, here's my code: http://github.com/alex/evolves/tree/master

Perhaps we both have the same error in our code?

tripwire
Nov 19, 2004

        ghost flow

Swanson Broth posted:

I am new to Python and Pygame (and coding). In what module/function should I be looking in order to draw a line at exactly a 45 degree angle from a point?

Well what are you you drawing to, a display? Check this out http://www.pygame.org/project/997/?release_id=1737

Jo
Jan 24, 2005

:allears:
Soiled Meat

tripwire posted:

code:
...
    def _inherit_genes(child, parents):
        if len(parents) > 1: #perform crossover
            for i in xrange(100):
                child._genes.extend(random.choice((parent1,parent2))._genes[i*10:i*10+10])
        else: #just a clone
            child._genes.extend(parents[0]._genes)
...

It could be me, but I thought when inheriting genes, one would select a random splice point and then take all material up to said splice from one parent and all material from the splice from the other parent. Here, it looks as if you're alternating your selection of chunks of genes.

I.E. Instead of...
P1: [ABCDEFG]
P2: [HIJKLMN]
CH: [ABCKLMN]

You're doing...
P1: [ABCDEFG]
P2: [HIJKLMN]
CH: [ABJKLFG]

tripwire
Nov 19, 2004

        ghost flow

Jo posted:

It could be me, but I thought when inheriting genes, one would select a random splice point and then take all material up to said splice from one parent and all material from the splice from the other parent. Here, it looks as if you're alternating your selection of chunks of genes.

I.E. Instead of...
P1: [ABCDEFG]
P2: [HIJKLMN]
CH: [ABCKLMN]

You're doing...
P1: [ABCDEFG]
P2: [HIJKLMN]
CH: [ABJKLFG]

Hmm good point. I think crossover in general is not really applicable to a genome as specified here, I want to eventually replace this with the pyGEP implementation someone has on google code but I was trying to get this working as proof of concept, thats why I have the crossover functionality effectively cancelled out either with comments or just ensuring its not called.

There has to be something wrong with the way I extend the parents genes or something that ends up making all the genomes identical inspite of the constant mutations. Can you try the code and see if you can get it to work? I'm sure this is like a stupid mistake where a list is made by reference rather than by value or something, theres no reason for the genepool to end up identical in perpetuity in the face of constant mutations.

Kire
Aug 25, 2006
I'm trying to use Tkinter to get a single line of input, then turn that line into a tuple, parsed by the commas that the user enters.

code:
from Tkinter import *

portfolio = []

class MyDialog:

    def __init__(self, parent):

        top = self.top = Toplevel(parent)

        Label(top, text="Portfolio: Date, Purchase Price, Ticker,\
 Current Price:").pack()

        self.e = Entry(top)
        self.e.pack(padx=5)

        b = Button(top, text="Process", command=self.ok)
        b.pack(pady=5)

    def ok(self):

        portfolio = [(self.e.get())]

        self.top.destroy()
        for item in portfolio:
            date = item[0]
            purchase = item[1]
            ticker = item[2]
            current = item[3]
            print 'Date:', date
            print 'Purchase price:', purchase
            print 'Ticker symbol:', ticker
            print 'Current price:', current
        
root = Tk()
root.update()
d = MyDialog(root)
root.wait_window(d.top)

If I input "Jan, 45.0, EK, 55.5" in the popup window then it just spits out "J" "a" "n" "," for the Date, Purchase Price, etc. Any ideas?

tripwire
Nov 19, 2004

        ghost flow
If anyone cares I think I just about got it, although it seems to be very slow. Heres a couple of minutes:


and a little further along:

tripwire fucked around with this message at 09:07 on Jan 8, 2009

whoknew
Sep 18, 2004


oh shit
im lit



Kire posted:

I'm trying to use Tkinter to get a single line of input, then turn that line into a tuple, parsed by the commas that the user enters.

code:
from Tkinter import *

portfolio = []

class MyDialog:

    def __init__(self, parent):

        top = self.top = Toplevel(parent)

        Label(top, text="Portfolio: Date, Purchase Price, Ticker,\
 Current Price:").pack()

        self.e = Entry(top)
        self.e.pack(padx=5)

        b = Button(top, text="Process", command=self.ok)
        b.pack(pady=5)

    def ok(self):

        portfolio = [(self.e.get())]

        self.top.destroy()
        for item in portfolio:
            date = item[0]
            purchase = item[1]
            ticker = item[2]
            current = item[3]
            print 'Date:', date
            print 'Purchase price:', purchase
            print 'Ticker symbol:', ticker
            print 'Current price:', current
        
root = Tk()
root.update()
d = MyDialog(root)
root.wait_window(d.top)

If I input "Jan, 45.0, EK, 55.5" in the popup window then it just spits out "J" "a" "n" "," for the Date, Purchase Price, etc. Any ideas?

portfolio = [(self.e.get().split(','))]

Although it's a lot clearer if you just do:

code:
portfolio = self.e.get().split(',')
self.top.destroy()
try:
    date = item[0]
    purchase = item[1]
    ticker = item[2]
    current = item[3]
    print 'Date:', date
    print 'Purchase price:', purchase
    print 'Ticker symbol:', ticker
    print 'Current price:', current
except IndexError:
    print 'Improper input.'
If you really, really need a tuple, you can do tuple(self.e.get().split(',')), mainly if you want this data to be immutable for whatever reason (usually if you're going to hash it for use as a dict key.).

whoknew fucked around with this message at 08:33 on Jan 8, 2009

UnNethertrash
Jun 14, 2007
The Trashman Cometh
I've been doing the Euler problems after I read about them (here in fact) the other day. Here is the code I used on number 47 (basically it involves calculating prime factors):

code:
#Find first 4 consecutive integers with 4 distinct prime factors
#Answer: 134043

from math import sqrt

def get_prime_factors(n): #only distinct prime factors
    prime_factors = []
    x = n
    max_factor = int(sqrt(n))
    current = 2
    while current <= x and current <= max_factor:
        while x % current != 0:
            current += 1
        if current not in prime_factors:    #remove this if to get all prime factors
            prime_factors.append(current)
        x = x/current
    return prime_factors



current = 5

while True:
    if current % 1000 == 0:
        print current
    fac_1 = False
    fac_2 = False
    fac_3 = False
    factors_1 = get_prime_factors(current)
    if len(factors_1) == 4:
        fac_1 = True
        factors_2 = get_prime_factors(current + 1)
        if len(factors_2) == 4:
            fac_2 = True
            factors_3 = get_prime_factors(current + 2)
            if len(factors_3) == 4:
                fac_3 = True
                factors_4 = get_prime_factors(current + 3)
                if len(factors_4) == 4:
                    print "Answer is", current
                    break
    if (fac_1 == True) and (fac_2 == False) and (fac_3 == False):
        current += 2
    elif (fac_1 == True) and (fac_2 == True) and (fac_3 == False):
        current += 3
    elif (fac_1 == True) and (fac_2 == True) and (fac_3 == True):
        current += 4
    else:
        current += 1
#    current += 1
This is a basic brute force attack with a few truth statements to raise the efficiency. It solves the problem in about 2 minutes.

In the answer forum someone posted this:

code:
import sys
 
def find_factor(n) :
    for i in xrange(2, int(n**0.5 + 1)) :
        if n % i == 0 :
            return i
    return n
 
def find_factors(n) :
    factors = []
    while n != 1 :
        fac = find_factor(n)
        if fac not in factors :
            factors.append(fac)
        n /= fac
    return factors
 
lastfail = 0
for i in xrange(1, 1000000) :
    if len(find_factors(i)) != 4 :
        lastfail = i
    else :
        if i - lastfail > 3 :
            print i, lastfail
            sys.exit()
As far as I can tell the method of finding the factors is identical to what I came up with (just in the form of 2 functions instead of 1), but his code solves the problem in a few seconds.

Ninja edit: My question, of course, is why is his code so much faster when they seem identical?

functional
Feb 12, 2008

UnNethertrash posted:

Ninja edit: My question, of course, is why is his code so much faster when they seem identical?

I read his code. After the second time through it was clear to me what he was doing. Very straightforward, simply accomplished. I've tried to read through your code four or five times now. Each time I give up because the spaghetti logic is simply too much. I have a question for you and a recommendation.

Question: If your code is running so much slower than this person's code, are your if statements really improvements? If you determine that they're not improvements of the most important type, perhaps you could go back to before you added them in, and optimize there.

Recommendation: If you were truly interested in improving your ability, rather than continue on solving each of these Euler problems, none of which will pose any real challenge to you, you would start over at the beginning and play golf with each of the problems. What is the shortest amount of code you can solve each of the problems in? I think this exercise will do you a lot more good.

functional fucked around with this message at 19:42 on Jan 8, 2009

UnNethertrash
Jun 14, 2007
The Trashman Cometh

functional posted:

I read his code. After the second time through it was clear to me what he was doing. Very straightforward, simply accomplished. I've tried to read through your code four or five times now. Each time I give up because the spaghetti logic is simply too much. I have a question for you and a recommendation.

Question: If your code is running so much slower than this person's code, are your if statements really improvements? If you determine that they're not improvements of the most important type, perhaps you could go back to before you added them in, and optimize there.

Recommendation: If you were truly interested in improving your ability, rather than continue on solving each of these Euler problems, none of which will pose any real challenge to you, you would start over at the beginning and play golf with each of the problems. What is the shortest amount of code you can solve each of the problems in? I think this exercise will do you a lot more good.

Thanks, I will try redoing the if blocks and see what happens. I didn't look at them before because I assumed that the processing of if statements would be trivial compared to the time necessary to factor numbers. Perhaps I was wrong about that.

I realized the same thing as your recommendation, which is why I started looking at other peoples code to begin with. Thanks.

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

tripwire posted:

:words:

Note that a leading __ does not make private attributes. It invokes name mangling, which is most definitely not what you want. It's only useful if you're trying to prevent naming conflicts in subclasses. If you're trying to annotate an attribute as private, use a better lexical convention: a leading _. Everyone knows what you mean and it doesn't mangle names.

tripwire
Nov 19, 2004

        ghost flow

Habnabit posted:

Note that a leading __ does not make private attributes. It invokes name mangling, which is most definitely not what you want. It's only useful if you're trying to prevent naming conflicts in subclasses. If you're trying to annotate an attribute as private, use a better lexical convention: a leading _. Everyone knows what you mean and it doesn't mangle names.

Thanks for the heads up. I knew I read before that you were supposed to use just one underscore for private names but apparently I'm an idiot and forgot. Using it on the functions i want to override is ok though right? I based most of this off of a neuroevolution package I used to use because I had no idea where to start so theres still some boneheaded artifacts of my unthinking copy+pasting in there.

My population class has its own iteration and comparator methods though, in that case double underscores are correct right? Like if you are trying to override a builtin function?

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

tripwire posted:

Thanks for the heads up. I knew I read before that you were supposed to use just one underscore for private names but apparently I'm an idiot and forgot. Using it on the functions i want to override is ok though right? I based most of this off of a neuroevolution package I used to use because I had no idea where to start so theres still some boneheaded artifacts of my unthinking copy+pasting in there.

My population class has its own iteration and comparator methods though, in that case double underscores are correct right? Like if you are trying to override a builtin function?

Yes, name mangling only takes effect if there's a leading __ with no trailing __. So, __iter__ et al. are safe.

Regarding your code itself, I haven't used (or even installed) cairo myself, so I unfortunately can't be much help there.

e: Oh, I missed a part of your question. No, you don't want to use name mangling if you want to override the methods. In fact, name mangling is intended to prevent things from being (implicitly) overridden.

Habnabit fucked around with this message at 09:54 on Jan 9, 2009

tripwire
Nov 19, 2004

        ghost flow

Habnabit posted:

Yes, name mangling only takes effect if there's a leading __ with no trailing __. So, __iter__ et al. are safe.

Regarding your code itself, I haven't used (or even installed) cairo myself, so I unfortunately can't be much help there.

e: Oh, I missed a part of your question. No, you don't want to use name mangling if you want to override the methods. In fact, name mangling is intended to prevent things from being (implicitly) overridden.

Hrm, guess I've misunderstood. I'm a little confused, can you help me to understand the reasons why its used in this code?
http://code.google.com/p/neat-python/source/browse/trunk/neat/population.py

Thats the thing I'm ripping off. The writer specifically prefixes function definitions and private variables with double underscores, in some cases with trailing double underscores as well, but only for functions which I assume are inherited from object?
Right here:
code:
class Population(object):
.
.
.
.
    def __len__(self):
        return len(self.__population)

    def __iter__(self):
        return iter(self.__population)

    def __getitem__(self, key):
        return self.__population[key]
Whats the idea of that? I thought it was so then when you call len() on a population object it would use the __len__ definition provided above.

Allie
Jan 17, 2004

It could just be overzealous use. I've seen developers do that in other projects, perhaps thinking __foo is like "private" in other languages (it isn't). I haven't delved into the code there deep enough to see if there are complex inheritance issues that really warrant it.

For what it's worth, I almost never use name mangling.

tbradshaw
Jan 15, 2008

First one must nail at least two overdrive phrases and activate the tilt sensor to ROCK OUT!

tripwire posted:

Whats the idea of that? I thought it was so then when you call len() on a population object it would use the __len__ definition provided above.

Yes, that's what it is. The convention is:

  • public - a public member
  • _private - a (semi) private member. The _ demarcation is a convention to notify anyone that uses this member that they probably shouldn't be.
  • __private - a private member. The __ demarcation will trigger a name mangling of the member to make it unique to the class and very unlikely that anything that inherits, uses, or calls the member will actually do so.

An example of this would be:

code:
class A(object):
  def __do(self):
    pass


class B(A):
  def __thing(self):
    pass
  
  def stuff(self):
    self.__do()
Is actually something like:

code:
class A(object):
  def _A_do(self):
    pass


class B(A):
  def _B_thing(self):
    pass
  
  def stuff(self):
    self._B_do()
Which, as you can see from the second example, calling stuff() on B will return an error because _B_do() doesn't exist. The idea is that since you named A.__do with a __, even though B inherited the method, it's not easily called. The reason people say "THIS ISN'T PRIVATE!" is that you can definitely reach into the __dict__ of B, find the mangled name of _A_do, and call it.

However, that certainly doesn't mean that you shouldn't use __ as private. That's exactly what it's for. The real lesson here is that with the duck typing that Python uses, a "real" private member isn't nearly as useful at it seems when you're working in a static language like Java or something. So just use _private unless you find that you truly need __private.

Regarding __magic__. Things that start with double underscore and end with double underscore are lovingly called "magic" attributes. They do things like you noticed, interacting with built in functions. Generally speaking, you shouldn't ever name your own stuff __magic__, instead, you should just use those whenever you want to do something "magic" that you found in the Python documentation.

tbradshaw fucked around with this message at 20:00 on Jan 9, 2009

Id4ever
Dec 2, 2004
I've got a small problem with Python 2.6 on my Vista computer. I have a Python script 'foo.py' containing the following code:
code:
import sys
print sys.argv
If I enter "python foo.py bar" in cmd.exe (in the directory containing foo.py), then it prints "['foo.py', 'bar']" as expected. If I instead enter just "foo.py bar", it prints "['F:\\Code\\Python\\foo.py']". Note that the 'bar' argument is missing.

On my laptop running XP and Python 2.6 the latter prints "['F:\\Code\\Python\\foo.py', bar]", which again is what one would expect. Why the strange behavior on my Vista computer?

Edit: This was caused by some major problems with the .py file associations, fixed by manually editing the relevant registry keys.

Id4ever fucked around with this message at 00:10 on Jan 10, 2009

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

tbradshaw posted:

:words:

Using __dict__ isn't necessary. __foo in class Bar turns into _Bar__foo, and since that doesn't have two leading underscores, subclasses can use self._Bar__foo to access the attribute just fine. Overzealous use is definitely the problem here, because it just makes it a pain in the rear end for people who have to use the library later on.

Another caveat: this is done by the compiler, so getattr(self, '__foo') sidesteps the name mangling.

tripwire
Nov 19, 2004

        ghost flow
Thanks for the help guys, I think I understand now.

I tidied up my code a bit but now I've run into another problem as I started adding in things- rather than steadily ratcheting up in similarity over time, the fitness seems to fluctuate wildly now with no clear indication of progress.
Is there some obvious error anyone can see? I screwed up before because I stupidly didn't realize that doing
code:
for gene in self.genome:
    gene += random.gauss(0,1)
was actually just making a temporary copy of each element and throwing it away after. So I ended up with a lot of ugly looking xranges assigning to genes by index now, if someone can suggest a nicer more idiomatic way of phrasing some of it I'd appreciate the help, the last version I saved is here

http://pastebin.com/m109364c
Its frustrating trying to figure out where I screwed up, I'm sure there has to be a much more elegant way of accomplishing what I'm doing in python than what I've got.

dorkanoid
Dec 21, 2004

code:
self.genome = [gene + random.gauss(0,1) for gene in self.genome]
...should work :)

tripwire
Nov 19, 2004

        ghost flow

dorkanoid posted:

code:
self.genome = [gene + random.gauss(0,1) for gene in self.genome]
...should work :)
How do you build a list comprehension that would mimic the functionality of going through every gene in the genome and conditionally applying a mutation if random.random() is greater or less than than some threshold value?
I can get as far as
code:
self.genome = [gene + random.gauss(0,1) for gene in self.genome if r() < threshold]
but that obviously won't work because whenever that last if statement is not true a gene will get cut from the genome.
Using a lambda and map I guess?

No Safe Word
Feb 26, 2005

tripwire posted:

How do you build a list comprehension that would mimic the functionality of going through every gene in the genome and conditionally applying a mutation if random.random() is greater or less than than some threshold value?
I can get as far as
code:
self.genome = [gene + random.gauss(0,1) for gene in self.genome if r() < threshold]
but that obviously won't work because whenever that last if statement is not true a gene will get cut from the genome.
Using a lambda and map I guess?
Don't use a lambda, just use a real function:
code:
def foo(gene):
    if random.random() < threshhold:
        return gene + random.gauss(0, 1)
    else:
        return gene

map(foo, self.genome)
You can even define it inside the function that uses it so that it doesn't pollute the namespace, in fact I'd recommend it.

hey mom its 420
May 12, 2007

I don't see what's wrong with this:
code:
self.genome = [gene + random.gauss(0,1) if random.random() < threshold else gene for gene in self.genome]
if you don't want the gene variable leaking out, you can do
code:
self.genome = list(gene + random.gauss(0,1) if random.random() < threshold else gene for gene in self.genome)
because generator expressions don't leak out variables whereas list comprehensions do, although this is fixed in Py3k.

Also, just doing map(foo, self.genome) won't do a thing because map doesn't modify the list, it just returns a new copy, so you have to do self.genome = map(foo, self.genome)

hey mom its 420 fucked around with this message at 16:39 on Jan 10, 2009

No Safe Word
Feb 26, 2005

Honestly I forgot you could do the if clause there and not just on the for portion of the list comprehension. That should be fine :)

tripwire
Nov 19, 2004

        ghost flow

Bonus posted:

I don't see what's wrong with this:
code:
self.genome = [gene + random.gauss(0,1) if random.random() < threshold else gene for gene in self.genome]
if you don't want the gene variable leaking out, you can do
code:
self.genome = list(gene + random.gauss(0,1) if random.random() < threshold else gene for gene in self.genome)
because generator expressions don't leak out variables whereas list comprehensions do, although this is fixed in Py3k.

Also, just doing map(foo, self.genome) won't do a thing because map doesn't modify the list, it just returns a new copy, so you have to do self.genome = map(foo, self.genome)
I didn't realize list comprehensions were that powerful. Thanks a lot for your help!

tripwire
Nov 19, 2004

        ghost flow
Ok, I tried to tidy up my code some more and use list comprehensions and generators where applicable, but theres still some dumb mistake somewhere that is preventing it from working.

Can anyone take another look and see if anything looks off?
http://codepad.org/qIW3uWEp
I'm stumped.
It should generally be going UP in fitness over time because its always removing the worst of the population and the majority of mutations should have negligible effect on fitness, yet the fitness always seems to fluctuate wildly with no apparant direction.

I think my code for mutation must be screwed up somewhere but I can't figure out where.
Aw what the hell...
I just checked generation by generation to see what was going on and it looks like every member of the new generation is a clone, or they all receive the same mutations.
How can this be? Garghfghdfj
(each row is a new generation)
---------
update: Hurray!! I realized that
code:
self.genome = list(gene + random.gauss(0,1) if random.random() < threshold else gene for gene in self.genome)
would end up passing that actual instance of gene class.
What this ended up doing was making 10 children whose genomes all pointed to genes that existed in someone elses genomes.

Obviously I needed to copy them rather then just refer to them so I gave my genes a copy class which simply returns a new instance with the same members.
I feel so good about finally getting this to a workable state!

tripwire fucked around with this message at 08:30 on Jan 11, 2009

Modern Pragmatist
Aug 20, 2008
I am attempting to develop a portion of a GUI using Tkinter. I have a listbox where I select an image file, the file is loaded and then displayed on a canvas. I am having a difficult time getting the image created within the subfunction to display in the canvas.

I initialize the canvas with a logo for the program, but I can't get my callback function for the listbox to update the image on the canvas. It just remains blank.

code:
import Tkinter
from PIL import Image, ImageTk

def updateImages(evnt):
    image = Image.open(imList.get(imList.curselection()[0]))
    image.thumbnail((300,300))

    im = ImageTk.PhotoImage(image)

    can.itemconfigure(img,image=im)

root = Tkinter.Tk()

can = Tkinter.Canvas(root, height=300, width=300, bg='#FFF')
can.grid(row=0,column=0)

logo = Image.open("logo.jpg")
logo.thumbnail((300,300))

logoIm = ImageTk.PhotoImage(logo)

img = can.create_image((150,150),anchor=Tkinter.CENTER,image=logoIm)

imList = Tkinter.Listbox(root)
imList.grid(row=0,column=1)

imList.bind("<<ListboxSelect>>",updateImages)

imList.insert(Tkinter.END,"/path/to/new/image.jpg")

root.mainloop()
Any insight would be greatly appreciated. thanks.

ionn
Jan 23, 2004

Din morsa.
Grimey Drawer
I have a bunch of hostnames that I need to check if they exist in DNS. I'm not connecting to them (that is done later by passing it on to ssh), I just need to verify that they're there. I'm using plain socket.gethostbyname() which works well, but the problem is that hostnames that do not exist take forever to time out and throw the "not found" exception. Well, not forever, but a few seconds which add up to a whole lot when doing lots of hosts. All valid names resolve very quickly (usually <10ms, since it's all nearby dns zones) and non-existend ones return NXDOMAIN just as fast (checked using dig).

I would like to set a short timeout (of, say, 100ms), and if it does not get a response in that time just say that it isn't there, but I'm not sure how to control that timeout. socket.setdefaulttimeout() seemingly has no effect on socket.gethostbyname() or socket.getaddrinfo().

I'm not very skilled with threads in Python, but would it work to start a thread to do the gethostbyname() call, and if it doesn't respond in time just kill it off?
Would it be possible to run several such calls in parallel, or could there be something thread-unsafe about that (especially when killing them off)? I've read something about the C function for gethostbyname() being thread-unsafe in some way, but I have no idea about how that turns out in Python.

ionn fucked around with this message at 00:01 on Jan 12, 2009

WickedMetalHead
Mar 9, 2007
/dev/null
twisted has a lot of non blocking type stuff and has a dns specific implementations if i recall correctly.

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.
Seconding this. Don't bother with threads when twisted provides an async DNS resolver already.

ionn
Jan 23, 2004

Din morsa.
Grimey Drawer

WickedMetalHead posted:

twisted has a lot of non blocking type stuff and has a dns specific implementations if i recall correctly.

Habnabit posted:

Seconding this. Don't bother with threads when twisted provides an async DNS resolver already.

Twisted looks like the way to go for lots of network-related stuff, there are many other cases where I could imagine it being very useful to me.
I'm building the dns-resolution thingy, it still takes some thread-awareness (a thread-safe place to keep track of outstanding requests) but shouldn't be too difficult.

tripwire
Nov 19, 2004

        ghost flow
Has anyone used the multiprocessing module before? I'm trying to use a pool to split up the work in my mona lisa triangle mover thing but it doesn't look like you can call it from an instance of a class and point to an instance method (it raises a pickle error, 'can't pickle instancemethod' or something like that. Do I have to make it use a global function? Is there some idiom or pattern for doing this?

No Safe Word
Feb 26, 2005

tripwire posted:

Has anyone used the multiprocessing module before? I'm trying to use a pool to split up the work in my mona lisa triangle mover thing but it doesn't look like you can call it from an instance of a class and point to an instance method (it raises a pickle error, 'can't pickle instancemethod' or something like that. Do I have to make it use a global function? Is there some idiom or pattern for doing this?

I haven't, but I know one poster here has it's m0nk3yz, he was like the main guy who wrote that module got it into python

edit: fixed

No Safe Word fucked around with this message at 10:02 on Jan 13, 2009

Adbot
ADBOT LOVES YOU

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

tripwire posted:

Has anyone used the multiprocessing module before? I'm trying to use a pool to split up the work in my mona lisa triangle mover thing but it doesn't look like you can call it from an instance of a class and point to an instance method (it raises a pickle error, 'can't pickle instancemethod' or something like that. Do I have to make it use a global function? Is there some idiom or pattern for doing this?

Yes, it has to be a pickle-able function/class instance/etc. Make it a function, and you're golden. choo choo! I'm not good with the GUI stuff, so I will help where I can.

Also, I didn't write it, I just got it into python.

  • Locked thread