Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Dren
Jan 5, 2001

Pillbug
uhhh...

quote:

Developers are often guys that live on another planet. Some think this is because software development is so exciting that everything else is poorly considered. We are deeply convinced that most developers do not enjoy themselves. They spend their lives together with complex problems that never go away from their minds. Nobody understands them. Their family? Their managers? Their clients? Their friends? No. Nobody. Nobody is able to imagine how huge and complex their tasks are. Consequently, what they do is underestimated. Worst: they can't communicate. Don't believe this is due to some intrinseque geek attitude. Their geekness results from the global incapacity to apprehend the very nature of their abilities. So they are forced to work harder while experiencing the true impossibility to share their permanent software experience. Of course, it may lead to social disasters.

Adbot
ADBOT LOVES YOU

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
You know it's going to be good when you read that.

"I can't convince anybody how good my lovely idea was, because I'm a nerd"

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
This web framework works by adding Python expressions to the "Review Comments" section of Word/Excel documents.

Dren
Jan 5, 2001

Pillbug

Suspicious Dish posted:

You know it's going to be good when you read that.

"I can't convince anybody how good my lovely idea was, because I'm a nerd"

The worst part is that's not just in there, it's how it starts. "I am incapable of communicating effectively. But listen, this idea I have is really good so it is totally worth slogging through my barely comprehensible bullshit, I promise."

Dominoes
Sep 20, 2007

quote:

So they are forced to work harder while experiencing the true impossibility to share their permanent software experience. Of course, it may lead to social disasters.
Looks like the author ops-tested this one.

Hey, so PyQt's flagging some of the methods of my Qt main window class as static. I read up on static methods and the @staticmethod decorator. Most of what I read pointed it to being pointless, and for the examples posted, I agree. I feel like this is a somewhat special case where @staticmethod may be more clear. Ie: These are functions that are only called from pressing a button etc in a QT window, and belong to the GUI structure. They seem to make sense grouped in with the other QT methods, even though the code they affect isn't part of the GUI. What's your take on this?


Python code:
class Main(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        QtWidgets.QWidget.__init__(self, parent)
        self.ui = Ui_Main()
        self.ui.setupUi(self)
        self.ui.btn_refresh.clicked.connect(self.refresh)

    def refresh(self):
        t = RefreshThread()
        t.start()
  
Other examples are a method that pulls up a matplotlib chart when clicking tree widet items, and one that sets GUI text color dynamically by returning a value.

How would you handle refresh?
-A normal method that's static (like above)
-Decorated with '@staticmethod', no self arg
-A function in the program body

Dominoes fucked around with this message at 18:38 on Dec 9, 2013

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Sometimes I try to do a hybrid between type checking and duck typing. By this I mean that I immediately check that the passed-in object has the attributes and methods I expect to use later on. I do this for the following reasons:
  • It makes explicit, early on, what kind of stuff we want.
  • It helps me remember in two weeks what the hell I'm doing.
  • I can raise more appropriate exceptions with better error messages.
  • If it's a long running-process that may only occasionally use an attribute, I'd rather know right off the bat rather than at 2AM in three weeks.
Whenever I do this, I always feel like the idealized Python programmer looking over my shoulder is shaking his head at me. On the other hand, I always shake my head at the programmer who slavishly sticks to some idealized set of rules at the expense of lots of time or verbosity or whatever.

So, if you came across code doing this would you shake your head?

Dominoes
Sep 20, 2007

Thermopyle posted:

Sometimes I try to do a hybrid between type checking and duck typing. By this I mean that I immediately check that the passed-in object has the attributes and methods I expect to use later on. I do this for the following
So, if you came across code doing this would you shake your head?

Example?

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Thermopyle posted:

Sometimes I try to do a hybrid between type checking and duck typing. By this I mean that I immediately check that the passed-in object has the attributes and methods I expect to use later on.
That's not a hybrid; it's just duck typing with some error checking. Nothing about duck typing requires that you not verify that the thing you're passed actually does look and quack like a duck; merely that you don't care about it's actual type so long as it does implement the interface you need.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell


Python code:
class WhatsUpHolmes(object):
  def __init__(self):
    self.an_attribute = 'butts'

class Howdy(object):
  def __init__(self, holmes_greeting):
    if not hasattr(holmes_greeting, 'an_attribute'):
      raise TypeError("Provide an object with an_attribute, you idiot")

Plorkyeran posted:

That's not a hybrid; it's just duck typing with some error checking. Nothing about duck typing requires that you not verify that the thing you're passed actually does look and quack like a duck; merely that you don't care about it's actual type so long as it does implement the interface you need.

Yeah, I guess you're right. Typically I'll see code just blindly using whatever objects its passed and relying on downstream exceptions to raise when its the wrong type of object.

Dominoes
Sep 20, 2007

I do stuff like that when it seems appropriate, only subbing what would be exception handling for raising an error in your example. It's due to the explicit > implicit paradigm, which doesn't always agree with duck-typing.

For example:

Python code:
if not x:
    return
instead of traditional duck typing:
Python code:
try:
    #do something with x here
except TypeError:  # This error means x wasn't assigned a value in some_function.
    return
You have to explain what the error means in comments, when it's clear what the first example's checking for. Case-by-case. I usually go with try/except/comment. Some things, like KeyError, FileNotFoundError and PermissionError are pretty explicit as-is, so I'll duck-type them every time, without needing a comment to explain. Another example of a case where I wouldn't duck-type is when using a complex statement like a dictionary comprehension, where it may appear ambiguous what raised the exception, but I'm checking for something specific.

Dominoes fucked around with this message at 05:49 on Dec 10, 2013

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Thermopyle posted:

Yeah, I guess you're right. Typically I'll see code just blindly using whatever objects its passed and relying on downstream exceptions to raise when its the wrong type of object.
That's a mix of laziness and the idea of asking for forgiveness rather than permission. Ultimately the only entirely reliable way to know if you can do a thing with an object is to try to do and handle the errors if it fails, so any sort of preemptive checking is going to be redundant code. I do think it's sometimes worth doing for the reasons you gave, but if you find yourself wanting to do it all over the place then you'd probably be happier in a language with static type checking.

tef
May 30, 2004

-> some l-system crap ->

Thermopyle posted:

So, if you came across code doing this would you shake your head?

As long as you're checking for quacks and not for class types, it would seem ok. I often skip the checks and convert to the type I want (so I'll call tuple, or list, or dict, etc, making a copy but it catches a lot of annoyances)

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

tef posted:

I often skip the checks and convert to the type I want (so I'll call tuple, or list, or dict, etc, making a copy but it catches a lot of annoyances)

I like that.

In other news, Jetbrains is having a 1 hour webinar wherein they build a Pinterest clone with PyCharm. I know there's a few newerish users around here, so I thought I'd mention it.

quote:

We'll demonstrate the workflow and tools to make using databases, Flask, HTML, CSS, and JavaScript a breeze by building and deploying a Pinterest clone. Come learn many of the tips and tricks PyCharm provides to turbocharge your daily tasks.

Dren
Jan 5, 2001

Pillbug
This article is pretty cool. It's about how the LA Times used NLTK to parse recipes out of news articles and create a cookbook.

http://datadesk.latimes.com/posts/2013/12/natural-language-processing-in-the-kitchen/

Dominoes
Sep 20, 2007

Thermopyle posted:

PyCharm doesn't flag those for me.

I'm running version 131.618.
Just updated to this version, and the bug's still there, every time. (without ignoring Pep8 # 226 in settings)

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Dominoes posted:

Just updated to this version, and the bug's still there, every time. (without ignoring Pep8 # 226 in settings)

I'd contact support and/or file a bug on their issue tracker. Jetbrains is quite responsive.

Dominoes
Sep 20, 2007

Done.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell


From their response it looks like this is where you need to file a report:

https://github.com/jcrocholl/pep8/issues

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
A previous version of PEP8 suggested (a + b) * (c + d). It seems that changed recently:

http://hg.python.org/peps/rev/37af28ad2972
http://hg.python.org/peps/rev/16dd63848921

Dominoes
Sep 20, 2007

Sounds like that's the issue; the external tool is out of date.

The new method's easier to read, and I'd probably break the Pep8 rule if it still said to use spaces always.

Reposted

Pollyanna
Mar 5, 2005

Milk's on them.


Tried to take the Python test on E-Lance and I couldn't even get past the first question :suicide:

The question was "write a problem that takes an input "n" and returns the maximum" blah blah gently caress I don't totally remember it was something about finding the maximum number of range(1, 9xn) that is evenly divisible by n. Like, 9x2 would be "99", and the result would be "98" since that's the maximum number that's evenly divisible by 2.

I submitted an answer for that one and ragequit when it was wrong. Especially since it was taking too long for me to write it. I may have freaked out a little too harshly :( Has anyone seen that problem before? Am I retarded for not finishing with 5 minutes or am I retarded for blowing the whole thing on one fuckup? (it's the latter)

ManoliIsFat
Oct 4, 2002

Pollyanna posted:

Tried to take the Python test on E-Lance and I couldn't even get past the first question :suicide:

The question was "write a problem that takes an input "n" and returns the maximum" blah blah gently caress I don't totally remember it was something about finding the maximum number of range(1, 9xn) that is evenly divisible by n. Like, 9x2 would be "99", and the result would be "98" since that's the maximum number that's evenly divisible by 2.
final_answer = number - (number % 2)

And in python you can multiply strings by numbers, so 'x' * 4 which will give you 'xxxx'

http://repl.it/NFZ

ManoliIsFat fucked around with this message at 21:11 on Dec 12, 2013

Pollyanna
Mar 5, 2005

Milk's on them.


ManoliIsFat posted:

final_answer = number - (number % 2)

And in python you can multiply strings by numbers, so 'x' * 4 which will give you 'xxxx'

http://repl.it/NFZ

:negative: I am loving retarded.

This was my implementation:

Python code:
def greatest_n_digit(param):

	n = param

	list = []

	for i in range(1, (int("9" * n))):
		if i % n == 0:
			list.append(i)

	return max(list)

for i in range(1, 10):
	print str(greatest_n_digit(i)) + ", "
I tried running it and it ate up all my memory and CPU and took over two minutes to complete. I really don't know what the gently caress I'm doing, do I?

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Pollyanna posted:

Tried to take the Python test on E-Lance and I couldn't even get past the first question :suicide:

The question was "write a problem that takes an input "n" and returns the maximum" blah blah gently caress I don't totally remember it was something about finding the maximum number of range(1, 9xn) that is evenly divisible by n. Like, 9x2 would be "99", and the result would be "98" since that's the maximum number that's evenly divisible by 2.

I submitted an answer for that one and ragequit when it was wrong. Especially since it was taking too long for me to write it. I may have freaked out a little too harshly :( Has anyone seen that problem before? Am I retarded for not finishing with 5 minutes or am I retarded for blowing the whole thing on one fuckup? (it's the latter)

Don't get so down on yourself and keep practicing. It takes time to build up the mental toolbox of things you can use to solve problems. You aren't alone in experiencing these frustrations. You do yourself a disservice though if you don't learn from these moments!

Sometimes for those types of problems, it's better to just bust out a pen and paper and play around with the numbers to figure out how your program might work.

a lovely poster
Aug 5, 2011

by Pipski

Pollyanna posted:

I really don't know what the gently caress I'm doing, do I?

Write out the answer in English before you start coding.

ManoliIsFat
Oct 4, 2002

Pollyanna posted:

I really don't know what the gently caress I'm doing, do I?
Naw, that divisible by 2, modulo stuff is classic CS written exam "ehhh, aint I clever" poo poo. You had a pretty good handle on it.

Maybe I'm reading the question wrong, but I'd assume the largest number is always either the last number in the list, or the last number -1. There's no reason to save a list of every even number between 2->n, then take the max of the list. If you really needed to run through the whole range, you might wanna keep a running max.

code:
max = -1
for i in range(1, (int("9" * n))):
    if(new_number > max)
        max = new_number
since you now that if it's greater than the current max, it's also greater than all those other maxes you saw previously.

Just keep on keeping on! These type of exam problems will always be tough for a dude starting totally fresh, teaching himself.

Dren
Jan 5, 2001

Pillbug

Pollyanna posted:

Tried to take the Python test on E-Lance and I couldn't even get past the first question :suicide:

The question was "write a problem that takes an input "n" and returns the maximum" blah blah gently caress I don't totally remember it was something about finding the maximum number of range(1, 9xn) that is evenly divisible by n. Like, 9x2 would be "99", and the result would be "98" since that's the maximum number that's evenly divisible by 2.

I submitted an answer for that one and ragequit when it was wrong. Especially since it was taking too long for me to write it. I may have freaked out a little too harshly :( Has anyone seen that problem before? Am I retarded for not finishing with 5 minutes or am I retarded for blowing the whole thing on one fuckup? (it's the latter)

Python code:
for n in range(1, 10):
    max_number = int('9' * n)
     
    for x in range(max_number, max_number - n - 1, -1):
        if x % n == 0:
            print 'n: %d, x: %d' % (n, x)
            break
These are CS/math type problems. Maybe they are trying to weed out people who don't have a degree? You could try doing some project euler stuff. Solving those will necessitate learning stuff about math and algorithmic efficiency.

duck monster
Dec 15, 2004

Just won an old client back, and some old django I worked on about a year has been returned to me for updates, well, in updated form.

The coder they hired was a very special person.

code:
    supplierName_one = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_one = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_two = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_two = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_three = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_three = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_four = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_four = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_five = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_five = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_six = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_six = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )
    supplierName_seven = models.CharField(
        max_length=30, null=True, blank=True
    )
    supplierPrice_seven = models.DecimalField(
        max_digits=11, decimal_places=2, default=0,     
    )

...... and so on up to 20
This was latched , along with an incredible amount of other unrelated poo poo on the *User profile* model.

One problem the client has is that when someone else logs on the supplier data isn't there.

And there is so much more of this sort of horseshit. He pulled out all the nato-standard django auth I used and made his own homebaked one that stores all the user data (username/non hashed password/email/etc in profile because "cant find User model" (according to the #comment). And on it goes.

:fuckoff:

I informed the client I'm rolling that poo poo back 6 months and spending the week rewriting it and creating migration scripts to renormalize, resane-ify, and generally unfuck the whole thing.

Its like having my beloved child return after a year on the run and she's now a meth-head pregnant to corrupt cop and a rampant STD infection. Just horrible.

Mrs. Wynand
Nov 23, 2002

DLT 4EVA
Heh I just had pretty much the exact same thing happen just now. I passed the project on to a friend instead... not his baby, he can handle it.

SurgicalOntologist
Jun 17, 2004

What the hell is the deal with using pip with a python3 virtualenv? When I'm not in a virtualenv, pip installs to python2.7, pip3 to python3.3, and pip-3.3 isn't recognized. When I'm in a python3 virtualenv, pip points back to the root python2.7, pip3 points to the root python3.3, and pip-3.3 installs into the virtualenv.

Is it supposed to work this way?? pip3 outside of a virtualenv and pip-3.3 inside it?

Dominoes
Sep 20, 2007

Looking for advice on atabase / python data structure use.

I've been working with some persistent data for an algorithmic stock trader. Two basic types of data I'm saving. 1: relatively small lists (ie < 1000) of objects representing stock transactions. 2: Large collections of historical data. (hundreds of mbs)

I load the object lists from a YAML file (similar to JSON in that it mimics Python data structures, but supports objects natively) when starting the program, and periodically save over the file with updated data. I load the historical data from a sqlite database, and convert it to a 3-axis dict when the program starts. I previously saved and loaded it the same way I did with transactions, but with JSON instead of YAML. The database system seems more robust, and has an immediate advantage of being able to insert individual records, instead of periodically saving a (huge) file.

Is loading a database to a python data structure like this required for fast operation? Would it be appropriate to skip loading the historical data to a dict, and directly make database queries (SQL or an abstraction like alchemy) whenever I need data? Is it doing a hard drive pull whenever I make changes? Is it appropriate to use databases for relatively small data sets like the transactions list?

Dominoes fucked around with this message at 01:23 on Dec 15, 2013

Malcolm XML
Aug 8, 2009

I always knew it would end like this.

Dominoes posted:

Looking for advice on atabase / python data structure use.

I've been working with some persistent data for an algorithmic stock trader. Two basic types of data I'm saving. 1: relatively small lists (ie < 1000) of objects representing stock transactions. 2: Large collections of historical data. (hundreds of mbs)

I load the object lists from a YAML file (similar to JSON in that it mimics Python data structures, but supports objects natively) when starting the program, and periodically save over the file with updated data. I load the historical data from a sqlite database, and convert it to a 3-axis dict when the program starts. I previously saved and loaded it the same way I did with transactions, but with JSON instead of YAML. The database system seems more robust, and has an immediate advantage of being able to insert individual records, instead of periodically saving a (huge) file.

Is loading a database to a python data structure like this required for fast operation? Would it be appropriate to skip loading the historical data to a dict, and directly make database queries (SQL or an abstraction like alchemy) whenever I need data? Is it doing a hard drive pull whenever I make changes? Is it appropriate to use databases for relatively small data sets like the transactions list?

Pandas but 300mb is pretty tiny

QuarkJets
Sep 8, 2008

Dominoes posted:

Looking for advice on atabase / python data structure use.

I've been working with some persistent data for an algorithmic stock trader. Two basic types of data I'm saving. 1: relatively small lists (ie < 1000) of objects representing stock transactions. 2: Large collections of historical data. (hundreds of mbs)

I load the object lists from a YAML file (similar to JSON in that it mimics Python data structures, but supports objects natively) when starting the program, and periodically save over the file with updated data. I load the historical data from a sqlite database, and convert it to a 3-axis dict when the program starts. I previously saved and loaded it the same way I did with transactions, but with JSON instead of YAML. The database system seems more robust, and has an immediate advantage of being able to insert individual records, instead of periodically saving a (huge) file.

Is loading a database to a python data structure like this required for fast operation? Would it be appropriate to skip loading the historical data to a dict, and directly make database queries (SQL or an abstraction like alchemy) whenever I need data? Is it doing a hard drive pull whenever I make changes? Is it appropriate to use databases for relatively small data sets like the transactions list?

Have you considered using HDF5 for your historical data? HDF5 is a file format that works kind of like a miniature chunked file system that stores N-dimensional data with whatever dimensions or chunking sizes you prefer. HDF5 is generally faster, more robust, and easier to use than MySQL, especially in a Python environment. HDF5 files are also highly portable, so they get used a lot in the scientific community (they are obsoleting FITS files and similar flat formats). However, HDF5 lacks the power of a database, so if you want to run relational queries then that's going to require a lot more work. You can think of HDF5 as a materialized view, which is fine if you just want an archive of historical data.

Using HDF5 goes something like this:

Python code:
#Opening file myfile.hdf5
#This file has a group /Data
#A dataset /Data/mydataset
#And another dataset: /another_dataset
import h5py
#HDF5 upports similar file access flags as python's open() function:
#'w' for writing, 'r' for reading, etc
file = h5py.open("myfile.hdf5", 'r')
#HDF5 objects can be treated like dictionaries.  Here I've grabbed a dataset: 
mydataset = file['/Data/mydataset'] 
#Here I've grabbed a group (which is kind of like a folder in HDF5-speak):
mygroup = file['/Data/'] 
#This is a different way of grabbing mydataset, using a group object
mydataset_again = mygroup['mydataset'] 

#I want to do something with all datasets in the Data group
for ds in mygroup:
  #Do something

#Now I want to do something with all objects in the root of the file:
for obj in file:
  #Do something
Datasets are basically numpy arrays and are stored/accessed very efficiently. Groups are sort of like folders in a file system. You can also assign attributes to groups and datasets, which act as single-element variables for things that you don't need an array for. So you could have a group with the dataset "Foo" that has the attribute "myattribute" which is a string filled with "this is an attribute", or the attribute could be an int or a double or whatever.

For financial data, if you want 2D or 3D (or ND) datasets then you could just categorize them into various groups and be able to easily access them elsewhere.

QuarkJets fucked around with this message at 01:49 on Dec 15, 2013

SurgicalOntologist
Jun 17, 2004

I'm having virtualenvwrapper problems. For some reason running which pip is no longer changing to the virtualenv pip after running workon .... Any ideas? I ran
code:
sudo apt-get install --reinstall python-pip python3-pip
sudo pip install --upgrade virtualenvwrapper
source .bashrc # to reload virtualenvwrapper.sh
but no luck.

Note: this is unrelated to the question about pip I asked a few posts up, that behavior is on my remote machine. There, which pip properly reflects the active virtualenv.

E: poo poo. There is no pip in the virtualenv bin directory. I must have installed this env with PyCharm, and that's probably the issue.

SurgicalOntologist fucked around with this message at 04:43 on Dec 15, 2013

Pollyanna
Mar 5, 2005

Milk's on them.


What's a good way to do XML-based GUI programming in Python? I've heard of Glade, but I don't think it's unique to Python.

OnceIWasAnOstrich
Jul 22, 2006

QuarkJets posted:


Using HDF5 goes something like this:

Pandas supports HDF5 backed data structures that have worked really well for me. Have the awesome data table structure but automatically set up all the HDF5 stuff transparently.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

Pollyanna posted:

What's a good way to do XML-based GUI programming in Python? I've heard of Glade, but I don't think it's unique to Python.

Glade is an app for building GTK+ UIs. It exports it into an XML format called GtkBuilder and it's built directly into GTK+.

A simple app I maintain is Alacarte. It's a bit old, but it uses Python and GtkBuilder:

GtkBuilder file, associated Python code

I write my GtkBuilder XML files by hand, and don't use Glade, though.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I was hoping for some PyQt4 help. I am preparing a polling loop to show some analog-digital conversion on a Raspberry Pi once the right hardware shows up. So I just have a separate thread setting some progress bar widgets forwards and backwards based on a sweep of values over time. I am sure the code as-is is a big problem because that's a separate thread screwing with the GUI thread; I don't know of a GUI toolkit that tolerates having other threads jack up its widgets directly. I just don't know what the policy is supposed to be in PyQt4 for getting the GUI thread to handle these correctly. I suspect it involves signals and slots, but I can only ever find examples where, say, changing one widget changes another directly. This is pretty useless because that connection stays within the GUI thread. I need to understand how to coordinate the GUI with asynchronous behavior. What suprises me is how often this code actually succeeds, but I have had it die impressively and arbitrarily too. How should I properly set the GUI components? Here's some code:

Python code:
import sys
import math
import time
import threading
from PyQt4 import QtGui, QtCore

class ProgressBarOscillator(object):
    def __init__(self, amplitude, frequency):
        self.x = 0
        self.amplitude = amplitude
        self.frequency = frequency

    def next(self):
        self.x += 1
        return self.amplitude * math.sin(self.x * self.frequency / (2 * math.pi)) + (self.amplitude / 2.0)

    def reset(self):
        self.x = 0


def oscillation_loop(gui):

    print("Started oscillation loop")

    osc1 = ProgressBarOscillator(100, 250)
    osc2 = ProgressBarOscillator(100, 350)
    osc3 = ProgressBarOscillator(100, 450)
    osc4 = ProgressBarOscillator(100, 550)

    ########################################################################
    # This is where I am sure I need to do something else than directly hitting the GUI widgets from an external thread.
    ########################################################################
    while True:
        gui.pbar1.setValue(osc1.next())    # Bad
        gui.pbar2.setValue(osc2.next())    # Still bad
        gui.pbar3.setValue(osc3.next())    # Not getting any better
        gui.pbar4.setValue(osc4.next())    # A miracle this doesn't crash all the time
        time.sleep(0.01)


class Example(QtGui.QWidget):

    def __init__(self):
        super(Example, self).__init__()

        self.initUI()

    def initUI(self):

        self.pbar1 = QtGui.QProgressBar(self)
        self.pbar1.setGeometry(30, 10, 200, 25)
        self.pbar2 = QtGui.QProgressBar(self)
        self.pbar2.setGeometry(30, 10+30, 200, 25)
        self.pbar3 = QtGui.QProgressBar(self)
        self.pbar3.setGeometry(30, 10+60, 200, 25)
        self.pbar4 = QtGui.QProgressBar(self)
        self.pbar4.setGeometry(30, 10+90, 200, 25)

        self.setGeometry(300, 300, 280, 170)
        self.setWindowTitle('ADC sliders')
        self.show()


if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    ex = Example()

    osc_thread = threading.Thread(group=None, target=oscillation_loop, args=(ex,))
    osc_thread.start()

    print("Starting the app.exec_()")

    sys.exit(app.exec_())

Dominoes
Sep 20, 2007

QuarkJets posted:

Have you considered using HDF5 for your historical data? HDF5 is a file format that works kind of like a miniature chunked file system that stores N-dimensional data with whatever dimensions or chunking sizes you prefer. HDF5 is generally faster, more robust, and easier to use than MySQL, especially in a Python environment. HDF5 files are also highly portable, so they get used a lot in the scientific community (they are obsoleting FITS files and similar flat formats). However, HDF5 lacks the power of a database, so if you want to run relational queries then that's going to require a lot more work. You can think of HDF5 as a materialized view, which is fine if you just want an archive of historical data.

Using HDF5 goes something like this:

I read through the quickstart and high level docs for H5PY, but can't figure out how to effectively store the data, either as groups or datasets. The three axes are stock symbol, date, and attribute (ie high, close prices). Would I use groups, or datasets? If I use datasets to store the prices, how can I correlate what each array value is for? I feel like while datasets (arrays) would be fast and compact, ensuring each array values matches up with what stock, date and attribute it's for would be a disaster.

For groups would I set it up like this?
/dataset/GOOG/2013-01-01/close = 800.74
/dataset/GOOG/2013-01-01/high = 810.34
/dataset/GOOG/2013-01-02/close = 801.25

etc Would I use attributes?

Haven't tried Pandas yet.

Dominoes fucked around with this message at 16:35 on Dec 15, 2013

Adbot
ADBOT LOVES YOU

Dominoes
Sep 20, 2007

Rocko Bonaparte posted:

I was hoping for some PyQt4 help. I need to understand how to coordinate the GUI with asynchronous behavior.
You're right; the answer is Signals and slots.

Python code:

def oscillation_loop(gui):
...
    while True:
        ex.updatePbarSignal.emit(osc1.next())
        ... 
        time.sleep(0.01)


class Example(QtGui.QWidget):
    updatePbarSignal = QtCore.pyqtSignal(float)

    def __init__(self):
        super(Example, self).__init__()

        self.initUI()

    def initUI(self):

        ...
        self.pbar4 = QtGui.QProgressBar(self)
        self.pbar4.setGeometry(30, 10+90, 200, 25)

        self.updatePbarSignal.connect(lambda value: self.update_pbar(value))

        self.setGeometry(300, 300, 280, 170)
        self.setWindowTitle('ADC sliders')
        self.show()

    def update_pbar(self, value):
        self.pbar1.setValue(value)

I left out the pbars past the first to keep the example simple, but you could pass the pbar object's name through emit, connect, and the set method to keep the code compact while adding the extra pbars.

Here are the four bits of code I added. You can use them as a template for doing this in the future.
  • ex.updatePbarSignal.emit(osc1.next()) - use this in the place you'd normally update the widget with setValue() or w/e.

  • updatePbarSignal = QtCore.pyqtSignal(float) - Boilerplate that has to include the data type of the signal. You can use Python or QT data tapes. I like putting it in the GUI's class, above the __init__ method.

  • self.updatePbarSignal.connect(lambda value: self.update_pbar(value)) - Lambda's used here as a workaround; the connect command only likes a single line, and can't pass variables directly. Usually goes in the gui's __init__.

  • def update_pbar(self, value):
    self.pbar1.setValue(value)
    - You can update with the GUI in this method without triggering errors.

quote:

I suspect it involves signals and slots, but I can only ever find examples where, say, changing one widget changes another directly.
Yea - the easiest-to-find tutorials and instructions don't cover custom signals... They should.

Dominoes fucked around with this message at 14:59 on Dec 15, 2013

  • Locked thread