Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Linear Zoetrope: Nov 28, 2011; A hero must cook

What's the best tool for documenting Python 3 for an open source project (there are multiple languages in this project, but let's assume I'm gluing together the docs from different languages manually)? It looks like there are several options, and I'm not familiar enough with the Python open source world to know what's standard. I feel like I rarely see the builtin PyDoc, and see sphinx referenced more often, but I don't really know the tradeoffs and pros/cons of all the various tools.

# ? Sep 20, 2017 20:26

Adbot: ADBOT LOVES YOU

# ? May 16, 2024 17:29

accipter: Sep 12, 2003

Linear Zoetrope posted:

What's the best tool for documenting Python 3 for an open source project (there are multiple languages in this project, but let's assume I'm gluing together the docs from different languages manually)? It looks like there are several options, and I'm not familiar enough with the Python open source world to know what's standard. I feel like I rarely see the builtin PyDoc, and see sphinx referenced more often, but I don't really know the tradeoffs and pros/cons of all the various tools.

I would suggest Sphinx and ReadTheDocs. I am not sure how well that works for not automatic documentation.

# ? Sep 20, 2017 22:07

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Sphinx is the way to go here unless you're looking to generate API docs and you like how Slade (or whatever) looks like. Though you can definitely modify the Sphinx output pretty endlessly.

# ? Sep 20, 2017 22:58

mbt: Aug 13, 2012

pyautogui question

I'm trying to automate some dumb work thing and I want to setup a mouse to raster across a screen very quickly

moveTo works great except I need it to run a function if the pixel under the cursor changes. As far as I can see nothing can be checked while moveTo is currently moving (with a duration of 0.5). My only other option is to use moveTo but divide the screen into squares and have it check every time it moves to another square, but that is far far too slow.

Any ideas? I sure hope it's me being bad at programming

# ? Sep 21, 2017 18:23

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Meyers-Briggs Testicle posted:

pyautogui question

I'm trying to automate some dumb work thing and I want to setup a mouse to raster across a screen very quickly

moveTo works great except I need it to run a function if the pixel under the cursor changes. As far as I can see nothing can be checked while moveTo is currently moving (with a duration of 0.5). My only other option is to use moveTo but divide the screen into squares and have it check every time it moves to another square, but that is far far too slow.

Any ideas? I sure hope it's me being bad at programming

If moveto doesn't offer any sort of callback functionality the only thing you can do is move in steps.

# ? Sep 21, 2017 19:12

Dominoes: Sep 20, 2007

Hey dudes, I set up some Powershell scripts to automate installing a calculator-like virtual env.

Running the setup script does the following:
-Creates a virtual environment in the directory you downloaded the scripts
-Installs the scipy stack, PyQt, and Spyder. (Downloads Scipy and its numpy+MKL prereq from Chris Gohlke's site, then the rest from PyPi. See this for why this is currently necessary; something to do with a Fortran compiler license)
-Setups up shortcuts that run Qtconsole or Spyder in that env.

I made bash scripts to do the same, but can't test it since my Ubuntu version has broken venvs. Might try setting up a special ipython config to automatically import numpy as np etc, but haven't figured out how to do that on a per-env or per-instance basis. Stack Overflow post I made describing my troubles here.

Related: Spyder feels appropriate for this type of use; I didn't give it a chance before, but it's great for writing and running one-off scripts for solving math and science problems.

Dominoes fucked around with this message at 16:21 on Sep 22, 2017

# ? Sep 22, 2017 02:47

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I just know we've got data and science types ITT, so I thought I'd mention that the new PyCharm EAP has a scientific mode.

edit: oh I forgot a link: https://blog.jetbrains.com/pycharm/2017/09/pycharm-2017-3-eap-1/

Thermopyle fucked around with this message at 15:12 on Sep 22, 2017

# ? Sep 22, 2017 13:58

Dominoes: Sep 20, 2007

Sick!

# ? Sep 22, 2017 17:58

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Here's a pretty elite variation on Python.

quote:

Often, a function will require that parameters have certain desirable properties. A function in Vigil can state what it requires by using implore:
Python code:
def square_root(n):
    implore n >= 0
    return math.sqrt(n)
If a caller fails to provide valid arguments, it is wrong and must be punished.

and

quote:

This is where Vigil sets itself apart from weaker languages that lack the courage of their convictions. When a Vigil program is executed, Vigil itself will monitor all oaths (implorations and swears) that have been made. If an oath is broken, the offending function (the caller in the case of implore and the callee in the case of swear) will be duly punished.

How?

Simple: it will be deleted from your source code.

Eternally morally vigilant.

# ? Sep 22, 2017 19:36

The Fool: Oct 16, 2003

This was my favorite line from that:

quote:

Vigil deleted a function. Won't that cause the functions that call it to fail?

It would seem that those functions appear to be corrupted as well. Run Vigil again and it will take care of that for you. Several invocations may be required to fully excise all bugs from your code.

# ? Sep 22, 2017 19:55

Linear Zoetrope: Nov 28, 2011; A hero must cook

Thermopyle posted:

Here's a pretty elite variation on Python.

This is not elite, merely just.

# ? Sep 22, 2017 20:33

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Linear Zoetrope posted:

This is not elite, merely just.

I stand corrected!

# ? Sep 22, 2017 20:44

QuarkJets: Sep 8, 2008

Burn the Heretic. Kill the Mutant. Purge the Unclean.

... using assertions

# ? Sep 22, 2017 22:38

pmchem: Jan 22, 2010

Thermopyle posted:

Here's a pretty elite variation on Python.

and

Eternally morally vigilant.

haha that's loving great, should be added to Python 3 IMO

# ? Sep 22, 2017 23:05

Proteus Jones: Feb 28, 2013

Thermopyle posted:

Here's a pretty elite variation on Python.

and

Eternally morally vigilant.

These file descriptions are fantastic

# ? Sep 23, 2017 00:57

Sockser: Jun 28, 2007; This world only remembers the results!

What's the go-to tool for compiling a python app into an eve these days? Is it still py2exe or has something better and cooler shown up on the scene?

# ? Sep 23, 2017 04:17

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

Sockser posted:

What's the go-to tool for compiling a python app into an eve these days? Is it still py2exe or has something better and cooler shown up on the scene?

http://nuitka.net/

# ? Sep 23, 2017 14:34

Dominoes: Sep 20, 2007

Hey dudes, looking for database-design/Django wisdom. I'm storing and retrieving tabular data. Ie there are ~60 people, ~20 things to track per person. I have a table that has rows foreign-keying to a person, an item, and a few bits of info like dates. This means I'm querying 1200 rows of this each page load... it's bogging the server down to the point where the site takes minutes to load or times out. Is there a better way to store this type of data? I imagine I could fix it by serializing a dict of data in a text field or JSON field associated with each person (items as keys, dates etc as values), but I suspect this is frowned up. What do you suggest?

# ? Sep 24, 2017 00:38

Ahz: Jun 17, 2001; PUT MY CART BACK? I'M BETTER THAN THAT AND YOU! WHERE IS MY BUTLER?!

Dominoes posted:

Hey dudes, looking for database-design/Django wisdom. I'm storing and retrieving tabular data. Ie there are ~60 people, ~20 things to track per person. I have a table that has rows foreign-keying to a person, an item, and a few bits of info like dates. This means I'm querying 1200 rows of this each page load... it's bogging the server down to the point where the site takes minutes to load or times out. Is there a better way to store this type of data? I imagine I could fix it by serializing a dict of data in a text field or JSON field associated with each person (items as keys, dates etc as values), but I suspect this is frowned up. What do you suggest?

Should profile your view with something like the Django debug tools or whatever they're called. There's no point guessing where the bottleneck is if you have a exponential algorithm for querying rows. 1200 should be fairly simple with the right query.

# ? Sep 24, 2017 04:58

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Yeah, you probably just need to finesse your query. Might have to use raw SQL if the ORM isn't expressive enough.

# ? Sep 24, 2017 14:49

Cingulate: Oct 23, 2012; by Fluffdaddy

A while ago, I complained about dict(this_will_become_a_str=var) syntax. Somebody told me I was silly because dict is highly convergent with regular kwargs. I have in the meantime understood your point, and it has essentially made me more comfortable using **kwargs well.

# ? Sep 25, 2017 11:39

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Cingulate posted:

A while ago, I complained about dict(this_will_become_a_str=var) syntax. Somebody told me I was silly because dict is highly convergent with regular kwargs. I have in the meantime understood your point, and it has essentially made me more comfortable using **kwargs well.

I love it when this happens to me.

Someone will explain something to me and I'll be thinking "sure, whatever dude" and then a year later I'm like "gently caress that dude was right, I apologize for dismissing you under my breath".

# ? Sep 25, 2017 17:33

Dominoes: Sep 20, 2007

Thermopyle posted:

Yeah, you probably just need to finesse your query. Might have to use raw SQL if the ORM isn't expressive enough.

Fixed it. Apparently there's a huge performance penalty for doing:

Python code:

 [gt for gt in ground_trainings if gt.person == person and gt.item == training_item]

instead of:

Python code:

 [gt for gt in ground_trainings if gt.person_id == person.id and gt.item_id == training_item.id]

Seems like another good ROT is when in doubt, use normal Python code to filter your initial query results; re-filtering/getting can cause extra DB pulls.

Dominoes fucked around with this message at 22:05 on Sep 25, 2017

# ? Sep 25, 2017 20:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I can't really do it for you without having access to your codebase and database, but I'm 95% sure you can do that purely in the DB with the ORM or failing that raw sql. Probably be less resource intensive too.

The Django rule of thumb is don't do filtering in python when your highly-optimized database software can do it for you. Databases are built for filtering.

Though, like all rules of thumb, there's quite a bit of leeway there dependent upon your data structure, data size, and machine/network resource constraints.

# ? Sep 25, 2017 22:33

Dominoes: Sep 20, 2007

Thermopyle posted:

I can't really do it for you without having access to your codebase and database, but I'm 95% sure you can do that purely in the DB with the ORM or failing that raw sql. Probably be less resource intensive too.

The Django rule of thumb is don't do filtering in python when your highly-optimized database software can do it for you. Databases are built for filtering.

Though, like all rules of thumb, there's quite a bit of leeway there dependent upon your data structure, data size, and machine/network resource constraints.

How can I do this efficiently in Django query language? people is a query result.

Python code:

matching_people = (person for person in people if person.last_name.lower() == last_name and person.first_name[0].lower() == first_initial)
person = next(matching_people)

It's not this:

Python code:

person = people.get(last_name__iexact=last_name, first_name__istartswith=first_initial)

Another example:
Fast:

Python code:

matches = (i for i in items if i.pex_task_id == item['Task ID'])
item_db = next(matches)

Slow:

Python code:

item_db = items.get(pex_task_id=item['Task ID'])

Dominoes fucked around with this message at 01:21 on Sep 26, 2017

# ? Sep 25, 2017 22:39

Ahz: Jun 17, 2001; PUT MY CART BACK? I'M BETTER THAN THAT AND YOU! WHERE IS MY BUTLER?!

Dominoes posted:

How can I do this efficiently in Django query language? people is a query result.

I think you need to post your generated SQL to better evaluate why one is fast vs. slow. There's no way a straight up "select x from y where z = abc" should be slower than "select * from y"(+ post-processing in python) in a real DBMS

# ? Sep 26, 2017 15:40

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Ahz posted:

I think you need to post your generated SQL to better evaluate why one is fast vs. slow. There's no way a straight up "select x from y where z = abc" should be slower than "select * from y"(+ post-processing in python) in a real DBMS

I agree with this.

Dominoes posted:

Slow:

Python code:

item_db = items.get(pex_task_id=item['Task ID'])

This should be almost your most basic SQL statement ever and should be faster (or at least there should be a negligible difference) than fetching all items from the database and then iterating over them in python.

It almost makes me think there's a problem with your database. Are you using Postgres or what?

# ? Sep 26, 2017 16:20

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I came across this neat theme for PyCharm, and because I'm always fiddling with stuff, I decided to try it out.

It's called Material Theme and it's pretty great and customizable.

I'm actually most impressed by the fact that it's very thorough and doesn't seem to break anything...which is, IME, not par for the course when it comes to open source themes for stuff.

I also added the Code Glance plugin (just search the Plugins in Settings) which adds the "minimap" of the code in your files like SublimeText does. It also seems to work well without slowdowns or anything.

Look to the right side in this screenshot:

# ? Sep 26, 2017 16:47

Proteus Jones: Feb 28, 2013

Thermopyle posted:

I came across this neat theme for PyCharm, and because I'm always fiddling with stuff, I decided to try it out.

It's called Material Theme and it's pretty great and customizable.

I'm actually most impressed by the fact that it's very thorough and doesn't seem to break anything...which is, IME, not par for the course when it comes to open source themes for stuff.

I also added the Code Glance plugin (just search the Plugins in Settings) which adds the "minimap" of the code in your files like SublimeText does. It also seems to work well without slowdowns or anything.

Look to the right side in this screenshot:

I like the look of that Material Theme. I'll have to give it a go later.

I've been rocking Code Glance for a while, and that minimap is worth it's weight in gold.

From the screen shot I see you're on a Mac. Did you upgrade yet? Any gotchas or caveats you found with Python (specifically Anaconda if you use it) or PyCharm?

# ? Sep 26, 2017 16:51

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Proteus Jones posted:

From the screen shot I see you're on a Mac. Did you upgrade yet? Any gotchas or caveats you found with Python (specifically Anaconda if you use it) or PyCharm?

Nah, I'm on Windows, I just stole those screenshots from elsewhere.

# ? Sep 26, 2017 16:53

Ahz: Jun 17, 2001; PUT MY CART BACK? I'M BETTER THAN THAT AND YOU! WHERE IS MY BUTLER?!

Thermopyle posted:

I came across this neat theme for PyCharm, and because I'm always fiddling with stuff, I decided to try it out.

It's called Material Theme and it's pretty great and customizable.

I'm actually most impressed by the fact that it's very thorough and doesn't seem to break anything...which is, IME, not par for the course when it comes to open source themes for stuff.

I also added the Code Glance plugin (just search the Plugins in Settings) which adds the "minimap" of the code in your files like SublimeText does. It also seems to work well without slowdowns or anything.

I've been using this theme for a while. Make sure you get the latest fork, there are two in the repo.

# ? Sep 26, 2017 17:45

creatine: Jan 27, 2012

Does anybody know of a python3 module that let's you work with .FCS files? All the ones I can find are 2.7 only

# ? Sep 28, 2017 00:12

huhu: Feb 24, 2006

creatine posted:

Does anybody know of a python3 module that let's you work with .FCS files? All the ones I can find are 2.7 only

Perhaps you could do a .FCS -> .txt with another tool and go from there? If .FCS doesn't have formatting, you could probably just do with open -

http://www.pythonforbeginners.com/files/with-statement-in-python

# ? Sep 28, 2017 02:56

Foxfire_: Nov 8, 2010

creatine posted:

Does anybody know of a python3 module that let's you work with .FCS files? All the ones I can find are 2.7 only

https://github.com/eyurtsev/fcsparser claims to do it, I've only ever used FlowCytometryTools (which is 2.7 only)

Depending on what exactly you need, it's not too horrible to write something yourself to yank out the data and compensation matrix.

# ? Sep 28, 2017 06:16

creatine: Jan 27, 2012

Foxfire_ posted:

https://github.com/eyurtsev/fcsparser claims to do it, I've only ever used FlowCytometryTools (which is 2.7 only)

Depending on what exactly you need, it's not too horrible to write something yourself to yank out the data and compensation matrix.

Yeah I just need data from it to try tSNE analysis with it. It's not critical to our work, it's more a side project at work for me.

huhu posted:

Perhaps you could do a .FCS -> .txt with another tool and go from there? If .FCS doesn't have formatting, you could probably just do with open -

http://www.pythonforbeginners.com/files/with-statement-in-python

Thanks I'll give this a shot if the other package doesn't work.

# ? Sep 28, 2017 11:40

PoizenJam: Dec 2, 2006; Damn!!!
It's PoizenJam!!!

I use PsychoPy to run Psychology experiments. Right now, I'm doing an experiment where participants view a word in multiple different fonts. Here's my problem: The fonts need to be markedly different between presentations, but the letters themselves should always appear in the same spot. So while common monospaced fonts are far too similar to one another, variable spaced fonts tend to result in a much smaller word width with smaller spacing between words.

In my case, I use 5 letter words with Lucida Console for one presentation, and Brush Script for the other. Is there any way in Python or PsychoPy that I can fix/justify the text display so the width and/or spacing of these two fonts are the same? Save for manually setting the X/Y coordinates of each individual letter?

I'm displaying text in my Experiment using TextStim.

# ? Sep 28, 2017 22:26

Cingulate: Oct 23, 2012; by Fluffdaddy

JVNO posted:

I use PsychoPy to run Psychology experiments. Right now, I'm doing an experiment where participants view a word in multiple different fonts. Here's my problem: The fonts need to be markedly different between presentations, but the letters themselves should always appear in the same spot. So while common monospaced fonts are far too similar to one another, variable spaced fonts tend to result in a much smaller word width with smaller spacing between words.

In my case, I use 5 letter words with Lucida Console for one presentation, and Brush Script for the other. Is there any way in Python or PsychoPy that I can fix/justify the text display so the width and/or spacing of these two fonts are the same? Save for manually setting the X/Y coordinates of each individual letter?

I'm displaying text in my Experiment using TextStim.

Displaying the letters individually is probably gonna be very simple.

code:

## boilerplate here
from psychophy import visual

x_pos = (-3, -1, +1, +3)
def display_word(word, x_pos, font):
    for x, character in zip(word, x_pos):
        visual.TextStim(win, text=x, font=font', pos=(x_pos, 0.0)).draw()
    win.flip()

display_word("code", x_pos, this_font)

or something like that.

# ? Sep 28, 2017 22:36

PoizenJam: Dec 2, 2006; Damn!!!
It's PoizenJam!!!

If I'm understanding that correctly- you built a function to display the word that iterates through the letters and places them in fixed coordinates? I can certainly give that solution a try, and it was my fallback method. Just seems somewhat inelegant- you'd think there'd be a 'Justify Text' setting so I could set the size of the bounding box and forget it.

Oh well! Thanks regardless :v:

Edit: Is there any way I can get Python to return the coordinates of the letters already in my study, or will this be trial and error until I replicate the spacing of Lucida Console? I'd like to fix the new font to those locations.

PoizenJam fucked around with this message at 22:46 on Sep 28, 2017

# ? Sep 28, 2017 22:44

Ellie Crabcakes: Feb 1, 2008; Stop emailing my boyfriend Gay Crungus

JVNO posted:

Edit: Is there any way I can get Python to return the coordinates of the letters already in my study, or will this be trial and error until I replicate the spacing of Lucida Console? I'd like to fix the new font to those locations.

Not really, since the advance width is not likely to be identical between glyphs. If you want absolute, pixel-perfect alignment, there are a lot of considerations to take into consideration, because working with fonts is loving terrible.

# ? Sep 28, 2017 23:25

Adbot: ADBOT LOVES YOU

# ? May 16, 2024 17:29

PoizenJam: Dec 2, 2006; Damn!!!
It's PoizenJam!!!

Well gently caress me, cause creating 1440 pixel perfect images of all the word stimuli I used would be a horrifying endeavour. I'll either deal with the weird spacing, or use Cingulate's method :

PoizenJam fucked around with this message at 01:27 on Sep 29, 2017

# ? Sep 29, 2017 01:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »