Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Eela6
May 25, 2007
Shredded Hen
I am unreasonably excited for python 3.6. f-strings! Asynchronous generators and comprehensions! Underscores in numeric literals! Preservation of argument order in class instantiation and **kwargs! Low-level optimizations I don't fully understand!

Python is a great language that's getting better all the time. It's certainly not perfect and it's not appropriate for everything, but it's so good most of the time. I enjoy progamming in python and the python community and I hope you all do too.

Adbot
ADBOT LOVES YOU

Baby Babbeh
Aug 2, 2005

It's hard to soar with the eagles when you work with Turkeys!!



I use .format() probably way more than I should, so I'm pretty hyped for f-strings. Can't wait to write the lovely, un-maintainable code ya'll are talking about.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

The great thing for me is that I'm almost always in charge of my software stack so I can switch to 3.6 on day one!

Feral Integral
Jun 6, 2006

YOSPOS

Baby Babbeh posted:

I use .format() probably way more than I should, so I'm pretty hyped for f-strings. Can't wait to write the lovely, un-maintainable code ya'll are talking about.

Is the f-string thing just a shorthand for .format()?

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
I'm trying to learn/use Sphinx but I swear every single tutorial I've found is completely worthless and leaves more questions than answers. For example I just want to do something simple like this:

code:
def foo():
    """Some information about this function."""

    ...

    # Something very important about the next line:
    two = 1 + 1

    ....

    return two
I can get Sphinx to generate documentation for foo() containing that bit in the """ """, but I want it to include also the comments in #. How can I do that?

Another seemingly simple but difficult thing I want to do is something like: I have a dict with some parameters. I want sphinx to document each key, and I want it to take whatever I already wrote in inline comments.

code:
some_dict_of_parameters = {
    'a_flag':          #< `a_flag` controls something and does something else
        True,        
    'some_other_flag': #< `some_other_flag` controls the blah blah blah.
        False,
}
In doxygen I think I could do this with the #< (or eg !< for fortran) like I show. I want a similar feature in sphinx because the other alternatives I can think of is copying and pasting all the keys into a docstring placed above/after that dictionary and that's just stupid.

Boris Galerkin fucked around with this message at 00:21 on Dec 22, 2016

huhu
Feb 24, 2006
Has anyone ever been able to pin a python script to the start menu?

Master_Odin
Apr 15, 2010

My spear never misses its mark...

ladies
Are f-strings going to be back ported in some way to older versions of Python or is it 3.6 only?

Space Kablooey
May 6, 2009


Boris Galerkin posted:

I'm trying to learn/use Sphinx but I swear every single tutorial I've found is completely worthless and leaves more questions than answers. For example I just want to do something simple like this:
*snip*

Do you know any OSS project that does what you want in their documentation? I never came across this before.

Eela6
May 25, 2007
Shredded Hen

Master_Odin posted:

Are f-strings going to be back ported in some way to older versions of Python or is it 3.6 only?

As far as I'm aware 3.6 and above only.

accipter
Sep 12, 2003

Boris Galerkin posted:

I'm trying to learn/use Sphinx but I swear every single tutorial I've found is completely worthless and leaves more questions than answers. For example I just want to do something simple like this:

code:
def foo():
    """Some information about this function."""

    ...

    # Something very important about the next line:
    two = 1 + 1

    ....

    return two
I can get Sphinx to generate documentation for foo() containing that bit in the """ """, but I want it to include also the comments in #. How can I do that?

Another seemingly simple but difficult thing I want to do is something like: I have a dict with some parameters. I want sphinx to document each key, and I want it to take whatever I already wrote in inline comments.

code:
some_dict_of_parameters = {
    'a_flag':          #< `a_flag` controls something and does something else
        True,        
    'some_other_flag': #< `some_other_flag` controls the blah blah blah.
        False,
}
In doxygen I think I could do this with the #< (or eg !< for fortran) like I show. I want a similar feature in sphinx because the other alternatives I can think of is copying and pasting all the keys into a docstring placed above/after that dictionary and that's just stupid.

Have you taken a look at autodata (need to scroll down to autodata). You can do the following:
Python code:
class Foo:
    """Docstring for class Foo."""

    #: Doc comment for class attribute Foo.bar.
    #: It can have multiple lines.
    bar = 1

    flox = 1.5   #: Doc comment for Foo.flox. One line only.

    baz = 2
    """Docstring for class attribute Foo.baz."""

    def __init__(self):
        #: Doc comment for instance attribute qux.
        self.qux = 3

        self.spam = 4
        """Docstring for instance attribute spam."""

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

HardDiskD posted:

Do you know any OSS project that does what you want in their documentation? I never came across this before.

Yes, doxygen lets me do this. For example in fortran syntax, I can do normal comments that won't show up in the auto generated documentation with the normal '!' prefix, but I can also choose to use '!<' and doxygen will include that particular comment line. For my second example, standard fortran doesn't really have dicts so you could use something like a derived type or whatever.

code:

type :: parameters
     int :: a_flag !> info about this flag
     int :: a_different_flag !> info about this one

     ! this is a regular comment like that doxygen will ignore

end type parameters

And in the documentation it automagically documents each member "key" without me having to copy and paste it and put it all into a docstring.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

Have you taken a look at autodata (need to scroll down to autodata). You can do the following:
Python code:

class Foo:
    """Docstring for class Foo."""

    #: Doc comment for class attribute Foo.bar.
    #: It can have multiple lines.
    bar = 1

    flox = 1.5   #: Doc comment for Foo.flox. One line only.

    baz = 2
    """Docstring for class attribute Foo.baz."""

    def __init__(self):
        #: Doc comment for instance attribute qux.
        self.qux = 3

        self.spam = 4
        """Docstring for instance attribute spam."""

Hmm ok, will try this again tomorrow. I did some searching and i did see some examples using the "#: etc" syntax and I tried it, but it did nothing. Does "autodata" not get enabled by default? Is being able to explicitly tell sphinx to put this comment line in the documents not a common thing for python developers?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Feral Integral posted:

Is the f-string thing just a shorthand for .format()?

We just talked about it on the previous page.

Space Kablooey
May 6, 2009


Boris Galerkin posted:

Yes, doxygen lets me do this. For example in fortran syntax, I can do normal comments that won't show up in the auto generated documentation with the normal '!' prefix, but I can also choose to use '!<' and doxygen will include that particular comment line. For my second example, standard fortran doesn't really have dicts so you could use something like a derived type or whatever.

code:

type :: parameters
     int :: a_flag !> info about this flag
     int :: a_different_flag !> info about this one

     ! this is a regular comment like that doxygen will ignore

end type parameters

And in the documentation it automagically documents each member "key" without me having to copy and paste it and put it all into a docstring.

I meant any projects that were generated with Sphinx

accipter
Sep 12, 2003

Boris Galerkin posted:

Hmm ok, will try this again tomorrow. I did some searching and i did see some examples using the "#: etc" syntax and I tried it, but it did nothing. Does "autodata" not get enabled by default? Is being able to explicitly tell sphinx to put this comment line in the documents not a common thing for python developers?

Usage guidance is really meant to go into the docstring. Comments are there to help the understand the code, and shouldn't really be separated from the code.

If you want to have the autodata fields show up, you have to request them in the .rst files, or modify the autosummary templates to include the information by default.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

Usage guidance is really meant to go into the docstring. Comments are there to help the understand the code, and shouldn't really be separated from the code.

If you want to have the autodata fields show up, you have to request them in the .rst files, or modify the autosummary templates to include the information by default.

It's starting to sound like sphinx isn't the right tool for this case then. Not knocking on it, but in this case I don't want to separate usage vs what the code does because the developers are going to be the only users. I want high level information (this function solves df(x)/dx = exp(a(x)*q(x) - 1), where a(x) = ...) but I also want selected comments to appear both in the documentation with latex or latex like rendered math that show what I actually implemented/am solving since the actual equation implemented might not resemble the one in the docstring, and for it to appear in the code next to the relevant lines.

accipter
Sep 12, 2003

Boris Galerkin posted:

It's starting to sound like sphinx isn't the right tool for this case then. Not knocking on it, but in this case I don't want to separate usage vs what the code does because the developers are going to be the only users. I want high level information (this function solves df(x)/dx = exp(a(x)*q(x) - 1), where a(x) = ...) but I also want selected comments to appear both in the documentation with latex or latex like rendered math that show what I actually implemented/am solving since the actual equation implemented might not resemble the one in the docstring, and for it to appear in the code next to the relevant lines.

You can definitely make Sphinx work for what you want to do, but it isn't standard practice. The biggest issue that I see with what you are trying to do is that including a # comment in the documentation seems worthless to me without the code associated with that comment. If you agree, then it would be easy to include links to highlighted source code using sphinx.ext.viewcode. This would allow a user to browse the API and readily access the source code to see implementation. If you are set on including some # comments within the generated documentation that is possible with Sphinx and how you accomplish that depends on how you are generating the documentation. If you are using autodoc and/or autosummary, you need to modify the templates to include the autodata field. This would take about 10 minutes to modify the templates.

BigRedDot
Mar 6, 2008

accipter posted:

You can definitely make Sphinx work for what you want to do, but it isn't standard practice.
Echoing this. I can almost guarantee I've written more Sphinx extensions than anyone here. You can make Sphinx do almost any crazy thing but you have to decide if the cost is justified. As an example Bokeh is based around a declarative system, where everything is determined by the values of trait-like properties on classes that automagically handle type validation and serialization. There are so many of them that it absolutely makes sense to have the documentation for the property with the property (anything else would be a maintenance nightmare). But I had to write a Sphinx extension to help with that, because there's nothing built into Sphinx that does anything like what we needed. The corollary to all of this is also that writing Sphinx extensions is fairly hard, because (irony!) the Sphinx and Docutils docs are atrocious. I had to trawl through both projects source code for a few days to really understand the zen of them.

Anyway, it sort of sounds like you are looking for a "literate programming" tool for Python. If that's the case you might find Pycco more suited.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

You can definitely make Sphinx work for what you want to do, but it isn't standard practice. The biggest issue that I see with what you are trying to do is that including a # comment in the documentation seems worthless to me without the code associated with that comment. If you agree, then it would be easy to include links to highlighted source code using sphinx.ext.viewcode. This would allow a user to browse the API and readily access the source code to see implementation. If you are set on including some # comments within the generated documentation that is possible with Sphinx and how you accomplish that depends on how you are generating the documentation. If you are using autodoc and/or autosummary, you need to modify the templates to include the autodata field. This would take about 10 minutes to modify the templates.

I do agree and that viewcode extension looks like it might work out. I'll give it a shot thanks.

BigRedDot posted:

Anyway, it sort of sounds like you are looking for a "literate programming" tool for Python. If that's the case you might find Pycco more suited.

That seems way too verbose without looking into the configuration. I'm going to give the viewcode extension a shot.

Unrelated question, tell me if I'm doing this wrong:

I have a module called message_tools.py that contains some functions for, well, messaging/output. Is it wrong to add this to my __init__.py?

code:
# __init__.py

import mypackage.message_tools as msg    # dir(mypackage) now shows both message_tools and msg
del mypackage.message_tools                         # i don't want message_tools to show up
I don't want to rename message_tools.py to msg.py because it's not immediately obvious looking at the filename 'msg.py' what it contains, but in the code itself mypackage.msg.warn() is much more obvious, and mypackage.message_tools.warn() just seems too long.

e: I made a setup.py for this package and got it working smoothly, except that the PKG INFO file says 'PLATFORM: UNKNOWN'. I tried adding the classifier for OS :: POSIX :: Linux but that didn't change the PLATFORM string. What do I need to add to fix that?

Boris Galerkin fucked around with this message at 23:15 on Dec 23, 2016

VikingofRock
Aug 24, 2008





Hey so I finally got around to watching this talk, and it was pretty good! Thanks for linking it.

Kuule hain nussivan
Nov 27, 2008

I'm going to be making a very simple local db for a small organisation, but couldn't find any SQL frontends for querying only. I thougt I could mock one up in Python relatively quickly, but what's the go to framesork for simple UIs?

QuarkJets
Sep 8, 2008

TKinter (based on TK) is native to Python and lets you make really simple UIs really fast. For a simple querying-only interface it'll probably work in a pinch and won't take long to write

PyQt or PySide if you want it to look nicer / you want to do more complicated stuff, and especially if you want the user to actually modify fields

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Hey BigRedDot: I just noticed the link to the weather source code here goes to a 404 on Github.

QuarkJets
Sep 8, 2008

What's the word on fastest plotting in Python? I read that matplotlib with the TkAgg backend is probably the fastest way to dump plots, does anyone know for sure?

BigRedDot
Mar 6, 2008

Thermopyle posted:

Hey BigRedDot: I just noticed the link to the weather source code here goes to a 404 on Github.

Ah thanks, that got fixed on the front page of the demo site but the links in the gallery there are separate.

Xarn
Jun 26, 2015

QuarkJets posted:

What's the word on fastest plotting in Python? I read that matplotlib with the TkAgg backend is probably the fastest way to dump plots, does anyone know for sure?

No idea on speed, but recently I started using Plotly, for reasons like "actually reasonable to install" and its not bad.

No Safe Word
Feb 26, 2005

Python using the Go runtime? Why not: https://opensource.googleblog.com/2017/01/grumpy-go-running-python.html

Baby Babbeh
Aug 2, 2005

It's hard to soar with the eagles when you work with Turkeys!!



What's the point of that? Is Go concurrency really that much better than asyncio that you'd give up access to some of python's most powerful libraries to use it?

more like dICK
Feb 15, 2010

This is inevitable.
I feel like "real world" scenarios is only true within Google if they're not including c extensions.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Baby Babbeh posted:

What's the point of that? Is Go concurrency really that much better than asyncio that you'd give up access to some of python's most powerful libraries to use it?

I'd really like to see some more in-depth discussion of this.

I mean, they'd have to move to Python 3+ to get asyncio, but it seems like that'd be less work than what they did, so maybe Go concurrency really is that much better? Also, there's other lightweight thread alternatives like eventlet/gevent...

Of course, Google also has a real hard-on for Go, so maybe it's just the new and shiny factor?

Cingulate
Oct 23, 2012

by Fluffdaddy
Shouldn't this work?

code:
$python --version
Python 3.5.2 :: Anaconda custom (x86_64)

$conda install python=3.6
Fetching package metadata ...........
Solving package specifications: ....


UnsatisfiableError: The following specifications were found to be in conflict:
  - conda -> python 2.7*|3.3*|3.4*|3.5*
  - python 3.6*
Use "conda info <package>" to see the dependencies for each package.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Cingulate posted:

Shouldn't this work?

code:
$python --version
Python 3.5.2 :: Anaconda custom (x86_64)

$conda install python=3.6
Fetching package metadata ...........
Solving package specifications: ....


UnsatisfiableError: The following specifications were found to be in conflict:
  - conda -> python 2.7*|3.3*|3.4*|3.5*
  - python 3.6*
Use "conda info <package>" to see the dependencies for each package.

I think you have to install it in an environment. The base conda install is python 3.5 and you can't change it.

(I dont do a lot with conda other than set up my environments so this post is mostly speculation)

Eela6
May 25, 2007
Shredded Hen
Not until Anaconda updates to python 3.6 by default, which will happen by the end of the month.

For now, you can create a new environment for python 3.6.
code:
conda create -n py36 python=3.6
then activate it
windows code:
code:
activate py36
linux code:
code:
source activate py36

more like dICK
Feb 15, 2010

This is inevitable.
Last year Dave Beazly gave a talk about removing the GIL, and mentioned that moving to fine grained locks (which is what GrumPy does) had a big performance regression on single threaded code, making it a no-go for mainline CPython.

The chart in the blog post seems to indicate that GrumPy has this same regression, but they obviously don't have to stick the same guarantees as the CPython team.

Cingulate
Oct 23, 2012

by Fluffdaddy
Ah okay, thanks. I will just wait then.

I didn't want to make a new env because I have tons of packages and I don't think there is a simple way of migrating my py3.5 packages to a py3.6 env right?

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Thermopyle posted:

I'd really like to see some more in-depth discussion of this.

I mean, they'd have to move to Python 3+ to get asyncio, but it seems like that'd be less work than what they did, so maybe Go concurrency really is that much better? Also, there's other lightweight thread alternatives like eventlet/gevent...

Of course, Google also has a real hard-on for Go, so maybe it's just the new and shiny factor?

I bet it's more work to take a big codebase like YouTube to 3+ than to write a compiler that'll drop-in replace 2.

The fib test they cite just used threads, which would underperform an approach that spun off processes (last I checked, on Windows, for something that uses lots of memory). I'm not familiar enough with how Go handles that sort of thing to know if it's a good comparison, but I'm guessing it's weak and meant to say "Look, ma: no GIL!"

They should have included comparisons to the other alternatives they explored to make a good case, but at the end of the day we're not the audience they need to justify this to.

Dex
May 26, 2006

Quintuple x!!!

Would not escrow again.

VERY MISLEADING!
from some stuff i looked at last year, go's concurrency for cpu intensive threads was around 50x cpython's. however, go has some seriously nasty leaks that are completely beyond your ability to control, so you will almost certainly end up with memory issues that are near impossible to isolate, so if you have even the slightest reason to care about memory consumption, it seems like a terrible choice

i'd aim for pypy before running python in go if i was willing to make tradeoffs on certain libs since it's apparently significantly more performant than cpython, but i don't actually need to so i've only done some light reading on it

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Dex posted:

from some stuff i looked at last year, go's concurrency for cpu intensive threads was around 50x cpython's. however, go has some seriously nasty leaks that are completely beyond your ability to control, so you will almost certainly end up with memory issues that are near impossible to isolate, so if you have even the slightest reason to care about memory consumption, it seems like a terrible choice

i'd aim for pypy before running python in go if i was willing to make tradeoffs on certain libs since it's apparently significantly more performant than cpython, but i don't actually need to so i've only done some light reading on it

Oh that's interesting and now you've got me wishing for a Grumpy vs IronPython cage match since neither gets C extensions.

My money's on IPY because I know I've seen CPU-heavy code run better there than CPython and it has a really good GC behind it :getin:

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Cingulate posted:

Ah okay, thanks. I will just wait then.

I didn't want to make a new env because I have tons of packages and I don't think there is a simple way of migrating my py3.5 packages to a py3.6 env right?

I always use pip to install stuff in my conda environments, so if that was the case for you, I'd do:

code:
pip freeze > requirements.txt

....create new conda 3.6 environment...

pip install -r requirements.txt
I assume there's some way to do the same sort of thing with conda-installed packages...

Adbot
ADBOT LOVES YOU

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Munkeymon posted:

I bet it's more work to take a big codebase like YouTube to 3+ than to write a compiler that'll drop-in replace 2.

Yeah, after I thought about it some more I think you're probably right. I've literally never attempted to move a python 2 project to python 3...

  • Locked thread