Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
If I want to import say just sqrt from numpy I could do

code:

from numpy import sqrt

But what do I do if I want to import just sqrt but keep the same numpy namespace?

code:

from numpy import sqrt as numpy_sqrt
from numpy import sqrt as numpy.sqrt

The first one works but I'd rather have it called as numpy.sqrt still. The second one doesn't seem to work.

Adbot
ADBOT LOVES YOU

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
Is numpy vs np a religious thing like vim vs emacs?

e: I guess if you're forced to 79 characters (lololol what year is this) then that makes sense yeah.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

QuarkJets posted:

That's technically correct, which is the best kind of correct. Kind of like how a bicycle is basically a Ferrari, your smart phone is basically a supercomputer, or how genital herpes is basically a cold sore

(I'm a scientific computing guy so JS is a total non-starter; but I have messed around with Javascript integration in a Qt application for funsies, which is kind of cool because it makes it not too hard to create your C++ objects from Javascript. But you can also just do Python integration, which I've found is way more powerful)

Do you have any good resources for scientific computing with python you could recommend? I'm a scientific computing guy too and until a few months ago I've been working exclusively with Fortran and C. But now I've/we've switched to a new software package that's built on C++ but with python interfaces. I've managed to jump into python and get things running smoothly though the first few weeks were really confusing especially cause even though I "knew" what a class/oop was I didn't really know how to use them. Basically I've been reading a lot of "how to do things the python way" which are okay but I haven't found anything that was more on the lines of "how to do things the python way for fortran programmers with no oop experience."

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

QuarkJets posted:

Like many/most scientific computing people, I'm self-taught (because even graduate-level scientific computing courses are mostly worthless), so I don't really know of any good resources are for learning OOP with scientific programming. But I'm pretty sure they exist! I'd recommend asking in the scientific/maths computing thread

For using Python you'll want to be aware of Numpy, which gives bindings to most of the good Fortran matrix math functions. It sounds like you're using a package that uses C++ for the lifting and Python as a glue language, which is really common and often very effective. Be aware that loops in Python are much slower than loops in C/Fortran, so the goal is to either write clever Numpy expressions or to export computationally-heavy loops to C/C++/Fortran (eg adding 1 to every element in an NxN array with Python for loops is slow, but the one-liner Numpy expression to do this is as fast as C/Fortran). The best advice is probably to write code in Python without caring much about efficiency, and then if speed is an issue start looking at how to make things faster; preemptive optimization is a trap and usually a waste of time.

Since you come from Fortran/C, just think of a class as a special type of structure, in that it can hold both variables (attributes) and functions (methods).

Got any examples for this numpy stuff?

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
I'm trying to learn/use Sphinx but I swear every single tutorial I've found is completely worthless and leaves more questions than answers. For example I just want to do something simple like this:

code:
def foo():
    """Some information about this function."""

    ...

    # Something very important about the next line:
    two = 1 + 1

    ....

    return two
I can get Sphinx to generate documentation for foo() containing that bit in the """ """, but I want it to include also the comments in #. How can I do that?

Another seemingly simple but difficult thing I want to do is something like: I have a dict with some parameters. I want sphinx to document each key, and I want it to take whatever I already wrote in inline comments.

code:
some_dict_of_parameters = {
    'a_flag':          #< `a_flag` controls something and does something else
        True,        
    'some_other_flag': #< `some_other_flag` controls the blah blah blah.
        False,
}
In doxygen I think I could do this with the #< (or eg !< for fortran) like I show. I want a similar feature in sphinx because the other alternatives I can think of is copying and pasting all the keys into a docstring placed above/after that dictionary and that's just stupid.

Boris Galerkin fucked around with this message at 00:21 on Dec 22, 2016

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

HardDiskD posted:

Do you know any OSS project that does what you want in their documentation? I never came across this before.

Yes, doxygen lets me do this. For example in fortran syntax, I can do normal comments that won't show up in the auto generated documentation with the normal '!' prefix, but I can also choose to use '!<' and doxygen will include that particular comment line. For my second example, standard fortran doesn't really have dicts so you could use something like a derived type or whatever.

code:

type :: parameters
     int :: a_flag !> info about this flag
     int :: a_different_flag !> info about this one

     ! this is a regular comment like that doxygen will ignore

end type parameters

And in the documentation it automagically documents each member "key" without me having to copy and paste it and put it all into a docstring.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

Have you taken a look at autodata (need to scroll down to autodata). You can do the following:
Python code:

class Foo:
    """Docstring for class Foo."""

    #: Doc comment for class attribute Foo.bar.
    #: It can have multiple lines.
    bar = 1

    flox = 1.5   #: Doc comment for Foo.flox. One line only.

    baz = 2
    """Docstring for class attribute Foo.baz."""

    def __init__(self):
        #: Doc comment for instance attribute qux.
        self.qux = 3

        self.spam = 4
        """Docstring for instance attribute spam."""

Hmm ok, will try this again tomorrow. I did some searching and i did see some examples using the "#: etc" syntax and I tried it, but it did nothing. Does "autodata" not get enabled by default? Is being able to explicitly tell sphinx to put this comment line in the documents not a common thing for python developers?

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

Usage guidance is really meant to go into the docstring. Comments are there to help the understand the code, and shouldn't really be separated from the code.

If you want to have the autodata fields show up, you have to request them in the .rst files, or modify the autosummary templates to include the information by default.

It's starting to sound like sphinx isn't the right tool for this case then. Not knocking on it, but in this case I don't want to separate usage vs what the code does because the developers are going to be the only users. I want high level information (this function solves df(x)/dx = exp(a(x)*q(x) - 1), where a(x) = ...) but I also want selected comments to appear both in the documentation with latex or latex like rendered math that show what I actually implemented/am solving since the actual equation implemented might not resemble the one in the docstring, and for it to appear in the code next to the relevant lines.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

accipter posted:

You can definitely make Sphinx work for what you want to do, but it isn't standard practice. The biggest issue that I see with what you are trying to do is that including a # comment in the documentation seems worthless to me without the code associated with that comment. If you agree, then it would be easy to include links to highlighted source code using sphinx.ext.viewcode. This would allow a user to browse the API and readily access the source code to see implementation. If you are set on including some # comments within the generated documentation that is possible with Sphinx and how you accomplish that depends on how you are generating the documentation. If you are using autodoc and/or autosummary, you need to modify the templates to include the autodata field. This would take about 10 minutes to modify the templates.

I do agree and that viewcode extension looks like it might work out. I'll give it a shot thanks.

BigRedDot posted:

Anyway, it sort of sounds like you are looking for a "literate programming" tool for Python. If that's the case you might find Pycco more suited.

That seems way too verbose without looking into the configuration. I'm going to give the viewcode extension a shot.

Unrelated question, tell me if I'm doing this wrong:

I have a module called message_tools.py that contains some functions for, well, messaging/output. Is it wrong to add this to my __init__.py?

code:
# __init__.py

import mypackage.message_tools as msg    # dir(mypackage) now shows both message_tools and msg
del mypackage.message_tools                         # i don't want message_tools to show up
I don't want to rename message_tools.py to msg.py because it's not immediately obvious looking at the filename 'msg.py' what it contains, but in the code itself mypackage.msg.warn() is much more obvious, and mypackage.message_tools.warn() just seems too long.

e: I made a setup.py for this package and got it working smoothly, except that the PKG INFO file says 'PLATFORM: UNKNOWN'. I tried adding the classifier for OS :: POSIX :: Linux but that didn't change the PLATFORM string. What do I need to add to fix that?

Boris Galerkin fucked around with this message at 23:15 on Dec 23, 2016

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
I'm documenting some tutorials/examples with Sphinx and for each tutorial (it's a python module/script) I'm writing the documentation at the module level docstring and then using automodule in my docs/source directory to get it generated.

I want to include the actual code into the HTML page that gets generated. How do I do that? I tried with 'literalinclude' but this, well, literally includes also its docstring which I don't want. I guess I could just specify which lines to include but it would be better if it would automatically just include everything below the module level docstring.

Secondly is there a variable I can tell the docstring to include itself (with its path automatically expanded)? Right now I have to do "literalinclude: ../../tutorials/.." and so on. I'd like to just do something like "literalinclude: $SELF".

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

VikingofRock posted:

Hey so what's the state of the art way of doing packaging? I want something where I can run some command(s) on the command line and it'll download all the right dependencies for my script, download the right version of python, and run the script. But it seems like there are a million different tools that each do a part of this and it's not really clear to me how they intersect.

I'm not an expert either but I think these two links should help:

https://packaging.python.org/distributing/
https://github.com/pypa/sampleproject

If you clone the sampleproject you should be able to customize it to your own package and then just run a pip install . from your project directory. You should be able to configure it to download Python dependencies as well.

I think if you want to manage non-Python dependencies then you'll have to use Conda/Anaconda, like "download the right version of python." Though I gotta wonder why do you think you need to manage the actual Python installation yourself? That just seems unnecessarily complicated and unless you specifically need to do for whatever reason that then wouldn't it be better off just testing your script against different Python versions? The sampleproject I linked has some scripts/tools that do just that for you, automatically.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
I wanna use virtualenv like all the cool kids but one of the python packages/libraries I'm using isn't managed by pip. It's something I need to compile myself with cmake and is linked against the systemwide Intel MKL and other dependencies. It doesn't show up when I type "pip list" but is importable/usable.

How can I use virtualenv --no-site-packages to get a clean environment, but with this one package included/available? Can I just symlink something to my virtualenv folder (and what)?

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
Do I use 3 or 4 spaces in my docstrings if I'm using sphinx?

e: logging chat:

If I use a third party package to build an app, and that third party package has a logger already set up, should I just hook into their logger or should I need to create my own and keep them separated? The info that their logger outputs is useful to me; my extra log messages are just additional things I've added for my app.

Boris Galerkin fucked around with this message at 16:01 on Mar 4, 2017

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Thermopyle posted:

Create your own logger and logging config.

Okay thanks, will do.



Another question:

I've been trying out pytest and it's really neat. My test codes look much cleaner than with unittest, but what I like the most is the colored red/green output. I do something like TDD so I'm constantly alt-tabbing or tmux pane switching etc to another terminal to re-run tests. Is there already an established way to tell pytest something like "monitor test folder x and re-run your tests every single time source code in folder y changes"?

And going further (but maybe too Vim specific) is there a way to have pytest color my vim status bar red/green depending on status so I don't even have to look over to the other terminal unless I need to read the status messages?

e: what would be even better is if it would add information to my vim status line, just something short like: "b3,r66; b8,r23", telling me that one test failed in buffer 3 on line 66 so I could just switch my vim buffer directly to "3" and then see which test failed.

Boris Galerkin fucked around with this message at 15:34 on Mar 5, 2017

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
Should also put in big bold letters at the top when the OP was last updated and keep it updated. IMO the problem with these huge threads/ops is that I click one one and see it posted in 20xx and think "hmm okay I'm just gonna assume everything is outdated."

Adbot
ADBOT LOVES YOU

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

VikingofRock posted:

Okay, that's more similar to how I thought it was (I'm also in science academia). Mostly to me it seems like academics use python 2 because that's what's provided by default by Mac OS, CentOS, and Ubuntu, and that's like 99% of what your code is going to be running on.

That's my experience as well in academia. Outside of a very narrow group of people and/or field I honestly don't think that academics even know there's a python 2 and python 3, all they know is there is a "python" with numpy and sympy etc and that's that. At the end of the day there's basically only two things that matter: 1) does my code work, and 2) does it run on my laptop and/or hpc cluster. Most people aren't going to care about writing reusable library code to do the things they need to do. They're perfectly fine with one giant script.py file where they manually go in and comment/uncomment out lines of code to run different scripts and such. Either that or old school style script_do_x.py and script_do_x_with_different_y.py, where they are just copies of each other save for some inputs.

That's non-CS academia for you.

  • Locked thread