|
CAO Apr 2020Python code:
Python’s a high-level language that’s easy to learn, read, and write. It’s popular, has a robust collection of third-party modules, and a large community willing to help. It’s friendly to multiple programming styles: procedural and functional; object-oriented, and not. Generally, it runs slowly compared to other languages like C, Go, Java, Julia, Rust, and Haskell. Python is versatile, and is especially popular for scientific computing and web development. It’s not suitable for systems programming, and is weak when making distributable stand-alone programs. What makes Python different from other languages? -Significant whitespace. In other words, you don’t use braces to indicate blocks of code, you use indentation. -Duck typing: Python doesn’t care about the actual type of the objects you’re working with as long as the object implements the correct attributes and methods. Python is also strongly typed, meaning that you can’t can add a string, a number, and a file object together to get some guessed-at result. -Extensive standard library. Python has a split userbase between (incompatible) versions 2 and 3. Many old code bases use v2. Highlights: The versions handle character encoding differently, v3 uses lazy evaluation in the standard library when possible, and v3 includes many new features. General links
Learn
Installing Python and third-party packages There are two main paths for installing Python. If you’re new, download Anaconda; it has many common third-party packages built-in. You can install additional packages later, with its conda package manager, or using the official one, pip. You can also install Python directly from the Official downloads page.The default links on this page are for x86-versions; browse to the OS-specific pages to find the 64-bit downloads. If you’re on Linux, you probably have Python installed already – perhaps multiple versions. Using virtual environments is especially important on Linux; altering the system python may break your OS. See the Virtual environments section below. Third-party packages can be installed with the built-in package manager pip. Run pip install packagename. Or create a text file of package names, and install with pip install -r requirements.txt. Anaconda users can install most packages with conda install packagename. Consider using Poetry or pipenv to install packages, rather than using pip and virtual environments separately. Use commands like this: pipenv install requests, or pipenv install, if you already have a pipfile established. (pipenv creates and manages this file automatically). Selected third-party packages
Virtual environments Using virtual environments lets you isolate a project's dependencies from your system's Python installation. This is important on Linux, where altering the system Python can break your OS, and useful on any OS to keep packages separate and easy-to-manage. Official python virtual environment reference. Consider using pipenv instead of this directly; it combines workflows for package management and virtual environments. You can access the environment from a terminal using commands such as pipenv shell, or pipenv run, followed by a command. Anaconda has its own virtual-environment tools. Scientific computing The Scipy Stack is a collection of modules that expand python’s numerical-computing capability. They work well together, but learning when to use each package can be tricky. The Python Data Science Handbook, linked below, can help. All of these packages can be installed with conda; most with Pip. Numpy: Provides a multi-dimensional array data type that allows fast vectorized options. Great for linear algebra, and nearly all numerical computing in Python relies on it. Scipy – A collection of unrelated tools, divided into submodules. Includes modules for statistics, Fourier transforms, scientific constants, linear algebra, signal processing, image manipulation, root-finding, ODE-solvers, and specialized functions (Bessel etc) etc. If you're considering implementing a common scientific operation by hand, check Scipy first. Matplotlib– Plotting. Robust and flexible. Has two APIs, both of which are awkward. Sympy – symbolic computation. Manipulate equations abstractly, take derivatives and integrals analytically, manipulate variables. Pandas – Used in statistics and data analysis; wraps Numpy arrays with labels and useful methods. Ipython / Jupyter. Exists as three related components: Ipython terminal: A powerful improvement over the default. Ipython QtConsole: Similar to the terminal app, but has advantages provided by a GUI. Notebook: Work on pages of mixed code, LaTeX, and text on redistributable web pages. Set to be replaced by a new project called Jupyter Lab Related: Scikit-learn is a machine-learning package with a simple API. Solves classification, regression, and clustering problems with a range of techniques. Tensorflow and Theano provide neural net frameworks. Code editors
Web development Python is great for server-side web development, when paired with one of these packages:
Warning: Diving into these packages directly can be challenging if you’re new to web development, since it requires proficiency in several skills. I recommend learning Python, database basics, HTML, CSS, and Javascript independently first. Check out Tango with Django for a Django tutorial. huhu posted a Flask intro in this thread. Heroku is a service that makes hosting Python websites easy; it has free plans for development, and can quickly scale up for production use. Microcontrollers Micropython allows you to run Python on embedded systems. Additionally, Python has good library support on single-board-computers like the Rasperry Pi. Alternatives For scientific computing: Julia is fast, and has more natural syntax for math and equations. Similar syntax to Python; it’s easy to pick up one if you know the other. code:
Python code:
Pure functional programming: Haskell has a steep learning curve, and is a robust functional language. Is a high-level langauge like Python. Statistics and data analysis: R. Specialized, popular language with a large collection of stats packages. Web development: Ruby on Rails is another high-level framework with nice syntax. Complementary languages
Expanding Python into new realms
Neat specialized tutorials
(By Randall Munroe) This OP’s a community effort; PM me for updates and edits. Dominoes fucked around with this message at 16:22 on Apr 23, 2020 |
# ? Mar 6, 2017 15:36 |
|
|
# ? May 16, 2024 18:28 |
|
RIP The old thread. To open the new thread, here's something I wish I knew all those years ago: code:
|
# ? Mar 6, 2017 15:54 |
|
tef posted:RIP The old thread. Potentially a double-edged sword, because if I try to write a list of strings as in code:
|
# ? Mar 6, 2017 16:39 |
|
I don't use pylint, only the linter that's built in to PyCharm -- does pylint flag implicit string concatenation? At first glance it seems tricky to figure out when it might be intentional, since I relatively commonly doPython code:
Python code:
|
# ? Mar 6, 2017 17:10 |
|
What I normally do is type out my longass string and then find spots in it that make sense to break it up. Then I put the cursor there and press enter and PyCharm moves it all to the next line while taking care of quotes. Also, for whatever reason I find myself using long strings as function params (logging usually) more often which doesn't require the extra set of parentheses which is nice.
|
# ? Mar 6, 2017 17:24 |
|
So anaconda comes with an assload of its own libraries and binaries, and sometimes this causes problems for me. For instance, I can't just set my PATH to the anaconda/bin folder because then I'm overriding the default path for various tools in a build environment where the versioning is actually pretty important. The only solution to this is to manually downgrade anaconda packages one at a time to match the system configuration. This isn't easy or convenient. I wish there was a way to tell anaconda that I want it to use and link against the system libraries for everything that it can find I'm not really asking a question, just venting over some issues that I've been having lately
|
# ? Mar 6, 2017 23:07 |
|
QuarkJets posted:So anaconda comes with an assload of its own libraries and binaries, and sometimes this causes problems for me. For instance, I can't just set my PATH to the anaconda/bin folder because then I'm overriding the default path for various tools in a build environment where the versioning is actually pretty important. I'm having difficulty picturing your exact problem, but the general idea is why I always use miniconda...have you looked at using that?
|
# ? Mar 6, 2017 23:13 |
|
I have not, but now I will look into it
|
# ? Mar 6, 2017 23:44 |
|
Personally, even though I use miniconda I don't put its main bin folder on my path. I symlink activate and conda into my path, and then never use the root environment. So I don't get conda stuff on my path until I explicitly activate an environment.
|
# ? Mar 7, 2017 02:30 |
|
What if you're deploying code to multiple users on a common system? I usually write a shell wrapper that inline modifies the path and ld_library_path to use anaconda, if it's necessary (such as if my application links against the specific version of pythonxx.so that comes with anaconda). Things can also become very complicated if your project is using something like cmake to specify your Python area
|
# ? Mar 7, 2017 05:20 |
|
I asked about a pytest plugin to watch my test folders and automatically run a test if the files get modified and someone suggested pytest-xdist. It works great but it has the side effect of making "all tests passed" not green when I'm using the follow (-f) option. It does mark things as red though when things go wrong. Is there a way to make it also do green when good? Also are there any other neat pytest plugins I should know about? I just started using pytest so this is the first one I've used.
|
# ? Mar 7, 2017 08:47 |
|
QuarkJets posted:What if you're deploying code to multiple users on a common system? I usually write a shell wrapper that inline modifies the path and ld_library_path to use anaconda, if it's necessary (such as if my application links against the specific version of pythonxx.so that comes with anaconda). I've only had to face compilation issues with Anaconda a couple of times but I solved it by keeping everything inside Anaconda. Installed all the dependencies as Anaconda packages and then made a conda recipe for the actual compilation. Then you can distribute the precompiled conda package.
|
# ? Mar 7, 2017 14:30 |
|
Why not environment module files? They're pretty neat.
|
# ? Mar 7, 2017 15:11 |
|
https://github.com/kennethreitz/pipenv
|
# ? Mar 7, 2017 15:31 |
|
Why doesn't this work? https://repl.it/GMLO/1 code:
I guess I could just remove the property decorator and it would work, kind of. I don't want to type "thing.foo()", I just want to type "thing.foo". I could make two properties i guess, but I don't want that either. I want a single "foo" property that could return either things. What the hell is the point of the property anyway? When I google I only see tutorials for making getters and setters and none of that explains to me why I should even bother. At the moment I use @property on a class method when I want that docstring to show up in the help/pydoc for that class. Boris Galerkin fucked around with this message at 12:27 on Mar 8, 2017 |
# ? Mar 8, 2017 12:25 |
|
You're catching the TypeError before it occurs; ie once you've gotten thing.foo, you have no more error checking, and 10 doesn't have any keys. @property makes methods look like properties; I think they're for writing Java-style code. I wouldn't use them, since they mask that calculations are being run. Boris Galerkin posted:Basically I want a dict/container object that has a default return value when I don't give it an index or key or whatever. If I do then I want it to return the value of that key, which internally in my class is saved as a dictionary. Python code:
Python code:
Dominoes fucked around with this message at 21:24 on Mar 8, 2017 |
# ? Mar 8, 2017 13:12 |
There are two main reasons for @property. One is when you want to present a uniform API to your users. If you have a single method that takes only self and will always return the same result for a given object, it's "really" a property. For example, you might lazily compute the perimeter of a 2d shape when asked for, but the users don't need to know whether it's computed during __init__ or when you ask. Plus, if we only implement a 'getter' for perimeter and not a 'setter', our code won't let us create a rectangle where P! = 2(WH) well, within the limits of Python floats, anyways... This helps makes some guarantees about our rectangles being actual rectangles. The second is as a step in refactoring. Suppose we have a class Frisbee. You initially had a property, Frisbee.curve that was a public attribute. Later on, you implemented an advanced physics and materials simulation; it models the deformation of the Frisbee during its flight! That is, Frisbee.curve is no longer a static property - it can change over time! Clearly, Frisbee.curve needs to be a method that works with your physics simulation. The problem is, all over your codebase and your customers' codebases, there are calls to some_frisbee.curve, not some_frisbee.curve(). Using @property makes the change "invisible" so that the old calling signature will still work. Excessive use of @property is a warning sign of brittle code, though. At that point you might want to start raising warnings on old API usage that tell the user about your new API that more closely suits how your new code actually works. if you want to know more, there are sections in 'Effective Python', and a speech by Brett Slatkkn, the author of that book, about this. Eela6 fucked around with this message at 17:30 on Mar 8, 2017 |
|
# ? Mar 8, 2017 17:15 |
|
I'm reading about RESTful web services as they relate to Flask. In the introduction, It talks about how you might need to have multiple versions of your website because not everyone can access the new version immediately (i.e. a smartphone app won't be up to the newest version until the user downloads the new version). The smartphone app was the only example given and I'm struggling to understand what else would justify having separate versions of a website with Flask? Edit: Perhaps a better question, is there a good article/resource that explains RESTful web services well? huhu fucked around with this message at 19:23 on Mar 8, 2017 |
# ? Mar 8, 2017 19:20 |
|
You might have other sites using your service (if you're providing an API), which aren't really any different from a mobile app. Also I guess it's possible that even if you only have your own frontend accessing it, people might have 'stale' pages loaded that make calls to the old API, and there's a possibility of losing data if they can't talk to the server anymore
|
# ? Mar 8, 2017 20:12 |
|
huhu posted:I'm reading about RESTful web services as they relate to Flask. In the introduction, It talks about how you might need to have multiple versions of your website because not everyone can access the new version immediately (i.e. a smartphone app won't be up to the newest version until the user downloads the new version). The smartphone app was the only example given and I'm struggling to understand what else would justify having separate versions of a website with Flask? If you're providing an API for consumption by third parties you need a versioning and deprecation policy. If you're providing an API for consumption only by yourself, you should probably version it because it's easy to do and will help you in the future if your project grows or you open it up to others or whatever. Any decent REST API will make versioning a simple matter of setting a config variable to True. It seems a little odd to word it as having different versions of a website...I've never heard of that. Only different versions of the API. Though, with modern web development, your website is often just another consumer of your API.
|
# ? Mar 8, 2017 20:37 |
|
Thanks for the Miniconda recommendation, it's nice knowing that there is a minimalist version of anaconda out there. But for some reason I'm still having trouble with cmake; it claims to be using the HDF5 libraries and headers in my miniconda area, but it's apparently grabbing my system headers because when I launch my application I get a big gently caress-off warning about a library/header version mismatch (HDF5 at least gives a very thorough warning, as it specifies all of the paths and shows that system headers were used and that miniconda libraries are what is being linked against). This is probably going to require getting a lot more intimate with cmake and/or moc than I'd like
|
# ? Mar 8, 2017 22:40 |
|
I switched to PyCharm EAP (basically jetbrains beta program) today just for the hell of it. The first thing I've noticed is this goddamn amazing "zero latency typing" thing. I've had zero issues with typing latency in PyCharm, but its amazing how great it feels now.
|
# ? Mar 8, 2017 23:08 |
|
I tried to add code:
quote:werkzeug.routing.BuildError: Could not build url for endpoint 'auth.login'. Did you mean 'login' instead? quote:The default is login but since you don't have a view named 'login' code:
|
# ? Mar 9, 2017 04:23 |
|
QuarkJets posted:Thanks for the Miniconda recommendation, it's nice knowing that there is a minimalist version of anaconda out there. But for some reason I'm still having trouble with cmake; it claims to be using the HDF5 libraries and headers in my miniconda area, but it's apparently grabbing my system headers because when I launch my application I get a big gently caress-off warning about a library/header version mismatch (HDF5 at least gives a very thorough warning, as it specifies all of the paths and shows that system headers were used and that miniconda libraries are what is being linked against). This is probably going to require getting a lot more intimate with cmake and/or moc than I'd like I have this same problem with cmake and HDF5. I'm not sure why but I've heard from others with similar problems as well. I was able to "fix" the issue with compiling one thing by just exporting HDF5_ROOT with the cmake command: $ HDF5_ROOT=/path/to/hdf5/root cmake ... Hope that helps.
|
# ? Mar 9, 2017 10:55 |
|
Eela6 posted:if you want to know more, there are sections in 'Effective Python', and a speech by Brett Slatkkn, the author of that book, about this. you're talking about this talk i'm guessing? i liked it huhu posted:werkzeug.routing.BuildError: Could not build url for endpoint 'auth.login'. Did you mean 'login' instead? i haven't used flask in a while but i've had that exact error when using url_for - have you imported login into the auth module there? if you have the same structure as that post you linked but you're not importing login into the __init__, you probably want auth.view.login instead
|
# ? Mar 9, 2017 13:11 |
|
huhu posted:I tried to add I'm assuming that you are using Flask-Login here, but the gist of my advice should apply to you. Somewhere in the code, Flask-Login is using this code to redirect your user to your login page: Python code:
- Look for the Blueprint named auth and; - Build a url for the login endpoint in the blueprint. You are using @app.route, which means that your login endpoint is directly under the application object. There's nothing wrong with this per se, but blueprints are mostly used to organize code, which is a good thing. The biggest drawback is that, compared to the normal Flask way of creating routes, they have a bit of architectural overhead. That said, you have two options here: 1) Create an auth blueprint and put your login route in there; 2) Change Flask-Login's login route in the login manager.
|
# ? Mar 9, 2017 14:37 |
|
Dominoes posted:
As an aside, what syntax is this? I've not seen this before in Python.
|
# ? Mar 9, 2017 15:22 |
|
NtotheTC posted:As an aside, what syntax is this? I've not seen this before in Python.
|
# ? Mar 9, 2017 15:30 |
|
And I'd like to clarify: That type annotation's not accurate for the function I posted; I got confused part way through about what Boris wanted. Union[dict, int] means it could return either a dict or int. As posted, my example was really just int, or Union[Any, int] if you expect to pass it arbitrary dicts.
|
# ? Mar 9, 2017 16:00 |
|
IAmKale posted:That's Python type annotations, available since 3.5. It doesn't activate strong typing or anything, it's just a way to add in expected inputs and outputs to method definitions. The part after -> is the method's return value type. And by "expected" he means expected by Python tools. For example IDE's can use it as hints to highlight when you're trying to pass an int to a function expecting a tuple or whatever. Also, you can use MyPy to do static typing on your code.
|
# ? Mar 9, 2017 16:55 |
|
huhu posted:I tried to add You might have done this, but the login manager class has a field "login_view" that can be set, like code:
I just realized I was using the login extension wrong, I'm glad you asked this ... I've never done any web-dev before, and despite c# being my goto language I decided to follow https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world up to the point where openid gets involved. I really like using flask, sqlalchemy, the whole bit is really good when I don't really know what I'm doing python-wise. I really like writing python3 though, and it's super loving easy to read other's python3 code. I just can't handle the runtime type checking though, it makes me real uneasy. I annotate all of my functions and still gently caress everything up all of the time. The best fuckup was when I passed a string into a function that was supposed to take a list of strings and make directories out of them ... I guess I have two questions for the thread: - Is there an advantage to annotating my function definitions beyond having some documentation? - Is there a less heavy handed way to do m:n style threading in python besides using celery/rabbitmq? I'm looking for something like .Net's Task Parallel Library or whatever Go does. vvv lol now if I can only get this web app to read forum posts for me i'd be solid dougdrums fucked around with this message at 19:39 on Mar 9, 2017 |
# ? Mar 9, 2017 19:33 |
|
dougdrums posted:- Is there an advantage to annotating my function definitions beyond having some documentation? Thermopyle posted:And by "expected" he means expected by Python tools. For example IDE's can use it as hints to highlight when you're trying to pass an int to a function expecting a tuple or whatever. Also, you can use MyPy to do static typing on your code. For example, PyCharm uses type annotations to save you from yourself.
|
# ? Mar 9, 2017 19:37 |
|
I like being able to get an idea of what the function does by looking at its signature.
|
# ? Mar 9, 2017 20:41 |
|
The one thing I don't like about more advanced typing annotations is that they can make the signature too long and noisy and those are the signatures they're most useful with. Syntax highlighting and other IDE features help with that, though.Python code:
|
# ? Mar 9, 2017 22:38 |
|
Thermopyle posted:The one thing I don't like about more advanced typing annotations is that they can make the signature too long and noisy and those are the signatures they're most useful with. Syntax highlighting and other IDE features help with that, though. I know you know better, but one might suggest you could either get rid of the type hints or clean up that signature with a better function name.
|
# ? Mar 9, 2017 22:40 |
|
Dominoes posted:I like being able to get an idea of what the function does by looking at its signature. Also makes compiling a function with Numba a good deal easier having the signature right there
|
# ? Mar 9, 2017 22:42 |
|
Tigren posted:I know you know better, but one might suggest you could either get rid of the type hints or clean up that signature with a better function name. Yes, it was just an example of what can happen.
|
# ? Mar 9, 2017 23:13 |
|
I've just been doing like:code:
|
# ? Mar 10, 2017 00:27 |
|
dougdrums posted:I've just been doing like: Yeah, that's basically what I do
|
# ? Mar 10, 2017 00:51 |
|
|
# ? May 16, 2024 18:28 |
|
dougdrums posted:You might have done this, but the login manager class has a field "login_view" that can be set, like Thank you so much.
|
# ? Mar 10, 2017 01:09 |