Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›16 »

Dominoes: Sep 20, 2007

Nippashish posted:

Its a convention that more or less everyone follows, like using snake_case instead of camelCase for variables or naming classes with PascalCase instead of something else. I've never encountered anyone with strong opinions about it though.

Technique I use: import numpy as np standard for the most part. If I'm doing anything with messy math, I'll import the funcs I need separately, ie from numpy import sqrt, sin, cos, arctan2, pi, e etc to make equations easier to read.

# ¿ Dec 1, 2016 05:43

Adbot: ADBOT LOVES YOU

# ¿ May 8, 2024 21:38

Dominoes: Sep 20, 2007

Hey dudes, running into issues scraping this image: http://www.feynmanlectures.caltech.edu/img/FLP_I/f16-01/f16-01_tc_big.svgz

Getting 403s, despite having a user agent header. Ie:

Python code:

url = 'http://www.feynmanlectures.caltech.edu/img/FLP_I/f16-01/f16-01_tc_big.svgz'
requests.get(url, headers={'User-agent': 'Mozilla/5.0'}, stream=True)

Any ideas?

# ¿ Dec 6, 2016 21:11

Dominoes: Sep 20, 2007

If you click the link above, it'll work. These guys have complained about people scraping the site before, so I suspect they're doing something deliberate; can't figure it out!

# ¿ Dec 6, 2016 23:44

Dominoes: Sep 20, 2007

taqueso posted:

Clicking that link 403s for me. Maybe it would work if you set the referrer header to their site?

That works - thanks much!

Dominoes fucked around with this message at 00:28 on Dec 7, 2016

# ¿ Dec 7, 2016 00:09

Dominoes: Sep 20, 2007

mystes posted:

Can you even do this in MyPy? If OrderedDict is just a normal class, isn't there no way for MyPy to understand that it should actually take type parameters? Surely this is why you need the separate "typing" namespace?

Nailed it; it's the same reason you important List, Dict etc from typing, and use those rather than the builtin list, dict.

I'm don't know if there's a clean solution to this. You could probably call it a Dict and things would mostly work out. (although you're losing some of the self-documentation value; add a comment?). Or leave it as OrderedDict, but skip defining the key/val types.

Here's a SA post with another thought: post

# ¿ Dec 19, 2016 04:15

Dominoes: Sep 20, 2007

Hey dudes; trying to run a java file from Python, using subprocess. It's not doing anything. The most basic example:

Python code:

subprocess.run([java], shell=True, check=True)

This should show the java help page, but is returning CalledProcessError: Command '['java']' returned non-zero exit status 1. The docs mention this, but I still don't get it. Without check=True, it doesn't do anything. The actual code I'm trying to run looks like this:

Python code:

subprocess.run(['java', '-jar', 'mathtoweb.jar'], shell=True)

# ¿ Jan 12, 2017 20:36

Dominoes: Sep 20, 2007

Thansk for the subprocess words. Got it sorting using that advice!

# ¿ Jan 14, 2017 15:26

Dominoes: Sep 20, 2007

Anecdote of the current state of installing Python packages that include C/Fortran code in windows with pip instead of conda:

The following packages work without a problem (Ie I loaded them in a requirements.txt, then 'ran pip install -r requirements.txt'.)

numpy
pandas
matplotlib
jupyter
sympy
PyQt5
keras
requests
requests_oauthlib
pytest
toolz
cytoolz
beautifulsoup4
django-toolbelt

The following required Chris Gohlke's site:

scipy
scikit-learn
h5py
tables

Numba installs from pip only after installing llvmlite from CG's site.

Dominoes fucked around with this message at 12:05 on Jan 21, 2017

# ¿ Jan 21, 2017 00:25

Dominoes: Sep 20, 2007

pdfminer works well enough, but is slow.

# ¿ Jan 21, 2017 17:07

Dominoes: Sep 20, 2007

No Safe Word posted:

code:
>>> year, month, day, _, _, _, _, _, _ = tuple(datetime.datetime.now().utctimetuple())
>>> year, month, day, unused, unused, unused, unused, unused, unused = tuple(datetime.datetime.now().utctimetuple())
Contrived but you pick which one is better. You generally shouldn't have unused stuff at all, and the rare cases where it is used is because you need a terse way to skip it. A single, obviously-intended-to-not-be-used character, does that job more effectively than a word. And it's already internationalized to boot!

This is the way to handle it:

code:

>>> year, month, day, *rest = tuple(datetime.datetime.now().utctimetuple())

*what Ela said.

# ¿ Feb 2, 2017 00:56

Dominoes: Sep 20, 2007

Hey dudes. How do I make pip work with Ubuntu on Python 3.6? Ie pretend I'm an idiot and need basic instructions.

I've been using Anaconda for a while on Windows and Ubuntu. Thought I'd try 3.6 on Windows, so uninstalled Anaconda, and got 3.6 up and running smooth by installing packages with pip from a requirements.txt, and a few from Chris Gohlke's site.

Trying on Ubuntu, now, and it's a mess. There's a beta in the official packages you can run with 'python3.6', and a source dist on python's site that I can get sort of working (make works, but make install crashes, and it sort of works, but the files are in th wrong place?), and there are some unofficial repos. In all cases, I can't get pip to work with the 3.6 install. Any ideas, other than going back to Anaconda? The main issue seems to be having 3 version of Python floating around.

The command 'python3.6 -m pip...' results in an error: 'ModuleNotFoundError: No module named 'pip._vendor'

edit: After experimenting with the default 3.5 setup, the amount of packages that won't install with pip's about the same as on Windows! Rather than CG's site, these can be installed with 'sudo apt install'.

Packages that install with pip in Win and Ubuntu:

code:

# Scipy stack
numpy  # mkl??
pandas
matplotlib
jupyter
sympy


# Science extras
pandas-datareader
numba  # once llvmlite's installed from a binary
scikit-image
pillow
seaborn


# Misc
requests
requests_oauthlib
pytest
toolz
cytoolz
beautifulsoup4
lxml
sqlalchemy

Packages that install with pip on Ubuntu but not Win:

code:

scikit-learn
h5py

Packages that install with pip on Win but not Ubuntu:

code:

PyQt5
psycopg2

Scipy won't install on either.

Dominoes fucked around with this message at 15:42 on Feb 3, 2017

# ¿ Feb 3, 2017 13:56

Dominoes: Sep 20, 2007

Eela6 posted:

Scipy is notoriously finicky. My honest advice is to start with anaconda and expand on it with pip from there. I know that's not what you want to hear, but it's probably the easiest solution. The good news is Anaconda is finally up and running with Python 3.6!

You're right, that would be the easiest solution. This is more of a curiosity. Since switching to stock Python on Win, I appreciate having one package man instead of 2. I've gotten burned a few times with Anaconda when trying to manually update one package that breaks another, and Qt version headaches. It's a wash either way.

Dominoes fucked around with this message at 12:53 on Feb 4, 2017

# ¿ Feb 4, 2017 12:47

Dominoes: Sep 20, 2007

Thx. It installs fine from CG's site on Win, and sudo apt install on Ubuntu; I was hoping it'd work with pip, now that pip supports binary wheels.

# ¿ Feb 4, 2017 15:46

Dominoes: Sep 20, 2007

A few highlights:
-Built-in functions tend to return iterators rather than lists. Ie range(), .items(), map() etc.
-Division doesn't auto-round.
-string encoding is handled differently
-print is a function
-Python 3 supports a number of new features.

I'd just dive in, and fix things as they break. You can try the new features once you're comfortable with the changes.

Dominoes fucked around with this message at 22:02 on Feb 7, 2017

# ¿ Feb 7, 2017 21:59

Dominoes: Sep 20, 2007

Hey dudes. Looking to see if there are pre-built solutions that would help for a project I'm going to undertake for work. I'm trying to build a scheduler app for a flying club. In early planning stages. Currently, we have a white board, and magnetic name tags we shift around. The schedule changes frequently, and there are different qualifications, training requirements etc each person maintains that place constraints on the scheduling. Subject to chaos, people getting sick, last-minute changes etc.

Thought 1: Scheduling? Aren't there a bunch of already made solutions, since it's such a common task?

Thought 2: This seems like a unique use case, with many non-standard variables; roll your own rather than fit a square peg in a round hole.

Thoughts?

# ¿ Feb 9, 2017 22:52

Dominoes: Sep 20, 2007

Munkeymon posted:

Sounds like you want a rules engine.

Thanks, homes. From what I gather so far, sounds like I need to roll my own, STS!!! Django.

# ¿ Feb 11, 2017 02:44

Dominoes: Sep 20, 2007

Rosalind - Python's a general-purpose language, so the tools you need for things like statistics will be included in third-party packages. Finding the right package, and info on how to use it can be confusing, as you've found out. Like Quarkjets said, start with Anaconda, since the packages you need will already be included.

The packages you need in this case are pandas (to read the CSV, and do some stats), and scipy.stats, and statsmodels for the anaysis. Post what you're specifically looking for, or post equivalent R code, and we'll give you example code. Translating between Python and R is usually easy.

Dominoes fucked around with this message at 11:29 on Feb 11, 2017

# ¿ Feb 11, 2017 11:12

Dominoes: Sep 20, 2007

Munkeymon posted:

There aren't some already kicking around out there?

I dunno; no luck yet.

# ¿ Feb 13, 2017 20:05

Dominoes: Sep 20, 2007

Hey dudes. What are the best ways to learn Python, for someone with no experience? I learned from Codeacademy, which was OK. The OP looks out of date.

Dominoes fucked around with this message at 17:16 on Feb 27, 2017

# ¿ Feb 27, 2017 17:05

Dominoes: Sep 20, 2007

Isn't that by the guy who ranted about Python 3 in the old V, then justified it recently with odd reasoning?

Dominoes fucked around with this message at 19:13 on Feb 27, 2017

# ¿ Feb 27, 2017 19:09

Dominoes: Sep 20, 2007

Thanks for the Udacity rec. The guy looking for the rec is new to programming, other than a C++ course in college a decade ago.

# ¿ Feb 27, 2017 19:28

Dominoes: Sep 20, 2007

onionradish posted:

It is; I'd avoid him on principle based on how he lashed out against critics, aside from suspecting anything he says as being out of date since he's been so anti-Python3. Suddenly, he'd be worth listening to?

Automate the Boring Stuff is probably the most-recommended starter since it emphasizes doing something practical with the language.

Other sources I found useful when learning:

PyCharm as an IDE to kick me in the pants on style and provide easy access to method calls

Reddit's /learnpython wiki

Reddit's /learnpython and /Python groups; /learnpython has a mix of beginner questions and lazy student questions, which means anyone can ask beginner questions without standing out, or see how many of the existing questions you already know the answer to - it's something a beginner outgrows, which is a decent milestone

PyVideo.org, which has hundreds of video sessions from Python conferences, ranging from brief overviews of modules, to 3+ hour tutorials on topics from basic Python, to good Pythonic code (Hettinger, et al), to heavily-specific scientific topics (using modules for DNA analysis, real-time bidding for online ads, etc.)

DevDocs.io to copy a local, offline, browser-accessible version of Python docs (as well as bunch of other languages)

Thx! Sent him the ATBS link; seems perfect. I'm recommending Anaconda since it has all the packages.

Eela6 posted:

BTW, should we make a new Python thread? This one is pretty crusty. It might be nice to have an updated OP with some of this material.

Congrats, you just nominated yourself. I bet you can't have it up by this time tmw.

# ¿ Feb 27, 2017 19:43

Dominoes: Sep 20, 2007

Some highlights you could put in:

-Tutorials and instructional books
-Strengths and weaknesses (ie easy, versatile, sucks at distributable standalone progs and inappropriate for sys programming)
-Installation guides (eg Anaconda vs pip wheels / linux packages / Chris Gohlke's installers)
-IDEs
-Scipy-stack info
-Popular packages (requests, pytz, sqlalchemy, beautifulsoup, toolz etc)
-GUI (Qt5, Kivy, TKinter etc)
-Web dev section, with a link to the Django thread
-Resources (like onion's reddit recommendations)
-Alternatives (ie R for stats, Julia for numerical things, Ruby for web dev)
-Neat projects (micropython, numba, pypy etc)
-Good articles and notebooks (Kalman filters, Data Science Handbook)

Dominoes fucked around with this message at 20:55 on Feb 27, 2017

# ¿ Feb 27, 2017 20:45

Dominoes: Sep 20, 2007

Eela6 posted:

I can only type with one hand because of health issues, so it will take a little longer than that. But I'm happy to do it.

Thermopyle posted:

I've got a new thread for Python pretty much ready to go from a couple years ago, I just never got around to posting it.

I'm going to do it if y'all don't.

# ¿ Mar 5, 2017 17:51

Dominoes: Sep 20, 2007

Ha! Just clicked this as I'm drafting one; we have the same idea about the opener! Merging them now.

Dominoes fucked around with this message at 23:27 on Mar 5, 2017

# ¿ Mar 5, 2017 23:23

Dominoes: Sep 20, 2007

Draft.
CAO March 2017

Python code:

import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

About
Python�s a high-level language featuring code that�s easy to learn, read, and write. It�s popular, has a robust collection of third-party modules, and a large community willing to help. It�s friendly to multiple programming styles; procedural and functional; object-oriented, or not. Generally, it runs slowly compared to other languages; eg C, Java, Julia, Rust, and Haskell.

Python is versatile, and is especially popular for scientific computing and web development. It�s not suitable for systems programming, and is weak when dealing with distributable stand-alone programs.

What makes Python different from other languages?
-Significant whitespace. In other words, you don�t use braces to indicate blocks of code, you use indentation.
-Duck typing. �If it walks like a duck�.�What this means is that Python doesn�t care about the actual type of the objects you�re working with as long as the object implements the correct attributes and methods.�Note that Python is also strongly typed, meaning that you can�t can add a string, a number, and a file object together to get some guessed-at result.
-Batteries included.�Python comes with an extensive standard library.

General links
Official docs
Style guide
Official third-party package repository
Pyhon subreddit, Subreddit for beginners

Learn
Automate the Boring Stuff Dive into small projects
Codeacademy provides interactive exercises.
MIT OpenCourseware
Think Python

Installing Python and third-party packages
There are two main paths for installing Python. If you�re new, download Anaconda; it has many common third-party packages built-in. You can install additional packages later, with its conda package manager, or using the official one, pip. Enthought Canopy is similar to Anaconda.

You can also install Python directly from the Official downloads page.The default links on this page are for x86-versions; browse to the OS-specific pages to find the 64-bit downloads.

If you�re on Linux, you probably have Python installed already � perhaps multiple versions� be careful!

Third-party packages can be installed with the built-in package manager pip. Just run pip install packagename. Or create a text file of package names, and install with pip install -r requirements.txt. Packages that include code from other languages like C may work this way, or you may have to install using your system package manager (ie in Linux), or installers from This page on Windows. Anaconda users can install most packages with conda install packagename.

Selected third-party packages

Requests - for making web requests with easy syntax
Pytest - unit tests; cleaner than the built-in module.
SQLalchemy - for working with databases.
BeautifulSoup - for parsing HTML and XML. Recommended over a built-in alternative
Websockets � Live communication betweeen browser and server.
PyQt5 Lets you use the C++ GUI library Qt with Python
. Steep learning curve, but QT is a powerful, well-documented way to make desktop and mobile GUIs. Used by Ipython Qtconsole, and Spyder.
Toolz - functional programming tools like currying, accumulate, take, compose, and pipe. Some of this functionality's included in the built-in itertools and functools libraries, or is easily created from them. You should check those libraries out too!

Virtual environments
Check out virtualenv and related tools. Virtualenv is a tool to create isolated Python environments.

The introduction explains what it is and why you want want it. Many people use virtualenvwrapper to make working with virtualenvs easier.Virtualenv-burrito makes setting up the two easy. Anaconda has its own virtual-environment tools.

Scientific computing
The Scipy Stack is a collection of modules that expand python�s numerical-computing capability. They work well together, but learning when to use each package can be tricky. The Python Data Science Handbook, linked below, can help. All of these packages can be installed with conda; most with Pip.

Numpy: Provides a multi-dimensional array data type that allows fast vectorized options. Great for linear algebra, and nearly all numerical computing in Python relies on it.

Scipy � A collection of unrelated tools, divided into submodules. Includes modules for statistics, Fourier transforms, scientific constants, linear algebra, signal processing, image manipulation, root-finding, ODE-solvers, and specialized functions (Bessel etc) etc If you're considering implementing a common scientific operation by hand, check Scipy first.Documentation here

Matplotlib� Plotting. Robust and flexible. Has two APIs, both of which are awkward.

Sympy � symbolic computation. Manipulate equations abstractly, take derivatives and integrals analytically, manipulate variables.

Pandas � Used in statistics and data analysis; wraps Numpy arrays with labels and useful methods.

Ipython / Jupyter. Exists as three related components: Ipython terminal: A powerful improvement over the default. Ipython QtConsole: Similar to the terminal app, but has advantages provided by a GUI. Notebook: Work on pages of mixed code, LaTeX, and text on redistributable web pages. Set to be replaced by a new project called Jupyter Lab

Related: Scikit-learn is a machine-learning package with a simple API. Solves classification, regression, and clustering problems with a range of techniques. Tensorflow and Theano provide neural net frameworks.

IDEs
PyCharm Robust and full-featured. Its community edition is free, and works well for Python. Its Profession edition costs ~$60/year, and includes addition features like web-development tools. Free for students and contributors to open-source projects.

Spyder is targeted at scientific computing, and is simpler than PyCharm. Can be installed with pip, and is included with Anaconda.

If you'd prefer a text-editor with language-specific features, try Visual Studio Code or Atom

We development
Python is great for server-side web development, when paired with one of these packages:
Django � Batteries-included and popular. Intimidating to start with, and involves many files working together, but includes most of what you need to build a website. Extensive docs, and many people who can help on StackOverflow.

Flask Minimalist, and easy to start with. Customizable. As your projects grow, you�ll likely want to add other modules for things like database management, migrations, admin, and authentication.

Pyramid and Pylons are other popular frameworks.This page shows an overview of what's available.

Warning: Diving into these packages directly can be challenging if you�re new to web development, since it requires proficiency in several skills. I recommend learning Python, database basics, HTML, CSS, and Javascript independently first.

Heroku is a service that makes hosting Python websites easy; it has free plans for development, and can quickly scale up for production use.

Alternatives
For scientific computing: Julia is fast, and has more natural syntax for math and equations. Similar syntax to Python; it�s easy to pick up one if you know the other.

code:

f(x) = 2x^2

Python code:

f = lambda x: 2 * x**2

Pure functional programming: Haskell has a steep learning curve, and is a robust functional language. Is a high-level langauge like Python.

Statistics and data analysis: R. Specialized, popular language with a large collection of stats packages.

Web development: Ruby on Rails is another high-level framework with nice syntax.

Complementary languages
C: Python runs on C, so it's a natural language to write high-speed extensions in.
HTML, CSS, Javascript, and JQuery for web development.
SQL, if you use Python to manage databases.

Expanding Python into new realms
PyPy is a Just-in-time (JIT) compiler for Python that allows a subset of the language to run very fast.

Numba is another way to speed up Python to near-C-speeds with a JIT. By applying a decorator, can make python functions run much faster; but limits which parts of the language you can use in these functions. Usually requires writing out loops manually, where otherwise you might used vectorized code.

Micropython allows you to code custom microcontrollers with Python.

Neat specialized tutorials
Python Data Science Handbook � Introduction to scientific programming in Python.
Kalman and Bayesian Filters in Python � A detailed introduction to Kalman filters, making heavy use of Python.

This OP�s a community effort; post in the OP or PM-me for updates and edits.

Dominoes fucked around with this message at 15:03 on Mar 6, 2017

# ¿ Mar 5, 2017 23:37

Dominoes: Sep 20, 2007

Done. Sorry Therm - I'm really butchering your material!

Dominoes fucked around with this message at 00:38 on Mar 6, 2017

# ¿ Mar 6, 2017 00:26

Dominoes: Sep 20, 2007

I agree Nip; going to cut them entirely.

# ¿ Mar 6, 2017 01:58

Dominoes: Sep 20, 2007

VC Code feels like a faster Atom.

Dominoes fucked around with this message at 14:16 on Mar 6, 2017

# ¿ Mar 6, 2017 14:11

Dominoes: Sep 20, 2007

Thread's up.

Still looking for additions and edits.

# ¿ Mar 6, 2017 15:36

Dominoes: Sep 20, 2007

VikingofRock posted:

Maybe add some stuff about Python 2 vs. Python 3? Someone new to the language is bound to have questions about it, since it comes up fairly often in Python discussions, and there is still a lot of Python 2 code floating around out there (particularly in scientific circles). Even if the section is just "Python 3 is better and you should definitely use it and here is why", it's probably worth having that section IMO.

I deliberately omitted that; too political.If anything, it would be a note about missing features etc when working with old codebases.

Dominoes fucked around with this message at 10:43 on Mar 7, 2017

# ¿ Mar 7, 2017 10:39

Dominoes: Sep 20, 2007

# ¿ Mar 7, 2017 19:05

Adbot: ADBOT LOVES YOU

# ¿ May 8, 2024 21:38

Dominoes: Sep 20, 2007

QuarkJets posted:

For some reason Enthought (their product Canopy is basically like Anaconda) only supports Python2.

Edited to reflect that!

# ¿ Mar 7, 2017 20:17

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›16 »