Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

It's more that if you were to do:

code:

>>> q = OrderedDict({1: 'first', 2: 'second', 3: 'third'})
>>> q.update({4: 'fourth', 5: 'fifth'})
>>> for i, j in q.iteritems():
...     print i, j
...

Then you wouldn't be guaranteed to get:

code:

1 first
2 second
3 third
4 fourth
5 fifth

As the order of the dictionaries you're passing in as a paramater to the constructor and update are both normal unordered and could be added to the OrderedDict in any order (based on python spec).

However, the way you were adding to the dictionary is the recommended way that would ensure the order always.

# ¿ Mar 14, 2017 21:19

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 21:34

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Newf posted:

Do you know (or care to share) any os projects with a python / electron setup? I've only glanced at electron, and then only with JS/TS projects. Haven't seen used as a UI layer with other languages.

https://github.com/fyears/electron-python-example

You have to have some boilerplate to spin up electron and get python to initially communicate, but after that, it's super easy to just use Python/Flask to design your entire app. It's also pretty easy to bundle this (though again requires some more boilerplate in the node layer).

# ¿ Aug 1, 2017 23:55

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

NtotheTC posted:

So my new project is sufficiently out of the quagmires of legacy code that I get to start using the latest versions of everything- including python3.6 over python2.7. It's perhaps a bit shameful that I've done virtually no commercial coding with python3, but I'm wondering if there's a list of the biggest changes I'll need to wrap my head around? f_string_literals and type hinting are two that I've been made aware of but I'm sure there's more out there.

The biggest (to me) is the asyncio stuff. But here's a nice list of neat things in Python3 as compared to Python2: https://powerfulpython.com/blog/whats-really-new-in-python-3

Master_Odin fucked around with this message at 13:12 on Sep 7, 2017

# ¿ Sep 7, 2017 12:53

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

If anything, it looks like you could create a dictionary where the keys are the categories and the values are dictionaries where the keys are the names of the views and values are view object. Then you'd just need a nested for loop to iterate through all this to make the repeated function call.

Moving stuff around the menus would be moving key/values around the inner dicts.

# ¿ Sep 8, 2017 16:09

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Sphinx is the way to go here unless you're looking to generate API docs and you like how Slade (or whatever) looks like. Though you can definitely modify the Sphinx output pretty endlessly.

# ¿ Sep 20, 2017 22:58

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Slimchandi posted:

If I have a container class, holding some instances of another class, is there a simple way to get the container to aggregate the results of the inner classes, without writing separate methods for the container for each inner class methods

How can I avoid writing similar methods for hearts and diamonds, on the basis that the logic will be the same for all three? Preferably in a way that still works with code completion in Pycharm?

What are you trying to do here? Your example makes little sense to me. I'd probably have each suit (spade, heart, etc) be a class that inherits Inner, setting an internal class variable common to Inner to also denote suit and value.

# ¿ Jan 13, 2018 14:10

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

HardDiskD posted:

Which testing framework is the new hotness nowadays? I've been using nose for the longest time and I want to get on with the times.

Assuming a modern version of Python3, either pytest or unittest. I prefer the latter when possible as it's part of the standard library, but the former is more powerful. nose has seen work in a couple years and nose2 sort of fell flat and has this on their github page: "However, given the current climate, with much more interest accruing around pytest, nose2 is prioritizing bugfixes and maintenance ahead of new feature development."

# ¿ Apr 18, 2018 19:41

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

The main takeaway I got from that whole thing is that Python dependency management is becoming increasingly like Go's was with multiple different "standards" (for Python, we've got setup.py, requirements.txt, Pipfile, pyproject.toml, setup.cfg, MANIFEST.in, ...) and a bunch of churn over "the way forward" especially given that some are specified in PEPs (pyproject.toml) and accepted and others are the defacto (Pipfile) based on usage in the wild.

Like, is the suggested solution right now to have a Pipfile and pyproject.toml and then to keep them manually synced for publishing/development, even though they're largely going to be the same thing (and I fail to see why we need three files considering the success of yarn/npm/cargo on a two file format).

# ¿ May 15, 2018 01:41

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

I think the way forward is too in flux right now.

But, I think if you choose the most popular one (pipenv), no matter what happens you will end up being fine.

Like there will probably be tools to convert your Pipfile to FuturePipfileReplacement. Actually, if whatever you use can export a requirements.txt you can probably move between all of the solutions without much effort.

While I agree in theory that tying oneself to the most popular one will generally be fine, after looking through poetry, I think it's probably taking the better approach of rewriting what pip and pipenv on top of it does into one package that then uses one single format and having that format be used for installations, building, and packaging (instead of just the first two in the case of Pipenv and then leaving pip to do the third totally separately). It also doesn't help that Pipenv literally advertises itself as THE "Python packaging tool" when it's more of an installation dependency management system.

SurgicalOntologist posted:

I think what's often missing in these discussions is the difference between applications and libraries. I didn't read the whole reddit thread but KR does mention that pipenv is only for applications, I'm just not sure it got much attention.

Except what about for developing libraries? I think it's more that including/excluding setup.py should be determined on if you're writing an application or a library (exclude on former, include on latter), but you should probably always have a Pipfile and Pipfile.lock for ease of development? KR has certainly included them in requests at the very least.

# ¿ May 15, 2018 04:23

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

SurgicalOntologist posted:

Genuinely asking, what are the benefits for ease of development? Once I have setup.py, if I want to develop a new machine I just run pip install -e. and I'm ready to go.

The answer to that probably depends on your use of virtualenv/pipenv.

# ¿ May 15, 2018 04:59

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

Though, if you wanted to use pipenv for some reason, you can of course download the numpy-mkl wheel and install it by pipenv install numpy-mkl.whl.

To be clear, I'm very torn between several of these attempted solutions. For example, I do not like that pipenv defaults to wildcard versions on installed packages. It should pin the version number by default.

In fact, poetry seems to do a lot better with regards to a lot of this stuff. Unfortunately, I like the transparent virtualenv handling that pipenv does so I'm constantly torn between the two.

edit: oh another thing that poetry does better than pipenv...you don't have to manage a Pipfile (or requirements.txt) in addition to a setup.py file.

edit2: Like I said on that reddit thread, I think its better that a good-enough tool becomes a standard in the python community soon rather than a perfect tool at some later time.

Yeah, and poetry is completely unusable for Homebrew Python installations so good luck getting that to catch on till that's fixed.

# ¿ May 15, 2018 21:42

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Sad Panda posted:

Very. I'll be logging in to each site and there's not going to be much standardisation.

There are updates every now and then but infrequently enough that once I get it all done at first I don't mind tweaking it when they change.

As vikingstrike said, if it's possible to generalize, you should try and do that. But if each site is unique enough that they require their own parser, your best bet is to store the scraping code in a DB or dynamically load it from a directory (I'd chose the latter as it'll be way easier to update and version control). But even in those cases, see if there isn't generalized helper functions you can write for it.

# ¿ May 16, 2018 01:14

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

The March Hare posted:

I've got a real puzzling guy on my hands right now, could use outside thoughts.

Say we have some string:
'\x1b[1mThis is the string with some random ANSI garbage.\x1b(B\x1b[m'

That string has a length of n, but a printed length of x, where x is len(This is the string with some random ANSI garbage.).

I need to split lines whose printed len is > terminal width in order to display them properly.

My gut instinct is that I should strip all of the ANSI escape sequences from the strings, then use something like textwrap.wrap, then reinsert the ANSI stuff. This would involve some kind of tricky index tracking, and would be generally annoying.

Is there any simpler/more elegant way any of y'all can think of to solve this here problem?

I'd probably subclass textwrap.TextWrapper to work mostly the same, just in that it ignores ANSI escape sequences in calculating the length of a given work chunk.

# ¿ May 18, 2018 01:07

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

SpaceSDoorGunner posted:

Anyone know why this:

https://stackoverflow.com/questions/23287/largest-prime-factor-of-a-number/412942#412942

is better what seemed to me to be the obvious solution:

def factor(num):
->factors = [ ]
->for x in range(1,num):
->-> if num%x == 0:
->->-> factors.append(x)
->factors.append(num)
->return(factors)
factor(42)

Your algorithm returns something different? It returns all possible factors, not the prime factorization of the number.

code:

>>> def prime_factors(n):
...     """Returns all the prime factors of a positive integer"""
...     factors = []
...     d = 2
...     while n > 1:
...         while n % d == 0:
...             factors.append(d)
...             n /= d
...         d = d + 1
...     return factors
...
>>> def factor(num):
...     factors = []
...     for x in range(1, num):
...             if num % x == 0:
...                     factors.append(x)
...     factors.append(num)
...     return factors
...
>>> prime_factors(42)
[2, 3, 7]
>>> factor(42)
[1, 2, 3, 6, 7, 14, 21, 42]

# ¿ Jun 2, 2018 22:30

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

I almost always have a "main()" function that "if __name__ == '__main__'" block calls if the script is to be run directly. pylint trained me that doing anything in the if statement complicated is bad, and also to never do tests based around testing if __name__ is equal to __main__ within your actual function code. This makes it way easier also to turn your script into an installed application on the commandline if you go that route.

# ¿ Jun 7, 2018 23:24

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Is there a good list of released python versions somewhere that is easy to grep? I'm trying to make a docker image that's based on stretch-slim but can be used for python 3.4+ (as one of the dependencies of the project will produce a different output depending on version so I can't use python:3.4-jessie-slim and python:3.5-stretch-slim). Alpine also isn't an option as it requires me to build several of my dependencies and include a third-party glibc layer which somewhat ruins the point of using alpine for me.

Master_Odin fucked around with this message at 20:36 on Jun 25, 2018

# ¿ Jun 25, 2018 20:25

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

What exactly would you want users to pass to your function? A multidimensional list of strings that evaluate to functions or Boolean operators?

# ¿ Jul 2, 2018 20:16

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

duck monster posted:

Functions and booleans, but the __operator__ seems the way to go

That still doesn't really tell me much on what exactly your data structures are or how you're looking to combine them.

You can though use the operator module (or __operator__ magic functions on the classes) and then do what sqlalchemy does and define and_, or_, etc to then combine the functions you're calling into a meaningful way, or decompose it into a minimum set of operations if that's your aim.

# ¿ Jul 3, 2018 02:53

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

unpacked robinhood posted:

I'd like to "live-update" a geographical map with rectangular overlays, as soon as background processes bring up fresh data.
So far I have an ugly blocking thing in matplotlib/cartopy that requires manually closing the window to resume processing, until updated content pops up in a new window.
Can I make a non blocking display in matplotlib, or is there something more suitable out there ? I've seen a few SO posts on the issue but the solutions looked super clunky and confusing.

Alternatively, your best bet is just to use Leaflet.js (or just D3) with some JS around it that makes ajax calls/websocket to your python backend to get the data and then updates the Leaflet.js, all on the frontend.

# ¿ Jan 24, 2019 16:03

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Except at least they have a folder within a project they install modules to and I don't have to explain virtualenvs to people starting out. Can't wait for PEP-582 to be implemented.

# ¿ Feb 5, 2019 18:30

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

necrotic posted:

Our approach for awful projects like that was to say if you touched a file in a change you fixed it then. This made the up front lift small, but could cause small changes to take longer. It was worth it and eventually only 10% of files which were never touched remained and somebody just went in and fixed them later.

This is tricky with CI though since you can only run the lint against files changed in the commit or it'll fail for you constantly.

Yeah, this is what I've done for large projects that hadn't yet been linted continuously. Just defined each file individually within whatever the ignore file format is (.flake8 -> exlude=) and then just over time submit a PR that fixes all style issues in a file and remove it from that list. Prevents any new files from not following the style, and then can overtime fix up old code. I do prefer to not mix PRs of bugfixes/features with PRs fixing style changes as that just leads to it being a pain to actually properly review a change amongst all the noise.

# ¿ Apr 18, 2019 17:15

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

What's the suggest structure for tests for a module that is not installable? I've got the standard structure of:

pre:

pkg/
  __init__.py
  module_1.py
tests/
  __init__.py
  test_module_1.py

where test_module_1.py at the top has a line to import pkg.module_1.

and so I can use

code:

python -m unittest discover

and it works totally fine, but if I try and do

code:

python tests/test_module_1.py

(or the equivalent command through python -m unittest) it will complain about how it cannot import pkg.module_1. Is the suggested fix here to use sys to insert the path to the package or is there something I'm missing here? Would pytest offer some sort of solution?

# ¿ Apr 20, 2019 16:22

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Neslepaks posted:

I'd suggest invoking through pytest instead yeah. Tends to be less finicky that way.

I guess a follow-up question would be how does pytest under the hood resolve things so that I can just do `pytest tests/test_module_1.py` or even just `pytest test_module_1.py` and it will know what `pkg` is and is importable?

e: Actually, I think I found why I'm so confused on the manner. Using the example from above, why it is that unittest seems to choke if I do an import like this:

code:

import pkg

def test_something():
    pkg.module_1.do_something()

and it throws an error about not being able to find pkg.module_1, but if I just import pkg.module_1 it works fine. Not sure what that's about, but now at least things are working as I'd expect.

Master_Odin fucked around with this message at 20:14 on Apr 21, 2019

# ¿ Apr 21, 2019 19:53

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

Chris has a few errors in that blog post, but anyway thats old news that doesn't need rehashed.

The thing with pipenv is that it's not terrible, it's just not as good as the alternatives. Or, rather, they're all terrible in their own ways, pipenv is just somewhat more terrible.

On the other hand, it has/had a good chance of becoming The One True Thingamajig. When it comes to packaging, there being The One True Thingamajig is way more important than whether we end up with a tool that has a rating of 4/10 Terribleness or a tool that is a 6/10 Terribleness. It's much easier to fix a tool than to fix fragmentation.

In other words the world where we have fragmentation on packaging tools is worse than the world where pipenv is the packaging tool.

However, for me, the advantages that pipenv has in its head start at becoming The One True Thingamajig have just become outweighed by Reitz and how he handles the project.

Yeah, the PSF really needs to stop letting the PyPA be its own thing that takes a laissez-faire approach to "the one true thing" and just mandate everyone use tool foo, that uses format bar (like pyproject.toml which has an actual accepted PEP), and that we stop dealing with the absolute nightmare that is python packaging and dependency resolution. Hopefully with PEP-582 coming in python 3.8, we might finally get a point where the tooling isn't an absolute mess for people, especially when coming from other languages.

# ¿ May 6, 2019 03:50

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

Sounds like there was some drama at the Python language summit.

The general idea:

Python is loving up it's ecosystem with it's poor standard library.

I like where Amber Brown is coming from there.

In a lot of ways I think that if you're making Guido mad you've got good ideas!

On the other hand though, he does have an excellent point that developers really need to stop supporting python 2 universally in their packages and just let the version just sort of organically die regardless of if RedHat will support it forever.

Also, I like how a talk that focuses on how we should more to pypi doesn't mention at all the huge mess that is python package installation.

# ¿ May 18, 2019 22:25

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Well, I guess that talk resonated with the core python developers with the proposed PEP-0594 to deprecate and remove some real old/unused stdlib modules.

# ¿ May 23, 2019 00:39

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

e;f: double post

Master_Odin fucked around with this message at 16:09 on May 25, 2019

# ¿ May 25, 2019 16:06

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

It's not necessarily "bad" for what it does, but it does mean that Python ships with a Tcl interpreter and some other stuff to make it work which causes some amount of explosion in the building python. This isn't a great thing given how little tkinter is really used out in the real world.

Ghost of Reagan Past posted:

But yes just rip off the Python 2 bandaid. Now that supervisor 4 has Python 3 support there is basically zero excuse for it. Your lovely legacy application needs to get ported to Python 3, and that's all there is to it.

Except that they continue support of Python 2.7 in supervisor 4 giving people an easier time of just not migrating. I hope in the next couple of versions they drop that and just focus on Python 3. Major libraries that keep supporting Python 2 will continue to allow people to keep that bandaid on that much longer.

# ¿ May 25, 2019 16:09

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

the yeti posted:

Is there a way to organize a Flask app that doesn't involve hypothetical end users either relying on the developer to host the app or arranging their own server?

Use electron?

# ¿ May 26, 2019 03:51

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

You can pretty easily just iterate through possible python versions available on the path?

In bash for Linux with "future-proofing":

code:

if [ -x "$(command -v "python")" ]; then
  echo "Found python"
fi
for MAJOR in 2 3 4; do
  if [ -x "$(command -v "python${MAJOR}")" ]; then
    echo "Found python${MAJOR}"
  fi
  for MINOR in {0..20}; do
    if [ -x "$(command -v "python${MAJOR}.${MINOR}")" ]; then
      echo "Found python${MAJOR}.${MINOR}"
    fi
  done
done

Would probably expand this to get the actual version from the alias (and potentially follow symlinks, but probably not) for asking the user.

I believe pipenv does something similar when you specify --three or whatever when creating an environment. Could also test for tools like pyenv if you wanted to get really fancy for people who install versions of python but do not add them to their path, but trying to deal with users who have python installed, but not on the path is a pointlessly hard endeavor.

For the second point, you could potentially use the

code:

--target

option on pip to specify the __pypackages__ directory and then just append that directory to PYTHONPATH when you activate the environment?

Master_Odin fucked around with this message at 17:17 on Jul 15, 2019

# ¿ Jul 15, 2019 17:13

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

I guess Kennith Reitz is getting out of the game? https://github.com/not-kennethreitz/team/issues/21

I sincerely hope that requests falls into the PSF organization and that they take over governance. It'll also be interesting to see if we do end up with another event-stream event.

# ¿ Jul 18, 2019 22:59

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

I hope someone hires him.

Looks like PSF is taking over all of KR's stuff.

Also, in reading that issue thread I found out that PSF now administers Black!

Eventually, the black repo will be moving out of the python repo and into the psf repo.

He's already given away quite a few of them with the remaining real high profile ones left being just requests, records, and clint. But yeah, it's cool that the PSF is taking over some number of projects that are indispensable to the community as I trust them not to ghost a repo.

# ¿ Jul 19, 2019 00:34

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

What's the suggested way to handle dependencies for a larger system installation getting installed on the local machine and that we expect local system users to be manually executing some number of the python scripts? The python scripts are going to be located at /usr/local/butts/bin and that we expect the users to just cd into the directory and directly run ./script.py and not do python3 script.py. Would the best option just be to put a virtualenv in the directory and make the shebang point to the python executable in the venv or is there some better way that I'm missing. These are not power users and I'd like to avoid having to explain what is virtualenv or pipenv or anything like that to them.

# ¿ Aug 16, 2019 16:59

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

For cx_freeze or nuikta, I'm guessing I go into a virtualenv with my necessary dependencies, compile, leave the virtualenv and then the scripts can be executed as is?

# ¿ Aug 16, 2019 18:49

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

necrotic posted:

The output is a single binary with everything embedded.

Sorry, I meant that I'd have to do the compile step on the machine that I was planning on using these scripts, e.g. compile on Ubuntu if using on Ubuntu.

# ¿ Aug 16, 2019 20:15

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Boris Galerkin posted:

I�m making a toy script to check the git repo status of all the repos I have on my computer. Right now I�m just using subprocess.run and doing some regex on stdout to check things like if I�m behind or have new files or whatever.

Is there an easier way to do this with a package and which one?

https://github.com/gitpython-developers/GitPython is the one I normally use when doing stuff with Git.

# ¿ Aug 29, 2019 17:42

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Can you not use the --user parameter to also specify only install to your user account? Or just a virtualenv?

You could also go with https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb?hl=en and be able to do a bit more fine-grained HTML updating through JS. What I mean here would be that your python code writes the results to a JSON file, an then a basic HTML file that just shows the body and has a JS function that pings the JSON file every x seconds and updates the HTML structure accordingly.

# ¿ Sep 8, 2019 13:59

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 21:34

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Generate your requirements.txt file via pip freeze. Itwill output a full list of installed packages, including what version they are.

# ¿ Aug 18, 2020 18:29

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python