Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »

QuarkJets: Sep 8, 2008

Opencv and PIL are both good, pretty sure numpy/scipy have image reading functions that just use PIL

# ? Aug 14, 2019 23:53

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 15:38

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

crazysim posted:

Unfortunately, WSL2 does appear to require Hyper V.

https://devblogs.microsoft.com/commandline/wsl-2-post-build-faq/

Bah!

though...according to that page VirtualBox post 6.0 should work if you have Hyper-V installed?

QuarkJets posted:

Opencv and PIL are both good, pretty sure numpy/scipy have image reading functions that just use PIL

PIL looks like what I'll use for now, thanks.

Except that you should actually use the PIL fork Pillow now instead of PIL since PIL stopped development back in 2009.

Thermopyle fucked around with this message at 00:26 on Aug 15, 2019

# ? Aug 15, 2019 00:21

KICK BAMA KICK: Mar 2, 2009

Thermopyle posted:

If you think that's nice, wait until you learn about Docker!

That one-liner you posted the other day impressed me; I'll take a crack at it tonight.

e: wtf how does it pull an Ubuntu image in 26mb?

KICK BAMA KICK fucked around with this message at 01:24 on Aug 15, 2019

# ? Aug 15, 2019 00:50

necrotic: Aug 2, 2005; I owe my brother big time for this!

KICK BAMA KICK posted:

e: wtf how does it pull an Ubuntu image in 26mb?

Its not the same as the server ISO, which includes a lot of packages. The image is basically just the rootfs.

https://github.com/tianon/docker-brew-ubuntu-core/blob/9db8c72dd02e8f9fd5dba82ff9266174b088e2e6/bionic/Dockerfile

Docker images can grow in size pretty quickly if you aren't good about cleaning up each run step. I went through one of ours recently and stripped out nearly a gig of trash from the image. It's still 600mb, but the majority of that are artifacts for the application itself.

# ? Aug 15, 2019 02:00

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

Thermopyle posted:

Bah!

though...according to that page VirtualBox post 6.0 should work if you have Hyper-V installed?

PIL looks like what I'll use for now, thanks.

Except that you should actually use the PIL fork Pillow now instead of PIL since PIL stopped development back in 2009.

Yeah I've done a fair amount of image manipulation in Python and Pillow is the best library I've tried for it.

# ? Aug 15, 2019 04:29

Dominoes: Sep 20, 2007

QuarkJets posted:

pretty sure numpy/scipy have image reading functions that just use PIL

Yep. There's a scipy module dedicated to it; used to be just a read fn in `scipy.misc`.

# ? Aug 15, 2019 05:04

KICK BAMA KICK: Mar 2, 2009

Following up from yesterday yes Docker is quite good. I spent Tuesday evening figuring out Vagrant/Ansible/VMs in general and thinking that was awesome, and now that I see how Docker works I feel like yesterday me was the character in like a movie set in the past who is set up to make the joke of saying like "ok so this is called a TELEFACSIMILIE and it can transmit FOUR WORDS in only SEVEN HOURS, pretty cool, right?"

So the idea is if I set up my docker-compose.yml right on the local machine I code on and everything works there then I can go do the same on my "server" to "deploy" it and it will work exactly the same?

# ? Aug 16, 2019 03:18

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

KICK BAMA KICK posted:

Following up from yesterday yes Docker is quite good. I spent Tuesday evening figuring out Vagrant/Ansible/VMs in general and thinking that was awesome, and now that I see how Docker works I feel like yesterday me was the character in like a movie set in the past who is set up to make the joke of saying like "ok so this is called a TELEFACSIMILIE and it can transmit FOUR WORDS in only SEVEN HOURS, pretty cool, right?"

So the idea is if I set up my docker-compose.yml right on the local machine I code on and everything works there then I can go do the same on my "server" to "deploy" it and it will work exactly the same?

Yep, you got it.

# ? Aug 16, 2019 06:26

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

What's the suggested way to handle dependencies for a larger system installation getting installed on the local machine and that we expect local system users to be manually executing some number of the python scripts? The python scripts are going to be located at /usr/local/butts/bin and that we expect the users to just cd into the directory and directly run ./script.py and not do python3 script.py. Would the best option just be to put a virtualenv in the directory and make the shebang point to the python executable in the venv or is there some better way that I'm missing. These are not power users and I'd like to avoid having to explain what is virtualenv or pipenv or anything like that to them.

# ? Aug 16, 2019 16:59

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I'd consider using something like cx_freeze or nuitka to just make it super simple for the end users and get rid of a lot of administration hassles.

# ? Aug 16, 2019 18:32

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

For cx_freeze or nuikta, I'm guessing I go into a virtualenv with my necessary dependencies, compile, leave the virtualenv and then the scripts can be executed as is?

# ? Aug 16, 2019 18:49

necrotic: Aug 2, 2005; I owe my brother big time for this!

The output is a single binary with everything embedded.

# ? Aug 16, 2019 18:54

QuarkJets: Sep 8, 2008

Master_Odin posted:

What's the suggested way to handle dependencies for a larger system installation getting installed on the local machine and that we expect local system users to be manually executing some number of the python scripts? The python scripts are going to be located at /usr/local/butts/bin and that we expect the users to just cd into the directory and directly run ./script.py and not do python3 script.py. Would the best option just be to put a virtualenv in the directory and make the shebang point to the python executable in the venv or is there some better way that I'm missing. These are not power users and I'd like to avoid having to explain what is virtualenv or pipenv or anything like that to them.

The venv solution is what I would do

And I would put it all together in an rpm or deb package

# ? Aug 16, 2019 19:57

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

necrotic posted:

The output is a single binary with everything embedded.

Sorry, I meant that I'd have to do the compile step on the machine that I was planning on using these scripts, e.g. compile on Ubuntu if using on Ubuntu.

# ? Aug 16, 2019 20:15

mr_package: Jun 13, 2000

It's not impossible to cross compile but yeah, just do that.

I use pyinstaller to deploy to Windows. On Mac I got lazy and just update system Python to whatever I need. I am terrible at devops.

# ? Aug 16, 2019 20:21

Dominoes: Sep 20, 2007

mr_package posted:

On Mac I got lazy and just update system Python to whatever I need. I am terrible at devops.

You've been dealt a bad hand.

# ? Aug 16, 2019 20:26

Dominoes: Sep 20, 2007

Is there a way to execute a package's CLI script from the console without creating the script/bin file? Scripts are identified in a package's dist-info folder, in a file named `entry_points.txt`. Example:

code:

[console_scripts]
ipython = IPython:start_ipython
ipython3 = IPython:start_ipython

where `IPython` is the module, and `start_ipython` is a function imported in that module.

They're normally executed by running python with a script in the env's bin or Scripts folder. On Linux, this appears to be a python script. On Windows, it's an executable file. (eg `bin/ipython`, or `Scripts/ipython.exe` respectively) The Linux scripts generally import the function from the module, and run it with sys.argv, with some boilerplate.
Example:

code:

import re
import sys

from IPython import start_ipython

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(start_ipython())

Is there a way to run the console scripts directly? I've found that running `python -m IPython` works in the above case, but this is likely incomplete, as it ignores the `start_ipython` part, so it may not be generally applicable. Have tried various combos of `.`, `:`, quotes etc.

I'm attempting to incorporate CLI scripts into the package manager I'm working on without executing `setup.py`. Could do this on linux by just genning the bin-folder script and executing it, but I think storing the `entry_points.txt` info in a single file without the boilerplate would be more elegant, and I'm not sure how to approach this on Win. edit: I think genning the linux-style py script should work for win too. Will probably take this approach.

Dominoes fucked around with this message at 20:22 on Aug 17, 2019

# ? Aug 17, 2019 18:35

QuarkJets: Sep 8, 2008

You can added a shebang to the top of any script, if you set the shebang to use a python executable then it will execute as though you typed "python <script_name>". You can also package these with the scripts keyword (which targets files) instead of console-scripts (which targets functions in files)

# ? Aug 17, 2019 23:40

Dominoes: Sep 20, 2007

Much apprec. Hadn't given thought to packaging yet - on depacking now. Have a (so-far) working soln, where we generate the file with this code:

Rust code:

fn make_script(path: &PathBuf, name: &str, module: &str, func: &str) {
    let contents = format!(
        r"import re
import sys

from {} import {}

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit({}())",
        module, func, func
    );

    fs::write(path, contents).expect(&format!("Problem creating script file for {}", name));
}

Save it to `__pypackages__/bin`, and call the env's python to execute it.
Where we've parsed `name`, `module`, and `func` using a regex from `[console_scripts]`. Sounds like I need to check for `scripts` as well, or could break on some packages.

I noticed the shebang in the linux scripts; ommitted it from this generated one since it seemed OS-specific. What's your take on including/excluding a shebang with this flow?

Dominoes fucked around with this message at 23:55 on Aug 17, 2019

# ? Aug 17, 2019 23:51

QuarkJets: Sep 8, 2008

Windows installs of python associate .py files with the python executable, so no action needed there. *nix users get script support by you adding a shebang. Thus, a shebang is all that's needed for cross-platform script support.

# ? Aug 18, 2019 00:15

Dominoes: Sep 20, 2007

Got another brain buster: If a package specifies `sys_platform == darwin`, should this be applied when run on macOs?

And another: How are extra features which are required specified? The tag `extra == "test"` etc is common, and means to only install that dep if the `test` flag is included in its parents' `extras_require` in `setup.py`. But how do we know a dependency requires its sub-dependency to be installed with an extra?

ie I've found no `features == "test"; there must be some way to indicate a sub-dep requires this. I hope it's not relying on parsing sub-deps' setup.py files.

Dominoes fucked around with this message at 02:23 on Aug 18, 2019

# ? Aug 18, 2019 02:15

22 Eargesplitten: Oct 10, 2010

Newbie question: if Python is so much slower than C, why are so many libraries in Linux written in Python rather than C, even stuff meant for SOC use?

Or are we at the point where even SOCs are miles ahead of full-blown workstations from when Python was invented?

# ? Aug 18, 2019 20:52

QuarkJets: Sep 8, 2008

22 Eargesplitten posted:

Newbie question: if Python is so much slower than C, why are so many libraries in Linux written in Python rather than C, even stuff meant for SOC use?

Or are we at the point where even SOCs are miles ahead of full-blown workstations from when Python was invented?

The concept is that Python makes a very nice glue and prototype language. You get the best of both worlds by writing a Python library where any performance-critical stuff is written in a very fast compiled language like C or Fortran. One way to reach that point is to write the library entirely in Python and then profile it. If performance optimization isn't needed, then you're done; the pure python implementation is fast enough.

QuarkJets fucked around with this message at 21:05 on Aug 18, 2019

# ? Aug 18, 2019 21:03

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

That and inertia, it's been around for long enough that it's already been a glue language for a while, and a lot of system components have (often first-party) bindings like systemd

# ? Aug 18, 2019 21:25

taqueso: Mar 8, 2004

Plus there are plenty of people that can write python code but don't want to write / don't want to learn C.

# ? Aug 18, 2019 22:25

22 Eargesplitten: Oct 10, 2010

Makes sense, I didn�t think about the possibility of using a profiler to pick out individual bits that need to be more optimized, doing them the hard way, and then wrapping them in Python.

I�m getting involved in a project with a friend that knows way more poo poo than I do and it�s either going to be a trial by fire or the end of a friendship when he realizes that picking up new stuff fast is a poor substitute for knowing what the gently caress you�re doing :v:

I mean I have an AS in computer science and have written a decent amount of hobby stuff in various languages, but most of the time if something has to interact with other languages I�m writing it to plug in to a black box of a game engine or web server or something. SOC design probably won�t be nearly as kind to me. Although maybe knowing C++ will help me not be completely lost at C.

# ? Aug 18, 2019 22:41

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

22 Eargesplitten posted:

Although maybe knowing C++ will help me not be completely lost at C.

Oh, you sweet summer child.

# ? Aug 19, 2019 13:52

The Fool: Oct 16, 2003

C++ does a better job of keeping you from getting lost at sea.

# ? Aug 19, 2019 17:00

22 Eargesplitten: Oct 10, 2010

I�m glad someone picked up on my terrible pun.

Turns out there�s out of the box open source software that would do more of what we need than the other options and it supports .NET so I might go with that since about half my experience is with Java and another bit with C#

# ? Aug 19, 2019 22:42

mr_package: Jun 13, 2000

Dominoes posted:

You've been dealt a bad hand.

Oops it's not quite so terrible as I made it sound: I'm using homebrew on Macs so not actually touching the "Mac System Python" 2.x stuff. But it's all still by hand, I go do brew install and then pip3 install etc. for the libraries I need on all the 'servers' (old mac minis).

Unsurprisingly a tradeoff of this is that I try to not use 'good' libraries like requests, lxml, etc. and tend to try to use the built-in modules first. urllib3 ain't so bad! j/k, I installed requests.

# ? Aug 21, 2019 22:46

QuarkJets: Sep 8, 2008

Install anaconda

# ? Aug 22, 2019 07:43

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Or pyenv

# ? Aug 22, 2019 16:37

the yeti: Mar 29, 2008; memento disco

If I wanted to write unit and integration tests for a small project (as an exercise more then anything) should I be using pytest or is there something nicer?

# ? Aug 23, 2019 14:51

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Pytest

# ? Aug 23, 2019 15:13

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

the yeti posted:

If I wanted to write unit and integration tests for a small project (as an exercise more then anything) should I be using pytest or is there something nicer?

unittest is in the standard library and is just fine for a small project.

# ? Aug 23, 2019 15:18

Dominoes: Sep 20, 2007

Pytest's syntax is more concise, and uses native assert statements:

Unittest:

Python code:

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

Pytest:

Python code:

def test_upper():
    assert 'foo'.upper() ==  'foo'

# ? Aug 23, 2019 16:24

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

We shouldn't be encouraging people to add dependencies unnecessarily and for a small project the benefits of pytest are pretty limited.

# ? Aug 23, 2019 17:24

QuarkJets: Sep 8, 2008

Thermopyle posted:

We shouldn't be encouraging people to add dependencies unnecessarily and for a small project the benefits of pytest are pretty limited.

I agree with this.

# ? Aug 23, 2019 20:42

Dominoes: Sep 20, 2007

Question for anyone experience with c python extensions, and how we import them:

I'm attempting to rename Python modules, to accommodate multiple installed versions. I've been parsing the python files, and renaming import statements. I'm now running into an issue where a .so file is imported. It's imported like this:

Python code:

from . import mtrand

Where mtrand appears to be pointing to a file named mtrand.cpython-37m-x86_64-linux-gnu.so. The error message after attempting to use the renamed module is:

Python code:

 File "mtrand.pyx", line 1, in init numpy.random.mtrand
ModuleNotFoundError: No module named 'numpy.random.common'

Presumably the python file imports the `.so` file I mentioned, transparently represents it as `.pyx`, and then calls another file named common.cpython-37m-x86_64-linux-gnu.so.

Note that there is a common.pxd file, in which I can perform a rename that does not affect the error. Is this insolvable without recompiling the C/Fortran, or is there a way to do this? Presumably this could be fixed by editing the `.pyx` file, if it's real.

edit: The file that needs to be modified exists on the Numpy github, but not the unpacked wheel. Leaning unfixable without a recompile.

Dominoes fucked around with this message at 23:18 on Aug 25, 2019

# ? Aug 25, 2019 22:09

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 15:38

Banjo Bones: Mar 28, 2003

I'm trying to learn on my own. I've googled this a lot and I'm kind of stuck.

I want to just write a simple 'while' loop where if a person pets the cat with 'Y' input the cat purrs and the loop ends.
If they press 'N' , then the loop should start over until they choose 'Y'.
If they press anything else than 'Y' or "N', they are told to please press Y or N and the loop restarts.

I'm kind of stuck on how to write it correctly without being an infinite loop.

I'm not asking for anyone to just give me the solution, but maybe just a hint for where I'm going wrong. I know now to write any repetitious code. I want it condensed to just a clean little loop.

code:

def pet_cat(self):
		"""Pets the cat, causing it to react accordingly."""
		answer = input("Do you pet the cat? Please enter Y or N: ")
		while answer:
			if answer == 'Y':
				print(f"{self.name} purrs.")
				break
			if answer == 'N':
				print(f"{self.name} is upset you are not petting him.")
			else:
				print("Please enter Y or N.")

# ? Aug 28, 2019 14:24

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »