Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
NinpoEspiritoSanto
Oct 22, 2013




I've used C stuff before, ffi makes it so much nicer. Cython is useful too and worth reading about.

Adbot
ADBOT LOVES YOU

M. Night Skymall
Mar 22, 2012

Hughmoris posted:

Anyone ever futz with extending Python using C or Rust to gain performance?

I've been reading some articles on it and the idea seems intriguing but I'm sure the reality can be a headache.

I did a proof of concept where I hooked a rust program into python and got it to import and work, it wasn't too hard really although I forgot how it all worked now. I think it was a little annoying to pass objects back into python, but I also didn't spend that long on it and it was almost 2 years ago so things have probably improved since then. I think the headaches are more around getting it to work on a variety of systems, if you're in control of the environment and you don't care about making it easily distributed to everyone on the planet, it's probably not bad at all. Or I guess just shove it into docker.

Da Mott Man
Aug 3, 2012


I've been using this lately to speed up critical areas of projects. https://github.com/pybind/pybind11 Syntax is good, passing typed data is easy, its been a treat.

ryanrs
Jul 12, 2011

Hughmoris posted:

Anyone ever futz with extending Python using C or Rust to gain performance?

I've been reading some articles on it and the idea seems intriguing but I'm sure the reality can be a headache.

Yeah, I wrote a python module in C to expose some system calls. I used the straight C style of the core libraries. That kind of C code tends to be a bit wordy, but overall it was easy to write and a nice API.

How much do you need to cooperate with the rest of the python code? Are you just doing some math, or more involved things with threads or blocking i/o?

QuarkJets
Sep 8, 2008

Hughmoris posted:

Anyone ever futz with extending Python using C or Rust to gain performance?

I've been reading some articles on it and the idea seems intriguing but I'm sure the reality can be a headache.

Yeah I've written a ton of C functions explicitly so that they could be called in Python. It's pretty easy to do, if you're going with a lazy approach there are some pitfalls you can get into but if you're explicitly creating types with the ctypes module to feed in data or receive output from some C function then everything works extremely well. Sometimes I do it for performance reasons, sometimes it's to access some specific functionality that already exists as a C library

There are all sorts of different schemes for doing this but I just use ctypes. It's simple and effective. The only downside is that your C code needs to already exist in a shared library; ctypes will not create one for you.

If you just want performance you might try looking at numba, another thing that I've sometimes used. Numba is nice for accelerating basic numeric computations, you can do neat things like releasing the GIL and it even has the ability to target a GPU (and iirc is the NVidia-endorsed way to do CUDA programming in Python, but I prefer cupy for that). The advertisement has always been "just put this decorating on your normal python functions for a speedup!!!1" but the advertising is kind of a lie, I've never been in a situation where I could just stick the JIT decorator on a function for any kind of performance boost, it's always required that I write a new function that I intend to compile with numba (which is fine by me).

ryanrs
Jul 12, 2011

Ctypes is cool, but you can also use the low-level C API to do Pythonic things like iterators and generators and such. You can make your C code feel extremely Python-y, down to the doc strings.

NinpoEspiritoSanto
Oct 22, 2013




Just a caveat ctypes run like poo poo with pypy whereas cffi is much better. A nicer interface too imo.

Hughmoris
Apr 21, 2007
Let's go to the abyss!
Thanks for the thoughts and ideas.

I don't have a particular use-case at the moment but I'm always intrigued about big performance gains. I know just enough Python to hack together small personal/work scripts, and I'm now walking myself through learning C.

Being able to turbo-charge your Python program by seamlessly dropping down to C/C++/Rust seems like a great tool to have at your disposal.

ryanrs
Jul 12, 2011

In practice it's usually about accessing C libraries or odd system call or other things not available as a python library. Writing a C module for just straight speed optimization is not the common case, I think (could be wrong though?).

12 rats tied together
Sep 7, 2006

It is often either convenience or performance based, performance for example in the case of uvloop, which is cython and written on top of libuv.

I also worked at a data science consulting firm where the r&d team would occasionally build some complex/performance bound functionality in C/C++/Matlab/Rust and then publish both Python and R bindings for it so that the teams involved with consulting engagements could benefit regardless of language chosen. The guy who pioneered this process at that org gave a talk about it, if you're interested.

QuarkJets
Sep 8, 2008

ryanrs posted:

In practice it's usually about accessing C libraries or odd system call or other things not available as a python library. Writing a C module for just straight speed optimization is not the common case, I think (could be wrong though?).

I have no idea how you would even begin to estimate the number of instances of one use case versus the other, but I don't think there's much point in distinguishing those cases, either. I think that numpy.ndarray having a ctypes interface was probably meant to simplify access to old numeric C libraries that are highly performant but that interface is also extremely helpful when writing highly performant C code for your python library, it's all good

The "Python as a glue language" ecosystem usually emphasizes writing C (or whatever) code for performance reasons, in my experience. That's one of the explanations behind why people went and created the numba module: instead of writing Python code, then rewriting bottlenecks in something else, you're supposed to be able to skip a step by just writing numba code in Python. In practice I'm really just replacing "rewrite function in C" with "rewrite function with numba" but the thought is there, and the portability is definitely still an advantage!

QuarkJets fucked around with this message at 09:59 on Apr 6, 2021

accipter
Sep 12, 2003

Hughmoris posted:

Anyone ever futz with extending Python using C or Rust to gain performance?

I've been reading some articles on it and the idea seems intriguing but I'm sure the reality can be a headache.

I have used C and numba. For my problem, using numba was just a fast and much easier to implement.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
On the C runtime front: An old anecdote I keep in mind from last time I did stuff with that is "Python hates PCP." Disregarding what you first thought that meant, it means "Python calling C calling Python." If your runtime calls back to stuff or exposes abstractions then you're probably running into the GIL as a functional thing you'll have to work with. It's not the end of the world by any means but it's a fun little land mine the first time it happens.

QuarkJets
Sep 8, 2008

The fact that numba can compile functions to run on a CPU or a GPU is also an extremely powerful reason to use it.

I do sometimes write C code to be executed from Python but there's no denying that numba has a number of advantages. Usually if I'm deferring to C it's because I have need of some specific C library (for astronomers: cfitsio is way way more efficient than astropy.io.fits)

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4
Python GUIs: tkinter is still not great to work with, right? What's the good poo poo for making a semi-decent cross-platform GUI without giving yourself hives, PyQt? guizero?

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Epsilon Plus posted:

Python GUIs: tkinter is still not great to work with, right? What's the good poo poo for making a semi-decent cross-platform GUI without giving yourself hives, PyQt? guizero?

What's the use case?

Personally, I 100% won't do anything in tkinter or pyqt and instead make my GUIs in the web browser using Dash. https://dash-gallery.plotly.host/Portal/

I find Dash ridiculously productive and bonus that I can deploy it to free tier heroku in about 5 minutes.

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4

CarForumPoster posted:

What's the use case?

Personally, I 100% won't do anything in tkinter or pyqt and instead make my GUIs in the web browser using Dash. https://dash-gallery.plotly.host/Portal/

I find Dash ridiculously productive and bonus that I can deploy it to free tier heroku in about 5 minutes.

Good question: there's someone who wants a desktop version of VARIA, the Super Metroid randomizer. You can run it from the command line but not everyone is like, comfortable with that, so I figured I'd take the time to learn how to make a decent Python GUI and package the whole thing as an .exe for ease of use. I could probably cram it all into an Electron app but I'm more interested in using this as an excuse to learn a GUI toolkit than I am in learning how to get Electron going.

QuarkJets
Sep 8, 2008

PyQt is the best imo, you can consistently make nice-looking GUIs out of it and it's simultaneously easy to use yet versatile.

You should basically avoid tkinter, it sucks and the GUIs you can make with it universally look bad.

There are a few other good ones but PyQt is an excellent default choice

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Epsilon Plus posted:

Good question: there's someone who wants a desktop version of VARIA, the Super Metroid randomizer. You can run it from the command line but not everyone is like, comfortable with that, so I figured I'd take the time to learn how to make a decent Python GUI and package the whole thing as an .exe for ease of use. I could probably cram it all into an Electron app but I'm more interested in using this as an excuse to learn a GUI toolkit than I am in learning how to get Electron going.

At the risk of being obtuse and pumping out hot takes, learning a python gui package is a complete waste of time. Deploying python poo poo to run locally is absolutely terrible.

Also I cant help but notice that website you linked is doing exactly the thing I suggested. Its a python web app "Powered by web2py". That said their github indicates web2py requires tremendously more code and number of files than a comparable Dash app.

CarForumPoster fucked around with this message at 01:26 on Apr 9, 2021

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4
Agreed, more or less, but there's a few people who seem to want it for one reason or another (unstable/metered internet connection or frequent travel) and it's giving me a good excuse to poke at some stuff relating to the project I'm interested in.

QuarkJets
Sep 8, 2008

CarForumPoster posted:

At the risk of being obtuse and pumping out hot takes, learning a python gui package is a complete waste of time. Deploying python poo poo to run locally is absolutely terrible.

If you're about to write a post and your first thought is "I'm being really obtuse with this hot take" maybe just don't bother.

Macichne Leainig
Jul 26, 2012

by VG
PyQt with PyInstaller makes it pretty dang easy to get a UI up and running and package it into a single .exe for deployment. Sure there are probably a million other "better" options but I wouldn't go as far as to say it's a complete waste of time.

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Protocol7 posted:

PyQt with PyInstaller makes it pretty dang easy to get a UI up and running and package it into a single .exe for deployment. Sure there are probably a million other "better" options but I wouldn't go as far as to say it's a complete waste of time.

Okay I challenge you to make a GUI based Python distributable that can do this common task. This is trivial to do in a Python web app.

Task:
Mail merge a docx template and save as pdf.

Only requirements:
-runs on regular win10 with ms word
-Uses an MS word template, in docx format.
-Replace 4 strings with user input boxes from the GUI
-save as a PDF with a user defined file name.
-It needs to just work, can’t require a user to install Python on their system or interact with a CLI.
-Can’t require admin privileges.

I’ll post the same app as a web app to heroku free tier. It’ll be about 30 lines of code in one file.

Macichne Leainig
Jul 26, 2012

by VG
I won’t disagree that Dash is better in your oddly specific use case. Maybe PyQt isn’t even the best desktop GUI library. I will still vouch for PyInstaller making it easy to package Python projects in a single .exe though.

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Protocol7 posted:

I won’t disagree that Dash is better in your oddly specific use case. Maybe PyQt isn’t even the best desktop GUI library. I will still vouch for PyInstaller making it easy to package Python projects in a single .exe though.

I used PyInstaller in 2016 and attempted to use it again in 2019 and staunchly disagree.

It was easy when it worked. There is a set of circumstances that go into that happening. Want to package an ML model? Make interactive visualizations? Interact with the Win32 API? Well those are all super common business tasks and all things that you're likely to run into trouble with with PyInstaller and GUIs in my limited experience.

The OPs goal was to make something for their friend that can't use a CLI. "Simple to use" and "self contained python packages" are not good combos in my, again limited, experience.

QuarkJets posted:

If you're about to write a post and your first thought is "I'm being really obtuse with this hot take" maybe just don't bother.

bellicose shitposting about resumes, planes and python is all I have left

QuarkJets
Sep 8, 2008

CarForumPoster posted:

Okay I challenge you to make a GUI based Python distributable that can do this common task. This is trivial to do in a Python web app.

Task:
Mail merge a docx template and save as pdf.

Only requirements:
-runs on regular win10 with ms word
-Uses an MS word template, in docx format.
-Replace 4 strings with user input boxes from the GUI
-save as a PDF with a user defined file name.
-It needs to just work, can’t require a user to install Python on their system or interact with a CLI.
-Can’t require admin privileges.

I’ll post the same app as a web app to heroku free tier. It’ll be about 30 lines of code in one file.

FYI you've been getting really aggro over insisting that a widely-used and well-supported set of tools are completely unusable and a waste of time, when you seem to have little experience in actually using them. Heroku is great but not all products make sense for a PaaS deployment.

No one is asking you to use these tools, nor is anyone even saying that they're superior to deploying something on Heroku. Please don't feel like you have to poo poo on what other people want to do

Hughmoris
Apr 21, 2007
Let's go to the abyss!
Honest question when people say to use a web app instead of a traditional GUI...

Are they talking about using something like Flask and just hosting it locally? E.g. I want to create a UI that lists my media files, their file size, and thumbnails. Would I just build that out with Flask/HTML/CSS/JS and just host it locally? Then whenever I want to use the UI I have to launch the server?

QuarkJets
Sep 8, 2008

CarForumPoster posted:

I used PyInstaller in 2016 and attempted to use it again in 2019 and staunchly disagree.

It was easy when it worked. There is a set of circumstances that go into that happening. Want to package an ML model? Make interactive visualizations? Interact with the Win32 API? Well those are all super common business tasks and all things that you're likely to run into trouble with with PyInstaller and GUIs in my limited experience.

The OPs goal was to make something for their friend that can't use a CLI. "Simple to use" and "self contained python packages" are not good combos in my, again limited, experience.


bellicose shitposting about resumes, planes and python is all I have left

The OPs goal was to convert something that already exists as a web application into a standalone GUI application that doesn't require internet access. "Just make another web application and deploy it with heroku" is, as you put it, an obtuse response

CarForumPoster
Jun 26, 2013

⚡POWER⚡

QuarkJets posted:

The OPs goal was to convert something that already exists as a web application into a standalone GUI application that doesn't require internet access. "Just make another web application and deploy it with heroku" is, as you put it, an obtuse response

QuarkJets posted:

FYI you've been getting really aggro over insisting that a widely-used and well-supported set of tools are completely unusable and a waste of time, when you seem to have little experience in actually using them. Heroku is great but not all products make sense for a PaaS deployment.

No one is asking you to use these tools, nor is anyone even saying that they're superior to deploying something on Heroku. Please don't feel like you have to poo poo on what other people want to do

Telling someone that the way things were done when they first learned has evolved into a much easier to do thing now isnt making GBS threads on them, it is helping them. The thing the OP linked to for metroid or whatevs, does that work with pyinstaller? If someone was coming in asking how to make beautiful plots in matplotlib I'd tell them seaborn and plotly exist and are WAY faster to learn with much better outputs.

The thing is that they're not that widely used. The reason is, they're not that useful. Here's real data:
PyInstaller, 10 years old, 7.9k stars
Flask, 11 years old, 54k stars
Plotly Dash, 4 years old, 14.3k stars


Hughmoris posted:

Honest question when people say to use a web app instead of a traditional GUI...

Are they talking about using something like Flask and just hosting it locally? E.g. I want to create a UI that lists my media files, their file size, and thumbnails. Would I just build that out with Flask/HTML/CSS/JS and just host it locally? Then whenever I want to use the UI I have to launch the server?

Basically, though this has the same "trying to distrbute python to run locally on non technical peoples computers sucks" issues. If you need to access the local file system you'd run the server that is included with flask/dash/django and access it through a web browser, yea. You wouldnt need to touch HTML/CSS/JS with Dash though.

CarForumPoster fucked around with this message at 05:07 on Apr 10, 2021

QuarkJets
Sep 8, 2008

CarForumPoster posted:

Telling someone that the way things were done when they first learned has evolved into a much easier to do thing now isnt making GBS threads on them, it is helping them. The thing the OP linked to for metroid or whatevs, does that work with pyinstaller? If someone was coming in asking how to make beautiful plots in matplotlib I'd tell them seaborn and plotly exist and are WAY faster to learn with much better outputs.

That's correct, but it's also not all that you did and I think that you know that.

Macichne Leainig
Jul 26, 2012

by VG

CarForumPoster posted:

:words:

The OPs goal was to make something for their friend that can't use a CLI.

Then it’s a good thing PyInstaller works fine for this exact purpose and fabricating other situations that don’t apply to the OP’s question whatsoever is completely unproductive and adds nothing of value to the conversation. :thumbsup:

susan b buffering
Nov 14, 2016

musicbrainz picard is built using Python and PyQt and it’s no more difficult to install and use than any other software I’ve used

Hughmoris
Apr 21, 2007
Let's go to the abyss!
After all that GUI talk last night, I woke up and built a modern UI for my dictionary app using Tkinter. 100% pure standard library, even uses urllib. :smug:



And yes, my entry field automatically resizing with the definition length is a feature.

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4

skull mask mcgee posted:

musicbrainz picard is built using Python and PyQt and it’s no more difficult to install and use than any other software I’ve used

Oh hey I use Picard for FLAC file management, neat!

anyways I'm sorry to have summoned this tedious conversation into existance

QuarkJets
Sep 8, 2008

Hughmoris posted:

After all that GUI talk last night, I woke up and built a modern UI for my dictionary app using Tkinter. 100% pure standard library, even uses urllib. :smug:



And yes, my entry field automatically resizing with the definition length is a feature.

That looks pretty good. I've seen some ugly Tkinter interfaces but yours is not one of those

NinpoEspiritoSanto
Oct 22, 2013




Lol just to pop in on the end of this and mention kivy https://kivy.org/#home

punished milkman
Dec 5, 2018

would have won
i think dearpygui makes pretty beautiful GUIs. i never have a reason to make GUI stuff so i haven’t actually built anything legitimate with it but i want to real bad just for the aesthetics

https://github.com/hoffstadt/DearPyGui

Bad Munki
Nov 4, 2008

We're all mad here.


I was curious about all this so I slapped a Tkinter GUI on top of a thing I've been working on and at least if you keep your usage simple and just use the basic elements, lol dear god it's fuckin' hideous

My little test app was a couple text areas and a go button, the top one takes some json, the button sends it off to a service, and the bottom text area displays the results. By default,

1) the text areas have no separation from the background. It's a white area on a white background, unless
2) you click on one of the areas to set focus, at which point it's just got a black rectangle around it, but only one of the area can have focus at a time, obviously
3) the button's default color scheme makes it almost invisible. White text on a white background with a faint grey outline
4) when you click the button, it gets a more colorful background, but
5) when you release the button, there's a big grey-filled rectangle all around it with like a 10-pixel margin that sits there for a second? I think it's supposed to flash real quick but oof, it's ugly
6) scrolling the text areas produces a whole load of lovely artifacts as lines of text flash that same grey highlight seemingly at random
7) I'm firing this off from within pycharm, but it starts in the background and I have to cmd-tab to it every time after starting it
8) it's incredibly easy to soft-lock it with a slightly poor selection of Tkinter calls, which then means having to force-quit the app, and then each time it runs after that python complains about having quit in a bad state previously and would I like to restore my windows? Which it asks every time no matter what from then on, until I finally say yeah, sure, do it buddy, and it does nothing, but stops asking at least. I imagine this is less about Tkinter though.
9) it won't accept a paste into these text areas? Copy works though. WHY
10) text areas don't get scrollbars on demand automatically?

I will happily believe that these are problems that could be overcome, but why would they exist in the first place?! Even overlooking the cosmetic issues with styling, these are some embarrassing fundamental problems.

https://i.imgur.com/yYDfs5S.mp4

The code that produced this mess, for reference:

code:
import tkinter as tk
import json
import asf_search as asf

def do_search():
    try:
        search_output.delete(1.0, tk.END)
        search_params = json.loads(search_input.get("1.0", tk.END))
        results = asf.search(**search_params)
        search_output.delete(1.0, tk.END)
        search_output.insert(1.0, json.dumps(results, indent=2))
    except Exception as e:
        search_output.delete(1.0, tk.END)
        search_output.insert(1.0, e)
    return

window = tk.Tk()

tk.Label(window, text="Search Parameters").grid(row=0)
search_input = tk.Text(window, height=10, width=200)
search_input.grid(row=1, column=0)

search_button = tk.Button(window, text="Search", command=do_search).grid(row=2)

tk.Label(window, text="Search Results").grid(row=3)
search_output = tk.Text(window, height=30, width=200)
search_output.grid(row=4, column=0)

window.mainloop()

Bad Munki fucked around with this message at 05:26 on Apr 14, 2021

Hughmoris
Apr 21, 2007
Let's go to the abyss!

Bad Munki posted:

I was curious about all this so I slapped a Tkinter GUI on top of a thing I've been working on and at least if you keep your usage simple and just use the basic elements, lol dear god it's fuckin' hideous
...

I know you did this as an exercise and it's a draft, but I wonder if you're experiencing some wonkyness specific to Apple. I'm running Win10 and used your code:

1. The text area does have separation (see screenshot)
2. The button looks normal and behaves normally
3. I can paste text just fine in your text boxes
4. Scrolling through a 4MB text file, I don't see any artifacts. Still no scroll buttons though

I have next to zero experience with UIs but for 29 lines of code I don't think it looks half bad.

Adbot
ADBOT LOVES YOU

Macichne Leainig
Jul 26, 2012

by VG
Doesn’t tkinter have color themes too? Most of them are pretty awful, yeah, but might help it look more consistent across macOS and Windows.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply