|
I've got a computer science background, though out of college I immediately went into the operations side. I've done lots of "scripting" first in PowerShell and have relearned Python. But I find myself still thinking in PowerShell, and missing out on some of the more powerful Python patterns, like passing around functions in variables. Does anyone have a suggestion on some kind of material that might me help think a little bit more in Python? Ideally, I'm picturing some content that lays out a "problem" and solves it in more of a "scripting" way but then shows how to refactor and solve the problem in a more Pythonic way. But anything that pushes me in that direction would be helpful.
|
#
¿
Jan 24, 2023 17:13
|
|
|
|
| # ¿ Apr 29, 2024 05:24 |
|
|
And now technology has come full circle, and dockerfiles are full of sed and awk commands to manipulate configuration files etc etc.
|
|
#
¿
Feb 3, 2023 23:31
|
|
|
Zugzwang posted:You could instead do this as a one-liner: Thank you for this. What I'd been doing in the past (when I was quite sure that the element I wanted would be in the list, and only exactly once) was use list comprehension and just tack a [0] at the end, but this is a bit more elegant. I just used it to filter through a list of GitHub release assets to find the right file to install during a pipeline run.
|
|
#
¿
Mar 3, 2023 19:09
|
|
|
A friend pointed out to me that I don't need the default value in next, especially when the next step in my code (loading the URL I found) would fail if the default value happened, so I just took that out. It'll cause a StopIteration exception which I'm not handling, but what it really means is something is messed up with the project I'm reading from and I'll need to fix my code regardless.
|
|
#
¿
Mar 3, 2023 20:07
|
|
|
I'm not entirely sure how to phrase this question so my google searching is really missing the mark. I'm wondering if there's some kind of generic Python framework that will allow me to write a declarative statement about how a thing should look, and then gives me some kind of skeleton I can plug in to tell the framework how to make the changes needed. My specific usage is creating group entries in a particular tool. What I'd like is to have some definition of how the groups/folders should look, and then I can tell this "framework" how to read what exists, how to remove something, how to add something, and then I can just say "make it look like this" and the framework will figure out what needs to be deleted and what needs to be added. Basically I'm looking for something like what Terraform or Ansible do (a declarative declaration of what the product should be, rather than an imperative declaration of how to make it) but either it doesn't exist, or I'm searching with the wrong words. "Declarative" isn't a helpful search term, because that has a specific meaning in programming that's different form what I'm looking for. Maybe there's no value in a framework that just helps decide went to create and when to delete, and anyone doing such a thing is better off just writing it all themselves. Or maybe the whole thing is a waste of time trying to make something extremely robust that only gets executed a few hundred times ever.
|
|
#
¿
Mar 9, 2023 16:26
|
|
|
The Fool posted:If what you're working with is loaded into a dict would something like deepdiff work? Hmm, that might work. Unfortunately I got sucked into a different hyper-focus with this project so it might be bit before I can come back to this particular problem kicking around in the back of my head.
|
|
#
¿
Mar 9, 2023 18:20
|
|
|
I'm quite familiar with Pydantic, since this is part of a project using FastAPI, which relies heavily on Pydantic. And that's not what I'm after here. As part of the application (so it can't really be some external tool) I'm managing groups and folders in this tool called Grouper (which is basically only in use at higher-ed institutions and the web API is terrible and there's nothing really in the ecosystem publicly outside the tool itself). For n departments, the application creates a structure in Grouper with about 15 groups spread out across a couple different folders. The groups needed have changed over time. So some departments have, for example, an access group to a service that no longer exists. When the tool creates a new department it doesn't create a group for that unused service, but the existing departments have those groups. And sometimes we might introduce a new service that needs a new group, and so we have to add that group to all the existing departments. So I might define an ideal layout that's something like code:code:
|
|
#
¿
Mar 10, 2023 22:32
|
|
|
It would be a very "me" thing to go in a couple of weeks from never touching terraform or go to learning go and writing a terraform provider.
|
|
#
¿
Mar 10, 2023 22:46
|
|
|
I'm writing a Python module, and I've gotten myself stuck into some circular dependencies. I've found some "basic" stuff about fixing that, but I seem to have gotten myself into a deeper pickle. Basically, I have a User class, and that User class has methods that will return a Group. And I have a Group class that has methods that will return a User. And I'm trying to define these in separate files, otherwise this is gonna be some massive 2000 line single-file module. I've finally just "thrown in the towel" so to speak and stopped importing at the top of the file, and instead am importing only when a function is called, but that doesn't feel great either. So any tips or links on how to "design" my way out of this?
|
|
#
¿
Apr 10, 2023 19:15
|
|
|
A User object has a get_groups() method that will return all the Groups the User is a member of. A Group has a get_members() method that will return all Users of the Group. So a User object has no Group property, and a Group has no User property, but they each have methods that will call the constructor for the opposite object. I've gotten things pretty well cleaned up with a combination of doing local imports and importing TYPE_CHECKING to enable my type annotations without needing full imports at the root of the file.
|
|
#
¿
Apr 14, 2023 21:55
|
|
|
Working at a large public research university, I know that I enjoy automating the lovely things, but as a bonus I'm also making it slightly easier for someone else to cure cancer. Best of both worlds!
|
|
#
¿
May 10, 2023 12:14
|
|
|
Just as a few of my own cents on testing... I've spent the last year on and off working on a FastAPI project. My general structure is that I have methods that represent the endpoints of the API (obviously) but they do very little work directly, and basically just assemble data to be sent to various "utility" functions. I could test all those utility functions directly, but instead I'm using pytest and pytest-cov to test my API endpoints, generally running through multiple scenarios to cover all the paths of that utility code. I've done quite a bit of major refactoring under the hood, but my tests have barely changed because I'm testing the functionality of the app directly, rather than what's underneath. Not that 100% code coverage is the be-all-end-all, but I have gotten it to 100% code coverage (with a few "# pragma: no cover" comments here and there). And it's pretty common to have something like package[dev] that will install whatever tools are needed for development and testing. Technically speaking you could do that when pulling a package from pypi, but it's really for development (in which case you'd run pip install .[dev] in your local copy of the repo).
|
|
#
¿
May 12, 2023 01:14
|
|
|
Speaking of testing, and discussion a little while back about next(), I've run into some fun bugs around that related AnyIO and respx, the testing framework for httpx. There's some kind of issue where when the AnyIO backend is asyncio and a StopIteration error is raised, it just hangs instead of throwing the error. I was using next() a few places in my code and if the iterator was exhausted my tests would freeze (and would likely freeze in production as well). In addition, respx uses next() under the hood to find the right response to return to a mocked httpx call, and if there's no response found it just hangs. And then when it comes to coverage, apparently the coverage plugin counts an iterator as an untriggered branch if the default value or StopIteraion isn't thrown, which is... kind of weird and surprising. Now I'm super leery of next(), even though I probably shouldn't be. And the issue in AnyIO will get fixed in the 4.0 release, whenever that happens (and whenever it percolates down to starlette and fastapi).
|
|
#
¿
May 16, 2023 18:54
|
|
|
None of my test functions are asynchronous, which I think is what that page is describing. All my tests are synchronous, which works. What I'm testing is a FastAPI app, and testing is just using starlette's TestClient functionality, so even if my app is async, the tests don't need to be. And the AnyIO isn't a choice I made, it's what's given to me by FastAPI via starlette. I think it might actually be an edge case with asyncio directly and its Future ability specifically, but reading too much into that my head starts to spin. It appears that maybe the "pattern" when using asyncio is to do a try/except anytime you think a StopIteration might be raised to catch it and do... literally anything but raise StopIteration. It works fine with Trio, so what AnyIO has done is just add a wrapper when the engine is asyncio to raise a different error instead. I'm just a simple country sysadmin, waiting for the 4.0 release of AnyIO, which has apparently been in the works since like October.
|
|
#
¿
May 16, 2023 20:30
|
|
|
Something I've enjoyed in PowerShell is something called Parameter Sets. Basically, you can have a whole mess of parameters for a function, and you can define them such that what's required or optional or even allowed is dependent on other parameters you've specified. As a somewhat simple example, if you were writing a function that added a user to a group, you would need a username, and then either a group name or group id. You could configure this easily in PowerShell, such that the interpreter would handle the case where someone passed in a username and nothing else, or passed in both a group name or group id. Is there anything like that in Python? I know I could make both group_name and group_id optional, and then in the code raise a ValueError if both group_name and group_id are given, or if neither are given, but that's a lot of boilerplate code. I know with typing you can import overload so that you can at least define those cases for type hinting, but nothing at run time will actually enforce that. For my case I'm working with this godawful Grouper API and there are some API endpoints that will do a ton of things based on what parameters are sent in, but I'd like to do some sanity checking beforehand because the errors returned from the API are often not clear, and also the API will often do stupid things so passing in "invalid" parameters in a certain combination might not cause any errors, but might cause unexpected results. Also as I'm thinking to myself, maybe I right one mega function that will just take everything as an optional input as a private function, but then write simpler functions that use the private function but with more limited parameters so it's much harder for someone to get themselves in trouble.
|
|
#
¿
May 18, 2023 16:10
|
|
|
A Pydantic dataclass runs validations on the data before just returning a regular dataclass. So would the expectation be that the person consuming my functions would be generating the correct dataclass and then passing that into a function? Or I'm using a dataclass under the hood somehow?
|
|
#
¿
May 19, 2023 02:47
|
|
|
Flask is lightweight and lets you bolt lots of things onto it, which means it's more flexible, but some things are more repetitive. FastAPI is specifically for writing Rest APIs, so there's less boilerplate involved, but it lacks some of Flask's flexibility. A bit over a year ago I did a rewrite of an internal RestAPI tool we use from Groovy into Python. I'd done a bit with Flask at that point and so started to look at that, and Flask was able to do it, but I also spent a few days running through a FastAPI tutorial and decided to do my rewrite in that instead, as it was tailored to exactly what I was doing. FastAPI (via Starlette) can do templating using something like Jinja2, but I don't have any direct experience with that to compare it to Flask, if that's what you're doing.
|
|
#
¿
Jun 22, 2023 15:23
|
|
|
Ease of development aside, the big difference between FastAPI and Flask is that FastAPI is asynchronous, and Flask is not. Which may or may not matter to you. Strictly speaking, FastAPI and Flask probably aren't really "equivalent" tools, Flask is probably more equivalent to Starlette, which is the framework that FastAPI is built on. You've probably built something more equivalent to FastAPI on top of Flask. This really sounds like a case where momentum is a key deciding factor. The calculation would be different if you were starting from nothing, and maybe if you'd built all that framework on top of FastAPI instead of Flask the final product would be "better" but I'm not sure it would be "better" enough to warrant throwing away what you've done and re-implement it in FastAPI. Then again, I'll say I've had a blast developing in FastAPI, and maybe you'd have fun trying it out. It sounds like you've built out functionality (like auto-generation around models) that isn't in FastAPI, so maybe you'd have a blast porting that over to work with FastAPI. I've structured my app so that all the data interaction happens outside the "routes" so I could, if I wanted to, pretty easily port that stuff to some other tool as well, so that's certainly still possible with FastAPI.
|
|
#
¿
Jun 22, 2023 17:35
|
|
|
I've been using SQLModel, which is written by the same guy that made FastAPI. Basically it mixes Pydantic and SQLAlchemy together into a single object. So I can define a class that will be both a Pydantic model with all the typing and validation that entails, as well as a database table. Though, in practice, I've found it's not always as simple as it sounds. There are quite a few cases where the model you store in the database is not the model you want to present to the user, so you still end up writing two things, one a SQLModel object that defines the database, and another a Pydantic BaseModel object that you can present to the user.
|
|
#
¿
Jun 23, 2023 19:32
|
|
|
Continuing the Flask vs something else discussion, if you were to build a Python webapp where you were rendering and serving HTML with something like Jinja2, what would you recommend as a framework that supports async? Starlette seems like the closest equivalent to Flask as a "micro-framework". However because of the nature of ASGI, something like FastAPI just adds on to Starlette, so you can use it to render and serve Jinja2 templates just as well as Starlette, with the added bonus of routes being defined in a way I'm already familiar with (since I'm familiar with FastAPI). Is there another framework built on Starlette that would be better suited to rendering and serving HTML?
|
|
#
¿
Jun 28, 2023 17:20
|
|
|
I think the problem you actually want to solve is a better way to deploy the instrument control software onto the users' devices, because it seems like the pain point you have is having to get hands on keyboard to set this stuff up. Engineering a web app using cutting edge java script is probably way more complicated and prone to failure than a better system for deploying the control software you have already.
|
|
#
¿
Jul 6, 2023 04:26
|
|
|
I know a moderate amount of info about git but I still use GitHub Desktop as my primary interface. It does obfuscate some of the workings of git but also they end up being some of the stuff about git that really sucks. But I've found it meets the vast majority of needs, and certainly all the day to day needs. Even merge conflicts are not so bad, paired with vscode. When GitHub Desktop detects a merge conflict I open the file in vscode and vscode helps me fix it and I save the file and GitHub Desktop is happy.
|
|
#
¿
Aug 3, 2023 02:25
|
|
|
Had an opportunity to play around in Flask today to try something, after spending the last year in FastAPI and holy moly are these entirely different ecosystems.
|
|
#
¿
Aug 3, 2023 22:03
|
|
|
oatmealraisin posted:What have you noticed? I moved to fastapi a while ago and honestly was just using Flask for small APIs, and honestly beyond the endpoint niceties I dont really remember what I'm missing I'm working on a GUI app with Flask, specifically trying to build some forms, so obviously it's a totally different case than writing an API in FastAPI. But there's so many batteries included in Flask, and batteries you can easily add in. I was fine in FastAPI using Pydantic BaseSettings to import my settings from a .env file, but Flask just does it for me (as long as I install python-dotenv). And I'm using forms so I can just bolt-in Flask-WTF. And I thought it was annoying to have to hand code my forms into the jinja template, but it turns out I can just install bootcamp-flask to do that for me. And etc etc. I don't really think it's better or worse, it's just... different.
|
|
#
¿
Aug 4, 2023 17:33
|
|
|
This is maybe more of a DB question than a Python question, but I'm using SQL Alchemy so I'll ask it here... I've got a FastAPI project that's storing some stuff in a DB (SQLite locally for development, MySQL for production). I have a few columns that are strings, but are actually lists of data. So to read them I read the string and split on the comma to get my list of strings. I know I could setup a relationship and store these items in the list in another table, to do it in a more "native" way, but that adds complexity to the DB, and there's never a reason why I would want only one of these values in the list: I always want only a single one. These entries are rather "static", they don't get changed by user input, and in fact once created they rarely if ever change, only get read. I'm thinking I could do something like this but I'm not sure if there's any better options these days.
|
|
#
¿
Sep 13, 2023 21:39
|
|
|
I'm very upset that that is perfectly valid Python
|
|
#
¿
Sep 27, 2023 03:53
|
|
|
Why don't you just base Gem on one of the Pydantic objects?
|
|
#
¿
Sep 27, 2023 14:07
|
|
|
Macichne Leainig posted:I would say "just replace the abandonware" but I think we all know it's load-bearing abandonware at this point. It's one of the big downsides of open source ecosystems with a centralized package repository. When someone abandons "pillow" then the community can pick it up, but the package "pillow" still belongs to that abandoned project. So everybody has to work around that limitation.
|
|
#
¿
Sep 27, 2023 17:15
|
|
|
I've never tried it myself, but can you use Gem and BaseModel as mixins to make a new class?
|
|
#
¿
Sep 27, 2023 17:25
|
|
|
I wouldn't think about this problem as how you save data to disk, but instead what form do you want this data to take as you're using it (probably just a list or set of ticket IDs?). You could pickle that and write that to disk, or output as JSON and write that. Don't over think it.
|
|
#
¿
Sep 28, 2023 14:31
|
|
|
You can structure each "script" so that it can also be imported. So, not sure how it's structured now, but get each script into such a form where it gets executed from a single function call. Then at the end do something like if __name__ == "main" and have it call that single function call. That way if you execute the file as a script it will execute, but then you could import that function from another file and execute it there.
|
|
#
¿
Oct 11, 2023 22:12
|
|
|
I always lean towards existing standards so that I can use built in tools and make it easier for other systems to interact with it, so have you considered something like JSON?
|
|
#
¿
Nov 6, 2023 18:39
|
|
|
I'm confused because the callback is for your app's authentication, why does your app need to authenticate to itself to function? FWIW, using Google's oauth provider allows setting localhost as the callback URL.
|
|
#
¿
Nov 23, 2023 17:17
|
|
|
Is the not negating the entire statement or just the part before the or? I think you need more parenthesis, because the not isn't applying precisely how you want it. I realize this is a simplified example to demonstrate the issue, but in production do you want "right" to be an outcome? Because if so, I don't see why you're even bothering with the not to start with the negative case.
|
|
#
¿
Feb 25, 2024 17:07
|
|
|
And the reason it's absolute hell is... exactly the case you're running into now. Sometimes programming conversations can feel preachy when doing it the "wrong" way works for now, but this is a pretty clear case of how it being done the "wrong" way is right now directly making the program harder to understand and troubleshoot.
|
|
#
¿
Feb 25, 2024 17:28
|
|
|
This is probably a very luddite opinion of mine, and I'm sure it's objectively wrong, but I really hate when there are too many functions being called (especially within functions within functions etc). I somewhat frequently find myself tracing very closely through some random code base trying to figure out exactly what's going on, and going deeper and deeper with functions makes that more difficult. You'd think in theory that I should be able to look at a function and say "It's going to do X" the same way I can look at a line like "a = b + c" and know what's going to happen. But in practice, it doesn't work out that way, and I end up having to read through those functions to figure out exactly what's happening.
|
|
#
¿
Feb 27, 2024 18:49
|
|
|
mypy --strict or gtfo To be clear this is trolly advice, but the last big project I did at my old job I was able to get it to that point.
|
|
#
¿
Mar 23, 2024 00:28
|
|
|
|
| # ¿ Apr 29, 2024 05:24 |
|
|
I've done magical things with mkdocs material, except everyone at work liked it too much and I got fired over it.
|
|
#
¿
Mar 29, 2024 01:15
|
|