|
Hed posted:Did you uninstall Python 3.11 or upgrade to 3.12 since last time? No, I haven't done either of those things. I haven't messed with the Python files at all. Hed posted:It looks like your venv is messed up. I’d take that “create process using” line one argument at a time and make sure all those files exist. Probably easier to bring up project settings and try to make a new venv dedicated to the project. I'll take a look at the directories and see if the files have changed at all. After that I'll try making a new venv and see if that fixes it. EDIT: My directories show the files, but when I click the icon for python.exe, it says the path doesn't exist. I tried creating a new venv and got the following error message: pre:Cannot run program "C:\Users\[USER_NAME]\AppData\Local\Microsoft\WindowsApps\python3.11.exe": CreateProcess error=3, The system cannot find the path specified I. M. Gei fucked around with this message at 19:43 on Nov 28, 2023 |
# ? Nov 28, 2023 19:23 |
|
|
# ? May 30, 2024 14:06 |
|
Okay I (re)installed Python 3.12 and set the directory for the new venv to that instead of 3.11, and that seems to have fixed the problem. Now if only I could remember how my old code worked.
|
# ? Nov 28, 2023 22:30 |
|
I. M. Gei posted:Now if only I could remember how my old code worked. I don’t remember how my new code works
|
# ? Nov 28, 2023 22:36 |
|
i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction: I have a standalone API with endpoints to issue users JWTs either through /login, /register endpoints. These JWTs are signed to expire in 7 days and are never stored in the database, except for when users hit /logout, which will then add the token to a blacklist table. That's probably more information then needed, but I'm looking for a way to compile the list of active users within the last 15 minutes, and I'm not sure the best way to do it. I also want to get a number of people who are hitting the API not logged in. My API is a flask application, so I'm assuming there's some way to intercept all endpoints, and add them to "active users" and "guests" lists in application state if they don't already exist and timestamp them, but I'm not sure if there's a Flask-y way to do this? teen phone cutie fucked around with this message at 23:20 on Nov 28, 2023 |
# ? Nov 28, 2023 23:15 |
|
necrotic posted:I don’t remember how my new code works Yeah as soon as I press enter/return, that line is unfortunately meaningless to me
|
# ? Nov 28, 2023 23:24 |
|
sugar free jazz posted:this is in fact a personal workflow i can hand manage! pyautogui also feels very familiar since i've done some screen scraping with selenium, so this will hopefully work. wish it could be done quietly but don't let the perfect be the enemy of the good etc. appreciate it! FWIW there's nothing wrong with hacky/scrappy solutions, as long as you go in open eyed and aware of the limitations. I'm a big fan of weird lovely scripts, because the alternative is often 'nothing gets even halfway automated'. The trick is knowing when to say 'okay cool this has gotten too big, this needs to be refactored properly', which often means not letting management know about your scrappy bullshit, because they often don't know the difference.
|
# ? Nov 28, 2023 23:46 |
|
teen phone cutie posted:i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction: If you want to execute this logic on each request so you can gather stats, what you're after is the before_request decorator. It doesn't pass any params, but you can import flask.request to have access to a request object to inspect. As for the actual stat storage, you haven't said anything about what your backing data store is, but I'd probably do something like: for each request, extract the user ID out of the JWT and upsert a row in a DB table with the user ID and current timestamp + 15 minutes. Then, when you want to view active users, query the table for rows with a timestamp greater than current. Depending on space constraints, you'll likely want to clean this up periodically by dropping all rows with an expired timestamp. Tracking users not logged in depends on how you want to use that data. Do you need to bucket it day by day? As a percentage of total requests? Just an absolute count? Harder to suggest an approach to that one without more details but maybe just a table of unique IPs + day of access would suffice.
|
# ? Nov 29, 2023 05:21 |
|
teen phone cutie posted:i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction: OpenTelemetry is your friend, but any APM implementation does the trick here. As above, parse the received JWT for the user ID, and store that. Whenever you receive an authenticated request, store that and the user ID as a span attribute. So too if you receive unauthenticated requests. Then just query over the last fifteen minutes for all the unique user IDs, which you can then correlate across any number of other interesting attributes about your service. The advantage of using opentelemetry over tossing the data in a DB is that you get much more interesting information about the way your application is behaving, get robust monitoring and alerting, and can perform all kinds of interesting queries to better understand how your service is being used and its interactions with dependencies. OpenTelemetry has an auto-instrumentation library for flask so you can leverage that as well. Jaeger is a good free OSS backend for visualizing traces. All commercial observability vendors I know of also support opentelemetry (Datadog, New Relic, Elastic, etc). Use opentelemetry rather than a vendor specific APM library so your data is portable. You can send your telemetry directly to the backend, or deploy an opentelemetry collector to handle extra parsing, processing, or other enrichment as desired. The Iron Rose fucked around with this message at 08:08 on Nov 29, 2023 |
# ? Nov 29, 2023 07:56 |
|
I have another question that should be pretty easy to solve, but I want to make sure I don't gently caress it up. I've got a .txt file with lines of numbers. Each line has 5 numbers, each number is either 1 or 2 digits, and the numbers in each line are separated by tab spaces. So in total, each line has somewhere between 10 and 15 characters(?) with tab spaces, new-line characters, and the end-of-file character included. I need to read all 5 numbers in each line of my file, convert them to ints, filter out all of the tabs and other non-numerical characters, and save the numbers to a list of length 5. I know how to open the file for reading, I *think* I know how to convert my numbers from strings to ints (just save them to a variable that's already defined as an int), and I know how to save everything to a list. What I'm not sure about is how to filter out all the whitespace characters without loving up the numbers, especially since the lines vary in length depending on how many total digits are in my numbers and I apparently have to read them all as strings (unless there's another way besides .read() and .readline() that I'm not aware of?). What's the best way to go about this? It seems like a relatively simple solve, but my head hurts and I kinda wanna hurry up and get to the fun parts of my code rather than waste more brain cells on this one stupid part. I. M. Gei fucked around with this message at 20:28 on Nov 29, 2023 |
# ? Nov 29, 2023 20:20 |
|
Just use a CSV reader like pandas has.
|
# ? Nov 29, 2023 20:28 |
|
I. M. Gei posted:I have another question that should be pretty easy to solve, but I want to make sure I don't gently caress it up. If you use split() without an argument, it will split on any whitespace (that includes tabs). so list = f.readline().split() will populate the list with the numbers and throw away the whitespace.
|
# ? Nov 29, 2023 20:29 |
|
Chin Strap posted:Just use a CSV reader like pandas has. This is the embarrassing part where I have to ask how do I do this? y'all I am very bad at programming, and my brain is the kind that's not good at filling in blanks when it's told to do something. saintonan posted:If you use split() without an argument, it will split on any whitespace (that includes tabs). so list = f.readline().split() will populate the list with the numbers and throw away the whitespace. I'll give this a try. I'm guessing it saves each number in the list as a string and I just have to replace them with ints?
|
# ? Nov 29, 2023 20:51 |
|
I. M. Gei posted:I'm guessing it saves each number in the list as a string and I just have to replace them with ints? Python is a dynamically typed language, which means variable type only matters when you do something with it. I'm not entirely clear on what you want to do with this list of values, so your question is hard to answer.
|
# ? Nov 29, 2023 21:02 |
|
Your file is really just bytes. Python's `open()` function has configuration for how it should try to interpret the file -- if it should decode the bytes into strings / "words" or if it should just treat each line as bytes. The way to configure this is described here. If you did not specify binary mode, python is reading bytes off of the file system and doing this:quote:[...] in text mode (the default, or when 't' is included in the mode argument), the contents of the file are returned as str, the bytes having been first decoded using a platform-dependent encoding or using the specified encoding if given. As saintonan mentioned, Python is dynamically typed, so however you read the file you're free to convert the data from it into whatever type you need. You can tell what kind of a thing you have by feeding it into `type()`. You can cast a thing to a different type by feeding it into the built-in function for that type, for example, int(), str(), float(). Variables aren't really declared with a type either. A variable's name is just a label that gets attached to an object that lives in memory. The objects have the type, not the labels. Be careful that you don't misinterpret re-assigning a label as mutating an object: Python code:
|
# ? Nov 29, 2023 21:38 |
|
Here's a sort of minimal example:Python code:
I. M. Gei posted:y'all I am very bad at programming, and my brain is the kind that's not good at filling in blanks when it's told to do something. https://automatetheboringstuff.com/#toc Is what I would recommend if you're coming at this from a non-programmer perspective. This book is free online and is basically written to say 'Hey, you're not a developer, but you'd like to automate simple boring things' and I found it extremely helpful to me when I was getting started. Falcon2001 fucked around with this message at 23:22 on Nov 29, 2023 |
# ? Nov 29, 2023 22:56 |
|
I. M. Gei posted:This is the embarrassing part where I have to ask how do I do this? That is correct, split() returns a list of strings. You can convert that into a list of ints or floats if you need that
|
# ? Nov 30, 2023 01:11 |
|
I don't want to sound old, but what's up with kids today and their async statements? My real question is I have some code running in a gui, my model may be modified by mouse events and I'm pretty sure the gui backend will make sure mouse events and display won't overlap, but I'd like to be sure so I wanted to add a mutex. Being lazy, I thought I'd just add a threading mutex, since they should all be pretty much the same, right? That's when I found asyncio.Lock. The documentation says it is "Not thread-safe", but I think it must mean you may need to wait while accessing it, as opposed to ROM which is perfectly thread safe? I feel like I'm clear on the differences between threads and async. async seems neat and different, but a bit jankier than threads. Is there any real difference or preference between the threading.Lock vs asyncio.Lock? At a low level, they must be doing the same thing, but I could imagine differences in what happens when you need to wait for the lock.
|
# ? Dec 3, 2023 21:47 |
|
I take it to mean that it's designed for use if you have multiple coroutines running on a single underlying thread, and will fail in weird and wonderful ways if you try to use it when more than one actual thread is involved.
|
# ? Dec 3, 2023 22:30 |
|
Falcon2001 posted:Anyone have any experience with one of the Python-only frontend frameworks like Anvil or JustPy? I have some internal tooling I'd like to build with not a lot of time, and all things being equal, it'd be nice to not have to learn JS and a modern frontend library. The alternative would just be a bunch of CLI stuff using Click/etc, which isn't awful (or Textual if I decide to get really fancy) but it'd be nice to be able to just link someone to a URL. The Python front-end world seems pretty experimental to me at this point. Would absolutely love to be proven wrong because as someone who loves Python but also loves UI/UX and data viz, I'm sort of just delaying learning any JS frameworks. Are Jupyter notebooks at all in your POC here? Cause there's some half decent looking Vuetify extension for ipywidgets. Otherwise the only other one I'm familiar with is Plotly Dash which was made for data science dashboards rather than any UI. Seems all the cool kids who want to avoid the JavaScript ecosystem are using HTMX + Flask/Django. Dunno how well that would work for an internal tool.
|
# ? Dec 4, 2023 13:55 |
|
StumblyWumbly posted:Is there any real difference or preference between the threading.Lock vs asyncio.Lock? At a low level, they must be doing the same thing, but I could imagine differences in what happens when you need to wait for the lock. StumblyWumbly posted:I feel like I'm clear on the differences between threads and async. Vulture Culture fucked around with this message at 17:34 on Dec 4, 2023 |
# ? Dec 4, 2023 17:23 |
|
Oysters Autobio posted:The Python front-end world seems pretty experimental to me at this point. Would absolutely love to be proven wrong because as someone who loves Python but also loves UI/UX and data viz, I'm sort of just delaying learning any JS frameworks. Jupyter is on my list but more as a different project I posted about in here before for automating runbooks. I'll take a look at HTMX again and see about using that.
|
# ? Dec 4, 2023 18:14 |
|
Thanks for the help, that clarifies things. I think I expected async to be more just because it's new. Is it true that if I have thread lock protecting a resource that ends up getting used in async, things are protected? Or is there a potential issue because an async lock would automatically yield or await but the thread lock will not? This question is purely academic
|
# ? Dec 4, 2023 23:19 |
|
StumblyWumbly posted:Thanks for the help, that clarifies things. I think I expected async to be more just because it's new. So consider what Lock and RLock each do. Lock will block the thread until another thread releases the lock, but all the other tasks on your event loop are running in the same thread, so now as soon as one of them tries to wait on the lock, no other task will ever run again and you're deadlocked. RLock is re-entrant, meaning the same thread can call it multiple times and safely re-acquire the lock, which puts you in the opposite situation: attempts to acquire the lock from other coroutines in the loop, which all run in the same thread as the one you locked from, will always succeed and your lock will never, ever wait.
|
# ? Dec 5, 2023 03:59 |
|
Vulture Culture posted:You need to be really careful addressing synchronization in programs that are simultaneously using asyncio and worker threads. Trying to acquire a locked lock will block the thread, right? And as a general rule, you don't want to put blocking operations into the event loop. If you spin up worker threads for every synchronous task that could possibly hit a blocking operation, including working with thread-locked resources, you're fine. Ok, thanks, I see what you're getting at. Thread locks will bounce you to another thread, but because of their tie to the thread it wouldn't make sense to send you to another asyncio. I guess the other part is that since async won't yield unless explicitly told to, it doesn't make sense to have an async lock unless you do an async yield. My background is in FPGAs and microcontrollers, where things are just different. It seems like asyncio is mainly for servicing high latency calls like network queries. Maybe they could be used to help with multiprocessing (which I haven't used) or maybe CUDA, but I assume those systems already have features so async isn't necessary. Is there a use case for this that I'm missing?
|
# ? Dec 5, 2023 05:34 |
|
CUDA is a fully-featured task manager internally, you'd want to rely more on those built-in capabilities (which are basically transparent). But there's probably some usecase here I'm just not imagining, because I'm bad at asyncio
|
# ? Dec 5, 2023 05:50 |
|
When plotting stuff with Matplotlib/Seaborn, is there a simple way to ensure that the legend always winds up in the same spot outside of the plot(s)? I generally use the loc='upper right', bbox_to_anchor=(X, 1) arguments to get the legend outside and to the right of the plots, where X is some number like 1.2 or 1.4, but then I need to keep manually tweaking X to get the legend exactly where I want it (with its top roughly aligned to the top of the plots, and a bit of whitespace between the right border of the plots and the left border of the legend). This of course involves constantly regenerating the plot(s), which may or may not be fast depending on what I'm doing. Surely there is a less annoying way to do this?
|
# ? Dec 5, 2023 09:05 |
|
StumblyWumbly posted:
The io in asyncio is important: it is specifically for async _io_ operations. So yes, it’s so one network request doesn’t mean no other work can occur.
|
# ? Dec 6, 2023 00:19 |
|
SQLAchemy question here. I have two models:Python code:
This is the query I have so far, but it does not work (results in a DetachedInstanceError error message): Python code:
|
# ? Dec 7, 2023 17:52 |
|
Jose Cuervo posted:SQLAchemy question here. I have two models:
|
# ? Dec 7, 2023 21:29 |
|
Vulture Culture posted:DetachedInstanceError usually results from trying to query without a session, or using a session that's already been closed. What's your full error text, and what's the code that actually executes this statement? I guess I should have said I am using a Jupyter notebook and in previous cells I have been able to successfully run other (simpler) queries against the database using the with Session() context managers, so I do not think the issue is that the session has already been closed. The code for executing the statement is: Python code:
Python code:
So that solves the error message, but I would still appreciate any pointers on how to write the correct query, because what I have is not correct. As written it only returns a single result, but if I remove the .limit(1) then it returns all visits ordered by date, and not just the single most recent visit for each subject. Jose Cuervo fucked around with this message at 02:17 on Dec 8, 2023 |
# ? Dec 7, 2023 22:02 |
|
Jose Cuervo posted:SQLAchemy question here. I have two models: Once I addressed the issue of trying to access the result outside of the context manager, the query which returns what I want is: Python code:
|
# ? Dec 8, 2023 19:14 |
|
Is anyone doing the advent of code by any chance? I started with day 1 and while my code works fine for the test data set it’s returning a wrong anwser for the real data set. I’m getting a file with a string of alohanumeric data per line. I need to get the first and last number of each line and combine them. “bein3bdj4bdn9” would mean 3 and 9, making 39. This was fairly simple. I used a regex to match all digits in a list and concat the first/last items. Part 2 was where the data set could also include written numbers (one, two, etc). Tuned the regex to include those which worked fine on the test data set. On the real data set the code runs with no problems, but the number I get is rejected by AoC saying it’s wrong. I’ve been looking at samples and they all seem to be correctly taking the first and last number from the string, spelled out number or digit. Any ideas on how to troubleshoot such an issue would be appreciated.
|
# ? Dec 12, 2023 16:25 |
|
LochNessMonster posted:Is anyone doing the advent of code by any chance? Yeah this is a common trip up right on day one that got me too. Without giving the game away here’s two things you should be aware: * some words might overlap (“twone”) * regex-replacing, given the above, might give you two results depending on which direction you do it from (“2ne” vs “tw1”) Don’t give up the elves need you
|
# ? Dec 12, 2023 16:46 |
|
Armitag3 posted:Yeah this is a common trip up right on day one that got me too. Without giving the game away here’s two things you should be aware: Thanks, I didn’t take the overlap into account. Coming up with a solution is going to be challenging, but interesting. How did you find out this was the problem?
|
# ? Dec 12, 2023 17:00 |
|
LochNessMonster posted:Thanks, I didn’t take the overlap into account. Coming up with a solution is going to be challenging, but interesting. My solution involved reversing the string (I’m doing it in C++ this year). When I ran it with the string already reversed, I got different results and that’s when i realised that it matters which way you parse the string from first.
|
# ? Dec 12, 2023 17:04 |
|
Curious to hear about other folks' solutions! I used a trie for part two so I could iterate over the string and check substring prefixes as I went along until I hit a valid word. However, I hadn't implemented a trie since I was in school, which was close to a decade ago now, so re-learning it all burned me out on puzzles and I haven't bothered doing part two of any of the other days.
|
# ? Dec 12, 2023 18:18 |
|
WHERE MY HAT IS AT posted:Curious to hear about other folks' solutions! I used a trie for part two so I could iterate over the string and check substring prefixes as I went along until I hit a valid word. I solved it by adjusting my regex and using a look ahead. So instead of r”\d|one|two|three|etc” I went with r”(?=(\d|one|two|three|etc))” I never heard of a trie before (had to look it up), good to know about though. Can you share your implementation? LochNessMonster fucked around with this message at 19:58 on Dec 12, 2023 |
# ? Dec 12, 2023 19:43 |
|
I had to use a third party regex library that allows for overlapping results; I had the exact same issue.
|
# ? Dec 12, 2023 20:38 |
|
There may be some cutesy itertools solution, I always like something that avoids regex even if it winds up being a little slower. Might think about it tonight
|
# ? Dec 12, 2023 22:39 |
|
|
# ? May 30, 2024 14:06 |
|
Maybe check out the 'sliding_window' iterator from the itertools recipes, which continues to drive me insane for not being importable but instead requires copy-pasting.
|
# ? Dec 12, 2023 23:11 |