Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

I. M. Gei: Jun 26, 2005; CHIEFS

BITCH

Hed posted:

Did you uninstall Python 3.11 or upgrade to 3.12 since last time?

No, I haven't done either of those things. I haven't messed with the Python files at all.

Hed posted:

It looks like your venv is messed up. I�d take that �create process using� line one argument at a time and make sure all those files exist. Probably easier to bring up project settings and try to make a new venv dedicated to the project.

I'll take a look at the directories and see if the files have changed at all. After that I'll try making a new venv and see if that fixes it.

EDIT: My directories show the files, but when I click the icon for python.exe, it says the path doesn't exist. I tried creating a new venv and got the following error message:

pre:

Cannot run program "C:\Users\[USER_NAME]\AppData\Local\Microsoft\WindowsApps\python3.11.exe":
CreateProcess error=3, The system cannot find the path specified

so...... maybe I reinstall Python?

I. M. Gei fucked around with this message at 19:43 on Nov 28, 2023

# ? Nov 28, 2023 19:23

Adbot: ADBOT LOVES YOU

# ? May 28, 2024 08:33

I. M. Gei: Jun 26, 2005; CHIEFS

BITCH

Okay I (re)installed Python 3.12 and set the directory for the new venv to that instead of 3.11, and that seems to have fixed the problem.

Now if only I could remember how my old code worked. :doh:

# ? Nov 28, 2023 22:30

necrotic: Aug 2, 2005; I owe my brother big time for this!

I. M. Gei posted:

Now if only I could remember how my old code worked.

I don�t remember how my new code works

# ? Nov 28, 2023 22:36

teen phone cutie: Jun 18, 2012; last year i rewrote something awful from scratch because i hate myself

i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction:

I have a standalone API with endpoints to issue users JWTs either through /login, /register endpoints. These JWTs are signed to expire in 7 days and are never stored in the database, except for when users hit /logout, which will then add the token to a blacklist table.

That's probably more information then needed, but I'm looking for a way to compile the list of active users within the last 15 minutes, and I'm not sure the best way to do it. I also want to get a number of people who are hitting the API not logged in. My API is a flask application, so I'm assuming there's some way to intercept all endpoints, and add them to "active users" and "guests" lists in application state if they don't already exist and timestamp them, but I'm not sure if there's a Flask-y way to do this?

teen phone cutie fucked around with this message at 23:20 on Nov 28, 2023

# ? Nov 28, 2023 23:15

Macichne Leainig: Jul 26, 2012; by VG

necrotic posted:

I don�t remember how my new code works

Yeah as soon as I press enter/return, that line is unfortunately meaningless to me

# ? Nov 28, 2023 23:24

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

sugar free jazz posted:

this is in fact a personal workflow i can hand manage! pyautogui also feels very familiar since i've done some screen scraping with selenium, so this will hopefully work. wish it could be done quietly but don't let the perfect be the enemy of the good etc. appreciate it!

FWIW there's nothing wrong with hacky/scrappy solutions, as long as you go in open eyed and aware of the limitations. I'm a big fan of weird lovely scripts, because the alternative is often 'nothing gets even halfway automated'.

The trick is knowing when to say 'okay cool this has gotten too big, this needs to be refactored properly', which often means not letting management know about your scrappy bullshit, because they often don't know the difference.

# ? Nov 28, 2023 23:46

WHERE MY HAT IS AT: Jan 7, 2011

teen phone cutie posted:

i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction:

I have a standalone API with endpoints to issue users JWTs either through /login, /register endpoints. These JWTs are signed to expire in 7 days and are never stored in the database, except for when users hit /logout, which will then add the token to a blacklist table.

That's probably more information then needed, but I'm looking for a way to compile the list of active users within the last 15 minutes, and I'm not sure the best way to do it. I also want to get a number of people who are hitting the API not logged in. My API is a flask application, so I'm assuming there's some way to intercept all endpoints, and add them to "active users" and "guests" lists in application state if they don't already exist and timestamp them, but I'm not sure if there's a Flask-y way to do this?

If you want to execute this logic on each request so you can gather stats, what you're after is the before_request decorator. It doesn't pass any params, but you can import flask.request to have access to a request object to inspect.

As for the actual stat storage, you haven't said anything about what your backing data store is, but I'd probably do something like: for each request, extract the user ID out of the JWT and upsert a row in a DB table with the user ID and current timestamp + 15 minutes. Then, when you want to view active users, query the table for rows with a timestamp greater than current. Depending on space constraints, you'll likely want to clean this up periodically by dropping all rows with an expired timestamp.

Tracking users not logged in depends on how you want to use that data. Do you need to bucket it day by day? As a percentage of total requests? Just an absolute count? Harder to suggest an approach to that one without more details but maybe just a table of unique IPs + day of access would suffice.

# ? Nov 29, 2023 05:21

The Iron Rose: May 12, 2012; Cat Army

teen phone cutie posted:

i have a general question that i'm just now starting to look into, but would be great if someone could point me in the right direction:

I have a standalone API with endpoints to issue users JWTs either through /login, /register endpoints. These JWTs are signed to expire in 7 days and are never stored in the database, except for when users hit /logout, which will then add the token to a blacklist table.

That's probably more information then needed, but I'm looking for a way to compile the list of active users within the last 15 minutes, and I'm not sure the best way to do it. I also want to get a number of people who are hitting the API not logged in. My API is a flask application, so I'm assuming there's some way to intercept all endpoints, and add them to "active users" and "guests" lists in application state if they don't already exist and timestamp them, but I'm not sure if there's a Flask-y way to do this?

OpenTelemetry is your friend, but any APM implementation does the trick here. As above, parse the received JWT for the user ID, and store that. Whenever you receive an authenticated request, store that and the user ID as a span attribute. So too if you receive unauthenticated requests. Then just query over the last fifteen minutes for all the unique user IDs, which you can then correlate across any number of other interesting attributes about your service. The advantage of using opentelemetry over tossing the data in a DB is that you get much more interesting information about the way your application is behaving, get robust monitoring and alerting, and can perform all kinds of interesting queries to better understand how your service is being used and its interactions with dependencies.

OpenTelemetry has an auto-instrumentation library for flask so you can leverage that as well.

Jaeger is a good free OSS backend for visualizing traces. All commercial observability vendors I know of also support opentelemetry (Datadog, New Relic, Elastic, etc). Use opentelemetry rather than a vendor specific APM library so your data is portable. You can send your telemetry directly to the backend, or deploy an opentelemetry collector to handle extra parsing, processing, or other enrichment as desired.

The Iron Rose fucked around with this message at 08:08 on Nov 29, 2023

# ? Nov 29, 2023 07:56

I. M. Gei: Jun 26, 2005; CHIEFS

BITCH

I have another question that should be pretty easy to solve, but I want to make sure I don't gently caress it up.

I've got a .txt file with lines of numbers. Each line has 5 numbers, each number is either 1 or 2 digits, and the numbers in each line are separated by tab spaces. So in total, each line has somewhere between 10 and 15 characters(?) with tab spaces, new-line characters, and the end-of-file character included.

I need to read all 5 numbers in each line of my file, convert them to ints, filter out all of the tabs and other non-numerical characters, and save the numbers to a list of length 5.

I know how to open the file for reading, I *think* I know how to convert my numbers from strings to ints (just save them to a variable that's already defined as an int), and I know how to save everything to a list. What I'm not sure about is how to filter out all the whitespace characters without loving up the numbers, especially since the lines vary in length depending on how many total digits are in my numbers and I apparently have to read them all as strings (unless there's another way besides .read() and .readline() that I'm not aware of?).

What's the best way to go about this? It seems like a relatively simple solve, but my head hurts and I kinda wanna hurry up and get to the fun parts of my code rather than waste more brain cells on this one stupid part.

I. M. Gei fucked around with this message at 20:28 on Nov 29, 2023

# ? Nov 29, 2023 20:20

Chin Strap: Nov 24, 2002; I failed my TFLC Toxx, but I no longer need a double chin strap; Pillbug

Just use a CSV reader like pandas has.

# ? Nov 29, 2023 20:28

saintonan: Dec 7, 2009; Fields of glory shine eternal

I. M. Gei posted:

I have another question that should be pretty easy to solve, but I want to make sure I don't gently caress it up.

I've got a .txt file with lines of numbers. Each line has 5 numbers, each number is either 1 or 2 digits, and the numbers in each line are separated by tab spaces. So in total, each line has somewhere between 9 and 15 characters(?) with tab spaces, new-line characters, and the end-of-file character included.

I need to read all 5 numbers in each line of my file, convert them to ints, filter out all of the tabs and other non-numerical characters, and save the numbers to a list of length 5.

I know how to open the file for reading, I *think* I know how to convert my numbers from strings to ints (just save them to a variable that's already defined as an int), and I know how to save everything to a list. What I'm not sure about is how to filter out all the whitespace characters without loving up the numbers, especially since the lines vary in length depending on how many total digits are in my numbers and I apparently have to read them all as strings (unless there's another way besides .read() and .readline() that I'm not aware of?).

What's the best way to go about this? It seems like a relatively simple solve, but my head hurts and I kinda wanna hurry up and get to the fun parts of my code rather than waste more brain cells on this one stupid part.

If you use split() without an argument, it will split on any whitespace (that includes tabs). so list = f.readline().split() will populate the list with the numbers and throw away the whitespace.

# ? Nov 29, 2023 20:29

I. M. Gei: Jun 26, 2005; CHIEFS

BITCH

Chin Strap posted:

Just use a CSV reader like pandas has.

This is the embarrassing part where I have to ask how do I do this?

y'all I am very bad at programming, and my brain is the kind that's not good at filling in blanks when it's told to do something.

saintonan posted:

If you use split() without an argument, it will split on any whitespace (that includes tabs). so list = f.readline().split() will populate the list with the numbers and throw away the whitespace.

I'll give this a try.

I'm guessing it saves each number in the list as a string and I just have to replace them with ints?

# ? Nov 29, 2023 20:51

saintonan: Dec 7, 2009; Fields of glory shine eternal

I. M. Gei posted:

I'm guessing it saves each number in the list as a string and I just have to replace them with ints?

Python is a dynamically typed language, which means variable type only matters when you do something with it. I'm not entirely clear on what you want to do with this list of values, so your question is hard to answer.

# ? Nov 29, 2023 21:02

12 rats tied together: Sep 7, 2006

Your file is really just bytes. Python's `open()` function has configuration for how it should try to interpret the file -- if it should decode the bytes into strings / "words" or if it should just treat each line as bytes. The way to configure this is described here. If you did not specify binary mode, python is reading bytes off of the file system and doing this:

quote:

[...] in text mode (the default, or when 't' is included in the mode argument), the contents of the file are returned as str, the bytes having been first decoded using a platform-dependent encoding or using the specified encoding if given.

It's not really possible to store "an integer" or "a word" in a file. You can only write bytes, and then interpret them in a certain way.

As saintonan mentioned, Python is dynamically typed, so however you read the file you're free to convert the data from it into whatever type you need. You can tell what kind of a thing you have by feeding it into `type()`. You can cast a thing to a different type by feeding it into the built-in function for that type, for example, int(), str(), float().

Variables aren't really declared with a type either. A variable's name is just a label that gets attached to an object that lives in memory. The objects have the type, not the labels. Be careful that you don't misinterpret re-assigning a label as mutating an object:

Python code:

>>> word = "abcdef"
>>> type(word) # it's a string
<class 'str'>

>>> number = 123456
>>> type(number) # it's a number 
<class 'int'>

>>> word = number
>>> word
123456

>>> type(word) # its a number now.
<class 'int'>

>>> id(number)
2451604671088
>>> id(word) # they both point to the same object id, which was never mutated, and was always a number
2451604671088

# ? Nov 29, 2023 21:38

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

Here's a sort of minimal example:

Python code:

from pathlib import Path
filepath = Path("C:/Code/Butts/buttfile.txt")

def load_file_to_str(filename: Path) -> str:
    # This will load a text file (assuming default decoding) and return it as a big string
    with open(filename, 'r') as txtfile:
        return txtfile.read()

print(load_file_to_str(filepath))

This would return a string with line breaks for the whole file and then print it out, which should basically match what's in your text file, there are other methods too such as .readlines() etc that try and split it on line breaks.

I. M. Gei posted:

y'all I am very bad at programming, and my brain is the kind that's not good at filling in blanks when it's told to do something.

https://automatetheboringstuff.com/#toc Is what I would recommend if you're coming at this from a non-programmer perspective. This book is free online and is basically written to say 'Hey, you're not a developer, but you'd like to automate simple boring things' and I found it extremely helpful to me when I was getting started.

Falcon2001 fucked around with this message at 23:22 on Nov 29, 2023

# ? Nov 29, 2023 22:56

QuarkJets: Sep 8, 2008

I. M. Gei posted:

This is the embarrassing part where I have to ask how do I do this?

y'all I am very bad at programming, and my brain is the kind that's not good at filling in blanks when it's told to do something.

I'll give this a try.

I'm guessing it saves each number in the list as a string and I just have to replace them with ints?

That is correct, split() returns a list of strings. You can convert that into a list of ints or floats if you need that

# ? Nov 30, 2023 01:11

StumblyWumbly: Sep 12, 2007; Batmanticore!

I don't want to sound old, but what's up with kids today and their async statements?

My real question is I have some code running in a gui, my model may be modified by mouse events and I'm pretty sure the gui backend will make sure mouse events and display won't overlap, but I'd like to be sure so I wanted to add a mutex. Being lazy, I thought I'd just add a threading mutex, since they should all be pretty much the same, right? That's when I found asyncio.Lock. The documentation says it is "Not thread-safe", but I think it must mean you may need to wait while accessing it, as opposed to ROM which is perfectly thread safe?

I feel like I'm clear on the differences between threads and async. async seems neat and different, but a bit jankier than threads. Is there any real difference or preference between the threading.Lock vs asyncio.Lock? At a low level, they must be doing the same thing, but I could imagine differences in what happens when you need to wait for the lock.

# ? Dec 3, 2023 21:47

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

I take it to mean that it's designed for use if you have multiple coroutines running on a single underlying thread, and will fail in weird and wonderful ways if you try to use it when more than one actual thread is involved.

# ? Dec 3, 2023 22:30

Oysters Autobio: Mar 13, 2017

Falcon2001 posted:

Anyone have any experience with one of the Python-only frontend frameworks like Anvil or JustPy? I have some internal tooling I'd like to build with not a lot of time, and all things being equal, it'd be nice to not have to learn JS and a modern frontend library. The alternative would just be a bunch of CLI stuff using Click/etc, which isn't awful (or Textual if I decide to get really fancy) but it'd be nice to be able to just link someone to a URL.

Like sure, at some point I'll come back to it but I'd like to be able to slap together some basic UIs in front of some automation/diagnostic stuff. My only real requirements would be something that lets you build the whole UX in Python (so not just Django/Flask), and ideally something with a decent default style setup so stuff looks reasonably 'built this century'.

The Python front-end world seems pretty experimental to me at this point. Would absolutely love to be proven wrong because as someone who loves Python but also loves UI/UX and data viz, I'm sort of just delaying learning any JS frameworks.

Are Jupyter notebooks at all in your POC here? Cause there's some half decent looking Vuetify extension for ipywidgets.

Otherwise the only other one I'm familiar with is Plotly Dash which was made for data science dashboards rather than any UI.

Seems all the cool kids who want to avoid the JavaScript ecosystem are using HTMX + Flask/Django. Dunno how well that would work for an internal tool.

# ? Dec 4, 2023 13:55

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

StumblyWumbly posted:

Is there any real difference or preference between the threading.Lock vs asyncio.Lock? At a low level, they must be doing the same thing, but I could imagine differences in what happens when you need to wait for the lock.

Async locks are for when you have multiple waiters in the same asyncio event loop. Trying to access any async resource (lock, task result, etc.) from another event loop will typically generate an exception.

StumblyWumbly posted:

I feel like I'm clear on the differences between threads and async.

Honestly, no. They couldn't be more different. asyncio is a model for running multiple tasks in a single thread, to ensure the thread always has something to do even when a task is waiting for a result. This keeps the runtime, not the operating system, in charge of when the event loop may be yielded from one task to another one. This is the reason locking primitives are so lightweight in the async model: it's literally impossible for two tasks in the same event loop to concurrently access the same memory, because they cannot ever run at the same time.

Vulture Culture fucked around with this message at 17:34 on Dec 4, 2023

# ? Dec 4, 2023 17:23

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

Oysters Autobio posted:

The Python front-end world seems pretty experimental to me at this point. Would absolutely love to be proven wrong because as someone who loves Python but also loves UI/UX and data viz, I'm sort of just delaying learning any JS frameworks.

Are Jupyter notebooks at all in your POC here? Cause there's some half decent looking Vuetify extension for ipywidgets.

Otherwise the only other one I'm familiar with is Plotly Dash which was made for data science dashboards rather than any UI.

Seems all the cool kids who want to avoid the JavaScript ecosystem are using HTMX + Flask/Django. Dunno how well that would work for an internal tool.

Jupyter is on my list but more as a different project I posted about in here before for automating runbooks. I'll take a look at HTMX again and see about using that.

# ? Dec 4, 2023 18:14

StumblyWumbly: Sep 12, 2007; Batmanticore!

Thanks for the help, that clarifies things. I think I expected async to be more just because it's new.

Is it true that if I have thread lock protecting a resource that ends up getting used in async, things are protected? Or is there a potential issue because an async lock would automatically yield or await but the thread lock will not? This question is purely academic

# ? Dec 4, 2023 23:19

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

StumblyWumbly posted:

Thanks for the help, that clarifies things. I think I expected async to be more just because it's new.

Is it true that if I have thread lock protecting a resource that ends up getting used in async, things are protected? Or is there a potential issue because an async lock would automatically yield or await but the thread lock will not? This question is purely academic

You need to be really careful addressing synchronization in programs that are simultaneously using asyncio and worker threads. Trying to acquire a locked lock will block the thread, right? And as a general rule, you don't want to put blocking operations into the event loop. If you spin up worker threads for every synchronous task that could possibly hit a blocking operation, including working with thread-locked resources, you're fine.

So consider what Lock and RLock each do. Lock will block the thread until another thread releases the lock, but all the other tasks on your event loop are running in the same thread, so now as soon as one of them tries to wait on the lock, no other task will ever run again and you're deadlocked. RLock is re-entrant, meaning the same thread can call it multiple times and safely re-acquire the lock, which puts you in the opposite situation: attempts to acquire the lock from other coroutines in the loop, which all run in the same thread as the one you locked from, will always succeed and your lock will never, ever wait.

# ? Dec 5, 2023 03:59

StumblyWumbly: Sep 12, 2007; Batmanticore!

Vulture Culture posted:

You need to be really careful addressing synchronization in programs that are simultaneously using asyncio and worker threads. Trying to acquire a locked lock will block the thread, right? And as a general rule, you don't want to put blocking operations into the event loop. If you spin up worker threads for every synchronous task that could possibly hit a blocking operation, including working with thread-locked resources, you're fine.

So consider what Lock and RLock each do. Lock will block the thread until another thread releases the lock, but all the other tasks on your event loop are running in the same thread, so now as soon as one of them tries to wait on the lock, no other task will ever run again and you're deadlocked. RLock is re-entrant, meaning the same thread can call it multiple times and safely re-acquire the lock, which puts you in the opposite situation: attempts to acquire the lock from other coroutines in the loop, which all run in the same thread as the one you locked from, will always succeed and your lock will never, ever wait.

Ok, thanks, I see what you're getting at. Thread locks will bounce you to another thread, but because of their tie to the thread it wouldn't make sense to send you to another asyncio. I guess the other part is that since async won't yield unless explicitly told to, it doesn't make sense to have an async lock unless you do an async yield. My background is in FPGAs and microcontrollers, where things are just different.

It seems like asyncio is mainly for servicing high latency calls like network queries. Maybe they could be used to help with multiprocessing (which I haven't used) or maybe CUDA, but I assume those systems already have features so async isn't necessary. Is there a use case for this that I'm missing?

# ? Dec 5, 2023 05:34

QuarkJets: Sep 8, 2008

CUDA is a fully-featured task manager internally, you'd want to rely more on those built-in capabilities (which are basically transparent). But there's probably some usecase here I'm just not imagining, because I'm bad at asyncio

# ? Dec 5, 2023 05:50

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

When plotting stuff with Matplotlib/Seaborn, is there a simple way to ensure that the legend always winds up in the same spot outside of the plot(s)? I generally use the loc='upper right', bbox_to_anchor=(X, 1) arguments to get the legend outside and to the right of the plots, where X is some number like 1.2 or 1.4, but then I need to keep manually tweaking X to get the legend exactly where I want it (with its top roughly aligned to the top of the plots, and a bit of whitespace between the right border of the plots and the left border of the legend). This of course involves constantly regenerating the plot(s), which may or may not be fast depending on what I'm doing. Surely there is a less annoying way to do this?

# ? Dec 5, 2023 09:05

necrotic: Aug 2, 2005; I owe my brother big time for this!

StumblyWumbly posted:

It seems like asyncio is mainly for servicing high latency calls like network queries.

The io in asyncio is important: it is specifically for async _io_ operations. So yes, it�s so one network request doesn�t mean no other work can occur.

# ? Dec 6, 2023 00:19

Jose Cuervo: Aug 25, 2004

SQLAchemy question here. I have two models:

Python code:

class Subject(Base):
    __tablename__ = 'subjects'

    id: Mapped[int] = mapped_column(primary_key=True)
    first_name: Mapped[str] = mapped_column(String(60), nullable=False)
    last_name: Mapped[str] = mapped_column(String(60), nullable=False)
    visits: Mapped[list['Visit']] = relationship(cascade='all, delete-orphan',
                                                                      back_populates='subject')

class Visit(Base):
    __tablename__ = 'visits'

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    date: Mapped[str] = mapped_column(DateTime, nullable=False)
    amount_spent: Mapped[int] = mapped_column(Integer, nullable=False)
    units: Mapped[str] = mapped_column(String, nullable=False)
    subject_id: Mapped[int] = mapped_column(Integer, ForeignKey('subjects.id'), index=True)
    subject: Mapped['Subject'] = relationship(back_populates='visits')

I now want to write a query which returns the most recent visit for each subject in the database.
This is the query I have so far, but it does not work (results in a DetachedInstanceError error message):

Python code:

    q = select(Visit) \
                .join(Subject.visits) \
                .order_by(Visit.date.desc()).limit(1)

Any thoughts on what I am doing wrong?

# ? Dec 7, 2023 17:52

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

Jose Cuervo posted:

SQLAchemy question here. I have two models:

...

I now want to write a query which returns the most recent visit for each subject in the database.
This is the query I have so far, but it does not work (results in a DetachedInstanceError error message):
Python code:
    q = select(Visit) \
                .join(Subject.visits) \
                .order_by(Visit.date.desc()).limit(1)
Any thoughts on what I am doing wrong?

DetachedInstanceError usually results from trying to query without a session, or using a session that's already been closed. What's your full error text, and what's the code that actually executes this statement?

# ? Dec 7, 2023 21:29

Jose Cuervo: Aug 25, 2004

Vulture Culture posted:

DetachedInstanceError usually results from trying to query without a session, or using a session that's already been closed. What's your full error text, and what's the code that actually executes this statement?

I guess I should have said I am using a Jupyter notebook and in previous cells I have been able to successfully run other (simpler) queries against the database using the with Session() context managers, so I do not think the issue is that the session has already been closed.

The code for executing the statement is:

Python code:

with Session() as session:
    with session.begin():
        q = select(Visit) \
                .join(Subject.visits) \
                .order_by(Visit.date.desc()).limit(1)
        r = session.scalars(q).all()
        print(r[0])

and the full error message is:

Python code:

DetachedInstanceError                     Traceback (most recent call last)
File ~\Anaconda3\envs\cam\Lib\site-packages\IPython\core\formatters.py:708, in PlainTextFormatter.__call__(self, obj)
    701 stream = StringIO()
    702 printer = pretty.RepresentationPrinter(stream, self.verbose,
    703     self.max_width, self.newline,
    704     max_seq_length=self.max_seq_length,
    705     singleton_pprinters=self.singleton_printers,
    706     type_pprinters=self.type_printers,
    707     deferred_pprinters=self.deferred_printers)
--> 708 printer.pretty(obj)
    709 printer.flush()
    710 return stream.getvalue()

File ~\Anaconda3\envs\cam\Lib\site-packages\IPython\lib\pretty.py:410, in RepresentationPrinter.pretty(self, obj)
    407                         return meth(obj, self, cycle)
    408                 if cls is not object \
    409                         and callable(cls.__dict__.get('__repr__')):
--> 410                     return _repr_pprint(obj, self, cycle)
    412     return _default_pprint(obj, self, cycle)
    413 finally:

File ~\Anaconda3\envs\cam\Lib\site-packages\IPython\lib\pretty.py:778, in _repr_pprint(obj, p, cycle)
    776 """A pprint that just redirects to the normal repr function."""
    777 # Find newlines and replace them with p.break_()
--> 778 output = repr(obj)
    779 lines = output.splitlines()
    780 with p.group():

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\engine\row.py:245, in Row.__repr__(self)
    244 def __repr__(self) -> str:
--> 245     return repr(sql_util._repr_row(self))

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\sql\util.py:598, in _repr_row.__repr__(self)
    595 def __repr__(self) -> str:
    596     trunc = self.trunc
    597     return "(%s%s)" % (
--> 598         ", ".join(trunc(value) for value in self.row),
    599         "," if len(self.row) == 1 else "",
    600     )

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\sql\util.py:598, in <genexpr>(.0)
    595 def __repr__(self) -> str:
    596     trunc = self.trunc
    597     return "(%s%s)" % (
--> 598         ", ".join(trunc(value) for value in self.row),
    599         "," if len(self.row) == 1 else "",
    600     )

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\sql\util.py:565, in _repr_base.trunc(self, value)
    564 def trunc(self, value: Any) -> str:
--> 565     rep = repr(value)
    566     lenrep = len(rep)
    567     if lenrep > self.max_chars:

File models.py:93, in Visit.__repr__(self)
      92 def __repr__(self):
 ---> 93     return f"{self.date.strftime('%Y-%m-%d')}"

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\orm\attributes.py:566, in InstrumentedAttribute.__get__(self, instance, owner)
    564 except AttributeError as err:
    565     raise orm_exc.UnmappedInstanceError(instance) from err
--> 566 return self.impl.get(state, dict_)

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\orm\attributes.py:1086, in AttributeImpl.get(self, state, dict_, passive)
   1083 if not passive & CALLABLES_OK:
   1084     return PASSIVE_NO_RESULT
-> 1086 value = self._fire_loader_callables(state, key, passive)
   1088 if value is PASSIVE_NO_RESULT or value is NO_VALUE:
   1089     return value

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy \orm\attributes.py:1116, in AttributeImpl._fire_loader_callables(self, state, key, passive)
   1108 def _fire_loader_callables(
   1109     self, state: InstanceState[Any], key: str, passive:  PassiveFlag
   1110 ) -> Any:
   1111     if (
   1112         self.accepts_scalar_loader
   1113         and self.load_on_unexpire
   1114         and key in state.expired_attributes
   1115     ):
-> 1116         return state._load_expired(state, passive)
   1117     elif key in state.callables:
   1118         callable_ = state.callables[key]

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\orm\state.py:798, in InstanceState._load_expired(self, state, passive)
    791 toload = self.expired_attributes.intersection(self.unmodified)
    792 toload = toload.difference(
    793     attr
    794     for attr in toload
    795     if not self.manager[attr].impl.load_on_unexpire
    796 )
--> 798 self.manager.expired_attribute_loader(self, toload, passive)
    800 # if the loader failed, or this
    801 # instance state didn't have an identity,
    802 # the attributes still might be in the callables
    803 # dict.  ensure they are removed.
    804 self.expired_attributes.clear()

File ~\Anaconda3\envs\cam\Lib\site-packages\sqlalchemy\orm\loading.py:1582, in load_scalar_attributes(mapper, state, attribute_names, passive)
   1580 session = state.session
   1581 if not session:
-> 1582     raise orm_exc.DetachedInstanceError(
   1583         "Instance %s is not bound to a Session; "
   1584         "attribute refresh operation cannot proceed" % (state_str(state))
   1585     )
   1587 no_autoflush = bool(passive & attributes.NO_AUTOFLUSH)
   1589 # in the case of inheritance, particularly concrete and abstract
   1590 # concrete inheritance, the class manager might have some keys
   1591 # of attributes on the superclass that we didn't actually map.
   1592 # These could be mapped as "concrete, don't load" or could be completely
   1593 # excluded from the mapping and we know nothing about them.  Filter them
   1594 # here to prevent them from coming through.

DetachedInstanceError: Instance <Visit at 0x166c3b74250> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: https://sqlalche.me/e/20/bhk3)

EDIT: You are correct. I was attempting to access r[0] from outside the cell/ scope of the context manager.

So that solves the error message, but I would still appreciate any pointers on how to write the correct query, because what I have is not correct. As written it only returns a single result, but if I remove the .limit(1) then it returns all visits ordered by date, and not just the single most recent visit for each subject.

Jose Cuervo fucked around with this message at 02:17 on Dec 8, 2023

# ? Dec 7, 2023 22:02

Jose Cuervo: Aug 25, 2004

Jose Cuervo posted:

SQLAchemy question here. I have two models:

Python code:

class Subject(Base):
    __tablename__ = 'subjects'

    id: Mapped[int] = mapped_column(primary_key=True)
    first_name: Mapped[str] = mapped_column(String(60), nullable=False)
    last_name: Mapped[str] = mapped_column(String(60), nullable=False)
    visits: Mapped[list['Visit']] = relationship(cascade='all, delete-orphan',
                                                                      back_populates='subject')

class Visit(Base):
    __tablename__ = 'visits'

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    date: Mapped[str] = mapped_column(DateTime, nullable=False)
    amount_spent: Mapped[int] = mapped_column(Integer, nullable=False)
    units: Mapped[str] = mapped_column(String, nullable=False)
    subject_id: Mapped[int] = mapped_column(Integer, ForeignKey('subjects.id'), index=True)
    subject: Mapped['Subject'] = relationship(back_populates='visits')

Python code:

    q = select(Visit) \
                .join(Subject.visits) \
                .order_by(Visit.date.desc()).limit(1)

Any thoughts on what I am doing wrong?

Once I addressed the issue of trying to access the result outside of the context manager, the query which returns what I want is:

Python code:

q = Subject.id, func.max(Visit.date)) \
        .join(Subject.visits) \
        .group_by(Visit.subject_id) \
        .order_by(Subject.last_name)

# ? Dec 8, 2023 19:14

LochNessMonster: Feb 3, 2005; I need about three fitty

Is anyone doing the advent of code by any chance?

I started with day 1 and while my code works fine for the test data set it�s returning a wrong anwser for the real data set.

I�m getting a file with a string of alohanumeric data per line. I need to get the first and last number of each line and combine them. �bein3bdj4bdn9� would mean 3 and 9, making 39. This was fairly simple. I used a regex to match all digits in a list and concat the first/last items.

Part 2 was where the data set could also include written numbers (one, two, etc). Tuned the regex to include those which worked fine on the test data set.

On the real data set the code runs with no problems, but the number I get is rejected by AoC saying it�s wrong. I�ve been looking at samples and they all seem to be correctly taking the first and last number from the string, spelled out number or digit.

Any ideas on how to troubleshoot such an issue would be appreciated.

# ? Dec 12, 2023 16:25

Armitag3: Mar 15, 2020; Forget it Jake, it's cybertown.

LochNessMonster posted:

Is anyone doing the advent of code by any chance?

I started with day 1 and while my code works fine for the test data set it�s returning a wrong anwser for the real data set.

I�m getting a file with a string of alohanumeric data per line. I need to get the first and last number of each line and combine them. �bein3bdj4bdn9� would mean 3 and 9, making 39. This was fairly simple. I used a regex to match all digits in a list and concat the first/last items.

Part 2 was where the data set could also include written numbers (one, two, etc). Tuned the regex to include those which worked fine on the test data set.

On the real data set the code runs with no problems, but the number I get is rejected by AoC saying it�s wrong. I�ve been looking at samples and they all seem to be correctly taking the first and last number from the string, spelled out number or digit.

Any ideas on how to troubleshoot such an issue would be appreciated.

Yeah this is a common trip up right on day one that got me too. Without giving the game away here�s two things you should be aware:
* some words might overlap (�twone�)
* regex-replacing, given the above, might give you two results depending on which direction you do it from (�2ne� vs �tw1�)

Don�t give up the elves need you

# ? Dec 12, 2023 16:46

LochNessMonster: Feb 3, 2005; I need about three fitty

Armitag3 posted:

Yeah this is a common trip up right on day one that got me too. Without giving the game away here�s two things you should be aware:
* some words might overlap (�twone�)
* regex-replacing, given the above, might give you two results depending on which direction you do it from (�2ne� vs �tw1�)

Don�t give up the elves need you

Thanks, I didn�t take the overlap into account. Coming up with a solution is going to be challenging, but interesting.

How did you find out this was the problem?

# ? Dec 12, 2023 17:00

Armitag3: Mar 15, 2020; Forget it Jake, it's cybertown.

LochNessMonster posted:

Thanks, I didn�t take the overlap into account. Coming up with a solution is going to be challenging, but interesting.

How did you find out this was the problem?

My solution involved reversing the string (I�m doing it in C++ this year). When I ran it with the string already reversed, I got different results and that�s when i realised that it matters which way you parse the string from first.

# ? Dec 12, 2023 17:04

WHERE MY HAT IS AT: Jan 7, 2011

Curious to hear about other folks' solutions! I used a trie for part two so I could iterate over the string and check substring prefixes as I went along until I hit a valid word.

However, I hadn't implemented a trie since I was in school, which was close to a decade ago now, so re-learning it all burned me out on puzzles and I haven't bothered doing part two of any of the other days.

# ? Dec 12, 2023 18:18

LochNessMonster: Feb 3, 2005; I need about three fitty

WHERE MY HAT IS AT posted:

Curious to hear about other folks' solutions! I used a trie for part two so I could iterate over the string and check substring prefixes as I went along until I hit a valid word.

However, I hadn't implemented a trie since I was in school, which was close to a decade ago now, so re-learning it all burned me out on puzzles and I haven't bothered doing part two of any of the other days.

I solved it by adjusting my regex and using a look ahead. So instead of r�\d|one|two|three|etc� I went with r�(?=(\d|one|two|three|etc))�

I never heard of a trie before (had to look it up), good to know about though. Can you share your implementation?

LochNessMonster fucked around with this message at 19:58 on Dec 12, 2023

# ? Dec 12, 2023 19:43

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

I had to use a third party regex library that allows for overlapping results; I had the exact same issue.

# ? Dec 12, 2023 20:38

QuarkJets: Sep 8, 2008

There may be some cutesy itertools solution, I always like something that avoids regex even if it winds up being a little slower. Might think about it tonight

# ? Dec 12, 2023 22:39

Adbot: ADBOT LOVES YOU

# ? May 28, 2024 08:33

Zoracle Zed: Jul 10, 2001

Maybe check out the 'sliding_window' iterator from the itertools recipes, which continues to drive me insane for not being importable but instead requires copy-pasting.

# ? Dec 12, 2023 23:11

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »