Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

WHERE MY HAT IS AT: Jan 7, 2011

We�re pretty good about hinting in first party code but it all falls down as soon as you use a third party library for which there are no types. Fine for Flask/Django/whatever other popular library but just lol if I try to get anyone to write our own type packages for third party deps we use.

# ? Sep 3, 2023 02:26

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 22:36

QuarkJets: Sep 8, 2008

Good third party packages like numpy tend to use good type consistency

Good groups too

# ? Sep 3, 2023 02:35

Generic Monk: Oct 31, 2011

Generic Monk posted:

Good point

https://github.com/poisonwomb/flaskblog

I think that's provided by flask_login

Using pycharm I dropped a breakpoint on the line where it is meant to query the user from the database but it never actually pauses the execution, so it looks like that isn't even running. So I would assume it's a problem with the form validators because of the validate_on_submit conditional, but those seem to be working ok. Am I missing a trick?

# ? Sep 3, 2023 18:57

spiritual bypass: Feb 19, 2008; Grimey Drawer

Keep dropping in breakpoints until you find where the execution path goes. Might have to dive into framework/library code to understand it.

# ? Sep 3, 2023 20:01

teen phone cutie: Jun 18, 2012; last year i rewrote something awful from scratch because i hate myself

this is probably an extremely dumb question but what's the preferred approach to install a new package with pip-tools?

I have my requirements.in where I have my pinned packages, and my requirements.txt that's autogenerated, but I'm not sure how to install a new package with the command line and get it in the requirements.in

Do i just have to manually add that line?

teen phone cutie fucked around with this message at 20:16 on Sep 3, 2023

# ? Sep 3, 2023 20:13

QuarkJets: Sep 8, 2008

teen phone cutie posted:

this is probably an extremely dumb question but what's the preferred approach to install a new package with pip-tools?

I have my requirements.in where I have my pinned packages, and my requirements.txt that's autogenerated, but I'm not sure how to install a new package with the command line and get it in the requirements.in

Do i just have to manually add that line?

According to the documentation you would manually add that new package to requirements.in

# ? Sep 3, 2023 23:51

rowkey bilbao: Jul 24, 2023

Anyone familiar with Pydantic v2 ?

You used to be able to do this:

Python code:

import uuid
from typing import Any, Dict, Union

from pydantic import UUID4,UUID5, BaseModel, Field
from pydantic import __version__ as pdversion
from pydantic import validator


def uuid_factory():
    return str(uuid.uuid4())


class Pet(BaseModel):
    name: str
    owners_id:Union[UUID4, UUID5]


class PetOwner(BaseModel):
    person_id: Union[UUID4, UUID5] = Field(default_factory=uuid_factory)
    name: str
    animals: Dict[str, Pet]

    @validator("animals", pre=True)
    def enforce_dog_properties(cls, v, values, **kwargs):
        print("enforce_dog_properties()")
        for name, animal in v.items():
            if "owners_id" not in animal:
                animal["owners_id"] = values['person_id'] # we set some value on the pet based on an attribute on the owner
        return v

print(f"using pydantic v{pdversion}")
a = PetOwner(name="henry", animals={"animal_1": {"name": "sultan"}}) # we don't need to set the Pet.owners_id just now
print(a.json())
"""
{"person_id": "e3a81267-0d13-4f76-8183-1e9443e843cc", "name": "henry", "animals": {"animal_1": {"name": "sultan", "owners_id": "e3a81267-0d13-4f76-8183-1e9443e843cc"}}}
"""

Now validator is deprecated and the recommended replacement is field_validator. Replacing the validator method with

Python code:


    @field_validator('animals', mode="before")
    def enforce_dog_properties(cls, v):
       for critter_id, critter_specs in v.items():
          if "owners_id" not in critter_specs:
              critter_specs["owners_id"] = cls.person_id
       return v

doesn't work, because the person_id value isn't set.
How do can I create instances like in my first examples, while being able to set values in the Pet instance based on the PetOwner.person_id value ?

I've been trying field_validator, but I run into similar issues.

# ? Sep 4, 2023 17:41

Armitag3: Mar 15, 2020; Forget it Jake, it's cybertown.

rowkey bilbao posted:

Anyone familiar with Pydantic v2 ?

You used to be able to do this:

Python code:

import uuid
from typing import Any, Dict, Union

from pydantic import UUID4,UUID5, BaseModel, Field
from pydantic import __version__ as pdversion
from pydantic import validator


def uuid_factory():
    return str(uuid.uuid4())


class Pet(BaseModel):
    name: str
    owners_id:Union[UUID4, UUID5]


class PetOwner(BaseModel):
    person_id: Union[UUID4, UUID5] = Field(default_factory=uuid_factory)
    name: str
    animals: Dict[str, Pet]

    @validator("animals", pre=True)
    def enforce_dog_properties(cls, v, values, **kwargs):
        print("enforce_dog_properties()")
        for name, animal in v.items():
            if "owners_id" not in animal:
                animal["owners_id"] = values['person_id'] # we set some value on the pet based on an attribute on the owner
        return v

print(f"using pydantic v{pdversion}")
a = PetOwner(name="henry", animals={"animal_1": {"name": "sultan"}}) # we don't need to set the Pet.owners_id just now
print(a.json())
"""
{"person_id": "e3a81267-0d13-4f76-8183-1e9443e843cc", "name": "henry", "animals": {"animal_1": {"name": "sultan", "owners_id": "e3a81267-0d13-4f76-8183-1e9443e843cc"}}}
"""

Now validator is deprecated and the recommended replacement is field_validator. Replacing the validator method with

Python code:


    @field_validator('animals', mode="before")
    def enforce_dog_properties(cls, v):
       for critter_id, critter_specs in v.items():
          if "owners_id" not in critter_specs:
              critter_specs["owners_id"] = cls.person_id
       return v

Try adding the @classmethod decorator to your validator function, then you should have access to cls as you're expecting.

Python code:


    @field_validator('animals', mode="before")
    @classmethod
    def enforce_dog_properties(cls, v):
       for critter_id, critter_specs in v.items():
          if "owners_id" not in critter_specs:
              critter_specs["owners_id"] = cls.person_id
       return v

# ? Sep 4, 2023 18:34

rowkey bilbao: Jul 24, 2023

That doesn't seem to do much, at least on my install.
It still goes

code:

AttributeError: person_id

I suppose the validator runs before

Python code:

person_id: Union[UUID4, UUID5] = Field(default_factory=uuid_factory)

sets PetOwner.person_id to a new uuid.

rowkey bilbao fucked around with this message at 22:05 on Sep 4, 2023

# ? Sep 4, 2023 22:02

QuarkJets: Sep 8, 2008

Have you tried setting the mode to "after"?

I would have guessed that you want a normal method, not a class method, since you're validating the values attached to an instance of the class. But you want the factory function to run and assign a default value to the missing attribute before this validator runs, so I'd think you'd want this validator to be in "after" mode

QuarkJets fucked around with this message at 22:15 on Sep 4, 2023

# ? Sep 4, 2023 22:11

nullfunction: Jan 24, 2005; Nap Ghost

Pretty sure the v2 equivalent you're looking for is FieldValidationInfo, which is passed into each validated field like the values dict was in 1.x, it just has a slightly different shape.

Python code:


from pydantic import FieldValidationInfo
...

class PetOwner(BaseModel):
    person_id: Union[UUID4, UUID5] = Field(default_factory=uuid_factory)
    name: str
    animals: Dict[str, Pet]

    @field_validator("animals", mode="before")
    @classmethod
    def enforce_dog_properties(cls, v, info: FieldValidationInfo):
       print(info)
       # FieldValidationInfo(config={'title': 'PetOwner'}, context=None, data={'person_id': UUID('c60d44c7-ee94-4995-9a84-934102e63a77'), 'name': 'henry'}, field_name='animals')
       for _, animal in v.items():
          if "owners_id" not in animal:
              animal["owners_id"] = info.data["person_id"]
       return v

# ? Sep 5, 2023 05:30

QuarkJets: Sep 8, 2008

Python code:

for _, animal in v.items():

Just use for animal in v.values():

# ? Sep 5, 2023 05:50

nullfunction: Jan 24, 2005; Nap Ghost

Yeah, fair. I was mucking about with the rest of the code to play with v2 a bit more and didn't include half of the things that would make this better.

So far v2 seems good, though I need to spend some time with their automated upgrade tool.

# ? Sep 5, 2023 05:55

Data Graham: Dec 28, 2009; 📈📊🍪😋

QuarkJets posted:

Python code:
for _, animal in v.items():
Just use for animal in v.values():

It's like wasted spark ignition

# ? Sep 5, 2023 10:46

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

QuarkJets posted:

Python code:
for _, animal in v.items():
Just use for animal in v.values():

Similarly I've had to point out to people you don't need to do index-based list iteration a bunch of times:

Python code:

For x in range(len(list_to_review)):
    do_thing(list_to_review[x])

It's pretty minor but god I'm glad we can just do for x in y.

# ? Sep 5, 2023 17:11

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

Cursed comedy option: use for i, _ in enumerate(list), then access elements via the index :getin:

# ? Sep 5, 2023 18:16

rowkey bilbao: Jul 24, 2023

QuarkJets posted:

Have you tried setting the mode to "after"?

I did, iirc it expects all fields (ex Pet.owners_id) to be set when creating a PetOwner instance.

nullfunction posted:

Pretty sure the v2 equivalent you're looking for is FieldValidationInfo, which is passed into each validated field like the values dict was in 1.x, it just has a slightly different shape.

This was the one, thanks !

# ? Sep 5, 2023 21:53

Jigsaw: Aug 14, 2008

Zugzwang posted:

Cursed comedy option: use for i, _ in enumerate(list), then access elements via the index

for x in list(<generator>) or things like {max, all, any}([<generator expression>]) drive me insane. You were so close! Why did it have to be a list?!

# ? Sep 6, 2023 14:55

Olly the Otter: Jul 22, 2007

During code review I noticed a routine that modifies a dict while iterating over it. I've always understood this to be a no-no. But this code seems to work in practice. It's not removing or adding keys, just re-assigning the value for some keys. Here's a simplified example to show what I mean:

code:

Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> mydata = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k, d in mydata.items():
...     if d % 2 == 0:
...         mydata[k] = d * 100
... 
>>> mydata
{'one': 1, 'two': 200, 'three': 3, 'four': 400}

Is this type of thing guaranteed to work under Python standards?

# ? Sep 6, 2023 18:46

Foxfire_: Nov 8, 2010

Olly the Otter posted:

Is this type of thing guaranteed to work under Python standards?

There aren't any Python standards besides "what does CPython do?" combined with vibes/occasional comments about what is only an implementation detail.

Mutating an item linked to a key ought to be safe.

Replacing the object a key points to with a different one like that code does is fuzzier. CPython probably doesn't actually rebalance the backing tree so previous iterators are staying valid. But it also doesn't seem that unreasonable for some hypothetical future version to consider replacing a value as a time that it could rebalance

# ? Sep 6, 2023 20:31

Slimchandi: May 13, 2005; That finger on your temple is the barrel of my raygun

Foxfire_ posted:

There aren't any Python standards besides "what does CPython do?" combined with vibes/occasional comments about what is only an implementation detail.

I still can't bring myself not to use OrderedDict.

But yeah, why not make that a dict comprehension? Just never modify the thing you are iterating over.

# ? Sep 6, 2023 20:48

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

The big problem with mutating collections is that you can't add or remove things as you go, mutating them is perfectly fine. Agree that a dict comprehension is cleaner though.

# ? Sep 6, 2023 20:52

Foxfire_: Nov 8, 2010

The gray area is that this code isn't mutating things, it's replacing the value associated witha key with a completely new unrelated one (python integers are immutable like strings are)

A well optimized dictionary implementation probably doesn't mess with any structure for that, but you could also implement it with "delete the old (key, value)" operation followed by "insert the new (key, value)", which could plausibly restructure a backing tree/hash table

# ? Sep 6, 2023 22:08

Pie Colony: Dec 8, 2006; I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

Foxfire_ posted:

There aren't any Python standards besides "what does CPython do?" combined with vibes/occasional comments about what is only an implementation detail.

Mutating an item linked to a key ought to be safe.

Replacing the object a key points to with a different one like that code does is fuzzier. CPython probably doesn't actually rebalance the backing tree so previous iterators are staying valid. But it also doesn't seem that unreasonable for some hypothetical future version to consider replacing a value as a time that it could rebalance

Why would you rebalance a tree when modifying the value for some key? Neither the size of the tree nor the bucket of the value changes.

To answer the original question, I agree there probably isn't anything specifying this behavior, but modifying or replacing the value of some existing key should be fine, adding or removing keys would break iteration.

Pie Colony fucked around with this message at 00:22 on Sep 7, 2023

# ? Sep 7, 2023 00:18

Olly the Otter: Jul 22, 2007

Foxfire_ posted:

The gray area is that this code isn't mutating things, it's replacing the value

Yeah, this gray area is why I posted -- if it were just mutating an object then I wouldn't be concerned. (The original code does stuff that's more obviously replacing the value, such as mydata[k] = UUID(d), but the principle is the same.)

If it were my own code then I'd avoid stuff like this altogether, but it's trickier when it comes to asking someone else to change their code. I suppose I was hoping there were clear standards somewhere that I could point to.

# ? Sep 7, 2023 00:51

QuarkJets: Sep 8, 2008

Olly the Otter posted:

During code review I noticed a routine that modifies a dict while iterating over it. I've always understood this to be a no-no. But this code seems to work in practice. It's not removing or adding keys, just re-assigning the value for some keys. Here's a simplified example to show what I mean:
code:
Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> mydata = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k, d in mydata.items():
...     if d % 2 == 0:
...         mydata[k] = d * 100
... 
>>> mydata
{'one': 1, 'two': 200, 'three': 3, 'four': 400}
Is this type of thing guaranteed to work under Python standards?

Views are dynamic, so changes to the values in a dictionary are permitted even while you're iterating over them. If keys were being added or removed then I'd expect issues
https://docs.python.org/3/library/stdtypes.html#dictionary-view-objects
Per the docs, this is a-okay

This is okay too:

Python code:

>>> x = {0: 1, 1:2, 2:3, 3:4, 4:5, 5:6}
>>> for key, value in x.items():
...     if key < 5:
...         x[key+1] += 10
...
>>> x
{0: 1, 1: 12, 2: 13, 3: 14, 4: 15, 5: 16}
>>>

No problem here, changes to the values are allowed even before we've iterated over them. But what about key and value, are those up to date too?

Python code:

>>> x = {0: 1, 1:2, 2:3, 3:4, 4:5, 5:6}
>>> for key, value in x.items():
...     print(key, value)
...     if key < 5:
...         x[key+1] += 10
...
0 1
1 12
2 13
3 14
4 15
5 16
>>> x
{0: 1, 1: 12, 2: 13, 3: 14, 4: 15, 5: 16}

Yup!

QuarkJets fucked around with this message at 07:26 on Sep 7, 2023

# ? Sep 7, 2023 07:14

Foxfire_: Nov 8, 2010

Pie Colony posted:

Why would you rebalance a tree when modifying the value for some key? Neither the size of the tree nor the bucket of the value changes.

If you specify that changing the value for an existing key will never invalidate iterators, it's requiring that Change be a new primitive operation because it has different behavior from a "Delete (key, value1)" => "Create (key, value2)" sequence. Delete and Create may invalidate iterators, so you can't just compose them to create Change.

Essentially:

Python code:

for k, v in mapping:
    mapping[k] = None

and

Python code:

for k, v in mapping:
    del mapping[k]
    mapping[k] = None

have different observable behavior in CPython

Requiring Change in the interface isn't unreasonable, but it doesn't feel so fundamental to the concept of a mapping type that I'd confidently infer a "The Python language requires that all Dict and Dict-compatible objects support a Change operation" rule instead of it just being an implementation detail that CPython dict's happen to do this

Nothing in the documentation says anything definitively either way, there's nothing talking about changes at all. It just says:

quote:

d[key] = value
Set d[key] to value

and

quote:

Iterating views while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries.

# ? Sep 8, 2023 04:44

QuarkJets: Sep 8, 2008

The definition of the dict type also says:

quote:

Note that updating a key does not affect the order. Keys added after deletion are inserted at the end.

and

quote:

Keys and values are iterated over in insertion order.

This is explicit and defines the behavior that we've each shown

# ? Sep 8, 2023 07:41

Foxfire_: Nov 8, 2010

Yeah, I'd take that as sufficient to say the language is requiring update/change as a distinct thing. Looks like it got promoted from being an implementation detail to language requirement at the same time as requiring dict's to be ordered did. The text for it is just off with the stuff about order in the main dict documentation instead of the part about dictviews.

# ? Sep 8, 2023 09:24

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

I've got a bit of a design question. I'm working on a project that involves a lot of infrastructure tech clients and has to do with various forms of infrastructure validation across a variety of infra types. To keep the backstory short: the APIs that these services expose have a fairly poor Python integration; for this reason and wanting to glue stuff together we have a package that does a lot of this. For example, you might have an ClusterPlus object, which recreates the hierarchical data of the Cluster object from the API, as well as loading things like ClusterResources/etc.

These classes are pretty heavy, but they make it a lot easier to iterate quickly because a lot of the weird interplay is handled for you. However, now we have a problem: We want to be able to freeze these objects so we can use them in a situation where you cache first and then call later, and currently they have a lot of lazy loading functions etc involved that check if the object is loaded and grab it if it isn't.

There's a few options I'd like to suggest moving forward for refactoring these, and I'm not sure which one would be better or if it's even workable. Note that these classes are all in active use by other major projects so ideally we'd like these refactors to be as drop-in as possible.

One option would be to have all these objects take in an optional argument that disables all API calls; this requires that it's implemented properly but is (at least from a high level perspective) fairly straightforward.

Another option would be to separate the data structure from the functions - For example, having a FrozenClusterPlus class that is either inherited from (probably bad) or passed into the ClusterPlus object, so something like this:

Python code:

class ClusterPlusData:
    cluster_name: str
    cluster_children: List[Cluster]

class ClusterPlus:
    cluster_data = ClusterPlusData()

    @property
    def cluster_name(self):
        return self.cluster_data.cluster_name

    @property
    def cluster_children(self):
        if not self.cluster_data.cluster_children:
            self.cluster_data.cluster_children = self.cluster_child_client.get_children()
        return self.cluster_data.cluster_children

Or invert it where you work with the base ClusterPlus object and have a ClusterPlusFuncs class that adds on things, although I think the above approach is cleaner.

Any suggestions on how to approach this?

# ? Sep 11, 2023 20:36

QuarkJets: Sep 8, 2008

Have you looked at frozen dataclasses?

Python code:

from dataclasses import dataclass

@dataclass(frozen=True)
class ClusterPlus:
    cluster_name: str
    cluster_children: tuple[Cluster]

You can use `__post_init__(self)` to perform assignment after initialization, but after that any attempts to modify the attributes will result in an error. Rather than __post_init__ I think it's cleaner to have a factory function or class method to do that for you.

To store off intermediate state you could have a `to_json` method, and then you could have a class method `from_json` that also handles initialization

Without knowing what your data model looks like it hard to know how drop-in this would be. Frozen dataclasses are as close to "immutable" as things tend to get in python, and you get all of the other advantages of dataclasses (a nice __repr__, trivial conversion to dictionary, etc).

# ? Sep 12, 2023 05:42

Gothmog1065: May 14, 2009

Hey all, getting back into some basic Python scripting and my smooth brain finally hit a wall. The basic is that the script is parsing a file, putting it all in a dictionary, then pulling specific elements of that out and creating a list of lists. However, I want the functionality to sort that list. That all came out well and sorting was working. Then I wanted to be able to add the sort ability to the command line with an argument, so in comes argparser. The issue I'm probably having, however is scoping. Here's what I've got so far:

Python code:

import re,argparse

def <functions that work>():
    stuff

def data_display(dataDict): 
    list = []
    for key in dataDict.keys():
        <all of this works fine>
        thread = "column1"
        thDir = "column 2" 
        typ = "column 3" 
        data = "column 4" 

        thdLine = ((thread.ljust(25, ' '), (thDir.ljust(2, ' '), (typ.ljust(4, ' '), (data))
        thdList.append(thdLine)

    thdList=(sorted(thdList, key = lambda x: x[args.sort])) #args.sort can be replaced with a numeric value or global basic 'sort' and works fine. 

def main(): 
    parser = argparse.ArgumentParser(description =  "NetConfig Display")
    parser.add_argument("-s", "--sort", help="Sort by column", default=1, required=False)
    parser.parse_args()
    args = parser.parse_args()

    data_display(dataDict)

data_dict = {}
sort = 0

main()

Doing it this way throws a name error: "NameError: name 'args' is not defined.

If I attempt to add a variable of 'args' in the global area, I get TypeErrors, no matter what I attempt to set it to (string, list, tuple, etc). If I global another variable (sort) and put sort = args.sort inside the main function, it doesn't update the global variable (without having to put 'global sort' in there which is silly). So I'm probably doing something really dumb right now and completely missing an easy solution.

# ? Sep 13, 2023 21:36

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

This is maybe more of a DB question than a Python question, but I'm using SQL Alchemy so I'll ask it here...

I've got a FastAPI project that's storing some stuff in a DB (SQLite locally for development, MySQL for production). I have a few columns that are strings, but are actually lists of data. So to read them I read the string and split on the comma to get my list of strings. I know I could setup a relationship and store these items in the list in another table, to do it in a more "native" way, but that adds complexity to the DB, and there's never a reason why I would want only one of these values in the list: I always want only a single one.

These entries are rather "static", they don't get changed by user input, and in fact once created they rarely if ever change, only get read. I'm thinking I could do something like this but I'm not sure if there's any better options these days.

# ? Sep 13, 2023 21:39

necrotic: Aug 2, 2005; I owe my brother big time for this!

Gothmog1065 posted:

Hey all, getting back into some basic Python scripting and my smooth brain finally hit a wall. The basic is that the script is parsing a file, putting it all in a dictionary, then pulling specific elements of that out and creating a list of lists. However, I want the functionality to sort that list. That all came out well and sorting was working. Then I wanted to be able to add the sort ability to the command line with an argument, so in comes argparser. The issue I'm probably having, however is scoping. Here's what I've got so far:
Python code:
import re,argparse

def <functions that work>():
    stuff

def data_display(dataDict): 
    list = []
    for key in dataDict.keys():
        <all of this works fine>
        thread = "column1"
        thDir = "column 2" 
        typ = "column 3" 
        data = "column 4" 

        thdLine = ((thread.ljust(25, ' '), (thDir.ljust(2, ' '), (typ.ljust(4, ' '), (data))
        thdList.append(thdLine)

    thdList=(sorted(thdList, key = lambda x: x[args.sort])) #args.sort can be replaced with a numeric value or global basic 'sort' and works fine. 

def main(): 
    parser = argparse.ArgumentParser(description =  "NetConfig Display")
    parser.add_argument("-s", "--sort", help="Sort by column", default=1, required=False)
    parser.parse_args()
    args = parser.parse_args()

    data_display(dataDict)

data_dict = {}
sort = 0

main()
Doing it this way throws a name error: "NameError: name 'args' is not defined.

If I attempt to add a variable of 'args' in the global area, I get TypeErrors, no matter what I attempt to set it to (string, list, tuple, etc). If I global another variable (sort) and put sort = args.sort inside the main function, it doesn't update the global variable (without having to put 'global sort' in there which is silly). So I'm probably doing something really dumb right now and completely missing an easy solution.

args is a local to the main function. Pass it in as an argument to data_display.

# ? Sep 13, 2023 22:16

12 rats tied together: Sep 7, 2006

you could also define data_display inside main's scope. which is not usually recommended but is sometimes a useful pattern

# ? Sep 13, 2023 22:19

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

QuarkJets posted:

Have you looked at frozen dataclasses?
Python code:
from dataclasses import dataclass

@dataclass(frozen=True)
class ClusterPlus:
    cluster_name: str
    cluster_children: tuple[Cluster]
You can use `__post_init__(self)` to perform assignment after initialization, but after that any attempts to modify the attributes will result in an error. Rather than __post_init__ I think it's cleaner to have a factory function or class method to do that for you.

To store off intermediate state you could have a `to_json` method, and then you could have a class method `from_json` that also handles initialization

Without knowing what your data model looks like it hard to know how drop-in this would be. Frozen dataclasses are as close to "immutable" as things tend to get in python, and you get all of the other advantages of dataclasses (a nice __repr__, trivial conversion to dictionary, etc).

Familiar with Frozen dataclasses, but this solves a problem I'm not actually trying to fix. The issue isn't 'you can reassign/update attributes after creation', the problem is 'certain attributes make calls out to clients to populate data, and we need to ensure that doesn't happen.'

The idea is that all the dependencies/etc of a Cluster might be very complex and big, and span multiple other systems. This framework was built as an accelerator to other projects so it basically works a glue between multiple infrastructure technologies - when you ask a cluster about network details, instead of just giving you a pointer, it actually calls and loads the object from that other client, then it's available to you cached locally from there on out. (See the example of the cluster_children note above)

The problem is that I basically want a way of saying 'hey just return a version of yourself that just contains the data and won't try and fetch anything else - we either initialized it correctly, or datasets will be empty.

One approach would be to construct complex dataclasses and have each of these objects basically turn themselves into those objects, but that's a huge amount of basically copy/paste work there, so the idea is how to build it into the class itself in the least weird way possible.

There's some really long ways of going about it - basically writing two almost identical classes that both fulfill the same interface - so you have a ClusterPlus object and then a ClusterPlusButThisOneDoesntMakeCalls object - the second of which is basically just a dataclass representation of what the first one looks like after you fetched all the data you need. But that's a ton of duplicate code, not to mention the entire problem of drift.

# ? Sep 14, 2023 03:17

QuarkJets: Sep 8, 2008

You should copy-paste whatever Python spits out when the issue occurs because that's very helpful, Gothmog1065. I can see that dataDict isn't defined when main() runs, that would be one problem

# ? Sep 14, 2023 03:51

QuarkJets: Sep 8, 2008

Falcon2001 posted:

Familiar with Frozen dataclasses, but this solves a problem I'm not actually trying to fix. The issue isn't 'you can reassign/update attributes after creation', the problem is 'certain attributes make calls out to clients to populate data, and we need to ensure that doesn't happen.'

The idea is that all the dependencies/etc of a Cluster might be very complex and big, and span multiple other systems. This framework was built as an accelerator to other projects so it basically works a glue between multiple infrastructure technologies - when you ask a cluster about network details, instead of just giving you a pointer, it actually calls and loads the object from that other client, then it's available to you cached locally from there on out. (See the example of the cluster_children note above)

The problem is that I basically want a way of saying 'hey just return a version of yourself that just contains the data and won't try and fetch anything else - we either initialized it correctly, or datasets will be empty.

One approach would be to construct complex dataclasses and have each of these objects basically turn themselves into those objects, but that's a huge amount of basically copy/paste work there, so the idea is how to build it into the class itself in the least weird way possible.

There's some really long ways of going about it - basically writing two almost identical classes that both fulfill the same interface - so you have a ClusterPlus object and then a ClusterPlusButThisOneDoesntMakeCalls object - the second of which is basically just a dataclass representation of what the first one looks like after you fetched all the data you need. But that's a ton of duplicate code, not to mention the entire problem of drift.

Have you looked at the cached property decorator?

# ? Sep 14, 2023 03:55

WHERE MY HAT IS AT: Jan 7, 2011

Falcon2001 posted:

Familiar with Frozen dataclasses, but this solves a problem I'm not actually trying to fix. The issue isn't 'you can reassign/update attributes after creation', the problem is 'certain attributes make calls out to clients to populate data, and we need to ensure that doesn't happen.'

The idea is that all the dependencies/etc of a Cluster might be very complex and big, and span multiple other systems. This framework was built as an accelerator to other projects so it basically works a glue between multiple infrastructure technologies - when you ask a cluster about network details, instead of just giving you a pointer, it actually calls and loads the object from that other client, then it's available to you cached locally from there on out. (See the example of the cluster_children note above)

The problem is that I basically want a way of saying 'hey just return a version of yourself that just contains the data and won't try and fetch anything else - we either initialized it correctly, or datasets will be empty.

One approach would be to construct complex dataclasses and have each of these objects basically turn themselves into those objects, but that's a huge amount of basically copy/paste work there, so the idea is how to build it into the class itself in the least weird way possible.

There's some really long ways of going about it - basically writing two almost identical classes that both fulfill the same interface - so you have a ClusterPlus object and then a ClusterPlusButThisOneDoesntMakeCalls object - the second of which is basically just a dataclass representation of what the first one looks like after you fetched all the data you need. But that's a ton of duplicate code, not to mention the entire problem of drift.

Assuming there's a naming convention for the underlying properties, you could make a class that just dynamically returns those when fetching a property. Something like:

Python code:

class MyClientClass():
    def __init__(self):
        self._fooprop = None

    @property
    def foo(self):
        if self._fooprop:
            return self._fooprop
        else:
            self._fooprop = "EXPENSIVE OPERATION"
            return self._fooprop

class DisabledProxy():
    def __init__(self, proxied):
        self._proxied = proxied

    def __getattr__(self, name):
        return getattr(self._proxied, f"_{name}prop")
                       
foo = MyClientClass()
proxy = DisabledProxy(foo)

print(proxy.foo) # prints None

This gets tricky if you have nested props/sub objects you also need to disable but makes things easy in the simple case as long as people can follow the naming convention or you find some other way to determine it like a dict or something.

# ? Sep 14, 2023 04:22

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 22:36

duck monster: Dec 15, 2004

Zugzwang posted:

Yeah I really wish Python had something like "type hinting is purely optional, but they will be enforced if you specify them." As opposed to "the Python interpreter does not give the slightest gently caress about type hints."

Julia has the former as part of its multiple dispatch model, but sadly, adoption of Julia continues to be pretty underwhelming.

loving VB6 had a solution for this.

"Option Explicit On"
"Option Strict On"

It would be a highly good thing to have the option to just stick something like that at the top of a python file, so that it explodes at import if it fails a sanity linting.

# ? Sep 14, 2023 05:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »