Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
lazerwolf
Dec 22, 2009

Orange and Black

D34THROW posted:

I cannot for the life of me remember the name of the tool I used which walked your code and created a requirements.txt based on what was actually used. Like...someone running my app doesn't need black or flake8 and poo poo like that.

Correct that’s why you can specify those requirements in a requirements_dev.txt file for those who would contribute to your project but not for running in production.

I haven’t gotten into poetry yet seems like a lot of overhead that hasn’t been widely adopted yet.

What can it do better that I can’t achieve with pyenv and pinned reqs files?

Adbot
ADBOT LOVES YOU

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:

Foxfire_ posted:

I'm having trouble thinking of something that's actually useful for. Display code would want some application-specific approximation instead of the exact values (humans usually don't want to be told that "0.1" is 3602879701896397 / 36028797018963968 instead of 1/10). And if you were doing arbitrary precision math, you shouldn't have floats to begin with

Fractional inches.

I just had something happened that has never happened to me before. I'm following the first-fit descending algorithm here and adapting it to fit my needs as follows. The first function calls the second, and it caused an infinite loop or memory leak. I'm trying to determine exactly where.

Python code:
def cutlist():
    item = "HSM 3 X 2 X 24"
    cutlist = [192, 192, 96, 96]
    print(item, calculate_required(item, cutlist))

def calculate_required(item: str, cutlist: list) -> int:
    size = MATERIAL_LENGTHS[item] if item in MATERIAL_LENGTHS.keys() else 288
    remain = [size]
    solution = [[]]
    for piece in sorted(cutlist, reverse=True):
        for num, free in enumerate(remain):
            if free >= piece:
                remain[num] -= piece
                solution[num].append(piece)
                break
            else:
                solution.append([piece])
                remain.append(size - piece)

    return len(solution)
I mean, I'd really like to know what the gently caress I did because when I directly copy the FFD() function, change variable names for clarity, and call it from calculate_required then it works. Manually copied by typing, it does not work. :shepface:

D34THROW fucked around with this message at 20:47 on Apr 4, 2022

ExcessBLarg!
Sep 1, 2001

D34THROW posted:

The first function calls the second, and it caused an infinite loop or memory leak. I'm trying to determine exactly where.
You're appending to remain while iterating through it. It's generally discouraged to modify the collection you're iterating over.

Edit: Looked at the original FFD function. The problem with your code is that the "else:" block is on the inner-most "if". It's supposed to be on the inner-most "for" (the infrequently-used for/else clause).

ExcessBLarg! fucked around with this message at 21:17 on Apr 4, 2022

Foxfire_
Nov 8, 2010

D34THROW posted:

Fractional inches.
I guess those happen to work out since all the values you care about happen to be exactly representable fractions of two.

For the cut list, a straightforward debugging method is to print out stuff and see where what actually happens splits from what you think happens:
code:
def calculate_required(item: str, cutlist: list) -> int:
     size = MATERIAL_LENGTHS[item] if item in MATERIAL_LENGTHS.keys() else 288
     remain = [size]
     solution = [[]]
     for piece in sorted(cutlist, reverse=True):
         print(f"Fitting piece {piece} into free space {remain}")
         for num, free in enumerate(remain):
             print(f"Considering free chunk of length {free}")
             if free >= piece:
                 print(f"It fits, leftover freechunk is length {remain[num]-piece}")
                 remain[num] -= piece
                 solution[num].append(piece)
                 break
             else:
                 print("It doesn't fit in this chunk")
                 print("(Why is it moving into the solution and the uncut lengths changing)")
                 solution.append([piece])
                 remain.append(size-piece)
     return len(solution)
Python list iterators internally happen to be implemented by tracking indices. The infinite loop happens when it considers a length, appends it to remain, then the next loop considers that length again.
The else path on that doesn't make sense to me glancing through, the piece doesn't fit into the freespace, but it's messing with the solution and remaining freespace

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4

The March Hare posted:

Something like Poetry will keep your requirements in separate files - one will contain packages that are top-level required, and then a lockfile with those packages deps.

Also, pipenv or venv or... god there's a bunch of them, huh

punk rebel ecks posted:

So I finally finished my pokedex project.

The file is on my Github.

Would anyone like to check it out and give me their thoughts?

I was wondering if I was at the point where I can start applying for (likely entry level) software developer/engineer/programming jobs?

Definitely learn about virtual environments; generally Python projects each use their own virtual environment to keep projects that need two different incompatible versions of a package kind of isolated from each other. It also helps with making the requirements.txt file like as mentioned before - you don't have to manually pare out anything the project isn't using.

QuarkJets
Sep 8, 2008

Data Graham posted:

And while this will do for a quick-n-dirty first pass, ideally you would want to go back and weed out all the things that got installed that aren't strictly "requirements" for your project and that you don't care about pinning.

Right? My instinct is to pin as little as possible, but is that best practice? I'm second-guessing myself now and thinking that might lead to more trouble than overspecifying.

I think of environment handling from two perspectives: 1) the minimum packages required to install and use a package vs 2) a frozen, tested environment. The first should be handled via a setup.cfg or pyproject.toml that specifies as few packages as possible using the loosest version requirements possible. The second should be a conda.yaml or requirements.txt dumped from `conda list` or `pip freeze`.

D34THROW posted:

I cannot for the life of me remember the name of the tool I used which walked your code and created a requirements.txt based on what was actually used. Like...someone running my app doesn't need black or flake8 and poo poo like that.

pipreqs does this, it's pretty cool

QuarkJets
Sep 8, 2008

punk rebel ecks posted:

So I finally finished my pokedex project.

The file is on my Github.

Would anyone like to check it out and give me their thoughts?

I was wondering if I was at the point where I can start applying for (likely entry level) software developer/engineer/programming jobs?

I didn't review your code because it's in a zip file, sorry :shrug:

punk rebel ecks
Dec 11, 2010

A shitty post? This calls for a dance of deduction.

QuarkJets posted:

I didn't review your code because it's in a zip file, sorry :shrug:

I’ll fix it into the correct format and ask again in two weeks.

I’m on a trip right now.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I appreciate the help y'all. It generates the cutlists now and the next step is to make it calculate the material required. From there, I have to go back to any classes that involve cut material and modify them to do, say, self.add_to_cutlist("PT 1 X 4 X 8", 37, 2) instead of self.add_to_dict("materials", f'PT 1 X 4 X 8 @ {length}"', 2. Then

Sometimes I hate the end of the day because I have something in my head and then forget it 16 hours later when I start up again. :v:

Slimchandi
May 13, 2005
That finger on your temple is the barrel of my raygun

D34THROW posted:


Python code:

def calculate_required(item: str, cutlist: list) -> int:
    size = MATERIAL_LENGTHS[item] if item in MATERIAL_LENGTHS.keys() else 288


May I introduce you to dict.get() to simplify this?

https://python-reference.readthedocs.io/en/latest/docs/dict/get.html

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:

:stare: I use that comprehension so many times and you've just made my life easier, holy poo poo!

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Is there something like a "semiautomatic automation" framework in Python? I'd like to script something that has to break off at points for manual intervention, resume, and recover from goofiness that can come from that. I figured I could scribble together my own thing with that but the Python community has all kinds of goofy things.

I was figuring something like this would be like a re-entrant state machine and I'd be scripting the transitions and what happens with them. The state would be saved off when it exited and reloaded to continue at a spot.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I'm at the end of my loving rope here.

I'm refactoring my cli.py file into manage.py and I've added the #!usr/bin/env python shebang to the top line of the file, but in order to actually do anything with it at the terminal, I have to type ./manage.py.

What I want to be able to do is manage <args> like, say, black or flask or flake8. How are those able to do it? I'm looking at the source on GitHub but I can't figure it out.

Zephirus
May 18, 2004

BRRRR......CHK

D34THROW posted:

I'm at the end of my loving rope here.

I'm refactoring my cli.py file into manage.py and I've added the #!usr/bin/env python shebang to the top line of the file, but in order to actually do anything with it at the terminal, I have to type ./manage.py.

What I want to be able to do is manage <args> like, say, black or flask or flake8. How are those able to do it? I'm looking at the source on GitHub but I can't figure it out.

You're looking for entry points in setuptools

https://setuptools.pypa.io/en/latest/userguide/entry_point.html

This is where flake8 configures it https://github.com/PyCQA/flake8/blob/3ce76158a0e18605e9701bd3a2a25716fea411de/setup.cfg#L51

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Rocko Bonaparte posted:

Is there something like a "semiautomatic automation" framework in Python? I'd like to script something that has to break off at points for manual intervention, resume, and recover from goofiness that can come from that. I figured I could scribble together my own thing with that but the Python community has all kinds of goofy things.

I was figuring something like this would be like a re-entrant state machine and I'd be scripting the transitions and what happens with them. The state would be saved off when it exited and reloaded to continue at a spot.

I’m kinda confused as to what someone would make that wouldn’t just be typing those rules into code?

I use AWS step functions to define a state machine that runs lambdas based on business rules. Important bits like max concurrency and error handling are reliable.

I do a similar thing with zapier for simple flows. All those are things that require internet connected code to run though.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:

Yeah that looks like it. Do I have to reconfigure my ./manage folder as a module and run setup,py before I can make this work?


EDIT: Holy gently caress, I did it. Thank you! Now I think I'm starting to understand packages a little more. But it doesn't ultimately work because it needs access to the app context of my flask app. Back to bash aliases :v:

D34THROW fucked around with this message at 18:08 on Apr 7, 2022

ExcessBLarg!
Sep 1, 2001

D34THROW posted:

What I want to be able to do is manage <args> like, say, black or flask or flake8.
You have to copy the file into one of the directories specified by $PATH.

Zephirus
May 18, 2004

BRRRR......CHK

D34THROW posted:

Yeah that looks like it. Do I have to reconfigure my ./manage folder as a module and run setup,py before I can make this work?

if you don't have it set up with setuptools already then I would follow this advice:

ExcessBLarg! posted:

You have to copy the file into one of the directories specified by $PATH.

Assuming you're on a unix-alike you would rename the manage.py file to manage and set it as executable

mv manage.py manage; chmod +x manage

You then have to put it in a folder in your PATH environment variable (or add a path to it to your environment).

The shebang tells the shell what to call to execute the file (it doesn't look at the file extension).

Note this doesn't work on windows outside of WSL.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I gave the gently caress up on it because a module can't access the overall app context for Flask. Instead, I'm doing away with manage.py, moving everything into the main .py file I use for Flask, and doing everything with the CLI through that.

Which brings me to my next :smuggo: moment looking at myself a year ago learning Python. Instead of doing 19 different parser.add_argument() calls, I'm looping over a dict of dicts that defines the arguments, the functions to call, and the help text. Doing this with a pair of subparsers for tests and db_func, with an additional run argument to actually run the server.

I'll post code if anyone's interested in telling me how bad it is. Or I'll shut up if I'm boring y'all. Maybe this all belongs in project.log :shrug:

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4
Question about Python style/best practice - for context, I teach Python to newbies at a bootcamp and I'm trying really hard to get them to do things correctly the first time around instead of them getting jobs and having all their coworkers bitch about how they can't code properly:

The following code works, because the attributes for an object are basically just a dictionary you can gently caress around with however you want:

code:
class SampleClass():

	def __init__():
		pass

x = SampleClass()

x.new_attribute = 3
From my understanding though, it's a bad idea to just assign attributes to instances of a class willy-nilly, because then there's no guarantee that all instances of a class have the same attributes. There are situations where this could be useful, but that doesn't make it a good idea - you could solve whatever problem you need to solve by keeping track of the data elsewhere. This is generally correct thinking for newbies to Python/programming, or even just in general, right? If any instance of a class has new_attribute, then they really all should by default. My logic is that it's similar to how Python will let you assign one type of data to a variable, and then replace it with a completely different type - you can do it, but there's definitely a high risk of shooting yourself in the foot when something in your code later is expecting it to be of one type when it's actually another.

Macichne Leainig
Jul 26, 2012

by VG
I'm not an expert enough to speak authoritatively on this in any manner, but I also imagine in that scenario that IDEs would have trouble picking up on the fact that SampleClass has a new_attribute so that might also induce some development difficulties with that specific class.

ploots
Mar 19, 2010

death cob for cutie posted:

I'm trying really hard to get them to do things correctly the first time around instead of them getting jobs and having all their coworkers bitch about how they can't code properly

I've got some bad news for you about everyone, everywhere

death cob for cutie
Dec 30, 2006

dwarves won't delve no more
too much splatting down on Zot:4
oh it's an impossible task because no two people can agree on what is right outside of a vacuum, and everyone thinks everyone else is an idiot, but I should at least try to smooth out the real bad parts

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I use flake8 and black so I don't have to think about that poo poo. For the most part, flake8 only catches dumb poo poo (like trailing whitespace) and black cleans up poo poo I can't be bothered with, like long comprehensions and statements.

QuarkJets
Sep 8, 2008

death cob for cutie posted:

Question about Python style/best practice - for context, I teach Python to newbies at a bootcamp and I'm trying really hard to get them to do things correctly the first time around instead of them getting jobs and having all their coworkers bitch about how they can't code properly:

The following code works, because the attributes for an object are basically just a dictionary you can gently caress around with however you want:

code:
class SampleClass():

	def __init__():
		pass

x = SampleClass()

x.new_attribute = 3
From my understanding though, it's a bad idea to just assign attributes to instances of a class willy-nilly, because then there's no guarantee that all instances of a class have the same attributes. There are situations where this could be useful, but that doesn't make it a good idea - you could solve whatever problem you need to solve by keeping track of the data elsewhere. This is generally correct thinking for newbies to Python/programming, or even just in general, right? If any instance of a class has new_attribute, then they really all should by default. My logic is that it's similar to how Python will let you assign one type of data to a variable, and then replace it with a completely different type - you can do it, but there's definitely a high risk of shooting yourself in the foot when something in your code later is expecting it to be of one type when it's actually another.

It's bad practice, yes. There's virtually no good reason to do this. Classes should be well defined, and defined once.

ExcessBLarg!
Sep 1, 2001

death cob for cutie posted:

From my understanding though, it's a bad idea to just assign attributes to instances of a class willy-nilly, because then there's no guarantee that all instances of a class have the same attributes.
Thought exercise: How is this really any different?
Python code:
class SampleClass():
	def __init__(self):
		pass

class DerivedSampleClass(SampleClass):
    def __init__(self, na):
        super().__init__()
        self.new_attribute = na

x = DerivedSampleClass(3)
assert isinstance(x, SampleClass)
assert vars(x).keys() != vars(SampleClass()).keys()

ExcessBLarg! fucked around with this message at 04:48 on Apr 13, 2022

QuarkJets
Sep 8, 2008

ExcessBLarg! posted:

Thought exercise: How is this really any different?
Python code:
class SampleClass():
	def __init__(self):
		pass

class DerivedSampleClass(SampleClass):
    def __init__(self, na):
        super().__init__()
        self.new_attribute = na

x = DerivedSampleClass(3)
assert isinstance(x, SampleClass)
assert vars(x).keys() == vars(SampleClass()).keys()

This implementation defines a new class and is self-documenting, the other does not and is not.

QuarkJets fucked around with this message at 04:53 on Apr 13, 2022

CarForumPoster
Jun 26, 2013

⚡POWER⚡

QuarkJets posted:

This implementation defines a new class and is self-documenting, the other does not and is not.

Well said.

ExcessBLarg!
Sep 1, 2001

QuarkJets posted:

This implementation defines a new class and is self-documenting, the other does not and is not.
True.

I'd argue the biggest issue here is that Python, by having all attributes be public and (easily) assignable outside of the defining context can lead to a situation where an inexperienced programmer might adopt non-idiomatic--but functionally OK--style.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!

CarForumPoster posted:

I’m kinda confused as to what someone would make that wouldn’t just be typing those rules into code?

As an example, the user at some point has to cherry pick a bunch of commits that cause merge conflicts and resolve them. That process can be non-trivial and span days. I'd like them to be able to resume there. There are a few things like that which can span a few days of interactive work.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I am having an absolute bitch of a time with a problem I am going to have a hard time explaining, but here goes.

I'm using Flask-Security, but all the examples use a single module to hold everything. What I am trying to do is have user_datastore globally available. To do that, I'm putting as a module-level variable in the app's __init__.py file, which also contains the app factory.

I can't set user_datastore at the module level until it's set in the app factory - because I need the db object that isn't created until , but setting a user_datastore variable in create_app doesn't change the module-level init.

I need a way to set the __init__.py level variable from within a function and I am out of my loving mind on how to do it.

EDIT: Here's the dumb hacky poo poo I'm trying now.
Python code:
# Set up everything needed for `create_app`
db = SQLAlchemy()migrate = Migrate()
...
user_datastore = None


def create_app(config_class=Config):
    """
    Flask app factory function.

    Parameters
    ----------
    config_class : Config
        The configuration object, passed to `app.config` to set things
        up from `config.py`.
    """
    # Set up the app.
    app = Flask(__name__)
    ...
    # Set up the security objects.
    from app_data.models import User, Role

    globals()["user_datastore"] = SQLAlchemyUserDatastore(db, User, Role)
    security.init_app(app, user_datastore)
In my @app.before_first_request function in the top-level .py, which calls user_datastore, it kicks back:

code:
from app_data import create_app, db, user_datastore
...
if not user_datastore.find_user(email="jjohnson@whitealuminum.com"):
     > AttributeError: 'NoneType' object has no attribute 'find_user'

D34THROW fucked around with this message at 21:05 on Apr 13, 2022

lazerwolf
Dec 22, 2009

Orange and Black
I have a best practices question.

Preface: I’m working in Django and DRF. I try to keep any complex business logic in services.py and call those functions from my DRF views. DRF serializers are great because they can pass validated data to a function with the following syntax
code:
my_function(**serializer.validated_data)
What’s the best way in py3.7+ to structure arguments for my_function if the args could be many nested serializers?

Example
code:

from models import Animal, Species, Demographic

def my_function(animal_id: str, species: str, demographics: Dict[str, Any] = {}) -> Animal:
  species = Species.objects.create(label=species)
  animal = Animal.objects.create(animal_id=animal_id, species=species)
  Demographic.objects.create(
    animal=animal,
    sex=demographics.get("sex", "U"),
    date_of_birth=demographics.get("date_of_birth"),
    weight=demographics.get("weight")
    ...etc
  )
  return animal

Should I create a dataclass to define the acceptable fields for demographic or is that overkill?

I certainly don’t want to pass each value as it’s own arg. I have functions that created 4-5 nested models so the number of args gets large quickly

lazerwolf fucked around with this message at 20:38 on Apr 13, 2022

QuarkJets
Sep 8, 2008

D34THROW posted:

I am having an absolute bitch of a time with a problem I am going to have a hard time explaining, but here goes.

I'm using Flask-Security, but all the examples use a single module to hold everything. What I am trying to do is have user_datastore globally available. To do that, I'm putting as a module-level variable in the app's __init__.py file, which also contains the app factory.

I can't set user_datastore at the module level until it's set in the app factory - because I need the db object that isn't created until , but setting a user_datastore variable in create_app doesn't change the module-level init.

I need a way to set the __init__.py level variable from within a function and I am out of my loving mind on how to do it.

EDIT: Here's the dumb hacky poo poo I'm trying now.
Python code:
# Set up everything needed for `create_app`
db = SQLAlchemy()migrate = Migrate()
...
user_datastore = None


def create_app(config_class=Config):
    """
    Flask app factory function.

    Parameters
    ----------
    config_class : Config
        The configuration object, passed to `app.config` to set things
        up from `config.py`.
    """
    # Set up the app.
    app = Flask(__name__)
    ...
    # Set up the security objects.
    from app_data.models import User, Role

    globals()["user_datastore"] = SQLAlchemyUserDatastore(db, User, Role)
    security.init_app(app, user_datastore)
In my @app.before_first_request function in the top-level .py, which calls user_datastore, it kicks back:

code:
from app_data import create_app, db, user_datastore
...
if not user_datastore.find_user(email="jjohnson@whitealuminum.com"):
     > AttributeError: 'NoneType' object has no attribute 'find_user'

Each module has its own scope. When you import from one module to another, those values are copied to the new scope. Altering that new variable, even via globals(), will not alter the variables that exist in another module's scope.

Even if you could do this, you absolutely shouldn't anyway; this kind of design is incredibly frustrating to try and debug when problems occur. I don't know what the examples you're seeing look like, but if they're doing things like this then you should look for new tutorials.

Why do you need a global user_datastore variable? Just pass it around to whatever objects need it. Stick it in a class and pass that around instead, if you want

QuarkJets
Sep 8, 2008

lazerwolf posted:

I have a best practices question.

Preface: I’m working in Django and DRF. I try to keep any complex business logic in services.py and call those functions from my DRF views. DRF serializers are great because they can pass validated data to a function with the following syntax
code:
my_function(**serializer.validated_data)
What’s the best way in py3.7+ to structure arguments for my_function if the args could be many nested serializers?

Example
code:

from models import Animal, Species, Demographic

def my_function(animal_id: str, species: str, demographics: Dict[str, Any] = {}) -> Animal:
  species = Species.objects.create(label=species)
  animal = Animal.objects.create(animal_id=animal_id, species=species)
  Demographic.objects.create(
    animal=animal,
    sex=demographics.get("sex", "U"),
    date_of_birth=demographics.get("date_of_birth"),
    weight=demographics.get("weight")
    ...etc
  )
  return animal

Should I create a dataclass to define the acceptable fields for demographic or is that overkill?

I certainly don’t want to pass each value as it’s own arg. I have functions that created 4-5 nested models so the number of args gets large quickly

If you have a bunch of grouped-together data that you're using over and over, then that sounds like a slam dunk case for a dataclass. I don't think "overkill" is the right way to describe this, it's likely going to make your code a lot simpler to read and write once you've defined the object. "I have these 5 values that a bunch of different functions need" is totally dataclass territory

Sleepy Robot
Mar 24, 2006
instant constitutional scholar, just add astonomist
Is it best practice to have some python packages installed on the system python for programs that you don't really use in a project but instead use kind of like command line utilities? I mean rather than having them installed from a virtual environment. For example, I have yt-dlp installed for downloading youtube videos, or dte for doing date calculations on the fly. I just have these in my system python and it seems to be working ok so far.

12 rats tied together
Sep 7, 2006

QuarkJets posted:

If you have a bunch of grouped-together data that you're using over and over, then that sounds like a slam dunk case for a dataclass. I don't think "overkill" is the right way to describe this, it's likely going to make your code a lot simpler to read and write once you've defined the object. "I have these 5 values that a bunch of different functions need" is totally dataclass territory

absolutely agreed, because then you can also annotate your functions as accepting objects of that type, and then you can run one of the static type checkers to give you red squiggly lines if you ever screw that up

Sleepy Robot posted:

Is it best practice to have some python packages installed on the system python for programs that you don't really use in a project but instead use kind of like command line utilities? I mean rather than having them installed from a virtual environment. For example, I have yt-dlp installed for downloading youtube videos, or dte for doing date calculations on the fly. I just have these in my system python and it seems to be working ok so far.

I use a ton of python-based utilities at my job, which is being a janitor for computers, and the thing I usually do is have pyenv set a version inside of my "junk drawer" folder that all of my stuff uses. I never install anything to the system python on purpose.

lazerwolf
Dec 22, 2009

Orange and Black

12 rats tied together posted:

absolutely agreed, because then you can also annotate your functions as accepting objects of that type, and then you can run one of the static type checkers to give you red squiggly lines if you ever screw that up

This was my intent. I’d like to ensure the “demographic” dict has a certain form. This might be the only function that would use the dataclass but I want to utilize serializer functionality in non DRF context.

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug

lazerwolf posted:

This was my intent. I’d like to ensure the “demographic” dict has a certain form. This might be the only function that would use the dataclass but I want to utilize serializer functionality in non DRF context.

If a dictionary leaves the scope it was made in, it's a good candidate for sticking in a data class, IMO.

I took over some software and we have a couple places where it's just like
code:
requests: dict
and I'm like WHAT THE gently caress IS IN HERE. I'm gonna go through and fix that poo poo up at some point, because it drives me crazy to have to figure out where we make this drat thing like eight steps away from where I'm working with it again.

QuarkJets
Sep 8, 2008

Sleepy Robot posted:

Is it best practice to have some python packages installed on the system python for programs that you don't really use in a project but instead use kind of like command line utilities? I mean rather than having them installed from a virtual environment. For example, I have yt-dlp installed for downloading youtube videos, or dte for doing date calculations on the fly. I just have these in my system python and it seems to be working ok so far.

Most linux flavors ship with a system python (and make a ton of python packages available via yum or apt) for exactly this reason. Python is pretty handy and has a lot of cool stuff built from it, so it's reasonable to make that stuff available from a standard system location for multiple users to use. So if you have some Python package that you want to make available to your users, especially users who aren't doing their own development, putting it in the system location is a-okay in my opinion.

Falcon2001 posted:

If a dictionary leaves the scope it was made in, it's a good candidate for sticking in a data class, IMO.

I took over some software and we have a couple places where it's just like
code:
requests: dict
and I'm like WHAT THE gently caress IS IN HERE. I'm gonna go through and fix that poo poo up at some point, because it drives me crazy to have to figure out where we make this drat thing like eight steps away from where I'm working with it again.

I too am flummoxed when I come situations where dicts are being passed around. It's not ideal, even when the dict is documented someplace (and let's face it, the documentation will wind up being full of lies at some point, if not immediately). It works fine enough obviously, you can robustly handle missing keys and whatnot, but it just feels a lot more fragile and harder to maintain.

Adbot
ADBOT LOVES YOU

teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself
HI! I have a SQLAlchemy question about getting additional data from an association object. The gist is that I have threads and users and I want to get total posts read and last post ID read from an association object given a user ID

The context for the issue I'm trying to solve is located at this post in the SQL thread:

teen phone cutie posted:

Not sure if there's a better thread for this, but I have a performance questions re: a personal project I'm working on: What do you people think is the most performant way to do the "unread count" for threads that Something Awful has? Talking about this thing:



I'm building this feature into my project where I have "threads," "posts," and "users" and I'm thinking about making association table to store maybe the thread ID, user ID, and just arbitrary "number of posts read," but I also feel like that table could get really big, really quick and I wonder if there's a better way to handle something like this.

I've been poking around at other forums and SA seems to be one of the only forums that even try to show that count, so I wonder if other forums just said gently caress it, we're not doing that because of this exact performance concern.

And now I've begun implementation

So after some reading through the SQLAlchemy docs, I've come to the conclusion that since I need an association table with additional data, what I need is an association object: https://docs.sqlalchemy.org/en/14/orm/basic_relationships.html#association-object

So now my models look something like this:

Python code:
class ReadCount(db.Model):
    __tablename__ = "read_count"

    def as_dict(self):
        return {c.name: getattr(self, c.name) for c in self.__table__.columns}

    user_id = db.Column(db.ForeignKey("users.id"), primary_key=True)
    thread_id = db.Column(db.ForeignKey("threads.id"), primary_key=True)
    last_read_post_id = db.Column(db.ForeignKey("posts.id"), nullable=True)
    number_posts_read = db.Column(db.Integer, nullable=False, default=0)
    thread = db.relationship("Thread", backref="read_count")
    user = db.relationship("User", backref="read_count")


class Thread(db.Model):
    __tablename__ = "threads"

    def as_dict(self):
        return {c.name: getattr(self, c.name) for c in self.__table__.columns}

    id = db.Column(db.Integer, primary_key=True, autoincrement=True)
    title = db.Column(db.String(255), unique=False, nullable=False)

    def __init__(
        self,
        title,
    ):
        self.title = title

class User(db.Model):
    __tablename__ = "users"

    def as_dict(self):
        return {c.name: getattr(self, c.name) for c in self.__table__.columns}

    id = db.Column(db.Integer, primary_key=True, autoincrement=True)
    email = db.Column(EmailType, unique=True, nullable=False)

    def __init__(
        self,
        email
    ):
        self.email = email
What I'm stuck on now is this: how do I go about querying my association object to get both last_read_post_id and number_posts_read where the user ID === XXX

So far, my GET /threads logic looks like this

Python code:
	user_id = 1234

	"""
	how does this query change to get the additional fields where user ID is equal to the one provided?
	"""
        assoc_query = ReadCount.query.filter_by(user_id=user_id)
        filtered_query = (
            Thread.query.filter(
                and_(Thread.is_deleted == False, Thread.forum_id == int(forum_id))
            )
            .order_by(Thread.is_sticky.desc(), Thread.last_post_at.desc())
        )
So essentially my returned JSON for a thread is something like this:

JavaScript code:
[{
  "id": 1,
  "title": "hello world",
  "is_sticky": true
}, {
  "id": 2,
  "title": "hello blurld",
  "is_sticky": false
}]
and I want it to look like this

JavaScript code:
[{
  "id": 1,
  "title": "hello world",
  "is_sticky": true,
  "last_read_post_id": 23,
  "number_posts_read": 40
}, {
  "id": 2,
  "title": "hello blurld",
  "is_sticky": false,
   "last_read_post_id": null,
  "number_posts_read": 0
}]

teen phone cutie fucked around with this message at 07:12 on Apr 14, 2022

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply