Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
QuarkJets
Sep 8, 2008

Yeah can't you just assign a variable and use that in the f-string several times? I don't think there's any actual advantage to using format()

Adbot
ADBOT LOVES YOU

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
The advantage being talked about is you can assign your format string to a variable and use it in several different places, instead of repeating it everywhere.

QuarkJets
Sep 8, 2008

I'm still not sure that I understand. Could an example help?

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Sure. With format strings you can do something like:

code:
user_fmt = '<a href="/users/{id}">{name}</a>'

// later...

user_fmt.format(**thread.first_poster)
user_fmt.format(**thread.last_poster)
It's kind of marginal because you could always write a format_user function, but it is something you can do with .format() that you can't do directly with fstrings.

QuarkJets
Sep 8, 2008

Ah, that makes a lot of sense! Thank you

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug
I've used that same concept with dealing with templates without kinja for simple email forms at work; we just load the template and then .format() it. But everywhere else is fstrings.

bigperm
Jul 10, 2001
some obscure reference

Falcon2001 posted:

Depending on how much you want to try a new tech, I've been seeing HTMX a lot lately as a JS-less (but built in JS) library that seems to be a nice lightweight alternative to vue/react/etc. I haven't tried it myself though, and I'm curious how other view it, but the videos/etc seem pretty promising.

edit: here's a video example that seems to at least be somewhat near what you're looking at :
https://www.youtube.com/watch?v=kNXVFB5eQfo&t=1079s

I like htmx a lot. Just ask for some html, put it somewhere (I guess it does more than that though). Can even use flask/django to send some jinja template partial. I made a website that generates 'poems' from a hilarious list of "1000 Most Common English Phrases" and used htmx for the infinite scroll and it was dead simple.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
While I've not yet learned how to do pull requests and poo poo like that, I raised an issue on borb and the author fixed it for the next release. It was literally just a from x import y quality-of-life suggestion but it's a good feeling :shobon:

Hughmoris
Apr 21, 2007
Let's go to the abyss!

D34THROW posted:

While I've not yet learned how to do pull requests and poo poo like that, I raised an issue on borb and the author fixed it for the next release. It was literally just a from x import y quality-of-life suggestion but it's a good feeling :shobon:

:hfive:

That's a great feeling, ain't it? I made a couple of extremely minor changes for two Microsoft repos, that were surprisingly accepted. So I now tell my non-tech friends that I've contributed to Microsoft's code base. :smug:

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug

bigperm posted:

I like htmx a lot. Just ask for some html, put it somewhere (I guess it does more than that though). Can even use flask/django to send some jinja template partial. I made a website that generates 'poems' from a hilarious list of "1000 Most Common English Phrases" and used htmx for the infinite scroll and it was dead simple.

Out of curiosity do you have the source for that lying around? Would be interesting to see.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Motherfucker. I thought dumping n DB records to a file (without committing) for testing purposes was going to be a simple process. First, I was just dumping raw formatted JSON but I didn't like the readability. Then I decided to use tabulate

I didn't expect getting this
code:
=================== Report ==================
        Column  Value
--------------  -----------------------------
          data  {
                 "placeholder": "placeholder"
                }
create_user_id  1
       type_id  5
        job_id  0

====== PanRoof ======
      Column  Value
------------  -------
       width  192
  projection  96
fascia_sides  2
      gutter  True
   insulated  True
to take this
Python code:
def bulk_dump(*records) -> bool:
    import tabulate as tb

    path = "./test_output/db_obj/test_dump.txt"
    with open(path, "w") as file:
        for record in records:
            name = f" {get_classname(record)} "
            # Remove the SQLAlchemy instance state from the dict so it
            # can serialize properly
            record.__dict__.pop("_sa_instance_state")
            record = record.__dict__
            for k, v in record.items():
                if type(v) == dict:
                    record[k] = dumps(v, indent=1)
            table = tb.tabulate(
                [(k, v) for k, v in record.items()],
                headers=["Column", "Value"],
                tablefmt="simple",
                colalign=("right", "left"),
            )
            header = name.center(
                (table.find("\n", table.find("\n") + 1) - (table.find("\n")))
                - 1,
                "=",
            )
            data = f"{header}\n{table}\n\n"
            file.write(data)
    size = getsize(path)
    print(
        f"  Output file size: {size} bytes ({ceil(size / 1024)} kB / "
        f"{round((size / 1024 / 1024), ndigits=2)} MB)"
    )
    return True
I just want to verify that Flask forms are passing the proper values to the loving DB records :shepicide:

(The return True is there so that it gives pytest something to assert.)


EDIT: Crossposting from coding horrors for self-schad:

D34THROW posted:

I just found one of my very first Python projects, forked off from an old CLI iteration of the project I'm working on now. I thought I was loving slick.

I was so loving wrong.

D34THROW fucked around with this message at 17:59 on May 13, 2022

bigperm
Jul 10, 2001
some obscure reference

Falcon2001 posted:

Out of curiosity do you have the source for that lying around? Would be interesting to see.

I think I saved the page I got them from initially but that was a few laptops ago and I can't find it. I had to clean it up quite a bit and it's all in an sqlite database now. I added a page to the site to list them all though : https://zerorg.com/list

bigperm fucked around with this message at 00:34 on May 14, 2022

pmchem
Jan 22, 2010


anaconda 2022.05 was released earlier this week:
https://www.anaconda.com/blog/new-release-anaconda-distribution-now-supporting-m1
https://docs.anaconda.com/anaconda/reference/release-notes/

native support for m1 macs is the highlight

also has conda 4.12.0 which supports libmamba for faster dep solves:
https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community

CompeAnansi
Feb 1, 2011

I respectfully decline
the invitation to join
your hallucination

pmchem posted:

anaconda 2022.05 was released earlier this week ... native support for m1 macs is the highlight

Hurray! Been waiting for that. Although I wonder how nicely that is going to play with some of the libraries that use executables that haven't been ported over yet (e.g., opencv).

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Love writing a function that's 31 lines of docstring (so I don't forget how the gently caress to use it), and 4 lines of code (that really could fit on one line but for PEP8) :shepicide:

MightyBigMinus
Jan 26, 2020

hello pypeople, now that aws lambda supports direct function urls I was able to get a python function working without getting lost in apigateway again.

when i test it with curl it works great. when i load it in chrome it runs twice, because chrome automatically asks for favicon and my script is so loving basic it doesnt even check what url is being called

so I figure i'm at the part where I should have something like "routing" and "error handling". where would you go from here? Ideally i'd like to take my current thing and make it only actually run when the request is a post to /foo with a json body with specific keys meeting specific options. then later on maybe i could turn GET /foo into something useful, and maybe even GET / into some kind of base/home page?

should I go back to trying to move that into apigateway? should I start watching intro to flask tutorials on youtube? is fastapi really the place to start now or is it just a new fad?

i would very much like to avoid the kitchen-sink/week-long-project feeling I'm getting from "the serverless framework" and django. I don't even have a db. all the tutorials for them are about my dev environment on my laptop or a deploy pipeline. neither of those are my problem, its already deployed and I don't give a poo poo if it ever runs on my laptop i want it to run in lambda.

MightyBigMinus fucked around with this message at 20:52 on May 18, 2022

CarForumPoster
Jun 26, 2013

⚡POWER⚡

MightyBigMinus posted:

hello pypeople, now that aws lambda supports direct function urls I was able to get a python function working without getting lost in apigateway again.

when i test it with curl it works great. when i load it in chrome it runs twice, because chrome automatically asks for favicon and my script is so loving basic it doesnt even check what url is being called

so I figure i'm at the part where I should have something like "routing" and "error handling". where would you go from here? Ideally i'd like to take my current thing and make it only actually run when the request is a post to /foo with a json body with specific keys meeting specific options. then later on maybe i could turn GET /foo into something useful, and maybe even GET / into some kind of base/home page?

should I go back to trying to move that into apigateway? should I start watching intro to flask tutorials on youtube? is fastapi really the place to start now or is it just a new fad?

i would very much like to avoid the kitchen-sink/week-long-project feeling I'm getting from "the serverless framework" and django. I don't even have a db. all the tutorials for them are about my dev environment on my laptop or a deploy pipeline. neither of those are my problem, its already deployed and I don't give a poo poo if it ever runs on my laptop i want it to run in lambda.

Its unclear what your requirements are. If you just want an API that eats your POST, Lambda + API Gateway is probably the easiest. If your whole python deployment will always be under 250MB. Zappa will do most of the config for you, no API Gateway worries. If it might ever be bigger than that, check out the AWS SAM CLI, specifically deploying container images with it. Still pretty easy, now your deployments can be 10GB.

If you want to return a webpage, now you're chattin about web frameworks. What are the requirements?

CarForumPoster fucked around with this message at 21:05 on May 18, 2022

MightyBigMinus
Jan 26, 2020

total opposite end of the spectrum, i don't have 250mb problems or 10gb problems or container problems or setup & deployment problems.

i'm using the standard library + boto3 in a single file and editing in the web interface. i don't even have a build step yet. there is no requirements.txt.

all of the aws/iam stuff is already done (in tf) so zappa and sam look... wrong/different-issue?

i wanna take the fully working thing i have now and wrap it in something that let me constrain it to only execute on POST /foo and also check the message body that {'property': 'value'} is present.

now i can obviously just code that myself, parsing the event object and writing a bunch of if/else poo poo, but this feels super boilerplatey and frameworky so surely there's a better way.

even more magic would be coughing up an openapi/swagger-ui easily so i don't have to explain how to use curl to anyone

CarForumPoster
Jun 26, 2013

⚡POWER⚡

MightyBigMinus posted:

total opposite end of the spectrum, i don't have 250mb problems or 10gb problems or container problems or setup & deployment problems.

i'm using the standard library + boto3 in a single file and editing in the web interface. i don't even have a build step yet. there is no requirements.txt.

all of the aws/iam stuff is already done (in tf) so zappa and sam look... wrong/different-issue?

i wanna take the fully working thing i have now and wrap it in something that let me constrain it to only execute on POST /foo and also check the message body that {'property': 'value'} is present.

now i can obviously just code that myself, parsing the event object and writing a bunch of if/else poo poo, but this feels super boilerplatey and frameworky so surely there's a better way.

even more magic would be coughing up an openapi/swagger-ui easily so i don't have to explain how to use curl to anyone

Sounds like you want FastAPI.

It has a swagger UI out of the box, extremely easy to use. It was easy enough to get up and running on Lambda that I can't remember how long the setup took because it wasn't a pain.

Doing what you describe looks like this:
code:
from typing import Union

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
    return {"item_id": item_id, "q": q}

QuarkJets
Sep 8, 2008

D34THROW posted:

Love writing a function that's 31 lines of docstring (so I don't forget how the gently caress to use it), and 4 lines of code (that really could fit on one line but for PEP8) :shepicide:

That's probably too much docstring or a function with too much unrelated behavior

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Probably more too much docstring. I use numpydoc-style docstrings because I have an easier time visually parsing them, so I have parameters, returns, notes, and examples sections. It's a template function to generate the class attribute for an HTML tag that depends on the session["ADMIN_MODE"], essentially a shortcut for a big jinja if/else block that exists to assign the proper class for admin or user mode.



I mean, if that's too much docstring, I'm more than open to advice on how to pare it down. I want to contribute more to open-source projects and stuff like that so a lesson on brevity is welcome.

Armitag3
Mar 15, 2020

Forget it Jake, it's cybertown.


D34THROW posted:

Probably more too much docstring. I use numpydoc-style docstrings because I have an easier time visually parsing them, so I have parameters, returns, notes, and examples sections. It's a template function to generate the class attribute for an HTML tag that depends on the session["ADMIN_MODE"], essentially a shortcut for a big jinja if/else block that exists to assign the proper class for admin or user mode.



I mean, if that's too much docstring, I'm more than open to advice on how to pare it down. I want to contribute more to open-source projects and stuff like that so a lesson on brevity is welcome.

Python is pretty flexible when it comes to how exhaustive the docstring needs to be. I tend to use the 6 month/peer rule: how complicated would it be to use this function as me 6 months from now/as another user or peer?

If there are a fair amount of gotchas or lots of parameters, you’ll want a good exhaustive docstring, there’s no real upper limit to how helpful you want to be here. Otherwise, you’ll find that a succinct passage is enough, especially for functions that take none or one or two params and are like 5 lines in length. If you have empathy for your fellow users then you’ll find the right balance no problem.

(e: docstrings in a lot of pythons own api can be pretty succinct and short if you take a look)

SirPablo
May 1, 2004

Pillbug
Someone explain or direct me somewhere... What does the "-> str" do when defining the function?

Armitag3
Mar 15, 2020

Forget it Jake, it's cybertown.


SirPablo posted:

Someone explain or direct me somewhere... What does the "-> str" do when defining the function?

-> at the end of the function hints at the return type. -> str means it returns a string.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
It can also be used by static type checkers like mypy to make sure that functions are returning properly. Even though it's ducktyped, I migrated to Python from Visual Basic for Applications; VBA is a statically typed language and I vastly prefer static typing. I try as hard as I can to keep

Get into the typing module and the return annotations and you get crazy poo poo like def with_lock(f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]: Typehints are fun :v:

SirPablo
May 1, 2004

Pillbug

Armitag3 posted:

-> at the end of the function hints at the return type. -> str means it returns a string.

Ah interesting. So it's almost a comment of sorts, doesn't actually do anything functionally by itself.

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug
Can't stress enough the sanity-saving that type hints offers for python. Sure, it's not perfect, but it is at minimum a nice way to make your IDE way more effective.

Data Graham
Dec 28, 2009

📈📊🍪😋



SirPablo posted:

Ah interesting. So it's almost a comment of sorts, doesn't actually do anything functionally by itself.

It makes it so your IDE can say "oh, this function you're calling here says it returns a string, but then you're adding a number to it so that might be a type mismatch" and give you squiggly lines so you know whether or not to expect it'll run right.

Armitag3
Mar 15, 2020

Forget it Jake, it's cybertown.


SirPablo posted:

Ah interesting. So it's almost a comment of sorts, doesn't actually do anything functionally by itself.

Yeah, python is still ducktyped so you can go against your own hints. Like D34THROW says, they can be used by libs like mypy to actually enforce types by giving you errors on a pre-build.

QuarkJets
Sep 8, 2008

D34THROW posted:

Probably more too much docstring. I use numpydoc-style docstrings because I have an easier time visually parsing them, so I have parameters, returns, notes, and examples sections. It's a template function to generate the class attribute for an HTML tag that depends on the session["ADMIN_MODE"], essentially a shortcut for a big jinja if/else block that exists to assign the proper class for admin or user mode.



I mean, if that's too much docstring, I'm more than open to advice on how to pare it down. I want to contribute more to open-source projects and stuff like that so a lesson on brevity is welcome.

The biggest problem with lengthy comments is the notion of "decay" or "rot". Longer comments rot faster than shorter comments. Longer docstrings need more review every time that you make a change to the code (or, if you don't review, will be more likely to contain lies when you're done). Misdocumented code is worse than undocumented code, so try to write documentation that is succinct and difficult to accidently break later.

Are you running flake8 or black on your code? If not, you should. Look into setting up pre-commit hooks and define flake8 as a step

I think the steps inside the function can be broken up to improve readability. Assign a string to a "user_class" variable. Assign another string to "joined_classes". Then return an f-string containing both, or just concatenate them.

Remove the parens around the returned value

Brush up on what pep8 has to say about docstrings.

Where does the session variable come from? The session or maybe the session mode should be a function argument.

Your docstring doesn't need a labeled Notes section. Docstrings are already notes

You don't need 2 examples. You may not even need 1. This is a simple enough function, it's just concatenating a bunch of strings with "user" or "admin" at the front.

If you use typehints, then don't put types in your docstring. This is not a mild suggestion, it's important that you break this habit.

Tayter Swift
Nov 18, 2002

Pillbug
I'm starting to get some real imposter syndrome when it comes to pandas and dask. Here's some simple code:

code:
print('writing parquet')

big_future = client.scatter(ddf)

def doit(_):
    _.to_parquet(path='raw 21q4/', schema='infer')
future = client.submit(doit, big_future)
dask.distributed.wait(future)
dask was yelling at me about the task graph when I was simply writing "ddf.to_parquet()," so okay I'm doing it as a scatter like it says. I expect this code to create a parquet file series in the specified path. Instead it churns forever, and deposits nothing. The dask dashboard shows that it's clearly working during this time. Is there something in this code that makes it just ignore the actual 'write the goddamn parquet' step?

Maybe I just don't understand how the hell dask works.

edit: never loving mind, I was writing to the wrong goddamned directory :ughh:

Tayter Swift fucked around with this message at 21:56 on May 19, 2022

QuarkJets
Sep 8, 2008

I am familiar with spark but not dask. I'd really like to learn more about dask since it seems like that's the tool I'd need to make all of my dreams come true with GPUs

Tayter Swift
Nov 18, 2002

Pillbug
In my case I'm porting some rather involved ETL on some large fixed-width format files (40GB, 50 million rows by 100+ columns) over from ancient SAS code. It's been pretty slow going as I'm feeling a bit over my head, and right now it's way the hell slower than the SAS code.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:

QuarkJets posted:

docstring rot

I run both, black on file save and flake8 on pre-commit. I would be running mypy pre-commit too if I had started looking into typehints earlier.

The original version of the function worked the way you said, helper strings defined then returned in an f-string.

The session variable comes from Flask, it's imported at the top of the module.

Understood on the notes, I can stick that right after the opening paragraph and it makes more sense that way.

I included the examples to make it more obvious what will be output; maybe something like that can go in the opener?

I include types because numpydoc seemed to want them; looking at numpy's code, they don't typehint in the function definition, it's in the docstring. I suppose it makes sense then that the function definition does the job of typehinting, especially if I'm doing something like Optional[int] = 0 to make it obvious to 6-months-from-now me.

Revised:


I take this input seriously and appreciate it :)

D34THROW fucked around with this message at 19:39 on May 20, 2022

QuarkJets
Sep 8, 2008

That's way cleaner! The single example output is fine I think

lazerwolf
Dec 22, 2009

Orange and Black
There's a small bug in your code if you don't pass any additional classes you end up with an additional trailing space
code:
# session['ADMIN_MODE'] = True
print(get_class_by_mode())
"class=admin "
why not just cast *classes as a list and use join for all parts? Empty lists won't add additional trailing spaces

code:
def get_class_by_mode(*classes: Optional[str]) -> str:
    mode = "admin" if session['ADMIN_MODE'] else "user"
    return f"class={' '.join([mode] + list(classes))}"

# session['ADMIN_MODE'] = True
print(get_class_by_mode())
"class=admin"

print(get_class_by_mode("test", "me"))
"class=admin test me"

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:

lazerwolf posted:

There's a small bug in your code if you don't pass any additional classes you end up with an additional trailing space
code:
# session['ADMIN_MODE'] = True
print(get_class_by_mode())
"class=admin "
why not just cast *classes as a list and use join for all parts? Empty lists won't add additional trailing spaces

code:
def get_class_by_mode(*classes: Optional[str]) -> str:
    mode = "admin" if session['ADMIN_MODE'] else "user"
    return f"class={' '.join([mode] + list(classes))}"

# session['ADMIN_MODE'] = True
print(get_class_by_mode())
"class=admin"

print(get_class_by_mode("test", "me"))
"class=admin test me"

I wasn't sure how to get around that without more code. I did not think of list concatenation, thank you!

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
In my latest bit of surprising-myself code, I realized that a function I was writing with @singledispatch and would have violated DRY with could be boiled down quite easily - it takes an id, start date, and end date, builds a query, and executes it against the DB. returning the Decimal result.

Originally, it would have been get_sales_by_rep, get_sale_by_branch, etc. but I realized that I can map the table object names to a dict and use that to assign the table to get the FK `id` from. If that makes sense.

Python code:
def get_sales(
    name: Union[str, db.Model],
    id: int,
    date_start: datetime = None,
    date_end: datetime = None,
) -> Decimal:
    """ <docstring removed for brevity> """
    TABLE_IDS = {
        "Salesperson": Job.sales_rep_id,
        "Branch": Job.branch_id,
        "Municipality": Job.municipality_id,
        "User": Job.create_user_id,
        "Product": Job.product_id,
    }

    if not type(name) == str:
        name == name.__name__

    criterion = TABLE_IDS.get(name, None)

    if not criterion:
        # TODO Write some error logic here.
        pass

    if date_start and date_end:
        query = (
            db.select(func.sum([Job.contract_amount]))
            .filter(criterion == id)
            .filter(Job.contract_date >= date_start)
            .filter(Job.contract_date <= date_end)
        )
    elif date_start and not date_end:
        query = (
            db.select(func.sum([Job.contract_amount]))
            .filter(criterion == id)
            .filter(Job.contract_date >= date_start)
        )
    elif not date_start and date_end:
        query = (
            db.select(func.sum([Job.contract_amount]))
            .filter(criterion == id)
            .filter(Job.contract_date <= date_end)
        )
    elif not date_start and not date_end:
        query = db.select(func.sum([Job.contract_amount])).filter(
            criterion == id
        )

    return db.session.execute(query).all()[0]
Obviously a switch case would be more elegant here but I am stuck without upgrading for now. Just one of those moments that made me :unsmith:


EDIT: I've also completely eschewed sesson["ADMIN_MODE"] as something I'm using. I'm using flask-admin for that now, and you can only get to that index page with the proper roles. :v:

Deffon
Mar 28, 2010

Nice!

One possible improvement is to separate the general query handling from the date range filtering.

You can always start out with a basic query and then add stuff to it before executing it.

Something like this:

Python code:
query = db.select(func.sum([Job.contract_amount]))
    .filter(criterion == id)

if date_start:
    query = query.filter(Job.contract_date >= date_start)
if date_end:
    query = query.filter(Job.contract_date <= date_end)

return db.session.execute(query).all()[0]

Adbot
ADBOT LOVES YOU

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Thanks! That was my original plan but I brain-farted on the implementation and forgot I could apply .filter to an object that was already a .select:dumb:

D34THROW fucked around with this message at 20:47 on Jun 3, 2022

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply