Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

cinci zoo sniper: Mar 15, 2013

Boris Galerkin posted:

What is the easiest/proper way of securing an api endpoint with a token that only one person (me) will ever use?

My (naive) way is that I'm thinking I need to generate a random token and put this value in a read only text/config file that my script can read, and then on the user end I just set an Authorization header with that token and in my code I would compare the values and exit if they don't match.

I have no idea what I'm doing though so I feel like someone is going to tell me that this is a bad idea.

OAuth 2.

# ? Jan 28, 2019 15:49

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 10:05

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

cinci zoo sniper posted:

OAuth 2.

do this

dont do the stupid thing

its a pain in the butt but do it

# ? Jan 28, 2019 15:54

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Maybe I�m misunderstanding something but oauth 2 looks like the system where I log into Facebook to get a token?

I�m not building a website where I can log in and stuff, it�s just an api endpoint that I get data from via curl

code:

curl -H �Authorization: my_token� -X POST my_website.com/api/get/data

I don�t understand how authenticating via Facebook will help me here.

# ? Jan 28, 2019 16:08

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

Boris Galerkin posted:

Maybe I’m misunderstanding something but oauth 2 looks like the system where I log into Facebook to get a token?

I’m not building a website where I can log in and stuff, it’s just an api endpoint that I get data from via curl
code:
curl -H &#8220;Authorization: my_token&#8221; -X POST my_website.com/api/get/data
I don’t understand how authenticating via Facebook will help me here.

You are facebook in this scenario

(Oauth 2 is a standard)

# ? Jan 28, 2019 16:15

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

bob dobbs is dead posted:

You are facebook in this scenario

(Oauth 2 is a standard)

So if I'm understanding correctly, I need to create an oauth2 server to handle giving out tokens and authorizing them when I use curl.

How would I get the token in the first place then? Like I said this website is just the one api url. Would I need to create an admin.html page or something? That just seems excessive.

(Sorry if I'm being difficult, I genuinely do not understand how this would work.)

# ? Jan 28, 2019 16:27

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

bob dobbs is dead posted:

its a pain in the butt but do it

# ? Jan 28, 2019 16:36

Scrapez: Feb 27, 2004

I'm not familiar with Python and am trying to augment an AWS Lambda script written in Python. I'm getting tripped up trying to assign a variable to be a multi-line string where the contents of the string are contents of other variables.

iE:
var1 = foo
var 2 = bar

var 3 = foo
bar

I've tried:
var 3 = '''\
{var1}
{var2}\
'''

But instead of the variable contents for var1 and var2, I just get var1 and var2. Any pointers in the right direction are appreciated.

# ? Jan 28, 2019 16:45

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

Scrapez posted:

I'm not familiar with Python and am trying to augment an AWS Lambda script written in Python. I'm getting tripped up trying to assign a variable to be a multi-line string where the contents of the string are contents of other variables.

iE:
var1 = foo
var 2 = bar

var 3 = foo
bar

I've tried:
var 3 = '''\
{var1}
{var2}\
'''

But instead of the variable contents for var1 and var2, I just get var1 and var2. Any pointers in the right direction are appreciated.

you want f-strings

they're only available in really new pythons

the older ones, you do, "{var1}, {var2}".format(var1=blah, var2=bleh)

you will also see c-style format strings

see dealio for more info

https://realpython.com/python-string-formatting/#2-new-style-string-formatting-strformat

# ? Jan 28, 2019 16:49

Scrapez: Feb 27, 2004

bob dobbs is dead posted:

you want f-strings

they're only available in really new pythons

the older ones, you do, "{var1}, {var2}".format(var1=blah, var2=bleh)

you will also see c-style format strings

see dealio for more info

https://realpython.com/python-string-formatting/#2-new-style-string-formatting-strformat

This appears to be a Python 2.x script so I will probably have to go the older route. So the part I'm missing is the .format section it would appear?

Edit: Adding the .format section indeed worked. Thank you for the assistance.

Scrapez fucked around with this message at 17:07 on Jan 28, 2019

# ? Jan 28, 2019 16:57

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

f-strings are the best.

i just wanted to say that

# ? Jan 28, 2019 17:22

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

Thermopyle posted:

f-strings are the best.

i just wanted to say that

tbh ruby did em better and a fair bit earlier

cant do numerics worth poo poo in ruby tho

# ? Jan 28, 2019 17:25

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

bob dobbs is dead posted:

Yeah the problem isn't that it's a pain in the butt to set up, the problem is I literally don't understand what it is I need to set up and all the tutorials I've found online are for use cases where you have users who can register accounts. In my case I have only one user and it's myself. I don't really want to bother with registering an account because then I'd just be registering an account for myself when I don't expect to have any other users. And also because it's not a website. All I want is to generate a token for myself and to have some kind of mechanism to check if the token is valid. Setting up a Postgres database just to handle my one user, along with a server to handle the oauth stuff just means now I have two more passwords to worry about.

What am I missing because I don't get it.

# ? Jan 28, 2019 17:52

The Fool: Oct 16, 2003

DRF supports token authentication.

https://www.django-rest-framework.org/api-guide/authentication/#tokenauthentication

# ? Jan 28, 2019 18:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

bob dobbs is dead posted:

tbh ruby did em better and a fair bit earlier

cant do numerics worth poo poo in ruby tho

yeah, I know of multiple languages that do them in varying ways. JS has template literals, C# has string interpolation. perl, shell, a bunch of others

wherever it is implemented it is great

# ? Jan 28, 2019 21:16

Scrapez: Feb 27, 2004

I feel like this is probably an easy thing to do but I'm struggling mightily.

I'm trying to put the value portions of one variable into their own unique variables.

The output of RR looks like: [{u'Value': '1 10 5060 10.100.75.237'}, {u'Value': '1 10 5060 10.100.75.238'}]

The following for loop will print out the values by themselves:
1 10 5060 10.100.75.237
1 10 5060 10.100.75.238

for RR in RRS['ResourceRecords']:
VALUE = RR['Value']
print VALUE

I want to put each line in its own variable but no matter how I've tried to do it, I end up with the second line as the value for both variables. Is there an easy way to do this?

# ? Jan 28, 2019 21:47

Nippashish: Nov 2, 2005; Let me see you dance!

Scrapez posted:

I want to put each line in its own variable

Don't do this. Make a list instead. You want 'var0', 'var1', 'var2', etc but what you should want is 'var[0]', 'var[1]', 'var[2]' ....

# ? Jan 28, 2019 21:56

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

take good care of the data structures and the algorithms will take care of themselves

# ? Jan 28, 2019 22:19

PBS: Sep 21, 2015

How do you guys handle internal PKI related trust issues with python?

Appending internal certs to certifi's pem seems to resolve the issue for a number of popular modules, but I don't really like it as a solution for a few reasons.

My preference would be to have an additional module you can import on top of or in place of certifi that would let you load in additional certs, while still working with other modules that expect certifi.

The best solution I've managed to think of is to just have another module that executes some code when imported that appends the certs to the certifi module, but this solution seems a little kludgy. The benefit though is I main maintain an internal build of this module that users could easily import and be done.

I haven't been able to find a true good solution for this yet, but I've seen a litany of posts about it when searching around. Internally a number of people have just been disabling cert verification, which I'd like to help prevent. Without an easy solution no one's going to bother.

PBS fucked around with this message at 06:03 on Jan 29, 2019

# ? Jan 29, 2019 06:00

breaks: May 12, 2001

PBS posted:

Internally a number of people have just been disabling cert verification, which I'd like to help prevent.

if you don't trust anybody, you don't have to trust anybody *taps head*

# ? Jan 30, 2019 06:32

ninepints: Sep 7, 2017; four and a half quarts

Boris Galerkin posted:

Wouldn�t I need to define a setter method for every attribute I declare then? I was hoping attrs provided a hook for a generic one that gets called for all attributes I define with attrib.

E: yeah, it looks like what I want isn�t possible so I�ll just rethink this entirely. Thanks.

If I'm understanding the question correctly, it's definitely possible. Something like this:

Python code:

class Foo(object):
    _MY_SPECIAL_ATTRIBUTES = ['bar', 'baz']

    def __init__(self):
        for attr in self._MY_SPECIAL_ATTRIBUTES:
            super().__setattr__(attr, None)
    
    def __setattr__(self, attr, value):
        if attr in self._MY_SPECIAL_ATTRIBUTES:
            value *= 2
        super().__setattr__(attr, value)

Edit: Wasn't understanding the question correctly/hadn't seen the attrs library before. It sounds like attrs is incompatible with defining your own __setattr__ method, both literally and in the sense that the point of the library is to generate no-surprises boilerplate attribute-setting code.

ninepints fucked around with this message at 09:17 on Jan 30, 2019

# ? Jan 30, 2019 08:41

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Well poo poo, I use type annotations extensively and somehow I never knew TypedDict existed. Pretty great for dealing with REST APIs...

Python code:

from mypy_extensions import TypedDict

Movie = TypedDict('Movie', {'name': str, 'year': int})

movie: Movie = {'name': 'Blade Runner', 'year': 1982}

Kind of makes me think of a TypeScript interface.

My understanding after spending a half hour in github issues is that this will likely come to the standard library typing module.

# ? Feb 1, 2019 00:31

Dominoes: Sep 20, 2007

Neat! Dataclasses also are similar to TS interfaces. (And structs in other languages), and are my go-to for collections of related data. The free things (like display and equality) they come with make them appropriate for Python's high-level, batteries-included niche.

It's interesting how diff languages handle data structures on a spectrum of objects, structs, interfaces, dataclasses, classes etc. There are varying degrees of overlap. For example, pure "Map" types exist in the form of python Dicts, JS Maps, and Rust HashMaps. JS Objects are an awkward mix of Dict and Classes, and as far as I can tell, are the most common way of binding keys and values. Bundles of data with optional methods are seen in Rust/C structs, and python Dataclasses. Python and JS classes can work in a similar way, but with their state setup accompanied by verbose syntax, making it seem like it's not the priority. TS Interfaces and Python NamedTuples set up state strictly, but don't allow methods, and in the latter case, are always immutable.

Dominoes fucked around with this message at 03:42 on Feb 1, 2019

# ? Feb 1, 2019 03:33

NinpoEspiritoSanto: Oct 22, 2013

Pyrsistent is a nice lib for immutable data structures and most of the boilerplate pain with classes is resolved by attrs. I've not played with the new dataclasses in 3.7 yet (too busy playing with Trio!) but intend to.

# ? Feb 1, 2019 09:25

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Another thing thats always irritated me is how limited the Callable type is.

Good luck defining a Callable type for a variadic callback!

Well turns out they added¹ a Protocol type a year ago and no one notified me. Now you can define a type for basically any function signature in a way that's reminiscent (again) of TypeScript interfaces.

Python code:

from typing import Optional, Iterable, List
from typing_extensions import Protocol

class Combiner(Protocol):
    def __call__(self, *vals: bytes, maxlen: Optional[int] = None) -> List[bytes]: ...

def batch_proc(data: Iterable[bytes], cb_results: Combiner) -> bytes:
    for item in data:
        ...

def good_cb(*vals: bytes, maxlen: Optional[int] = None) -> List[bytes]:
    ...
def bad_cb(*vals: bytes, maxitems: Optional[int]) -> List[bytes]:
    ...

batch_proc([], good_cb)  # OK
batch_proc([], bad_cb)   # Error! Argument 2 has incompatible type because of
                         # different name and kind in the callback

¹To the official typing_extensions module.

# ? Feb 1, 2019 18:16

PBS: Sep 21, 2015

Reading the posts in this thread regularly makes me feel inadequate.

# ? Feb 2, 2019 03:33

Dr Subterfuge: Aug 31, 2005; TIME TO ROC N' ROLL

Yeah actually Thermopyle could you explain what's going on in that code?

# ? Feb 2, 2019 04:43

Data Graham: Dec 28, 2009; 📈📊🍪😋

I mean, much as I appreciate the level of mastery and art that such patterns can make possible, there�s a point at which I�m like �sure hope I�m not the guy who has to inherit this code after the person who wrote it leaves�

# ? Feb 3, 2019 01:21

bob dobbs is dead: Oct 8, 2017; I love peeps; Nap Ghost

it's types for functions, that's it

# ? Feb 3, 2019 02:19

NinpoEspiritoSanto: Oct 22, 2013

bob dobbs is dead posted:

it's types for functions, that's it

Yeah it's just type annotations I don't get the freak out? They're a good thing.

# ? Feb 3, 2019 02:23

punished milkman: Dec 5, 2018; would have won

Dr Subterfuge posted:

Yeah actually Thermopyle could you explain what's going on in that code?

it's sorta like an interface you might see in C# or something if you're familiar with that.

the class Combiner is declaring that anything with the type 'Combiner' must have the parameters *vals: bytes and maxlen: Optional[int]. Might be worth googling what the '*' and 'Optional[some type]' parts mean if you're not familiar with them.

the function batch_proc is defined so that its parameter cb_results is expected to have the Combiner type. Anything passed to it has to conform to the scheme declared in the Combiner class up at the top. Thermopyle's code snippet shows this when batch_proc is called with good_cb (no error) and then called with bad_cb (throws an error). That's because bad_cb has "maxitems" which does not conform to the expected name and kind of the "maxlen" parameter defined in Combiner.

hope I didn't botch this explanation too bad.

punished milkman fucked around with this message at 02:42 on Feb 3, 2019

# ? Feb 3, 2019 02:40

NinpoEspiritoSanto: Oct 22, 2013

Also unless I missed a change, that's all but an exercise for the reader or a tool like mypy or one of the 3rd party libs that'll toss an exception if you gently caress up types. It does look a bit messy especially with complicated data structures coming in or being returned, however it's worth getting your head around. Loose typing and type coercion was one of the worst loving ideas to hit programming since java butchered oop or the goto like hell that is callbacks/futures for async programming. The day python lets us enforce typing natively will be a good one.

# ? Feb 3, 2019 02:52

punished milkman: Dec 5, 2018; would have won

as an aside, PyCharm has awesome type annotation support and will constantly scream at you if you're loving up

# ? Feb 3, 2019 03:01

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

JetBrains gradually re-enabling all the IntelliJ features :unsmigghh:

# ? Feb 3, 2019 03:22

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Dr Subterfuge posted:

Yeah actually Thermopyle could you explain what's going on in that code?

Protocols are what you might call 'static duck typing'. If the interface of an object matches the Protocol, then its considered the same type.

Usually, for a type to match, it has to inherit from the class in the type hint. But with a Protocol type, it just has to implement the methods you specify. Just like you're used to with regular, un-type-hinted python. When you're writing a function, you don't care if someone calls you with (for example) a list as an argument, you just care that it acts like a list when you iterate it or sort it or whatever.

A Protocol lets you do the same thing with type hints. It's a way of saying "I don't care what the actual type is as long as it implements the behavior and state that I expect".

Python code:

from typing import Protocol

class SupportsClose(Protocol):
    def close(self) -> None:
        ...   # note that this is the actual code.  
              # Three periods (aka an ellipsis).  Valid python for indicating an empty function body.  
              # It's the same as using the keyword `pass`, and you could use `pass` if you wanted.
              # Anyway...

If you have another class like this:

Python code:

class Resource:
    def close(self) -> None:
        self.file.close()
        self.lock.release()

You can pass it to functions like this and mypy doesn't bitch about it:

Python code:

def do_a_thing_and_close_a_thing(closable_thing: SupportsClose) -> None:
    doing_things()
    closable_thing.close()

def main():
    resource = Resource()
    do_a_thing_and_close_a_thing(resource)

Note that SupportsClose does not have to inherit from the same base class that Resource does. It is a structural type because it inherits from Protocol and implements the close method that Resource also does.

OK, another thing you need to know.

These two things are equivalent:

Python code:

def i_am_a_function(some_arg):
    print(some_arg)

>>> i_am_a_function('hi')
hi

and

Python code:

class i_am_a_function_maker:
    def __call__(self, some_arg):
        print(some_arg)

i_am_a_function = i_am_a_function_maker()

>>> i_am_a_function('hi')
hi

In other words, an instance of a class that implements the __call__ special method works just like a function does.

And another thing you need to know. When python introduced type hints, one of the provided types was Callable. This works fine for simple cases:

Python code:

class SomeClass:
    ...

def i_am_a_callable(my_first_arg: str, my_second_arg: SomeClass) -> bool:
    # blah blah blah
    return True

def i_take_a_callback(callback_fn: Callable[[str, SomeClass], bool]) -> None:
    """
    `callback_fn` parameter is typed with `Callable` and a signature that matches
    the `i_am_callable` function.  Or any function that has the same signature.
    """
    ...

The problem with Callable is that it doesn't have a way of specifying the type of functions that take *args or **kwargs.

Now there is some info, but I'm kinda tired of typing right now and I want to go play overwatch. Maybe it's enough to understand the code I posted earlier? I can write more tomorrow if anyone wants me to.

# ? Feb 3, 2019 04:32

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Why does it seem like there�s suddenly a drive to static type everything now? I thought not having to do that was one of the core things with python.

# ? Feb 3, 2019 08:10

cinci zoo sniper: Mar 15, 2013

Boris Galerkin posted:

Why does it seem like there�s suddenly a drive to static type everything now? I thought not having to do that was one of the core things with python.

Then Guido worked on a real life Python project.

# ? Feb 3, 2019 08:19

Hadlock: Nov 9, 2004

Boris Galerkin posted:

Why does it seem like there�s suddenly a drive to static type everything now? I thought not having to do that was one of the core things with python.

Any project that's going to be maintained over many years (2+) by more than 2 generations of programming teams, as a practical matter, should be strongly typed. Especially as you get past a low mark like 20K LOC. Otherwise you're wasting hours to weeks trying to support the walls of your jello castle as you keep building them ever higher. It's possible, but it's also frequently turns in to a loving mess.

Typescript is wildly popular because it forces front end developers to adhere to some basic conventions, and static typing makes code moderately maintainable.

Edit: that reminds me, I need to open a ticket to get our eastern european java developers to consolidate this XML config file in to the primary config yaml file

# ? Feb 3, 2019 08:59

Nippashish: Nov 2, 2005; Let me see you dance!

Static typing is exactly as useful as the tools it enables. For example, refactoring tools for Java and C# are much better than for python because a lot more is statically knowable about the runtime behavior of code in those languages.

# ? Feb 3, 2019 10:26

cinci zoo sniper: Mar 15, 2013

Nippashish posted:

Static typing is exactly as useful as the tools it enables. For example, refactoring tools for Java and C# are much better than for python because a lot more is statically knowable about the runtime behavior of code in those languages.

That and at least rudimentary capability of code to document itself - both intrinsically and via every failure being a loud and unmissable compiler-time event.

# ? Feb 3, 2019 10:45

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 10:05

Dr Subterfuge: Aug 31, 2005; TIME TO ROC N' ROLL

punished milkman posted:

Thermopyle posted:

Thank you both for the explanations. I sort of got the "what" of it when I first saw it, but not really the "why" or the "how." Makes more sense now.

# ? Feb 3, 2019 15:14

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »