Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›13 »

tef: May 30, 2004; -> some l-system crap ->

Thermopyle posted:

lol, give the poor guy a break!

Maybe I have no patience because it's been seven years in this goddamn thread

quote:

How many posts do we need to tell him this?

One didn't seem to be enough.

Symbolic Butt posted:

I kind of understand Dominoes because writing self over and over is kind of frustrating when you don't need to do that as much in other languages like C++, Java and Ruby.

When I was a younger, petulant mess of a programmer, I too waltzed into python and the explicit self. I'd done Java, JavaScript, and a handful of terrible languages not even worth mentioning. I didn't like it much, but I'd often used /this.foo/ instead of /foo/ in Java. I liked to be able to see what I was doing. I wasn't so keen on what seemed to be self sneaking into my method arguments. Now I've been using python for a bit, it's probably the best feature of python.

Let's take a step back and look why it's there and what it does for us: Python doesn't have methods: It has functions in classes. This means you can call a method explicitly on an object, ClassName.method(object, args). This also means you can pull the function out and assign it to other classes, ClassOne.foo = ClassTwo.foo. It also means that obj.foo() is actually two operations, method = obj.foo; method(). The choice of making methods out functions gives us unique ways to build classes, and also lets us take a method and pass it around as a function.

This is incredibly useful, and it's obvious to see why when we compare it to other languages with objects.

In Ruby, you don't even have functions, there isn't really a notion of them. A top level def foo ... end is actually defining a private method on Object. I'll let that sink in: there are no functions in ruby. defining a top level method is monkey patching object. You also have to clumsily call method = object.method(:name_of_method), method.call(), to get the same results. Ruby ends up with a plethora of things that look like functions but all behave differently. Lambdas, Blocks, Procs, Methods.

In javascript, when do obj.foo(), and you do method = obj.foo; method() they do two entirely different things. JavaScript programmers must manually bind methods to objects if they want to capture them to pass around. Let us not forget var that = this � this isn't like a variable, nor acts like one, it doesn't obey the scoping rules that other languages have.

In python: methods are functions, and when you lookup obj.foo, it partially applies the function, setting the first argument to obj. self is just like any other argument, and can be captured inside nested function and class definitions. In many ways, it's not explicit self, but lexical self. it's just another variable, which is why you can call it anything you like. Python could chose to have both method and function types, and special self keyword, but we'd be doing this = self to lexically bind it, and also fun doing bind and unbind to play about with methods.

It gives you all of this expressive power and how do you thank it? Trying to rename it from something *every* python programmer calls it, and *every python programmer* has a habit of writing it. Why do you claim to do it? Readability? Optimization? Time saving. You have already saved yourself so much time by using lexical self that one or two keystrokes isn't going to make a project late.

Whining about lexical self and arguing to remove or shorten it for the very reasons lexical self exists. Lexical self is the best goddam thing in python, take a seat, shut the gently caress up, and learn why it's there. I have no loving patience for such foolish behaviour.

# ¿ Nov 18, 2014 19:27

Adbot: ADBOT LOVES YOU

# ¿ May 12, 2024 09:22

tef: May 30, 2004; -> some l-system crap ->

The absolute broken nature of ruby's ersatz functions is that *adding methods to Object* is considered routine, and *returning a class from inside a method, which captures method arguments* is considered *wtf*. They are living in a world where mashing everything into one namespace makes sense and Foo = namedtuple('Foo', 'x y') is crazy talk. If you want to go over to the world of implict self you are free to go but don't bring any of those trash ideas into python. We have already suffered enough.

# ¿ Nov 18, 2014 19:32

tef: May 30, 2004; -> some l-system crap ->

I pray that the next bad idea you have doesn't take five people repeating the same thing to you for you to listen :3:

# ¿ Nov 18, 2014 21:10

tef: May 30, 2004; -> some l-system crap ->

Munkeymon posted:

It's more annoying to write self.property than simply property as you would in most languages with the concept of a class method, for one.

(Notably, most languages doesn't include JavaScript, Ruby, Lua. Ruby has uninheritable singleton class methods, and uh, javascript is prototypical)

So we're talking C# or Java here. Python actually has class methods too.

Jewel posted:

Most other languages, at least that I use, implicitly give "this" to a function in a class, and if python forces you to have a keyword representing "this"/"self" in every member function, why not just provide it via a builtin keyword instead of making you add the argument manually every time?

It has *entirely* different semantics. It makes it hard to know to bind this as a classmethod, staticmethod, or instancemethod. Maybe we do something like JavasScript where obj.method and obj.method() have different semantics, or lua's foo.method and foo:method for binding.

Instead of one way of having lookup, we have two. We'd might have to special case binding too, but maybe the descriptor protocol can be saved.

quote:

And then to go a step further, why not implicitly scope local arguments on the class.

In C# you can tell if a property exists because it is checked at compile time. With ruby, it works on the basis that if you assign to it, it's a local variable, and if you only read from it, it's a method call.

Python on the other hand has lexical scope, and global variables. Like ruby, anything not locally set is treated as an outer value, but it is lexical rather than object scoped. To do x = y in python you will have to check if it's currently being called as a method, or bound to an object, and also make a choice as to wether the outer x or the object x takes precedence.

This isn't easy: say you have a function like so, and you bind it to a class, should x be self.x or the argument?

code:

def foo(x):
  def bar():
     return x
  return bar

And this only gets you so far: Because python implicitly treats variables as local when they are assigned too, you're going to need to distinguish between x = 1 and self.x = 1. Ruby solves this by adding a @ operator (as well as a @@operator (and a few more operators help with this issue too for the other namespaces ruby has)

So to get this to work while preserving python semantics, we have to introduce let x = ... or @x to work out which one is which, as we can't implicitly tell from assignment anymore, or we go full hog and implement static typing. Ruby doesn't have functions or lexical scoping either.

JavaScript is the halfway point between python and ruby's this: Python requires it everywhere, Javascript doesn't need it in the function def, and ruby doesn't have it anywhere.

quote:

personally I think is unnecessary as the only thing it lets you do is have a local and a member variable with the same name which seems dirty).

In a dynamic language what happens if I do obj.x; obj.method() where method has a local variable assignment to x? it wasn't to know that x was created by someone else. Your handwaving boils down to "Why can't I just write ruby or c# in python" and the answer is: If you want these features, you're going to have to use the languages built around them.

Python is a language where the OO has been built atop of functions and containers. The meta object protocol itself is actually quite nice, and as a result doesn't depend on special builtins or features of the language as much as others. Languages built around objects and methods have these implicit self, either via static typing, or via special operators to distinguish which context you are in. You end up with crap functions.

Python gets a lot of power out of the Object model, and foo.x doesn't care if it's a module, a method, an attribute or whatever. Overriding how binding works can be done at the object level with __getattr__ or __setattr__, and at the attribute level with __get__ and __set. Treating objects as containers of functions gives easier composition and simpler language rules.

Ruby by comparison has one syntax for method calls, obj.foo, one syntax for module lookups, obj::foo, another for attribute lookup, @foo, another for class lookup @foo, two ways of defining a method, ordinary and singleton, classes and also modules, which compose differently. No class methods, no functions, but methods, blocks, procs, lambda, all with slightly different semantics.

What you are wanting is to have a little bit of sloppy, implicit code, without having to pay the price of static typing or a plethora of operators to distinguish it. The answer is no, you can't. Stop being a babby.

Jewel posted:

Why did python choose that design philosophy in the first place, actually?

aesthetics. I rather like em.

# ¿ Nov 20, 2014 16:07

tef: May 30, 2004; -> some l-system crap ->

Jewel posted:

Why did python choose that design philosophy in the first place, actually?

http://python-history.blogspot.co.uk/2009/02/adding-support-for-user-defined-classes.html :toot:

# ¿ Nov 20, 2014 16:11

tef: May 30, 2004; -> some l-system crap ->

Ghost of Reagan Past posted:

I'm wondering if abstracting some of this into something more object-oriented would be fruitful, since the biggest issue is that the functions end up requiring between 4-7 arguments to do all the work and the code can be somewhat ugly when that's all floating around (the functions are as separate as they can be at this point).

If you have more than 6 you've probably missed one (Kernighan or Ritchie, probably (maybe Plaugher))

quote:

There are three ways I think this could be done to clean things up: one, throw all the parameters into a class and let the functions pull what they need out. Two, throw all the data structures into a class and pass that around. Three, build an all-encompassing class, instantiate. I don't like the third option that much, since it seems mostly pointless (everything would need to be called the exact same way, except now as an object! WOW!). The second is better, but there are three data structures I use and a single class for them seems gratuitous and dangerous. Finally, the first: I like it, but since different functions need different arguments, the only thing it would do is move the location of the ugliness. Now, there's something to be said for cleaning up the ugliness a bit, but since I don't have any real idea whether object-oriented design would be remotely useful in this case at all, I'm curious if this sounds like a good idea, or if it's just overcomplicating things.

It depends how these functions compose and interact. You may find it easier to take option three: one big rear end class with the state and methods, and a simpler interface on top for what the program does, or at least a simpler api to call and interact with it. From there you may find it easier to slowly clump things together.

quote:

Basically, I'm asking a more general question about designing Python applications: should I be wary of throwing around objects when it's not clear to me (an amateur) that there's a very good reason (I'm not sure slightly deuglifying code is a very good reason, but I could be wrong!)? And what are some ways to tell that there's a good reason for throwing around objects (beyond the obvious cases like representing objects and their properties)?

The reason other languages (Java, Ruby especially) use objects all the goddam time is because they have no other choice. Instead of a function do_thing we have a tyranny of ThingDoer objects with a do_thing method. Don't feel you have to use classes to get things done in python, but there are some good reasons to use classes:

To hide a decision that you might want to change in future, to hide a difficult mechanism (eg requests vs urllib2)

To spread out the moving parts, so they don't touch each other (but this requires care, it's still easy to have all the methods in different files but yet touching all the same datastructures at runtime)

To pull out some feature into stand alone code.

The code shouldn't try to be more elegant than the problem it solves. Many problems are ugly, clunky, and clumsy, and it's not unheard of for some numerical computation methods to be pages upon pages of work (See also automation scripting). Sometimes code is best left to be ugly and rudimentary over stylish with a veneer of elegance.

# ¿ Aug 29, 2015 03:11

tef: May 30, 2004; -> some l-system crap ->

Maybe they're using a five year old python due to some legacy constraints?

# ¿ Aug 29, 2015 22:34

tef: May 30, 2004; -> some l-system crap ->

The code looks reasonably ok for the interface you've been given. i'd probaby do a few things differently.

Take a Grid class, which has open, is_open, close, is_close, open_random, and neighbours methods, handling the virtual top and bottom row.

The percolator would take a Grid, along with your DisjointSet (QuickUnion), and expose __init__, step() and percolates methods. The step method would be a cleaned up version of open_and_union.

code:

p = self.grid.open_random()
for n in self.grid.neighbours(p): 
    if self.grid.is_open(n):
        self.disjoint_set.union(n, p)

Then i'd have a simulator function that returned a stats object

code:

def simulate(percolator, times):
    s = Stats()
    for n in range(times):
        p = percolator()
        while not p.percolated():
            p.step()
        s.add_sample(p.steps)
    return s

tef fucked around with this message at 03:55 on Aug 31, 2015

# ¿ Aug 31, 2015 03:50

tef: May 30, 2004; -> some l-system crap ->

duck monster posted:

I still havent evolved from 2.7 yet. I probably should update my thinking. async seems to be the selling point to my line of thinking.

As to why? Lotta legacy, and I've been hacking on python since the late 90s and old dogs/new tricks/etc

3.5 for me is the point where using python 3 and rewriting in python 3 seem like a good choice. I dunno if this is true for other people (i do unix plumbing stuff bolting api calls to syscalls).

It's not just the asyncio stuff, it's also the performance improvements over 2.7, and little things like bytestring formatting operations. The third party libraries i rely on are there too! venvs and pips are finally part of the batteries.

# ¿ Sep 15, 2015 02:58

tef: May 30, 2004; -> some l-system crap ->

qntm posted:

I'm working on a library for finite state machines. When you make a finite state machine you have to supply an alphabet of symbols, which in this case is a set of hashable values (they get used as keys in a dict). But I would also like there to be a special "anything else" value which you can add to your alphabet. So if your alphabet is {"a", "b", "c", "d", fsm.anything_else}, and you pass "e" into your FSM, then the library treats that as "anything else" and follows the transition you selected for that special value.

Problem is, what should I set the value fsm.anything_else to? I can't set it to "e" because that might be part of the user's chosen alphabet. I can't set it to None for the same reason. In fact the alphabet could in theory legitimately contain any hashable value. Users don't care, of course, unless there's a clash, because they'll use the symbol, not the value. Is the best approach really to just use a very large integer which nobody is ever likely to run into by accident?

foo = object()

>>> foo = object()
>>> bar = object()
>>> foo == bar
False
>>> foo is bar
False
>>> any(foo == x for x in [1,2,3,"a","b","c", None, bar])
False

# ¿ Sep 17, 2015 14:17

tef: May 30, 2004; -> some l-system crap ->

FoiledAgain posted:

How can the super() called get "nullified" like this? Am I using super() incorrectly?

super takes args in 2.x: https://docs.python.org/2/library/functions.html#super

in the current version of python, you can use the no-arg form: https://docs.python.org/3/library/functions.html#super "The zero argument form only works inside a class definition, as the compiler fills in the necessary details to correctly retrieve the class being defined, as well as accessing the current instance for ordinary methods."

my advice is to do SuperClass.__init__(self) explicitly and just avoid super altogether unless you're doing multiple inheritance

# ¿ Sep 20, 2015 21:08

tef: May 30, 2004; -> some l-system crap ->

mekkanare posted:

My questions:
Is there such a thing as flushing a TCP buffer? If so, how do I do it for Python?

You could try

code:

f = connectionSoc.makefile(); f.readline()

# ¿ Feb 28, 2016 00:28

tef: May 30, 2004; -> some l-system crap ->

the talent deficit posted:

i know python hates functional programming, but is there really no filtermap in the standard library?

x = [f(y) for y in c if y < 2] ?

# ¿ May 17, 2016 01:53

tef: May 30, 2004; -> some l-system crap ->

i can tell you a few unhelpful lies, but the nature objects is finicky when you consider EOL'd python (2.7 and below), so i'll cover how objects work in python 3 and new style objects in python 2. just use class Foo(object): in python 2, don't ask.

so, in python we have dictionaries.

we can get from them: del d["key"], we can set them d["key"]=1 and we can get them d["key"]

objects sorta work the same way, they're kinda like dictionaries, except we use foo.bar instead of foo["bar"], and python does *magic* behind the scenes when you lookup an object's attribute

code:

>>> class Example:
...     pass
... 
>>> e = Example()
>>> e.a = 1
>>> e.a
1
>>> del e.a
>>> e.a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Example' object has no attribute 'a'
>>>

we can assign functions to instances too:

code:

>>> def same(x):
...     return x
... 
>>> e.same = same
>>> e.same(1)
1

although I said foo.bar does magic in python, it doesn't really do it for instances.

instances are really just a wrapper around a class, an a dictionary for attributes

code:

>> e.__class__
<class '__main__.Example'>
>>> e.__dict__
{'same': <function same at 0x104b33730>}
>>>

for instances, foo.bar is really foo.__dict__['bar']

if python can't find 'bar' in the instance dictionary, it looks it up in the class instead, but classes are where the magic starts to happen for foo.bar

code:

>>> class Example:
...     a = 1
... 
>>> Example.a
1
>>> Example.__dict__
mappingproxy({..... some poo poo that's too long but it's a dict ok ? .... })
>>> e = Example()
>>> e.a
1
>>> e.a = 2 # We'd want to do Example.a = 2 to set that one
>>> e.__dict__
{'a': 2}
>>> e.a
2
>>> Example.a
1
>>>

in python, classes are just templates for instances, which contain functions.

code:

>>> class Another:
...    def example(self): 
...        return "nice"
... 
>>> a = Another()
>>> a.example()
'nice'
>>> Another.example
<function Another.example at 0x104b33840>
>>> def example2(self, a):
...     return a
... 
>>> Another.example2 = example2
>>> a.example2(1)
1

when you call foo.bar, if python finds a bar function in the class, it binds it - it returns a new function, calling the original, but passing foo as self.

you can think of foo.bar being partially appliying foo.__class__.bar(self=foo), if you want to, i guess

code:

>>> Another.example2
<function example2 at 0x104b338c8>
>>> a.example2
<bound method Another.example2 of <__main__.Another object at 0x104bc1fd0>>
>>>

this is actually a huge difference between python and many of it's peers: in ruby, you always call a method, in javascript, if you lookup the method you must bind it manually

there is no special this or self keyword, python's self is just named thay by convention, and it's a lexically scoped variable too, so no more "var that=this" as seen in other languages

recap:

so, objects all wrap some dictionary (o.__dict__) which stores attributes.

when you call foo.bar, you're looking up bar inside foo's dictionary

when you lookup an attribute of a class directly, or lookup something inside an instance, no real magic is performed

classes are just wrapped dictionaries of functions and values too.

when an instance calls up to the class to find an attribute, it perfoms binding, turning the function into a bound method - where the first argument is the instance itself

i'm too tired and i dunno how accurate my memory is but from here you can start to explore the metaobject protocol: a class with __getattr__/__setattr__ methods can start to change the behaviour of `foo.bar`

it is worth shouting out to descriptors, which allow you to overwrite foo.bar by setting methods on bar, not on foo.

# ¿ Jul 16, 2016 02:14

tef: May 30, 2004; -> some l-system crap ->

there are plenty of places where this is not true but it's a reasonable set of lies to go on i hope

# ¿ Jul 16, 2016 02:16

tef: May 30, 2004; -> some l-system crap ->

there's a reason i didn't cover super.

# ¿ Jul 16, 2016 02:40

tef: May 30, 2004; -> some l-system crap ->

The March Hare posted:

Mock question!

If I have something like:
Python code:
# utils.py
def some_func():
    foo = db_call_to_get_an_instance_of_some_class()
    bar = foo.method(baz)

    for thing in bar:
        do_some_thing()
I want to test this function without hitting the db, so I'm currently just patching the return value of db_call_to_get_an_instance_of_some_class to be a Mock() object, but then how do I make the foo.method(baz) return an iterable for the purposes of the later for thing in bar?

unless you're using magic wizard sql, you could always test against sqlite? you've kinda already given up on testing on the db you use in production, but using sqlite might give you a much more useful mock object than one you can write yourself :v:

# ¿ Jul 22, 2016 06:57

tef: May 30, 2004; -> some l-system crap ->

UberJumper posted:

Can someone tell me what is the preferred practice for doing imports inside a package for third party packages?

it depends :eng101:

quote:

# Foo/bar.py

code:

import moo

def do_something():
    return moo.cat(....)

This is good when you're doing something a lot with moo, or cat isn't a clear function

quote:

Or doing this?

Foo/utils.py
code:
from moo import cat 

This is good when you're only doing cat and it's an obvious name: from urlparse import urlparse

If you're writing a standalone executable script, from x import * is acceptable, but rarely otherwise.

quote:

Foo/bar.py
code:
from .utils import cat

def do_something():
    return cat(....)
Basically hiding usages of the third party library moo in utils, or using the third party moo explicitly. Some people at work say the first one others say the second one. We already have a pretty big utils sub package that is more or less imported throughout the project.

`
This is good when you might change the moo library later. It's heavyweight when you won't do it. It might not be worth wrapping pytz or requests, but if the API is ugly, a wrapper gives you the chance to make a smaller easier api for the things you need without coupling your code.

It's all a tradeoff.

I tend to

- wrap any big 3rd party module, especially one i'm trying out for size. Often just to set up defaults.
- use import foo over from foo import bar

but usually when i'm writing larger chunks of code, for cheap hacks i'll do whatever

# ¿ Aug 23, 2016 17:34

tef: May 30, 2004; -> some l-system crap ->

UberJumper posted:

Thanks!

I am not really wrapping the moo library (it is actually a fairly nice library, just has horrible method names). There is just a bunch of really cryptic method names, that are not upto the PEP8 standard (e.g. 'g_cfg_l' which literally means load_config).

It is also a library that almost every single module in our entire package (probably 100+ files), imports. So i have been debating just putting a bunch of "from moo import g_cfg_l as load_config" into utils/__init__.py then in our code just use:
code:
from .utils import load_config

never have a file called utils

put it in a file called "moo_wrapper" or "mootools" or "mooutils"

never have a file called utils

it's a broken window and it will attract detritus

# ¿ Aug 24, 2016 11:11

tef: May 30, 2004; -> some l-system crap ->

LochNessMonster posted:

Not sure if this belongs here, but I'm starting to learn programming with Python and am trying to build a webscraper that stores information in some sort of a database.

Planning to scrape like 10 different small websites with items and a handfull of attributes.

What database would be easy to use with Python? I was thinking something like CouchDB, or SQLite.

i'd recommend sqlite, but i'd also suggest maybe peewee, depending how much sql you want to pick up

ps, i also like lxml, requests, you might too.

# ¿ Sep 3, 2016 13:48

Adbot: ADBOT LOVES YOU

# ¿ May 12, 2024 09:22

tef: May 30, 2004; -> some l-system crap ->

As much as i'd like to retire this thread, well, it's almost a decade now, but it's been a long decade.

king_kilr posted:

I got started with Python a few months a go, I've been using it for exclusively web stuff so far, using Django. I really like it so far, I would say the best part is code is so readable.

# ¿ Mar 2, 2017 04:42

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›13 »