Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Eela6: May 25, 2007; Shredded Hen

There are two main reasons for @property.

One is when you want to present a uniform API to your users. If you have a single method that takes only self and will always return the same result for a given object, it's "really" a property. For example, you might lazily compute the perimeter of a 2d shape when asked for, but the users don't need to know whether it's computed during __init__ or when you ask.

Plus, if we only implement a 'getter' for perimeter and not a 'setter', our code won't let us create a rectangle where
P! = 2(WH) well, within the limits of Python floats, anyways...

This helps makes some guarantees about our rectangles being actual rectangles.

The second is as a step in refactoring. Suppose we have a class Frisbee. You initially had a property, Frisbee.curve that was a public attribute. Later on, you implemented an advanced physics and materials simulation; it models the deformation of the Frisbee during its flight! That is, Frisbee.curve is no longer a static property - it can change over time! Clearly, Frisbee.curve needs to be a method that works with your physics simulation. The problem is, all over your codebase and your customers' codebases, there are calls to some_frisbee.curve, not some_frisbee.curve().

Using @property makes the change "invisible" so that the old calling signature will still work.

Excessive use of @property is a warning sign of brittle code, though. At that point you might want to start raising warnings on old API usage that tell the user about your new API that more closely suits how your new code actually works.

if you want to know more, there are sections in 'Effective Python', and a speech by Brett Slatkkn, the author of that book, about this.

Eela6 fucked around with this message at 17:30 on Mar 8, 2017

# ¿ Mar 8, 2017 17:15

Adbot: ADBOT LOVES YOU

# ¿ Apr 29, 2024 12:00

Eela6: May 25, 2007; Shredded Hen

I wrote something pretty fun today. I got sick of writing __repr__ for functions, so I made a decorator that does it for me!

Python code:

def easyrepr(cls):
    """class decorator. tries to create self-documenting __repr__. 
    such thast, where possible for immutable objects with 
    implemented __equals__, 
    eval(repr(x)) == x 
    
    """
    _cls_new = cls.__new__
    def _repr(self):
        return self._easyrepr
    def _new(cls, *args, **kwargs):
        instance = _cls_new(cls)
        argstr = (f'{arg!r}' for arg in args)
        kwargstr = (f'{arg} = {kwargs[arg]!r}' for arg in kwargs)
        args = ', '.join(chain(argstr, kwargstr))
        instance._easyrepr = f'{cls.__name__}({args})'
        instance.__repr__ = _repr
        return instance
    cls.__new__ = _new
    cls.__repr__ = _repr
    return cls


if __name__ == '__main__':
    @easyrepr
    class TestClass():
        def __init__(self, foo, *, bar):
            self.foo, self.bar = foo, bar
        def __eq__(self, other):
            return self.foo, self.bar == other.foo, other.bar
    a = TestClass('foo', bar = 'bar')
    b = TestClass('spam', bar = 'eggs')
    c = TestClass(a, bar = b)
    assert repr(c) == "TestClass(TestClass('foo', bar = 'bar'), bar = TestClass('spam', bar = 'eggs'))"
    assert c == eval(repr(c))

# ¿ Mar 10, 2017 02:02

Eela6: May 25, 2007; Shredded Hen

That's the way I tried to do it at first, but it wasn't inherited properly through subclassing, which bugged me. You get the name (and signature) of the base decorated class.

E.G:

Python code:

>>>class SubClassReprTester(ReprTester):
    pass
>>>print(SubClassReprTester('foo', 'bar', c=1))
ReprTester('foo', 'bar', c=1)

Obviously you can fix this by decorating the subclass with repr_as_initialized.

Eela6 fucked around with this message at 04:35 on Mar 10, 2017

# ¿ Mar 10, 2017 04:31

Eela6: May 25, 2007; Shredded Hen

Why not just have two attributes? country.name being the default, and country.name_in_lang being a dictionary that holds the other names? I think you're overthinking this.

# ¿ Mar 10, 2017 09:04

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

Because it was just an example.

e: And also because I come from a procedural/imperative programming background (my first language was Fortran back when it was still FORTRAN [77]) and I've never learned anything else other than modern versions of Fortran and MATLAB so honestly I have no clue what the point of objects are or when and why I should use them.

I just think it's pretty neat to be able to do:
code:
new_tensor = old_tensor.rotate()
Instead of
code:
new_tensor = rotate_tensor(old_tensor, rotation_matrix)
But honestly the only reason I think the first one is neat is because it looks better to me. I really wasn't joking when I said I liked "foo.thing" better than "foo.thing()" because it looks better without the parenthesis (but mostly because I don't think it's semantically correct to call a function to return a value that I've already computed).

I too came from MATLAB, where you can call a zero-argument function without the parentheses.

The reason why you can't (well, shouldn't) in python is because Python has functions as first class objects. That means you can put functions into containers, have functions with extra attributes (that could be functions themselves), all sorts of weird stuff. This is powerful, but it means you need a way to refer to the function itself as an object.

When you use the parentheses you are referring to the action of the function.

When you don't use the parentheses you refer to the function itself. This is a crucial distinction!

Here's an example that might help.

IN:

Python code:

def print_0():
    print(0)
def print_1():
    print(1)
def print_2():
    print(2)
    
function_list = [print_0, print_1, print_2]

for func in function_list:
    func.extra_attribute = 'hello, I am '+func.__name__
    print(func.extra_attribute)
    print(func)
    func()

OUT

code:

hello, I am print_0
<function print_0 at 0x000001E71F2AB400>
0
hello, I am print_1
<function print_1 at 0x000001E71F2ABE18>
1
hello, I am print_2
<function print_2 at 0x000001E71F2AB378>
2

If you write Python, people have expectations about 'how' your code works. You can break this with weird metaprogramming and operator overloading - Python gives you a lot of power. But this is strongly discouraged! If everyone write Python in the 'normal', idiomatic way, the community is more easily able to use and reason about each others' code.

Other languages do not have this philosophy. I am of the opinion it is one of the best things about Python, but you may disagree. That's OK; just be warned the majority of the community will not be on your side.

The Zen Of Python posted:

Explicit is better than implicit.
Special cases aren't special enough to break the rules.

Eela6 fucked around with this message at 22:00 on Mar 10, 2017

# ¿ Mar 10, 2017 21:48

Eela6: May 25, 2007; Shredded Hen

Elm is a neat language! I'm not sure if I love it but there are a lot of interesting ideas there.

# ¿ Mar 11, 2017 18:03

Eela6: May 25, 2007; Shredded Hen

Whenever you call a function, it does some action, then returns an object.

The function sorted() takes an iterable* and returns a sorted list of the contents. This is a new list; not the old one.

list.sort(), on the other hand, is a method of a list (a special kind of function, attached to an object), that sorts it without making a new copy. To remind programmers of the difference, it returns None.

Try this

code:


a_list = [3, 1, 2]
b_list = sorted(a_list)
print(a_list)
print(b_list)
print(a_list.sort())
print(a_list)
a_list.append(4)
print(a_list)
print(b_list)

What are the results? Are they what you expected?

*An iterable is an abstraction of anything that can give you a 'flow' of objects, one at a time. Tuples, lists, sets, and dictionaries are all iterable. For now, all you need to know is anything that supports

code:

for x in iterable:
    #code

is iterable.

# ¿ Mar 12, 2017 04:14

Eela6: May 25, 2007; Shredded Hen

Thermopyle posted:

There's a not-entirely-unjustified school of thought in python-land that too many developers pull out classes unnecessarily.

I am of this opinion.

This is a good video on the subject:
[video type=""]https://m.youtube.com/watch?v=o9pEzgHorH0[/video]

Classes are useful when they make your code easier to reason about. Too many classes are often a signal of code without forethought.

# ¿ Mar 12, 2017 20:44

Eela6: May 25, 2007; Shredded Hen

Finding the correct level of abstraction is like tuning a guitar string by ear. The correct tuning depends on the pitch of the strings around it, and large changes might require re-tuning of the surrounding strings.

Similarly, there is no uniform level of abstraction that is 'always correct' for a particular concept. Whether or not a particular piece of code is correctly abstracted is not just a function of the code itself, but the larger program as a whole.

# ¿ Mar 14, 2017 19:37

Eela6: May 25, 2007; Shredded Hen

For now, if you are relying on the order, use OrderedDict all the way through. In CPython 3.6 it's just the new dict implementation under the hood, so you lose nothing, but you keep back-and-forward compatibility.

(I will never go back! f-strings are too useful.)

# ¿ Mar 15, 2017 00:59

Eela6: May 25, 2007; Shredded Hen

Cingulate posted:

I often do something like this:
code:
parameter = 999
while does_not_match_some_condition(parameter):
    parameter = stochastic_process()

do_things_with_parameter(parameter)
I.e., to find a good value for my parameter (might be a string, a number, a sequence ...), I use a while loop to try a few values for my parameter, until I have one that matches some conditions. Then, I use that parameter for some purpose.

Specifying the parameter to some arbitrary value before the while loop feels inelegant. Is there a better way for doing this?

A more fleshed-out example, maybe I want to randomly draw an even integer between 140 and 75858 whose third-to-last digit is a 4 or a 7 for whatever reason:
code:
from random import randint
my_number = -1

while my_number % 1 and str(my_number)[-3] in {'4', '7'}:
    my_number = randint(140, 75859)

print(my_number)
(Ignore that this is probably a very inefficient solution for solving this specific task.)

I guess this is a very minor thing, but it's a pattern I use all the time and it feels weird.

Python lacks the construct DO -> WHILE, so you have to figure out what makes sense given your level of abstraction. Personally, since this is a single construct ('get a number that satisfies this condition'), I would abstract it out to a function. This also lets you skip the 'pointless' initialization of your parameter that you find inelegant. This is similar to SurgicalOntologist's suggestion of using break.

I just like functions.

Python code:

def get_parameter_that_satisfies_conditions():
    while True:
        n = randint(140, 75858)
        if n % 2 == 0 and ((n % 1000) // 100) in {4, 7}:
            return n
parameter = get_parameter_that_satisfies_conditions()
do_things_with(parameter)

Eela6 fucked around with this message at 19:26 on Mar 15, 2017

# ¿ Mar 15, 2017 19:23

Eela6: May 25, 2007; Shredded Hen

SurgicalOntologist posted:

Not sure if I should put this here or in the Scientific Computing thread, but whatever.

I have a data analysis project that involves computing a series of variables on a dataset (specifically, I'm adding DataArrays to an xarray DataSet, but it's the same idea as if I were adding columns to a DataFrame). Each calculation requires some variables to already be available and adds one or more variables to the dataset. Some of these are expensive to calculate and may not be necessary to run in all circumstances so I don't want to run everything. In short, I want a way to handle this dependency graph.

This seems like it should be a common pattern but I don't know of an existing easy solution. Closest thing I can think of is dask's DAG compute engine but this doesn't really fit into that model since I'm enriching the dataset object rather than passing around values.

If there's no existing solution the the best option I can think of is to have something like a require method call at the top of every function that checks if the required elements are present and then runs the corresponding function if they are not. Any better ideas?

You want to implement lazy attributes - only calculated if they don't already exist.

Here's an implementation, taken from Python Cookbook, 3rd Ed by David Beazley and Brian K. Jones, pg.267-278. (The example class is my own.)

This works because a __get__() method associated with an attribute is only called if the attribute being accessed is not in the underlying instance dictionary.

Python code:

from time import time

class LazyProperty():
    def __init__(self, func):
        self.func = func
    def __get__(self, instance, cls):
        if instance is None:
            return self
        else:
            value = self.func(instance)
            setattr(instance, self.func.__name__, value)
            return value

class Divisors():
    def __init__(self, n):
        self._n = n

    @LazyProperty
    def divisors(self):
        print('calculating divisors')
        return [x for x in range(1, self._n) if self._n % x  == 0]
        #intentionally slow
        
if __name__ == '__main__':
    
   D = Divisors(500000)
   print('calling D.Divisors')
   t0 = time()
   a = D.divisors
   elapsed = time()-t0
   print(f'first access ran in {elapsed}')
   print('calling D.divisors')
   t0 = time()
   b = D.divisors
   elapsed = time()-t0
   print(f'second access ran in {elapsed}')

OUT

code:

calling D.Divisors
calculating divisors
first access ran in 0.054593801498413086
calling D.divisors
second access ran in 0.0

Eela6 fucked around with this message at 00:00 on Mar 17, 2017

# ¿ Mar 16, 2017 23:45

Eela6: May 25, 2007; Shredded Hen

I agree with Thermopyle. @classmethod is the way to make alternative constructors. If your method could be described as

Python code:

cls.from_bytes
cls.from_str
cls.from_some_other_format

It's a good place for @classmethod.

Generally speaking, overriding __new__ is deeper magic than you need to solve a problem in python. However, it is very fun! This happens most often when you want to create new classes 'on demand'. I don't know if your problem requires this, but if you want to get into runtime construction of classes, you should look into Metaclasses.

Python Classes make Instances
- > Python Metaclasses make Classes

I am not confident enough in my python magic to give a good example, but both Fluent Python and the Python Cookbook, 3rd. Ed, have great tutorials.

(I'm sure they exist online, but I don't know them well enough to suggest one).

As a final point, your override of __new__ won't work in the first place. It is a class method that implicitly takes cls as the first argument. The signature is as follows:

Python code:

def __new__(cls, *args, **kwargs):

Eela6 fucked around with this message at 18:20 on Mar 18, 2017

# ¿ Mar 18, 2017 18:09

Eela6: May 25, 2007; Shredded Hen

The Gunslinger posted:

I decided to pick up Python, I have a fair amount of experience with Perl and C++ but want to update my skillset a bit. My laptop that runs Linux is having hardware problems so I decided to fart around on my Windows gaming PC with this. I'm working through a tutorial book but just encountered an issue.

They want you to mess around with turtle which is part of the standard library. If I call my script or invoke Python in a command shell from my base user directory (C:\users\TGS) I can import turtle and work with it just fine. If I try to import it from a folder on my desktop, I can't. I just get an unknown attribute error since it can't find the library. I've checked the PATH and the installation directory for Python is there so uh, anyone know what gives?

This is a bit strange. How did you install Python? If you're a beginner, especially on windows, I would try using Anaconda.

As mentioned, Spyder is a great beginner's IDE. I personally use Visual Studio Code

# ¿ Mar 21, 2017 04:32

Eela6: May 25, 2007; Shredded Hen

As a small note, this part of your code:

Python code:

csvFile = open("test.csv", 'a')
try:
    writer = csv.writer(csvFile)
    writer.writerow((ad_title, date_posted, body, mapaddress, lat, lon))
finally:
    csvFile.close()

Is best expressed in idomatic python with a context manager ('with statement'), as follows:

Python code:

with open('test.csv', 'a') as file:
	writer = csv.writer(file)
	writer.writerow((ad_title, date_posted, body, mapaddress, lat, lon))

The file will automatically be closed when the interpreter exits the scope of the with block for any reason, including raising an error.

edit: EFB

# ¿ Mar 23, 2017 21:47

Eela6: May 25, 2007; Shredded Hen

^^ No idea, but I would love to know the answer.

I gave my second talk at San Diego Python last night! It went really well.

Who in the thread is going to PyCon this year? It might be fun to have a goon Python dev lunch.

# ¿ Mar 24, 2017 18:03

Eela6: May 25, 2007; Shredded Hen

tricksnake posted:

Hi I am a new python user just now working my way through Learn Python The Hard Way

I'm wondering what a good framework for python is? Especially for beginners if that's a thing. I have 2.x and 3.x installed but I'm reading in a lot of places that 3.x is the way to go so I'm gonna need a framework that's for 3.x work.

edit: Crap... is Atom text editor what I'm looking for? It says to download it but that's just a text editor. I'm guessing all I need is that at this point in my quest for programming skills.

I would recommend downloading Anaconda. It comes with a lightweight IDE, Spyder, that I find excellent for beginners to-intermediate programmers. In addition, Anaconda includes many of the most useful packahes and dependencies 'in-box', some of which can be frustrating to install otherwise. It has decent autocomplete and will warn you about syntax errors, and it has a very easy REPL work flow.

Other good choices for IDEs are PyCharm and Visual Studio Code, but you might find them a little heavyweight for what you're doing right now.

Good luck with Python! I hope you enjoy it

# ¿ Mar 28, 2017 02:52

Eela6: May 25, 2007; Shredded Hen

huhu posted:

Would this be the best way to log errors on a script that I'm running?

code:

try:
    # Threading here for 3 different functions
except Exception as e:     
    logf = open("log.txt", "a")
    logf.write("Error: {} \n".format(str(e)))
    logf.close()
finally:
    pass

Python has a standard logging module .

However, if you're just doing something very basic, this is the 'pythonic' way to do what you're asking.

Python code:

try:
    #threading
except Exception as err:
    with open('log.txt', 'a') as log:
        print(f'Error: {err}', file=log)

This is syntatically equivalent to the following:

Python code:

try:
    #threading
except Exception as err:
    logf = open('log.txt', 'a')
    logf.write('Error: {}\n'.format(str(e)))
finally:
    logf.close()

If you are going to use try/except, it's important to have the closure of the file be in the finally clause; the way you have it will leave the file open if an error occurs during logf.write(). A context manager ('with') is a cleaner way to handle file IO and the current preferred idiom.

# ¿ Mar 28, 2017 20:49

Eela6: May 25, 2007; Shredded Hen

Indeed I did. Thank you.

Edit: this is actually a great example of why to use 'with' over manual open/close: even with experience it's easy to gently caress it up doing it manually.

Eela6 fucked around with this message at 21:41 on Mar 28, 2017

# ¿ Mar 28, 2017 21:33

Eela6: May 25, 2007; Shredded Hen

Why not? I'd love to know ~~more~~ anything about flask

# ¿ Mar 30, 2017 06:23

Eela6: May 25, 2007; Shredded Hen

oliveoil posted:

Is there anything like the JVM spec but for Python?

I want to know what I can assume about the way Python works. I want to know what's the memory overhead for creating an object, for example. How many bits are in an integer? And so on. I don't want to cobble together answers from StackOverflow that maybe describe the behavior of a specific Python implementation that isn't necessarily fixed, for example.

Also, I'd like to separately (so that I know what is prescribed for all Python implementations, and what is specific to or just an implementation detail of the most popular implementation) about the internals of the most popular Python implementation. I'm guessing that's CPython, but I imagine 2.7 and 3.blah are pretty different. In general, though, I want to know stuff like how does garbage collection work in CPython? What happens when I create a thread? How do locks work? Java has a bunch of explanation about "happens-before" relationships when writing multi-threaded programs - what's the Python equivalent?

CPython is the standard. The best place to start is probably the Python/C API Reference Manual

Garbage Collection is handled by reference counting. Objects whose live references hit 0 are immediately. Cpython countains an occasional cyclic garbage detector; while the running of the garbage collection can be forced, outside of that it's behavior is not (meant to be) predictable.

Python is more about knowing how things behave than 'what they are'. EG, All integers are implemented as �long� integer objects of arbitrary size.

Python Doubles behave as IEEE 64-bit doubles, full stop.

If you want to know more about Python's data model and how things work behind the scenes, I recommend Fluent Python, as I do to everyone.

# ¿ Apr 1, 2017 20:30

Eela6: May 25, 2007; Shredded Hen

OnceIWasAnOstrich posted:

numpy for the array and matplotlib imshow() to render?

A uint8 array of dimensions [m, n, 3] works well to represent images on a pixel level. The library skimage (installed by default in anaconda) works well for basic image manipulation as well. E.G:

Python code:

import numpy as np
from skimage.io import imshow
RED, GREEN, BLUE = 0, 1, 2
img = np.zeros([100, 100, 3], dtype='uint8')
img[0:30, :, RED] = 255
img[31:61, :, GREEN] = 255
img[62:, :, BLUE] = 255
imshow(img)

[timg]http://i.imgur.com/bccSZGh.png][/timg]

Eela6 fucked around with this message at 16:57 on Apr 8, 2017

# ¿ Apr 8, 2017 16:54

Eela6: May 25, 2007; Shredded Hen

I mostly use visual studio code as well. I don't do any webdev so I can't comment on its use for that.

# ¿ Apr 12, 2017 21:36

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

https://github.com/python/cpython/blob/3.6/Lib/collections/__init__.py
code:
    def __init__(*args, **kwds):

        [snip]

        self, *args = args
Is there a reason ever to write code this way like the init method of OrderedDict? Why wouldn't you explicitly write out "self" as the first argument?

I have no idea. This is really strange.

# ¿ Apr 15, 2017 23:40

Eela6: May 25, 2007; Shredded Hen

The first rule of namedtuple is 'don't look at the implementation of namedtuple'

# ¿ Apr 16, 2017 19:10

Eela6: May 25, 2007; Shredded Hen

Thermopyle posted:

IMO, typing.NamedTuple is also better than collections.namedtuple with no extra dependency. I haven't used attrs, though.

Thanks for pointing out typing.NamedTuple! It has a really cool syntax with support for type hints:

Python code:

# python 3.6 and up ->
class Employee(typing.NamedTuple):
    name: str
    id: int

# equivalent to
Employee = collections.namedtuple('Employee', ['name', 'id'])

Eela6 fucked around with this message at 03:04 on Apr 17, 2017

# ¿ Apr 17, 2017 01:39

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

I need to be able to inject a function into a class when the object gets created

Why do you need to?

Metaclasses are the preferred way of customizing classes, but most of the time you think you need them (or the even deeper magic of method injection) they are not necessary.

To quote Tim Peters

Tim Peters posted:

[Metaclasses] are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why)

Eela6 fucked around with this message at 05:06 on Apr 18, 2017

# ¿ Apr 18, 2017 05:04

Eela6: May 25, 2007; Shredded Hen

You don't need to do any special magic to create a method and attach it to an object. So long as the first parameter is self it should 'just work'. Phoneposting so i cant go into more detail, but if you want an example I do so in my easyrepr implementation from earlier in this thread.

# ¿ Apr 18, 2017 19:00

Eela6: May 25, 2007; Shredded Hen

SurgicalOntologist posted:

That's what I thought would happen (it would get self passed automatically), but I just checked and Boris is right, it doesn't. Huh. This is the first time in a while I've been surprised by Python. I guess you do need to do that thing with the types module (or use the first method I suggested).

Edit: Eela6, IIRC about your easyrepr thing, it attaches to the class rather than an instance. In that case it works as expected.

Ah, I see! Thank you for the correction. It's a technical point but an important one.

# ¿ Apr 18, 2017 20:48

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

Thanks for all the help and discussion. I don't want to sound dismissive, but it looks like the original method I proposed works and Dex's solution is more or less the same, so I'm just going to stick with what I have.

You took a good look at alternative suggestions and then made an informed decision. That's not dismissive in the least. You brought up an interesting question in an area that's relatively niche/complex within python. If nothing else, I learned a lot from the discussion!

Just because Metaclasses and method injection are the wrong solution most of the time doesn't mean they're the wrong solution all of the time.

# ¿ Apr 20, 2017 17:52

Eela6: May 25, 2007; Shredded Hen

Epsilon Plus posted:

I've tried to figure out super() in Python a few times and everything I look at makes no goddamn sense; do you ? have any good resources covering it? I don't mind doing regular old DI for the project I'm doing in Python it would be nice to properly understand how super() works.

Super() considered super by Raymond Hettinger is a good resource.

Raymond did this as talk at PyCon a couple years back so it's available as a video presentation too. Raymond is a fantastic presenter.

As always, Fluent Python covers this in some detail.

# ¿ Apr 21, 2017 02:40

Eela6: May 25, 2007; Shredded Hen

The best thing you can do to get basic fluency is to take a project you've written before in another language and re-inplement it in Python.

# ¿ Apr 22, 2017 22:57

Eela6: May 25, 2007; Shredded Hen

baka kaba posted:

pathlib is cool, Path.iterdir() is not because it's not sorted. Is there a natural sort I'm missing (where 'file2' < 'file10') or do I need to screw around parsing numbers from filenames here

pathlib Paths are hashable and orderable (and therefore sortable) so you can just call sorted!

EX:

code:


 Directory of C:\exampledir

04/24/2017  11:44 AM    <DIR>          .
04/24/2017  11:44 AM    <DIR>          ..
04/24/2017  11:44 AM                 0 barfile.txt
04/24/2017  11:37 AM    <DIR>          definitely_not_porn
04/24/2017  11:37 AM    <DIR>          folder_0
04/24/2017  11:37 AM    <DIR>          folder_1
04/24/2017  11:44 AM                 0 foofile.txt
04/24/2017  11:37 AM    <DIR>          not_porn
               2 File(s)              0 bytes
               6 Dir(s)  93,747,830,784 bytes free

IN:

Python code:

from pathlib import Path
p = Path(r'C:/exampledir')
for path in sorted(p.iterdir())
    print(path)

OUT:

code:

C:\exampledir\barfile.txt
C:\exampledir\definitely_not_porn
C:\exampledir\folder_0
C:\exampledir\folder_1
C:\exampledir\foofile.txt
C:\exampledir\not_porn

(Note that I am on a windows rather than POSIX system)

Death Zebra, can you give us the full stack trace?

Eela6 fucked around with this message at 19:50 on Apr 24, 2017

# ¿ Apr 24, 2017 19:46

Eela6: May 25, 2007; Shredded Hen

Ordering is lexigrapic according to the path of the file. It works 'like you would expect' - file0 < file1 < file9.

More specifically, Lexigraphic ordering of strings and paths works more or less as follows:

Note that

Python code:


def __eq__(self, other):
        if len(self) == len(other) and all(ord(c) < ord(k) for c, k in zip(self, other)):
             return True
        return False

def __lt__(self, other):
          return any(ord(c) < ord(k) for c, k in zip(self, other)) or( 
               len(self) < len(other) and all(c == k for c, k in zip(self, other))
           )

(This is true at least for ASCII and the corresponding part of UTF-8.

EDIT: Oh, I see, your files aren't left-padded with zeros! I'm dumb. You can fix this by re-labeling the files. I'll have a solution up in a jiffy.

Eela6 fucked around with this message at 20:32 on Apr 24, 2017

# ¿ Apr 24, 2017 20:01

Eela6: May 25, 2007; Shredded Hen

baka kaba posted:

Badly, I was using sorted() until I realised my data was out of order, tracked it down to the files coming in all file1, file10, file11. That's why I was hoping a natural sort was lurking in the standard library

Oh well, maybe I can work around it with that split tuple suggestion up there, thanks! It just feels bad because it's brittle and Windows does natural sort ordering, missed opportunities

This is kind of an overwrought solution but it should work just fine. I'm sure a REGEX wizard can do this in three lines if not less.

Python code:

from pathlib import Path
def by_file_number(path):
    parent, name = path.parent, path.name
    digits = frozenset(str(x) for x in range(10))
    for i, char in enumerate(reversed(name), 1):
        if char == '.':
            break
    else: # NO BREAK
        raise ValueError('no extension in filename')
    extension = name[-i:]
    for k, char in enumerate(reversed(name[:-i])):
        if char not in digits:
            break
    try:
        number = int(name[(-k-i):-i])
    except ValueError:
        number = None # We choose None since it is NOT ordered.
    prefix = name[:(-k-i)]
    return (parent, (prefix, number, extension))
    # Note we're returning a tuple of a the form (str, (str, union[int, None], str))

IN:

Python code:

filenames = (
        'C:\exampledir/file0.bat', 'C:\exampledir/file1.txt',
        'C:\exampledir/file2.txt', 
        'C:\exampledir/file10.txt', 
        'C:\exampledir/file11.txt', 
        'C:\otherdir/file21.txt',
        'C:\exampledir/foobar.jar')
paths = sorted((Path(name) for name  in filenames), key = by_file_number)
for path in paths:
    print(path)

OUT:

code:

C:\exampledir\file0.bat
C:\exampledir\file1.txt
C:\exampledir\file2.txt
C:\exampledir\file10.txt
C:\exampledir\file11.txt
C:\exampledir\foobar.jar
C:\otherdir\file21.txt

Eela6 fucked around with this message at 21:14 on Apr 24, 2017

# ¿ Apr 24, 2017 21:07

Eela6: May 25, 2007; Shredded Hen

ynohtna posted:

Here's my common snippet to convert an iterable into natural sort order:

Python code:

import re
def sorted_nicely(iter):
    """ Sort the given iterable in the way that humans expect."""
    convert = lambda text: int(text) if text.isdigit() else text
    alphanum_key = lambda key: [ convert(c) for c in re.split('([0-9]+)', key) ]
    return sorted(iter, key=alphanum_key)

Usage is:

Python code:

    for p in sorted_nicely(seen_paths):
        print(p)

Ooh, this one is nice.

# ¿ Apr 25, 2017 00:26

Eela6: May 25, 2007; Shredded Hen

Cingulate posted:

Crank it up a notch.

Python code:

noisymodule.print = lambda x : del random.choice(noisymodule.keys())

noisymodule.warn = lambda x : os.remove(glob.glob(noisymodule.__file__.replace("__init__.py", "*"))

Mods?

# ¿ Apr 27, 2017 06:46

Eela6: May 25, 2007; Shredded Hen

QuarkJets posted:

Probably execution is supposed to stop at that point, once the error is raised? Definitely throw a raise at the end of that except, if so

A catch of Exception (or worse yet, BaseException) without a re-raise is a huge red flag.

# ¿ May 9, 2017 23:09

Eela6: May 25, 2007; Shredded Hen

Hi Python goons! PyCon starts next week. I will be in Portland, OR for it. Anyone else going?

# ¿ May 14, 2017 16:30

Adbot: ADBOT LOVES YOU

# ¿ Apr 29, 2024 12:00

Eela6: May 25, 2007; Shredded Hen

LochNessMonster posted:

So I've been messing around with Python for a few months but I feel I'm stuck at a level slightly above complete beginner. I know how to work with lists/dicts/tuples and create loops/conditions.

What would be a good resource to gain knowledge on working with concepts that are new to me? Things I regularly see and don't understand are __init__ or __main__ funtions, when I should use classes or how to create a decent structure in my code. Currently I create a few functions and below them I write loops/conditions which use them. When looking at code from others I usually see them split over multiple files.

I'm working with python3.

You actually want to know two things:
0. How to write pythonic code;
Solution: Read Fluent Python. It is by far the best intermediate-level book I know of. You don't have to read it cover to cover; read the first few chapters and then skip around to stuff that interests you. As you learn more, you can go back to the more advanced topics. Plus, it's an excellent reference.

1. How to organize your code in general.

Solution: this is a little tougher! I would recommend Clean Code by Robert C. Martin. This is the best book I know of on the topic. It's about Java, but the concepts are universal. Honestly, though, it's just going to take a lot of practice.

Eela6 fucked around with this message at 14:51 on Jun 1, 2017

# ¿ Jun 1, 2017 14:45

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›3 »