Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Hed: Mar 31, 2004; Fun Shoe

Yeah I don't know it's how every hand calculator I've ever had has worked so it didn't surprise me at all.

# ? Apr 26, 2016 22:31

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 07:46

Tigren: Oct 3, 2003

Hed posted:

Yeah I don't know it's how every hand calculator I've ever had has worked so it didn't surprise me at all.

Hand... calculator? Like, you push buttons on a thing that only calculates?

# ? Apr 26, 2016 23:01

Hed: Mar 31, 2004; Fun Shoe

It can spell.... boobs.

# ? Apr 27, 2016 01:10

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Tigren posted:

Hand... calculator? Like, you push buttons on a thing that only calculates?

You know how you prove an RCE with a screenshot of calc.exe? Well, that program was actually made into a handheld device, like a Tiger Electronics portable game but for an application.

Edit: just for clarification, I don't think handheld calculators can be used to demonstrate an RCE unless the thing you hacked is a 3D printer.

# ? Apr 27, 2016 03:06

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

QuarkJets posted:

To be fair, it's how most languages work under-the-hood. Even just written out on paper it's a bit ambiguous

I believe that if I saw -2� written on paper I'd read it as -4 without thinking hard about it, and I'd expect to see (-2)� if it were to be evaluated the other way.

Or to put it another way, I think in written mathematics the unary minus sign is just shorthand for "0 -" and with that understood, it works with the same precedence as subtraction usually has.

# ? Apr 27, 2016 14:31

Dominoes: Sep 20, 2007

Hey dudes. I know this is an area with many attempts and no popular results, but I'm trying to override datetimes. Ie a wrapper for datetime and pytz with easy syntax, but without some of the downfalls of attempts like Arrow. [slowness, requiring the user to call timezone libraries explicitly, and obtuse naming conventions etc]

I suck at classes, but I think they're required h ere. Datetime objects' tzinfo attribute can't be set by a hard block, so I need to return a new one If I initiate a new instance, or convert a timezone-naive datetime. Here's my mess of confusion:

Python code:

class Instant(datetime.datetime):
    def __init__(self, *args, **kwargs):
        super().__init__()  # Tbh I'm not sure what this does
	# Returning doesn't work in this func, nor does setting tzinfo

    def __new__(cls, *args, **kwargs):
        # I think this is how I do it? Not working, but at least I can return from __new__.
        if self.tzinfo == None:
            return self.replace(tzinfo=pytz.utc)

I'm not sure where to start; how would it work? If tzinfo were settable, it would look like this:

Python code:

class Instant(datetime.datetime):
    def __init__(self, *args, **kwargs):
        if not self.tzinfo:
            self.tzinfo = pytz.utc

edit: This seems to be working. Is it OK?

Python code:

class Instant(datetime.datetime):
    def __init__(self, *args, **kwargs):
        super().__init__()

    def __new__(cls, *args, **kwargs):
        dt = datetime.datetime(*args, **kwargs)

        if dt.tzinfo:
            return dt
        else:
            return dt.replace(tzinfo=pytz.utc)

Dominoes fucked around with this message at 05:40 on Apr 29, 2016

# ? Apr 28, 2016 20:17

KICK BAMA KICK: Mar 2, 2009

Not 100% I'm following what you want but to simply that, could you, inside your __init__ look for a tzinfo argument inside **kwargs, test it, and pass it on either as is or modified by your logic to super.__init__(*args, **kwargs)? Now if tzinfo can be given as a positional argument it's messier.

# ? Apr 28, 2016 22:25

Space Kablooey: May 6, 2009

Does anyone know if it is it possible to run UWSGI from a Raspberry Pi? AFAIK you can run PyPy on it, but I'm not sure if it just works with my current application set or if I have to whip up something else.

I don't have a Pi or else I would test it myself. :smith:

# ? Apr 29, 2016 00:31

creatine: Jan 27, 2012

HardDisk posted:

Does anyone know if it is it possible to run UWSGI from a Raspberry Pi? AFAIK you can run PyPy on it, but I'm not sure if it just works with my current application set or if I have to whip up something else.

I don't have a Pi or else I would test it myself.

I can test it out if you tell me what I need to do for it. I'm installing UWSGI now on my pi2

edit: following this guide http://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html

I was able to do the hello world successfully.

creatine fucked around with this message at 02:53 on Apr 29, 2016

# ? Apr 29, 2016 02:48

Space Kablooey: May 6, 2009

Wow, thank you! :3:

Any specific headaches? Are you running Noobs or Raspbian? Do you have to run PyPy or does cPython work?

Can you test Flask on it, please?

# ? Apr 29, 2016 03:47

creatine: Jan 27, 2012

HardDisk posted:

Wow, thank you!

Any specific headaches? Are you running Noobs or Raspbian? Do you have to run PyPy or does cPython work?

Can you test Flask on it, please?

NOOBS with the default python 2.7. Installing flask to try it.

edit: so I don't really know flask at all so I don't really know what's supposed to go on with this. When I run a hello world flask app through uwsgi the console doesnt give any errors but I don't know what supposed to happen next

This is what the console shows.

creatine fucked around with this message at 04:09 on Apr 29, 2016

# ? Apr 29, 2016 03:52

Space Kablooey: May 6, 2009

It seems to be working to me. Try accessing localhost:8080.

# ? Apr 29, 2016 04:25

creatine: Jan 27, 2012

Connection refused error on localhost and with :8080 but I'm probably doing something wrong. Like I said I've never used flask or uwsgi before so I'm going in blind.

# ? Apr 29, 2016 04:27

Space Kablooey: May 6, 2009

Probably because it needs an nginx running as well. But uWSGI seems to be working. Thanks a lot for that!

# ? Apr 29, 2016 04:29

creatine: Jan 27, 2012

No problem! I can fiddle with getting nginx running tomorrow to test if it fully works too

# ? Apr 29, 2016 04:34

Dominoes: Sep 20, 2007

Dominoes posted:

edit: This seems to be working. Is it OK?

Python code:

class Instant(datetime.datetime):
    def __init__(self, *args, **kwargs):
        super().__init__()

    def __new__(cls, *args, **kwargs):
        dt = datetime.datetime(*args, **kwargs)

        if dt.tzinfo:
            return dt
        else:
            return dt.replace(tzinfo=pytz.utc)

Not working - it looks like this is just returning a dt object that always has tzinfo, rather than my custom object. I tried this:

Python code:

class Instant(datetime.datetime):
    # def __init__(self, *args, **kwargs):
    def __init__(self, year, month, day, hour=0, minute=0, second=0, microsecond=0, tzinfo=None):
        super().__init__()

        # Boilerplate
        self.year = year
        self.month = month
        self.day = day
        self.minute = minute
        self.second = second
        self.microsecond = microsecond
        self.tzinfo = tzinfo if tzinfo else pytz.utc

But am still unable to write to these methods due to datetime's built-in blocks: "AttributeError: attribute 'year' of 'datetime.date' objects is not writable" Any OOP wizards know what to do?

# ? Apr 29, 2016 18:06

SurgicalOntologist: Jun 17, 2004

I've honestly never seen __new__ used before (although I do know what it does)... but I think you need to use the cls input. The first line should be dt = cls(*args, **kwargs). Then you'll get your Instant object and not a regular datetime. I also like to use self for what you call dt when I'm writing an object-creation classmethod, to make it clear that it's for object creation.

You also still have the same problem as before: you need to be passing datetime arguments to super().__init__(). In the first method, doing this might make the __new__ method unnecessary. In the second, you can set the datetime fields by passing them to super().__init__() rather than setting them as attributes (since perhaps datetime is immutable).

I would also question using inheritance here at all, although I'm not an OOP wizard and try to avoid classes as much as possible.

SurgicalOntologist fucked around with this message at 18:32 on Apr 29, 2016

# ? Apr 29, 2016 18:29

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

I highly recommend using functions and not custom subclasses of Python builtins.

# ? Apr 29, 2016 18:31

Dominoes: Sep 20, 2007

Suspicious Dish posted:

I highly recommend using functions and not custom subclasses of Python builtins.

Yea, but you can't make a type with functions :/ I'm making most of what are traditionally methods into standalone functions that operate on the class (or normal datetime objects... bonus!)

SurgicalOntologist posted:

I've honestly never seen __new__ used before (although I do know what it does)... but I think you need to use the cls input. The first line should be dt = cls(*args, **kwargs). Then you'll get your Instant object and not a regular datetime. I also like to use self for what you call dt when I'm writing an object-creation classmethod, to make it clear that it's for object creation.

You also still have the same problem as before: you need to be passing datetime arguments to super().__init__(). In the first method, doing this might make the __new__ method unnecessary. In the second, you can set the datetime fields by passing them to super().__init__() rather than setting them as attributes (since perhaps datetime is immutable).

I would also question using inheritance here at all, although I'm not an OOP wizard and try to avoid classes as much as possible.

I get max recursion errors when I do this: I think it's bouncing back between new and init indefinitely.

I'm trying a non-inheriting object that converts to datetime.datetime when added etc, but I suspect this will cause slowness, which is one of the complaints I have about Arrow.

Dominoes fucked around with this message at 18:40 on Apr 29, 2016

# ? Apr 29, 2016 18:35

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Your code will be inherently more fragile and harder to read if you do it this way, even if you do succeed in getting it to work. Write code that operates on normal datetime objects.

# ? Apr 29, 2016 18:42

Dominoes: Sep 20, 2007

Suspicious Dish posted:

Your code will be inherently more fragile and harder to read if you do it this way, even if you do succeed in getting it to work. Write code that operates on normal datetime objects.

Considering this approach; most of what I'm doing works on either; A lot of the cleanup doesn't need a fancy object; might have been lead down this road by the existing attempts to improve dts. A lot can be done by applying functions to datetime.datetime. Ringer?: I'm raising errors if my funcs operate on a tz-naive dt.

Some of my goals:
-Not slow like Arrow
-Intuitive syntax. No strftime/strptime, no weird replace(days)/replace(day) stuff or jumping through hoops to localize.
-Unsure if I want to merge dt/date/time. If I do, won't just set the unused parts to 0 like Arrow.
-Clean format method to convert to/from string: no '%'s.
-Iteration
-Function-based rather than method-based
-Don't deal with tz-naive dts.
-One module to import: Wraps datetime and pytz, and doesn't require calling them separately.
-Easy tz localization.

Most of this should be pretty easy, and worth doing.

Dominoes fucked around with this message at 18:53 on Apr 29, 2016

# ? Apr 29, 2016 18:43

SurgicalOntologist: Jun 17, 2004

Dominoes posted:

I get max recursion errors when I do this: I think it's bouncing back between new and init indefinitely.

Right, that makes sense. I see no reason you need the new though. Do any input mangling you need in init.

Dominoes posted:

I'm trying a non-inheriting object that converts to datetime.datetime when added etc, but I suspect this will cause slowness, which is one of the complaints I have about Arrow.

Instead of an object that converts, what I had in mind was composition. Store the datetime as an attribute and you'll have much more freedom operating with it. In other words, make your class a wrapper of datetime rather than a modified datetime.

But really, you should listen to Suspicious Dish.

# ? Apr 29, 2016 18:46

Dominoes: Sep 20, 2007

Both good ideas; both better than what I was trying to do. TBH I'm not sure what I would do with custom methods on top of datetime.datetime's, other than forcing the object to have a tzinfo.

Here's your idea (having dt as an attribute rather than canning the custom object):

Python code:

class Instant:
        def __init__(self, year: int, month: int, day: int, hour:int=0, minute:int=0,
                second:int=0, microsecond:int=0, tzinfo=pytz.utc):
            # Boilerplate
            self.year = year
            self.month = month
            self.day = day
            self.hour = hour
            self.minute = minute
            self.second = second
            self.microsecond = microsecond
            self.tzinfo = tzinfo

            self.datetime = datetime.datetime(year, month, day, hour, minute,
                second, microsecond, tzinfo)


            def __add__(self, other):
                return self.datetime + other

            def __repr__(self):
                # return to_iso(self)
                return self.datetime.isoformat()

Dominoes fucked around with this message at 19:00 on Apr 29, 2016

# ? Apr 29, 2016 18:54

SurgicalOntologist: Jun 17, 2004

Oh, if forcing tzinfo is all you want to do that's probably overkill. In that case I would consider making a function that fakes being a class.

Python code:

def Instant(*args, **kwargs):
    dt = datetime.datetime(*args, **kwargs)

    if dt.tzinfo:
        return dt
    else:
        return dt.replace(tzinfo=pytz.utc)

# ? Apr 29, 2016 19:01

Dominoes: Sep 20, 2007

That's slick too.

# ? Apr 29, 2016 19:06

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

the only thing you changed was a single letter

# ? Apr 29, 2016 19:09

SurgicalOntologist: Jun 17, 2004

I changed class to def. Is that not the kind of thing you had in mind when you encouraged using functions? Whether it's capitalized or not isn't really the issue; it may be better if it doesn't pretend to be a class to be honest.

# ? Apr 29, 2016 19:15

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Yeah, my point is that he was somehow against functions, but when capitalizing it, suddenly it's super slick.

# ? Apr 29, 2016 19:44

Dominoes: Sep 20, 2007

Suspicious Dish posted:

Yeah, my point is that he was somehow against functions

Why would you think that‽

Python code:

def datetime(*args, **kwargs):
    """Create a datetime instance, with default tzawareness at UTC."""
    ...

code:

In [3]: instant.datetime(2016, 1, 1)
Out[3]: datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<UTC>)

# ? Apr 29, 2016 19:52

Dominoes: Sep 20, 2007

~~https://github.com/David-OConnor/instant~~
edit: Name change due to Pypi availability https://github.com/David-OConnor/saturn

Basic documentation. The functionality it describes works atm except for string formatting (Uses strptime/strftime format until I can figure out the regex). Suggestions?

Dominoes fucked around with this message at 19:15 on May 1, 2016

# ? Apr 29, 2016 21:40

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

It's always bothered me how slow Arrow and dateutil are. I've been limited by their speed many times.

I haven't actually investigated to see whether the slowness is because of the way they are coded or if it's just inherent in the types of operations they do...but I'm glad someone else is working on it because gently caress dealing with dates and times.

# ? Apr 29, 2016 23:23

Dominoes: Sep 20, 2007

Same- not sure why Arrow's slow. I started using it for being always tz-aware, iteration, and curiosity about merging dates and dts. I was driven away by slowness, awkward syntax, lack of support for timedeltas, and requiring multiple imports.

So far I've cleaned up irritating syntax and tz-naive pitfalls that affect my own code. What functionality do you think would be useful, besides what I've listed on the github page?

Dominoes fucked around with this message at 19:57 on Apr 30, 2016

# ? Apr 30, 2016 06:48

QuarkJets: Sep 8, 2008

Would it maybe make more sense to try investigating and improving Arrow's efficiency, rather than making a new class that replicates the same functionality but faster? It could be a real shitshow and you might get roadblocked, but if you're successful then it would help a lot more people

# ? Apr 30, 2016 19:56

Dominoes: Sep 20, 2007

QuarkJets posted:

Would it maybe make more sense to try investigating and improving Arrow's efficiency, rather than making a new class that replicates the same functionality but faster? It could be a real shitshow and you might get roadblocked, but if you're successful then it would help a lot more people

I submitted a cut and dry improvement suggestion to Arrow syntax a while back and it got shut down hard; Arrow has multiple issues, and I don't think the creators are willing to fix them. The adding/subtracting syntax (replace) is awkward to use. (ie replace.day= vs replace.days=). Figuring out, and submitting a pull (If practical) for Arrow's speed issue would be a good goal; haven't dug through their code yet. Arrow's home page lists 'fewer imports' as a feature, but you still need to import arrow, pytz or dateutil, and datetime to do basic stuff.

Does anyone have a guess as to why Arrow's slow? I picked an arbitrary task to demonstrate:

Python code:

def test_arrow():
    a = arrow.Arrow(2016,4,30)
    for i in range(100000):
        a = a.replace(minutes=1)


def test_dt():
    d = pytz.utc.localize(dt.datetime(2016, 4, 30)
    for i in range(100000):
        d = d +  dt.timedelta(minutes=1)


%timeit test_dt()
10 loops, best of 3: 64 ms per loop

%timeit test_arrow()
1 loop, best of 3: 1.18 s per loop

Dominoes fucked around with this message at 20:19 on Apr 30, 2016

# ? Apr 30, 2016 20:01

QuarkJets: Sep 8, 2008

From https://github.com/crsmithdev/arrow/blob/master/arrow/arrow.py

1) It looks like replace() creates an entire new Arrow instance with updated attributes instead of just updating the attributes of the current Arrow instance. This might not be the worst idea, if there's a good reason for it.

2) Inside of replace() there's a big ugly comparison between the strings in kwargs and the strings in two internal lists, _ATTRS and _ATTRS_PLURAL. This happens every time that replace() is called. Yikes. Replacing those ATTR lists with sets would speed up the comparisons significantly, but it looks like the fact that these are lists in a specific order might be used elsewhere (in which case they could probably use OrderedDict and still get a huge speedup, but that's sloppy; is there really no OrderedSet anywhere in core Python?). This right here is probably the bulk of the problem

# ? Apr 30, 2016 20:40

Dominoes: Sep 20, 2007

Good find! Regarding point 1, I'm guessing it's to imitate datetime.datetime's immutability.

# ? Apr 30, 2016 20:49

QuarkJets: Sep 8, 2008

It might be fun just to try it. "Hey I turned these lists into OrderedDict objects and the speedup was insane. Patch?"

# ? May 1, 2016 03:25

Dominoes: Sep 20, 2007

No reason not to. You already understand the issue - give it a go!

Edit: Poking through Arrow's code, I noticed that formatting as a string requires creating a datetime object and changing its timezone... I'm guessing this library isn't designed with speed as a goal.

Do y'all think these these func arguments should be set up with the most important arguments first (ie the datetime), per convention, or with the ones that are less likely to change first, which would make partial/currying easier? Ie:

Python code:

def to_str(dt: DateOrDateTime, str_format: str) -> str:
    pass  # Most important first; logical and per most conventions

# or

def to_str(str_format: str, dt: DateOrDateTime, ) -> str:
    pass  
    # Can curry easier, ie my_format = partial(instant.to_str, 'YYYY-MM-DD')
    # (my_format(dt_string) for dt_string in dt_strings) or 
    # map(my_format, dt_strings)

edit: Looking through Arrow's code more... lots of obfuscation. I looked at the 'get' function and 'format' method to figure out how they did the regex parsing, and went on an adventure through several modules and classes before finding the code.

Why would you they this:

Python code:

def format(self, fmt, locale='en_us'):
    return formatter.DateTimeFormatter(locale).format(self._datetime, fmt)

#...

class DateTimeFormatter(object):
    _FORMAT_RE = re.compile('(YYY?Y?|MM?M?M?|Do|DD?D?D?|d?dd?d?|HH?|hh?|mm?|ss?|SS?S?S?S?S?|ZZ?|a|A|X)')

    def __init__(self, locale='en_us'):
        self.locale = locales.get_locale(locale)

    def format(cls, dt, fmt):
        return cls._FORMAT_RE.sub(lambda m: cls._format_token(dt, m.group(0)), fmt)

    def _format_token(self, dt, token):
        if token == 'YYYY':
            return self.locale.year_full(dt.year)
        # ...

instead of this?:

Python code:

def format(dt, str_format, locale='en_us'):
    locale = locales.get_locale(locale)
    _FORMAT_RE = re.compile('(YYY?Y?|MM?M?M?|Do|DD?D?D?|d?dd?d?|HH?|hh?|mm?|ss?|SS?S?S?S?S?|ZZ?|a|A|X)')

    return _FORMAT_RE.sub(lambda m: _format_token(dt, m.group(0), locale), str_format)


def format_token(dt, token, locale):
    if token == 'YYYY':
        return locale.year_full(dt.year)
        # ...

Dominoes fucked around with this message at 19:37 on May 1, 2016

# ? May 1, 2016 11:49

QuarkJets: Sep 8, 2008

Is DateTimeFormatter used in a lot of other places? This might just be a one-time cludge to call a class method as a function, which is fine if the class method needs to be used as a class method in other places. Or it could just be someone going overboard with OO programming

# ? May 1, 2016 21:00

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 07:46

Dominoes: Sep 20, 2007

No, only in the place I mentioned.

# ? May 1, 2016 21:27

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »