Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Thermopyle posted:

Sounds like there was some drama at the Python language summit.

The general idea:

Python is loving up it's ecosystem with it's poor standard library.

I like where Amber Brown is coming from there.

In a lot of ways I think that if you're making Guido mad you've got good ideas!

On the other hand though, he does have an excellent point that developers really need to stop supporting python 2 universally in their packages and just let the version just sort of organically die regardless of if RedHat will support it forever.

Also, I like how a talk that focuses on how we should more to pypi doesn't mention at all the huge mess that is python package installation.

# ? May 18, 2019 22:25

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 12:32

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Master_Odin posted:

On the other hand though, he does have an excellent point that developers really need to stop supporting python 2 universally in their packages and just let the version just sort of organically die regardless of if RedHat will support it forever.

Also, I like how a talk that focuses on how we should more to pypi doesn't mention at all the huge mess that is python package installation.

Yes, people should just rip off the python 2 bandaid.

She might very well have mentioned python packaging, the article is just a summary of her talk.

They didn't record the talks so people could say more controversial things. Sounds like Guido should've thought about that before attending talks there...

# ? May 18, 2019 22:51

Hed: Mar 31, 2004; Fun Shoe

Hollow Talk posted:

And it will only get better: http://charlesleifer.com/blog/new-features-planned-for-python-4-0/

I enjoyed this.

# ? May 18, 2019 22:55

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

Thermopyle posted:

Yes, people should just rip off the python 2 bandaid.

She might very well have mentioned python packaging, the article is just a summary of her talk.

They didn't record the talks so people could say more controversial things. Sounds like Guido should've thought about that before attending talks there...

Easier said than done. Would be a lot easier if Guido had engineered (lol) a proper migration strategy instead of winging it and complaining when people dont spend millions upgrading.

# ? May 18, 2019 23:44

the yeti: Mar 29, 2008; memento disco

Malcolm XML posted:

Lmao at Guido taking it persomally when someone with expertise makes constructive criticism about his baby and he storms out

Literally the core tenet of open source development

# ? May 19, 2019 00:16

Dominoes: Sep 20, 2007

From the parody article on Python 4:

quote:

In addition to namedtuple and dataclasses (3.7), Python 4.0 will include several new thousand line decorator-hacks to implement simple struct types.
The GIL has been removed.

Seriously. I appreciate the attempts to make a struct with Dataclass, but why did it take so long, and why's something like Dataclass not the base way of storing related data? I imagine it being more common than normal classes, NamedTuples etc in my code in the future.

And... love the KR drama.

# ? May 19, 2019 00:19

breaks: May 12, 2001

Hey man we�ve had ~~namedtuples~~ ~~NamedTuples~~ ~~attrs~~ dataclasses for a long time now

If only this level of effort had been put into package management

# ? May 19, 2019 02:33

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Thermopyle posted:

Sounds like there was some drama at the Python language summit.

This is ok and good but gently caress stop supporting python 2 aaaahhhh

Hollow Talk posted:

And it will only get better: http://charlesleifer.com/blog/new-features-planned-for-python-4-0/

quote:

No switch statement (and yes, yes, I know you can use a dict to do dispatching).

Ahahahah

# ? May 19, 2019 05:03

QuarkJets: Sep 8, 2008

Thermopyle posted:

Sounds like there was some drama at the Python language summit.

The general idea:

Python is loving up it's ecosystem with it's poor standard library.

I like where Amber Brown is coming from there.

In a lot of ways I think that if you're making Guido mad you've got good ideas!

Brown starts off saying that the stdlib doesn't have enough useful features, so users need to rely on third party stuff, which sucks

Then Brown points out that the stdlib is bloated and hard to build, with features that are simply worse than 3rd party modules, which is problematic because simply being in the stdlib means that people will sacrifice features for convenience.

I don't think you can truly fix both of these things.

QuarkJets fucked around with this message at 05:43 on May 19, 2019

# ? May 19, 2019 05:27

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Maybe "batteries included" isn't the best idea because those batteries are usually crappy and get discarded quickly??

Make it easier for people to add their own quality batteries

# ? May 20, 2019 01:04

General_Failure: Apr 17, 2005

shrike82 posted:

My bad... Why not give Google Colab a try? It's free and would let you play with CV frameworks?

I did. It reminded me I have to do the tutorial. I have a knack for breaking it. Thanks for the reminder.

I also just tried running a jupyter notebook. While I'm not fond of web interfaces it seems pretty good. Being able to connect to a server on a remote machine is a big plus.

It seems kind of obtuse but I can only really use the PC with a head occasionally because of a lack of space.Running cost is a concern these days too. Doing stuff on Google's money seems like a good idea. I mostly use my pinebook to access other things anyway.
Admittedly way before posting here I got lovely and pre-ordered a Jetson Nano. If / when it arrives I'm really looking forward to giving it a workout.

# ? May 20, 2019 11:35

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

baka kaba posted:

Maybe "batteries included" isn't the best idea because those batteries are usually crappy and get discarded quickly??

Make it easier for people to add their own quality batteries

This way lies npm and I don't think we want that road

# ? May 21, 2019 03:14

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Phobeste posted:

This way lies npm and I don't think we want that road

Ehh, npm is basically no batteries included.

Changeable batteries is what ensurepip kind of does now already.

# ? May 21, 2019 03:42

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

The standard library might be better if updates weren't directly tied to Python updates.

Of course, then you have a whole 'nother problem...

# ? May 21, 2019 03:44

NinpoEspiritoSanto: Oct 22, 2013

I think there's a lot of well solved problems in stdlib, I feel a lot of the heat coming lately is because of solving as yet unsolved (well) problems such as packaging and deployment, py core have decided to put in their own flawed solutions to already solved problems in the shape of asyncio (twisted well established if somewhat flawed but better positioned to resolve those flaws) and data classes (attrs seems to me like it solved that problem all too well).
Core just doesn't get the development and release cycle such things benefit from.

# ? May 21, 2019 10:40

Nippashish: Nov 2, 2005; Let me see you dance!

I think python would benefit from an unstable place to grow the standard lib. Sort of like what boost does for c++.

# ? May 21, 2019 20:29

Sylink: Apr 17, 2004

This is sort of a general programming question but I'm using Python specifically -

I am writing a small script that will take a list of people and their shift schedules, and then it needs to detect from that list and the current datetime whether that person should be on shift now or not.

I have two problems:

1) What is the best way to encode/store the shift schedules in a format that the script would import? The schedules are the same weekly so the exact date isn't important but the weekday and hour of the day are important.

Right now I record the start hour as number and the weekday as a number (0-6 per the weekday() function in datetime), then the length of the shift in hours so you can compute the end by adding X hours.

2) What is the best way to detect if they are within the range? It seems like I need to generate a table of values (probably in pandas) with one column being the start time and the other an end time, then you can do a start < current < end to detect if they should be on shift.

Mostly point 1 and making it easy to get the shifts in is the hard part, I could just generate the next 48 hours of date ranges for each person I suppose? (48 because some shifts go through midnight so over two different days)

# ? May 22, 2019 00:22

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

Sylink posted:

This is sort of a general programming question but I'm using Python specifically -

I am writing a small script that will take a list of people and their shift schedules, and then it needs to detect from that list and the current datetime whether that person should be on shift now or not.

I have two problems:

1) What is the best way to encode/store the shift schedules in a format that the script would import? The schedules are the same weekly so the exact date isn't important but the weekday and hour of the day are important.

Right now I record the start hour as number and the weekday as a number (0-6 per the weekday() function in datetime), then the length of the shift in hours so you can compute the end by adding X hours.

2) What is the best way to detect if they are within the range? It seems like I need to generate a table of values (probably in pandas) with one column being the start time and the other an end time, then you can do a start < current < end to detect if they should be on shift.

Mostly point 1 and making it easy to get the shifts in is the hard part, I could just generate the next 48 hours of date ranges for each person I suppose? (48 because some shifts go through midnight so over two different days)

You want a date/time library that can
1. Serialize to and deserialize from strings (when saving/reading the file defining the shifts)
2. Do nice comparisons so you can do now < shift_end

A standard library way to do this is datetime, using strftime and strptime for reading/writing and comparing datetime objects. As posters have mentioned above there are better libraries on pypi for this (I haven�t used them personally but I 100% believe they�re better than datetime); that said, a lot of the problems they solve are about things like timezone handling and localization that might not be an issue for you.

Keep in mind, on that subject, that if you ever have even one person using this - as either someone that will be defining shifts or getting angry texts that they�re not at work - who is in a different time zone than everybody else this gets a lot more complex

# ? May 22, 2019 00:36

Sylink: Apr 17, 2004

Maybe someone can recommend those datetime alternatives, because yah that standard library is tedious.

And yah timezones suck, which is why we do everything in UTC.

# ? May 22, 2019 00:50

vikingstrike: Sep 23, 2007; whats happening, captain

It would be super easy to do this in pandas. Store data at the Worker/Day of Week level of observation with a start time and end time? So if a worker worked 7 days of the week, they would have 7 rows. Index to the correct row based on day of week.

# ? May 22, 2019 01:38

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Sylink posted:

Maybe someone can recommend those datetime alternatives, because yah that standard library is tedious.

And yah timezones suck, which is why we do everything in UTC.

https://pendulum.eustace.io/

E: it�s made by the same person who made poetry (which is also great).

Boris Galerkin fucked around with this message at 05:45 on May 22, 2019

# ? May 22, 2019 05:35

NinpoEspiritoSanto: Oct 22, 2013

Boris Galerkin posted:

https://pendulum.eustace.io/

E: it’s made by the same person who made poetry (which is also great).

I was also going to recommend pendulum if someone else hadn't, it's a brilliant library.

Depending on your storage format, marshmallow might be helpful in serialising/deserialisjng your data as well, I've used it a lot with json and csv storage formats.

# ? May 22, 2019 08:14

Dominoes: Sep 20, 2007

Pendulum, like moment and arrow, has no discrete Date and Time types.

# ? May 22, 2019 08:51

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Well, I guess that talk resonated with the core python developers with the proposed PEP-0594 to deprecate and remove some real old/unused stdlib modules.

# ? May 23, 2019 00:39

Hughmoris: Apr 21, 2007; Let's go to the abyss!

I need some advice on how to approach a problem at work.

We have a rule engine that processes incoming orders. A new order comes in with a list of properties, and depending on what those properties and values are, the rule engine directs the order down a specific node to an action. Behind the GUI editor, the rule engine is essentially an XML file.

The problem is analysts can tweak the rules in order to account for a new type of order that will be processed, or to edit the current routing of a particular order. We don't have a good way of testing after an analyst makes a configuration change. It's easy to test their specific change (by submitting that specific order and sees if behaves as intended), but it could have unforseen consequences to the routing of other order types.

I'd like to create something that would allow for automated testing. I'm not sure how to start, given the XML file that represents the current conditional rules. Maybe recreate the entire XML file in Python as conditional statements? Create test orders that are essentially a JSON object {'Order123': {'Type:Sports', 'Method:Online', 'Cost:$530'}}. Then run that order through my test script before an analyst makes changes to the engine. Then run the test script again post configuration change, and verify it's the same result.

Any thoughts/advice on how I could try tackling this?

# ? May 24, 2019 02:14

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Hughmoris posted:

I need some advice on how to approach a problem at work.

We have a rule engine that processes incoming orders. A new order comes in with a list of properties, and depending on what those properties and values are, the rule engine directs the order down a specific node to an action. Behind the GUI editor, the rule engine is essentially an XML file.

The problem is analysts can tweak the rules in order to account for a new type of order that will be processed, or to edit the current routing of a particular order. We don't have a good way of testing after an analyst makes a configuration change. It's easy to test their specific change (by submitting that specific order and sees if behaves as intended), but it could have unforseen consequences to the routing of other order types.

I'd like to create something that would allow for automated testing. I'm not sure how to start, given the XML file that represents the current conditional rules. Maybe recreate the entire XML file in Python as conditional statements? Create test orders that are essentially a JSON object {'Order123': {'Type:Sports', 'Method:Online', 'Cost:$530'}}. Then run that order through my test script before an analyst makes changes to the engine. Then run the test script again post configuration change, and verify it's the same result.

Any thoughts/advice on how I could try tackling this?

Why not just have a set of orders that get run through each time a configuration change is made?

# ? May 24, 2019 02:21

Hughmoris: Apr 21, 2007; Let's go to the abyss!

CarForumPoster posted:

Why not just have a set of orders that get run through each time a configuration change is made?

That was my initial thought, too. I could have a defined order set, place the orders, then check what the final destination is for each order pre and post rule change.

Unfortunately, there isn't a clean way to automate data pulls and extractions after orders are ran through the system. Would have to use the system's search engine to run a manual search, then export the results to CSV files, then compare said CSV files. Going that route will definitely be better than what we have (nothing).

Would be nice to automate everything but I'm stuck on what recreating that XML in Python would look like.

*Side note: I recently joined this team and it blows my mind that this hasn't already got sorted out. The vendor is a multi-billion dollar company, I don't know they haven't provided customers with a tool for this yet.

Hughmoris fucked around with this message at 02:29 on May 24, 2019

# ? May 24, 2019 02:26

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Hughmoris posted:

That was my initial thought, too. I could have a defined order set, place the orders, then check what the final destination is for each order pre and post rule change.

Unfortunately, there isn't a clean way to automate data pulls and extractions after orders are ran through the system. Would have to use the system's search engine to run a manual search, then export the results to CSV files, then compare said CSV files. Going that route will definitely be better than what we have (nothing).

Would be nice to automate everything but I'm stuck on what recreating that XML in Python would look like.

If you can really reliably find them by searching you could use python/selenium to get the data out the other side automatically.

# ? May 24, 2019 02:29

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Unable to directly connect to the current results for a variety of reasons, but each night there is a dump to a reporting server that I could query. But that would cause long pauses between testing and validation.

I can copy the XML document that represents the rules engine in to my script. Not sure how to ask this but with a given XML document, can I take an object and see what node it would land on given a set of properties? Essentially, start the object at the top of the XML document and filter down until it matches an attribute for a given node. When it reaches the end of that node, see what the properties of that node are.

I'm doing a crappy job of explaining that since I don't have a ton of experience with XML.

* Thinking on it some more: I can try to parse my XML document in to a whole bunch of IF/ELSE statements, then run my object through those conditions to see where it lands.

Hughmoris fucked around with this message at 03:11 on May 24, 2019

# ? May 24, 2019 02:36

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Hughmoris posted:

Unable to directly connect to the current results for a variety of reasons, but each night there is a dump to a reporting server that I could query. But that would cause long pauses between testing and validation.

I can copy the XML document that represents the rules engine in to my script. Not sure how to ask this but with a given XML document, can I take an object and see what node it would land on given a set of properties? Essentially, start the object at the top of the XML document and filter down until it matches an attribute for a given node. When it reaches the end of that node, see what the properties of that node are.

I'm doing a crappy job of explaining that since I don't have a ton of experience with XML.

* Thinking on it some more: I can try to parse my XML document in to a whole bunch of IF/ELSE statements, then run my object through those conditions to see where it lands.

I haven't read your posts on this subject real closely, but look into xpath maybe?

# ? May 24, 2019 17:10

Tayter Swift: Nov 18, 2002; Pillbug

Stupid newbie matplotlib question that's frustrating the gently caress out of me.

I have a time series with daily data going back several years. On my work computer I can plot it and it gives one tick per year on the x-axis like a sane plot would do, but I get a dreaded FutureWarning I don't understand:

code:

FutureWarning: Using an implicitly registered datetime 
converter for a matplotlib plotting method. The converter was 
registered by pandas on import. Future versions of pandas 
will require you to explicitly register matplotlib converters.

To register the converters:
	>>> from pandas.plotting import register_matplotlib_converters
	>>> register_matplotlib_converters()
  warnings.warn(msg, FutureWarning)

At home, however, it's trying to label every day, resulting in an unreadable mess for the x-axis (see screenshot below). How do I fix this so that there is only one tick and label per year?

Only registered members can see post attachments!

# ? May 24, 2019 17:23

StashAugustine: Mar 24, 2013; Do not trust in hope- it will betray you! Only faith and hatred sustain.

Hey I'm doing some work with python on Ubuntu. What's a good text editor for that? Gedit is kinda lovely about indentation sometimes (or maybe I'm just dumb)

# ? May 24, 2019 20:49

The Fool: Oct 16, 2003

StashAugustine posted:

Hey I'm doing some work with python on Ubuntu. What's a good text editor for that? Gedit is kinda lovely about indentation sometimes (or maybe I'm just dumb)

https://code.visualstudio.com/docs/setup/linux

# ? May 24, 2019 20:50

larper: Apr 9, 2019

# ? May 24, 2019 21:50

Charles 2 of Spain: Nov 7, 2017

Tayter Swift posted:

Stupid newbie matplotlib question that's frustrating the gently caress out of me.

I have a time series with daily data going back several years. On my work computer I can plot it and it gives one tick per year on the x-axis like a sane plot would do, but I get a dreaded FutureWarning I don't understand:
code:
FutureWarning: Using an implicitly registered datetime 
converter for a matplotlib plotting method. The converter was 
registered by pandas on import. Future versions of pandas 
will require you to explicitly register matplotlib converters.

To register the converters:
	>>> from pandas.plotting import register_matplotlib_converters
	>>> register_matplotlib_converters()
  warnings.warn(msg, FutureWarning)
At home, however, it's trying to label every day, resulting in an unreadable mess for the x-axis (see screenshot below). How do I fix this so that there is only one tick and label per year?

Should be something like ax.set_xticks() and ax.set_xticklabels() and you just give the function a list of what you want.

# ? May 25, 2019 03:01

Tayter Swift: Nov 18, 2002; Pillbug

Turns out the reg field was read in as a string and not a date :doh:

I assumed pandas would read in a date when it was saved to CSV from pandas as a date without explicitly using parse_dates. Welp.

# ? May 25, 2019 06:26

vikingstrike: Sep 23, 2007; whats happening, captain

Need to use something like HDF or parquet for pandas to know dtypes without you being more explicit in the load.

# ? May 25, 2019 13:58

Ghost of Reagan Past: Oct 7, 2003; rock and roll fun

QuarkJets posted:

Brown starts off saying that the stdlib doesn't have enough useful features, so users need to rely on third party stuff, which sucks

Then Brown points out that the stdlib is bloated and hard to build, with features that are simply worse than 3rd party modules, which is problematic because simply being in the stdlib means that people will sacrifice features for convenience.

I don't think you can truly fix both of these things.

I think if they buckled down and made the stdlib APIs consistent and then treated them as a great base to build better, friendlier solutions on top of (like Requests, Arrow, etc.), that'd do a lot of good. Brown is right that some features need to also be dropped -- tkinter needs to go, if someone still uses it they can have it as a separate package and leave the rest of us in peace.

The fact is there isn't a great solution to the problems because Python and the library are nearly 30 years old now. But you can make it better, and as a Python developer I think that's a good goal.

But yes just rip off the Python 2 bandaid. Now that supervisor 4 has Python 3 support there is basically zero excuse for it. Your lovely legacy application needs to get ported to Python 3, and that's all there is to it.

# ? May 25, 2019 15:50

the yeti: Mar 29, 2008; memento disco

Is tkinter really that bad? I ask this relatively innocently--I've seen a small example and it seemed pretty clumsy, but I have almost no experience with Python GUIs besides learning PySimpleGUI to refactor said small example.

# ? May 25, 2019 15:58

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 12:32

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

e;f: double post

Master_Odin fucked around with this message at 16:09 on May 25, 2019

# ? May 25, 2019 16:06

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »