Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Bad Munki: Nov 4, 2008; We're all mad here.

Hughmoris posted:

I know you did this as an exercise and it's a draft, but I wonder if you're experiencing some wonkyness specific to Apple. I'm running Win10 and used your code:

I'm 100% sure it's related, but that's still not an excuse. I'd expect this stuff to work way better out of the box sans customization. Like, if I were going to package this up somehow, it looks like I'd have a ton of OS-specific adjustments to be made for who knows how many OSes. A simple, generic solution this is not.

# ? Apr 14, 2021 15:30

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 17:54

QuarkJets: Sep 8, 2008

Bad Munki posted:

I'm 100% sure it's related, but that's still not an excuse. I'd expect this stuff to work way better out of the box sans customization. Like, if I were going to package this up somehow, it looks like I'd have a ton of OS-specific adjustments to be made for who knows how many OSes. A simple, generic solution this is not.

I've heard people complain about TKinter in this way before, and that's why I recommend pyqt

Linux GUIs made in TKinter tend to look pretty ugly, too. Not as ugly as your example, but I don't use TKinter anymore

# ? Apr 14, 2021 22:29

QuarkJets: Sep 8, 2008

So I'm watching the NVidia GTC talks this week and I just want to express my tentative excitement over legate.numpy and legate.pandas. They were in closed-access last year, but now they're out, and they're basically drop-in replacements for numpy and pandas for parallelized GPU computing

As in you replace numpy imports with legate.numpy imports and now your software runs on NVidia GPUs. And it also provides scalability, as in the normally single-threaded numpy instructions can be executed in parallel on an HPC cluster (CPU or GPU or both) by just altering your launch script to define the resources you want to use

I'm not going to assume that this is without flaws; it's new, there are a bunch of missing API translations, and of the ones that exist there are very likely going to be problems. But I have a lot of numpy code that could probably use this, already running on systems with GPUs, and I want to try it out.

https://github.com/nv-legate

(I'm also excited to try out using C++17 execution policies to target GPUs, but that's a topic for a different thread)

# ? Apr 14, 2021 23:35

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

QuarkJets posted:

So I'm watching the NVidia GTC talks this week and I just want to express my tentative excitement over legate.numpy and legate.pandas. They were in closed-access last year, but now they're out, and they're basically drop-in replacements for numpy and pandas for parallelized GPU computing

This sounds awesome. I never could get a parallelized apply to work reliably. e.g.

code:

def func(x, y)
    #some code
    return z

df['somecol'].apply(lambda val: func(val, number))

I've tried a few packages but to no avail.

Also Sorry for being a dick earlier but if we are showing code:

Bad Munki posted:

I was curious about all this so I slapped a Tkinter GUI on top of a thing I've been working on and at least if you keep your usage simple and just use the basic elements, lol dear god it's fuckin' hideous

https://i.imgur.com/yYDfs5S.mp4

The code that produced this mess, for reference:

For comparison in Dash, just put your function in the callback bit.

code:

import dash
import dash_bootstrap_components as dbc
import dash_html_components as html
from dash.dependencies import Input, Output, State

app = dash.Dash(
    external_stylesheets=[dbc.themes.BOOTSTRAP]
)

app.layout = dbc.Container(
    [
        # Make a text area.
        dbc.Textarea(className="mb-3", placeholder="A Textarea", id="text-input"),
        html.Br(),
        dbc.Button("Search", id="search-button", color="primary"),
        # Make an output Div
        html.Div("", id="some-output", className="mt-3")
    ], className="p-5"
)

#This callback will do stuff when the button is pressed taking in the value of the text area and returning it to the output div.
@app.callback(
    Output("some-output", "children"),
    [Input("search-button", "n_clicks")],
    [State("text-input", "value")]
)
def on_button_click(n, your_textarea):
    if n is None:
        return ""
    else:
        return f"Clicked {n} times. Your text is: {your_textarea}"

if __name__ == "__main__":
    app.run_server()

# ? Apr 15, 2021 00:30

Bad Munki: Nov 4, 2008; We're all mad here.

Shifting gears from UI chat:

I have a subclass of list, and I would like all the usual list operations that return lists themselves to return that same list subclass instead. What�s gonna be the path of least resistance here?

Example: my_thing[3:5] returns a built-in list instead of my subclass. Of course, I have things setup such that its constructor takes a list of the objects it works with, so I could call mySubClass(my_thing[3:5]) and get exactly what I want, but that�s not as nice. I feel like I�m most of the way there, I just need to sort out this final step.

The subclass is not terribly interesting except that it only stores a certain kind of object and provides a few extra operations on those objects.

Bad Munki fucked around with this message at 03:22 on Apr 15, 2021

# ? Apr 15, 2021 03:18

QuarkJets: Sep 8, 2008

Try subclassing Userlist instead, it was created because subclassing list can be tricky

# ? Apr 15, 2021 04:08

Walh Hara: May 11, 2012

Have you considered using composition over inheritance? It's a well known design idea... Without knowing more about your problem difficult to say if it's better, but worth thinking about.

# ? Apr 15, 2021 08:32

SurgicalOntologist: Jun 17, 2004

QuarkJets posted:

So I'm watching the NVidia GTC talks this week and I just want to express my tentative excitement over legate.numpy and legate.pandas. They were in closed-access last year, but now they're out, and they're basically drop-in replacements for numpy and pandas for parallelized GPU computing

Nice! Any idea how it compares to JAX? Looks like legate is easier to work with multiple GPUs while JAX targets more the single GPU use case, at least it's my 5 minute analysis.

At work we use xarray so it always seems a bit farther off that we can put our entire codebase into one of these backends, although looks like there is the beginning of some support so maybe we should try it. We manually cast to JAX arrays in a few slow functions but it's annoying to lose the conveniences of xarray. I wonder if the fact that it supports pandas means xarray is more doable (xarray basically generalizes pandas for high-dimensional arrays so you can get named dimensions rather than "columns" and "index" -- it uses pd.Index under the hood).

I'm registered to the conference but haven't been able to attend anything -- I've got a long list of talks I want to try to watch over the weekend.

# ? Apr 15, 2021 11:34

QuarkJets: Sep 8, 2008

Not sure how they actually compare, but Tim Costa's talk seemed to be suggesting that legate has performance advantages over the graph-based llvm code generation that XLA/JAX uses

# ? Apr 15, 2021 11:51

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Can someone tell me a little bit about how they personally utilize Docker containers, for personal or work projects?

I have a few months before I'm going to start job hunting again. I have some experience using Python professionally for data analysis and basics visualizations but I feel like I'm missing the train on containers. I understand the gist of them but I'm trying to think of how a healthcare data team might use them in day to day work.

# ? Apr 17, 2021 03:29

QuarkJets: Sep 8, 2008

Hughmoris posted:

Can someone tell me a little bit about how they personally utilize Docker containers, for personal or work projects?

I have a few months before I'm going to start job hunting again. I have some experience using Python professionally for data analysis and basics visualizations but I feel like I'm missing the train on containers. I understand the gist of them but I'm trying to think of how a healthcare data team might use them in day to day work.

They are ideal for version control, even beyond what a venv or conda environment provides. So you can strictly define the operations or development environment that you want to use and you won't clash with the requirements that anyone else might have. And you can basically guarantee reproducibility of results, too

So Application A and Application B can run on the same system even if they have conflicting runtime requirements. And you can create dev environments that don't conflict with the host system

# ? Apr 17, 2021 03:52

Hed: Mar 31, 2004; Fun Shoe

Yep, I use �em now even for some relatively simple pure python programs. Controlling the input (it�s actually easier to muck with ports) and output (easy to use logging with stdout and have the log shipper be external to the program).

# ? Apr 17, 2021 03:57

QuarkJets: Sep 8, 2008

And the scalability is important too, but maybe not important in a medical setting. If you use a container then your software can be almost agnostic to choice of platform, operating system, etc. There are caveats of course but it's a very convenient kind of virtualization

# ? Apr 17, 2021 06:05

putin is a cunt: Apr 5, 2007; BOY DO I SURE ENJOY TRASH. THERE'S NOTHING MORE I LOVE THAN TO SIT DOWN IN FRONT OF THE BIG SCREEN AND EAT A BIIIIG STEAMY BOWL OF SHIT. WARNER BROS CAN COME OVER TO MY HOUSE AND ASSFUCK MY MOM WHILE I WATCH AND I WOULD CERTIFY IT FRESH, NO QUESTION

I hated python (undeservedly) on principle for a long time and now I've been forced to learn it and I gotta say, I'm a total convert. So greetings, Python thread.

# ? Apr 17, 2021 10:24

putin is a cunt: Apr 5, 2007; BOY DO I SURE ENJOY TRASH. THERE'S NOTHING MORE I LOVE THAN TO SIT DOWN IN FRONT OF THE BIG SCREEN AND EAT A BIIIIG STEAMY BOWL OF SHIT. WARNER BROS CAN COME OVER TO MY HOUSE AND ASSFUCK MY MOM WHILE I WATCH AND I WOULD CERTIFY IT FRESH, NO QUESTION

Hughmoris posted:

Can someone tell me a little bit about how they personally utilize Docker containers, for personal or work projects?

I have a few months before I'm going to start job hunting again. I have some experience using Python professionally for data analysis and basics visualizations but I feel like I'm missing the train on containers. I understand the gist of them but I'm trying to think of how a healthcare data team might use them in day to day work.

Using them as a cross platform way for devs to quickly get a working copy of a prod analogue up and running locally is super nice and helpful. Then taking that further, you can actually deploy that image to containers in production using AWS ECS or something similar. If it crashes ECS just starts it up again in a very short timeframe. It will auto scale for you too.

# ? Apr 17, 2021 10:26

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

Just be aware that what docker is optimized for is running on linux hosts where it can take advantage of mechanisms that provide strong per-process isolation and filesystem mounts (or bsd). It is slower on windows where it goes through an emulation layer; it is dogshit slow on OSX where it just runs in a linux vm, and eats a ton of resources, and particularly eats poo poo if you have the temerity to try and share files between the host OS and the VM, and also docker daemon runs a vm while active so it eats a bunch of system resources. It will make everything work cross platform but it won't necessarily make everything work well cross-platform.

# ? Apr 17, 2021 13:42

boofhead: Feb 18, 2021

+n'thing the "learn docker" thing, you'll likely be asked to share your code challenges via docker and if you're still learning that as you're doing the challenge, you won't have a great time of it. I.e. me, wednesday evening, the night before the code challenge is due, staring at a "entrypoint.sh file or folder not found" error for the millionth time while trying to build with docker compose to share to dockerhub when everything had been working perfectly fine on my local machine up til that point. I'd been using the docker files locally just fine and I naively assumed that it would then just continue working, but uh.. nope

whoops

# ? Apr 18, 2021 08:44

Wallet: Jun 19, 2006

Hughmoris posted:

Can someone tell me a little bit about how they personally utilize Docker containers, for personal or work projects?

I'm not sure exactly what you're looking for but I use Docker to deploy the APIs I develop for work to GCP because deploying anything like that without containers these days is a special kind of madness. I probably haven't actually touched Docker in six months at this point as once I got everything set up correctly with Cloud Build etc it just kind of handles itself. Merge into the staging branch and it auto builds and deploys within ~120 seconds to our staging environment�production works the same way (with some additional merge protection).

You can generally find a good base image suitable for whatever it is you want your container to be doing and have it up and running within a couple of hours once you have a basic idea of how Docker works. Unless you want to dig real deep I'd probably start with a prepared image like that while learning the ropes instead of trying to do everything from scratch.

It does feel kind of silly to me that people are using Docker as a way to dodge issues with running code across different Python environments because it feels like a massive hammer for what shouldn't be a massive nail, but I guess that's just where Python is at for now.

As an aside, PyCharm has pretty good Docker support (and everything else support, really).

Phobeste posted:

Just be aware that what docker is optimized for is running on linux hosts where it can take advantage of mechanisms that provide strong per-process isolation and filesystem mounts (or bsd). It is slower on windows where it goes through an emulation layer; it is dogshit slow on OSX where it just runs in a linux vm, and eats a ton of resources, and particularly eats poo poo if you have the temerity to try and share files between the host OS and the VM, and also docker daemon runs a vm while active so it eats a bunch of system resources. It will make everything work cross platform but it won't necessarily make everything work well cross-platform.

Also this is definitely a thing. Docker is incredible for containerized deployment and pretty poo poo for running things locally. I can't imagine trying to develop against it instead of using a venv.

a hot gujju bhabhi posted:

I hated python (undeservedly) on principle for a long time and now I've been forced to learn it and I gotta say, I'm a total convert. So greetings, Python thread.

Happy to see a convert but I'm sort of interested in what your opposition to Python was originally if you're willing to expand. I've met a number of devs who seem to disdain it out of hand but I've never gotten any substantial insight into what it is that offends them about it. The syntax and idiom of Python has always felt right to me but it almost feels like some folks feel like it's wrong for a programming language to care about how it looks (even though that has a major impact on how easy it is to parse).

# ? Apr 18, 2021 13:34

SurgicalOntologist: Jun 17, 2004

I do have a personal project that I develop against docker. If it was just the one conatainer I probably wouldn't bother, but once I had a docker compose set up with rq, Cloud SQL Proxy and some other dependencies, it was easier to use it for development than install those things "natively" (although that wouldn't be hard either). For development I just volume mount the source code directory, there are probably better ways but it works well for a Python flask project at least.

# ? Apr 18, 2021 16:12

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

If you develop python to end up on AWS Lambda, testing it locally in docker using AWS SAM owns.

Lets you run an AWS Lambda like environment in docker on your desktop, but you can remote into it and everything. Pretty great/easy to set up and use. I never developed an understanding of docker because following the SAM setup instructions worked for me.

CarForumPoster fucked around with this message at 17:44 on Apr 18, 2021

# ? Apr 18, 2021 17:36

QuarkJets: Sep 8, 2008

Wallet posted:

It does feel kind of silly to me that people are using Docker as a way to dodge issues with running code across different Python environments because it feels like a massive hammer for what shouldn't be a massive nail, but I guess that's just where Python is at for now.

That's just another way of saying that containers are being used to decouple Python applications from host environments, which is basically Docker's mission statement. You can accomplish something similar with a venv or conda environment if you only want to isolate an environment on a single host, but a container comes with a lot of other upside.

# ? Apr 19, 2021 12:43

Wallet: Jun 19, 2006

QuarkJets posted:

That's just another way of saying that containers are being used to decouple Python applications from host environments, which is basically Docker's mission statement. You can accomplish something similar with a venv or conda environment if you only want to isolate an environment on a single host, but a container comes with a lot of other upside.

They do, yes, but when you have to use Docker just so you can run someone's script on a machine other than the one they wrote it on that seems like a lot of faff for what should be a simple thing.

# ? Apr 19, 2021 14:09

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Wallet posted:

They do, yes, but when you have to use Docker just so you can run someone's script on a machine other than the one they wrote it on that seems like a lot of faff for what should be a simple thing.

The wind whispers through the trees ^{~*use a web app*~}

# ? Apr 19, 2021 14:19

QuarkJets: Sep 8, 2008

CarForumPoster posted:

The wind whispers through the trees ^{~*use a web app*~}

If anything, web-based applications are even more prone to getting containerized so as to be platform agnostic and more easily scalable.

# ? Apr 19, 2021 14:32

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

QuarkJets posted:

If anything, web-based applications are even more prone to getting containerized so as to be platform agnostic and more easily scalable.

Yea they def should be containerized IME. I learned "develop on X, deploy to Y" lessons the hard way...more than once. I more mean you dont need users to have docker or anything but mostly I'm just trollin from before.

# ? Apr 19, 2021 14:45

QuarkJets: Sep 8, 2008

CarForumPoster posted:

Yea they def should be containerized IME. I learned "develop on X, deploy to Y" lessons the hard way...more than once. I more mean you dont need users to have docker or anything but mostly I'm just trollin from before.

We weren't even talking about users, we were talking about why applications tend to get containerized when you Kramer'd into the discussion with a non sequitur

# ? Apr 19, 2021 23:54

Da Mott Man: Aug 3, 2012

If your application is developed well you should be able to run it with or without containerization. Docker or Kube etc just make everything cleaner, easier to figure out what went wrong, and faster to scale.

# ? Apr 21, 2021 07:45

Hughmoris: Apr 21, 2007; Let's go to the abyss!

What libraries should I be looking at if I want to automate form entry for something that I suspect is heavily javascript?

I don't have enough web dev knowledge but it's not a typical form where you can inspect a form field and see <input>. It's the C-herwell ticketing system, their web portal. 99% percent of the time I'm submitting a ticket, I'm clicking the same drat things.

# ? Apr 26, 2021 21:38

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Hughmoris posted:

What libraries should I be looking at if I want to automate form entry for something that I suspect is heavily javascript?

I don't have enough web dev knowledge but it's not a typical form where you can inspect a form field and see <input>. It's the C-herwell ticketing system, their web portal. 99% percent of the time I'm submitting a ticket, I'm clicking the same drat things.

It sounds like this is a form in a browser and I'm assuming your only interested in a python solution as opposed to like...a password manager browser plugin that has this capability. The way to interact with browsers like you're a human via python is usually selenium. Theres other ways I could maybe go into but selenium that you write code for a jupyter notebook would give you a pretty flexible way to interact with stuff in a browser.

That said it sounds like what you actually want is a browser plugin.

# ? Apr 26, 2021 23:00

Cyril Sneer: Aug 8, 2004; Life would be simple in the forest except for Cyril Sneer. And his life would be simple except for The Raccoons.

I have a situation where I'm extracting two sets of byte arrays from a larger byte array buffer as follows:

code:

msb = np.frombuffer( data[0::2] , dtype=np.int8, count=64)
lsb = np.frombuffer( data[1::2] , dtype=np.int8, count=64)

At this point, msb and lsb are two bytearray objects of length 64. What I want to do is somehow "match" the two bytes together to form a 16 bit integer. That is, by first 16 bit integer would be formed by merging msb[0] and lsb[0]. The second 16 bit integer would be formed by merging msb[1] and lsb[1], etc. Any clever ways to do this?

Edit:

Even better, can I directly interpret my original 128-byte byte array at 64 16-bit integers, rather than my split-and-combine approach? Looking through the struck package, there are some unpack functions that seem to be what I'm looking for?

Cyril Sneer fucked around with this message at 17:56 on Apr 30, 2021

# ? Apr 30, 2021 17:48

OnceIWasAnOstrich: Jul 22, 2006

Cyril Sneer posted:

I have a situation where I'm extracting two sets of byte arrays from a larger byte array buffer as follows:
code:
msb = np.frombuffer( data[0::2] , dtype=np.int8, count=64)
lsb = np.frombuffer( data[1::2] , dtype=np.int8, count=64)
At this point, msb and lsb are two bytearray objects of length 64. What I want to do is somehow "match" the two bytes together to form a 16 bit integer. That is, by first 16 bit integer would be formed by merging msb[0] and lsb[0]. The second 16 bit integer would be formed by merging msb[1] and lsb[1], etc. Any clever ways to do this?

Edit:

Even better, can I directly interpret my original 128-byte byte array at 64 16-bit integers, rather than my split-and-combine approach? Looking through the struck package, there are some unpack functions that seem to be what I'm looking for?

I am missing something. Why can't you just frombuffer() as a np.int16 dtype? Is the original data two 8-bit signed ints sequentially or something? If its something like that, just cast to 16-bit, multiple your more-significant part by the appropriate factor (or bit-shift), then add them together.

OnceIWasAnOstrich fucked around with this message at 18:25 on Apr 30, 2021

# ? Apr 30, 2021 18:22

Cyril Sneer: Aug 8, 2004; Life would be simple in the forest except for Cyril Sneer. And his life would be simple except for The Raccoons.

OnceIWasAnOstrich posted:

I am missing something. Why can't you just frombuffer() as a np.int16 dtype? Is the original data two 8-bit signed ints sequentially or something? If its something like that, just cast to 16-bit, multiple your more-significant part by the appropriate factor (or bit-shift), then add them together.

Thanks, yep, I figured out how to do it with both frombuffer() or struct.unpack().

# ? Apr 30, 2021 20:54

OnceIWasAnOstrich: Jul 22, 2006

If anyone has created a CLI tool with potentially dozens of configurable options before, I have some questions about how you like to structure this. I have inherited a tool with a CLI interface with a moderately complex argparse setup. We have a base parser and 3-4 specialized parsers for different invocations. Right now this gets processed by a main.py with functions run for the different CLI invocations. These functions do the basic argparse prepartion and parsing, and simply invoke a specific function in another module with all of the parsed arguments given to the function as named keyword arguments.

This means that the function signatures for those actual core processing functions are massive. Hypothetically someone might use them without the CLI interface from Python, so they have the full set of keyword arguments with the same default values as are set in argparse, although that isn't enforced in any way by code and just feels bad. How would you go about structuring this?

I could hand off the arguments to the function instead and require anyone using it via Python to manually construction a Namespace with the appropriate values, which makes that basically a non-starter because it would have to be done manually and doesn't benefit from any defaults. I would end up pulling specific values out of the namespace in that function but anyone looking at it would have to go backward and consult the argparse code to figure out what is available and what the defaults are. This is easy enough, just replace all the arguments with a single Namespace arg, and args. in front of every usage of one. It does break a lot of my nice IDE features because it can't know what is and isn't going to be in the Namespace.
I could create some sort of config class that gets instantiated with all of the arguments and passed around, although that just results in my duplicated argument definitions moving from a function to a dedicated class, which would still probably be an improvement. Would make it easier to check for errors because then its easy to check if something should actually exist in the class.
I could automatically create a config class with some code that constructs something from the argparse interface so that there is no duplication but then people who aren't insane will have no idea what my weird meta-code is that is constructing that class and I'm not sure how to get it documented properly. This always makes me feel smart but is terrible. Probably still won't help with pre-runtime inference of what items are actually in that class.
I could hypothetically have the main.py function split the arguments into more-logical groups and merge them into dicts or config classes based on what they end up being used for as some sort of hybrid so I don't need a monolithic config class. I'm not sure this solves any problems though, just moves some code around, maybe in a helpful way.

Is there some sort of intended solution that exists for this?

# ? May 2, 2021 17:17

NinpoEspiritoSanto: Oct 22, 2013

I'd personally look at click and refactor the mess you describe to that.

# ? May 2, 2021 18:25

QuarkJets: Sep 8, 2008

A common way to reduce the length of a function signature is to define a dataclass that contains the configurable parameters, then the function just accepts an instance of that class. If multiple functions can use the same dataclass then that's great, or you can define multiple dataclasses too - it's often better to have tailored instances than trying to create one big class that does everything

QuarkJets fucked around with this message at 23:54 on May 2, 2021

# ? May 2, 2021 23:51

OnceIWasAnOstrich: Jul 22, 2006

QuarkJets posted:

A common way to reduce the length of a function signature is to define a dataclass that contains the configurable parameters, then the function just accepts an instance of that class. If multiple functions can use the same dataclass then that's great, or you can define multiple dataclasses too - it's often better to have tailored instances than trying to create one big class that does everything

Yesss. Dataclasses were what I was thinking of but couldn't remember exactly because I have (regretfully) only used them once or twice a while back. It ends up basically identical to using the argparse-returned Namespace directly as far as the function is concerned, but it will require the same level of argument definition duplication unless I do something like generate the argparse parser from the dataclass. It does solve my problem of having it extremely difficult to use the functions outside of the CLI though. Click seems nice enough but mostly as a better argparse replacement, and I'm not struggling with that aspect right now, the actual interface is pretty nice and simple at the moment, outside of the huge number of optional arguments. Maybe it has a feature that would help me avoid the huge function signature, but it seems like it would just make me have a huge signature in addition to a huge number of decorators all stacked in the same place.

OnceIWasAnOstrich fucked around with this message at 01:14 on May 3, 2021

# ? May 3, 2021 01:10

Neslepaks: Sep 3, 2003

I'd probably just populate a big dict and pass that around because I'm a giant ape

# ? May 3, 2021 16:54

QuarkJets: Sep 8, 2008

Python classes are just dicts under the hood, so both approaches are really just big dicts

Python has that Big Dict Energy

# ? May 3, 2021 23:02

Deadite: Aug 30, 2003; A fat guy, a watermelon, and a stack of magazines?
Family.

Does anyone have experience running Dask on AWS? I'm trying to start a dask cluster on an EC2 instance but the dask scheduler never seems to start. I'm using EC2Cluster from dask-cloudprovider. Here's what I see on PuTTy when the cluster context manager executes:

code:

Creating scheduler instance
Created instance i-0be36abee1a3206c9 as dask-c958e0dc-scheduler
Waiting for scheduler to run at ip:port

and that's it. So the scheduler never starts running and no workers are created, but I'm not getting any errors or anything that I can trace back to a problem. Google is getting me nowhere with this issue either, so can anyone tell me what I'm doing wrong?

# ? May 4, 2021 03:57

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 17:54

Bourricot: Aug 7, 2016

I'm currently doing some Python for a personnal toy project. I'd like to use this as an opportunity to learn how to do things properly (re linting, auto-formatting...). So I'm asking for some goonpinions on the tools to do Python properly in 2021 (I'm already using mypy* and pylint):
- What is the recommended choice for code formatting: black? flake8? something else?
- Since it's going to be a basic script, I figure venv and a requirements.txt are good enough for my usecase. Am I mistaken?
- Any other pitfalls I might not be aware of?

*I know Python works well without static typing, but I wanted to give it a try.

# ? May 17, 2021 08:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »