Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

M. Night Skymall: Mar 22, 2012

Hadlock posted:

I'm getting ready to write, at least two microservices for deploying and maintaining state of an app, probably another 3-5 microservices related to deploying other apps. These will just be serving up json from a standard restful api using basic auth deep inside the protected network and not accessible outside the company. No UI or twitter for zombies, etc.

I'm fairly happy with using Flask at this point. It seems like Flask is the go-to library for microservices? I am looking at using PeeWee as my ORM and then connecting to a postgres db.

Is this a pretty good solution? Peewee seems actively developed despite having a fairly small user base. The big plus for me is that it's especially straightforward.

Other options include Flask/SQLAlchemy, SQLAlchemy being more difficult to use and a lot meatier.

Option C is something in the Django-world. Django has a fair bit of rails-magic to it, and I'm trying to imagine a future where my ops buddies can debug the code without having to call me when I'm on vacation in the bahamas someday.

Am I going to bet happy with Python/Flask/Peewee?

My predecessor used Cherry Py for something similar at another company.

I'm in the process of doing something similar, and I've pretty much also decided Flask is the way to go. I think Django REST Framework looks pretty nice if you need the features it provides like a built-in ORM, more robust user authentication methods, web browsable API, throttling, plenty of other stuff I care even less about. I don't really need any of that though and it doesn't sound like you do either, and Flask is nice since it's a lot easier to integrate whatever ORM you're comfortable with and is just lighter. I think DRF'd be great if I were more familiar with django already, or the API were public facing at all.

# ¿ Mar 26, 2018 17:37

Adbot: ADBOT LOVES YOU

# ¿ May 12, 2024 10:16

M. Night Skymall: Mar 22, 2012

Boris Galerkin posted:

Got it, thanks. I tend to just use PyCharm for writing code for all of the cool features but run everything from a terminal because sometimes I gotta do this remotely through ssh/vim.

PyCharm runs code remotely over ssh too. It'll auto deploy over sftp as well. You can also set up source roots etc remotely but it gets weird and I don't usually bother. Even if I'm running everything locally I still have it ssh into the windows Linux subsystem and use the interpreter there instead of setting everything up in windows.

# ¿ Apr 26, 2018 11:11

M. Night Skymall: Mar 22, 2012

SnatchRabbit posted:

I'm trying to do some port closure commands and the method I'm using is rather particular about how it formats ipv4 and ipv6 cidrs, so I need to run the same command a few times in order to make sure all the ports actually close properly. Ideally, I'd want my for loop to try a number of variations of ipv4 and ipv6 formats but if one fails I just want it to continue through the loop. Essentially, my code looks something like this:
code:
portlist = [1,2,3,4,5]

for port in portlist:
  try:
    closeport(ipv4, ipv6)
else:
  try: 
    closeport(ipv4)
  except ClientError as e: print (e)
  pass
How exactly should my for loop be constructed to try different variations of the ipv4 and ipv6 ports?

Based on your code I guess it throws an exception if it fails to close the port, if that's the case you need to retry in your except clause.

code:

portlist = [1,2,3,4,5]

for port in portlist:
  try:
    closeport(ipv4, ipv6)
  except ClientError:
    try:
      closeport(ipv4)
    except ClientError as e:
      print(e)

You can keep nesting try/except chains until you run out of methods, if you want it to keep going through the for loop even if it fails to close a port just make sure you catch all your port closing related exceptions. Alternatively if it's ok to attempt to close an already closed port you could just put your try/excepts all in a row instead of nesting them.

Most import things are: only catch exceptions related to closing ports so that other exceptions propagate back up, but catch all port related exceptions or you won't make it through your for loop if a port fails to close. If a port failing to close should be an exception then just don't have an except clause on your last port closing try and it'll propagate the exception up like normal.

If retrying the same method is something you need to do then adding the while loop huhu is talking about is a good way to do that, you just catch the exception in the while loop, and might want to add some kind of counter so it doesn't go on forever.

# ¿ Apr 27, 2018 12:23

M. Night Skymall: Mar 22, 2012

The other problem with that code is that the exception you're using the try/except block to catch happens when you convert the input from a string into an integer, you need to move that part inside the try.

code:

#This is a Guess the Number game
import random

print('Hello, what is your name?')
name = input()

print('Well ' + name + ' I am thinking of a number between 1 and 20.')
secretNumber = random.randint(1, 20)

for guessesTaken in range (1, 7):
    print('Take a guess.')

    try:
        guess = int(input())
        if guess > secretNumber:
            print('Your guess is too high.')
        elif guess < secretNumber:
            print('Your guess is too low.')
        else:
            break #This condition is for the correct guess
    except ValueError:
        print ('You did not enter a number.')

if guess == secretNumber:
    print('Good job ' + name + '!') 
else:
    print('Nope. The correct guess was ' + str(secretNumber))

print('You took ' + str(guessesTaken) + ' guesses')

# ¿ Aug 31, 2018 11:44

M. Night Skymall: Mar 22, 2012

SnatchRabbit posted:

I'm writing a function that checks over some db snapshots. First, it is going to check the timestamp on the snapshot. If older than 60 days it will check for some tags. If it finds the tag I want, it will delete the snapshot. If not I want it to give me a skipping message. The function seems to evaluate all my snapshots more or less correctly, the issue I am having is that I only get the "snapshotid does not have Weekly tag, skipping" only shows up when there is a list of other tags present, but not if the tags are completely empty. Is there a relatively simple way to adapt my if statement to accommodate an empty tag list?

I'd just add a check for an empty list like so:

code:

  for snapshot in dbsnapshots:
    dbsnapshotid = snapshot['DBSnapshotIdentifier']
    dbsnapshotcreatetime = snapshot['SnapshotCreateTime']
    dbsnapshotarn = snapshot['DBSnapshotArn']
    timedifference = currentdate - dbsnapshotcreatetime
    if timedifference.days>60:
      print (dbsnapshotid + " is older than 60 days. Checking tags")
      tags = client.list_tags_for_resource(ResourceName=dbsnapshotarn)
      tags = tags['TagList']
      for tag in tags:
        if tag["Key"] == 'DBBackupFrequency' and tag["Value"] == 'Weekly':
          print ('Weekly tag found, Deleting snapshot')
        else:
          print (dbsnapshotid + ' does not have Weekly tag, skipping')
      if not tags:
        print (dbsnapshotid + ' does not have Weekly tag, skipping')
    else:
      print (dbsnapshotid + " is not older than 60 days, skipping")

# ¿ Sep 12, 2018 18:03

M. Night Skymall: Mar 22, 2012

Furism posted:

I have a newbie question. I guess it's more algorithmic than Python-specific but I do use Python.

I'm uploading files to a REST API. I can't do anything with the files while they are being processed so I need to wait for them to be before I can move to the next action I need to do with them. I could just put a "sleep" I guess but I think it's ugly and dirty and won't cover edge cases. For now what I do is this:

* Parse some directory with all the files
* Upload them one by one to the API. Each API calls sends a response with some JSON object that I store in an array.
* Now I want to iterate through that array and check for the status of each file. When the status changed from "processing" to "complete", do something and keep iterating until all the files' status is "complete."

At first I wanted to remove the current entry from the array, but I don't think I can modify an array while I'm moving through it (especially from within a loop?). So I was thinking of creating another array and fill it with the completed JSON objects as they get completed. But that means I'd have to potentially loop through the array an unknown amount of times. I feel that this way of doing it (I'm not even sure I explain it right) is correct.

What would be the best way to solve this little issue?

Use a while loop and iterate through the array until it's empty by removing completed objects, you might still want to use a sleep or some kind of limiter so you aren't hammering the API though.

Edit: I realized this doesn't answer some of the issues you're having. I'd use a manual index into the array, and then just check the boundary at the start of the while loop to see if you're past the end, reset to beginning of array and continue iterating through, once the array is empty you're done. I guess technically you'll skip some entries in your array when you remove an entry, so you could not increment your iterator whenever you remove an item if it's important to you that you check each item once in order.

M. Night Skymall fucked around with this message at 13:46 on Jan 7, 2019

# ¿ Jan 7, 2019 13:39

M. Night Skymall: Mar 22, 2012

FCKGW posted:

I learned python in community college a few years back and I'm going to be going back to school soon and would like to get a refresher course.

Anyone have any online video series they could recommend? Something from Coursersa or Edx or Codeacademy, etc..?

I really liked the mit one on edx.

# ¿ Jun 18, 2019 16:58

M. Night Skymall: Mar 22, 2012

the yeti posted:

academic software is terrifying

Look, we can't all be the financial industry and run our critical services on *checks notes* giant excel spreadsheets.

# ¿ Nov 11, 2019 16:07

M. Night Skymall: Mar 22, 2012

Tortilla Maker posted:

Newbie question:

I'm currently getting the following error:
code:
def generate_df():
    df = pd.DataFrame({'user': ['bob', 'jane'],
                 'income': [20000, 45000]})
    return df
StackOverflow generally emphasized including 'return' but it doesnt seem to help. Other posts suggested placing this in a class but no luck for me either.

Appreciate any suggestions!

Is that the entirety of the code? What line is the actual error on.

# ¿ Mar 10, 2020 19:29

M. Night Skymall: Mar 22, 2012

Tortilla Maker posted:

Thank you all for the quick replies. And apologies for the poor explanation.

Issue is that I was attempting to call the dataframe elsewhere in my code.

For now, I can reference it by using "df = generate_df()" but I was hoping to just use the dataframe as it was defined inside the function (in this example, df).

Writing this on my phone and while on the go, so apologies for the poor example!

What you're confused about is called scope, looking that up and reading about it might help understand why you can't use "df" outside of the function you're defining it in.

# ¿ Mar 10, 2020 20:36

M. Night Skymall: Mar 22, 2012

Whybird posted:

Hi! I'm coming to Python from a more strongly typed language and there's an aspect of BAFP and duck typing that I'm trying to get my head round and which none of the tutorials I've read seem to focus on. It's more a question of what's right in principle than an actual live example.

Supposing I've got a list of objects of varying classes. Some of them have a particular kind of functionality, say, they define a particular method, and some of them don't. I want to run this method on any objects that have it defined, but I'd expect them to be in a minority.

Is it still the right thing to do to loop through all the items in my list and use a try/except clause on each to see if the method exists? It feels like the exception I'm catching in this case isn't really an exception but expected behaviour -- but is that just a question of semantics and actually exceptions can be things you'd expect to happen the majority of the time?

I think it's "pythonic" to use try/except for flow control, that's how I was taught forever ago and googling seems to back me up as that still being the intention. Obviously still up to you if you actually want to program that way, I do a lot of try/except for checking types/existence of things and once you get used to it in Python it's easy to read things written that way.

# ¿ Jul 16, 2020 16:42

M. Night Skymall: Mar 22, 2012

You should probably not use the MRN as the identifier throughout your database. It's generally best practice with PHI to isolate out the MRN into a de-identification table and use an identifier unique to your application to identify the patient. It's easy enough to do a join or whatever if someone wants to look people up by MRN, but it's useful to be able to display some kind of unique identifier that isn't immediately under all the HIPAA restrictions as PHI. Even if it's as simple as someone trying to do a bug report and not having to deal with the fact that their bug report must contain PHI to tell you which patient caused it. Not vomiting out PHI 100% of the time in error messages, things like that. Just make some other ID in the patient table and use that as the foreign key elsewhere.

# ¿ Mar 23, 2021 15:32

M. Night Skymall: Mar 22, 2012

xtal posted:

This isn't actual healthcare code being written in Python on something awful right?

I run a backend for a healthcare reporting tool in Python, would you uh..prefer MUMPS? You really think that's a good language?

# ¿ Mar 23, 2021 15:41

M. Night Skymall: Mar 22, 2012

xtal posted:

I guess I had hoped it was something with correctness guarantees, but the really troubling thing here is asking goons for help with healthcare code

Edit: upon further thought, Python would suck for healthcare too, you'd probably still be using 2.x and Therac-25 someone whenever a time objected tested falsy at midnight

I guess work in healthcare first and then lemme know if you still want to complain about Python of all things, definitely the least of healthcare software's problems. I mean, I doubt anyone's using it to administer drugs with a raspberry Pi, but "I need to aggregate a bunch of data in disparate formats and reason about it" is a very healthcare and python thing to do.

# ¿ Mar 23, 2021 15:47

M. Night Skymall: Mar 22, 2012

Jose Cuervo posted:

Would you have this same concern given the use case (a research database where the only people which access to it are on the IRB protocol - currently just me for now)?

It doesn't matter what the data is used for. The history of MRN as PHI is pretty dumb in my opinion, but per guidance from the government your MRN is as much PHI as your DOB or name. Having a de-identification table isn't a big deal, you can still store all your PHI in the patients table along with your new unique identifier. It's really *just* to remove the awkwardness of having everything in your DB keyed to a piece of PHI. I mean you're right, it's just you and it probably won't affect much now. But making good decisions about your schema is much..much easier now than it is later, and there's basically no way you will live to regret de-identifying your data in advance, and many ways you can live to regret spreading the MRN all over your database.

Anyone who's looking at this going "No no not python!" is just forcing this data to be sucked into a PowerBI tool instead. Don't make this poor person use PowerBI, that's just mean.

Edit: MRN is problematic because it's specifically listed as an identifier. If you have an internal ID in use for patients besides MRN that'd also be better than using the MRN directly. One more dumb thing about MRNs, they're specific to the hospital EMR so they aren't guaranteed to be unique.

M. Night Skymall fucked around with this message at 18:46 on Mar 23, 2021

# ¿ Mar 23, 2021 18:17

M. Night Skymall: Mar 22, 2012

Framboise posted:

As/for someone with very basic programming knowledge who wants to look into programming as a career: Is this Udemy course https://www.udemy.com/course/100-days-of-code/ worth looking into? Seems like a good deal for 63 hours of content, but if it's not good I don't want to put my time into it.*

* Specifically, it's on sale right now for $12.99 which sounds like a good deal, but that price for the bold claims they make sounds kinda sus to me.

Edit: there's also this one https://stacksocial.com/sales/the-2021-premium-python-certification-bootcamp-bundle from StackSocial that claims to be 41 hours of content worth $2585 for only 34.99 and again, sus.

Udemy courses are always "on sale" for like 10-15 dollars or something, regardless of what their claimed price is, so judge all the courses based on their standard pricing being 12 bucks. No idea about that particular one, I've done some Udemy courses and they were useful for getting up to speed on stuff. Highly rated ones like the one you linked tend to be good, but also MIT has a pretty good python course that at least used to be free. I think this one? You can take it on Edx, maybe this one. The edx course should be free if you just want to watch the videos and receive the assignments, you might have to click around a bit to convince it not to charge you. I don't know, it was a lot more obviously free when I did it many years ago.

Either way, Udemy typically just provides video lectures and problems without much interaction/grading, so for something as general as Python you can almost assuredly do just as well with the free offerings from coursera or edx or something I'm pretty sure. In the end it's going to be up to you to actually do the projects and learn things, programming's much more a skill you have to practice than knowledge you memorize or take notes on through lectures.

# ¿ Mar 24, 2021 13:53

M. Night Skymall: Mar 22, 2012

Hughmoris posted:

Anyone ever futz with extending Python using C or Rust to gain performance?

I've been reading some articles on it and the idea seems intriguing but I'm sure the reality can be a headache.

I did a proof of concept where I hooked a rust program into python and got it to import and work, it wasn't too hard really although I forgot how it all worked now. I think it was a little annoying to pass objects back into python, but I also didn't spend that long on it and it was almost 2 years ago so things have probably improved since then. I think the headaches are more around getting it to work on a variety of systems, if you're in control of the environment and you don't care about making it easily distributed to everyone on the planet, it's probably not bad at all. Or I guess just shove it into docker.

# ¿ Apr 4, 2021 14:41

M. Night Skymall: Mar 22, 2012

FISHMANPET posted:

This is probably a very luddite opinion of mine, and I'm sure it's objectively wrong, but I really hate when there are too many functions being called (especially within functions within functions etc). I somewhat frequently find myself tracing very closely through some random code base trying to figure out exactly what's going on, and going deeper and deeper with functions makes that more difficult. You'd think in theory that I should be able to look at a function and say "It's going to do X" the same way I can look at a line like "a = b + c" and know what's going to happen. But in practice, it doesn't work out that way, and I end up having to read through those functions to figure out exactly what's happening.

That's why docstrings and good function names were called out as the most important things earlier, that's what helps you to navigate code bases designed that way. Also good tooling that allows you to jump to/from function definition easily, and get hover over docstrings/types. Modern software engineering best practices make a lot of assumptions about the environment the person reading the code is operating in. That said, people can and often do go way too far with OOP stuff and make it an unreadable mess in the name of "clean code."

# ¿ Feb 27, 2024 19:05

Adbot: ADBOT LOVES YOU

# ¿ May 12, 2024 10:16

M. Night Skymall: Mar 22, 2012

Hughmoris posted:

Rookie question here:

What is the easiest way to convert/print a Jupyter notebook to PDF that will look good? I need to submit it for a class.

https://nbconvert.readthedocs.io/en/latest/usage.html probably.

# ¿ Mar 11, 2024 21:10

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python