|
Quick question about error handling - I'm doing some DB interaction using pandas methods that's reliant on boto for some AWS credentials handling. When I get an error, pandas always seems to return a pandas.io.sql.DatabaseError, no matter what the specific problem is. If I gently caress up my actual SQL I get this response: code:code:1. Parse the error message as a string and look for key phrases - ie "Unable to locate Credentials" or "Queries of this type are not supported". This feels gross, would break if an update changed the error messages, might not handle all error types. 2. Use the __context__ field to examine the chain of exceptions and look for something more specific - something like: code:code:Is there a better way to go about this?
|
#
?
Jan 10, 2019 20:02
|
|
|
|
| # ? May 14, 2024 04:06 |
|
|
If they dont provide more specific error types you are pretty much screwed. Both the error messages, and the nested errors may change under you. Imo, the error messages look more stable, cause noone ever changes them (case in point - enotty)
|
|
#
?
Jan 10, 2019 23:28
|
|
|
Thermopyle posted:I'd go with Docker. All our python code runs in containers at this point, especially the flask stuff.
|
|
#
?
Jan 11, 2019 01:25
|
|
|
DarthRoblox posted:
Good libraries provide SomeLibraryException.code set to some constant like SomeLibraryException.NO_RESPONSE that you can check. Then you have the 99% of libraries that don't do this  .
|
|
#
?
Jan 11, 2019 01:33
|
|
|
Thermopyle posted:I'd go with Docker. Why? They're already planning to deploy to a VM. A single docker image makes no sense on a VM (unless someone wants to educate me?) I realise docker images aren't solely intended for baremetal deployment, but a single app deployed to a VM doesn't sound like something that needs another layer of virtual encapsulation to get the job done, other than it's a Cool Tool. Depending on knowhow you can range from just sftp-ing code to the VM, to things like gitolite that mean you can push to a remote git that'll check out the code you push to a "live" location, to uploading a tarball to creating an ansible script that does it all for you. Ultimately, the question is "how do I get files from location A to location B and have them accessible from location B". This comes down to your own skills and what you're comfortable with. There's no point chatting about docker or various deployment tools if they just go over the head of the person trying to get something live as a learning experience. Personally, Grump, I'd be asking what your ultimate goal and your current knowledge is. You say you're just a clueless front end person, but you've put a flask app together so not that clueless, where does your comfort zone live? How are you with Linux? SFTP? SSH? Nginx/Apache? etc.
|
|
#
?
Jan 11, 2019 02:42
|
|
|
Bundy posted:Why? They're already planning to deploy to a VM. A single docker image makes no sense on a VM (unless someone wants to educate me?) Identical environment on dev and production and Docker is the easiest way to that end goal. Whether you're deploying to a VM or not is a red herring. Thermopyle fucked around with this message at 02:58 on Jan 11, 2019 |
|
#
?
Jan 11, 2019 02:45
|
|
|
Bundy posted:Personally, Grump, I'd be asking what your ultimate goal and your current knowledge is. You say you're just a clueless front end person, but you've put a flask app together so not that clueless, where does your comfort zone live? How are you with Linux? SFTP? SSH? Nginx/Apache? etc. I'm pretty comfortable with Linux and Apache, but that's basically the extend of my knowledge. I've deployed personal sites and React apps on Apache without too much trouble. I have no issue securing servers and getting something on the web, in a nutshell. This is really just a pet project, so I'm all for learning new tech. I didn't know any Flask / Python before this. Docker is still a mystery to me, and I've watched quite a bit of tutorials - none of that knowledge ever sticks with me. If I'm going to learn Docker, I need a tutorial that's going to spell everything out for me very clearly - I have yet to find any good ones and I even have a Pluralsight subscription teen phone cutie fucked around with this message at 04:18 on Jan 11, 2019 |
|
#
?
Jan 11, 2019 04:12
|
|
|
Docker is much easier than you think...once the concept behind it *clicks* in your brain. Just devote a morning to really messing around with Docker and going through multiple tutorials. The payoff once it clicks for you is worth it in the end for a lot of reasons. One way of thinking about Docker that might help it click for a developer: Think of containers like objects/classes but they encapsulate a whole application and everything needed for it to run instead of encapsulating data and code. And, just like with objects, you don't have to think about the nitty-gritty of how it works internally, just the functionality that the container/object exposes...aka which network ports and possibly where it writes data to. So, with a Flask application, it's very possible that the only public "API" is port 5000. The container hides away the operating system, python, everything. Then you might have the same thing for MySQL, and the two containers talk to each other. Nginx will reverse proxy requests coming in to (say) port 80 on the host machine to port 5000 which Docker has grabbed hold of for your Flask application, and while processing the request, Flask accesses MySQL on whatever port you set up when running the container. The big and important benefit of this is that it's super simple to run absolutely identical containers on your work computer and where the heck ever else you want to. There's no messing with trying to run multiple python versions or installing MySQL in multiple versions. If you can install Docker, you can run your containers, which have encapsulated the version of Python and the version of MySQL you're developing against.
|
|
#
?
Jan 11, 2019 04:40
|
|
|
It's super simple yeah. Install docker, make sure you have all your requirements in a requirements.txt and go find probably the official python 3.7 docker file and then do a docker build . -t hadlock/myapp: latest Then do a docker run -itd -p 5000:5000 --name=myapp hadlock/myapp:latest and then go browse to 0.0.0.0:5000 ... You can tail the logs via docker logs -f myapp Probably messed up some of the syntax due to being on a phone but that's really it. Then you can push the container to a remote registry and you can pull it down, libraries and all already correctly preconfigured along with the correct version of python etc etc... run that exact same thing anywhere inside your firewall, works exactly like it did on your dev laptop. No custom bullshit ansible that's slightly different than the ansible your co-worker wrote to deploy that other python app. Side load in config like, -v /localpath/to/application.yaml:/containerapp/application.yaml And then just jump into the app container like, docker exec -it myapp bash Hadlock fucked around with this message at 05:03 on Jan 11, 2019 |
|
#
?
Jan 11, 2019 05:00
|
|
|
baka kaba posted:dict1.update(dict2) basically adds all the key/value pairs from dict2 into dict1. So it'll overwrite any keys that already exist I ended up just having a list inside a dict. Noticed right after I posted. Thanks!
|
|
#
?
Jan 11, 2019 20:45
|
|
|
Thanks for the help guys. I think I'm going to look into some Docker tutorials and report back. On the same note, I'm having some issues with how I'm getting my data back from one of my endpoints, specifically with Marshmallow and SQLAlchemy https://gist.github.com/martinmckenna/eb5eeee5869663fc8f2e52a5e7ef72c9 I have a many-to-many relationship between cocktails and ingredients, but I also have more data than just foreign keys on the relational table, ings_in_cocktail, such as "ounces." When I GET /cocktails/, it returns something like this: code:I want the data to look like the following: code:teen phone cutie fucked around with this message at 02:52 on Jan 14, 2019 |
|
#
?
Jan 14, 2019 02:49
|
|
|
My wife is trying to pick up programming, and I thought Python was a good language to start with because of its whitespace rules and I'm assuming a lot of immutability. Does anyone have a good course or book recommendation to start with? I've been a dev for 7 years now so I can help fill in the blanks a bit, but I don't work with Python at all so something comprehensive and easy to understand would be best. Also what's being used for Python web projects? Is it Flask?
|
|
#
?
Jan 17, 2019 17:51
|
|
|
Modest Mouse cover band posted:
I've seen this conversation a million times now, and if you're interested in doing web dev stuff with Python you should be learning Django.
|
|
#
?
Jan 17, 2019 18:28
|
|
|
punished milkman posted:I've seen this conversation a million times now, and if you're interested in doing web dev stuff with Python you should be learning Django. yah, i do flask all day every day but thats cuz i know wtf im doing flask is simpler but it gives you lots less poo poo
|
|
#
?
Jan 17, 2019 18:31
|
|
|
punished milkman posted:I've seen this conversation a million times now, and if you're interested in doing web dev stuff with Python you should be learning Django. Awesome, thank you I knew there was another one. I found this and she already likes it better than CodeAcademy: https://tutorial.djangogirls.org/en/ plus she has a chromebook so it saves VM setup time for the moment.
|
|
#
?
Jan 17, 2019 18:35
|
|
|
Django or Pyramid are probably the frameworks to pick up for things that are somewhat sane. https://greenteapress.com/wp/think-python-2e/ Think Python is great for beginners.
|
|
#
?
Jan 17, 2019 19:08
|
|
|
For Django specifically, once youve done the tutorials (of which there are many excellent ones suxh as the one youve already found) the Two Scoops of Django book makes an excellent reference book. Its expensive in hardcopy form but I think you can get an electronic copy cheaper
|
|
#
?
Jan 17, 2019 19:54
|
|
|
Anyone know of a good project that can webscrape for pricing information? I was thinking beautifulsoup mostly. Just have a few pages from vendors I setup watches on (like camelcamelcamel but cross vendor) I’m sure this has probably been done already a million times so if I could just sponge off one and check out the code that’d be super.
|
|
#
?
Jan 17, 2019 22:12
|
|
|
priznat posted:Anyone know of a good project that can webscrape for pricing information? I was thinking beautifulsoup mostly. Just have a few pages from vendors I setup watches on (like camelcamelcamel but cross vendor) There's also scrapy iirc. Automate the boring stuff has some good stuff on web scraping.
|
|
#
?
Jan 18, 2019 02:16
|
|
|
Modest Mouse cover band posted:My wife is trying to pick up programming, and I thought Python was a good language to start with because of its whitespace rules and I'm assuming a lot of immutability. I guess I don't exactly know what you mean by this, but when I think of python I don't think "immutability."
|
|
#
?
Jan 18, 2019 04:59
|
|
|
So I've been playing with/now seriously considering contributing to a newish Python library called Trio It's a truly excellent approach to async programming in Python, based off ideas from Curio, Dave Beazley's hobby async project that began life when he did a talk on asyncio at Pycon Brasil after Python added async/await syntax to the core. I've rehosted the asyncio: A dumpster fire of bad design rant as the original site has been down for some time (I've let the original author know, author information is still in the left untampered html). This gives a decent, angry rundown of what's wrong with asyncio (mostly still today). Trio's author and now Python core developer Nathaniel J Smith has some great blog posts on different approaches to async now that async/await were added to the language (predates Trio, uses Curio as examples), as well as some very good post-Trio reasoning about just what's wrong with the traditional approach to async programming in Python here. Edit: There's also Andre Caron's take on NJ Smith's reasoning about asyncio. Last night I had a use case for checking this out and after about a half hour I'd knocked together a script to check a blacklist, then validate, check MX record and attempt to see if the mailbox exists for 10k email addresses to help filter out spam/invalid vs legit emails on a subscriber list for a friend of mine, total runtime was 3 minutes and change. I highly recommend having a read and a play, even if it's not mature enough for anything you're working on yet (it does lack some native libraries, but does provide a bridge for asyncio focused libs). Or hell if you're bored, join the project/contribute! If you prefer to watch presentations on such things, NJSmith did a decent talk at Pyninsula, with the interesting bits being how you can implement the Happy Eyeballs algorithm in Trio vs Twisted and asyncio: https://www.youtube.com/watch?v=i-R704I8ySE NinpoEspiritoSanto fucked around with this message at 13:59 on Jan 18, 2019 |
|
#
?
Jan 18, 2019 13:36
|
|
|
priznat posted:Anyone know of a good project that can webscrape for pricing information? I was thinking beautifulsoup mostly. Just have a few pages from vendors I setup watches on (like camelcamelcamel but cross vendor) ParseHub is a non programming (its firefoxes element picker for scraping best I can tell) solution that I found pretty easy to use after a few minutes of their tutorial videos. Has some limitations but if you want very quick and dirty, I like it and its free for up to 200 pages at once. You can load like search terms or whatever straight from a csv or json and loop over them which is nice.
|
|
#
?
Jan 18, 2019 14:24
|
|
|
trio>asyncio but then you have another dependency, which isn't worth it for some
|
|
#
?
Jan 18, 2019 21:20
|
|
|
Thermopyle posted:trio>asyncio Eh, maybe but most apps or devs worth their salt don't just rely on stdlib, considering the horror that is datetime, urllib, etc. Trio will end up to async as requests is to urllib, attrs to classes and pendulum to datetime, IMHO. Stdlib has some nice things, but is hampered in some ways when it comes to libraries as you're stuck on that release cycle. I'd take an extra line in requirements.txt over wrestling with asyncio, it's still somewhat painful even with the new syntax.
|
|
#
?
Jan 18, 2019 21:29
|
|
|
Bundy posted:Eh, maybe but most apps or devs worth their salt don't just rely on stdlib, considering the horror that is datetime, urllib, etc. Trio will end up to async as requests is to urllib, attrs to classes and pendulum to datetime, IMHO. Right, but there's a difference between not just relying on stdlib and adding an extra dependency. There's costs beyond the few extra bytes in requirements.txt to every dependency. Some dependencies are worth it for some projects and not for other projects. I'll quote myself here talking specifically about django packages, but the same holds in general... me posted:When and how to use other libraries is a hard question to answer. You can have a large application where very little of the logic has anything to do with async (in fact, I wouldn't be surprised if this holds true for most non-trivial applications). There's no point in tying yourself to a third party no matter how much easier they make async when you've got 10 lines of code devoted to it. Like I said, trio is definitely better than asyncio, which is why I've contributed a lot bug-fixing and features. However, people can be too eager to try the hot new library without considering the consequences...which is part of the reason you end up with the leftPad fiasco in Node-land.
|
|
#
?
Jan 18, 2019 22:33
|
|
|
What is the practical difference between attrs.attrib(init=False) and attrs.attrib(default=None)?
|
|
#
?
Jan 19, 2019 12:28
|
|
|
Boris Galerkin posted:What is the practical difference between attrs.attrib(init=False) and attrs.attrib(default=None)? One effects how __init__() is handled when initialising the class: https://www.attrs.org/en/stable/init.html The other is for specific value defaults: https://www.attrs.org/en/stable/init.html#defaults
|
|
#
?
Jan 19, 2019 14:53
|
|
|
Thermopyle posted:Docker is much easier than you think...once the concept behind it *clicks* in your brain So I dockerized my Flask app and have everything working locally no problem. Now how do I get this thing on a Linux box? I can't find one tutorial online about this. I'm at the point where running docker-compose will build my MySQL and app containers, but I have no idea what is required to get me a working app on a server teen phone cutie fucked around with this message at 09:20 on Jan 20, 2019 |
|
#
?
Jan 20, 2019 09:06
|
|
|
Well, what exactly are you having trouble with  ? Install Docker & Friends on your server, and run docker compose there. Sprinkle your compose.yml with `restart: always` so it can unshit itself if something happens and you are more or less good to go (well, that and obviously mount your app data in the external (server) fs).
|
|
#
?
Jan 20, 2019 09:25
|
|
|
cinci zoo sniper posted:Well, what exactly are you having trouble with Like.....just run git clone my-project in the root directory on the server and then run docker-compose? And curling the IP will just work? quote:(well, that and obviously mount your app data in the external (server) fs). uhhhhhh what? teen phone cutie fucked around with this message at 09:28 on Jan 20, 2019 |
|
#
?
Jan 20, 2019 09:26
|
|
|
I see. Read 1) https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-centos-7 2) https://www.digitalocean.com/community/tutorials/how-to-work-with-docker-data-volumes-on-ubuntu-14-04 for the Docker stuff then. But yeah you git clone your project into your server (I would not do it in "server root" (home directory), rather I prefer doing things like these in /opt/ but that's a matter of preference to a large extent), then `docker-compose up -d`. Assuming you configured restart behaviour and volumes correctly in compose.yml, the app will persist through docker and server (provided that you made Docker a service) restarts, or crashes if those happen (obviously depends on how hard and how much crashed). Without restart configuration it will stay dead, without volumes you will lose all data accumulated by app(s) - while that is far from always necessary, you mention MySQL container so you probably don't want that database to lose all data in it if anything happens. About curling, can you elaborate more? Also, figure out what firewall your server has and look up some configuration guide(s) on DigitalOcean, they have pretty good basic reference material for most things.
|
|
#
?
Jan 20, 2019 09:39
|
|
|
cinci zoo sniper posted:I see. Read wait holy poo poo...I just spun up a server really quick and followed these steps. It just works. Navigating to http://ip-address-here:5000 just works.....That's crazy it's like magic
|
|
#
?
Jan 20, 2019 10:04
|
|
|
Yep, that's the selling point of Docker and container tech in general. The solutions are portable, and if it works somewhere, it works [almost] anywhere.
|
|
#
?
Jan 20, 2019 10:08
|
|
|
One more question, which is probably better suited for another thread - what's the easiest way to get an SSL for a docker Flask app?
|
|
#
?
Jan 20, 2019 10:32
|
|
|
Make sure you read some stuff on securing your new server as well. Nobody is going to specifically target you, but they will throw out wide nets and see who they can get. Last thing you want is someone using your server in their bitcoin mining operation. https://www.digitalocean.com/community/tutorials/7-security-measures-to-protect-your-servers E: the most important thing there is to use ssh key authentication, disable password logins, and especially disable root logins. Everything else is secondary. Boris Galerkin fucked around with this message at 10:38 on Jan 20, 2019 |
|
#
?
Jan 20, 2019 10:34
|
|
|
Grump posted:One more question, which is probably better suited for another thread - what's the easiest way to get an SSL for a docker Flask app? letsencrypt provide free certs that you can renew automatically every 3 months
|
|
#
?
Jan 20, 2019 10:35
|
|
|
Grump posted:One more question, which is probably better suited for another thread - what's the easiest way to get an SSL for a docker Flask app? A few ways would be setting nginx up in front of Docker or just adding appropriate SSL context for your Flask app (e.g. like so).
|
|
#
?
Jan 20, 2019 10:39
|
|
|
cinci zoo sniper posted:A few ways would be setting nginx up in front of Docker or just adding appropriate SSL context for your Flask app (e.g. like so). I'd probably want to set up nginx in front of it if I want to be able to visit domain.com without having to do the additional :5000 at the end right?
|
|
#
?
Jan 20, 2019 10:57
|
|
|
You should use something other than the built in http server to talk to the world at large, yes. I'm on my phone, but search for Chris Warrick nginx uwsgi. He has a great guide for deploying python apps along with nginx. Even has an ansible book if that's your thing.
|
|
#
?
Jan 20, 2019 11:00
|
|
|
|
| # ? May 14, 2024 04:06 |
|
|
Grump posted:I'd probably want to set up nginx in front of it if I want to be able to visit domain.com without having to do the additional :5000 at the end right? Not necessarily, but I would roll with Nginx setup regardless, it's industry standard for this use case. Initial setup can be a bit of a hurdle, but once you are past that it's smooth sailing and will open a lot of options for you.
|
|
#
?
Jan 20, 2019 11:05
|
|