|
Thank you! I think that helps a lot. So far, it seems like "module of functions that take the browser object as an argument" matches my use case best, so far, but I'm going to look into the class thing, because I know nothing about it. I suspect that it's not what I need for this, though, because this is basically [browse to this page in a particular system, get this piece of information. Browse to this page, click on these buttons, report back on what happened.] Because it's limited scope and not something that could/would be reused on other websites/services, I can't yet imagine what a class would look like here. As for Serious Business ... I'm not getting paid to do it, yet, but I'd like to learn as if I were so that I can rely on it if I choose.
|
#
?
Jul 28, 2018 03:33
|
|
|
|
| # ? May 16, 2024 06:23 |
|
|
PyCharm and other JetBrains stuff is 50% off. https://www.jetbrains.com/promo/friends/
|
|
#
?
Jul 30, 2018 16:38
|
|
|
Any suggestions for career progression paths in Python/software development generally coming from a data analysis background? I have been using Python for about 18 months, with the last 12 months exclusively as my 9-5. It is my only experience of a programming language so far. The first 6 months were tooling around with pandas and seaborn, working with mid-size time series analysis. Then I took a job as a sole developer for a 6-month old startup, replacing Excel forecasting tools and streamlining rote data tasks with pandas/jupyter notebook. To give you an idea of my experience and development, I wrote my first Python class about 2 weeks into this job. I have since developed and packaged a fairly extensive tabbed GUI app in jupyter notebook using pandas, ipywidgets and bqplot to replace and extend on multiple 100mb+ spreadsheet tools, with multiple file inputs, outputs and config saving, taught myself OO principles, basic git and discovered pycharm. I feel quite fortunate to have been given the role and the time to develop, but I'm running out of tools to automate and processes to streamline; if I'm not careful I'll be just messing around with similar projects in a year's time. There is no-one there to guide my technical career development in specific toolsets, only to give me new challenges and see if I can find a solution on my own. Also, once I replicate an Excel tool, the original analyst can't really contribute to the development any more, and no-one else can maintain them except me. I'm fairly happy to stay with the company I work with as long as I don't stagnate. - One immediately obvious role would be to learn SQL and begin interfacing with our customer database which is managed by our franchise parent. They provide us with some standard extracts but we often need to combine them in pandas/excel before we can begin working with them. I think they have a nightly backup of production which I could begin practising with, but again no real guidance. - An alternative might be to begin doing some kind of web interface for data analytics and insights as we have quite a wealth of data on our customers and their purchases but this is not readily available to our C-levels, perhaps this could be using dash (and django?). The vast majority of Python jobs I see advertised seem to want django, so that seems like a useful option too. Any other paths I might have missed? Both SQL and web seem like desirable tools but I feel like they have a greater opportunity to learn the 'wrong' way without sufficient guidance.
|
|
#
?
Jul 30, 2018 21:40
|
|
|
SQL is simple as brick and you definitely should know it regardless of your plans or further career specifics. It also is something you can definitely pick up at your own pace with free materials online.
|
|
#
?
Jul 31, 2018 04:28
|
|
|
Django is definitely worth exploring, just because being able to turn anything into a website is a useful tool to have. You'll find you gain quite a lot knowledge about client/server architecture, deployment and packaging, SQL and RDBMSs just through osmosis, but at any time you can drill down into any one of those topics and they're each a discipline of their own
|
|
#
?
Jul 31, 2018 07:45
|
|
|
I'm using the new PyCharm What's the difference between these two commands and which one is the correct one to use after I install a package? Boris Galerkin fucked around with this message at 15:51 on Jul 31, 2018 |
|
#
?
Jul 31, 2018 11:23
|
|
|
Slimchandi posted:Any suggestions for career progression paths in Python/software development generally coming from a data analysis background? Do you want to become a data scientist, or a BI developer, or a (bigger, better) Data Analyst, or even a Web Developer? Those are all paths you can take from here on out once you have some mastery of Python. To become a data scientist you would learn a bit of SQL and lots of machine learning/statistics. To become a BI Developer you would learn how to design better dashboards, maybe Tableau and/or Power BI. For a Data Analyst, continue on the path that you are on, learn to tease out more insights from the data and make recommendations. For web development, more Flask and Django and CSS/JS. SQL is a good bet in all of these endeavours though, simply because of how widespread it is. However it's a big language and you won't need many analytical functions if you are doing web development, or DB management and indexing if you are doing Data Analysis. Learning how to grab data and pivot it is always good to know and doesn't take much effort. 
|
|
#
?
Jul 31, 2018 13:05
|
|
|
“A bit of SQL” is maybe a junior data scientist, or like data science intern. Serious data scientists have to be able to barf any complexity SQL in their sleep and understand basic DBA stuff (indices mainly) to be able to write queries that are not only complex but also don’t destroy databases and execute before heat death of the universe.
|
|
#
?
Jul 31, 2018 13:09
|
|
|
Boris Galerkin posted:I'm using the new PyCharm 2017.2 now with (better?) pipenv support so I'm playing around with pipenv as well. Whenever I install a package a dialog screen shows up in the corner telling me that my pipfile is out of date and that I should run either "pipenv lock" or "pipenv update." Both options are clickable links and I've tried clicking both of them now and they both seem to update my Pipfile the same way. PyCharm 2018.2 is the latest. It just dropped 6 days ago. It also calls out pipenv support as a new feature. quote:Support for pipenv https://www.jetbrains.com/pycharm/whatsnew/
|
|
#
?
Jul 31, 2018 13:15
|
|
|
Proteus Jones posted:PyCharm 2018.2 is the latest. It just dropped 6 days ago. I’m pretty sure 2017 was a typo.
|
|
#
?
Jul 31, 2018 14:27
|
|
|
Yeah I meant 2018.2.
|
|
#
?
Jul 31, 2018 15:50
|
|
|
I'm not sure why PyCharm recommends update. I mean, update will also lock, but that's not all it does. AFAIK, lock is what you want.
|
|
#
?
Jul 31, 2018 16:23
|
|
|
I've only recently started messing around with Python and I went from the interpreter on WSL to VS Code. Am I missing out on anything by not using PyCharm? I'm only doing simple data munging scripts for now and often use the built-in terminal/interpreter in VS Code. I generally dislike huge big IDEs, and do often work with SQL. Is that a reason to stick with VS Code for all my coding needs?
|
|
#
?
Jul 31, 2018 17:27
|
|
|
Mark Larson posted:I've only recently started messing around with Python and I went from the interpreter on WSL to VS Code. Am I missing out on anything by not using PyCharm? I'm only doing simple data munging scripts for now and often use the built-in terminal/interpreter in VS Code. Old, but a quick perusal makes it seem pretty relevant still: https://www.youtube.com/watch?v=iutkLjeGc6w It's basically the same editor vs IDE war as ever, but you owe it to yourself to at least watch some of JetBrains demo videos or maybe look over some of these articles/posts. FWIW, PyCharm has Jetbrains excellent SQL editor called DataGrip built-in. It's kind of hard to tell you specifically why it would be worth it for you since it's hard to say what you, specifically, would find important or useful.
|
|
#
?
Jul 31, 2018 17:42
|
|
|
The biggest PITA I have with Python and VS Code is that VS Code doesn't pick up the OpenCV bindings, doesn't give me IntelliSense for OpenCV classes, and gives me a red squiggly line for everything OpenCV. PyCharm does not. Also, PyCharm is stricter about PEP8, which is somewhat of a pain in the rear end, but I think it's a good thing still.
|
|
#
?
Jul 31, 2018 17:44
|
|
|
Protocol7 posted:The biggest PITA I have with Python and VS Code is that VS Code doesn't pick up the OpenCV bindings and doesn't give me IntelliSense and gives me a red squiggly line for everything OpenCV. PyCharm does not. FWIW, PyCharm and VS Code can both be configured to be as strict or not strict on PEP8 as you want.
|
|
#
?
Jul 31, 2018 17:45
|
|
|
Thermopyle posted:FWIW, PyCharm and VS Code can both be configured to be as strict or not strict on PEP8 as you want. Yeah, I figured. VS Code is crazy extensible, which is a pretty great thing. If you don't like how it does something, there's probably a setting or extension to change that. I use it for a lot of web stuff and it's definitely my favorite editor. Still contemplating if I want to upgrade to WebStorm, but that's a discussion for a different thread.
|
|
#
?
Jul 31, 2018 17:47
|
|
|
Protocol7 posted:Still contemplating if I want to upgrade to WebStorm, but that's a discussion for a different thread. If you care, WebStorm is integrated into PyCharm.
|
|
#
?
Jul 31, 2018 17:50
|
|
|
Small nitpick about DataGrip and WebStorm integrations, I don't think they are fully integrated within PyCharm. I'd be ready to bet about DataGrip, not sure about WebStorm. Nevertheless, you can competently develop web and SQL specific stuff in PyCharm, although you will most likely need to get yourself the Professional edition.Thermopyle posted:FWIW, PyCharm and VS Code can both be configured to be as strict or not strict on PEP8 as you want. Protocol7 posted:Yeah, I figured. VS Code is crazy extensible, which is a pretty great thing. If you don't like how it does something, there's probably a setting or extension to change that. I use it for a lot of web stuff and it's definitely my favorite editor. Yeah its under inspections in the settings. You can toggle various general Python and framework/stack-specific inspections on and off, as well as configure their severity and scope both in global and per-project basis. I don't know if you can integrate new custom inspections natively in PyCharm, but nothing stops your from strapping flake8 or similar as custom toolchain bit to your project to easily invoke extra checker. There's also stuff like SonarLint etc.
|
|
#
?
Jul 31, 2018 17:58
|
|
|
Mark Larson posted:I've only recently started messing around with Python and I went from the interpreter on WSL to VS Code. Am I missing out on anything by not using PyCharm? I'm only doing simple data munging scripts for now and often use the built-in terminal/interpreter in VS Code. A guy in my team does all this stuff in VS Code and his workflows are fine but SQL stuff definitely is way more awkward that it is for me with IntelliJ (I do use DataGrip and PyCharm separately). Also his code sometimes really needs an editors but that's adequately solvable in VS Code too, given some effort. In general, no, data science activities don't make you immune to advantages of fully fledged general programming tooling.
|
|
#
?
Jul 31, 2018 18:03
|
|
|
cinci zoo sniper posted:Small nitpick about DataGrip and WebStorm integrations, I don't think they are fully integrated within PyCharm. I'd be ready to bet about DataGrip, not sure about WebStorm. Nevertheless, you can competently develop web and SQL specific stuff in PyCharm, although you will most likely need to get yourself the Professional edition. I don't use DataGrip-ish features very much, but I use WebStorm-ish stuff extensively. In both cases, they're sometimes behind on features from the standalone version because they just get merged into PyCharm on major revisions or something like that. Anyway, I'm just going by what Jetbrains has told me on their issue tracker where they've told me that PyCharm and their more "full" IDEs include the features of WebStorm and DataGrip, but the user experience and how you access the features are different. I can't really commit to a position on this as I haven't done a full in-depth comparison.
|
|
#
?
Jul 31, 2018 19:21
|
|
|
Thermopyle posted:I don't use DataGrip-ish features very much, but I use WebStorm-ish stuff extensively. In both cases, they're sometimes behind on features from the standalone version because they just get merged into PyCharm on major revisions or something like that. Eh that sounds fair enough, similar to my experience with PyCharm and DataGrip stand-alone. To be fully honest I’ve used little of built in database capacity from PyCharm, but I was recently checking 2018.2 notes for both and there were some seeming differences.
|
|
#
?
Jul 31, 2018 19:31
|
|
|
I wish PyLint and PEP8 checkers were better about metaclasses, but there's only so much you can do. It's a running problem for me because I'm using protobuf and the generated files are 100% metaclasses and I hate to turn helpful lints off just because a bunch of classes from one file break the poor linter's brain when used.
Linear Zoetrope fucked around with this message at 20:37 on Jul 31, 2018 |
|
#
?
Jul 31, 2018 20:34
|
|
|
cinci zoo sniper posted:Eh that sounds fair enough, similar to my experience with PyCharm and DataGrip stand-alone. To be fully honest I’ve used little of built in database capacity from PyCharm, but I was recently checking 2018.2 notes for both and there were some seeming differences. In a complete coincidence I just stumbled across this SO post from someone on the DataGrip team while looking for something else. quote:I am Max, the PMM of DataGrip. https://stackoverflow.com/a/48862482/23972
|
|
#
?
Jul 31, 2018 20:46
|
|
|
E: iOS awful app is buggy garbage.
cinci zoo sniper fucked around with this message at 07:44 on Aug 1, 2018 |
|
#
?
Jul 31, 2018 21:02
|
|
|
Linear Zoetrope posted:I wish PyLint and PEP8 checkers were better about metaclasses, but there's only so much you can do. It's a running problem for me because I'm using protobuf and the generated files are 100% metaclasses and I hate to turn helpful lints off just because a bunch of classes from one file break the poor linter's brain when used. flake8 lets you configure directories or files to exclude entirely, not sure about pylint.
|
|
#
?
Jul 31, 2018 21:42
|
|
|
Thermopyle posted:I'm not sure why PyCharm recommends update. I mean, update will also lock, but that's not all it does. Cool thanks.
|
|
#
?
Aug 1, 2018 06:34
|
|
|
cinci zoo sniper posted:That settles it I guess. I think my recent discovery here was that I couldn’t find a DatGrip plugin for PyCharm, but I may have just fat fingered that. I can’t indeed get XPathView plugin on PyCharm, unlike DataGrip, for some reason. E: On a second thought, that seems to be mostly deprecated plugin, that I keep just as an “in case” measure. cinci zoo sniper fucked around with this message at 08:03 on Aug 1, 2018 |
|
#
?
Aug 1, 2018 07:44
|
|
|
I'm updating my raspberry pi based info screen thingy I have at home (think one of those magic mirrors without the mirror and more useful info), and wanted to do it in python this time. The thing pretty obviously breaks into a backend and a front end, so I was thinking of using bottle to serve a react app that then call different endpoints in the bottle to fetch updated info. The bottle app would also periodically make requests to third party APIs and fetch data that it then caches to limit the outbound request count. Now, for making the requests to the third party APIs, I was thinking of having a bunch of workers, each living in its own thread and periodically updating a local copy of the relevant data from a single 3rd party API. So something like this:  With the outline of the code looking something like this: Python code:2) Is there a more sensible way to have an eternal background worker that sleeps 90% of the time than the silly code above?
|
|
#
?
Aug 1, 2018 09:57
|
|
|
Loezi posted:
This looks ok, but I would do it with a Celery queue instead of always having 5 workers spawned, have your main process fire off the tasks to a queue and retrieve the data. So you end up with a solution that hits both points.
|
|
#
?
Aug 1, 2018 11:19
|
|
|
It seems way over engineered for such a display.
|
|
#
?
Aug 1, 2018 13:56
|
|
|
Yeah, that might be a bit much, I'm not sure exactly what you're using this for, but it makes my brain happy. As suggested by someone else, celery is your friend for task management. However, I think you don't do that instead, you add it! There's no reason you can't have separate queues and separate workers for each service you're working with there. Are you using docker to wrangle all of this? It would be extremely simple to spin up a quick docker-compose.yml that sets up all of your services, including one for celerybeat, then you just need one celery config to set up a broker that distributes tasks to the appropriate queue and its associated worker (probably a default redis container). Again, probably way too much for what you're doing, but I'm a docker convert and I've been in charge of dockerizing and overhauling celery systems at most of my recent jobs. It depends on how much of a load you're trying to handle...if it's really tiny, a single celery queue would handle it, but docker would let you scale up automatically with additional concurrent workers and stuff if you're handling a big load.
|
|
#
?
Aug 1, 2018 16:05
|
|
|
Write a small script that fetches whatever from whatever API. Repeat for each API. Add them all to your cron file on whatever repeating schedule you want. For bonus points add a lock file of some sort so subsequent runs don't stack up. That is likely going to be more maintainable and reliable than whatever system you're dreaming of. If you're just having fun thats cool, but usually for appliance-type things like this to be used by multiple members of your household you want something fast, reliable, and easy to troubleshoot.
|
|
#
?
Aug 1, 2018 16:49
|
|
|
Thermopyle posted:It seems way over engineered for such a display. Could be, but I'm not really sure how to cut down the complexity other than just moving the responsibilities around. The code for threading is pretty simple (20 lines global setup + a call to super().__init__() for every worker) and allows me to have a bunch of "do this every so often" modules that are 1) logically separated and 2) can be ran at different intervals. So I can update the estimated arrival time for the buses at the next door bus stop every 10 seconds, the weather every 10 minutes, the lunch menus at both my and my wife's work place every few hours and so forth. Probably going to skip Celery, seems like it requires me to run a separate broker and I don't really feel like setting this up. Optimally this whole thing should start with "source ~venv/bin/activate; python server.py &; firefox localhost:8080"
|
|
#
?
Aug 1, 2018 17:05
|
|
|
I’m trying to come up with a function that checks if a float is within the neighbourhood (+ or - 10) from eacht item in a list of floats (returned by a sqlite query). I’m having a hard time coming up with the logic for checking if it’s in or outside the range.
|
|
#
?
Aug 2, 2018 15:36
|
|
|
LochNessMonster posted:I’m trying to come up with a function that checks if a float is within the neighbourhood (+ or - 10) from eacht item in a list of floats (returned by a sqlite query). If you subtract two numbers and take the absolutely value of that, then you can see how close the values are. You could do this really easily with numpy. Something like code:e: numpy isn’t needed you could just do a lost comprehension to do the subtraction bit. Boris Galerkin fucked around with this message at 16:55 on Aug 2, 2018 |
|
#
?
Aug 2, 2018 16:53
|
|
Python code:
|
|
|
#
?
Aug 2, 2018 17:00
|
|
|
Thanks, that sounds like exactly what I am looking for. Is this the kind of stuff I’d learn from reading Fluent Python?
|
|
#
?
Aug 2, 2018 17:21
|
|
|
LochNessMonster posted:Thanks, that sounds like exactly what I am looking for. I don’t know that book but the abs(a-b) < tol part to find a distance in both answers is just math. You learned it when you learned about number lines 
Boris Galerkin fucked around with this message at 17:45 on Aug 2, 2018 |
|
#
?
Aug 2, 2018 17:42
|
|
|
|
| # ? May 16, 2024 06:23 |
|
|
Or if you just want to check it's close to all the valuesPython code:(In case you don't know generator expressions are like list expressions but inside ( ) instead of [ ], but because it's already the only thing in all's parentheses you can skip them)
|
|
#
?
Aug 2, 2018 18:48
|
|
