|
I've been stumbling my way through making a small Python 3.3 server monitor, because budget is $0, and have it (kind of) working. Each of the hosts runs an instance of Coretemp w/ the remote server plugin that runs a simple web server and spits out json to connected clients. This was my third approach after abandoning a couple wmi implementations because hosts ended up becoming practically unresponsive under load; these are rendering slaves and experience extended periods of 100% utilization, which is really what I'm concerned about monitoring in the first place. Anyway, the problem I'm having is with consuming the json the hosts provide using urllib. I know less about http than I do Python, so when readurl raises BadStatusLine, I'm capturing the json from it and parsing the CPU and ram data I want, rinse and repeat instead of maintaining a connection and parsing each new json as it refreshes. Christ, I feel dirty just typing that out. Is there a trick to doing this gracefully, or would using socket to build my own requests let me not rely on the whole thing routinely making GBS threads the bed? I mean, it works, but it feels shady as gently caress, and I'd love to not be a Coding Horrors celebrity. Handiklap fucked around with this message at 14:29 on Dec 15, 2013 |
# ? Dec 15, 2013 14:26 |
|
|
# ? Jun 12, 2024 13:30 |
|
Handiklap posted:I've been stumbling my way through making a small Python 3.3 server monitor, because budget is $0, and have it (kind of) working. Each of the hosts runs an instance of Coretemp w/ the remote server plugin that runs a simple web server and spits out json to connected clients. This was my third approach after abandoning a couple wmi implementations because hosts ended up becoming practically unresponsive under load; these are rendering slaves and experience extended periods of 100% utilization, which is really what I'm concerned about monitoring in the first place. How feasible would it be to create a Pip-like utility that cleanly handles compiling other language code? Ie something like the Ubuntu package manager, but for Python on any operating system. Pip works well for Python-only modules, but has a high chance of making GBS threads the bed if there's any C in it. Windows installers and Ubuntu packages are nice, but aren't available for everything. Dominoes fucked around with this message at 16:31 on Dec 15, 2013 |
# ? Dec 15, 2013 14:46 |
|
I just finished the Codecademy course for Python and now I don't know where to go to learn more, practical python. Any suggestions?
|
# ? Dec 15, 2013 17:16 |
|
rock2much posted:I just finished the Codecademy course for Python and now I don't know where to go to learn more, practical python. Any suggestions?
|
# ? Dec 15, 2013 17:22 |
|
Dominoes posted:You're right; the answer is Signals and slots. I couldn't get that connect statement to work. Just to confirm, is that likely from a change in recent releases of PyQt4? I had to use: Python code:
|
# ? Dec 15, 2013 17:43 |
|
Rocko Bonaparte posted:I couldn't get that connect statement to work. Just to confirm, is that likely from a change in recent releases of PyQt4? Could be. What version of PyQT are you using? My example used 'new style' signals and slots, and the the bit of code you posted uses the old style. The new style was introduced in PyQt 4.5. PyQt5 drops the old style entirely. The old style is based on the C++ QT. New style is cleaner and more Pythonic.
|
# ? Dec 15, 2013 17:52 |
|
Dominoes posted:Perhaps this is something Requests with streaming enabled would be good for. I tried this with the same results. Upon further inspection, the problem is that the server is returning an empty status code. Is there anyway to intercept the response and modify response.status before it throws a fit?
|
# ? Dec 15, 2013 18:13 |
|
Handiklap posted:I tried this with the same results. Upon further inspection, the problem is that the server is returning an empty status code. Is there anyway to intercept the response and modify response.status before it throws a fit? Python code:
|
# ? Dec 15, 2013 18:20 |
|
Dominoes posted:
But without a valid status, it raises BadStatusLine, and the assignment that raised the exception doesn't actually happen. I ended up using socket to just read it line by line: Python code:
|
# ? Dec 15, 2013 20:05 |
|
Dominoes posted:Start a project, then look up or ask about whatever you can't figure out. It makes it (for me at least) easier to commit things to memory, and understand concepts. So I made a thing, where at some point it sorts a dictionary and then prints the keys & values in a nice format. I googled and found code:
code:
code:
|
# ? Dec 15, 2013 23:11 |
|
That looks like Python 2.x code and you're trying to run it on Python 3.3.
|
# ? Dec 15, 2013 23:21 |
|
Oh, drat. I thought it would work backwards but I guess not. Thanks.
|
# ? Dec 15, 2013 23:26 |
|
Using 'items()' instead of 'iteritems()' implies Python 3 as well. Your code should work with parentheses around the print calls.
Dominoes fucked around with this message at 23:31 on Dec 15, 2013 |
# ? Dec 15, 2013 23:27 |
|
Dominoes posted:Using 'items()' instead of 'iteritems()' implies Python 3 as well. Your code should work with parentheses around the print calls. items works fine in Python2. In fact, it's what I've used for years (because I always forget about iteritems).
|
# ? Dec 15, 2013 23:34 |
|
Dominoes posted:
I thought I was using a release with the new style in it. This is what I installed anyways: PyQt4-4.10.3-gpl-Py3.3-Qt4.8.5-x64, which claims to be PyQt 4.10.3.
|
# ? Dec 16, 2013 00:20 |
|
Dominoes posted:How feasible would it be to create a Pip-like utility that cleanly handles compiling other language code? Ie something like the Ubuntu package manager, but for Python on any operating system. Pip works well for Python-only modules, but has a high chance of making GBS threads the bed if there's any C in it. Windows installers and Ubuntu packages are nice, but aren't available for everything. Down this road lies madness. What exactly is this for? Just over-ambitious side-project? It's a very hard problem to solve. If you're actually just trying to get poo poo done - just use the distribution-native packages for python wrappers. Any missing pure-python libraries are pretty easy to wrap in packages once you do it a few times. It's still a huge pain in the rear end though - like, can easily be somebody's full time job if you absolutely need to have reliable deploys + dependencies on python apps and you end up maintaining your own repos etc etc. Whatever it is you're doing, consider not doing it. Just make do with a bit of manual setup or make it a vagrant script or something.
|
# ? Dec 16, 2013 00:36 |
|
Dominoes posted:I read through the quickstart and high level docs for H5PY, but can't figure out how to effectively store the data, either as groups or datasets. The three axes are stock symbol, date, and attribute (ie high, close prices). Would I use groups, or datasets? If I use datasets to store the prices, how can I correlate what each array value is for? I feel like while datasets (arrays) would be fast and compact, ensuring each array values matches up with what stock, date and attribute it's for would be a disaster. Just think of groups as folders and datasets as N-dimensional tables of data that you put inside of groups. I'd think that having a group for each stock name would be best, and then you could set up datasets of dates and prices So your HDF5 file could look like this: /GOOG/date /GOOG/high_price /GOOG/close_price /GOOG/low_price /YHOO/date /YHOO/high_price /YHOO/close_price /YHOO/low_price /IBM/date /IBM/high_price /IBM/close_price /IBM/low_price So each group would be a stock name, and each group would be containing 4 1-D datasets: date, high_price, close_price, low_price. Each day, you'd put a new entry into each dataset. Maybe you make a new HDF5 file each month, or maybe each year. This is a nice and simple way to organize your data. There are lots of other ways that you could organize your data in HDF5 datasets, this is just a basic one. The analogue in MySQL would be to have a table of stock names with unique IDs and a table of price data that has a unique id, date, high_price, close_price, low_price, and a unique stockname id that maps back to the stock name table. This is fine and has advantages and disadvantages.
|
# ? Dec 16, 2013 01:02 |
|
Dominoes posted:How feasible would it be to create a Pip-like utility that cleanly handles compiling other language code? Ie something like the Ubuntu package manager, but for Python on any operating system. Pip works well for Python-only modules, but has a high chance of making GBS threads the bed if there's any C in it. Windows installers and Ubuntu packages are nice, but aren't available for everything. What exactly do you mean compiling other languages. Is this just in the context of compiling Python modules with other language code in them? Pip works perfectly well for that almost every time, assuming that you have your system set up right with all the dependencies you need. On the other hand that assumption is the single biggest challenge if you wanted to make a utility that took care of that for you. Are you looking for a replacement for setuptools that somehow makes the process of standardizing a build easier? There is a lot of work that goes into making any particular library able to compile everywhere, and making a tool that could do that sounds insane. I have yet to find a package that Pip doesn't install and compile correctly assuming I had brew or apt-get the dependencies correctly and there wasn't a bug in the code itself.
|
# ? Dec 16, 2013 01:24 |
|
Dominoes posted:How feasible would it be to create a Pip-like utility that cleanly handles compiling other language code? Ie something like the Ubuntu package manager, but for Python on any operating system. Pip works well for Python-only modules, but has a high chance of making GBS threads the bed if there's any C in it. Windows installers and Ubuntu packages are nice, but aren't available for everything. I've never really had any problem with pip on ubuntu. It Just Works and is the main reason for using linux instead of Windows for python development. You can get it to work on Windows by correctly configuring Visual Studio (but it isn't easy to configure that right...I had it done once years ago, but have no recollection of what I did).
|
# ? Dec 16, 2013 01:28 |
|
Anyone who's used Bokeh: exactly how do I create an independent plot that I can embed into a totally different page? There's create_html_snippet(), but I don't really understand how it works. It outputs something.embed.js, and I'm not sure how to work that into a Jinja2 template. Alternatively, has anyone figured out how to get bokeh-server working on Heroku?
|
# ? Dec 16, 2013 05:35 |
|
Mr. Wynand posted:Down this road lies madness. OnceIWasAnOstrich posted:What exactly do you mean compiling other languages. Is this just in the context of compiling Python modules with other language code in them? Pip works perfectly well for that almost every time, assuming that you have your system set up right with all the dependencies you need. On the other hand that assumption is the single biggest challenge if you wanted to make a utility that took care of that for you. Are you looking for a replacement for setuptools that somehow makes the process of standardizing a build easier? There is a lot of work that goes into making any particular library able to compile everywhere, and making a tool that could do that sounds insane. Thermopyle posted:I've never really had any problem with pip on ubuntu. It Just Works and is the main reason for using linux instead of Windows for python development. QuarkJets posted:Just think of groups as folders and datasets as N-dimensional tables of data that you put inside of groups. I'd think that having a group for each stock name would be best, and then you could set up datasets of dates and prices
|
# ? Dec 16, 2013 09:06 |
|
Disregard previous post! Got something working, finally. Go check out How are my stocks doing? and follow the instructions there for great fun! By the way, I mention it there, but I want the url to be (for example) /stocks?symbol=AAPL and that will return the stock chart for AAPL. How do I do this in Flask?
|
# ? Dec 16, 2013 09:26 |
|
Pollyanna posted:Disregard previous post! Got something working, finally. Go check out How are my stocks doing? and follow the instructions there for great fun! "Python flask get URL variables value" and you will see many ways.
|
# ? Dec 16, 2013 12:38 |
|
Dominoes posted:How feasible would it be to create a Pip-like utility that cleanly handles compiling other language code? Ie something like the Ubuntu package manager, but for Python on any operating system. Pip works well for Python-only modules, but has a high chance of making GBS threads the bed if there's any C in it. Windows installers and Ubuntu packages are nice, but aren't available for everything. Anacondas. It gets talked about a lot on here I don't know how you've missed it. Dominoes posted:I ran into a bit of trouble trying to get H5PY installed on Ubuntu. Installing PyQt5 was a pain before the latest Ubuntu release which included a package. I've failed 100% of the time when trying on Windows. A pip that downloads and configures dependencies and compilers so modules just worked would be nice. They have a prebuilt package for H5PY. The QT package is 4.8, but maybe there is a QT5 package available somewhere. http://docs.continuum.io/anaconda/pkgs.html
|
# ? Dec 16, 2013 14:55 |
|
Yeah like he said Anacondas does half of that, although it only works for a subset of packages and although a vaguely remember it installing a working copy of gcc and letting me compile a random library through pip at one point, although when I tried again just now it was still complaining about visual studio so who knows. I think you or anyone else dealing with relatively large amounts of data that isn't quite relational database complicated should really look into Pandas. DataTables give you amazing Numpy compatible support for row/column based data with annotation that stock to datapoints through operations. It was also a godsend when working with shittons of timepoint data. It's just a bonus that it can use HDF5/PyTables as a backend to do out of core computation. Also speaking of that PyTables is a good alternative to h5py that is less of a thin wrapper over the C library with a more Pythonic interface of you don't want to go all the way to using Pandas (which you should because it is awesome). OnceIWasAnOstrich fucked around with this message at 20:16 on Dec 16, 2013 |
# ? Dec 16, 2013 16:14 |
|
OnceIWasAnOstrich posted:Yeah like he said Anacondas does half of that, although it only works for a subset of packages and although a vaguely remember it installing a working copy of gcc and letting me compile a random library through pip at one point, although when I tried again just now it was still complaining about visual studio so who knows. Anaconda isn't meant to let you compile packages that aren't pure python. The package manager, conda, will let you grab prebuilt binaries for packages. Anaconda supports a good amount of packages across linux/windows/mac. That way when you want to install numpy you don't have to gently caress with getting all the numpy dependencies with apt-get or installing Visual Studio or whatever mac people do. If you want a package that needs to be compiled and isn't provided in the anacoda distro you can poke around on the net to see if someone else has packaged it for conda or you can do so yourself. Btw, I second the idea that he should switch to pandas. Pandas is basically awesome.
|
# ? Dec 16, 2013 17:45 |
|
I started a pandas tutorial last night and watched bits of videos from its creator. Looks like it could open interesting possibilities.
|
# ? Dec 16, 2013 18:49 |
|
Dominoes posted:I ran into a bit of trouble trying to get H5PY installed on Ubuntu. Installing PyQt5 was a pain before the latest Ubuntu release which included a package. I've failed 100% of the time when trying on Windows. A pip that downloads and configures dependencies and compilers so modules just worked would be nice. Right, so usually the way that problem is solved is through actual package managers. Linuxes have deb/rpm/etc, OS/X has the that weird homebrew system and Windows has gently caress-all which is why everyone hates developing on Windows even if they are Microsoft #1 fan. Your best bet is to find windows installers for specific packages you need (like the ones pointed out above) or maybe you can hack up some poo poo using cygwin - either way, it just isn't a pleasant experience
|
# ? Dec 17, 2013 00:16 |
|
I'm trying to add a script tag to a template in Flask. I have a variable snippet that looks like this:code:
code:
|
# ? Dec 17, 2013 02:47 |
http://flask.pocoo.org/docs/templating/#controlling-autoescaping
|
|
# ? Dec 17, 2013 03:05 |
|
Oh. I could have sworn I looked up the Jinja2 documentation for it... Anyway, I have yet another problem, and this one's bizarre. Lemme give you some background first: I have this script... Python code:
Python code:
code:
Particularly weird is this: code:
When I ask it to print input_symbol before querying web.DataReader: code:
Anyone got any ideas?
|
# ? Dec 17, 2013 03:58 |
|
Pollyanna posted:Anyone got any ideas? I'm assuming that your using the <script> tag you mentioned previously. The path to the JavaScript file is relative since it doesn't contain the domain etc. because of this it looks in the "current directory" for this file based on the structure of your URL. Based on the structure, it thinks the file is at stocks/some.idiotic.filename.js which just sends another request but now instead of the symbol the last part of the URL is the JavaScript file. You should probably specify the absolute path to the JavaScript file using url_for. Modern Pragmatist fucked around with this message at 04:58 on Dec 17, 2013 |
# ? Dec 17, 2013 04:56 |
|
What's this about?code:
|
# ? Dec 17, 2013 08:18 |
|
There's no guarantee that NoneType is in the global scope. It's in the types module.
|
# ? Dec 17, 2013 08:21 |
|
Suspicious Dish posted:There's no guarantee that NoneType is in the global scope. It's in the types module. That makes sense, but then this doesn't work: code:
|
# ? Dec 17, 2013 08:55 |
|
Are you on Python 2 or Python 3? NoneType isn't exposed anywhere in Python 3, from my recollection. The only way to get it is to do type(None) (which is actually what the types module did anyway)
|
# ? Dec 17, 2013 09:00 |
|
Suspicious Dish posted:Are you on Python 2 or Python 3? NoneType isn't exposed anywhere in Python 3, from my recollection. The only way to get it is to do type(None) (which is actually what the types module did anyway) I'm on 3.3. I have absolutely no need to do anything with this, it's just a curiosity that I came across. I also learned today that isinstance(True,int)==True.
|
# ? Dec 17, 2013 09:07 |
|
There's a lot of modules in PyPi to work with subversion, so I was hoping somebody could recommend a few before I just try to figure what a bunch of them can or can't do. I have to deal with a situation where subversion is being used basically for software distribution (sad) to some Windows machines. So we'd use it to revert and update to a specific revision of the repository. Is there a general heavyweight module for this, or at least one more suitable for this kind of thing?
|
# ? Dec 17, 2013 16:50 |
|
FoiledAgain posted:What's this about? Why the hell do you need this?
|
# ? Dec 17, 2013 17:32 |
|
|
# ? Jun 12, 2024 13:30 |
|
evilentity posted:Why the hell do you need this? FoiledAgain posted:I have absolutely no need to do anything with this, it's just a curiosity that I came across.
|
# ? Dec 17, 2013 19:00 |