|
ARACHNOTRON posted:It would be nice if it searched back until it didn't find an init file. I don't know. Ah, but Python also supports importing files that reside in directories without an init file. And you might have more than one area holding .py files that you want to use, which would ultimately necessitate some sort of PATH variable anyway (Java uses a PATH-like variable, too, it was probably set for you by your IDE if you're working in Windows. I think it's JAVAHOME? I haven't used Java in a long time)
|
# ? Mar 30, 2013 21:29 |
|
|
# ? May 8, 2024 05:33 |
|
ARACHNOTRON posted:Why does packaging in Python have to be so weird You are thinking in .NET (or java w/ eclipse). In .NET and in eclipse you can put your files any god drat place you please and the IDE will manage the visibility of the packages for you. The most you'll ever have to do is mess with the dependency checkboxes if you've set up multiple projects. In python your import visibility is defined by sys.path, as others have mentioned. Your import include path also has the working directory of the current module in it. This means that for the directory structure you've specified a file in subpackage1 cannot import package.subpackage2 or subpackage2 unless sys.path is modified (which you shouldn't do). If subpackage1 requires subpackage2 then subpackage2 should be a sub package of subpackage1. i.e. code:
turds.py can do: code:
code:
I believe in this way of thinking, as I have recently dealt with a compiling a C project in linux that someone had written in Visual Studio. Their directory structure was out of whack with their dependency structure in exactly the way you are describing. They had a subpackage2 at the same directory level as subpackage1, but subpackage1 depended on subpackage2 and the linux make system had no visibility to the files it needed. IDEs that build for you make awful directory layouts easy. Dren fucked around with this message at 08:30 on Mar 31, 2013 |
# ? Mar 31, 2013 08:23 |
|
What's the most current form of Oauth for Python 3? I'm trying to create a stock screener using my broker's API, which uses Oauth. Most of the info I find is out of date or conflicting. I've seen the following modules referenced: Oauth - seems to be the original, now outdated. I get an error of "'module' object has no attribute 'Consumer'" Oauth2 - Newer version, apparently also outdated? The one most referenced one online. Glitches out in pip/can't figure out how to install it. Oauthlib - IIRC, claims to be the new replacement for Oauth and Oauth2 Rauth.OAuth2Service - Also potentially replacement for Oauth and Oauth2? Requests - ? Oauth_hook - ? pyoauth2 - I get an error about not having a module named "client" in pyoauth2's init. Which should I use? Dominoes fucked around with this message at 18:39 on Mar 31, 2013 |
# ? Mar 31, 2013 18:29 |
|
Anyone ever found a database backend for Flask that supports generic foreign keys? I see there is some experimental support for it in Peewee, but it's not quite what I would really call fully generic. From what I can tell about SqlAlchemy, it doesn't really do generic foreign keys either.
|
# ? Mar 31, 2013 20:03 |
|
Dominoes posted:What's the most current form of Oauth for Python 3? I'm trying to create a stock screener using my broker's API, which uses Oauth. Most of the info I find is out of date or conflicting. I've seen the following modules referenced: I've used tweepy before: http://tweepy.github.com/ which uses OAuth to against twitter, but their setup.py doesn't really make it clear which it uses. It was solid overall. edit: although I guess I wouldn't have used it in 3.x, sorry :-/ Lurchington fucked around with this message at 22:53 on Mar 31, 2013 |
# ? Mar 31, 2013 22:30 |
|
My Rhythmic Crotch posted:Anyone ever found a database backend for Flask that supports generic foreign keys? I see there is some experimental support for it in Peewee, but it's not quite what I would really call fully generic. From what I can tell about SqlAlchemy, it doesn't really do generic foreign keys either. As far as SQLAlchemy goes, would the generic association examples talked about here and found here do the trick?
|
# ? Apr 1, 2013 05:03 |
|
Dren posted:You are thinking in .NET (or java w/ eclipse). In .NET and in eclipse you can put your files any god drat place you please and the IDE will manage the visibility of the packages for you. The most you'll ever have to do is mess with the dependency checkboxes if you've set up multiple projects. Pretty reasonable and understandable. I just wanted to group things up for now and have things look nice (even though I was going to split it all up across multiple servers later) because I'm dumb. QuarkJets posted:Ah, but Python also supports importing files that reside in directories without an init file. And you might have more than one area holding .py files that you want to use, which would ultimately necessitate some sort of PATH variable anyway Java's classpath is defined locally for every project at compile (basically just set to the current folder) which I guess doesn't help me at all when I work in other languages
|
# ? Apr 1, 2013 14:56 |
|
Haystack posted:As far as SQLAlchemy goes, would the generic association examples talked about here and found here do the trick? Purely as an exercise, I wanted to see how easy or difficult it would be to implement foreign keys which could point to an instance of any child class of a particular parent class. Turns out I guess you need generic foreign keys for that.
|
# ? Apr 1, 2013 19:39 |
|
Is anyone aware of a library or snippet for parsing ranges? Specifically, I'm looking for the ability to take string input like "1-7, 13, 24, 100-192", "K-W", or whatever else kind of weird range thing you can think of and be able to get a list or something sane out of that input. I just don't feel like writing it right now...
|
# ? Apr 2, 2013 23:47 |
|
Here's an untested braindead solution:Python code:
|
# ? Apr 3, 2013 00:43 |
|
Flask owns, why I have I been banging my head against the django wall for so long. Anyone know a good django admin replacement?
|
# ? Apr 3, 2013 02:45 |
|
Malcolm XML posted:Flask owns, why I have I been banging my head against the django wall for so long.
|
# ? Apr 3, 2013 02:54 |
|
So I learned the basics of Python from code academy and now I really want to start a small project to practice. I was thinking of a page I can upload/download files from sort of like my own ghetto megaupload. What's a good place to start? I looked at Flask and it looks like I can use this but I'm hitting a brick wall. When it comes to coding I am pretty bad (probably why I was attracted to Python's easy to use/read syntax.) The problem is it feels like frameworks are a whole different language in themselves, and I'm pretty lost. Can anyone please recommend me a good tutorial or path I should take to really start learning? Or recommend a good beginner's project I can start with? I really want to get into coding but I get overwhelmed easily.
|
# ? Apr 3, 2013 06:15 |
|
Social Animal posted:So I learned the basics of Python from code academy and now I really want to start a small project to practice. I was thinking of a page I can upload/download files from sort of like my own ghetto megaupload. What's a good place to start? I looked at Flask and it looks like I can use this but I'm hitting a brick wall. When it comes to coding I am pretty bad (probably why I was attracted to Python's easy to use/read syntax.) The problem is it feels like frameworks are a whole different language in themselves, and I'm pretty lost. Can anyone please recommend me a good tutorial or path I should take to really start learning? Or recommend a good beginner's project I can start with? Have you completed the Python Challenge yet? That's a really good first project
|
# ? Apr 3, 2013 06:55 |
|
This is the only page I could find: http://www.pythonchallenge.com/ Is this the one? If so love that page design.
|
# ? Apr 3, 2013 07:59 |
|
Social Animal posted:This is the only page I could find: That's the one. It was made in 2005, so not exactly the height of web design
|
# ? Apr 3, 2013 11:43 |
|
Social Animal posted:So I learned the basics of Python from code academy and now I really want to start a small project to practice. I was thinking of a page I can upload/download files from sort of like my own ghetto megaupload. What's a good place to start? I looked at Flask and it looks like I can use this but I'm hitting a brick wall. When it comes to coding I am pretty bad (probably why I was attracted to Python's easy to use/read syntax.) The problem is it feels like frameworks are a whole different language in themselves, and I'm pretty lost. Can anyone please recommend me a good tutorial or path I should take to really start learning? Or recommend a good beginner's project I can start with?
|
# ? Apr 3, 2013 13:49 |
|
I'm new to programming (Finished Codeacademy labs recently), and am running into long data-crunching times on a stock screener I'm working on. The functions below parse a .CSV file and compare its values to input numbers. Doing this for a maximum* of 20,000 cells takes 1 minute. *The actual number should be smaller, since a failed cell should skip the rest of its row. (The break in the second function below) Is the time due to sloppy code? Is it normal? Is it a limit of python, compared to C++? Python's using 35% of my CPU while the evaluation's occurring. The 'evaluate' function is a method of a class whose instances are contained in 'parameters' input for the 'passing_stocks' function. I removed some dubugging, output and formatting code. code:
code:
Dominoes fucked around with this message at 16:32 on Apr 3, 2013 |
# ? Apr 3, 2013 16:26 |
|
Is there any way you can the data you're pulling to do some of the heavy lifting for yourself? What kind of screens do you want to use? How many stocks are you pulling information for? Python is not really great for real-time financial stuff because of it's slower speed. Honestly anything dealing with real-time market data is not a good project for a beginner programmer. Good luck!
|
# ? Apr 3, 2013 16:32 |
|
a lovely poster posted:Is there any way you can the data you're pulling to do some of the heavy lifting for yourself? What kind of screens do you want to use? How many stocks are you pulling information for? I want to screen for price, % change, average volume, and possibly some other real-time criteria. I also want to screen for price changes over selectable increments of time - a function I haven't found on any existing screener. I'm using Yahoo Finance. For now, I'm using a canned list of stocks pulled from the nyse, nasdaq and s&p500 website - about 5,000 total. I'm pulling the real-time data into a .csv, in 200-stock-per-query increments, the max allowed by Yahoo. Surprisingly, this takes substantially less time than the actual number crunching. I'm also pulling and analyzing historical price data, which the functions I posted don't deal with. I'll post them, or the whole program if you're curious. Unfortunately, I can only seem to pull the historical data one-stock-at a time, so I've been pulling and analyzing a stock, then looping, instead of pre-downloading. I'm only pulling/analyzing historical data for stocks that passed the real-time screening. I'm trying to set up the program to work with my broker (tradeking) as well, but I'm having trouble getting Oauth to work with Python 3 - that's what my previous post in this thread was about. Dominoes fucked around with this message at 16:47 on Apr 3, 2013 |
# ? Apr 3, 2013 16:38 |
|
It's likely the reason it's slow is because it's reading the data file from the disk for every cell. If these are not sequential reads, then the disk is going to be bouncing around all over the place.
|
# ? Apr 3, 2013 16:39 |
|
Dominoes posted:Is the time due to sloppy code? What does the function read_cell look like?
|
# ? Apr 3, 2013 16:44 |
|
DARPA posted:What does the function read_cell look like? code:
Dominoes fucked around with this message at 16:57 on Apr 3, 2013 |
# ? Apr 3, 2013 16:46 |
|
[I think] read_cell is your issue. For each cell you check in evaluate() you are reopening the csv file and iterating through the lines. You are triple nesting loops for something like Number_Of_Stocks * Number_Of_Parameters * Lines_In_File. If you have the memory for it, open the file once, read it into a list, something like (assuming you're using csv.reader): Python code:
edit: evilentity suggests below using list() which definitely clearer than a loop or comprehension. DARPA fucked around with this message at 18:08 on Apr 3, 2013 |
# ? Apr 3, 2013 16:57 |
|
Dominoes posted:Here's the read_cell function: And thats the reason why its so slow. Read the whole file into memory and go from there. I assume reader is csv.reader. Something like that should do: Python code:
|
# ? Apr 3, 2013 17:07 |
|
Dominos, lets pump the brakes a little bit. The way you're accessing your data is very crude. Each time you need to access a cell you're opening the file, seeking to the row containing that record, parsing the row data, then returning the cell. Compared to memory access, disk access is very slow. To speed this operation up you should load all of your data into one of python's wonderful data structures and access it there. It doesn't sound like data.csv is very big so this won't be a problem. If your source data is too large to easily fit into memory you'll need to store it in a database and access it from there. Here's some sample code to get you started: Python code:
|
# ? Apr 3, 2013 17:09 |
|
Thanks guys - Really appreciate it. Going to fix it this evening!
|
# ? Apr 3, 2013 17:17 |
|
Dominoes posted:Thanks guys - Really appreciate it. Going to fix it this evening! Another tip, if there's a module for what you're trying to do in the standard library, use it instead of rolling your own solution.
|
# ? Apr 3, 2013 18:58 |
|
Thermopyle posted:Is anyone aware of a library or snippet for parsing ranges? Specifically, I'm looking for the ability to take string input like "1-7, 13, 24, 100-192", "K-W", or whatever else kind of weird range thing you can think of and be able to get a list or something sane out of that input. Have you tried numpy.fromstring? http://docs.scipy.org/doc/numpy/reference/generated/numpy.fromstring.html#numpy.fromstring I'm not certain it would handle multiple delimiters very well. My first impression with your problem is I'd just run a str.split() on it and than iterate over it to substitute range() for any instance where it saw "-". Of course, that doesn't address your question of a simple function to just do it. That kind of thing seems common enough that I'd be mildly surprised if NumPy didn't have a function to at least handle it halfway for you. Here's the list of all their array creation methods. http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html
|
# ? Apr 3, 2013 19:59 |
|
Pivotal Lever posted:Another tip, if there's a module for what you're trying to do in the standard library, use it instead of rolling your own solution. But he don't know what he don't know!
|
# ? Apr 3, 2013 20:52 |
|
Problem solved! I used evilentity's load_csv function. I added this line at the top of passing_stocks: "data = load_csv('data.csv')" Read cell now looks like this:code:
Pivotal Lever posted:Another tip, if there's a module for what you're trying to do in the standard library, use it instead of rolling your own solution.
|
# ? Apr 3, 2013 21:19 |
|
Dominoes posted:(slow code) Just to point out that you can better understand how much time is time spent on each portion of your code by profiling it. Python makes this pretty simple: code:
|
# ? Apr 3, 2013 21:25 |
|
Dominoes posted:
This is literally equivalent to data[x][y].
|
# ? Apr 3, 2013 21:54 |
|
Dominoes posted:Problem solved! I used evilentity's load_csv function. I added this line at the top of passing_stocks: "data = load_csv('data.csv')" Read cell now looks like this: You are still walking the full length of the lists. You can directly address the cell. You can write : Python code:
|
# ? Apr 3, 2013 22:04 |
|
DARPA posted:You are still walking the full length of the lists. You can directly address the cell. Dominoes fucked around with this message at 22:23 on Apr 3, 2013 |
# ? Apr 3, 2013 22:18 |
|
What could I have done to cause this behaviour? (Here x and y are instances of a class I created, so maybe I screwed something up there?)code:
|
# ? Apr 3, 2013 23:04 |
|
__eq__ evaluating to be the same but __hash__ evaluating to different hashes.
|
# ? Apr 3, 2013 23:21 |
|
You can do this with a mismatch between __eq__ and __hash__, like so:Python code:
code:
code:
The contract of Java's Object#hashCode is worth reading; it applies to Python also.
|
# ? Apr 3, 2013 23:26 |
|
I had set my own __eq__ and __ne__, but I didn't know there was a __hash__, so that really helps a lot. And thanks for the very nice demonstration of how this work, Lysidas!
|
# ? Apr 3, 2013 23:38 |
|
|
# ? May 8, 2024 05:33 |
|
So I'm learning Python and I was fiddling with list comprehensions:Python code:
|
# ? Apr 4, 2013 00:32 |