|
Gothmog1065 posted:Is there a good source so I can read up on the difference, etc? The book I used barely covered libraries and such. Modules are libraries. What you'd have is a file called something like my_commands.py that contains something like: code:
code:
Gothmog1065 posted:Eventually I want this to be part of a larger program that will do some of the tasks at work to make things easier. For example, we use IPs a lot, I'd like to eventually have it where I can put an IP in to a box and have it do various functions depending on the need. Designing your own language is a hard task. Right now you're simply splitting strings based on ",". What if you want to insert a comma character into one of your parameters? Then you have to stop using string manipulation and build a real language where you use something like: code:
code:
code:
Using these rules, a parse tree for the above stream might look like: code:
All of this work is rebuilding Python, which does all this already and a lot more, for no good reason. It's a good learning experience, but inventing your own language should never be done "in production". EDIT: To come from the other side, every custom language is something new for other people to learn. By making your IT staff learn a custom, less-powerful utility and custom, less-powerful language isn't really an efficient use of time either. This tool probably won't be used outside of your staff, so learning it isn't broadening their skills. Gothmog1065 posted:I'm on 3.2.1 right now, going to update, I though I was on 3.1.3, but whatever. I keep hearing different reasons to either be updated or go back. I know 2.x has much better library and addon support, but for the basic stuff right now, will it make a difference? Get used to examples and tutorials written for Python 2.x, not 3.x. Get used to being able to use third-party supplied libraries. Programmers quite often need to use third-party code because it's commonly a waste of time and effort to rebuild it themselves. Working with third-party libraries and frameworks is very good practice, and the availability of third-party libraries is much higher in 2.x than in 3.x. 2.7 has most of the good 3.x features anyway. Between 2.7 and 3.x, there aren't that many good reasons to switch over, while a lot of downsides. Gothmog1065 posted:anyhoo: Gah, as The Gripper said, I hosed up. Should be line.split. Suspicious Dish fucked around with this message at 08:52 on Nov 28, 2011 |
# ? Nov 28, 2011 08:41 |
|
|
# ? May 9, 2024 07:35 |
|
Suspicious Dish posted:2.7 has most of the good 3.x features anyway. Between 2.7 and 3.x, there aren't that many good reasons to switch over, while a lot of downsides. I'm just tired of typing from __future__ import division in every single goddamn file/interactive session NumPy + SciPy + NetworkX + BioPython = 3.2 supremacy
|
# ? Nov 28, 2011 16:02 |
|
So I need to write some unit tests for an app that I've written for sorting files based on meta-data. I was assuming I should just use PyUnit since it's part of the standard library, but I'm open to other suggestions. My main question though, was how exactly to go about setting up test cases. The most basic test would ensure that files (of various types) are sorted properly. I'm having a hard time thinking of a way to provide the correct sorting to validate against. My initial idea is to run the sorting, then get a directory listing of the destination directory and compare this to a known directory listing when the sorting occurs correctly. Where would I store this known listing? As a pickled list? A text file? I've done some unit tests in other languages before, but they all dealt with mathematical tests rather than something like a file sorting app.
|
# ? Nov 28, 2011 18:02 |
|
Why not just store the expected data as a list within the Python program?code:
|
# ? Nov 28, 2011 18:12 |
|
Suspicious Dish posted:Why not just store the expected data as a list within the Python program? Wow. Yeah that should work. Another question. All of my processing is done in a separate thread from my gui using threading.Thread A problem that I am having is that the thread never seems to release the memory that it allocates while running. Do I have to manually clean up the thread after it finishes executing the run method?
|
# ? Nov 30, 2011 03:20 |
|
That's a very generic question. You're not supposed to do any cleanup. The GC will do it for you. What sort of processing are you doing? EDIT: Do note that Python has the Global Interpreter Lock, meaning that two Python threads can't run at once. While your GUI will continue to update because it runs on its own thread, something like a Python callback will have to wait until a thread releases the GIL. If you're using numpy, that releases the GIL by itself. If you do any blocking IO, it will release the GIL while waiting for the IO to complete. If you're stuck in a while loop, the GIL won't be released and your callback won't be run. Suspicious Dish fucked around with this message at 03:37 on Nov 30, 2011 |
# ? Nov 30, 2011 03:34 |
|
Modern Pragmatist posted:A problem that I am having is that the thread never seems to release the memory that it allocates while running. Generally speaking, "dynamic" languages don't release memory to the OS during execution. The runtime allocates a heap at startup, and this is what your program sees as "memory". As your program's memory usage goes up, the runtime will allocate more memory from the OS and portion that out to your program, again on an as-needed basis. When your program's variables and/or structures go out of scope and/or their reference counts fall to zero and/or you explicitly destroy them, the language runtime marks that portion of its heap free. Memory is not returned to the OS until execution ends, because the runtime expects that it will use that heap again. You don't have direct control over memory from an OS perspective, and even indirectly you can only cause more of it to be used. You should expect that your process's overhead will grow to the size of the runtime overhead plus the size of your maximum heap allocation, and then remain near-constant for the life of the process.
|
# ? Nov 30, 2011 04:35 |
|
mdxi posted:Generally speaking, "dynamic" languages don't release memory to the OS during execution. Ok this is pretty helpful. I'll run a few more tests tomorrow to make sure it's working as expected. Thanks for the explanation.
|
# ? Nov 30, 2011 06:22 |
|
mdxi posted:Generally speaking, "dynamic" languages don't release memory to the OS during execution. Ah, this has been something I've been curious about as well. Do you have any sources that go into more detail about this? (Not to say your explanation isn't helpful! I'm just interested in some of the details.)
|
# ? Nov 30, 2011 07:45 |
|
mdxi posted:When your program's variables and/or structures go out of scope and/or their reference counts fall to zero and/or you explicitly destroy them, the language runtime marks that portion of its heap free. Memory is not returned to the OS until execution ends, because the runtime expects that it will use that heap again. http://hg.python.org/cpython/file/74d182cf0187/Objects/obmalloc.c#l1080 No, Python uses free when it frees an object. That should be returned to the OS.
|
# ? Nov 30, 2011 12:45 |
|
Suspicious Dish posted:http://hg.python.org/cpython/file/74d182cf0187/Objects/obmalloc.c#l1080 My apologies. I was working from my knowledge of Perl 5 internals, and betting that any language with a similar design would take a similar approach.
|
# ? Nov 30, 2011 14:15 |
|
mdxi posted:My apologies. I was working from my knowledge of Perl 5 internals, and betting that any language with a similar design would take a similar approach. Why would anyone ever think this. Perl internals are insane. quote:To reanimate the program, its parse tree must be reconstructed. This phase exists only if code generation occurred and you chose to generate bytecode. Perl must first reconstitute its parse trees from that bytecode sequence before the program can run. Perl does not run directly from the bytecodes; that would be slow.
|
# ? Nov 30, 2011 15:00 |
|
Suspicious Dish posted:http://hg.python.org/cpython/file/74d182cf0187/Objects/obmalloc.c#l1080 So if this is true, are all objects that exist within a thread not removed and their memory freed upon completion of threading.Thread.run()? As far as the contents of the thread, it loads a tar file, extracts certain parts and stores them into objects, which are then saved as modified files. It seems pretty trivial, however, the memory occupied by the objects is never released until I exit python.
|
# ? Nov 30, 2011 17:26 |
|
Modern Pragmatist posted:So if this is true, are all objects that exist within a thread not removed and their memory freed upon completion of threading.Thread.run()? Python objects are freed when their reference counts are dropped to zero, or when the cycle detector sweeps them away. I'll need a code sample to see whether you're retaining the objects somehow.
|
# ? Nov 30, 2011 22:12 |
|
Suspicious Dish posted:Python objects are freed when their reference counts are dropped to zero, or when the cycle detector sweeps them away. I'll need a code sample to see whether you're retaining the objects somehow. http://pastebin.com/5We2KrSq The thread is spawned in FileDropTarget.DoProcessing()
|
# ? Nov 30, 2011 23:23 |
|
code:
|
# ? Nov 30, 2011 23:57 |
|
tef posted:
I was under the impression that my very large variable (container) went out of scope after Processor.run() completed due to the fact that it is not a property of Processor. I ran a test by substituting: code:
Modern Pragmatist fucked around with this message at 06:43 on Dec 1, 2011 |
# ? Dec 1, 2011 01:58 |
|
I'm seeing something weird with urllib and simplejson. I'm sure I've just misunderstood something, but I can't see what. Therefore I've decided to throw it out here and see if any of you fine people can help. I've condensed the problem I'm experiencing down to the following code. code:
code:
Any ideas?
|
# ? Dec 1, 2011 16:18 |
|
Lister_of_smeg posted:What I don't understand is why urlencode is happy with the raw dict, but not with the dict that comes out of simplejson.dumps. code:
Why do you want to urlencode the simplejson output rather than just urlencode the request_dict? You're adding another step that doesn't really seem sensible.
|
# ? Dec 1, 2011 16:30 |
|
Kim Jong III posted:
That makes sense. Must remember to check with type() in the future. I was building out a RESTful interface that had been prototyped by one of the senior developers here. His prototype methods did it that way, so I figured I should follow suit. Looks like I need to ask some questions about how well tested this prototype is.
|
# ? Dec 2, 2011 11:02 |
|
I am taking a guess and I am imagining that by restful you mean 'homeopathic rest'
|
# ? Dec 2, 2011 13:48 |
|
Okay, I've run into a fairly annoying problem trying to develop a python module that interacts with two other modules that have a shared module hierarchy. Maybe this is me just coming from a mostly Java background and expecting this kind of thing to work, but hear me out, as I've been able to throw together a trivial example that illustrates what I'm trying to do. Let's say I have two projects, test-1 and test-2 with the following directory hierarchies: code:
My PYTHONPATH is set up as follows: code:
code:
edit: unbroke tables crazyfish fucked around with this message at 01:19 on Dec 3, 2011 |
# ? Dec 3, 2011 01:17 |
|
Every module is a singleton. This means that between test-1/foo/__init__.py and test-2/foo/__init__.py, one of them has to "win". You can see which one wins by doing:code:
|
# ? Dec 3, 2011 02:06 |
|
Suspicious Dish posted:Every module is a singleton. This means that between test-1/foo/__init__.py and test-2/foo/__init__.py, one of them has to "win". You can see which one wins by doing: Sigh. This just feels like another random instance of python's 'your way or the highway' kind of mentality, kind of ignoring the real need of enterprise types to organize their python modules, but never mind that. This is good information to know, don't get me wrong. But it's another instance of python breaking what I feel is a reasonable use case. Then again, no language would ever get developed without someone compromising someones' reasonable use cases.
|
# ? Dec 3, 2011 09:12 |
|
crazyfish posted:Sigh. This just feels like another random instance of python's 'your way or the highway' kind of mentality, kind of ignoring the real need of enterprise types to organize their python modules, but never mind that. What are you talking about? Enterprise types? code:
code:
|
# ? Dec 3, 2011 12:29 |
|
Hi again! I post here too rarely. I r nublet etc. I'm trying to implement a quicksort algorithm into my code(mostly to figure it out, partly for the satisfaction), and I've got it pretty much there, except now I'm getting an error I don't know what to do with. code:
The error I'm getting is tied specifically to anyplace sortThis[e] is mentioned in the above. The error itself: code:
|
# ? Dec 3, 2011 16:24 |
|
Fren posted:What are you talking about? Enterprise types? Yeah that was just me being dumb and angry. Thanks for the help.
|
# ? Dec 3, 2011 16:44 |
|
quaunaut posted:Hi again! I post here too rarely. I r nublet etc. "for" loops in python are not like for loops in C or C++ -- they're 'foreach' loops. When you say for e in sortThis, what you're doing is creating a loop where e gets assigned to be the elements inside sortThis successively. For Example: code:
program output posted:10 So, you can see that the problem here is that you're saying sortThis[e] - you're taking e, which is an element in your list, and you're using it as an index into list -- check this out: code:
program output posted:0 You can see that the IndexError exception is thrown on the last list iteration that it gets to, because the value 10000, when interpreted as an index into the list, is out of bounds. Also notice that even before that happens, because it's using the values in the list as indexes into the list, it's printing the list in reverse order -- probably not what was intended. For fun, check out what happens if you remove the element 10000 from that list and then execute the loop. if the value of e on that loop iteration is greater than the length of the list, you're gonna get that error you posted. Otherwise, the algorithm would just silently do the incorrect thing. TasteMyHouse fucked around with this message at 07:00 on Dec 4, 2011 |
# ? Dec 3, 2011 17:56 |
|
quaunaut posted:Hi again! I post here too rarely. I r nublet etc. As mentioned earlier, for iterates over the values in a list, not the indices . You might find it easier to implement quicksort that returns a new list before quicksorting in-place.
|
# ? Dec 3, 2011 19:47 |
|
quaunaut posted:Anyone have any ideas for why it would be out of the range? I'm a little late to the party on this, but for loops iterate over values. If you want indices, try for j in range(len(sortThis))
|
# ? Dec 4, 2011 06:22 |
|
FoiledAgain posted:I'm a little late to the party on this, but for loops iterate over values. If you want indices, try for j in range(len(sortThis)) I'm partial to for i, _ in enumerate(sortThis)
|
# ? Dec 4, 2011 20:53 |
|
For some reason, input() randomly stopped working properly and it doesn't receive input as strings, but as variable or numbers unless I put quotations around the input. How in the world do I fix that? I'm using IDLE if that matters.
|
# ? Dec 4, 2011 20:56 |
|
IShallRiseAgain posted:For some reason, input() randomly stopped working properly and it doesn't receive input as strings, but as variable or numbers unless I put quotations around the input. How in the world do I fix that? I'm using IDLE if that matters. In python 2, you should always use raw_input() - input() executes the input first and returns the result. In python 3 raw_input() changed to input().
|
# ? Dec 4, 2011 21:00 |
|
Thanks, I didn't realize input() had changed between versions, and I had to use an older version of Python for an assignment.
|
# ? Dec 4, 2011 21:05 |
|
I'd like to note that this is ENTIRELY for my own edification: I wanted to read 4 bytes from /dev/urandom, interpret those 4 bytes as an int, then use that int for some random purpose. The problem here is that if I do: code:
Again note though: I'm just using random now so it doesn't matter at all, practically. TasteMyHouse fucked around with this message at 19:44 on Dec 5, 2011 |
# ? Dec 5, 2011 19:35 |
|
TasteMyHouse posted:number is now a 4 char long string. I could write low-level byte twiddly bullshit using ord() and bitwise shifts, ORs, etc to take these 4 bytes and create a single int out of them... but this is Python, not C! Surely there's an easier way? code:
|
# ? Dec 5, 2011 19:42 |
|
Threep posted:I don't know if there's a better way but: Awesome. This was exactly what I was looking for, thanks!
|
# ? Dec 5, 2011 19:45 |
I'm starting learning design patterns, and the first thing I stumble across is interfaces. Since I was learning from a Java tutorial, I kind of had to scratch my head and work out how to implement it in Python. Would y'all be able to check my code and give any pointers / advice? (Unfortunately the output feature seems to be Python 2.x and I can't remember how to format strings in 2.x)
|
|
# ? Dec 5, 2011 21:45 |
|
I think you just need to do: print dave and print "Hello, %s" % dave At least, if I'm understanding what you're trying to do.
|
# ? Dec 5, 2011 21:50 |
|
|
# ? May 9, 2024 07:35 |
vikingstrike posted:I think you just need to do: Ah thanks, although that's more of an aside. I'm really just interested to see if my implementation of an Interface is useful/correct; it's all in the code.
|
|
# ? Dec 5, 2011 22:05 |