|
Asymmetrikon posted:I don't know how much of Python's standard types are defined by spec or left up to implementation; wouldn't it be technically possible for list to be backed as a linked list? Sure. It's also possible for every integer to be a linked list where the value it represents is the length of said list. Both are equally silly. Python programs basically expect O(1) access and a linked list doesn't really help you there a lot.
|
# ? Jan 12, 2017 08:43 |
|
|
# ? May 9, 2024 17:21 |
|
Hey dudes; trying to run a java file from Python, using subprocess. It's not doing anything. The most basic example:Python code:
Python code:
|
# ? Jan 12, 2017 20:36 |
|
Dominoes posted:Hey dudes; trying to run a java file from Python, using subprocess. It's not doing anything. The most basic example: Well java.exe with no params does return 1, so that's why it's throwing an exception when check=True. To actually get the output from java you have to pipe it as part of the call to subprocess.run. There are many ways to do this, depending on what you want to do with that output, but if you search "subprocess pipe output" you should find some useful info.
|
# ? Jan 12, 2017 20:45 |
|
Dominoes posted:Hey dudes; trying to run a java file from Python, using subprocess. It's not doing anything. The most basic example: You don't need shell=True if you're passing a list like that Python code:
What happens if you do the above without shell=True or check=True? Do you get any error messages? Sidenote, I love subprocess; it's like a poor man's parallel processing module and lets you do all sorts of crazy stuff. But it's also sometimes a messy crutch. In your case, you want to invoke a jar; have you considered installing a module like Pyjnius or py4J in order to invoke Java classes directly?
|
# ? Jan 13, 2017 08:36 |
|
Popen works as a context manager, fyi, you can use "with Popen(blah) as proc:" to get your waiting and file descriptor closing and so on for free
|
# ? Jan 13, 2017 10:36 |
|
Thansk for the subprocess words. Got it sorting using that advice!
|
# ? Jan 14, 2017 15:26 |
|
I would like to use Bokeh to plot data, and then select a portion of that data to work with. I don't understand how to get the data or the region back from the selector tools. Do I need to use a callback? Thanks.
|
# ? Jan 17, 2017 18:25 |
|
accipter posted:I would like to use Bokeh to plot data, and then select a portion of that data to work with. I don't understand how to get the data or the region back from the selector tools. Do I need to use a callback? Thanks. The relevant documentation appears to be listed here: CustomJS for Selections
|
# ? Jan 17, 2017 22:36 |
|
StormyDragon posted:The relevant documentation appears to be listed here: CustomJS for Selections Thanks. I found this, but then I thought there should be a more direct way to get access to the data, but I couldn't find it.
|
# ? Jan 17, 2017 23:43 |
|
accipter posted:Thanks. I found this, but then I thought there should be a more direct way to get access to the data, but I couldn't find it. I think you will have to put a few more words into your exact use case, by the sound of it I can only surmise that you need to use the Bokeh Server in order to get python callbacks.
|
# ? Jan 18, 2017 00:11 |
|
StormyDragon posted:I think you will have to put a few more words into your exact use case, by the sound of it I can only surmise that you need to use the Bokeh Server in order to get python callbacks. Fair enough. I am looking at porting an application that uses matplotlib to interactively select data to a Jupyter Notebook. It was my understanding that matplotlib couldn't be interacted with in a notebook. I was thinking that bokeh could be a potential solution. A cell would be used to create the plot and interactively select the data, and then the following cell would retrieve the selected region from the chart object.
|
# ? Jan 18, 2017 00:33 |
|
Question: I am using Tkinter and Tweepy to build a simple python twitter gui. I have an entry box and button that I can send tweets from (works fine). I then have a text box and two buttons for it. One button should start a twitter stream and print new tweets as they appear into the text box. The other button should stop the stream. I have my Tkinter GUI set up as one class and the twitter portion set up as another. Upon starting the stream I can get my script to print the tweets to the console but I can't figure out if there is a way to write them to the tkinter text box. I put the entire script on pastebin here :http://pastebin.com/S6sK92Hh the settings import is just a file that looks like this: code:
edit: I think I may have found a way to do this using tk.Frame that I was not utilizing earlier. Will update if I get it to work. 2edit: Rewrote a bunch http://pastebin.com/HEdzvJM9 still can't get it to write new tweets to textbox creatine fucked around with this message at 04:14 on Jan 19, 2017 |
# ? Jan 18, 2017 21:05 |
|
I'm picking up Python and I have a question about data structures, and which one's best. In my code I'm tracking which players have sight of which entities. If a player moves, I'll need to recalculate which entities they have visibility to; if an entity moves, I'll need to recalculate which players have visibility of it. I'll also need to iterate over which players can see an entity (for when the entity does something that players should find out about) and over which entities a player can see (for the player's UI). There will generally be quite a lot more entities than players, and a lot of them will not be seen by any player at any time. My current approach is for my "visibility" class to have two dictionaries: one with player ID as the key, and each entry being a set of all visible entities; the other with entity ID as the key and each entry a set of players who have visibility of the entity. Whenever the data changes, the class will have to update both dictionaries, but having the data held in this way will presumably mean that I don't have to search through the entirety of the visibility data each time I need to iterate through one player's visible entities or one entity's viewing players. Is the approach I'm describing reasonable, or is there something I've misunderstood about how fast different data structures are?
|
# ? Jan 19, 2017 13:25 |
|
Whybird posted:I'm picking up Python and I have a question about data structures, and which one's best.
|
# ? Jan 19, 2017 13:52 |
|
Dictionaries keep their keys as hashes, so accessing different sets shouldn't be too bad, and iterating through sets should also be fast enough. However, that architecture seems a bit strange to me. Why can't you keep a list of visible entities on each instance instead of having this Visibility class? Presumably, you are already holding a collection of players instances and entities instances around, and having this visibility data separate from the main instances can only hurt you in the long run.
|
# ? Jan 19, 2017 13:59 |
|
How many players and entities are we talking here? How often do you update the state, is it a fast action game or something turn-based? If the demands aren't high you can probably just do whatever feels simple and makes the most sense, without worrying about efficiency What you're saying makes sense, there's actually a bidirectional dict library that uses a pair of plain dicts under the hood, so maintaining two views (one from each side) is definitely an approach people use Do you actually need to maintain that full state though? It depends on what you need to do with the information, but from what you're saying everything relies on a player being able to see it happening - players need to know which entities they can see, and entities need to interact with each other if a player can see them. So you could just update the players' "what I can see" dict, and then combine their sets to get the set of all visible entities, and tell each of those to do their thing. Also saves you iterating over the larger group of entities to see if each is visible You could also drop the state and just let your visibility check function tell the entity and players to do something if any of them are visible - but it really depends on what your game is going to need and if it's worth storing the visibility information for further use baka kaba fucked around with this message at 14:12 on Jan 19, 2017 |
# ? Jan 19, 2017 14:06 |
|
creatine posted:Question: I am using Tkinter and Tweepy to build a simple python twitter gui. I have an entry box and button that I can send tweets from (works fine). Also, you should get rid of the Update class and move the methods to the GUI class. You never create an Update instance. That's why you're having to pass "self" as an argument to call those methods. It also makes "self" values misleading -- they look like they're referring to an instance of Update when they're really an instance of GUI. PM if you have questions.
|
# ? Jan 19, 2017 14:24 |
Dicts seem fine, so long as you don't have many players and you only need to track which players see enemies, not all entities which see each other. I would just use a set for each player. If you want to check if a monster is visible to a particular player, just check membership in the set. It's very fast! Checking membership for each player is not a hindrance so long as you have a small amount of players. Most games have 1-16, so I think you'll be OK. Also, as mentioned above, you might want to keep track of the intersection of all visible enemies and only check actions for enemies which are visible to some player. I. E, code:
Eela6 fucked around with this message at 15:24 on Jan 19, 2017 |
|
# ? Jan 19, 2017 15:10 |
|
Whybird posted:but having the data held in this way will presumably mean that I don't have to search through the entirety of the visibility data each time I need to iterate through one player's visible entities or one entity's viewing players. I'm not speaking directly to you as it sound like you're in a state where you're not even sure what makes the most conceptual sense, but...as a general rule you should use the data structure that makes the most sense conceptually and then if you hit a performance issue consider changing it.
|
# ? Jan 19, 2017 15:54 |
"Use the simplest thing that could possibly work" is one of the programming aphorisms that has served me best. With the following addendums: 'don't use a list to test membership ' 'don't use a list when you need a deque ' 'don't use sort() to maintain a sort when you could use bisect.insort()'
|
|
# ? Jan 19, 2017 16:16 |
|
onionradish posted:It works for me on 3.5.1/Windows 7. Change your except: pass line to except: raise to make sure you're not suppressing some error. Hey thanks! I moved some stuff around and got it to work as intended. Here's my working code as of now. http://pastebin.com/E8aD9q7v Basically make a file called settings.py like I said above or just plug in your own twitter app credentials and it should work. python 3.6 /win10
|
# ? Jan 19, 2017 17:49 |
|
Wow! Thank you all for the advice, this is really helpful!baka kaba posted:Do you actually need to maintain that full state though? It depends on what you need to do with the information, but from what you're saying everything relies on a player being able to see it happening - players need to know which entities they can see, and entities need to interact with each other if a player can see them. So you could just update the players' "what I can see" dict, and then combine their sets to get the set of all visible entities, and tell each of those to do their thing. I've just been watching https://www.youtube.com/watch?v=C4Kc8xzcA68 on hash tables and I realise I've been assuming that membership testing in a set is much, much slower than it is (also, I'm absolutely astounded by just how clever hash tables are!) -- this was why I was wary to have each entity check in turn whether or not it was in a player's visible_entities list, because it felt like that would be a lot of time-consuming checks to make. The idea of using the intersection of all visible entities instead is a really clever one, and I like it a lot!
|
# ? Jan 19, 2017 23:02 |
|
Whybird posted:Wow! Thank you all for the advice, this is really helpful! I haven't watched that video, but 7 years is a long time in Python time. Check out this video by Raymond Hettinger (a super awesome core dev who does great, accessible presentations) about dicts in Python 3.6. https://www.youtube.com/watch?v=p33CVV29OG8 A few other Hettinger videos https://www.youtube.com/watch?v=wf-BqAjZb8M https://www.youtube.com/watch?v=OSGv2VnC0go https://www.youtube.com/watch?v=Bv25Dwe84g0
|
# ? Jan 19, 2017 23:15 |
Hash tables are super cool. Remember that you can create your own __hash__() for classes, if you'd like. It's not usually necessary, because user-defined classes will default to the object id for __hash__(), but it can be useful if you're creating data types that should always hash the same. As a toy example (inspired by Fluent Python) code:
Raymond Hettinger is amazing.
|
|
# ? Jan 19, 2017 23:19 |
|
Whybird posted:Wow! Thank you all for the advice, this is really helpful! As I understand it, Python sets are also implemented as hash tables under the hood (only caring about the keys, not values), so checking membership in a set should be equivalent to checking if a key exists in a dict.
|
# ? Jan 19, 2017 23:35 |
|
Whybird posted:I'm picking up Python and I have a question about data structures, and which one's best. There are many ways of solving that general problem, but in molecular dynamics it's handled this way: https://en.wikipedia.org/wiki/Cell_lists https://en.wikipedia.org/wiki/Verlet_list
|
# ? Jan 20, 2017 00:27 |
|
So a system I work on is setup in that there are 20 cron jobs (run at 3 minute intervals) where each spawns a new instance of a shell script with a different argument (which says what user to use to run a different process). Essentially: code:
Each of these scripts then checks a directory for the next file in a queue, then operates on it. If no new file is found, it sleeps for 5 and then checks again (running for an hour, before dying and then restarting). It also deals with using file descriptors and locks so that only one instance is looking at the directory at a time and it's overly complicated for this. And then also does stuff like check all running processes in case a job that has gone over an hour long (so don't reuse the user) and all that. This is obviously something I think can be improved via a long running python script that handles which untrusted ids are available and incoming files. I'm guessing I need some combination of using multiprocessing and watchdog, but I'm not sure how I want to approach this and would love to know if anyone has any advice, since this is extremely new territory for me with python. I guess a simple approach would be to only use multiprocessing so that if an untrusted id is available, check the directory and spin up an instance if a file is there, otherwise sleep for 3 minutes, but that seems less "good" than also using watchdog. Master_Odin fucked around with this message at 03:45 on Jan 20, 2017 |
# ? Jan 20, 2017 03:38 |
This sounds like a job for coroutines and async/await, rather than multiprocessing. Are you sure you need multiprocessing? This sounds like an IO-bound rather than cpu-bound process, especially since you're hitting the filesystem so often. If you use async , you can avoid locks entirely. Which is pretty cool! If you're not familiar with async in python 3.6 (and considering it's so new, few are), consider watching Raymond Hettinger's recent talk from the python convention in Russia. The asyncio module in the standard library is what you're looking for. (asyncio is available in python 3.4 and 3.5, but it's easier and clearer in python 3.6 with the new async and await keywords, not to mention asynchronous comprehensions) Eela6 fucked around with this message at 04:03 on Jan 20, 2017 |
|
# ? Jan 20, 2017 03:55 |
|
Eela6 posted:This sounds like a job for coroutines and async/await, rather than multiprocessing. Are you sure you need multiprocessing? This sounds like an IO-bound rather than cpu-bound process, especially since you're hitting the filesystem so often. Like one of these! Tigren posted:https://www.youtube.com/watch?v=Bv25Dwe84g0 Tigren fucked around with this message at 04:46 on Jan 20, 2017 |
# ? Jan 20, 2017 04:41 |
|
Eela6 posted:Hash tables are super cool. Remember that you can create your own __hash__() for classes, if you'd like. It's not usually necessary, because user-defined classes will default to the object id for __hash__(), but it can be useful if you're creating data types that should always hash the same. Good example, with one minor tweak/caveat: it's very desirable for a hash function to be uniformly distributed (as much as possible), and a lot of work typically goes in to designing hash functions for built-in types to provide good performance when these are used as hash table keys. It's rather easy to accidentally destroy this property by mixing hash functions, and though using bitwise XOR isn't bad, if possible it's a good idea to use built-in functionality for this: code:
|
# ? Jan 20, 2017 17:11 |
Excellent point. Thanks for the correction!
|
|
# ? Jan 20, 2017 17:13 |
|
Was going to edit my post, but no point in doing that after a reply. I was going to say that implementing __hash__ as returning hash(tuple(stuff in your object)) also works with non-numeric data:code:
code:
Lysidas fucked around with this message at 17:21 on Jan 20, 2017 |
# ? Jan 20, 2017 17:17 |
|
This thread is good and I'm learning so many things.
|
# ? Jan 20, 2017 18:08 |
|
Lysidas posted:Was going to edit my post, but no point in doing that after a reply. I was going to say that implementing __hash__ as returning hash(tuple(stuff in your object)) also works with non-numeric data: This Stack Overflow post discusses the use of straightforward arithmetic involving prime numbers in creating a good hashcode implementation (in C#): http://stackoverflow.com/questions/263400/what-is-the-best-algorithm-for-an-overridden-system-object-gethashcode It is made a bit harder perhaps by the fact that Python integers are arbitrary-precision rather than 32-bit. I don't know how hashcodes work internally in Python so I don't know how big an issue this is or how hard it is to get around it.
|
# ? Jan 20, 2017 18:18 |
|
Are there any exe utilities that have been updated for 3.6?
|
# ? Jan 20, 2017 21:18 |
|
Anecdote of the current state of installing Python packages that include C/Fortran code in windows with pip instead of conda: The following packages work without a problem (Ie I loaded them in a requirements.txt, then 'ran pip install -r requirements.txt'.) numpy pandas matplotlib jupyter sympy PyQt5 keras requests requests_oauthlib pytest toolz cytoolz beautifulsoup4 django-toolbelt The following required Chris Gohlke's site: scipy scikit-learn h5py tables Numba installs from pip only after installing llvmlite from CG's site. Dominoes fucked around with this message at 12:05 on Jan 21, 2017 |
# ? Jan 21, 2017 00:25 |
|
I'm looking to parse text data from three kinds of sources: pdf, .doc and .docx. I have around a thousand of each. I'm surprised at how difficult it has been to extract the text of even a docx file into a Python string (which I'd then write to a plain txt). I'm using Python3 and don't have easy access to Windows to use wincom32. I have googled around and found a lot of solutions that don't seem to work. For instance PyPDF2 only prints blanks when I loop over the pages and print their text content. The python docx library doesnt work in Python3, even after tinkering with it (removing exception imports, adding () to print statements). I guess my question is if you guys have any good, simple method to turn any of these documents in txts?
|
# ? Jan 21, 2017 15:06 |
|
floppo posted:I'm looking to parse text data from three kinds of sources: pdf, .doc and .docx. I have around a thousand of each. I'm surprised at how difficult it has been to extract the text of even a docx file into a Python string (which I'd then write to a plain txt). I'm using Python3 and don't have easy access to Windows to use wincom32.
|
# ? Jan 21, 2017 15:18 |
|
floppo posted:I'm looking to parse text data from three kinds of sources: pdf, .doc and .docx. I have around a thousand of each. I'm surprised at how difficult it has been to extract the text of even a docx file into a Python string (which I'd then write to a plain txt). Have you looked at one in a text editor? They're not exactly trivial to parse Can't help with suggestions, but make sure your PDFs actually have text content - sometimes they're just page images
|
# ? Jan 21, 2017 16:27 |
|
|
# ? May 9, 2024 17:21 |
|
pdfminer works well enough, but is slow.
|
# ? Jan 21, 2017 17:07 |