Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.

accipter posted:

Would something like cx_freeze work for you? http://cx-freeze.sourceforge.net/

Seems worth a try. I'll tackle windows then Mac I guess.

Adbot
ADBOT LOVES YOU

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Maluco Marinero posted:

Seems worth a try. I'll tackle windows then Mac I guess.

Last time I did something like this I think I was also doing a Django project. For some reason I settled on PyInstaller, but I can't for the life of me remember why I decided on it over the alternatives.

BigRedDot
Mar 6, 2008

Hey guys, another month, another Bokeh release :)




For anyone interested we are having a webinar on May 13th, and if you want to see my ugly mug, you can watch the tutorial I gave last month at PyData London.

Speaking of PyData, all the talks from PyData just went up on the new PyData youtube channel. Lot's of great talks about lots of great tools for data analysis with python.

Pollyanna
Mar 5, 2005

Milk's on them.


I'm trying to learn async, JSON handling, REST and all that in Flask since I'm missing a lot of core web concepts. I'm having trouble getting my code to do what I want, though. What I wanted to do was send a GET request to a URL and echo back the arguments through jQuery.

This is the code I'm sending a GET to:

Python code:
@app.route('/get', methods = ['GET'])
def get():

	if request.method == 'GET':

		data = request.get_json(force=True)

		return jsonify(data=data)
and the script I'm using:

JavaScript code:
$(document).ready(function() {

	$("#get_json").click(function() {

		event.preventDefault();

		$.ajax({
			url: $("#user_form").attr('action'),
			type: 'GET',
			dataType: 'json',
			data: $("#user_form").serializeArray(),
		})
		.done(function() {
			console.log("success");
		})
		.fail(function() {
			console.log("error");
		})
		.always(function() {
			console.log("complete");
		});

	});

});
The URL I'm sending arguments to is /get?username=asdf&password=asdf. I just get a 400 when I do this. What am I doing wrong?

edit: Never mind, I changed what I'm gonna do. Now I just wish SQLAlchemy stuff was more easily serializable.

Pollyanna fucked around with this message at 16:55 on Apr 18, 2014

Megaman
May 8, 2004
I didn't read the thread BUT...
I use boto and fabric in my python scripts, but I have a problem, I don't know how to place a configuration on a machine without creating the following ugly lines to do it. My question is what is the real and clean way to do the following:

import pipes
some_config = ("""blah blah blah""")
run("sudo echo %s | sudo tee /some/location/some_config > /dev/null" % pipes.quote(some_config))

raymond
Mar 20, 2004

Megaman posted:

I use boto and fabric in my python scripts, but I have a problem, I don't know how to place a configuration on a machine without creating the following ugly lines to do it. My question is what is the real and clean way to do the following:

import pipes
some_config = ("""blah blah blah""")
run("sudo echo %s | sudo tee /some/location/some_config > /dev/null" % pipes.quote(some_config))

Would fabric's put method work?
http://docs.fabfile.org/en/latest/api/core/operations.html#fabric.operations.put

Megaman
May 8, 2004
I didn't read the thread BUT...

Doh! I just read about this, thanks, this will work just fine.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
One of the authors of Fabric is a goon, which is kind of neat.

Space Kablooey
May 6, 2009


Pollyanna posted:

The URL I'm sending arguments to is /get?username=asdf&password=asdf. I just get a 400 when I do this. What am I doing wrong?

You said you moved on, but what's causing the problem is (probably) that you are using GET. Notice how your data is being sent in the URL (BTW, Passing a password in the url is always a big no-no), not in a JSON. You should use POST instead. And you probably have to fiddle with the Content-Type somewhere.

Pollyanna posted:

Now I just wish SQLAlchemy stuff was more easily serializable.

I hear ya. :sigh:

Megaman
May 8, 2004
I didn't read the thread BUT...
This is not really a python question, but a python "IDE" question. I use VIM to write python, and have been doing so without any autocomplete up until now, don't ask me how. I just installed pathogen to manage my VIM plugins, and installed jedi-vim which appears to be a real awesome way to autocomplete just like ipython/bpython does but while you're actually writing code inside VIM. I can autocomplete as I go by typing Ctrl+P, but I want it to work on the fly, such as when I type a '.', etc. There appears to be an option "let g:jedi#popup_on_dot = 1" that you can throw into your .vimrc that will do this, but it doesn't seem to work. Does any one else have experience with this, or know what could be the culprit? The goal is to not have to press any autocomplete key.

Nimrod
Sep 20, 2003

Megaman posted:

This is not really a python question, but a python "IDE" question. I use VIM to write python, and have been doing so without any autocomplete up until now, don't ask me how. I just installed pathogen to manage my VIM plugins, and installed jedi-vim which appears to be a real awesome way to autocomplete just like ipython/bpython does but while you're actually writing code inside VIM. I can autocomplete as I go by typing Ctrl+P, but I want it to work on the fly, such as when I type a '.', etc. There appears to be an option "let g:jedi#popup_on_dot = 1" that you can throw into your .vimrc that will do this, but it doesn't seem to work. Does any one else have experience with this, or know what could be the culprit? The goal is to not have to press any autocomplete key.

I can't help you with your specific problem, but when I was using VIM, I used the following setup: http://sontek.net/blog/detail/turning-vim-into-a-modern-python-ide

I don't know if it all still works, but one of the pathogen plugins is a tab-complete for Python, with some documentation on setup/use. Maybe that will work for you.

EDIT: There's also a vim thread. Maybe some of the wizards there who actually know how to write plugins/vimrc's can help you:
http://forums.somethingawful.com/showthread.php?threadid=3552945

Nimrod fucked around with this message at 09:28 on Apr 21, 2014

sharktamer
Oct 30, 2011

Shark tamer ridiculous
Would someone be able to tell me how reasonable the comments on this pull request are? It's a bit worrying when one of the first comments is that requests is bad and it shouldn't be used, which just seems plain wrong to me. Also, further down, someone questions the commentors logic on using regex everywhere even when it doesn't make sense and their answer is pretty much "my repo, my rules".

It's not my code, I'm just looking to contribute to the same repo and I'm anticipating similar criticisms on reasonable code.

Kristler
Apr 19, 2014

sharktamer posted:

Would someone be able to tell me how reasonable the comments on this pull request are? It's a bit worrying when one of the first comments is that requests is bad and it shouldn't be used, which just seems plain wrong to me. Also, further down, someone questions the commentors logic on using regex everywhere even when it doesn't make sense and their answer is pretty much "my repo, my rules".

It's not my code, I'm just looking to contribute to the same repo and I'm anticipating similar criticisms on reasonable code.

While blunt and to the point, I think he has some valid points. His push for regex over string functions he claims is due to performance, but it's also important to homogenize solutions to similar problems as much as possible. This makes it easier for developers to do crossover work with as little friction as possible. Otherwise, it seems like that's just how this developer is in terms of critique. If or when you do end up contributing, I'd recommend taking the inevitable criticism in stride, contest differences you believe you are objectively right in, and if he doesn't want your involvement then it's his loss.

ahmeni
May 1, 2005

It's one continuous form where hardware and software function in perfect unison, creating a new generation of iPhone that's better by any measure.
Grimey Drawer
Right about everything except Requests. Requests is messy internally because it's beautiful externally. I don't think I've ever seen someone use Requests and then not use it whenever possible afterwards.

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
Wanting to use regular expressions to parse URLs rather than urlparse is pretty stupid.

The reason given for rejecting Requests is dumb, but not wanting to drag in a dep when you already have something that does the same thing is totally reasonable.

Nippashish
Nov 2, 2005

Let me see you dance!
Has anyone worked with remote interpreters in pycharm? I would like to run pycharm locally on my laptop for development, but the code I am writing needs to run on a remote server because it needs access to special hardware that my laptop doesn't have.

I can set up pycharm on my laptop to invoke python on the server when I press ctrl+R to run, but I don't understand how to have pycharm synchronize the code between my laptop and the server. It seems like I can set up a deployment in pycharm to copy the code to the server, but then I would need to run the upload manually each time I make changes and I don't want to do that.

I have also tried storing the code on the server and accessing it from my laptop through sshfs, but pycharm is completely unusable with this setup (frequently goes unresponsive for multiple minutes, locks up completely if the internet connection goes down, etc).

I think pycharm would be happy if I set up sshfs in the other direction (storing the files on my laptop and mounting my laptop via sshfs from the server) but I am frequently in places where I can't accept incoming connections so I don't see how I can make this viable. I'd also need to log in to the server to re-mount my laptop each time I start working which is not ideal.

ahmeni
May 1, 2005

It's one continuous form where hardware and software function in perfect unison, creating a new generation of iPhone that's better by any measure.
Grimey Drawer
Have you tried PyCharm's debug server?

That being said, I can highly advise writing mock services if you need to test against specific hardware. It's a bit of work up front but it saves heaps of trouble in the long run because you can write unit tests that both validate locally against the mock and remotely against the hardware on deployment.

Nippashish
Nov 2, 2005

Let me see you dance!

ahmeni posted:

Have you tried PyCharm's debug server?

That being said, I can highly advise writing mock services if you need to test against specific hardware. It's a bit of work up front but it saves heaps of trouble in the long run because you can write unit tests that both validate locally against the mock and remotely against the hardware on deployment.

That is more complicated than what I am trying to do (although remote debugging is something I'd like to get working eventually), and also doesn't solve the problem I'm having. Step 2 in "To prepare for remote debugging" on that page is "Copy the local script to the remote location, where you want to debug it.", which is precisely what I want pycharm to handle for me.

Right now I have a nice tweak script -> run script -> get results workflow on my local machine. I'd like to keep this workflow but have the "run script" step execute on the server instead of on my laptop. Something like tramp mode in emacs would be fantastic. Of course I can just work on the server directly with vim+ssh but I'd much rather use pycharm because it is very nice.

Being able to run the code from within pycharm isn't really essential (maybe my original question was a bit misleading), what I really want is something to handle the synchronization. I don't want to use source control for this because a big part of what I'm doing is making tiny changes between runs (like changing a 0.1 to 0.15 to see what happens) and I don't want a million tiny commits with useless parameter changes, and I also don't want tweak -> run to turn into tweak -> commit to git -> push to remote -> switch to server -> pull from remote -> run.

I don't think mocking the hardware makes sense here because what I really care about is the program output. The special hardware I'm using is a GPU to do fast linear algebra and as far as I know mocking this would mean either making up numbers (which doesn't help me because I need the non-made-up numbers) or running the calculations on the CPU, which is what I'm doing now, but the whole point of using the GPU is that it's a lot faster.

Nippashish fucked around with this message at 15:43 on Apr 23, 2014

vikingstrike
Sep 23, 2007

whats happening, captain
Install pycharm on the remote server and forward it over X? There are also commands like rmate (TextMate) and rsub (Sublime Text) that let you locally edit remote files through the respective applications.

Another option is to setup a Dropbox folder so syncs are handled automatically and then just keep an ssh session open to the remote server for execution.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

It's been awhile since I set it up, but it's definitely possible for PyCharm to do this. I remember having a difficult time getting it working as well.

Anyway, I'm not going to be at my dev machine all day to look in to it further, but I wanted to say that it is possible to do so don't give up!

Nippashish
Nov 2, 2005

Let me see you dance!

vikingstrike posted:

Install pycharm on the remote server and forward it over X?

I tried this too but the server is far away so it is pretty painful. Dropbox is a good idea though, I don't know why I didn't think of that.

evensevenone
May 12, 2001
Glass is a solid.
Seems like what you want could be achieved by a combination of inotifywait and rsync. This is a bash one-liner that I use a lot (the rsync line might be wrong, since I use a script that builds an packages, copies it, and pip installs it, but you get the idea).

code:
while true; do inotifywait -r . -e close_write; rsync -avz . remote_server: ; done;
Just leave that going in a terminal, and whenever you save a file everything gets synced remotely. If you wanna be a pro, throw your unit test runner in there too.

Scaevolus
Apr 16, 2007

Nippashish posted:

That is more complicated than what I am trying to do (although remote debugging is something I'd like to get working eventually), and also doesn't solve the problem I'm having. Step 2 in "To prepare for remote debugging" on that page is "Copy the local script to the remote location, where you want to debug it.", which is precisely what I want pycharm to handle for me.

Right now I have a nice tweak script -> run script -> get results workflow on my local machine. I'd like to keep this workflow but have the "run script" step execute on the server instead of on my laptop. Something like tramp mode in emacs would be fantastic. Of course I can just work on the server directly with vim+ssh but I'd much rather use pycharm because it is very nice.

Being able to run the code from within pycharm isn't really essential (maybe my original question was a bit misleading), what I really want is something to handle the synchronization. I don't want to use source control for this because a big part of what I'm doing is making tiny changes between runs (like changing a 0.1 to 0.15 to see what happens) and I don't want a million tiny commits with useless parameter changes, and I also don't want tweak -> run to turn into tweak -> commit to git -> push to remote -> switch to server -> pull from remote -> run.

I don't think mocking the hardware makes sense here because what I really care about is the program output. The special hardware I'm using is a GPU to do fast linear algebra and as far as I know mocking this would mean either making up numbers (which doesn't help me because I need the non-made-up numbers) or running the calculations on the CPU, which is what I'm doing now, but the whole point of using the GPU is that it's a lot faster.
Try sshfs.

SurgicalOntologist
Jun 17, 2004

I think PyCharm can automate sshfs for you. Check out Tools > Deployment > Automatic Upload. You will have to configure first using Tools > Deployment > Configure.

Nippashish
Nov 2, 2005

Let me see you dance!

SurgicalOntologist posted:

I think PyCharm can automate sshfs for you. Check out Tools > Deployment > Automatic Upload. You will have to configure first using Tools > Deployment > Configure.

I eventually found a stack overflow post this afternoon that pointed me at this option, and it does the trick. I've got remote execution and debugging working and it's really quite nice now that I know how to set it up.

namaste friends
Sep 18, 2004

by Smythe
I'm running pycharm on a Mac and a windows box. Getting remote compilation working wasn't hard once I figured out that you have to pay close attention to file encoding. This was for ide on Windows and compiler on Linux.

Hanpan
Dec 5, 2004

Solved. I was being an absolute moron and raising the wrong type of exception.

I am trying to handle various kinds of exceptions like so:

code:
try:
	some_method()
except CustomException1 as exc:
	# do some custom stuff here
	raise exc
except CustomException2 as exc:
	# do some other custom stuff here
	raise exc
except Exception as exc:
	print "beep"
	raise exc
The problem I have is the final part of that code (except Exception) is always running, because obviously both CustomException1 and CustomException2 inherit from Exception. I was hoping that the re-raise within each handler would bypass this but whatever exception I raise (CustomException1, CustomException2) is resulting in a "beep".

Hanpan fucked around with this message at 00:03 on Apr 24, 2014

SurgicalOntologist
Jun 17, 2004

Huh. I thought only one except block could fire.

I have a similar problem in my library, actually. I have a try-except block that catches any exception, backups any intermediate data then re-raises the exception, but it always fires twice and I can't figure it out. It must be something else though, there's only one except block in this case.

Dominoes
Sep 20, 2007

Looking for advice on generators/streaming. Generators are a tool I've read a lot about, and understand the basics of, but couldn't figure out where to use. I think I have a use: Streaming data.

Here's the main streaming function, which I think is set up correctly. The code's a bit messier than it should be, due to the source data not splitting up by line.
Python code:
def streaming(symbols):
    url = 'https://stream.tradeking.com/v1/market/quotes.json'
    resp = s.get(url, params={'symbols': ','.join(symbols)}, stream=True)
    buff = ""
    for chunk in resp.iter_content():
        buff += chunk.decode()
        if len(buff) > 2 and buff[-2:] == '}}':
            stream = buff
            buff = ""
            yield json.loads(stream)
Here's a function that should return a current stock quote based on the stream.
Python code:
def stream_last(streamer, symbol):
    for i in streamer:
        if trade_data['symbol'] == symbol:
            return float(trade_data['last'])

I think what it's actually doing is returning the next value, which may be long outdated. If I set up a variable equal to the streaming generator, I think it might be running in the background, and when iterated through, picks up where it left off - some time in the past. Is there a way to jump to the latest value in a generator? What can I do to pull the most recent data?

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
Why wouldn't it let me upgrade pip with the first attempt?

code:
$ pip install --upgrade https://my.url.to/pip-1.5.4-py2.py3-none-any.whl
Downloading/unpacking https://my.url.to/pip-1.5.4-py2.py3-none-any.whl
  Downloading pip-1.5.4-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.4.1
    Not uninstalling pip at /home/fletcher/.venvburrito/lib/python/pip-1.4.1-py2.7.egg, outside environment /home/fletcher/.virtualenvs/myvirtualenv
Successfully installed pip
Cleaning up...

$ pip --version
pip 1.4.1 from /home/fletcher/.venvburrito/lib/python/pip-1.4.1-py2.7.egg (python 2.7)

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/source/p/pip/pip-1.5.4.tar.gz#md5=834b2904f92d46aaa333267fb1c922bb
  Downloading pip-1.5.4.tar.gz (1.1MB): 1.1MB downloaded
  Running setup.py egg_info for package pip
    
    warning: no files found matching 'pip/cacert.pem'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
    no previously-included directories found matching 'docs/_build/_sources'
Installing collected packages: pip
  Found existing installation: pip 1.4.1
    Not uninstalling pip at /home/fletcher/.venvburrito/lib/python/pip-1.4.1-py2.7.egg, outside environment /home/fletcher/.virtualenvs/myvirtualenv
  Running setup.py install for pip
    
    warning: no files found matching 'pip/cacert.pem'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
    no previously-included directories found matching 'docs/_build/_sources'
    Installing pip script to /home/fletcher/.virtualenvs/myvirtualenv/bin
    Installing pip2.7 script to /home/fletcher/.virtualenvs/myvirtualenv/bin
    Installing pip2 script to /home/fletcher/.virtualenvs/myvirtualenv/bin
Successfully installed pip
Cleaning up...

$ pip --version
pip 1.5.4 from /home/fletcher/.virtualenvs/myvirtualenv/lib/python2.7/site-packages (python 2.7)

emoji
Jun 4, 2004
lol if you aren't using anaconda for your environments in TYOOL 2014

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

kraftwerk singles posted:

lol if you aren't using anaconda for your environments in TYOOL 2014

Can you elaborate??

emoji
Jun 4, 2004
https://store.continuum.io/cshop/anaconda/

You create your environments using the 'conda' tool. After that, it can be treated almost as a drop in replacement for virtualenv. It includes many packages such as pandas precompiled. Building environments using tools such as puppet takes seconds instead of minutes. I use it in dev and production.

Include pip when creating your environment to install packages outside the anaconda ecosystem.

bigreddot who posts in this thread works on this project I believe.

BigRedDot
Mar 6, 2008

kraftwerk singles posted:

bigreddot who posts in this thread works on this project I believe.

I work for Continuum, and I wrote the original version of conda but it has been taken over and taken much further by others in the last year. These days I mostly work on Bokeh. Happy to answer any Anaconda questions, though!

Nippashish
Nov 2, 2005

Let me see you dance!

BigRedDot posted:

I work for Continuum, and I wrote the original version of conda but it has been taken over and taken much further by others in the last year. These days I mostly work on Bokeh. Happy to answer any Anaconda questions, though!

When is conda going to support activating environments by relative path? It's silly to need to source activate $(pwd)/venv instead of just source activate venv to activate an environment in my working directory. Why does conda want to keep my environments in a central location by default? I don't understand the workflow that fits with. (I use conda every day tyvm for writing it).

ohgodwhat
Aug 6, 2005

kraftwerk singles posted:

lol if you aren't using anaconda for your environments in TYOOL 2014

So, uh, how do you install things into a conda environment?

BigRedDot
Mar 6, 2008

Nippashish posted:

When is conda going to support activating environments by relative path? It's silly to need to source activate $(pwd)/venv instead of just source activate venv to activate an environment in my working directory. Why does conda want to keep my environments in a central location by default? I don't understand the workflow that fits with. (I use conda every day tyvm for writing it).

Yah that was an early decision, we wanted simple named environments but also didn't want to have to introduce a persistence layer to map those names to absolute paths. conda started out as just a devops tool for us, if I'd known how popular it would become I probably would have made a different decision. Still I can imagine it would not be to difficult to add this kind of behavior: source activate -r <relpath> or something similar. I'll mention this to Aaorn and Ilan but the best way is to make a GH issue (https://github.com/conda/conda) or ask about it on the mailing list. (Edit: or even better a Pull Request!)

More Edit: I think source activate ./myenv should work too?

ohgodwhat posted:

So, uh, how do you install things into a conda environment?

pip still works fine for things that are not conda packages. For conda packages (which are really just tarballs of a /usr/local style hierarchy) you'd first create an environment:
code:
conda create -n <envname> python=3 # goes in <anaconda install dir>/envs/<envname>

# or

conda create -p /full/path/to/env python pyzmq
Then install into it:
code:
conda install -n <envname> numpy=1.7 pandas  # goes in <anaconda install dir>/envs/<envname>

# or

conda install -p /full/path/to/env numpy=1.8 scipy

BigRedDot fucked around with this message at 15:58 on Apr 25, 2014

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

BigRedDot posted:

pip still works fine for things that are not conda packages.

How do we know when to use which command?

Brennan
Jan 18, 2005

I just worked through Learning Python by Lutz and feel like I have a decent beginners level grasp of the language. I'm learning because I'm in a computer forensics program, and it's a good skill to have in the field but the program doesn't really touch on it. Are there any recommendations of newbie friendly places where I might be able to contribute and it would let me practice and learn through doing? I'm willing to put in the time, I'm just hoping for some direction.

Adbot
ADBOT LOVES YOU

OnceIWasAnOstrich
Jul 22, 2006

Thermopyle posted:

How do we know when to use which command?

The way I usually end up doing it is try conda first and if it isn't available as a conda package then use pip. Alternatively if you want to build the package as normal use pip anyway. At one point on one of my computers I ended up with a conda install command that would automatically call pip, although I am pretty sure I didn't do that myself so I'm not sure where it acquired that capability.

  • Locked thread