|
Much appreciated, I don'tknow anything about Selenium either so I'll look into that. A lot of folks at my company use Selenium so picking up some knowledge there might come in handy some time.
|
# ? Jan 26, 2017 21:58 |
|
|
# ? May 9, 2024 14:36 |
|
You used to be able to just pop a Firefox browser out of the box with one call, now you need to mess around configuring it a bit (which sucks) but otherwise it's still pretty straightforward. Start browser, get page, then you can grab the html like you're already doing, or do more complicated stuff like clicking elements or waiting for things to become visible
|
# ? Jan 26, 2017 22:41 |
|
I am having the worst trouble using Python to respond to a HTTP post request, I'm sending the post using jquery code:
code:
socialsecurity fucked around with this message at 23:04 on Jan 27, 2017 |
# ? Jan 27, 2017 23:00 |
Yesterday I went to my local Python Devs meet-up. (San Diego Python). It was a great time! I'd like to encourage everyone in this thread to check out their local python dev community. Also, I just bought tickets to my first ever PyCon - I'll be there in Portland, Oregon. I've gotten so much out of the recorded talks from PyCon - I can't wait to go! Tickets are sure to sell out soon, so if you're interested, try to register ASAP. If the standard price ($300) looks steep, student tickets are available, as well as financial aid. One of the best parts of Python is the community around the language. I'd really love to encourage anyone who reads the thread to become a closer part of it.
|
|
# ? Jan 27, 2017 23:17 |
|
socialsecurity posted:if anyone could even point me in the right direction here I'd be grateful. What library/framework are you using? If you're just writing to stdout, that's not going to get returned in the response - most libraries have a function that you can call or a value you can return that gets turned into the response.
|
# ? Jan 27, 2017 23:26 |
|
Asymmetrikon posted:What library/framework are you using? If you're just writing to stdout, that's not going to get returned in the response - most libraries have a function that you can call or a value you can return that gets turned into the response. Python 2.7 import requests import urllib import json import sys import ast import re from json import JSONEncoder are my imports, I tried just "print" at one point and "response" neither seemed to make a difference.
|
# ? Jan 27, 2017 23:32 |
|
socialsecurity posted:I am having the worst trouble using Python to respond to a HTTP post request, I believe "Status: %s\n" is supposed to be "HTTP/1.1 %s\n". The client may be failing to parse the response. Double-check that standard out is actually the output stream that goes back to the client. In particular, I'm surprised that you're not at least writing to a socket.
|
# ? Jan 28, 2017 15:02 |
|
Quick Flask question, trying to write a simple chat server and was going to just store messages in a list structurecode:
I'm sure I'll eventually move this over to some kind of fancier datastore but this is just for testing. How can I access the same variable across functions in a Flask program?
|
# ? Jan 28, 2017 18:52 |
|
You need to set that as global if you're writing to it, right? I mean technically it's a read then an assignation, but does Python still count it as a local variable from the beginning since it knows it's going to be written to?
|
# ? Jan 28, 2017 18:56 |
Bob Morales posted:Quick Flask question, trying to write a simple chat server and was going to just store messages in a list structure It's time for you to learn about scope! You have a couple options. You can use the global statement, which will do exactly what you want. This can be dangerous, though, as anything can touch the global namespace. It's better to wrap the whole thing in some kind of main() function and use the nonlocal statement. It's easiest to get a hang of how global and nonlocal work via example. So: IN: Python code:
code:
|
|
# ? Jan 28, 2017 19:27 |
|
If you want to do it with a global variable in Flask, you can do it this way. But note that it's a value attached to any instance of the Flask server running. So if you're running two Flask servers they would have different counts. Look into the Flask.g object as well, which might not fit your needs but it's worth knowing about.Python code:
|
# ? Jan 28, 2017 19:39 |
|
'ST posted:If you are writing a Python HTTP server as an exercise, go for it. However, if your main goal is to return data and not to explore HTTP response formats, I would recommend using an HTTP server library or BaseHTTPServer in the Python 2.7 standard library to handle the details for you. I'll admit I'm new to this and mainly helping out a friend with a little project he's working on, but the HTTP server is Apache hosting wordpress do we need to run a Python server library on top of that perhaps I am misunderingstanding what a server means in the context of Python 2.7
|
# ? Jan 28, 2017 19:50 |
|
socialsecurity posted:I'll admit I'm new to this and mainly helping out a friend with a little project he's working on, but the HTTP server is Apache hosting wordpress do we need to run a Python server library on top of that perhaps I am misunderingstanding what a server means in the context of Python 2.7
|
# ? Jan 28, 2017 20:11 |
|
socialsecurity posted:I'll admit I'm new to this and mainly helping out a friend with a little project he's working on, but the HTTP server is Apache hosting wordpress do we need to run a Python server library on top of that perhaps I am misunderingstanding what a server means in the context of Python 2.7 Your Python code is receiving data somehow, right? Then it does something and returns a result, and you want that result to eventually reach whatever made the POST request It depends how your Python code is receiving the data in the first place. Is the webserver piping it into your Python script , like myscript.py "some data" and looking for a result it can send back? In that case you can just print to stdout and the webserver will get it. In this case your script isn't actually handling HTTP, it's just working on some JSON data or whatever and spitting out the results. Or is your Python running a HTTP server that directly receives the POST request? In that case you need to send back the results in a HTTP response. Usually you'd have a library someone already wrote to do this, so you can handle the incoming request and form a response easily. There'd be some established workflow for this, so you don't have to worry too much about the technical details It's not really clear what you're doing, but if you're printing your results to stdout then that implies something else is handling all the HTTP, and if you're having problems with responses then it's probably occurring there (I might have said something stupid here because I don't know much about http )
|
# ? Jan 28, 2017 20:13 |
|
Ghost of Reagan Past posted:Well, what are you trying to do, specifically? Why are you sending JQuery POST requests? What's the server supposed to be doing? If you're running Wordpress, what do you need to do that can't be done with Wordpress/requires Python? You're probably overcomplicating this, and should probably evaluate what exactly you're doing (you might not need Python at all, and if you do need Python then you should probably use a preexisting solution such as a web framework or something similar) and get clear on why you think you need to do this. We are having someone send their address via a http wordpress form the info gets posted via j query Ajax to the python script, the python then takes the address info and runs it through a series of pre-made apis that it's supposed to send the info from the apis back. We are using python because that's what the api sending code was written in originally. So far the former sends the data to the python script which does process the data right if we pipe it to the console everything looks as it should just can't get it to respond properly to the Ajax post request.
|
# ? Jan 28, 2017 20:51 |
|
socialsecurity posted:We are having someone send their address via a http wordpress form the info gets posted via j query Ajax to the python script, the python then takes the address info and runs it through a series of pre-made apis that it's supposed to send the info from the apis back. We are using python because that's what the api sending code was written in originally. So far the former sends the data to the python script which does process the data right if we pipe it to the console everything looks as it should just can't get it to respond properly to the Ajax post request. I'm pretty sure your life would be a lot easier with Flask or Django here.
|
# ? Jan 28, 2017 23:27 |
|
Thermopyle posted:I'm pretty sure your life would be a lot easier with Flask or Django here. I've started looking into this but it seems it would require quite a bit of rewrite I didn't think sending a json back to a html post would be this much of an undertaking but thank you I think in the end going down this path would be easier to work on down the road.
|
# ? Jan 28, 2017 23:34 |
|
socialsecurity posted:I've started looking into this but it seems it would require quite a bit of rewrite I didn't think sending a json back to a html post would be this much of an undertaking but thank you I think in the end going down this path would be easier to work on down the road. http is a bitch with a bunch of weird edge cases and whatnot. It's just way easier to use a library that someone else dealt with all this crap.
|
# ? Jan 28, 2017 23:47 |
|
Also, both Flask and Django are perfectly capable of doing something super simple like you mentioned, but their getting started docs both assume you're wanting to do something more complicated so...that sucks for you. Just keep powering through and you're going to know stuff that is useful to know...
|
# ? Jan 28, 2017 23:48 |
|
socialsecurity posted:We are having someone send their address via a http wordpress form the info gets posted via j query Ajax to the python script, the python then takes the address info and runs it through a series of pre-made apis that it's supposed to send the info from the apis back. We are using python because that's what the api sending code was written in originally. So far the former sends the data to the python script which does process the data right if we pipe it to the console everything looks as it should just can't get it to respond properly to the Ajax post request. How does it actually work? How does your ajax request end up talking to the python script? How does the data get in?
|
# ? Jan 28, 2017 23:58 |
|
baka kaba posted:How does it actually work? How does your ajax request end up talking to the python script? How does the data get in? code:
code:
|
# ? Jan 29, 2017 00:30 |
|
socialsecurity posted:I've started looking into this but it seems it would require quite a bit of rewrite I didn't think sending a json back to a html post would be this much of an undertaking but thank you I think in the end going down this path would be easier to work on down the road. falcon is my go-to for quick and easy apis: Python code:
code:
Dex fucked around with this message at 02:38 on Jan 29, 2017 |
# ? Jan 29, 2017 02:35 |
|
Dex posted:falcon is my go-to for quick and easy apis: This makes sense more then other options I've seen up until a point, why does it look like I'm running another server to serve this out this is already hosted/exposed via my apache does it have to be hosted on its own to return properly?
|
# ? Jan 29, 2017 03:59 |
|
If you're using some kind of CGI thing where you're meant to just drop the results to stdout, and it looks fine on the command line, seems like the problem is elsewhere no? Did you debug whatever's making the ajax call and see exactly what the response it's getting is? Maybe adding an error callback would be useful too, instead of just the success one
|
# ? Jan 29, 2017 04:23 |
|
socialsecurity posted:This makes sense more then other options I've seen up until a point, why does it look like I'm running another server to serve this out this is already hosted/exposed via my apache does it have to be hosted on its own to return properly? https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
|
# ? Jan 29, 2017 06:25 |
|
I have written a small web scraper script that gets a certain piece of information from a website. I want to have the script run (i.e., get the piece of information from the website) every hour for a week. Is the Pythonic way to do this to simply write a for loop that calls the web scraper script and uses the time.sleep() function to implement an hour delay in between calls? One downside to this way is that if the computer restarts for updates etc. the for loop will stop. Is there a solution that works around this?
|
# ? Jan 30, 2017 21:32 |
|
Jose Cuervo posted:I have written a small web scraper script that gets a certain piece of information from a website. What OS are you running?
|
# ? Jan 30, 2017 21:43 |
|
Jose Cuervo posted:I have written a small web scraper script that gets a certain piece of information from a website. Leave the scheduling to the OS (cron if linux or mac and Task Scheduler if windows) and store state in a sqlite DB or CSV file. Simple is better than complex. Although practicality beats purity.
|
# ? Jan 30, 2017 21:55 |
|
I like to engineer the most impractical and complicated solution possible. In this case I'd set up celery workers and celery beat as daemons and then configure a periodic task to run. (no i wouldnt i'm just bitter that I worked on fixing up a project that did this once for just one little periodic thing instead of just using cron)
|
# ? Jan 30, 2017 22:05 |
|
accipter posted:What OS are you running? Tigren posted:Leave the scheduling to the OS (cron if linux or mac and Task Scheduler if windows) and store state in a sqlite DB or CSV file. So get task scheduler to run the script once an hour for a week and store the output in a file/ database so that if the computer restarts I have the information saved? Cool. Not quite sure what you mean by 'practicality beats purity'.
|
# ? Jan 30, 2017 22:12 |
|
Jose Cuervo posted:Not quite sure what you mean by 'practicality beats purity'. Open up python and type "import this" Or go read PEP 20.
|
# ? Jan 30, 2017 22:32 |
|
Jose Cuervo posted:I'm running Windows 7. Yep, you got it. Python has fantastic Sqlite support as well as csv support. Like FoiledAgain said, check out PEP 20. Sometimes, if it's more practical, it's better to do things either unpythonic or even outside of Python. PEP 20 is not the law, but typically a great set of guidelines.
|
# ? Jan 30, 2017 23:38 |
|
I hate getting assigned to a project and discovering that they wrote their own half-baked daemon and gave it a special service account just to run a process that has to execute every 10 minutes
|
# ? Jan 30, 2017 23:51 |
|
FoiledAgain posted:Open up python and type "import this" Tigren posted:Yep, you got it. Python has fantastic Sqlite support as well as csv support. Ah, I asked for a Pythonic solution but was given a practical solution instead. Got it.
|
# ? Jan 31, 2017 02:17 |
|
Okay, dumb question. I've got a set of words, and I want to create a list of every possible permutation of those words up to length n (so if n = 3, I want every three word permutation, every two word permutation and then every word.) I'm doing this by using itertools.permutations and a lot of nested For loops, but this seems like kind of a clunky way to implement it. (Also I'm not sure how to insert a space between the words, since permutations doesn't do this by default, i.e. for [1, 2] I want "1 2" rather than "12") There has to be some built-in that does this or something like it, right?
|
# ? Jan 31, 2017 04:48 |
|
Baby Babbeh posted:Okay, dumb question. I've got a set of words, and I want to create a list of every possible permutation of those words up to length n (so if n = 3, I want every three word permutation, every two word permutation and then every word.) I'm doing this by using itertools.permutations and a lot of nested For loops, but this seems like kind of a clunky way to implement it. (Also I'm not sure how to insert a space between the words, since permutations doesn't do this by default, i.e. for [1, 2] I want "1 2" rather than "12") I would do something like this: Python code:
|
# ? Jan 31, 2017 06:02 |
Baby Babbeh posted:Okay, dumb question. I've got a set of words, and I want to create a list of every possible permutation of those words up to length n (so if n = 3, I want every three word permutation, every two word permutation and then every word.) I'm doing this by using itertools.permutations and a lot of nested For loops, but this seems like kind of a clunky way to implement it. (Also I'm not sure how to insert a space between the words, since permutations doesn't do this by default, i.e. for [1, 2] I want "1 2" rather than "12") Remember, there are n!//(n-k)! k-permutations of n words. So if you want all the non-empty permutations up to k, that's Python code:
FUNCTIONS: Python code:
Python code:
['hello darkness old', 'hello darkness friend', 'hello old darkness', 'hello old friend', 'hello friend darkness', 'hello friend old', 'darkness hello old', 'darkness hello friend', 'darkness old hello', 'darkness old friend', 'darkness friend hello', 'darkness friend old', 'old hello darkness', 'old hello friend', 'old darkness hello', 'old darkness friend', 'old friend hello', 'old friend darkness', 'friend hello darkness', 'friend hello old', 'friend darkness hello', 'friend darkness old', 'friend old hello', 'friend old darkness', 'hello darkness', 'hello old', 'hello friend', 'darkness hello', 'darkness old', 'darkness friend', 'old hello', 'old darkness', 'old friend', 'friend hello', 'friend darkness', 'friend old', 'hello', 'darkness', 'old', 'friend'] Note: There is certainly nothing wrong with accipter's solution. My only advice would be to use generator functions for the intermediate comprehensions. Since you're not going to use the intermediate results in all_perms, building the intermediate list(s) just wastes memory and slows down the result (though probably not by too much). Luckily, you can do so simply by replacing two pairs of brackets with parentheses! Also, I tested it, and it turns out: Python code:
Python code:
['one', 'two', 'three', 'four', 'one two', 'one three', 'one four', 'two one', 'two three', 'two four', 'three one', 'three two', 'three four', 'four one', 'four two', 'four three', 'one two three', 'one two four', 'one three two', 'one three four', 'one four two', 'one four three', 'two one three', 'two one four', 'two three one', 'two three four', 'two four one', 'two four three', 'three one two', 'three one four', 'three two one', 'three two four', 'three four one', 'three four two', 'four one two', 'four one three', 'four two one', 'four two three', 'four three one', 'four three two'] Eela6 fucked around with this message at 22:57 on Jan 31, 2017 |
|
# ? Jan 31, 2017 19:38 |
|
Eela6 posted:I would do it like this... This is well done and better than my quick solution. I hadn't seen "yield from" and checking for an iter in that manner -- so thanks for those examples.
|
# ? Jan 31, 2017 19:56 |
Thanks! I picked up those tricks from Luciano Ramalho's Fluent Python and Brett Slatkin's Effective Python, respectively. I am writing a speech for my local python developers group. It's called 'The Elements of Style.' It's still in the works, but my rough draft is something like 80% complete. I would love advice on the slides. Take a look and let me know what you think! http://tinyurl.com/sdpystyle Eela6 fucked around with this message at 22:56 on Jan 31, 2017 |
|
# ? Jan 31, 2017 20:01 |
|
|
# ? May 9, 2024 14:36 |
|
Wow, thanks for that Eela6. That's a lot more elegant than what I was doing, which was creating a lot of intermediate data structures that I wasn't actually using that wasted memory. I think I really need to learn more about generators... I kind of know how to use them but I feel like I'm really just scratching the surface.
|
# ? Jan 31, 2017 20:18 |