Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›3 »

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Could this also be made to run on Android with NDK? You could have a neat cross-platform kit if you got that figured out, but just getting it working on iOS would be plenty cool all on its own.

# ¿ Nov 25, 2015 15:11

Adbot: ADBOT LOVES YOU

# ¿ May 9, 2024 11:36

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

duck monster posted:

I believe it already runs on Android.

Ah, so it does. Not sure why that didn't come up when I googled earlier.

E: google auto-corrects pyotherside to otherside haha thanks goog

Munkeymon fucked around with this message at 18:53 on Nov 25, 2015

# ¿ Nov 25, 2015 17:52

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Thermopyle posted:

I think Suspicious Dish first told me this, and I now agree with him: javascript is very similar to python.

And becoming more and more so with all the new features they're adding.

# ¿ Nov 25, 2015 18:54

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Use Selenium's Python bindings and http://stackoverflow.com/questions/3422262/take-a-screenshot-with-selenium-webdriver

Part of their API lets you pick a frame to interact with http://stackoverflow.com/questions/13567128/selenium-frames

Or you can just record your own actions and replay them automagically if hand-coding the workflow is too much http://www.seleniumhq.org/projects/ide/

# ¿ Dec 9, 2015 15:27

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Try

Python code:

import os
import sys

null = open(os.devnull, 'w')
temp = sys.stderr
sys.stderr = null

#let the extension do its thing here

sys.stderr = temp #restore it
null.close()

# ¿ Jan 20, 2016 21:27

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Tigren posted:

And where am I supposed to be seeing the output that I want to capture?

Before resetting sys.stderr, sys.stderr is just <open file '/dev/null', mode 'w' at 0x7fad40d5ec90>

Oh sorry I got it in my head that you just wanted to suppress it.

Python code:

import os
import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

#let the extension do its thing here

sys.stderr = temp #restore it
capture.seek(0) #readable as a normal file

# ¿ Jan 20, 2016 22:45

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Tigren posted:

Python code:

import os
import sys
from io import TextIOWrapper, BytesIO

import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

pacparser.init()
pacparser.parse_pac_file('test.pac')
pacparser.find_proxy('http://www.google.com')

sys.stderr = temp
capture.seek(0)
if capture:
    print "error captured"

No 'error captured' printing. 'capture' is empty.

You'd still have to do something about the exception:

Python code:

import sys
from io import TextIOWrapper, BytesIO
import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp

capture.seek(0)
print(capture.read())

Also capture is truthy as long as the call to TextIOWrapper didn't throw, so if capture: would always run the if body.

# ¿ Jan 21, 2016 17:18

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Tigren posted:

Thanks for being patient with me. stderr/stdout redirection seems like kind of a funky beast.

I tried running that from within a REPL and from a script
code:
$ python test.py
JSERROR: PAC script:3:
    ReferenceError: isnNet is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.
that did not go well!
I just get a blank line output for console.read(0).

Once I realized what you were doing, I looked into stderr redirection and saw mention of stderr caching. Do you think the JSERROR is cached and therefore not directly accessible from sys.stderr redirection?

Caching shouldn't be an issue because we're supplying our own buffer before the output is generated. Could be I made an invalid assumption and it might be sending the output on stdout. Could be that there's an init in the module that grabs copies of the std* streams before we're replacing them. This should mitigate both:

Python code:

import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp_err = sys.stderr
temp_out = sys.stdout
sys.stderr = sys.stdout = capture #mess with all the outputs

import pacparser #don't let it init until after we mess with outputs

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp_err
    sys.stdout = temp_out

capture.seek(0)
print("output: {0}|".format(capture.read()))

And, someone correct me if I'm wrong here, but I think the extension could be opening its own std* streams, which I don't think you can do anything about other than shove it in a subprocess and capture all the output.

# ¿ Jan 21, 2016 19:40

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

When you're searching for an ID like cur_pm25, you should just use find because there shouldn't be more than one on the page but you also seem to be confusing classes and IDs. aqivalue is a class, so there can be more than one, but you can do

Python code:

find(id='citydivouter').find(attrs={"class": "aqivalue"})

Python code:

select('#citydivouter .aqivalue')[0]

to get the big one.

E: forgot class was a keyword :downs:

Munkeymon fucked around with this message at 17:11 on Jan 28, 2016

# ¿ Jan 28, 2016 17:09

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

hooah posted:

Say I have a string test='hello\tworld'. Why does test.find(string.whitespace) return -1? I know tab is in string.whitespace.

I'm guessing tab isn't the only thing in string.whitespace? .find searches for a string(needle) in a string(haystack), not any arbitrary sub-part of the needle in the haystack.

E: are you trying to figure out if there's any whitespace or find the first whitespace character?

Munkeymon fucked around with this message at 17:25 on Mar 1, 2016

# ¿ Mar 1, 2016 17:20

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Well the regular expression /\s/ will match all white space characters, hint hint

Also it's really lovely of them to mix tab and space delimiters JFC

# ¿ Mar 1, 2016 17:51

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

They're not loving with the delimiter, you're just trying to treat the first column as two columns when it's actually a bucket. The CSV module with delimiter set to tabs should handle that just fine.

# ¿ Mar 1, 2016 19:51

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Hammerite posted:

* like not having a notion akin to Python's "class methods", where you have an essentially static method but it cares which class you used to call it

Extension methods don't do what you want?

# ¿ Apr 8, 2016 18:21

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Hammerite posted:

I don't know what you mean - I don't understand how extension methods would be relevant to this, but if you can point me to something I don't know about them that would be great.

Might be me misunderstanding why classmethods are useful, but in C# if I want to make a utility method that I can just call as if it were a member of a type when it's not, I'd write an extension method. Granted, this doesn't give the extension method access to private members or anything similarly magical, but it looks like a similar use case to me.

# ¿ Apr 11, 2016 15:05

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Hammerite posted:

No, that's not what I mean. I mean I wanted to have something like Python's "classmethod" where it acts like a static method (you don't need an instance to call it), but it does still care which class it's called through. A dumb, babby's- first-OOP type example would be
code:
import abc

class Animal(metaclass = abc.ABCMeta):
    @classmethod
    @abc.abstractmethod
    def noise(cls):
        return 'A noise an animal makes'

class Cat(Animal):
    @classmethod
    def noise(cls):
        return 'Meow'

class Dog(Animal):
    @classmethod
    def noise(cls):
        return 'Woof'

print(Cat.noise()) # Meow
print(Dog.noise()) # Woof
You can't do this in C#. An abstract method can't be static, and although you do have "type" objects you can't call type-specific methods on them.

You kind-of can, though: https://dotnetfiddle.net/TRTDYx E: well, you can't override the implementation's methods with an extension - you'd have to inherit and explicitly override to do that. To associate a static method with a type is totally doable, though.

Munkeymon fucked around with this message at 16:58 on Apr 11, 2016

# ¿ Apr 11, 2016 16:02

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

taqueso posted:

I was trying to think of where this might not be true, and in Rust, it seems like the unary operators (including -) have higher precedence than binary operators.
code:
fn main() {
    let x = 5;
    assert_eq!(  - x + 1     ,    -4);
    assert_eq!(  - (x + 1)   ,    -6);
    assert_eq!(  (- x) + 1   ,    -4);
}
https://play.rust-lang.org/?gist=0f0c7aecaa04d12beb7c23c369680f06&version=nightly&backtrace=0

Same in JS. It's almost like they come from the same organization heh

# ¿ Apr 26, 2016 19:58

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Loving Africa Chaps posted:

looks like it was the missing comma after twitterid

now i just need to work out why it thinks it's returned a negative number of rows!

Cheers for everyones help

The comma changes the type from a string to a tuple, in case that wasn't clear.

# ¿ May 5, 2016 20:12

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Loving Africa Chaps posted:

It wasn't and might explain my cool new errors. Got some more reading to do i guess

The method probably looks something like

Python code:

def execute(self, command, args):
    #... probably some important stuff going on up here
    for i, arg in enumerate(args):
        self._bindArgToCommand(i, arg)

Which strikes me as an odd way to handle it in Python because there's great language support for named arguments and argument list unpacking, but hey it's their party.

Anyway, both string and tuple are enumerable, so that for loop will happily consume either, member by member, so it will try to bind every character in a naked string (or code point in Python 3, I guess). The reason you need the (thing,) construct to pass a tuple instead of a string is that that's the special case syntax to create a one-element tuple. An empty tuple would just be empty matching parens. I would be a little surprised if a list didn't work just as well and the syntax there is a little more intuitive: [thing].

# ¿ May 9, 2016 15:27

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

QuarkJets posted:

Personally I feel like this kind of move is tacky

It's kinda rickety and weird, but for better or for worse the web browser is kinda becoming the lingua franca of UI toolkits.

OK yeah fine probably for worse

# ¿ Dec 5, 2016 23:02

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

It's becoming more Pythonic, but in a janky JS-y kind of way, and I say that as a front-end dev, so I'm doing mostly JS for my day job and have been for a while now.

For example, they added generator functions at the language level (yay) but they don't have a .map, .reduce, etc specified on them like regular arrays so you can't just write code that works for both without monkeypatching or going through a library :argh:

# ¿ Dec 7, 2016 15:16

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Suspicious Dish posted:

map in Python implicitly converts to an array. If you want that in JavaScript you have to explicitly convert to an array. [...myGeneratorFunc(foo)].map(...)

I think that's a better idea.

I like the way Py3k and LINQ work better :shrug:

# ¿ Dec 7, 2016 20:18

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Asymmetrikon posted:

You can technically kind of replicate f-strings with format by doing:
Python code:
"Hello, {thing}!".format(**locals())
but that's kinda nasty.

I wish there was room in the spec to add implicit naming to dictionary literals like JS is doing, you could just do something like

Python code:

meatbag = 'some guy - who cares'
"Hello, {meatbag}".format(**{meatbag})

Which would mostly obviate the need for a 'better' formatting option.

Huh, I thought 3 deprecated the modulo formatting, but I guess not? I mean, I would have, but whatever.

# ¿ Dec 19, 2016 17:13

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

They can't do what I did there because {thing} already means "make a set consisting of 'thing'". IDK how you get around that without doing something kind of gross and unintuitive.

# ¿ Dec 19, 2016 20:22

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Thermopyle posted:

I'd really like to see some more in-depth discussion of this.

I mean, they'd have to move to Python 3+ to get asyncio, but it seems like that'd be less work than what they did, so maybe Go concurrency really is that much better? Also, there's other lightweight thread alternatives like eventlet/gevent...

Of course, Google also has a real hard-on for Go, so maybe it's just the new and shiny factor?

I bet it's more work to take a big codebase like YouTube to 3+ than to write a compiler that'll drop-in replace 2.

The fib test they cite just used threads, which would underperform an approach that spun off processes (last I checked, on Windows, for something that uses lots of memory). I'm not familiar enough with how Go handles that sort of thing to know if it's a good comparison, but I'm guessing it's weak and meant to say "Look, ma: no GIL!"

They should have included comparisons to the other alternatives they explored to make a good case, but at the end of the day we're not the audience they need to justify this to.

# ¿ Jan 4, 2017 20:40

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Dex posted:

from some stuff i looked at last year, go's concurrency for cpu intensive threads was around 50x cpython's. however, go has some seriously nasty leaks that are completely beyond your ability to control, so you will almost certainly end up with memory issues that are near impossible to isolate, so if you have even the slightest reason to care about memory consumption, it seems like a terrible choice

i'd aim for pypy before running python in go if i was willing to make tradeoffs on certain libs since it's apparently significantly more performant than cpython, but i don't actually need to so i've only done some light reading on it

Oh that's interesting and now you've got me wishing for a Grumpy vs IronPython cage match since neither gets C extensions.

My money's on IPY because I know I've seen CPU-heavy code run better there than CPython and it has a really good GC behind it :getin:

# ¿ Jan 4, 2017 21:07

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

QuarkJets posted:

Wasn't Go also created at Google?

Yeah

Suspicious Dish posted:

CPython is really, really bad and even with the new "asyncio" stuff even something like node.js can run circles around it.

I said this in a different thread, I think, but Mozilla says you can get a pretty good performance boost by compiling Python to Web Assembly or asmjs (or whatever it's called this year) and running the result on SpiderMonkey. It's probably not a terribly fair comparison, though, because JS interpreters don't have to care about threading or C interop. Still kinda neat, though.

# ¿ Jan 5, 2017 15:05

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Thermopyle posted:

Where does one even get CSV-formatted data nowadays? Excel exports?

"See attached report"

Outside of that, in my experience, it's still a common data exchange format because it's so widely supported and 'easy', meaning the pain comes in dribs and drabs forever instead of being in the up-front setup effort.

# ¿ Jan 23, 2017 23:00

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Thermopyle posted:

The best way to do this is look at that javascript and see what it doing to get the date/time and then do that with python. So, maybe it's doing an AJAX request to some url, and you can do this with requests in python. This often fails because you've got to have specific cookies or something else the page sets up in its environment and it's a bitch to reverse engineer.

If it's too complicated, you can try PyExecJS, but that often fails because it's not running in a browser environment. Usually what I do when I get to this point is use PhantomJS with Selenium to do what I need to do...that usually means throwing away all your BS4 and requests code and doing it all in PhantomJS.

You can also have Phantom/Slimer save the rendered DOM as HTML to disk pretty easily if there's no complicated login process to get to it.

# ¿ Jan 26, 2017 18:51

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Dominoes posted:

Thoughts?

Sounds like you want a rules engine.

# ¿ Feb 10, 2017 14:43

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Dominoes posted:

Thanks, homes. From what I gather so far, sounds like I need to roll my own, STS!!! Django.

There aren't some already kicking around out there?

# ¿ Feb 13, 2017 18:38

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

https://pymotw.com/3/ deserves a mention. I know the standard modules are a functionality mausoleum but there's still good stuff in there i you know where to look.

# ¿ Feb 28, 2017 16:43

Adbot: ADBOT LOVES YOU

# ¿ May 9, 2024 11:36

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Dominoes posted:

Thread's up.

Still looking for additions and edits.

https://pymotw.com/3/

http://ironpython.net/ still technically exists!

# ¿ Mar 6, 2017 18:53

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›3 »