Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

ShadowHawk: Jun 25, 2000; CERTIFIED PRE OWNED TESLA OWNER

Python code:

for _ in (print(movie['title']) for movie in rt.lists('dvds', 'new_releases')['movies']): pass

Mine is clearly the best.

# ? Apr 5, 2015 02:09

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 20:31

cinci zoo sniper: Mar 15, 2013

The book I'm going through at the moment says that _ should return last output obtained. I reconstruct their example:

Python code:

run ball1.py
0.05095

_
Out[4]: ''

And I don't get the number I got from 'ball1.py', which is what book implies I should be getting. IPython QtConsole 3.0.0, Python 2.7.9.

# ? Apr 5, 2015 17:35

EAT THE EGGS RICOLA: May 29, 2008

kalstrams posted:

The book I'm going through at the moment says that _ should return last output obtained. I reconstruct their example:
Python code:
run ball1.py
0.05095

_
Out[4]: ''
And I don't get the number I got from 'ball1.py', which is what book implies I should be getting. IPython QtConsole 3.0.0, Python 2.7.9.

Can you post exactly what you're doing?

# ? Apr 5, 2015 17:45

good jovi: Dec 11, 2000; 'm pro-dickgirl, and I VOTE!

kalstrams posted:

The book I'm going through at the moment says that _ should return last output obtained. I reconstruct their example:
Python code:
run ball1.py
0.05095

_
Out[4]: ''
And I don't get the number I got from 'ball1.py', which is what book implies I should be getting. IPython QtConsole 3.0.0, Python 2.7.9.

The _ variable contains the return value of the last python function call you made. run isn't a python function, it's an ipython command that doesn't return anything.

Look in ball1.py, I'm betting it's just printing that number out, which isn't the same as returning it.

# ? Apr 5, 2015 17:53

cinci zoo sniper: Mar 15, 2013

EAT THE EGGS RICOLA posted:

Can you post exactly what you're doing?

ball1.py

Python code:

print 1*0.1 - 0.5*9.81*0.1**2

I have also tried programs that end with 'print y' after expression 'y=x+z' and programs that just end up with 'y=x+z', and there was no luck to be found either.

Edit:

good jovi posted:

Look in ball1.py, I'm betting it's just printing that number out, which isn't the same as returning it.

I get the same for programs like:

Python code:

v0 = 5
g = 9.81
t = 0.6
y = v0*t - 0.5*g*t**2

Python code:

C = 21
F = (9./5)*C + 32
print F

Maybe that's how it is intended to work, but then I have to wonder how the gently caress did guys in book get it done. with their code being:

Python code:

v0 = 5
g = 9.81
t = 0.6
y = v0*t - 0.5*g*t**2
print y

cinci zoo sniper fucked around with this message at 18:04 on Apr 5, 2015

# ? Apr 5, 2015 17:58

SurgicalOntologist: Jun 17, 2004

Either the book's wrong or you're misunderstanding it. Underscore contains the value of the last expression whose return was not assigned to a variable name. Print and run do not return anything, so they do not populate the underscore. Neither do any lines with an assignment (=).

# ? Apr 5, 2015 18:45

cinci zoo sniper: Mar 15, 2013

SurgicalOntologist posted:

Either the book's wrong or you're misunderstanding it. Underscore contains the value of the last expression whose return was not assigned to a variable name. Print and run do not return anything, so they do not populate the underscore. Neither do any lines with an assignment (=).

Alright. Can't expect everything be correct in a textbook, I guess.

# ? Apr 5, 2015 18:49

EAT THE EGGS RICOLA: May 29, 2008

kalstrams posted:

Maybe that's how it is intended to work, but then I have to wonder how the gently caress did guys in book get it done. with their code being:
Python code:
v0 = 5
g = 9.81
t = 0.6
y = v0*t - 0.5*g*t**2
print y

Open up a python console and type this:

Python code:

v0 = 5
g = 9.81
t = 0.6
v0*t - 0.5*g*t**2
_

# ? Apr 5, 2015 19:00

cinci zoo sniper: Mar 15, 2013

EAT THE EGGS RICOLA posted:

Open up a python console and type this:
Python code:
v0 = 5
g = 9.81
t = 0.6
v0*t - 0.5*g*t**2
_

I know that that works, thank you.

# ? Apr 5, 2015 19:27

Dominoes: Sep 20, 2007

Hey dudes, having an issue with Pypi. I'm following this tutorial. I registered and activated accounts on pypi and pypi test.

.pypirc

code:

[distutils]
index-servers =
    pypi
    pypitest

[pypi]
respository:[url]https://pypi.python.org/pypi[/url]
username: Redacted
password: Redacted

[pypitest]
respository: [url]https://testpypi.python.org/pypi[/url]
username: Redacted
password: Redacted

setup.py

Python code:

from setuptools import setup, find_packages

with open('README.rst') as f:
    readme = f.read()

setup(
    name = "quick",
    version = "0.1",
    packages = find_packages(),

    install_requires = ['numba>=0.17.0',
                        'numpy>=1.9.1',],

    url = 'https://github.com/David-OConnor/quick',
    description = "Fast implementation of numerical functions using Numba",
    long_description = readme,
    license = "apache",
    keywords = "fast, numba, numerical, optimized",

)

Command:

code:

python setup.py register -r pypitest

Error:

code:

Server response (403): You are not allowed to store 'quick' package information

I was able to successfully register using the form on pypi's test site, but when I upload using this:

code:

python setup.py sdist upload -r pypitest

I get this, similiar, message:

code:

error: HTTP Error 403: You are not allowed to edit 'quick' package information

I get the same error message when using Twine and Wheel, per these instructions.

Dominoes fucked around with this message at 20:28 on Apr 6, 2015

# ? Apr 6, 2015 16:55

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Is there some peculiar reason why somebody might want to explicitly pass self into an instance method rather than call it from self directly? I can't imagine what's going on. I'm reading some code from somebody I initially was giving a lot of slack since he was generally doing decent software engineering conceptually. However, bells started to go off when I saw some Systems Hungarian Notation.

# ? Apr 6, 2015 16:58

Space Kablooey: May 6, 2009

Rocko Bonaparte posted:

Is there some peculiar reason why somebody might want to explicitly pass self into an instance method rather than call it from self directly? I can't imagine what's going on. I'm reading some code from somebody I initially was giving a lot of slack since he was generally doing decent software engineering conceptually. However, bells started to go off when I saw some Systems Hungarian Notation.

You mean as an other argument other than the standard self?

Python code:

class Butt():
	def __init__(self, whatever):
		self.whatever = whatever
		self.fart(self)

	def fart(self, self2):
		print self2.whatever

If it is like this above then you should nip it before it gets worse.

# ? Apr 6, 2015 17:29

salisbury shake: Dec 27, 2011

Any reason alembic's autogenerate won't pick up a named UniqueConstraint added to a table? The table is a subclass of a parent table via joined table inheritance if that makes a difference.

# ? Apr 6, 2015 19:29

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Rocko Bonaparte posted:

Is there some peculiar reason why somebody might want to explicitly pass self into an instance method rather than call it from self directly? I can't imagine what's going on. I'm reading some code from somebody I initially was giving a lot of slack since he was generally doing decent software engineering conceptually. However, bells started to go off when I saw some Systems Hungarian Notation.

If the method is being called on a superclass I guess?

# ? Apr 6, 2015 20:12

Hughmoris: Apr 21, 2007; Let's go to the abyss!

I'm trying to learn how to scrape websites for useful information. For this, I was thinking I could scrape the local minor league baseball team's schedule for this year. I've found two websites that display their schedule, and they both seem to heavily use java script:
http://www.milb.com/schedule/index.jsp?sid=t4124
http://www.milb.com/milb/stats/stats.jsp?t=t_sch&cid=4124&sid=t4124&stn=true

Some google searches have pointed me towards using a combination of selenium and BeautifulSoup but I'm not having too much luck getting anything meaningful out of it.

Python code:

from selenium import webdriver  
from selenium.common.exceptions import NoSuchElementException  
from selenium.webdriver.common.keys import Keys  
from bs4 import BeautifulSoup

print "browser started"
browser = webdriver.Firefox()  
browser.get('http://www.milb.com/schedule/index.jsp?sid=t4124')  
html_source = browser.page_source  
browser.quit()
[s]soup[/s] = BeautifulSoup(html_source,'html.parser')  

comments = soup.findAll('div',{'id':'output'})

Any pointers?

# ? Apr 7, 2015 05:56

Dominoes: Sep 20, 2007

Hughmoris posted:

I'm trying to learn how to scrape websites for useful information. For this, I was thinking I could scrape the local minor league baseball team's schedule for this year. I've found two websites that display their schedule, and they both seem to heavily use java script:
http://www.milb.com/schedule/index.jsp?sid=t4124
http://www.milb.com/milb/stats/stats.jsp?t=t_sch&cid=4124&sid=t4124&stn=true

Some google searches have pointed me towards using a combination of selenium and BeautifulSoup but I'm not having too much luck getting anything meaningful out of it.
Python code:
from selenium import webdriver  
from selenium.common.exceptions import NoSuchElementException  
from selenium.webdriver.common.keys import Keys  
from bs4 import BeautifulSoup

print "browser started"
browser = webdriver.Firefox()  
browser.get('http://www.milb.com/schedule/index.jsp?sid=t4124')  
html_source = browser.page_source  
browser.quit()
[s]soup[/s] = BeautifulSoup(html_source,'html.parser')  

comments = soup.findAll('div',{'id':'output'})  
Any pointers?

Have you considered Regex?

# ? Apr 7, 2015 07:16

BabyFur Denny: Mar 18, 2003

Dominoes posted:

Have you considered Regex?

Are you serious? I hope you are not.

Hughmoris:
I would have suggested beautifulsoup.

# ? Apr 7, 2015 10:30

duck monster: Dec 15, 2004

Dominoes posted:

Have you considered Regex?

http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags

This should tell you everything you need to know about constructing regex html consumers. tl;dr ALL IS LOST the pony he comes.

Seconding beautifulsoup. Selenium can be handy for some things (Like trying to parse the insane mess facebook shits into your browser) but its brittle as hell, and frankly not as flexible.

At my old job we spidered about 500 different travel agents on a small cluster running stackless python greenlets (This was back in the mid 2000s) and beautifulsoup. Writing scrapers is never fun, writing LOTS of scrapers especially so. But bs takes the pain out of a lot of it.

duck monster fucked around with this message at 14:27 on Apr 7, 2015

# ? Apr 7, 2015 14:24

onionradish: Jul 6, 2006; That's spicy.

Hughmoris posted:

I'm trying to learn how to scrape websites for useful information. For this, I was thinking I could scrape the local minor league baseball team's schedule for this year. I've found two websites that display their schedule, and they both seem to heavily use java script:
http://www.milb.com/schedule/index.jsp?sid=t4124
http://www.milb.com/milb/stats/stats.jsp?t=t_sch&cid=4124&sid=t4124&stn=true

Some google searches have pointed me towards using a combination of selenium and BeautifulSoup but I'm not having too much luck getting anything meaningful out of it.

Any pointers?

All of the HTML on that site is a horrible mess. You're definitely going to want to use that second link, the one that gives you a list of tables.

I use LXML to parse rather than BeautifulSoup, so you''ll need to translate the syntax of the below:

Python code:

from selenium import webdriver  
import lxml.html

print "browser started"
browser = webdriver.Firefox()  
browser.get('http://www.milb.com/milb/stats/stats.jsp?t=t_sch&cid=4124&sid=t4124&stn=true')  
html_source = browser.page_source  
browser.quit()

# Should save/load the HTML here so you can do iterative development
# on your parser without re-scraping
    
doc = lxml.html.fromstring(html_source)

month_tables = doc.cssselect('.dataTableClass')

for month in month_tables:
    
    month_text = month.cssselect('tr.titleRow td')[0].text_content().strip()
    print month_text
    
    games = month.cssselect('tr td.dataCell:first-child')
    for game_details in games:
        print game_details.text_content().strip()

You'll need to do some plain text parsing to break the game_details results into times, opponents, and home/away, but that's something that regex is actually good for.

onionradish fucked around with this message at 15:12 on Apr 7, 2015

# ? Apr 7, 2015 14:43

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

BeautifulSoup uses lxml (and other parsers) under the hood so the only reason not to use it is if you really hate its API and know that your other favorite parser is perfectly capable of handling whatever you happen to throw at it.

# ? Apr 7, 2015 16:05

onionradish: Jul 6, 2006; That's spicy.

Munkeymon posted:

BeautifulSoup uses lxml (and other parsers) under the hood so the only reason not to use it is if you really hate its API and know that your other favorite parser is perfectly capable of handling whatever you happen to throw at it.

True. I didn't mean I used LXML "rather than" BeautifulSoup to be :smuggo:

. That was intended to be clarification on why I didn't use BeautifulSoup in the code snippet. I learned on and have done all my parsing with LXML so I know its syntax and just haven't had a reason to switch.

# ? Apr 7, 2015 16:49

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I think everyone missed your joke, Dominoes!

# ? Apr 7, 2015 16:51

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

onionradish posted:

True. I didn't mean I used LXML "rather than" BeautifulSoup to be . That was intended to be clarification on why I didn't use BeautifulSoup in the code snippet. I learned on and have done all my parsing with LXML so I know its syntax and just haven't had a reason to switch.

That's fair, but I feel like I see a lot of advice to avoid BS because lxml is better at parsing, which grates on me.

# ? Apr 7, 2015 16:54

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Munkeymon posted:

That's fair, but I feel like I see a lot of advice to avoid BS because lxml is better at parsing, which grates on me.

Didn't BS used to not use lxml? I feel like that was a thing and the reason people started recommending lxml instead.

# ? Apr 7, 2015 16:56

onionradish: Jul 6, 2006; That's spicy.

I thought it was the other way around, but I sure as hell didn't know what I was doing when I first started looking into scraping and parsing. I'll bet it had to do with XML parsing support, which BS says requires LXML. That might have even been the first type of parsing I needed to do — I can't remember.

Anyway, as a noob at the time, I was also looking to minimize dependencies because, uh, ?. So if I needed LXML anyway, why install something else, right? :cripes:

Of course that same kind of ignorance led me to use urllib far longer than I should until I finally installed requests and made my code a whole lot simpler.

From a quick scan of BS docs, the syntax looks pretty similar. Not sure how to do a "first-child" selector on games, though. We only want the first TD element in those rows.

Python code:

soup = BeautifulSoup(html_source)

month_tables = soup.select('.dataTableClass')

for month in month_tables:
    
    month_text = month.select('tr.titleRow td')[0].text.strip()
    print month_text
    
    games = month.select('tr td.dataCell')
    for game_details in games:
        print game_details.text.strip()

onionradish fucked around with this message at 17:31 on Apr 7, 2015

# ? Apr 7, 2015 17:25

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Thermopyle posted:

Didn't BS used to not use lxml? I feel like that was a thing and the reason people started recommending lxml instead.

Maybe at some point but it has for years and was using lxml when I first saw that advice pop up :shrug:

onionradish posted:

From a quick scan of BS docs, the syntax looks pretty similar. Not sure how to do a "first-child" selector on games, though. We only want the first TD element in those rows.

It's an iterator, so just call next() once.

# ? Apr 7, 2015 19:23

onionradish: Jul 6, 2006; That's spicy.

Calling next() seems icky. Isn't it possible to refine the selector to only return the items we care about in the first place? I'm sure it's just me not knowing the BS syntax here.

# ? Apr 7, 2015 19:28

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

I think it supports :first-of-type but you're still getting an iterator back, so...

E: you could .find('td')

Munkeymon fucked around with this message at 20:55 on Apr 7, 2015

# ? Apr 7, 2015 20:05

SurgicalOntologist: Jun 17, 2004

Just saw this today, which might be useful here, as it smooths over the BS interface: http://soupy.readthedocs.org/en/latest/

# ? Apr 7, 2015 20:31

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

They overcomplicated

Python code:

[str.strip() for str in BS(html).find(id='main').strings if len(str.strip())]

in their example :colbert:

I'm sure that can be done without calling strip twice, but I have a sinus headache and don't feel like giving myself a nested generator headache, too.

# ? Apr 7, 2015 20:58

Dominoes: Sep 20, 2007

Thermopyle posted:

I think everyone missed your joke, Dominoes!

Oh well. Reminded me to add The Center Cannot Hold to my reading list.

# ? Apr 7, 2015 22:29

Sinestro: Oct 31, 2010; The perfect day needs the perfect set of wheels.

Munkeymon posted:

They overcomplicated
Python code:
[str.strip() for str in BS(html).find(id='main').strings if len(str.strip())]
in their example

I'm sure that can be done without calling strip twice, but I have a sinus headache and don't feel like giving myself a nested generator headache, too.

Python code:

from operator import methodcaller as m

filter(len, map(m('strip'), BS(html).find(id='main').strings))

Sinestro fucked around with this message at 22:41 on Apr 7, 2015

# ? Apr 7, 2015 22:38

onionradish: Jul 6, 2006; That's spicy.

gently caress, I can't tell whether responses are real or just cruel obfuscation and code golf anymore....

I'm going back to LXML.

E: Dominoes' joke is the only one I got...

onionradish fucked around with this message at 23:10 on Apr 7, 2015

# ? Apr 7, 2015 22:57

SurgicalOntologist: Jun 17, 2004

Python code:

from toolz import pipe
from toolz.curried import map, filter
from operator import methodcaller, attrgetter

pipe(html,
    BeautifulSoup,
    methodcaller('find', id='main'),
    attrgetter('strings'),
    map(methodcaller('stripped_strings')),
    filter(len),
)

onionradish posted:

gently caress, I can't tell whether responses are real or just cruel obfuscation and code golf anymore....

You decide:

Python code:

from operator import methodcaller, attrgetter,
from toolz import pipe
from toolz.curried import map, take

pipe(html_source,
    BeautifulSoup,
    methodcaller('select', '.dataTableClass'),
    map(methodcaller('select', 'tr td.dataCell')),
    map(take(1)),
    map(attrgetter('stripped_strings')),
)

# ? Apr 7, 2015 23:13

onionradish: Jul 6, 2006; That's spicy.

Oh my god, is that still Python? :cry:

There's gotta be a point at which it isn't...

onionradish fucked around with this message at 23:26 on Apr 7, 2015

# ? Apr 7, 2015 23:23

hooah: Feb 6, 2006; WTF?

Is there a clean way to cause a key to be inserted in a dict if it doesn't exist, but increment it if it does? I tried

Python code:

my_dict['butts'] += 1

but of course that throws an error since 'butts' isn't in my_dict yet. I'm sure I could use try and catch, but that seems more complicated than it ought to be (I'm new-ish to Python).

# ? Apr 8, 2015 00:15

Hed: Mar 31, 2004; Fun Shoe

hooah posted:

Is there a clean way to cause a key to be inserted in a dict if it doesn't exist, but increment it if it does? I tried
Python code:
my_dict['butts'] += 1
but of course that throws an error since 'butts' isn't in my_dict yet. I'm sure I could use try and catch, but that seems more complicated than it ought to be (I'm new-ish to Python).

.get('keyname') on a dictionary will never raise a KeyError, and accepts a keyword argument for what to return in lieu of the value (default is None). So you could do

Python code:

my_dict['butts'] = my_dict.get('butts', 0) + 1

to get what I think you are probably wanting.

# ? Apr 8, 2015 00:19

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

hooah posted:

Is there a clean way to cause a key to be inserted in a dict if it doesn't exist, but increment it if it does? I tried
Python code:
my_dict['butts'] += 1
but of course that throws an error since 'butts' isn't in my_dict yet. I'm sure I could use try and catch, but that seems more complicated than it ought to be (I'm new-ish to Python).

d[key] = d.get(key, 0) + 1

efb

# ? Apr 8, 2015 00:20

Edison was a dick: Apr 3, 2010; direct current only

hooah posted:

Is there a clean way to cause a key to be inserted in a dict if it doesn't exist, but increment it if it does? I tried
Python code:
my_dict['butts'] += 1
but of course that throws an error since 'butts' isn't in my_dict yet. I'm sure I could use try and catch, but that seems more complicated than it ought to be (I'm new-ish to Python).

Use a collections.Counter instead of a regular dict.

# ? Apr 8, 2015 00:24

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 20:31

hooah: Feb 6, 2006; WTF?

Great, get(key, 0) worked fine!

Edison was a dick posted:

Use a collections.Counter instead of a regular dict.

How come?

# ? Apr 8, 2015 00:26

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »