Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
QuarkJets
Sep 8, 2008

Highblood posted:

Stupid question incoming: Big newbie to programming here. What's the point of dictionaries? I'm used to the more C-like struct and they seem similar but I don't get the differences. Why assign values to strings? Why not just a list of variables (man reaching for that " key is really hard okay)? From what I understand you can add and remove entries to the dictionary whenever you want but the applications for that fly completely over my head (I'm dumb sorry)

I just feel like whatever I could do with that could just as easily be done without it (C has no associative arrays right?)

Hopefully nobody got an aneurysm from that question.

Python has dict, C++ has std::unordered_map. Associative containers are extremely useful for a variety of use cases. For instance, in Python keyword arguments are passed basically as a dictionary; this allows you to check for and use optional arguments without mandating that they exist and without having to define several versions of the same function/class.

And deep down, most (all?) Python objects are actually just dictionaries anyway... so there's a use case

Adbot
ADBOT LOVES YOU

Nippashish
Nov 2, 2005

Let me see you dance!

Hammerite posted:

In the context of general Python programming, I would interpret "array" as synonymous with "list", since lists are the closest match for the notion of array in the Python standard library. In the context of using Python with Numpy, there might be an argument that this no longer holds; but as discussed, Numpy was at best implicitly introduced to the discussion context.

In the context of scientific python it's reasonable to assume the default unit of stuff is a numpy array. There is a whole world of python users for whom python without numpy is simply not a thing.

Cingulate
Oct 23, 2012

by Fluffdaddy

Nippashish posted:

In the context of scientific python it's reasonable to assume the default unit of stuff is a numpy array. There is a whole world of python users for whom python without numpy is simply not a thing.
I didn't make it clear enough I was indeed talking about scientific python.

On the other hand, it seems "import pandas as pd" is a part of like 75% of the answers in this thread. The main stuff that's not commonly helped by Pandas seems to be offering web services.

Amberskin
Dec 22, 2013

We come in peace! Legit!

Highblood posted:

Stupid question incoming: Big newbie to programming here. What's the point of dictionaries? I'm used to the more C-like struct and they seem similar but I don't get the differences. Why assign values to strings? Why not just a list of variables (man reaching for that " key is really hard okay)? From what I understand you can add and remove entries to the dictionary whenever you want but the applications for that fly completely over my head (I'm dumb sorry)

I just feel like whatever I could do with that could just as easily be done without it (C has no associative arrays right?)

Hopefully nobody got an aneurysm from that question.

Apart from the excelent reasons already given, think about a system or an application with hundreds or thousands of components. If you use structs (or the equivalent in other languages) you are dealing with data which resides at a fixed offset from the beginning of the structure. Now, if you have to add something to a structure which is widely used in your system (and you WILL need to do it, sooner or later) you will have to be very careful to add everything in a way that does not break any of your components (usually adding new stuff in the tail of the struct is enough, but that is not always posible). If you can't guarantee the offsets of the current variables are not affected, you will have to recompile everything.

If you use a dict and you need to add a new attribute, you just do it in runtime and just modify the components that need to use the new attribute. The other ones are not affected in any way.

BennyGsGhost
Jun 27, 2002

Low P/B or Bust
Pillbug
For purely educational purposes I've been trying different ways to extract information from a list of dictionaries. I know how to easily accomplish this with a list comprehension, but I'm wondering if it can also be done using map.

The dict:
code:
towns = [{'name': 'Manchester', 'population': 58241},
         {'name': 'Coventry', 'population': 12435},
         {'name': 'South Windsor', 'population': 25709}]
I'm trying to return the name for each town if population is under 50k. My working comp:
code:
[town.get('name') for town in towns if town.get('population') < 50000]
I've tried the following, which doesn't work (SyntaxError) as an if statement expects an else, and an else continue or pass doesn't work:
code:
list(map(lambda town: town.get('name') if town.get('population') < 50000, towns))
Is there a way to use a lambda to compare using one key/value in a dict, then return another key/value if True?

BennyGsGhost fucked around with this message at 19:29 on Jan 13, 2016

SurgicalOntologist
Jun 17, 2004

You need to use filter as well as map. Also, for educational purposes, you may be able to replace the lambdas with something from the operator module.

BennyGsGhost
Jun 27, 2002

Low P/B or Bust
Pillbug

SurgicalOntologist posted:

You need to use filter as well as map. Also, for educational purposes, you may be able to replace the lambdas with something from the operator module.

Thanks for the suggestion, I got it working. Also figured out how to utilize operator.itemgetter to clean it up a bit based on the extra suggestion, thanks for the idea!

nonathlon
Jul 9, 2004
And yet, somehow, now it's my fault ...
Quick, dumb stupid question: what's the simplest way to make a "beep" sound in Python.

Longer question, with context: I want to write a simple dumb little script that beeps at regular intervals until the script is terminated. That's all it should do. But the audio picture in Python looks completely fragmented. I'm on OSX, so that's the primary platform, but is there a simple way through this program or am I going to have to make a sound file and play it?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

outlier posted:

Quick, dumb stupid question: what's the simplest way to make a "beep" sound in Python.

Longer question, with context: I want to write a simple dumb little script that beeps at regular intervals until the script is terminated. That's all it should do. But the audio picture in Python looks completely fragmented. I'm on OSX, so that's the primary platform, but is there a simple way through this program or am I going to have to make a sound file and play it?

Python code:
print('\a')
That's the simplest and most cross platform way. It's the ASCII bell character.

But then it only works in a terminal and some terminals are configured to do something different with the bell character.

SurgicalOntologist
Jun 17, 2004

If you're on OSX you can also do os.system('say X') which can be a lot of fun.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
Is it common to have a method that can take multiple data types for the same parameter? I have a parameter that I want to accept either a string or a list. The string would just be for convenience instead of always having to pass in a list of 1.

Something like:
code:
def my_method(self, name, ids):
    if isinstance(ids, string_types):
        ids = [ids]
    # now do stuff

Cingulate
Oct 23, 2012

by Fluffdaddy
Super common, although ideally you just use duck typing.

The "check the type" (or duck check the type or however you call it - basically, just check if it quacks like a duck) is also very common though.

Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

fletcher posted:

Is it common to have a method that can take multiple data types for the same parameter? I have a parameter that I want to accept either a string or a list. The string would just be for convenience instead of always having to pass in a list of 1.

Something like:
code:
def my_method(self, name, ids):
    if isinstance(ids, string_types):
        ids = [ids]
    # now do stuff

In Python 2 you use isinstance(thing, basestring)

In Python 3 there isn't any such thing as basestring any more and you just use isinstance(thing, str)

qntm
Jun 17, 2009

Cingulate posted:

Super common, although ideally you just use duck typing.

The "check the type" (or duck check the type or however you call it - basically, just check if it quacks like a duck) is also very common though.

You mean by picking a random method which strings have but lists don't have, and testing whether the parameter has that method?

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord
But man lists and strings is a hairy situation where the "type distinction" can be confusing, I'm not sure if I'm a big fan of "duck typing" in this case...

Because of this I'd be careful with writing a function that takes both strings and lists of strings, maybe I'd separate into 2 different functions.

jerry seinfel
Jun 25, 2007


Does anyone have experience using selenium to work with dropdown menus? Mainly looking for help in speeding this script up based on where I think the slowdown is.

Part of my job has me cleaning up noisey data that comes in every day. It's through a vendor's system, so I can't perform operations on the data using SQL or anything else. I have to use this (really poorly put together) webpage to assign correct values to incorrect records.

The script is based in Selenium and works pretty well, but it takes a long time towards the end of each loop of the script. The slowdown occurs when the script is navigating dropdown values and searching for the correct value. The script has to select values in three drop down menus, so I just made it into a function that takes the name of each dropdown menu, and the text it should be looking for.

The dropdown menus get progressively longer, so by the time the script gets to the final dropdown menu it can take 5 to 10 seconds to identify the option to click. There are about 400~ dropdown items for it to iterate through by the end.

code:
def select_options(name_of_element, option_text):
    dropdown = driver.find_element_by_name(name_of_element)
    for option in dropdown.find_elements_by_tag_name('option'):
        if option.text == option_text:
            option.click()  
            break

SurgicalOntologist
Jun 17, 2004

Or just say "screw it" and require the caller to put that string in a single-item list. It's not that hard; I've been down that road before and I'm now convinced that the flexibility is not worth the trouble, even if the single-item call is by far the most common.

Another idea, if there aren't a lot of other parameters, is to use *args.

Python code:
def method(self, name, *ids):
   

Cingulate
Oct 23, 2012

by Fluffdaddy

qntm posted:

You mean by picking a random method which strings have but lists don't have, and testing whether the parameter has that method?
No - trying the method you'll actually use.

It really depends I guess.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I'll usually try to do the thing I want to do with the parameter and then catch exceptions to find out what went wrong.

Tigren
Oct 3, 2003
I am trying to use a Python module pacparser which is basically a wrapper for a C library. When the C library encounters an error, it spits out information and then raises a Python exception.

Python code:
In [5]: _pacparser.find_proxy('http://www.google.com', 'www.google.com')
JSERROR: NULL:285:
    ReferenceError: FindProxyForURL is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-5-0d32bdb423d6> in <module>()
----> 1 _pacparser.find_proxy('http://www.google.com', 'www.google.com')

error: Could not find proxy
I know how to capture the exception that Python raises, but how do I capture the information that is output from C?

code:
JSERROR: NULL:285:
    ReferenceError: FindProxyForURL is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.   

Tigren fucked around with this message at 22:35 on Jan 20, 2016

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Try
Python code:
import os
import sys

null = open(os.devnull, 'w')
temp = sys.stderr
sys.stderr = null

#let the extension do its thing here

sys.stderr = temp #restore it
null.close()

shrike82
Jun 11, 2005

Pythonista on iOS is a neat piece of kit if you enjoy hobbyist Python development.
Ver 2.0 exposes obj-C APIs so you can look up phone battery status, read in current GPS location, interact with Bluetooth devices (e.g., read in BPM from a BT heart rate monitor).
On the newer iPhones, you can also tie scripts to 3D touch shortcuts which makes app-replacement py scripts plausible. I spent a couple hours writing a decent Japanese ebook reader with auto-generated translations shown side-by-side.

Tigren
Oct 3, 2003

Munkeymon posted:

Try
Python code:
import os
import sys

null = open(os.devnull, 'w')
temp = sys.stderr
sys.stderr = null

#let the extension do its thing here

sys.stderr = temp #restore it
null.close()

And where am I supposed to be seeing the output that I want to capture?

Before resetting sys.stderr, sys.stderr is just <open file '/dev/null', mode 'w' at 0x7fad40d5ec90>

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Tigren posted:

And where am I supposed to be seeing the output that I want to capture?

Before resetting sys.stderr, sys.stderr is just <open file '/dev/null', mode 'w' at 0x7fad40d5ec90>

Oh sorry I got it in my head that you just wanted to suppress it.

Python code:
import os
import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

#let the extension do its thing here

sys.stderr = temp #restore it
capture.seek(0) #readable as a normal file

Tigren
Oct 3, 2003

Munkeymon posted:

Oh sorry I got it in my head that you just wanted to suppress it.

Python code:
import os
import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

#let the extension do its thing here

sys.stderr = temp #restore it
capture.seek(0) #readable as a normal file

Python code:
import os
import sys
from io import TextIOWrapper, BytesIO

import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

pacparser.init()
pacparser.parse_pac_file('test.pac')
pacparser.find_proxy('http://www.google.com')

sys.stderr = temp
capture.seek(0)
if capture:
    print "error captured"
No 'error captured' printing. 'capture' is empty.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Hammerite posted:

In Python 2 you use isinstance(thing, basestring)

In Python 3 there isn't any such thing as basestring any more and you just use isinstance(thing, str)

This was for python 2 & 3 using six.string_types =)

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Tigren posted:

Python code:
import os
import sys
from io import TextIOWrapper, BytesIO

import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

pacparser.init()
pacparser.parse_pac_file('test.pac')
pacparser.find_proxy('http://www.google.com')

sys.stderr = temp
capture.seek(0)
if capture:
    print "error captured"
No 'error captured' printing. 'capture' is empty.

You'd still have to do something about the exception:

Python code:
import sys
from io import TextIOWrapper, BytesIO
import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp

capture.seek(0)
print(capture.read())
Also capture is truthy as long as the call to TextIOWrapper didn't throw, so if capture: would always run the if body.

Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

fletcher posted:

This was for python 2 & 3 using six.string_types =)

Oh, I'm not familiar with it. In that case I would do exactly what you did

Tigren
Oct 3, 2003

Munkeymon posted:

You'd still have to do something about the exception:

Python code:
import sys
from io import TextIOWrapper, BytesIO
import pacparser

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp = sys.stderr
sys.stderr = capture

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp

capture.seek(0)
print(capture.read())
Also capture is truthy as long as the call to TextIOWrapper didn't throw, so if capture: would always run the if body.

Thanks for being patient with me. stderr/stdout redirection seems like kind of a funky beast.

I tried running that from within a REPL and from a script

code:
$ python test.py
JSERROR: PAC script:3:
    ReferenceError: isnNet is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.
that did not go well!

I just get a blank line output for console.read(0).

Once I realized what you were doing, I looked into stderr redirection and saw mention of stderr caching. Do you think the JSERROR is cached and therefore not directly accessible from sys.stderr redirection?

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Tigren posted:

Thanks for being patient with me. stderr/stdout redirection seems like kind of a funky beast.

I tried running that from within a REPL and from a script

code:
$ python test.py
JSERROR: PAC script:3:
    ReferenceError: isnNet is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.
that did not go well!

I just get a blank line output for console.read(0).

Once I realized what you were doing, I looked into stderr redirection and saw mention of stderr caching. Do you think the JSERROR is cached and therefore not directly accessible from sys.stderr redirection?

Caching shouldn't be an issue because we're supplying our own buffer before the output is generated. Could be I made an invalid assumption and it might be sending the output on stdout. Could be that there's an init in the module that grabs copies of the std* streams before we're replacing them. This should mitigate both:
Python code:
import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp_err = sys.stderr
temp_out = sys.stdout
sys.stderr = sys.stdout = capture #mess with all the outputs

import pacparser #don't let it init until after we mess with outputs

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp_err
    sys.stdout = temp_out

capture.seek(0)
print("output: {0}|".format(capture.read()))
And, someone correct me if I'm wrong here, but I think the extension could be opening its own std* streams, which I don't think you can do anything about other than shove it in a subprocess and capture all the output.

Tigren
Oct 3, 2003

Munkeymon posted:

Caching shouldn't be an issue because we're supplying our own buffer before the output is generated. Could be I made an invalid assumption and it might be sending the output on stdout. Could be that there's an init in the module that grabs copies of the std* streams before we're replacing them. This should mitigate both:
Python code:
import sys
from io import TextIOWrapper, BytesIO

capture = TextIOWrapper(BytesIO(), sys.stderr.encoding)
temp_err = sys.stderr
temp_out = sys.stdout
sys.stderr = sys.stdout = capture #mess with all the outputs

import pacparser #don't let it init until after we mess with outputs

try:
    pacparser.init()
    pacparser.parse_pac_file('test.pac')
    pacparser.find_proxy('http://www.google.com')
except Exception:
    print("that did not go well!")
finally:
    sys.stderr = temp_err
    sys.stdout = temp_out

capture.seek(0)
print("output: {0}|".format(capture.read()))
And, someone correct me if I'm wrong here, but I think the extension could be opening its own std* streams, which I don't think you can do anything about other than shove it in a subprocess and capture all the output.

code:
JSERROR: PAC script:3:
    ReferenceError: isnNet is not defined
pacparser.c: pacparser_find_proxy: Problem in executing findProxyForURL.
output: |
Seems like it's either time to subprocess it or write my own module.

Modulo16
Feb 12, 2014

"Authorities say the phony Pope can be recognized by his high-top sneakers and incredibly foul mouth."

I'm writing a python script that downloads the National Vulnerability Database from the nist.gov website and writes an xml file zipped into the directory. The other script I wrote that notifies the appropriate people in IRC with vulnerabilities that match the software and vendors we're using, just a hobby project while I learn python. That script works fine, however I can't seem to get this simple download using urllib2 to actually write the information to the new zip. It comes out as a blank XML file. What am I doing wrong?

Python code:
import urllib2

u = urllib2.urlopen('https://nvd.nist.gov/download/nvdcve-Recent.xml.zip')

localFile = open('nvd.xml.zip', 'w')
localFile.write(u.read())
localFile.close()

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Frank Viola posted:

I'm writing a python script that downloads the National Vulnerability Database from the nist.gov website and writes an xml file zipped into the directory. The other script I wrote that notifies the appropriate people in IRC with vulnerabilities that match the software and vendors we're using, just a hobby project while I learn python. That script works fine, however I can't seem to get this simple download using urllib2 to actually write the information to the new zip. It comes out as a blank XML file. What am I doing wrong?

Python code:
import urllib2

u = urllib2.urlopen('https://nvd.nist.gov/download/nvdcve-Recent.xml.zip')

localFile = open('nvd.xml.zip', 'w')
localFile.write(u.read())
localFile.close()

I'm not going to answer your question, but I just wanted to note that you might want to use the requests library instead of urllib2. I very rarely see anyone using urllib2 anymore...everyone seems to use requests.

Modulo16
Feb 12, 2014

"Authorities say the phony Pope can be recognized by his high-top sneakers and incredibly foul mouth."

Thermopyle posted:

I'm not going to answer your question, but I just wanted to note that you might want to use the requests library instead of urllib2. I very rarely see anyone using urllib2 anymore...everyone seems to use requests.

'wb'(binary) instead of 'w', rookie mistake; thanks for not telling me. Also I'll read up on requests. Thanks.

Modulo16 fucked around with this message at 07:01 on Jan 22, 2016

Edison was a dick
Apr 3, 2010

direct current :roboluv: only

Frank Viola posted:

I'm writing a python script that downloads the National Vulnerability Database from the nist.gov website and writes an xml file zipped into the directory. The other script I wrote that notifies the appropriate people in IRC with vulnerabilities that match the software and vendors we're using, just a hobby project while I learn python. That script works fine, however I can't seem to get this simple download using urllib2 to actually write the information to the new zip. It comes out as a blank XML file. What am I doing wrong?

Python code:
import urllib2

u = urllib2.urlopen('https://nvd.nist.gov/download/nvdcve-Recent.xml.zip')

localFile = open('nvd.xml.zip', 'w')
localFile.write(u.read())
localFile.close()

Also, rather than reading everything into memory at once then writing it all out at once, you might want to take a look at shutil.copyfileobj, context managers and contextlib.closing

Python code:
import contextlib
import shutil
import urllib2

# Some file objects don't act as context managers directly, but we can wrap them with contextlib.closing to make them behave
with contextlib.closing(urllib2.urlopen('https://nvd.nist.gov/download/nvdcve-Recent.xml.zip')) as u:
    # Regular file objects can be used as context managers directly
    with open('nvd.xml.zip', 'wb') as localFile:
        # This does reading from u and writing to localFile in a loop rather than all at once, so you don't need to fit the whole file in memory.
        shutil.copyfileobj(u, localFile)

jerry seinfel
Jun 25, 2007


hcenvirons posted:

question about selenium and selecting from dropdown

I managed to solve the issue I had (holla @ stackoverflow questions from 2011)


Instead of making a dropdown object and iterating through all the options it contains, I used Selenium's built in Select.

code:
from selenium.webdriver.support.ui import Select

def dropdown_alternative(name_of_element, option_text):
    select = Select(driver.find_element_by_name(name_of_element))
    select.select_by_visible_text(option_text)
So, so, so much faster and cleaner.

KernelSlanders
May 27, 2013

Rogue operating systems on occasion spread lies and rumors about me.

SurgicalOntologist posted:

If you're on OSX you can also do os.system('say X') which can be a lot of fun.

This is amazing. Thank you.

I can now annoy the hell out of my office mates.

e: Holy poo poo git log 2>&1 | say -f -

KernelSlanders fucked around with this message at 18:48 on Jan 24, 2016

IratelyBlank
Dec 2, 2004
The only easy day was yesterday
Does anyone know if NumPy/SciPy work with IronPython in C#? I am able to execute Python code and scripts just fine from within C#, but when I try to use one of my Python scripts that imports numpy, I immidately get an error "No module named numpy". My googling says that this wasn't possible a few years ago but all the questions about using SciPy/NumPy in Ironpython are now years old and it's unclear to me whether or not this is possible.

Nippashish
Nov 2, 2005

Let me see you dance!
As a general rule the numeric python ecosystem only works with CPython.

Adbot
ADBOT LOVES YOU

Dominoes
Sep 20, 2007

Is there a clean way to index a numpy array with interval, starting at the end instead of begining?

code:
x = np.array([1, 2, 3, 4, 5] )
x[::3]

out: array([1, 4])
I'd like it to be out: array([2, 5])

edit: Think I found it: x[::-3][::-1]

  • Locked thread