Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

lifg: Dec 4, 2000; <this tag left blank>; Muldoon

Edison was a dick posted:

Break only takes you out of the current for loop, you presumably have another loop around that, which you're not breaking out of.
The simplest thing might be to set a variable when you've found it, and check that variable at the top of your outer loop and break if the variable is set.

There's a really weird trick to break out of an outer loop without using a flag variable: using a for-loop else.

With a for-loop else, the "else" happens just once after a for-loop is finished. But if you break out of that for-loop, the "else" is skipped.

Which leads to this...

code:

for x in rows:
  for y in columns:
    if whatever(x,y):
      break
  else:
    continue
  break

# ? Sep 20, 2016 00:34

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 05:27

cinci zoo sniper: Mar 15, 2013

Is it straightforward to implement something so that when you have script.py that imports stuff from library.py, you can also get it to make you a copy of both library.py and script.py in the output folder? I'm specifically after doing it through Python, doing that via OS would be trivial.

# ? Sep 20, 2016 10:54

huhu: Feb 24, 2006

cinci zoo sniper posted:

Is it straightforward to implement something so that when you have script.py that imports stuff from library.py, you can also get it to make you a copy of both library.py and script.py in the output folder? I'm specifically after doing it through Python, doing that via OS would be trivial.

Is this what you want?

to_copy.py

code:

print("I am going to be copied")
import make_copy

make_copy.py

code:

import os
import shutil
folder_path = os.path.abspath("to_copy.py")
shutil.copy(folder_path, "./output")

# ? Sep 20, 2016 15:31

cinci zoo sniper: Mar 15, 2013

huhu posted:

Is this what you want?

to_copy.py

code:

print("I am going to be copied")
import make_copy

make_copy.py

code:

import os
import shutil
folder_path = os.path.abspath("to_copy.py")
shutil.copy(folder_path, "./output")

Yeah, thanks!

# ? Sep 20, 2016 16:09

BigRedDot: Mar 6, 2008

PyBay 2016 vieos are finally going up, including my Bokeh talk </shameless promotion> Raymond Hettinger also gave an, ah, interesting keynote.

# ? Sep 21, 2016 01:20

LochNessMonster: Feb 3, 2005; I need about three fitty

I'm probably going to figure this out within 5 minutes after posting, but I've been struggling with inserting data into sqlite 3 and I'm not quite sure why.

code:

import sqlite3
conn = sqlite3.connect('vehicles.db')
cursor = conn.cursor()

testDict = {'DealerId': '1',
            'Brand': 'bmw',
            'Milage': '13.909',
            'Dealer': 'crazy jack',
            'Price': '5.000',
            'Type': 'r90',
            'Year': '1980'}

for (DealerId, Dealer, Brand, Type, Price, Milage, Year) in testDict:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
                   DealerId,
                   Dealer,
                   Brand,
                   Type,
                   Price,
                   Milage,
                   Year)

Running this results in the following error message

code:

Traceback (most recent call last):
  File "/path/to/script/dbinsert.py", line 13, in <module>
    for (DealerId, Dealer, Brand, Type, Price, Milage, Year) in testDict:
ValueError: too many values to unpack (expected 7)

Please point and laugh at my failures :downs:

# ? Sep 21, 2016 08:54

ArcticZombie: Sep 15, 2010

I've never used sqlite3 so I'm winging this a bit, but I don't think your problem is with sqlite3. Looping over a dict doesn't work like that. The for loop will loop over each key in the dict:

Python code:

>>> testDict = {"foo":1,"bar":2,"baz":3}
>>> for key in testDict:
...    print(key)
foo
bar
baz

Your execute should look more like:

Python code:

cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
               testDict["DealerId"],
               testDict["Dealer"],
               testDict["Brand"],
               testDict["Type"],
               testDict["Price"],
               testDict["Milage"],
               testDict["Year"])

With no need for the loop over it.

# ? Sep 21, 2016 09:08

LochNessMonster: Feb 3, 2005; I need about three fitty

ArcticZombie posted:

I've never used sqlite3 so I'm winging this a bit, but I don't think your problem is with sqlite3. Looping over a dict doesn't work like that. The for loop will loop over each key in the dict:
Python code:
>>> testDict = {"foo":1,"bar":2,"baz":3}
>>> for key in testDict:
...    print(key)
foo
bar
baz
Your execute should look more like:
Python code:
cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
               testDict["DealerId"],
               testDict["Dealer"],
               testDict["Brand"],
               testDict["Type"],
               testDict["Price"],
               testDict["Milage"],
               testDict["Year"])
With no need for the loop over it.

The test dictionary only contains 1 entry, but the real resultset will have tens of entries. I just started out with a smaller data set to spot problems early on.

Your example should get me up and running though, I'm try and write something that inserts just the test dictionary and then see how I can write a loop around it so I can repeat it for multiple entries.

# ? Sep 21, 2016 09:47

ArcticZombie: Sep 15, 2010

The real results will be a list of dicts, no? You'll loop over the list, not over the dicts.

# ? Sep 21, 2016 09:53

LochNessMonster: Feb 3, 2005; I need about three fitty

ArcticZombie posted:

The real results will be a list of dicts, no? You'll loop over the list, not over the dicts.

yeah, you're right. It's a list containing dictionaries.

I'm really new to this, and still struggling a bit with the different types of data structures (and how to proces them)

# ? Sep 21, 2016 11:34

Dex: May 26, 2006; Quintuple x!!!

Would not escrow again.

VERY MISLEADING!

LochNessMonster posted:

yeah, you're right. It's a list containing dictionaries.

I'm really new to this, and still struggling a bit with the different types of data structures (and how to proces them)

since you're iterating through a list, just use arcticzombie's example for each dict in it:

Python code:

for testDict in mylist:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
        testDict["DealerId"],
        testDict["Dealer"],
        testDict["Brand"],
        testDict["Type"],
        testDict["Price"],
        testDict["Milage"],
        testDict["Year"]

# ? Sep 21, 2016 12:14

some kinda jackal: Feb 25, 2003; �
�

Man there are like 900 ways to install Py3 on OSX. I'm just going to brew install python3 and forget about it before I start to obsess about which is the best down the road. Never going to actually learn if I keep nitpicking my infrastructure :|

# ? Sep 21, 2016 12:58

Proteus Jones: Feb 28, 2013

Martytoof posted:

Man there are like 900 ways to install Py3 on OSX. I'm just going to brew install python3 and forget about it before I start to obsess about which is the best down the road. Never going to actually learn if I keep nitpicking my infrastructure :|

Anaconda will never let you down.

# ? Sep 21, 2016 12:59

some kinda jackal: Feb 25, 2003; �
�

I ended up settling on good ole dependable "brew install python3" which may or may not be terrible, I'll find out as I go along :haw:

# ? Sep 21, 2016 13:05

Tigren: Oct 3, 2003

LochNessMonster posted:

I'm probably going to figure this out within 5 minutes after posting, but I've been struggling with inserting data into sqlite 3 and I'm not quite sure why.

code:

import sqlite3
conn = sqlite3.connect('vehicles.db')
cursor = conn.cursor()

testDict = {'DealerId': '1',
            'Brand': 'bmw',
            'Milage': '13.909',
            'Dealer': 'crazy jack',
            'Price': '5.000',
            'Type': 'r90',
            'Year': '1980'}

for (DealerId, Dealer, Brand, Type, Price, Milage, Year) in testDict:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
                   DealerId,
                   Dealer,
                   Brand,
                   Type,
                   Price,
                   Milage,
                   Year)

Running this results in the following error message

code:

Traceback (most recent call last):
  File "/path/to/script/dbinsert.py", line 13, in <module>
    for (DealerId, Dealer, Brand, Type, Price, Milage, Year) in testDict:
ValueError: too many values to unpack (expected 7)

Please point and laugh at my failures :downs:

It looks like other people squared you away on your for loop. I wanted to show you this option for inserting from a dict as well. Instead of the question mark "parameterization", you can use named placeholders. Those names are the keys of your dict.

Python code:

In [38]: testDict                                                     
Out[38]:
{'Brand': 'bmw',
 'Dealer': 'crazy jack',
 'DealerId': '1',
 'Milage': '13.909',
 'Price': '5.000',
 'Type': 'r90',
 'Year': '1980'}

In [39]: cursor.execute('create table motorcycles (brand, dealer, dealerid, mileage, price, type, year)')
Out[39]: <sqlite3.Cursor at 0x7f1bf8216dc0>

In [40]: cursor.execute("INSERT INTO motorcycles VALUES (:Brand, :Dealer, :DealerId, :Milage, :Price, :Type, :Year)", testDict)
Out[40]: <sqlite3.Cursor at 0x7f1bf8216dc0>

In [41]: cursor.execute("SELECT * FROM motorcycles")
Out[41]: <sqlite3.Cursor at 0x7f1bf8216dc0>

In [42]: cursor.fetchall()
Out[42]: [('bmw', 'crazy jack', '1', '13.909', '5.000', 'r90', '1980')]

There's also the option of using cursor.executemany, which allows you to loop over a list of entries like so:

Python code:

In [82]: # starting with an empty db

In [83]: cursor.execute("SELECT * FROM motorcycles")
Out[83]: <sqlite3.Cursor at 0x7f1bebdd6d50>

In [84]: cursor.fetchall()
Out[84]: []

In [85]: # testList has two motorcycle dictionaries, but could have more
                                                                                                                                                                                                                                    
In [86]: testList
Out[86]:
[{'Brand': 'bmw',
  'Dealer': 'crazy jack',
  'DealerId': '1',
  'Milage': '13.909',
  'Price': '5.000',
  'Type': 'r90',
  'Year': '1980'},
 {'Brand': 'bmw',
  'Dealer': 'crazy jack',
  'DealerId': '2',
  'Milage': '13.909',
  'Price': '5.000',
  'Type': 'r90',
  'Year': '1980'}]

In [87]: cursor.executemany("INSERT INTO motorcycles VALUES (:Brand, :Dealer, :DealerId, :Milage, :Price, :Type, :Year)", testList)
Out[87]: <sqlite3.Cursor at 0x7f1bebdd6d50>

In [88]: cursor.execute("SELECT * FROM motorcycles")
Out[88]: <sqlite3.Cursor at 0x7f1bebdd6d50>

In [89]: cursor.fetchall()
Out[89]:
[('bmw', 'crazy jack', '1', '13.909', '5.000', 'r90', '1980'),
 ('bmw', 'crazy jack', '2', '13.909', '5.000', 'r90', '1980')]

# ? Sep 21, 2016 18:35

LochNessMonster: Feb 3, 2005; I need about three fitty

Tigren posted:

It looks like other people squared you away on your for loop. I wanted to show you this option for inserting from a dict as well. Instead of the question mark "parameterization", you can use named placeholders. Those names are the keys of your dict.

[..]

There's also the option of using cursor.executemany, which allows you to loop over a list of entries like so:

Both options seem like a pretty efficient way of doing things. I didn't get the for loop working yet so I'll give this a try as well.

Just for my understanding, the: in/out [number] is from your python environment?

# ? Sep 21, 2016 19:43

Tigren: Oct 3, 2003

LochNessMonster posted:

Both options seem like a pretty efficient way of doing things. I didn't get the for loop working yet so I'll give this a try as well.

Just for my understanding, the: in/out [number] is from your python environment?

Ya, those numbers are from iPython, which is an alternative python REPL. You can ignore those parts.

# ? Sep 21, 2016 20:51

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Tigren posted:

Ya, those numbers are from iPython, which is an alternative python REPL. You can ignore those parts.

That's the one thing I don't like about using iPython. It makes copy/pasting REPL sessions a little weird because of the in/out thing. Especially if you're copy/pasting to use in doctests.

# ? Sep 21, 2016 21:36

LochNessMonster: Feb 3, 2005; I need about three fitty

When using the for loops I keep getting the same error, no matter what I try.

When running this:

Python code:

import sqlite3
conn = sqlite3.connect('vehicles.db')
cursor = conn.cursor()

testList = [{'DealerId': '2',
            'Brand': 'bmw',
            'Milage': '13.909',
            'Dealer': 'crazy jack',
            'Price': '5.000',
            'Type': 'r90',
            'Year': '1980'},
            {'DealerId': '3',
            'Brand': 'goldwing',
            'Milage': '15.564',
            'Dealer': 'Mohawk Mike',
            'Price': '8.000',
            'Type': 'goldwing',
            'Year': '2015'}]


for dictionary in testList:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
                   dictionary["DealerId"],
                   dictionary["Dealer"],
                   dictionary["Brand"],
                   dictionary["Type"],
                   dictionary["Price"],
                   dictionary["Milage"],
                   dictionary["Year"])

I keep getting:

code:

Traceback (most recent call last):
  File "/path/to/program/dbinsert2.py", line 29, in <module>
    dictionary["Year"])
TypeError: function takes at most 2 arguments (8 given)

I'm trying to figure out why it only takes 2 arguments and how it's counting 8 that are being given. I really only count 7 question marks and 7 dictionary values.

So I went and tried Tigrens suggestion on how to tackle this. It doesn't give any errors, but for some reason it doesn't really enter the values either. When I do a SELECT * FROM motorcyles; it only returns 1 bogus value I added manually.

Python code:

import sqlite3
conn = sqlite3.connect('vehicles.db')
cursor = conn.cursor()

testList = [{'DealerId': '2',
            'Brand': 'bmw',
            'Milage': '13.909',
            'Dealer': 'crazy jack',
            'Price': '5.000',
            'Type': 'r90',
            'Year': '1980'},
            {'DealerId': '3',
            'Brand': 'goldwing',
            'Milage': '15.564',
            'Dealer': 'Mohawk Mike',
            'Price': '8.000',
            'Type': 'goldwing',
            'Year': '2015'}]

cursor.executemany("INSERT INTO motorcycles VALUES (:Brand, :Dealer, :DealerId, :Milage, :Price, :Type, :Year)", testList)

Time to figure out how to do error handling I guess.

LochNessMonster fucked around with this message at 10:19 on Sep 22, 2016

# ? Sep 22, 2016 09:54

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Do you need to commit the transaction?

# ? Sep 22, 2016 10:36

LochNessMonster: Feb 3, 2005; I need about three fitty

Hammerite posted:

Do you need to commit the transaction?

I feel pretty stupid now...

# ? Sep 22, 2016 11:29

Space Kablooey: May 6, 2009

LochNessMonster posted:

When using the for loops I keep getting the same error, no matter what I try.

When running this:

Python code:

import sqlite3
conn = sqlite3.connect('vehicles.db')
cursor = conn.cursor()

testList = [{'DealerId': '2',
            'Brand': 'bmw',
            'Milage': '13.909',
            'Dealer': 'crazy jack',
            'Price': '5.000',
            'Type': 'r90',
            'Year': '1980'},
            {'DealerId': '3',
            'Brand': 'goldwing',
            'Milage': '15.564',
            'Dealer': 'Mohawk Mike',
            'Price': '8.000',
            'Type': 'goldwing',
            'Year': '2015'}]


for dictionary in testList:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)",
                   dictionary["DealerId"],
                   dictionary["Dealer"],
                   dictionary["Brand"],
                   dictionary["Type"],
                   dictionary["Price"],
                   dictionary["Milage"],
                   dictionary["Year"])

I keep getting:

code:

Traceback (most recent call last):
  File "/path/to/program/dbinsert2.py", line 29, in <module>
    dictionary["Year"])
TypeError: function takes at most 2 arguments (8 given)

Tigren's solution is the better one IMO, but according to the docs (https://docs.python.org/2/library/sqlite3.html#sqlite3.Cursor.execute), if you want to use dictionaries, you have to use the named parameters style instead of the question marks one.

The question marks style is used when you have a list or tuple with the parameters already in order, for example:

Python code:

x = ['gassy', 'smelly', 'loud']
cursor.execute("INSERT INTO farts VALUES(?, ?, ?)", x)

If you wanted to use dicts, you would use it like this:

Python code:

x = {'type': 'gassy', 'smell': 'smelly', 'noise': 'loud'}
cursor.execute("INSERT INTO farts VALUES(:type, :smell, :noise)", x)

And it should work. I'm not sure the dictionary keys have to be equal to the columns names, but I don't think so.

Space Kablooey fucked around with this message at 16:00 on Sep 22, 2016

# ? Sep 22, 2016 15:56

SurgicalOntologist: Jun 17, 2004

...and if you want to do it the other way, you need to group the values into a list rather than including them all as function arguments directly.

Python code:


for dictionary in testList:
    cursor.execute("INSERT INTO motorcycles VALUES(?, ?, ?, ?, ?, ?, ?)", [
        dictionary["DealerId"],
        dictionary["Dealer"],
        dictionary["Brand"],
        dictionary["Type"],
        dictionary["Price"],
        dictionary["Milage"],
        dictionary["Year"],
    ])

(note the extra [ and ])

# ? Sep 22, 2016 16:13

LochNessMonster: Feb 3, 2005; I need about three fitty

Thanks for all the feedback. I managed to get it working with Tigrens way. The code is now looking like this:

Python code:

<scraping part>
motorcycle = { 'Brand': getBrand.get_text(),
                       'Type': getType.get_text(),
                       'Price': getPrice.get_text(),
                       'Milage': getMilage.get_text(),
                       'Dealer': dealerName,
                       'Year': getYear.get_text(),
                       'DealerId': getDealerId['id']}
        processingList.append(motorcycle)

dbPath = str(dataDir + dbName)

try:
    conn = sqlite3.connect(dbPath)
except Exeption as err:
    print('Could not connect to vehicles.db because reasons: ' + str(err))
cursor = conn.cursor()

try:
    cursor.executemany("INSERT INTO motorcycles VALUES (:DealerId, :Dealer, :Brand, :Type, :Price, :Milage, :Year)", processingList)
    conn.commit()

except Exception as err:
    print('An exception happened: ' +str(err))

conn.close()

It's really cool to see results while learning a useful new skill. I'm really enjoying this, and already have lots of other things in mind to incorporate in it.

For now it's still a lot of trial and error, and thinking about what I'm doing wrong but when I started a few weeks back I knew nothing at all (compared to you guys I guess that's still the case) but now I seem to find my way a bit.

LochNessMonster fucked around with this message at 17:26 on Sep 22, 2016

# ? Sep 22, 2016 17:19

Tigren: Oct 3, 2003

LochNessMonster posted:

Thanks for all the feedback. I managed to get it working with Tigrens way. The code is now looking like this:
Python code:
<scraping part>
motorcycle = { 'Brand': getBrand.get_text(),
                       'Type': getType.get_text(),
                       'Price': getPrice.get_text(),
                       'Milage': getMilage.get_text(),
                       'Dealer': dealerName,
                       'Year': getYear.get_text(),
                       'DealerId': getDealerId['id']}
        processingList.append(motorcycle)

dbPath = str(dataDir + dbName)

try:
    conn = sqlite3.connect(dbPath)
except Exeption as err:
    print('Could not connect to vehicles.db because reasons: ' + str(err))
cursor = conn.cursor()

try:
    cursor.executemany("INSERT INTO motorcycles VALUES (:DealerId, :Dealer, :Brand, :Type, :Price, :Milage, :Year)", processingList)
    conn.commit()

except Exception as err:
    print('An exception happened: ' +str(err))

conn.close()
It's really cool to see results while learning a useful new skill. I'm really enjoying this, and already have lots of other things in mind to incorporate in it.

For now it's still a lot of trial and error, and thinking about what I'm doing wrong but when I started a few weeks back I knew nothing at all (compared to you guys I guess that's still the case) but now I seem to find my way a bit.

Congrats, feels good doesn't it? Now read through these comics and check them off one by one when they apply to you. They will all apply to you eventually.

https://medium.freecodecamp.com/coding-explained-in-25-profound-comics-8847ea03819c#.lzka3j5rh

Tigren fucked around with this message at 17:55 on Sep 22, 2016

# ? Sep 22, 2016 17:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Tigren posted:

Congrats, feels good doesn't it? Now read through these comics and check them off one by one when they apply to you. They will all apply to you eventually.

https://medium.freecodecamp.com/coding-explained-in-25-profound-comics-8847ea03819c#.lzka3j5rh

I like how #12 "identifying if picture is of a bird" went from "virtually impossible" to pretty doable nowadays.

# ? Sep 22, 2016 18:19

Eela6: May 25, 2007; Shredded Hen

LochNess, glad to see you're enjoying python. It is by far my favorite language.

As a small piece of advice, try and avoid the construct

code:


except Exception:
#code

Unless you're planning to re-raise the exception. Catching and handling specific errors is OK, but generally ignoring ALL errors is asking for trouble later on. This is the sort of thing that seems exponent at the time but can make debugging your program down the line basically impossible.

Good luck with your coding!

# ? Sep 22, 2016 18:50

LochNessMonster: Feb 3, 2005; I need about three fitty

Eela6 posted:

LochNess, glad to see you're enjoying python. It is by far my favorite language.

As a small piece of advice, try and avoid the construct
code:
except Exception:
#code
Unless you're planning to re-raise the exception. Catching and handling specific errors is OK, but generally ignoring ALL errors is asking for trouble later on. This is the sort of thing that seems exponent at the time but can make debugging your program down the line basically impossible.

Good luck with your coding!

Thanks for the feedback, could you explain what is actually wrong and how I could improve on that? I thought I'd catch the errors for that specific code and print it to stdout. That way I knew what went wrong.

# ? Sep 23, 2016 12:28

'ST: Jul 24, 2003; "The world has nothing to fear from military ambition in our Government."

LochNessMonster posted:

Thanks for the feedback, could you explain what is actually wrong and how I could improve on that? I thought I'd catch the errors for that specific code and print it to stdout. That way I knew what went wrong.

There are three things to keep in mind here:

Do not use except Exception: at all in Python 2.4 and earlier unless you really know what you're doing.
It's fine to use it in Python 2.5+ and Python 3.x in limited cases, but be careful.
The best practice is to catch only the exceptions that you're sure you care about.

The first thing is a bit of dated advice. You can skip to the explanation of the second thing if you don't care about Python lore. Before Python 2.5, all exceptions inherited from Exception. Some of the exceptions that inherited from it included SystemExit and KeyboardInterrupt. That meant that you could conceivably catch and ignore some very fundamental built-in exceptions for exiting your Python program. Imagine hitting Control-C and having your exception handler print that out but not actually exit the program. In Python 2.5, they made a BaseException and made key system-internal exceptions inherit from that rather than from Exception. except Exception: will no longer catch these critical exceptions.

The second thing is to be careful about catching everything of type Exception. In particular, Python uses exceptions internally for normal iteration behavior. See https://docs.python.org/2.7/library/stdtypes.html#iterator.next. There are a few other potential gotchas, but it's not going to destroy your program like it would in Python 2.4.

That said, the third thing is to be specific about exceptions that you want to catch. In this case, you should use except sqlite3.Error: (https://docs.python.org/3/library/sqlite3.html?highlight=cursor#sqlite3.Error). This will catch all sqlite errors but allow the rest to bubble up out of your program. If you can identify a few very specific exceptions that you want to handle in the same way, it's even better to explicitly name them rather than catch some parent class of theirs. See http://stackoverflow.com/questions/6470428/catch-multiple-exceptions-in-one-line-except-block#6470452. The best practice is to be explicit and specific unless you have a very good reason not to be.

# ? Sep 23, 2016 13:36

SurgicalOntologist: Jun 17, 2004

The reason not to catch every exception is that if something goes terribly wrong, why continue? Specifically if you get an error connecting to the database, why then try to write something to it anyway? Clearly it's not going to work. You're just going to bury the useful exception message (the first one) under some other exceptions that you are sure to get if you continue to run your program.

You should catch an exception whenever there's a specific exception that you know you're able to handle and continue to run the program. For example maybe getDealerId['id'] returns a KeyError and you decide to handle that by writing in a default ID (note that there are still better ways to get this behavior). I can't think of anything that could go wrong connecting to the database and still have your program run correctly the rest of the way.

Finally, you say you'd print it to stdout so you know what's wrong. Well, usually the message will go to stderr, so unless you have some weird setup you should be able to see the message without needing to catch and print it.

# ? Sep 23, 2016 15:12

LochNessMonster: Feb 3, 2005; I need about three fitty

Reading these explenations and those links clears things up a bit (or a lot actually).

The reason I initially did it this way was a) because I didn't know any better, and b) I had no idea which other (specific) Errors I was looking for.

The reason I put it in there, was because my program seem to get all the input and put it in dictionary and added each dictionary to the list. But it didn't write the data to the database. This actually happened due to several reasons, some of which I found out with except Exception. Reasons were: database was in a different path than my program was looking in, table had a typo so couldn't insert and finally I didn't commit the changes.

What I didn't realize is that I could possibly catch lots of other Exceptions. So I'm going to rewrite this with trying to catch a sqlite3.DatabaseError for the connection, and a sqlite3.IntegrityError to see if my insert is working properly (if it isn't, I don't want to pollute the database).

# ? Sep 23, 2016 16:01

Eela6: May 25, 2007; Shredded Hen

Awesome. The posters above explained it better than I could have. As a quick note, if you're trying to debug, there's nothing wrong with putting off generic errors long enough to get some info first, I.e:

try:
// some problematic code
except exception as err:
// some additional debugging code that provides you info about the error
raise err

Later on you'll probably want to factor that out, but it can help you understand what's going on when your code behaves unexpectedly.

Eela6 fucked around with this message at 17:37 on Sep 23, 2016

# ? Sep 23, 2016 16:07

SurgicalOntologist: Jun 17, 2004

I'll just add that if you're only printing out the error message, you're not getting any more information than if you had just let it hit. Eela6's suggestion is a good reason to catch exceptions while debugging but for it to provide benefits you need to print out some other useful information after catching. E.g. the value of some variables that may be related to the error. Finally, as in Eela6's example, you should still re-raise the exception if you're only catching it for debugging purposes (that is, if you don't do some error-handling that will allow your program to continue regardless).

I'll also add that I usually re-raise an exception with a bare "raise" rather than "raise err". I think this provides a better traceback, but I was not able to confirm this with a quick Google.

# ? Sep 23, 2016 17:02

LochNessMonster: Feb 3, 2005; I need about three fitty

SurgicalOntologist posted:

I'll just add that if you're only printing out the error message, you're not getting any more information than if you had just let it hit. Eela6's suggestion is a good reason to catch exceptions while debugging but for it to provide benefits you need to print out some other useful information after catching. E.g. the value of some variables that may be related to the error. Finally, as in Eela6's example, you should still re-raise the exception if you're only catching it for debugging purposes (that is, if you don't do some error-handling that will allow your program to continue regardless).

I'll also add that I usually re-raise an exception with a bare "raise" rather than "raise err". I think this provides a better traceback, but I was not able to confirm this with a quick Google.

I still have to wrap my head around the raising and reraising you guys keep mentioning. I did print out a lot of other variables (and sometimes type(var) for the bs4 variables) tp get an idea what was going wrong.
I cleaned that part up before posting though.

# ? Sep 23, 2016 17:12

SurgicalOntologist: Jun 17, 2004

Oh, in that case catching the error does make some sense. Just add "raise" after though and you'll get more consistent behavior ("crashing" instead of chugging along like nothing is wrong).

# ? Sep 23, 2016 17:16

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Yeah, exceptions are there to tell you something went wrong, and give you information about exactly what happened. They should tell you which line in your code caused it, and what type of error it is. Not catching them means your program crashes (bad), because there's a serious problem (bad) which needs to be fixed (good!)

Catching exceptions is what you do when there are specific problems you want to handle. Things like I/O or network operations can be unreliable, there's a chance they could fail (throwing an exception) and you want your program to handle those situations gracefully. Catching those specific exceptions, in the places you expect they could happen, allows you to catch those errors and do whatever you need to do

You just need to be careful of the overlap here, the exceptions you expect and the ones you don't. You don't want to silently catch the errors that point to a bug in your program, you need it to let you know that Bad Thing has gone wrong. That's why it's best to limit your catching to the specific types of exceptions you expect, and only have your try block cover the places where you expect it to occur (like where you make a call to write to disk). That way you reduce the chances of catching and swallowing something you didn't mean to, and losing valuable information

Reraising is basically throwing an exception again - it's like saying "hang on a minute" with the catch, doing something, then saying "ok carry on crashing". Sometimes this is exactly what you want, or maybe something further back needs to see the exception too, and do some handling itself. Once you get the idea of exceptions in general, you should have a better idea of when you might want to do this kind of thing

# ? Sep 23, 2016 17:44

Eela6: May 25, 2007; Shredded Hen

quote:

I'll also add that I usually re-raise an exception with a bare "raise" rather than "raise err". I think this provides a better traceback, but I was not able to confirm this with a quick Google.

I was curious about this so I did a brief experiment. They appear to be identical in Cpython 3.5.

unnamed input

Python code:

def raise_unnamed_exception():
    def internal_unnamed_raise():
        try:
            raise ValueError("basic exception")
        except Exception as err:
            raise
            
    internal_unnamed_raise()

raise_unnamed_exception()

unnamed output

Python code:

runfile('C:/eela6_online_example/raise_unnamed_exception.py', wdir='C:/eela6_online_example')
Traceback (most recent call last):

  File "<ipython-input-15-83da24a50a8d>", line 1, in <module>
    runfile('C:/eela6_online_example/raise_unnamed_exception.py', wdir='C:/eela6_online_example')

  File "D:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 714, in runfile
    execfile(filename, namespace)

  File "D:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 89, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/eela6_online_example/raise_unnamed_exception.py", line 17, in <module>
    raise_unnamed_exception()

  File "C:/eela6_online_example/raise_unnamed_exception.py", line 15, in raise_unnamed_exception
    internal_unnamed_raise()

  File "C:/eela6_online_example/raise_unnamed_exception.py", line 11, in internal_unnamed_raise
    raise ValueError("basic exception")

ValueError: basic exception

named input:

Python code:

def raise_named_exception():
    def internal_named_raise():
        try:
            raise ValueError("basic exception")
        except Exception as err:
            raise err
            
    internal_named_raise()

raise_named_exception()

named output:

Python code:

runfile('C:/eela6_online_example/raise_named_exception.py', wdir='C:/eela6_online_example')
Traceback (most recent call last):

  File "<ipython-input-16-280b30bc959d>", line 1, in <module>
    runfile('C:/eela6_online_example/raise_named_exception.py', wdir='C:/eela6_online_example')

  File "D:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 714, in runfile
    execfile(filename, namespace)

  File "D:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 89, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/eela6_online_example/raise_named_exception.py", line 17, in <module>
    raise_named_exception()

  File "C:/eela6_online_example/raise_named_exception.py", line 15, in raise_named_exception
    internal_named_raise()

  File "C:/eela6_online_example/raise_named_exception.py", line 13, in internal_named_raise
    raise err

  File "C:/eela6_online_example/raise_named_exception.py", line 11, in internal_named_raise
    raise ValueError("basic exception")

ValueError: basic exception

Eela6 fucked around with this message at 22:07 on Sep 23, 2016

# ? Sep 23, 2016 17:48

QuarkJets: Sep 8, 2008

SurgicalOntologist posted:

The reason not to catch every exception is that if something goes terribly wrong, why continue? Specifically if you get an error connecting to the database, why then try to write something to it anyway? Clearly it's not going to work. You're just going to bury the useful exception message (the first one) under some other exceptions that you are sure to get if you continue to run your program.

You should catch an exception whenever there's a specific exception that you know you're able to handle and continue to run the program. For example maybe getDealerId['id'] returns a KeyError and you decide to handle that by writing in a default ID (note that there are still better ways to get this behavior). I can't think of anything that could go wrong connecting to the database and still have your program run correctly the rest of the way.

Finally, you say you'd print it to stdout so you know what's wrong. Well, usually the message will go to stderr, so unless you have some weird setup you should be able to see the message without needing to catch and print it.

You can raise caught exceptions. So maybe I want to print something specific anytime an exception is caught, even if the raised exception wasn't expected, and then raise the exception again

E: well gently caress this was talked about two posts later

# ? Sep 24, 2016 03:57

Death Zebra: May 14, 2014

This code:

code:

with open(filepath+'SearchResults. txt', 'w') as f:
    for x in range(len(searchresults)):
        s = searchresults[x][0]+'|'+searchresults[x][1]+'|'+searchresults[x][2]+'|'+searchresults[x][3]+'\n'
        f.write(s)

sometimes gets this error:

quote:

File "/storage/emulated/0/com.hipipal.qpyplus/scripts/.last_tmp.py", line 169, in <module>
f.write(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 53: ordinal not in range(128)

I've tried using f.write(s.encode("utf-8")) and f.write(str(s)) instead but that doesn't alleviate the problem.

# ? Sep 27, 2016 17:38

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 05:27

Eela6: May 25, 2007; Shredded Hen

Are you in python 2 or python 3?

# ? Sep 27, 2016 18:10

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »