Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Haystack posted:

When you go into Settings | Project Interpreter > Python Interpreters is your system python's lib folder included in the paths tab?

Yes it's there.

After some more research, I've found a bug on the Jetbrains issue tracker for this issue which arose with Ubuntu 13.04, which I am running.

Guess I've gotta wait for a fix from Jetbrains. Thankfully, I'm finally experiencing a bug on a piece of software that isn't marked "wontfix" or "lowpriority"! This one's priority is Major!

# ? May 7, 2013 22:23

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 04:14

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

ShoulderDaemon posted:

Note that there exist platforms where the endianness of integers is not the same as the endianness of floats, because the FPU and the ALU were made as separate units by different teams and cobbled together at the last minute or something.

Ah, interesting. I've never encountered any such platform, and I wonder how well Python would handle it. It looks like the struct module (Modules/_struct.c) treats whatever system it's running on as having a single native byte order that's defined as part of compiling the struct module.

Out of curiosity, what relatively modern CPUs (i.e can run an OS that can run Python) have different endianness for integers and floats?

# ? May 7, 2013 22:33

ShoulderDaemon: Oct 9, 2003; support goon fund; Taco Defender

Lysidas posted:

Out of curiosity, what relatively modern CPUs (i.e can run an OS that can run Python) have different endianness for integers and floats?

In terms of relatively modern... the whole ARM architecture is fairly modular, and I've seen a few different people independently make ARM-on-FPGA systems where they managed to flip byte order on the FPU, and then proceed to get an operating system running on it. I'm not aware of any actual ASIC-that-has-gone-to-market with such a bug in recent decades, however.

Historically, as with most crazy byte order shenanigans, it was more common. Back in the days of external FPUs and non-IEEE-754 there was a lot of room for exciting mismatches.

# ? May 7, 2013 22:42

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

Jewel posted:

Someone pointed out to me that multi-selection editing is in almost every text editor other than notepad.

Just hold alt and highlight stuff in notepad++ (and I think it's just alt in Visual Studio too) and there you go, multi-selection editing. I use it all the time.

Multi-select handles arbitrary cursor positions, it is not just column editing, mind you, my gif does not illustrate that.

For example you could do a find-all for "time" and two keystrokes convert all of them to "times" (right arrow and s) with multi-selection editing. (THIS IS A TERRIBLE EXAMPLE BUT YOU GET THE JIST.)

VVVVVVVVVVVVVVVV

Scaevolus posted:

Select "time", hit Ctrl-D a few times to expand the selection, right arrow, s.

Better example (still kinda dumb because search and replace).

deimos fucked around with this message at 23:03 on May 8, 2013

# ? May 8, 2013 22:03

Scaevolus: Apr 16, 2007

deimos posted:

For example you could do a find-all for "time" and two keystrokes convert all of them to "times" (right arrow and s) with multi-selection editing. (THIS IS A TERRIBLE EXAMPLE BUT YOU GET THE JIST.)

Select "time", hit Ctrl-D a few times to expand the selection, right arrow, s.

# ? May 8, 2013 22:44

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Is there a good page for explanation of how class variables work? I'm working on a poker game, and within the game, I have a global called deckStart that's an organized 52 card deck. I then do:

code:

self.__deck = self.__shuffle(self.deckStart)

def __shuffle(self, deck):
    for i in range(0,3):
        for j in range(0,52):
            rand = int(random.uniform(0,52))
            temp = deck[j];
            print(str(j) + " " + str(rand) + " " + str(len(deck)))
            deck[j] = deck[rand];
            deck[rand] = temp;
    return deck

and then I remove cards from the top of __deck during execution. I figured that deckStart would be left the same since I make no direct references to it and I assumed it'd be like Java/C++, but deckStart seems to lose 12 cards after the first execution.

# ? May 9, 2013 06:42

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Yeah, you're passing it by reference, so you're directly working on deckStart the whole time. You need to use copy from the standard library, or pull the base values from startDeck when you populate the new deck, rather than mutating it.

# ? May 9, 2013 07:36

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Maluco Marinero posted:

Yeah, you're passing it by reference, so you're directly working on deckStart the whole time. You need to use copy from the standard library, or pull the base values from startDeck when you populate the new deck, rather than mutating it.

Yeah, I just used self.deckStart[:] which makes a copy as I understand it. I guess I'm a bit confused as to when to know when something is passed by reference and when it's not. Running:

code:

a = 0
def update(temp):
    temp += 1

update(a)
print(a)

So is it just a list thing where it gets passed by reference while ints and strings and other basic types are just passed regularly?

I feel like I really should read a solid book on Python instead of haphazardly learning it.

# ? May 9, 2013 07:49

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Master_Odin posted:

Yeah, I just used self.deckStart[:] which makes a copy as I understand it. I guess I'm a bit confused as to when to know when something is passed by reference and when it's not. Running:
code:
a = 0
def update(temp):
    temp += 1

update(a)
print(a)
So is it just a list thing where it gets passed by reference while ints and strings and other basic types are just passed regularly?

I feel like I really should read a solid book on Python instead of haphazardly learning it.

Everything is passed by reference, absolutely everything. But ints and strings are immutable - can't be changed. Once you have a value of 4, that's it; that 4 can't be changed (to 5 say), though it can be replaced by a new value 5.

The reason why += changes a list in place whereas for your integer it acts like "temp = temp + 1" is that the behaviour of += is dependent on the type on its left. It behaves in the one way for lists and in the other for ints, strings and so on. This might seem confusing at first and almost like "cheating", but in fact you can set the behaviour of += for your own types however you want (by defining the __iadd__() method).

# ? May 9, 2013 08:45

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

Master_Odin, depending on how serious this poker app you're making is, you may want to think about the security implications of the tools you're using. Specifically you may want to look into os.urandom and random.SystemRandom instead of the standard random.Random class (which is what you're actually using when you call random.random(), random.uniform() etc.), as a smart attacker may be able to exploit some patterns in the latter. You could also look into the RNG in the PyCrypto package.

More importantly, there's a simpler and better algorithm for shuffling a deck. It's called the Fisher-Yates Shuffle, and it looks like this:

Python code:

def shuffle(l):
    for i in xrange(len(l)-1, 0, -1):
       j = random.randrange(0, i+1)
       l[j], l[i] = l[i], l[j]

Furthermore, this algorithm is already implemented in the standard random module: random.shuffle. I believe all you need to do have a quite secure implementation is create an instance of random.SystemRandom and call shuffle on it with your deck as the input argument. The behavior of the underlying os.urandom is strongly dependent on the platform you're running on, but it should be fine on both Unix and Windows.

# ? May 9, 2013 10:13

Nippashish: Nov 2, 2005; Let me see you dance!

LOOK I AM A TURTLE posted:

Master_Odin, depending on how serious this poker app you're making is, you may want to think about the security implications of the tools you're using.

The counterpoint being that unless there's real money involved you can ignore everything in this post (except for the part about random.shuffle, you should use random.shuffle).

# ? May 9, 2013 10:44

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

Nippashish posted:

The counterpoint being that unless there's real money involved you can ignore everything in this post (except for the part about random.shuffle, you should use random.shuffle).

Yeah, my main message would be to always look closely at the standard library before reinventing the wheel.

# ? May 9, 2013 11:08

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

Can someone point me towards a larger Python project that they think has really good unit tests?

# ? May 9, 2013 13:14

BigRedDot: Mar 6, 2008

PyTables

# ? May 9, 2013 19:44

tef: May 30, 2004; -> some l-system crap ->

Hammerite posted:

Everything is passed by reference, absolutely everything. But ints and strings are immutable - can't be changed. Once you have a value of 4, that's it; that 4 can't be changed (to 5 say), though it can be replaced by a new value 5.

It's more accurate to say that python uses call by object semantics. Every object is stored on the heap, and variables contain a reference to that object. When you pass in a list to a function, the reference is copied.

code:

def foo(x):
    x = 1

a = 0; foo(a); print a

If python were truly pass by reference, this would print 1, not 0.

# ? May 9, 2013 20:17

peepsalot: Apr 24, 2007; ��PEEP THIS...
��BITCH!

Can someone recommend a library to use to display vector graphics in python? I would like something where i can specify "hairline" or zero width lines and curves that display as 1 pixel wide no matter how zoomed or scaled it is. Also if there is some built in support for zooming that would be great. Also support for Bezier curves would be nice.

It would be for a simplistic 2D CAD sort of application.

# ? May 9, 2013 21:22

accipter: Sep 12, 2003

peepsalot posted:

Can someone recommend a library to use to display vector graphics in python? I would like something where i can specify "hairline" or zero width lines and curves that display as 1 pixel wide no matter how zoomed or scaled it is. Also if there is some built in support for zooming that would be great.

Why not use matplotlib and save as SVG?

# ? May 9, 2013 21:23

peepsalot: Apr 24, 2007; ��PEEP THIS...
��BITCH!

accipter posted:

Why not use matplotlib and save as SVG?

I had not considered matplotlib, I will have to look into that.

SVG doesn't support hairlines for one thing, and there are other issues I have with SVG that I won't get into, but it looks like matplotlib supports postscript export as well, which I think will work for what I need.

# ? May 9, 2013 22:06

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

peepsalot posted:

Can someone recommend a library to use to display vector graphics in python? I would like something where i can specify "hairline" or zero width lines and curves that display as 1 pixel wide no matter how zoomed or scaled it is. Also if there is some built in support for zooming that would be great. Also support for Bezier curves would be nice.

cairo?

# ? May 9, 2013 22:07

Emacs Headroom: Aug 2, 2003

Suspicious Dish posted:

cairo?

Nuthin wrong with postscript if that's what peepsalot is comfortable with :colbert:

# ? May 9, 2013 22:28

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

tef posted:

It's more accurate to say that python uses call by object semantics. Every object is stored on the heap, and variables contain a reference to that object. When you pass in a list to a function, the reference is copied.

Yeah, I knew I was most likely not using the correct terminology.

I hoped to just get across the idea of how Python behaves and what's going on with Master_Odin's code.

# ? May 10, 2013 03:29

evensevenone: May 12, 2001; Glass is a solid.

tef posted:

It's more accurate to say that python uses call by object semantics. Every object is stored on the heap, and variables contain a reference to that object. When you pass in a list to a function, the reference is copied.
code:
def foo(x):
    x = 1

a = 0; foo(a); print a
If python were truly pass by reference, this would print 1, not 0.

What's really happening is that x is added to a dict containing the attributes of foo, and the value of x is the id of a. So it's definitely passing a reference to an object, not the object itself. But then x = 1 creates a new int with a value of 1 and assigns its id to x. When foo ends, a obviously still refers to that old int.

If a were mutable you could change it from inside foo. But anything after an assignment would be referring to a different object.

Basically python is just a giant pile of dicts.

# ? May 10, 2013 07:33

Lichtenstein: May 31, 2012; It'll make sense, eventually.

Hey guys,

I'm a total programming novice, who has decided to pick up Python for fun. I'm currently going through the Al Sweigart books and loving around with doing a simple roguelike-like as an excercise. Which directly led me to wasting half a day on this retarded problem:

Why does this weird poo poo appears at the beginning of each line and how do I get rid of it? It's probably the most obvious thing in the word, but I can't get around it.

It's in pygcurse, if that changes anything.

Lichtenstein fucked around with this message at 19:41 on May 10, 2013

# ? May 10, 2013 19:07

n0manarmy: Mar 18, 2003

Are you using simple print statements or are you doing some kind of loop to output the text? Can you paste the code snippet for just that section?

# ? May 10, 2013 19:25

Lichtenstein: May 31, 2012; It'll make sense, eventually.

Currently it's basically this thrown in a loop along with a second function that simply changes the text color. win is my .PygcurseWindow:

code:

def introScreen():
    win.fill(bgcolor=(0,0,0))
    win.putchars('''    ____                         ___          \n   / __ \____  ____ ___  _____  / (_)___  ___ \n  / /_/ / __ \/ __ `/ / / / _ \/ / / __ \/ _ \\
 / _, _/ /_/ / /_/ / /_/ /  __/ / / / / /  __/
/_/ |_|\____/\__, /\__,_/\___/_/_/_/ /_/\___/ 
            /____/                            
           __  ____                 _         
          /  |/  (_)___ _____ ___  (_)        
         / /|_/ / / __ `/ __ `__ \/ /         
        / /  / / / /_/ / / / / / / /          
       /_/  /_/_/\__,_/_/ /_/ /_/_/           ''', win.centerx-23, win.centery-10, fgcolor=LSD, indent=True)
    if int(time.time() * 2) % 2 == 0:
        win.putchars('Press any key to begin', win.centerx-11, win.centery+5, fgcolor=LSD)
    win.update()
    exit()
    return

The horrible first line of the ASCII thing is me trying to see if perhaps it's using Enter rather than \n that causes the problem (didn't change poo poo).

I've loaded a system font (Consolas), but both courier and the default font also produce these boxes (though without question marks inside, but that's probably just a font thing).

# ? May 10, 2013 19:40

Smarmy Coworker: May 10, 2008; by XyloJW

I seem to be a huge idiot today.

I have this:

Python code:

mc = MongoClient('localhost', 27017)

docList = [Class(mc, m) for m in mc.jtest.docs.find()[0:]]

if not docList:
	print 'Hi, I work!'
	# DO A BUNCH OF STUFF THAT IS UNNECESSARY TO REPLICATE HERE #

mc.disconnect()

There is currently nothing in the docs collection, so docList is an empty list. I made sure of this by adding a print below it. not [] => True, but for some reason nothing under the conditional runs. This is a weird problem that I don't understand. Any possible solutions?

Lichtenstein posted:

Currently it's basically this thrown in a loop along with a second function that simply changes the text color. win is my .PygcurseWindow:
code:
def introScreen():
    win.fill(bgcolor=(0,0,0))
    win.putchars('''    ____                         ___          \n   / __ \____  ____ ___  _____  / (_)___  ___ \n  / /_/ / __ \/ __ `/ / / / _ \/ / / __ \/ _ \\
 / _, _/ /_/ / /_/ / /_/ /  __/ / / / / /  __/
/_/ |_|\____/\__, /\__,_/\___/_/_/_/ /_/\___/ 
            /____/                            
           __  ____                 _         
          /  |/  (_)___ _____ ___  (_)        
         / /|_/ / / __ `/ __ `__ \/ /         
        / /  / / / /_/ / / / / / / /          
       /_/  /_/_/\__,_/_/ /_/ /_/_/           ''', win.centerx-23, win.centery-10, fgcolor=LSD, indent=True)
    if int(time.time() * 2) % 2 == 0:
        win.putchars('Press any key to begin', win.centerx-11, win.centery+5, fgcolor=LSD)
    win.update()
    exit()
    return
The horrible first line of the ASCII thing is me trying to see if perhaps it's using Enter rather than \n that causes the problem (didn't change poo poo).

I've loaded a system font (Consolas), but both courier and the default font also produce these boxes (though without question marks inside, but that's probably just a font thing).

Try not having indent=True because it looks like it's trying to use some special character that isn't in your charset and doesn't seem indented anyway.

Smarmy Coworker fucked around with this message at 19:47 on May 10, 2013

# ? May 10, 2013 19:45

Lichtenstein: May 31, 2012; It'll make sense, eventually.

ARACHNOTRON posted:

Try not having indent=True because it looks like it's trying to use some special character that isn't in your charset and doesn't seem indented anyway.

The weird characters are left, except everything after the first line snaps to the left window border (which pretty much what indent=False is meant to do).

[edit] I've temporarily dealt with it by splitting the thing into separate strings for each line and manually placing them above each other, but there's got to be a better way.

Lichtenstein fucked around with this message at 21:08 on May 10, 2013

# ? May 10, 2013 21:06

peepsalot: Apr 24, 2007; ��PEEP THIS...
��BITCH!

try os.linesep instead of \n?

# ? May 10, 2013 21:24

Winkle-Daddy: Mar 10, 2007

Okay, I think it might be because it's Friday, but seriously, what the gently caress, Python?

code:

$ python myTest.py
Traceback (most recent call last):
  File "./myTest.py", line 1, in <module>
    from noise import pnoise2
  File "/Users/[my user]/python_stuff/tests/noise.py", line 1, in <module>
    from noise import pnoise2
ImportError: cannot import name pnoise2

$ cd ~/Downloads/noise-10b3/examples
$ python myTest.py
[outputs what I'd expect]
$ cd ..
$ sudo python setup.py install
Password:
running install
running build
running build_py
running build_ext
running install_lib
running install_egg_info
Removing /Library/Python/2.7/site-packages/noise-1.0b3-py2.7.egg-info
Writing /Library/Python/2.7/site-packages/noise-1.0b3-py2.7.egg-info

$ cd ~/python_stuff/tests
$ python myTest.py
Traceback (most recent call last):
  File "./myTest.py", line 1, in <module>
    from noise import pnoise2
  File "/Users/[my user]/python_stuff/tests/noise.py", line 1, in <module>
    from noise import pnoise2
ImportError: cannot import name pnoise2

I've never had this problem with any other module, is there something really stupid I'm doing? I've never had a problem with modules only working from the folder I installed them from. Maybe an OSX bug? Or a result of the hosed up permissions on my work computer?

edit:
File content I'm running:

code:

from noise import pnoise2

WORLD_SIZE = 10
for x in xrange(0,WORLD_SIZE):
	for y in xrange(0,WORLD_SIZE):
		print(pnoise2(float(x/100.),float(y/100.)))

edit:

Civil Twilight posted:

You have a file noise.py in ~/python_stuff/tests. When you try to "from noise import pnoise2" it's looking in your noise.py rather than the noise module you installed. Looks like noise.py also has "from noise import pnoise2", so it looks in itself...

Haha, holy poo poo! Well now, that is embarrassing. Thanks, Civil Twilight!

Winkle-Daddy fucked around with this message at 00:19 on May 12, 2013

# ? May 10, 2013 21:28

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

I'm parsing a CSV and have about 24,000 of these queries I need to run:

Python code:

"INSERT INTO words (seq_1, seq_2, fk, good_example) SELECT " + id1 + ", " + id2 + ", fk, " + str(boolCheck) + " FROM table WHERE value = '" + w + "';"

Is there a way to reduce the amount of time it takes to run all of those queries? Right now, based on some rough math, it's going to take about 3.5 hours for the script to finish. Some standard INSERT statements in the same script go blazingly fast thanks to cursor.execute("BEGIN TRANSACTION"), but I've yet to find a way to speed up the above query. I'm pretty sure it's the fact that I'm running a SELECT statement within INSERT, and each SELECT is being run in its own transaction.

I thought running all the SELECT's at once and then using their output as a parameter in executemany() would work, but doing a for loop and execute()'ing each SELECT statement took just as long. I'm stuck as to whether or not I should just resign myself to the fact that it's going to take for-loving-ever to run this script and let it do its thing overnight whenever I need to run it.

IAmKale fucked around with this message at 22:13 on May 10, 2013

# ? May 10, 2013 22:10

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

BeefofAges fucked around with this message at 02:57 on May 11, 2013

# ? May 11, 2013 00:41

Civil Twilight: Apr 2, 2011

Winkle-Daddy posted:

Okay, I think it might be because it's Friday, but seriously, what the gently caress, Python?

code:

$ python myTest.py
Traceback (most recent call last):
  File "./myTest.py", line 1, in <module>
    from noise import pnoise2
  File "/Users/[my user]/python_stuff/tests/noise.py", line 1, in <module>
    from noise import pnoise2
ImportError: cannot import name pnoise2

You have a file noise.py in ~/python_stuff/tests. When you try to "from noise import pnoise2" it's looking in your noise.py rather than the noise module you installed. Looks like noise.py also has "from noise import pnoise2", so it looks in itself...

Civil Twilight fucked around with this message at 01:21 on May 11, 2013

# ? May 11, 2013 01:19

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

Karthe posted:

I'm parsing a CSV and have about 24,000 of these queries I need to run:
Python code:
"INSERT INTO words (seq_1, seq_2, fk, good_example) SELECT " + id1 + ", " + id2 + ", fk, " + str(boolCheck) + " FROM table WHERE value = '" + w + "';"
Is there a way to reduce the amount of time it takes to run all of those queries? Right now, based on some rough math, it's going to take about 3.5 hours for the script to finish. Some standard INSERT statements in the same script go blazingly fast thanks to cursor.execute("BEGIN TRANSACTION"), but I've yet to find a way to speed up the above query. I'm pretty sure it's the fact that I'm running a SELECT statement within INSERT, and each SELECT is being run in its own transaction.

I thought running all the SELECT's at once and then using their output as a parameter in executemany() would work, but doing a for loop and execute()'ing each SELECT statement took just as long. I'm stuck as to whether or not I should just resign myself to the fact that it's going to take for-loving-ever to run this script and let it do its thing overnight whenever I need to run it.

Yuck, first of all, you should be using sqlalchemy's text escaping feature to interpolate in strings unless you're sure there can't be a sql injection attack.

Second, take a step back. Do you understand what that query is doing? You're basically inserting table for each value of w and selecting based on id1 and id2.
You might want to see if you can create a table based on id1, id2 and w and see if you can't construct the "words" table in one go using a few joins. You'll want to check out the sql questions thread since your problem is an ETL one more than a python one.

Third, do you undestand what "BEGIN TRANSACTION" means in the context of whatever sql implementation you're using? You should read the docs on cursor to see if you can disable the implicit transactions, or just find a better way to do standard inserts- postgres supports direct csv loading via psql and you can get very simple csvs (no nesting and quotes) into sqlite3 using the command line.

Really, whenever you run bunch of queries the question to ask is not "how do I speed this up?" but "what relationship between the parameters and the query am I missing"? Tables are cheap, time is not.

# ? May 11, 2013 01:40

Dren: Jan 5, 2001; Pillbug

Do you have an index on the value column of table to make your selects fast?

# ? May 11, 2013 03:28

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

Malcolm XML posted:

Yuck, first of all, you should be using sqlalchemy's text escaping feature to interpolate in strings unless you're sure there can't be a sql injection attack.

SQL injection attacks won't be an issue, this is simply a utility script I wrote to import values in a CSV file into a SQLite database I'm using in an Android app I'm working on. I'm running into issues when I want to include existing information from the database into the new record I'm inserting filled with information I've parsed from the CSV file.

That said, I suppose you're right that this is more of a SQL thing. I'll take my question to that thread.

Dren posted:

Do you have an index on the value column of table to make your selects fast?

Yeah, my indices are all up to snuff. Individual SELECTs on that table take ten-thousandths of a second to run across several tens of thousands of records, I'm pretty sure it's the individual SELECT query transactions that are bogging things down. Based on what I've read it's a limitation of SQLite, and not even committing records directly to the database helps to speed things up.

IAmKale fucked around with this message at 03:32 on May 11, 2013

# ? May 11, 2013 03:28

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

Edit: Double post

# ? May 11, 2013 03:32

Dominoes: Sep 20, 2007

How can I write to CSVs without getting a blank row between each real row? This page shows that the file needs to be written in binary mode to do this, but when I do it, I get the following error: 'TypeError: 'str' does not support the buffer infercace.'. Is there a way to convert a string to binary first?

I've been able to work with CSVs successfully before when downloading raw CSVs from websites, which work in binary mode. Now that I'm trying to make my own to save data, I'm running into this.

example:

Python code:

with open('file.csv', 'wb') as f:
    for row in data:
        csv.writer(f).writerow(row)

'writerow' sounds like something Scooby Doo would say

edit: Solved. The correct open line is:

Python code:

with open('file.csv', 'w', newline='') as f:

Dominoes fucked around with this message at 04:10 on May 11, 2013

# ? May 11, 2013 03:34

Dren: Jan 5, 2001; Pillbug

If there is a SQLite bottleneck with that statement you could probably do the queries 1000 at a time into a python list, them batch write them to the other table. Both of those ops should be fast.

# ? May 11, 2013 03:57

Scaevolus: Apr 16, 2007

Karthe posted:

Yeah, my indices are all up to snuff. Individual SELECTs on that table take ten-thousandths of a second to run across several tens of thousands of records, I'm pretty sure it's the individual SELECT query transactions that are bogging things down. Based on what I've read it's a limitation of SQLite, and not even committing records directly to the database helps to speed things up.

Try setting sqlite to a less durable commit level (look at the pragmas).

# ? May 11, 2013 05:40

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 04:14

duck monster: Dec 15, 2004

ShoulderDaemon posted:

Note that there exist platforms where the endianness of integers is not the same as the endianness of floats, because the FPU and the ALU were made as separate units by different teams and cobbled together at the last minute or something.

...and didn't lead to a round of project managers getting laid off for loving incompetence.

# ? May 11, 2013 06:12

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »