Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Nippashish: Nov 2, 2005; Let me see you dance!

duck hunt posted:

Here's my first try today.

Why are you using ord(item[0]) as the hash of an item? Python has a perfectly good hash function for this.
Why can't you specify a key when inserting?
What happens if you insert the same item twice? (I'd say two items with the same key but see the previous question).
Why do I need to know the index of the hash bucket to get an item out of the table? Same question for pop-ing.
What does getItem return? Why is this not crazy?

You basically want to emulate the interface of python's built in dict (minus the syntactic sugar that you get because dict is a builtin).

duck hunt posted:

What would be a better type to collect items? Set? Tuple? Some other class?

Why do you think the best type isn't a list?

# ? Aug 10, 2014 01:03

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 17:58

Jo: Jan 24, 2005; Soiled Meat

Forgive my naivete on the matter, but I'm sitting here wondering if my knowledge is lacking. Is there a reason not to use a dictionary, aside from the academic aspect? Is a dictionary not a hash map? I'd kinda' been using it like a simpler version of Java's HashMap.

EDIT: For context, I'm speaking with reference to...

Nippashish posted:

No, you should be learning how to make a hash table.

Jo fucked around with this message at 01:29 on Aug 10, 2014

# ? Aug 10, 2014 01:24

Nippashish: Nov 2, 2005; Let me see you dance!

Jo posted:

Forgive my naivete on the matter, but I'm sitting here wondering if my knowledge is lacking. Is there a reason not to use a dictionary, aside from the academic aspect? Is a dictionary not a hash map? I'd kinda' been using it like a simpler version of Java's HashMap.

I assumed the point was to not use a dict otherwise it's a silly question. The meta-point I was making with that post is that the correct response to not knowing how to do something is not to freak out, but to learn how to do it.

# ? Aug 10, 2014 01:34

Jewel: May 2, 2009

Is a dictionary not just a hash table internally anyway? I was under the assumption they were the same thing.

# ? Aug 10, 2014 03:04

the: Jul 18, 2004; by Cowcaster

Anyone here use Anaconda? How do I add modules to it?

edit: Specifically talking about a Windows environment

# ? Aug 10, 2014 03:19

BannedNewbie: Apr 22, 2003; HOW ARE YOU? -> YOSHI?
FINE, THANK YOU. -> YOSHI.

the posted:

Anyone here use Anaconda? How do I add modules to it?

edit: Specifically talking about a Windows environment

You can just install stuff exactly the same way you would in any python install. Anaconda should show up if you install from binaries. You can usually also use "conda install <module>" or "pip install <module>"

BannedNewbie fucked around with this message at 04:43 on Aug 10, 2014

# ? Aug 10, 2014 03:26

the: Jul 18, 2004; by Cowcaster

BannedNewbie posted:

You can just install stuff exactly the same way you would in any python install. Anaconda should show up if you install from binaries. You can usually also use "conda install <module>" or "pip install <module"

So why doesn't this work then?

code:

08/09/2014  07:33 PM            28,690 beatboxxx-21.5.tar.gz
               2 File(s)    360,472,594 bytes
               4 Dir(s)  943,833,022,464 bytes free

C:\Users\User\Downloads>conda install beatboxxx-21.5.tar.gz
Fetching package metadata: ..
Error: No packages found matching: beatboxxx-21.5.tar.gz

C:\Users\User\Downloads>

# ? Aug 10, 2014 03:37

vikingstrike: Sep 23, 2007; whats happening, captain

Try "pip install beatboxxx". Looks like you downloaded the source code, which you don't really need.

# ? Aug 10, 2014 03:42

the: Jul 18, 2004; by Cowcaster

That worked, thanks. Any idea why the conda command didn't? Their official docs say that you can install a local package using that command.

# ? Aug 10, 2014 03:43

vikingstrike: Sep 23, 2007; whats happening, captain

the posted:

That worked, thanks. Any idea why the conda command didn't? Their official docs say that you can install a local package using that command.

Not too familiar with anaconda but I don't think you can pass archives into it like that. You downloaded the source code when all you need is to tell it the module to install. See if "conda install beatboxxx" works.

Here is the Pypi page: https://pypi.python.org/pypi/beatboxxx/21.5 Pip downloads the file you did as part of the installation process. If you can find the module on Pypi you can install and upgrade through pip.

# ? Aug 10, 2014 03:45

SurgicalOntologist: Jun 17, 2004

You can install conda packages from tarballs, but since beatboxxx isn't on conda I highly doubt that you had a conda package tarball. If you had untarred you could have installed it locally with pip but it's the same thing.

# ? Aug 10, 2014 04:13

QuarkJets: Sep 8, 2008

Jewel posted:

Is a dictionary not just a hash table internally anyway? I was under the assumption they were the same thing.

Yes, a dictionary is a hash table. A set is a hash table, too.

# ? Aug 10, 2014 04:35

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

Nippashish posted:

Why are you using ord(item[0]) as the hash of an item? Python has a perfectly good hash function for this.

It feels like cheating not to write your own hash function when someone asks you to implement a hash table.

duck hunt posted:

you can see that in the constructor, I'm using a Python list. What would be a better type to collect items? Set? Tuple? Some other class?

Warning: I was never asked this stuff in an interview before, this is how I'd argue about it and I have no idea if it's satisfactory in any way.

The starting point of a hash table is that you have a simple indexed sequence structure that you can add and delete values from it. This sounds like a list to me. Now the main part: You want to look up a key associated with an index of this sequence via a hash function.

Assuming we already have a cool hash function that we can use:

Python code:

l = ['lol',    # value for key 'cat'
     'poop',   # value for key 'dog'
     'banana'] # value for key 'monkey'

def get_value(table, key):
    return table[cool_hash(key)]

# cool_hash('dog') == 1
# get_value(l, 'dog') == 'poop'

This works well if we don't have collisions. One way to deal with collisions is to store not just the value, but a sequence of every key/value pair associated with the same hash. This sounds like a job for a list of tuples.

Python code:

l = [[('cat', 'lol')],
     [('dog', 'poop')],
     [('monkey', 'banana'), ('human', 'computers')]]

def get_value(table, key):
    for (entry_key, entry_value) in table[cool_hash(key)]:
        if entry_key == key:
            return entry_value

# cool_hash('monkey') == 2
# cool_hash('human') == 2
# get_value(l, 'human') == 'computers'
# get_value(l, 'monkey') == 'banana'

And this is the idea for implementing the key lookup. I'll leave the rest (insertion and deletion) and wrapping the whole thing in a class to you. :v:

(actually I'd make get_value with a generator expression but it doesn't matter, the for loop is more straightforward if you want to add a KeyError exception)

Python code:

def get_value(table, key):
    return next(entry_value 
                for (entry_key, entry_value) in table[hash(key)]
                if entry_key == key)

# ? Aug 10, 2014 06:32

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

Oh I was just gonna say that you can also use a collections.deque as the table instead of a list but I noticed that I totally misread the final part of your post.

A python list is a dynamic array, you can use sys.getsize to check the growth of the list in memory as you append more items in it. Anyway, you're implementing a dynamic array using a dynamic array as base, I guess you kinda noticed how silly this looks.

I guess this isn't really a typical CS question for doing in python? Because, the way I see it, there isn't really a more primitive data structure behind it. The point of coding a dynamic array is to show that you know how to deal with memory management (malloc and realloc in C). It's just really awkward to think about these things in python. Implementing linked lists and trees with classes sounds way more natural to me.

# ? Aug 10, 2014 10:11

BigRedDot: Mar 6, 2008

SurgicalOntologist posted:

You can install conda packages from tarballs, but since beatboxxx isn't on conda I highly doubt that you had a conda package tarball. If you had untarred you could have installed it locally with pip but it's the same thing.

The conda tool is package manager for built, binary packages. Continuum maintains a large repository of pre-built conda packages, and there is a publicly available place (binstar.org) that users can build and upload and make their own packages available for conda to install. Conda does not build software though, so if you have a source tarball, you will have to run "python setup.py install" or whatever, which is fine to do.

# ? Aug 10, 2014 13:10

BigRedDot: Mar 6, 2008

Jo posted:

Forgive my naivete on the matter, but I'm sitting here wondering if my knowledge is lacking. Is there a reason not to use a dictionary, aside from the academic aspect? Is a dictionary not a hash map? I'd kinda' been using it like a simpler version of Java's HashMap.

You might not use a dictionary if you care about memory use. Dictionaries and classes (which are basically just dictionaries) grow faster than every other python container, because they have to maintain a certain percentage of the hash table free to ensure lookup performance. Consider:

code:

In [19]: d = {1:1, 2:2, 3:3, 4:4, 5:5}

In [20]: sys.getsizeof(d)
Out[20]: 280

In [21]: d[6] = 6

In [22]: sys.getsizeof(d)
Out[22]: 1048

This is why having "10 million small objects" is a bad idea usually, if your objects go from five to six attributes, you just quadrupled the amount of memory all those objects consume. "Fixing" this is actually what __slots__ is for; it makes objects that don't use dictionaries under the covers and don't have this rapidly growing space overhead, with the trade-off that you can't add new attributes to the objects anymore.

BigRedDot fucked around with this message at 13:21 on Aug 10, 2014

# ? Aug 10, 2014 13:18

SurgicalOntologist: Jun 17, 2004

BigRedDot posted:

The conda tool is package manager for built, binary packages. Continuum maintains a large repository of pre-built conda packages, and there is a publicly available place (binstar.org) that users can build and upload and make their own packages available for conda to install. Conda does not build software though, so if you have a source tarball, you will have to run "python setup.py install" or whatever, which is fine to do.

Right. When I said "it's the same thing" I meant that the tarball the had is the same as what pip downloaded and installed from PyPi.

Also, I would recommend pip install . over python setup.py instal, personally.

# ? Aug 10, 2014 16:03

Carabus: Sep 30, 2002; Booya.

What is the difference between `python-config --prefix`/Python vs `python-config --prefix`/lib/libpython2.7.dylib, or `python-config --prefix`/Headers vs `python-config --prefix`/include/python2.7. The former is missing on some Python distributions such as Anaconda, while Apple ships with both. I suppose the latter (/lib and /include) are best for linking with external programs/additional libraries?

edit: Turns out they're exactly the same.

Carabus fucked around with this message at 23:32 on Aug 10, 2014

# ? Aug 10, 2014 19:14

BigRedDot: Mar 6, 2008

SurgicalOntologist posted:

Right. When I said "it's the same thing" I meant that the tarball the had is the same as what pip downloaded and installed from PyPi.

Also, I would recommend pip install . over python setup.py instal, personally.

Actually I just took 15 seconds to build and upload a conda package for beatboxxx to binstar.

You (or the OP, really) can install it with

code:

conda install -c bryanv beatboxxx

For the curious all I did was:

code:

conda skeleton pypi beatboxxx
conda build --no-test beatboxxx

The ran the binstar upload command it gave me at the end.

# ? Aug 11, 2014 12:19

JayTee: May 24, 2004

edit: nevermind, trying to call an attribute as a method

JayTee fucked around with this message at 15:07 on Aug 11, 2014

# ? Aug 11, 2014 15:01

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

floppo posted:

apologies if this is too broad a question for this thread. I am interested getting data from twitter - specifically a list of all the people following a specific account. My goal is compare two such lists and see who is connected to who - this would require a second set of queries.

I know a little bit of python, but mostly from homework-type assignments that don't require getting data. I've found the twitter API website but I could use a bit more hand-holding. In short does anyone know a good guide to scraping data from twitter using python? I thought I would narrow my search to python stuff for now, but feel free to suggest alternatives if you know of them.

I assume this isn't some money-making venture and you're doing this to gain some experience from the project. In that case, the most valuable skill you will get out of a project like that is the ability to make sense of API docs and build a compliant interface to them. I would strongly suggest you just give it a try. If you're completely lost, the best first hint I can give is that the API will work by you formatting URLs a certain way (as described in the docs) and making web requests with them. The web service will reply with a text document with the data you want in a format also described in the docs.

I strongly suggest, you start by just trying to get something (anything) from the API and build from there. I've done some twitter integration before, so if you get stuck come back or shoot me a PM.

If you don't care about learning how to do it, there is a module called python-twitter that may help, but I haven't really played with it.

# ? Aug 11, 2014 17:25

floppo: Aug 24, 2005

KernelSlanders posted:

I assume this isn't some money-making venture and you're doing this to gain some experience from the project. In that case, the most valuable skill you will get out of a project like that is the ability to make sense of API docs and build a compliant interface to them. I would strongly suggest you just give it a try. If you're completely lost, the best first hint I can give is that the API will work by you formatting URLs a certain way (as described in the docs) and making web requests with them. The web service will reply with a text document with the data you want in a format also described in the docs.

I strongly suggest, you start by just trying to get something (anything) from the API and build from there. I've done some twitter integration before, so if you get stuck come back or shoot me a PM.

If you don't care about learning how to do it, there is a module called python-twitter that may help, but I haven't really played with it.

No money-making here. Just want to see how see how connected members of 'rival' groups in different countries.

Thanks for the link - I really should try it myself though as you suggest.

# ? Aug 11, 2014 18:16

ShadowHawk: Jun 25, 2000; CERTIFIED PRE OWNED TESLA OWNER

Python code:

#!/usr/bin/env python3
import itertools
print("\n".join(("".join(fizzbuzz) or str(number) for number, fizzbuzz in itertools.islice(enumerate(zip(itertools.cycle(("fizz","","")), itertools.cycle(("buzz","","","","")))), 1, 101))))

Do I get the job?

# ? Aug 11, 2014 18:52

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

ShadowHawk posted:

Python code:

#!/usr/bin/env python3
import itertools
print("\n".join(("".join(fizzbuzz) or str(number) for number, fizzbuzz in itertools.islice(enumerate(zip(itertools.cycle(("fizz","","")), itertools.cycle(("buzz","","","","")))), 1, 101))))

Do I get the job?

I always hated questions like that because there are so many possible answers and I never know what they're looking for. Are you trying to show how you're competent to write professional code rather than homework assignments? Use the library. Do you know how to solve basic programming problems? Write a legible homework-style function with superfluous comments. Do you really know the ins and outs of the platform? Do something like ShadowHawk.

It's a bit like that old Google interview question of, "On a new hardware platform, how would you determine if the stack grows up or down in memory?" I always thought the answer should be, "Read the documentation." Anyone writing code to solve that one is tunnel-visioning and just wasting company time. Of course, I don't work for Google, so what do I know.

# ? Aug 11, 2014 19:16

Space Kablooey: May 6, 2009

ShadowHawk posted:

Python code:

#!/usr/bin/env python3
import itertools
print("\n".join(("".join(fizzbuzz) or str(number) for number, fizzbuzz in itertools.islice(enumerate(zip(itertools.cycle(("fizz","","")), itertools.cycle(("buzz","","","","")))), 1, 101))))

Do I get the job?

You crashed my computer, so yes. :v:

EDIT: Changed zip to itertools.izip and it worked in python 2.7

Space Kablooey fucked around with this message at 20:21 on Aug 11, 2014

# ? Aug 11, 2014 20:10

Crosscontaminant: Jan 18, 2007

HardDisk posted:

You crashed my computer, so yes.

Your computer symlinks python3 to a Python 2 instance? That's a bit dubious. I'd do something about that, like installing Python 3.

KernelSlanders posted:

It's a bit like that old Google interview question of, "On a new hardware platform, how would you determine if the stack grows up or down in memory?" I always thought the answer should be, "Read the documentation."

Generally that's the correct answer, but this being Google it's very possible there is no documentation because you're on the team writing it. :haw:

# ? Aug 11, 2014 21:40

Space Kablooey: May 6, 2009

Crosscontaminant posted:

Your computer symlinks python3 to a Python 2 instance? That's a bit dubious. I'd do something about that, like installing Python 3.

Nah, it was my own fault, am I'm used to writing just "python" in the command line: "Oh, this is Python 3 code, I just have to import print function and it will work. What can go wrong? :downs:

# ? Aug 11, 2014 23:23

qntm: Jun 17, 2009

KernelSlanders posted:

I always hated questions like that because there are so many possible answers and I never know what they're looking for. Are you trying to show how you're competent to write professional code rather than homework assignments? Use the library. Do you know how to solve basic programming problems? Write a legible homework-style function with superfluous comments. Do you really know the ins and outs of the platform? Do something like ShadowHawk.

It's not supposed to test your professional code quality, language-agnostic programming ability or platform knowledge. It may be a test to see whether you're able to follow instructions and implement an unambiguous specification. But the real purpose of that question, if it gets asked, is to demonstrate in thirty seconds or less that the "Python" bullet point on your CV was not a bare-faced lie.

# ? Aug 12, 2014 11:09

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I was going to whip together a quick script to fix up my XBMC library (which is stored in MySQL) but something is stumping me, and I think this is a Python thing and not an SQL thing.

Python code:

# lol copy/paste formatting.  whatever.

sql = """
			 UPDATE path
			 SET strPath = REPLACE(strPath, 'smb://server/Media/', 'smb://server/Media3/')
			 WHERE strPath LIKE 'smb://server/Media/TV/The Show%'
      """
cur.execute(sql)
print("Rows updated: {}".format(cur.rowcount)

This outputs "Rows updated: 8", which is the correct row count for that LIKE statement.

However, it doesn't actually update the database.

If I copy/paste the sql into MySQL Workbench, it updates the database correctly.

What incredibly dumb and stupid thing am I doing wrong?

# ? Aug 12, 2014 22:05

accipter: Sep 12, 2003

Thermopyle posted:

SQL

Make sure you do

code:

cur.commit()

to apply the changes

# ? Aug 12, 2014 22:26

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

accipter posted:

Make sure you do
code:
cur.commit()
to apply the changes

Blah. Of course. conn.commit() did it.

Using ORM's makes the brain go dull.

Thanks!

# ? Aug 12, 2014 22:46

QuarkJets: Sep 8, 2008

accipter posted:

Make sure you do
code:
cur.commit()
to apply the changes

Is this MySQLdb or something else? For whatever reason I never have to use commit() with MySQLdb

# ? Aug 12, 2014 23:56

KernelSlanders: May 27, 2013; Rogue operating systems on occasion spread lies and rumors about me.

QuarkJets posted:

Is this MySQLdb or something else? For whatever reason I never have to use commit() with MySQLdb

Depending on whether you're using MyISAM or InnoDB, MySQL may not support transactions. It's also possible on InnoDB to set autocommit on.

# ? Aug 13, 2014 01:11

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

QuarkJets posted:

Is this MySQLdb or something else? For whatever reason I never have to use commit() with MySQLdb

It depends on how the db is configured, not the client you're using.

Also, for future reference, I'm using python 3 so it's "something else" since MySQLdb does not and will not exist for python 3.

I've used PyMySQL and mysql-connector to replace it and they seem to work fine. Also, they're pure python so no having to deal with compiling C packages and whatnot.

I'm not sure how their performance compares.

# ? Aug 13, 2014 01:53

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Thermopyle posted:

MySQLdb does not and will not exist for python 3

Hmm, this pull request is at least marked with the "1.3.0" milestone now. On one hand, it looks almost production-ready -- on the other hand, the maintainer seems to have been MIA for almost 4 months now. Not very promising.

I help maintain a decently-large Django project at a side job, and we switched from MySQL to Postgres in the middle of last year. We did this for non-Python reasons -- schema migrations are a real pain when you can't do things like 'alter table' inside a transaction. We weren't yet on Python 3 at that time, but it was nice to not have MySQL as a blocker when we fully switched to 3.4 this past April.

# ? Aug 13, 2014 03:43

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Lysidas posted:

Hmm, this pull request is at least marked with the "1.3.0" milestone now. On one hand, it looks almost production-ready -- on the other hand, the maintainer seems to have been MIA for almost 4 months now. Not very promising.

Yeah, when I said "will not" I was more making a prediction than repeating the official line of the mysqldb maintainer(s).

In researching mysql-python clients for python 3, people have been talking for years about mysqldb and python 3,so I'm not holding out any hope for anything "official".

# ? Aug 13, 2014 06:53

girl dick energy: Sep 30, 2009; You think you have the wherewithal to figure out my puzzle vagina?

So, I'm trying to learn Python! Specifically, I'm using it and the libtcod library to make a simple roguelike. I'm going through the tutorial, and everything's been good so far. But I've been stopped cold by an error while trying to implement FOV that I can't seem to figure out.

The error is

code:

TypeError: 'builtin_function_or_method' object has no attribute '__getitem__'

While trying to use the following

code:

fov_map = libt.map_new(MAP_WIDTH, MAP_HEIGHT)
for y in range(MAP_HEIGHT):
	for x in range(MAP_WIDTH):
		libt.map_set_properties(fov_map, x, y, not map[x][y].block_sight, not map[x][y].blocked)
fov_recompute = True

Specifically, it's citing line 268, which is the libt.map line. This part is entered exactly as it is in the tutorial, so at first I thought it might be something broken because of how a change was made, or how my dev environment is set up, but the example version of the code works just fine. It worked perfectly well before the FOV stuff went in, and I did my best to make sure it wasn't a simple sanity-check issue like forgetting a bracket somewhere, but that doesn't seem to be it, either. I could theoretically just copy the example text and re-implement all my personal changes, see if anything breaks. But somehow, I feel like that might not be the answer, since this is supposed to be a learning project, not a 'copy the answers'. I went through my code and theirs and did just about everything I could to make sure it wasn't just something I hosed up implementing, but if it is, I can't figure it out.

The tutorial has no mention of __getitem__ at all, which makes me think it was something added after the tutorial was made (it wouldn't be the first hiccup or unclear bit I've run into), but that's as much as I can figure out.

Here is my whole 347-line .py, needless changes (numpad movement!), dumb comments and all. And here is the tutorial's default stuff.

I'm sure it's something really stupid, but I've spent all day trying to figure out what the hitch is without any success. :downs:

# ? Aug 14, 2014 04:52

Space Kablooey: May 6, 2009

map is a function that is always available for every script, so you can't have anything that is called "map"

https://docs.python.org/2/library/functions.html

https://docs.python.org/3.4/library/functions.html

__getitem__ is what the "array[i]" actually calls down in the guts of python:

https://docs.python.org/2/library/operator.html#operator.__getitem__

Space Kablooey fucked around with this message at 05:06 on Aug 14, 2014

# ? Aug 14, 2014 05:02

girl dick energy: Sep 30, 2009; You think you have the wherewithal to figure out my puzzle vagina?

I use map[x][y].whatever in a lot of the code. Which parts would I need to replace/rename? And that still doesn't explain why the example code works, when it does the exact same thing? :confused:

# ? Aug 14, 2014 05:05

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 17:58

KICK BAMA KICK: Mar 2, 2009

Poison Mushroom posted:

code:

not map[x][y].block_sight, not map[x][y].blocked

Here's the problem. map is the name of a built-in function. Either a) the code you're c/ping must have elsewhere redefined map to refer to a list of lists or something to that effect, whose items you could access with the [x][y] syntax or b) you made a typo -- hopefully this, because the former would mean you're reading some dumb code. __getitem__ is the method that is silently invoked when you access an element of a list by index, or an element of a dictionary by key, etc. Because map is just a function, and functions don't have elements you can access with that syntax, it doesn't have a __getitem__ method and thus Python complains.

So either you mistyped map in place of something else (like fov_map perhaps?) or you need to incorporate the code that defines map. In the unlikely event that this code does redefine a very basic and useful built-in function rather than simply coming up with a unique name, consider learning from another source because that's a huge red flag.

# ? Aug 14, 2014 05:06

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »