Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

Titan Coeus: Jul 30, 2007; check out my horn

I have only used this briefly, but if you guys are trying to write code without the GIL slowing things down, there is the multiprocessing library. (Python 3 link)

# ¿ Jun 19, 2012 05:23

Adbot: ADBOT LOVES YOU

# ¿ May 19, 2024 22:01

Titan Coeus: Jul 30, 2007; check out my horn

OnceIWasAnOstrich posted:

Take a look at the BeautifulSoup library or Scrapy, much easier than trying to extract bits of HTML with string commands.

Seconding BeautifulSoup. Made my life much easier when parsing poorly formatted HTML.

# ¿ Aug 4, 2012 23:18

Titan Coeus: Jul 30, 2007; check out my horn

MC Fruit Stripe posted:

Either they changed the meaning of parsing (unlikely), what I am trying to do is more complicated than space exploration (less likely), or I am just not loving seeing this (very likely).

All I want is to take a page on Allmusic.com, say The Beatles page, located here: http://www.allmusic.com/artist/the-beatles-mn0000754032

And ultimately, after however many steps, have exactly the following result
code:
The Beatles	British Invasion;British Psychedelia;Contemporary Pop/Rock;Early Pop/Rock
(Styles pruned for table breakage)

That just does not feel that difficult. But I can get it to return the page, return the page with formatting stripped, I can get it to tell me where 'styles' is, return every link or bolded word in the page, but I just can not get it to grab a section of text.

Help?

I have no idea how to suggest improvements as I have no clue what you are doing.

# ¿ Aug 5, 2012 00:31

Titan Coeus: Jul 30, 2007; check out my horn

Ulio posted:

nvm fixed it, thanks. IDLE was just being weird.

edit: Everything I type on IDLE says invalid syntax. It even errors the text written at the very top which appears each time on its own.

Screenshot?

# ¿ Aug 8, 2012 04:23

Titan Coeus: Jul 30, 2007; check out my horn

A couple things make this code tricky to read. In particular, the long length of lines caused by comments. This is true for any language, but Python programs in particular tend to keep their line length to 80 characters or less, but no one ~~will~~ should be outraged if you go up to 100.

Instead of doing

Python code:

dict_keys = [] #it's a lot of work to iteratively create a tuple so I just make a list and turn it into a tuple with tuple()

Put the comment on it's own line and break after 80 characters

Python code:

# Create a list and convert it to a tuple with tuple() to avoid constantly
# making new tuples for each change
dict_keys = []

As far as Python specific stuff, the biggest thing I am noticing is your use of for loops:

Python code:

for x in range(len(data1_dict["key"])): dict_keys.append(data1_dict["key"][x])

In Python, think of a for loop as something that iterates over a collection, not something that increments a loop variable (like you would be used to in C).

Python code:

for value in data1_dict["key"]:
    dict_keys.append(value)

# ¿ Sep 18, 2012 07:50

Titan Coeus: Jul 30, 2007; check out my horn

MeramJert posted:

I hope NumPyPy exists soon because people have apparently donated $45,000 to fund its development

There was a commit 10 days ago that commented out some code... I'm not sure this was the best investment

# ¿ Sep 21, 2012 05:16

Titan Coeus: Jul 30, 2007; check out my horn

Kerpal posted:

Does anyone have experience with the _winreg module? I'm trying to write a script that deletes a registry key in HKLM\Software\Classes\Installer\Products, however the _winreg DeleteKey method cannot delete keys with subkeys, which gives a WindowsError exception "Access is denied" error 5. The only solutions I could think of was to create a function that recursively deletes every subkey in the input key before deleting the primary key. I'm trying to do this remotely, so I have to be able to connect to a remote registry. Another solution was to call the reg command like:

reg delete "\\\\%s1\\HKLM\Software\Classes\Installer\Products\%s2"

Where %s1 is the computer and %s2 is the product key I'm trying to delete. Running this command produces a different error, "The procedure number is out of range." Running the same command on the local machine worked fine.

The "rdelete" function here might help: http://code.activestate.com/recipes/476229-yarw-yet-another-registry-wrapper/

# ¿ Oct 4, 2012 16:38

Titan Coeus: Jul 30, 2007; check out my horn

the posted:

http://pastebin.com/3q8dW3EY

I'm doing an assignment where I have to do a certain number of "sweeps" in a loop to get an accurate value.

My professor's Fortran version does 2000 sweeps in 3 seconds on his laptop. Mine currently does 100 sweeps in about 10 minutes on a quad-core desktop. So, I think something is really wrong with my code.

The loop is here:
Python code:
change = 1.
while change < 101:
    for i in range(1, xx.size-1):
            for j in range(1, xx.size-1):
                for k in range(1, xx.size-1):
                    v[i,j,k] = (1./6)*(v[i+1,j,k]+v[i-1,j,k] + v[i,j+1,k]+v[i,j-1,k] + v[i,j,k+1]+v[i,j,k-1]) + (b_width**2/(6*enot))*rho_grid[i,j,k]
    print change
    change = change + 1
How could I do that differently to speed it up tremendously? Or is Fortran just that much faster than Python?

Perhaps someone who knows more about this than me see something I don't, but other than doing a couple calculations ahead of time (e.g. 1./6, (b_width**2/(6*enot))), I don't see anything in particular you can do to speed this up. This is not unusual, but I am surprised this is so much slower so hopefully I am missing some insight someone else will have.

# ¿ Oct 10, 2012 03:05

Titan Coeus: Jul 30, 2007; check out my horn

MeramJert posted:

See if you can reduce the number of nested for loops. Right now you have 4 nested for loops (your while loop is just another for loop in disguise), and that's what's slowing you down. Specifically, you should be able to eliminate all of the inner for loops using itertools.product. Itertools is made for these kinds of things, and I bet you'll see a nice speedup just from that.

Also, your quad core processor isn't being utilized. If you really need to speed stuff up, it may be worth looking into the built in multiprocessing library, but it will add a lot of complexity and you might need to rethink the problem a bit. I don't know if your professor's Fortran version can utilize multiple cores or not.

I don't know how much it would help in your situation, but you could also look into PyPy or Cython. I've heard very good things about them for speeding up numerical things like this.

Itertools might be useful (I'll be reading the documentation on itertools after this post..) but the other suggestions I don't think will be. Each iteration of the loop has a dependency on a prior iteration, so there isn't anything to parallelize. PyPy doesn't have NumPy support ("soon"), and if he is going to use Cython he might as well just write the entire thing in C.

# ¿ Oct 10, 2012 03:12

Titan Coeus: Jul 30, 2007; check out my horn

Misogynist posted:

I just finished my first legitimate (non-toy) Python project, Metricinga, a daemon that forwards Nagios/Icinga performance data to Graphite. Would anyone mind taking a minute to look over my code and tear it apart so I can write better code?

Some nitpicks:

Python code:

# this
    if metric_prefix != '':
# should be
    if metric_prefix:

Python code:

# this
try:
    pf = file(self.pidfile, 'r')
    pid = int(pf.read().strip())    
    pf.close()
except IOError:
    pid = None
# should be
try:
    with file(self.pidfile, 'r') as pf:
        pid = int(pf.read().strip())    
except IOError:
    pid = None

Python code:

# this
while 1:
# should be
while True:

Python code:

# this
if daemonize is True:
# should be
if daemonize:

Python code:

# this
(opts, args) = parser.parse_args()
# should be
opts, args = parser.parse_args()

Python code:

# this
file(self.pidfile,'w+').write("%s\n" % pid)
# should be
with file(self.pidfile,'w+') as pid_file:
    pid_file.write("%s\n" % pid)

I'm not sure about send_metric(self, metric). If you have no internet connection, this is going to loop indefinitely. You should probably set some limit (20?) on the number of failed attempts.

In stop() you call str(err) twice, the second time it will always already be a string.

In run(), your finally block has a "shutdown successfully" message. If joinall throws some other non-expected error, that message will be false.

See the following regarding string formatting: http://docs.python.org/2/tutorial/inputoutput.html#old-string-formatting

And this regarding your use of file vs. open: http://stackoverflow.com/questions/112970/python-when-to-use-file-vs-open

# ¿ Nov 2, 2012 02:07

Titan Coeus: Jul 30, 2007; check out my horn

Drunk Badger posted:

I'm making a serious attempt at learning Python, so I'm looking for a good Python IDE. The only other IDE I have good experience with is Visual Studio for a C# and C++ class, pop up tips and the ability to set break points with a nice list of variables and their values are probably the two things that helped the most in learning those, and as a novice programmer, probably things I require to hold my hand as I get into Python.

After looking at the IDE list I see that Spyder looks like a decent choice, the only downside is that it looks like it only supports Python 2, and I'd like to stick with version 3.

So my questions are:

1. Is there a Python 3 IDE that contains breakpoints and pop up hints?

2. Is it a terrible idea to use version 2 and Spyder instead of version 3 and some possibly lesser IDE?

I've always liked PyDev http://pydev.org/

# ¿ Nov 8, 2012 23:27

Titan Coeus: Jul 30, 2007; check out my horn

Thern posted:

Testing woes

Looking at it, my guess is you should change @patch("subprocess.Popen") to @patch("popen"). I'm not familiar with this testing framework though so that could be entirely off base.

# ¿ Nov 16, 2012 08:12

Titan Coeus: Jul 30, 2007; check out my horn

Masa posted:

What's the best way to copy text to the clipboard in a program using PyQt? I've tried using QApplication.clipboard, but it only works while the program that copied the text is still running.

What system is this on? That sounds like an X11 problem.

# ¿ Dec 2, 2012 07:42

Adbot: ADBOT LOVES YOU

# ¿ May 19, 2024 22:01

Titan Coeus: Jul 30, 2007; check out my horn

The Rosalind project has some good problems to practice programming on. I have done a few of them and didn't need any biology background. I can't speak for the higher level stuff.

# ¿ Dec 11, 2012 03:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.