Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

octobernight: Nov 25, 2004; High Priest of the Christian-Atheist Church

This is a dumb question, but I'm having an impossible time trying to debug a multi-threaded program being run remotely. I am running pdb and think I found the call that causes my program to hang.

Basically, when the get() function in Python's multiprocessing module is called, my program stops. When I inspect the get function, it lists:

<built-in method recv of _multiprocessing.Connection object at 0xd3683e40>

Is there any way in pdb to say, "get me the object at 0xd3683e40" so I can inspect it to see what it is, as I only have access to the function, not the object the function came from.

Thanks!

# ? Nov 28, 2014 21:30

Adbot: ADBOT LOVES YOU

# ? Jun 2, 2024 22:59

Mortanis: Dec 28, 2005; It's your father's lightsaber. This is the weapon of a Jedi Knight.; College Slice

Ultra newbie question here. I'm working my way through a Python Text Adventure platform from the ground up on my own, without any sort of book. I'm thinking I'd like to spend a bit to refactor before it gets unwieldy, as right now everything is in main.py. I'm having a hard time getting a handle on including things out, though. I want to move my item class definition and instances (nothing is database driven, I'm just hand creating them in the file), room class definition and instances, and mob class definition and instances to their own files, but there's a lot of cross talk. For instance, for my items, I've got them using lambdas to handle use so that a healing potion can do player.hp += 10 or automatically add an instance of an item to inventory or teleport a player to a specific room. Works fine when everything is in the same file, but as soon as I start splitting things into separate files, items can't see "player" or my maps, etc.

Is this because I'm just not coding in a way that Python works well in, or am I missing something that allows things to play nice? I get having everything be completely self-contained, but if I can't have instances seeing each other that's a serious blow to my methodology.

# ? Nov 29, 2014 04:30

SurgicalOntologist: Jun 17, 2004

Are these instances globals? That's something you should try to avoid. There may be other solutions, but a straightforward one is to add arguments to all these functions. So instead of adding 10 hp to the global player instead, add 10 hp to the player instance that was input.

This kind of design will aid in generality, now these potions will work on anything with an hp attribute, which may come in handy if you ever get to npc ai for example.

Edit: you may also want to put the instance creation separate from the classes. That's what makes more sense to me at least; separate the platform from from the game you're making with it.

SurgicalOntologist fucked around with this message at 04:49 on Nov 29, 2014

# ? Nov 29, 2014 04:46

KICK BAMA KICK: Mar 2, 2009

Yeah, it just sounds like you haven't quite wrapped your head around some OOP concepts yet, which is fine -- they're not at all intuitive until one day they totally are. Here's a super quick, dirty mock-up that I hope might send you down the right path, at least. You could put each of those classes in their own file and just import them where you need them. The idea is that each kind of thing exposes an interface that other things can use to get only the information they need to do their job.

# ? Nov 29, 2014 05:21

Mortanis: Dec 28, 2005; It's your father's lightsaber. This is the weapon of a Jedi Knight.; College Slice

I've been doing VERY BASIC OOP for awhile, just nothing like this and obviously very sloppy. At least Python is forcing me to clean up, which is great. It means I'm going to have to start over, though. I thought about going the route of having everything have methods for all the possible interactions (heal, damage, etc), but thought it would get unwieldy and tried to go with lambdas - seems I jut made more of a mess that way.

Using globals is right on the nose. I'd create all my instances of rooms, items, mobs and the player and then keep a global variable of which room the player is in, but every single method needed to pull in that room id and then check getRoomByCurrentID(current_room)... drat. I had just about everything done - all the verb functions for moving, using (including use x on y), attacking, picking up, dropping, equipping, giving, quests... But it's all slipshod and I need to learn this proper.

I think I'm just going to need to find a resource for something like this. I had all of it tied to a game state loop that just called the same functions over and over. I'm trying to think of how to structure the game more OOP. A master game class with methods for each phase maybe. Suppose it's time to research, but at least it's not all in vain.

Thanks all!

# ? Nov 29, 2014 05:44

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

Cingulate posted:

Ah, knew there'd be some map-like thing in there. Thanks.

I actually tried passing a map to pandas in the first example like this

Python code:

df['some_field'] = pd.Series(map(some_function(x), list_b), index=list_a)

but it didn't work as expected so I changed it to a generator. I sent a pull request fixing this, maybe it'll work correctly in the next pandas release :ssh:

# ? Nov 29, 2014 05:50

SurgicalOntologist: Jun 17, 2004

Mortanis posted:

I've been doing VERY BASIC OOP for awhile, just nothing like this and obviously very sloppy. At least Python is forcing me to clean up, which is great. It means I'm going to have to start over, though. I thought about going the route of having everything have methods for all the possible interactions (heal, damage, etc), but thought it would get unwieldy and tried to go with lambdas - seems I jut made more of a mess that way.

Using globals is right on the nose. I'd create all my instances of rooms, items, mobs and the player and then keep a global variable of which room the player is in, but every single method needed to pull in that room id and then check getRoomByCurrentID(current_room)... drat. I had just about everything done - all the verb functions for moving, using (including use x on y), attacking, picking up, dropping, equipping, giving, quests... But it's all slipshod and I need to learn this proper.

I think I'm just going to need to find a resource for something like this. I had all of it tied to a game state loop that just called the same functions over and over. I'm trying to think of how to structure the game more OOP. A master game class with methods for each phase maybe. Suppose it's time to research, but at least it's not all in vain.

Thanks all!

You don't necessarily have to start over. You could start by making everything that's a global variable, instead be an attribute of an instance of a GameState class. It would probably make sense to have some of those functions be its methods.

# ? Nov 29, 2014 06:53

KICK BAMA KICK: Mar 2, 2009

Mortanis posted:

At least Python is forcing me to clean up, which is great. It means I'm going to have to start over, though. I thought about going the route of having everything have methods for all the possible interactions (heal, damage, etc), but thought it would get unwieldy and tried to go with lambdas - seems I jut made more of a mess that way.

This all gets easier once you understand a few more OOP concepts like inheritance, interfaces and mixins. Want a thing to have health and respond to heals or damage?

Python code:

class HealthMixin:
    health = 100  # Could change this in the constructor of any class that inherits
    alive = True
    
    def heal(self, value):
        self.health += value

    def damage(self, value):
        self.health -= value
        if self.health <= 0:
            self.die()
    
    def die(self):
        self.alive = False

and have a class like Person inherit HealthMixin. Then you can use the heal, damage or die methods on any Player without cluttering up the definition of Player with that code. Even better, could also apply the HealthMixin to the class NPC to give them the same functionality. Then when you're writing any code that deals with a Player or NPC, you know it's OK to call those methods because you know they inherit them from HealthMixin.

That's a very simplified example and glides over some of the pitfalls of inheritance and such but that's the basic idea -- segregate functionality in the smallest units that make sense and combine them as needed.

KICK BAMA KICK fucked around with this message at 07:29 on Nov 29, 2014

# ? Nov 29, 2014 07:24

Mortanis: Dec 28, 2005; It's your father's lightsaber. This is the weapon of a Jedi Knight.; College Slice

No, that's helpful, thanks. I've been over all the OOP stuff before in other languages, but somehow completely forgot to apply it to Python and just was using objects as basic data shortage containers. This presents all sorts of new challenges, but that's that's learning the "right" way is better than locking myself into something sloppy. A rewrite would be good.

Double Edit: Removing my question. I took a second look and figured it out. Indentation is important!

Mortanis fucked around with this message at 02:16 on Nov 30, 2014

# ? Nov 29, 2014 20:42

FAT32 SHAMER: Aug 16, 2012

I have a python project that seems to be more about sqlite than about python, so if this is the wrong thread for this question please forgive me. I've never used databases before and while I get how to make tables and insert poo poo into them, I don't get my instructions. I'm supposed to have table Orders USE table Inventory and table ClientInfo MAKE Orders. I looked into it and read something about joining columns but then it launched into keys and aliases and basically I'm kind of lost

Python code:

import sqlite3
dbconn = sqlite3.connect('RCF.db',isolation_level=None)
c = dbconn.cursor()
c.execute(("""
    CREATE TABLE ClientInfo (
    ClientName VARCHAR(50),
    PhoneNumber NUMERIC(10,0),
    EmailAddress VARCHAR(255))"""))
c.execute(("""
    CREATE TABLE OrderInfo (
    OrderNumber INTEGER PRIMARY KEY AUTOINCREMENT,
    QuantityOrdered NUMERIC(6,0),
    ProductName VARCHAR(50),
    TotalPrice NUMERIC(7,0))"""))
#there is one order per type of item that the client wants, ie if a client wants 5 CB1000's and 5 CB1000s,
#there would be one order for CB1000 and another for CB1000s
c.execute(("""
    CREATE TABLE Inventory (
    ProductName VARCHAR(50),
    UnitPrice NUMERIC(7,0),
    Quantity NUMERIC(5,0))"""))

c.execute(("""
    INSERT INTO INVENTORY (ProductName, ItemNumber, UnitPrice, Quantity)
        VALUES('Circuit Board 1000', '1000', '1000');"""))
c.execute(("""
    INSERT INTO INVENTORY (ProductName, ItemNumber, UnitPrice, Quantity)
        VALUES('Circuit Board 1000s', '1200', '1000');"""))
c.execute(("""
    INSERT INTO INVENTORY (ProductName, ItemNumber, UnitPrice, Quantity)
        VALUES('Circuit Board 1000st', '1400', '1000');"""))
c.close()
dbconn.close()

To keep things easy, the professor suggested that I make it so that there is one order per type of product. My partner, who is building the GUI and the middleware, has it so that by pressing a button it will add an order, and subtract the inventory needed to fulfill also in Table OrderInfo I couldn't figure out how to make incrementing order numbers without using the INTEGER PRIMARY KEY AUTOINCREMENT thing.

Edit: Nevermind, I figured it out. Thanks anyways guys!

FAT32 SHAMER fucked around with this message at 04:35 on Dec 2, 2014

# ? Dec 2, 2014 03:42

zergstain: Dec 15, 2005

I'm trying to use subprocess.Popen() with a process that generates data forever at a high rate. I'd like it so that before each call to readline(), the pipe buffer is cleared such that I'm always reading the latest output, and anything that was received between calls to readline() falls into a blackhole.

Alternatively, I'd like a tiny pipe buffer, and writes from the child process to simply disappear when the buffer is full, as opposed to blocking. But I'm guessing that will require modifying the linux kernel.

# ? Dec 3, 2014 03:43

Edison was a dick: Apr 3, 2010; direct current only

zergstain posted:

I'm trying to use subprocess.Popen() with a process that generates data forever at a high rate. I'd like it so that before each call to readline(), the pipe buffer is cleared such that I'm always reading the latest output, and anything that was received between calls to readline() falls into a blackhole.

I can't think of any syscall that would allow you to do that. There's some bufferbloat stuff in-progress for the next version of Linux that would allow you to do this through the network stack, by having queue logic that drops packets from the end if the buffer is full, but that would be a massive hack.

The issue is that this is the opposite way around to the normal operation, as usually it's the reader that blocks waiting for the writer to provide data, whereas you want the writer to block waiting for the reader to be reading data.

quote:

Alternatively, I'd like a tiny pipe buffer, and writes from the child process to simply disappear when the buffer is full, as opposed to blocking. But I'm guessing that will require modifying the linux kernel.

If you can modify the process that's sending the data to set non-blocking mode on its output file descriptors, have it put a message length at the beginning of each message, and just silently drop writes where it either returned EAGAIN/EWOULDBLOCK, or write returned less than the message length, but you'd have to have the reader be reading the whole socket buffer in at once to be able to notice that it has to discard the message.

Honestly this sounds closer to SOCK_SEQPACKET instead, where writes will fail if the message is unable to fit in the buffer.
You'll still have at least 1 really old message in the buffer that you need to get through before new messages will be available, but if the message includes a timestamp you could drop old messages.

http://man7.org/conf/lca2013/IPC_Overview-LCA-2013-printable.pdf may be relevant, as I can't think of any IPC primitive that would do what you want, but rather than putting stuff in the kernel, you may be able to implement it on top of shared memory as a rotating buffer.

As an alternative to all this fiddling about with IPC primitives, you could try piping your process through svlogd, and when you want to read a message, you open the file path it's writing messages to and read that file.

Things get complicated when you reach the end of the file, as you need to re-open the path and use os.fstat to see if the inode of the new file is different. If that's the case then you have to start reading from the new one while closing the old one, if not then svlogd is still writing to that file, and you have to wait for it to provide more data.

You could lose messages if svlogd rotates yet another file between you finishing reading one file, and checking to see if there's a new file to read, but given you want to discard old messages anyway, this should be fine.

# ? Dec 3, 2014 13:15

Edison was a dick: Apr 3, 2010; direct current only

zergstain posted:

I'm trying to use subprocess.Popen() with a process that generates data forever at a high rate. I'd like it so that before each call to readline(), the pipe buffer is cleared such that I'm always reading the latest output, and anything that was received between calls to readline() falls into a blackhole.

I probably read far too much into your request, so if you just want to discard the contents of the buffer then do a blocking readline, then you can probably do.

code:

import fcntl
import os
import errno
def drain_pipe_buffer(fobj):
   fd = fobj.fileno()

   oldflags = fcntl.fcntl(fd, fcntl.F_UNCLK)
   newflags = nf | os.O_NONBLOCK
   fcntl.fcntl(fd, fcntl.F_SETFL, newflags)

   while True:
       try:
           os.read(fd, 4096)
       except OSError as e:
           if e.errno in (errno.EEXIST, errno.EAGAIN):
               break
           raise

   fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)

And call drain_pipe_buffer(p.stdout) before p.stdout.readline()

# ? Dec 3, 2014 14:54

zergstain: Dec 15, 2005

Edison was a dick posted:

I probably read far too much into your request, so if you just want to discard the contents of the buffer then do a blocking readline, then you can probably do.
code:
import fcntl
import os
import errno
def drain_pipe_buffer(fobj):
   fd = fobj.fileno()

   oldflags = fcntl.fcntl(fd, fcntl.F_UNCLK)
   newflags = nf | os.O_NONBLOCK
   fcntl.fcntl(fd, fcntl.F_SETFL, newflags)

   while True:
       try:
           os.read(fd, 4096)
       except OSError as e:
           if e.errno in (errno.EEXIST, errno.EAGAIN):
               break
           raise

   fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)
And call drain_pipe_buffer(p.stdout) before p.stdout.readline()

Thanks. My concern is that the other process will output data faster than your function will drain it.

I really just want to send the stdout over a UDP socket, so I'll see how the performance of just doing a shell redirect to /dev/udp/ip/port is.

The receiving process samples the input every 100ms, so I want to drop packets that are received between sample intervals.

# ? Dec 3, 2014 19:30

Edison was a dick: Apr 3, 2010; direct current only

zergstain posted:

Thanks. My concern is that the other process will output data faster than your function will drain it.

Then your best option is to do one read of the size of the pipe buffer and hope.
You can get the buffer size with socket.getsockopt(fd, socket.SO_RCVBUF).

quote:

I really just want to send the stdout over a UDP socket, so I'll see how the performance of just doing a shell redirect to /dev/udp/ip/port is.

The receiving process samples the input every 100ms, so I want to drop packets that are received between sample intervals.

It sounds like what you actually want is a daemon that does the sampling, which only reports the results when there's a client connected, so the daemon might need to run something like the built-in socketserver with UNIX domain sockets.

# ? Dec 3, 2014 20:33

zergstain: Dec 15, 2005

Edison was a dick posted:

Then your best option is to do one read of the size of the pipe buffer and hope.
You can get the buffer size with socket.getsockopt(fd, socket.SO_RCVBUF).

It sounds like what you actually want is a daemon that does the sampling, which only reports the results when there's a client connected, so the daemon might need to run something like the built-in socketserver with UNIX domain sockets.

So one thing sends output to the daemon and then the client connects to the daemon?

Basically I need to send the output of a tool that only runs on Linux to a program that only runs on Windows. After doing some post-processing. Meaning my bash redirection is a no go.

# ? Dec 3, 2014 22:26

Edison was a dick: Apr 3, 2010; direct current only

zergstain posted:

So one thing sends output to the daemon and then the client connects to the daemon?[quote]

I was thinking more that the thing doing the sampling would act as the daemon, so you don't need to send the data around when nobody is listening. Shoving a daemon in to receive output from the sampler would just be moving the problem. Inetd style socket activation might be a neat way to do this, as you could just stop logging samples out when it detects that its client has dicsonnected.

This assumes that you can alter your data logger will handle its standard output being some form of socket

[quote]Basically I need to send the output of a tool that only runs on Linux to a program that only runs on Windows. After doing some post-processing. Meaning my bash redirection is a no go.

How does the Windows program receive its input? What does it do with it? Can it take a stream?

# ? Dec 3, 2014 23:07

zergstain: Dec 15, 2005

Edison was a dick posted:

How does the Windows program receive its input? What does it do with it? Can it take a stream?

Currently through a UDP port. It needs to be pretty close to real time, so I don't think TCP is what I want. It takes data from multiple devices, and outputs them to CSV.

Most of that functionality I added myself. I moved the recv() call to a new thread today. It just overwrites the same buffer over and over, and the sampler thread gets whatever happens to be there when it comes around. With a Critical Section to prevent data corruption of course.

# ? Dec 4, 2014 01:02

Experto Crede: Aug 19, 2008; Keep on Truckin'

I want to scan through the output of a linux command to pull out a specific line which has a string in it.

If I use subprocess.check_output I can use re.findall to to show that string in the output, but I want to be able to add the whole line as a variable.

I'm thinking I could save each line into a list and search each one, then save that into a variable, but not sure how to do that. I think it's due to the object type check_output is creating, but I'm not 100%

Any thoughts?

# ? Dec 6, 2014 02:55

QuarkJets: Sep 8, 2008

Try this:

Python code:

my_process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT))
for line in my_process.stdout:
    print line
    if substring in line:
        do_something()

Alternatively, if you just want to do something with, say, the first instance of the substring, then you can use pipes with my_process.communicate() and then search through the returned iterables for just the first instance of your substring.

QuarkJets fucked around with this message at 03:51 on Dec 6, 2014

# ? Dec 6, 2014 03:49

Coin: Jan 9, 2006; I'm no shitposter; I always know how I'm posting is wrong. I'm just a guy that doesn't like reading the thread, effortposting, and respecting the mods. So if you think about it, I'm the best poster here.

I'm just finishing up my first-ever programming class. I couldn't find anything specifically forbidding homework help requests...if I am doing wrong, I will delete this with my apologies.

Assume there is a variable , mp_affiliation , that is associated with a dictionary that maps the names of parliament members to party affiliations, associate with the variable party_size a dictionary that maps party names to the number of members they have.

Some of the test data MPL uses:
mp_affiliation={'Dodds, Mr Nigel':'Democratic Unionist', 'Davies, Dai':'Independent', 'Robinson, Mrs Iris':'Democratic Unionist', 'Bacon, Mr Richard':'Conservative', 'Binley, Mr Brian':'Conservative'}

The code I have so far (probably too many comments, but my instructor loves these)

pre:

#Initializing the party_size dictionary
party_size={}
#Looking at each entry in the mp_affiliation dictionary.
for n in mp_affiliation:
	#If the party isn't in the party_size dictionary yet, I want to add it as a key with a value of one.
	if n not in party_size:
		party_size[mp_affiliation[n]]=1
	#If it is, then I want to increase that value by one. I'm pretty sure this is the part that doesn't work.
	else:
		party_size[mp_affiliation[n]]=(party_size[mp_affiliation[n]]+1)

My current code's output:
 party_size={'Independent': 1, 'Conservative':1, 'Democratic Unionist': 1}

edit: I'm sure you're all capable of the math, but to save you a few seconds the values for "Conservative" and "Democratic Unionist" should be 2.

Coin fucked around with this message at 06:13 on Dec 6, 2014

# ? Dec 6, 2014 06:03

Hughmoris: Apr 21, 2007; Let's go to the abyss!

What is an elegant way to rewrite a file as I go through it? I am searching a text file line by line and if I find a pattern match, I want to re-write that entire line. Right now, my script works by writing all the contents to a new file, then deleting the old file and renaming the new file to the old file name. Exhausting, I know. But it works. Any suggestions on how I should improve it?

code:

from os import remove, rename

perm_file = 'C:/Python27/Scripts/data/testdata.txt'
temp_file = 'C:/Python27/Scripts/data/new_testdata.txt'

f1 = open(perm_file, 'r')
f2 = open(temp_file, 'w')
for line in f1:
    if 'OIS=53212^^^' in line:
	    f2.write('EDIT,LOAD_ORDER,,,,"@LOAD_ORDER=OIS=12345^^^OOS=0^^^"\n')
    else:
	    f2.write(line)
f1.close()
f2.close()

remove(perm_file)
rename(temp_file, perm_file)

# ? Dec 6, 2014 06:15

suffix: Jul 27, 2013; Wheeee!

Coin posted:

I'm just finishing up my first-ever programming class. I couldn't find anything specifically forbidding homework help requests...if I am doing wrong, I will delete this with my apologies.

Assume there is a variable , mp_affiliation , that is associated with a dictionary that maps the names of parliament members to party affiliations, associate with the variable party_size a dictionary that maps party names to the number of members they have.

Some of the test data MPL uses:
mp_affiliation={'Dodds, Mr Nigel':'Democratic Unionist', 'Davies, Dai':'Independent', 'Robinson, Mrs Iris':'Democratic Unionist', 'Bacon, Mr Richard':'Conservative', 'Binley, Mr Brian':'Conservative'}

The code I have so far (probably too many comments, but my instructor loves these)
pre:
#Initializing the party_size dictionary
party_size={}
#Looking at each entry in the mp_affiliation dictionary.
for n in mp_affiliation:
	#If the party isn't in the party_size dictionary yet, I want to add it as a key with a value of one.
	if n not in party_size:
		party_size[mp_affiliation[n]]=1
	#If it is, then I want to increase that value by one. I'm pretty sure this is the part that doesn't work.
	else:
		party_size[mp_affiliation[n]]=(party_size[mp_affiliation[n]]+1)

My current code's output:
 party_size={'Independent': 1, 'Conservative':1, 'Democratic Unionist': 1}
edit: I'm sure you're all capable of the math, but to save you a few seconds the values for "Conservative" and "Democratic Unionist" should be 2.

Some general tips:

Instead of "a = a + 1" you can often use the in-place operator: "a += 1"

If you need both the key and the value when iterating over a dictionary, you can use the .items() method: "for name, party in mp_affiliation.items():"

If you're using PyCharm (which is a great Python IDE and has a free community edition), you can step through your code line for line in the debugger and check that it is working as you expect.

# ? Dec 6, 2014 07:28

Edison was a dick: Apr 3, 2010; direct current only

Hughmoris posted:

What is an elegant way to rewrite a file as I go through it? I am searching a text file line by line and if I find a pattern match, I want to re-write that entire line. Right now, my script works by writing all the contents to a new file, then deleting the old file and renaming the new file to the old file name. Exhausting, I know. But it works. Any suggestions on how I should improve it?

You could edit it in-place, but you may wish you didn't, as if your new line is longer or shorter than the old one, you need to rewrite everything after it too. Plus it can easily get corrupted by concurrent access.

With the "write new copy" then "rename into place" one of the writes will be successful.

If you have a posix compliant filesystem, you can rename a file on top of another, so you don't ever have a missing file there.

quote:

code:

from os import remove, rename

perm_file = 'C:/Python27/Scripts/data/testdata.txt'
temp_file = 'C:/Python27/Scripts/data/new_testdata.txt'

f1 = open(perm_file, 'r')
f2 = open(temp_file, 'w')
for line in f1:
    if 'OIS=53212^^^' in line:
	    f2.write('EDIT,LOAD_ORDER,,,,"@LOAD_ORDER=OIS=12345^^^OOS=0^^^"\n')
    else:
	    f2.write(line)
f1.close()
f2.close()

remove(perm_file)
rename(temp_file, perm_file)

If you want to do this with a bit less code, you can use the fileinput module from the standard library.

code:

import fileinput, sys
perm_file = 'C:/Python27/Scripts/data/testdata.txt'

for line in fileinput.input(files=[perm_file], inplace=True):
    if 'OIS=53212^^^' in line:
	    sys.stdout.write('EDIT,LOAD_ORDER,,,,"@LOAD_ORDER=OIS=12345^^^OOS=0^^^"\n')
    else:
	    sys.stdout.write(line)

fileinput is old and crufty, and full of global state, so avoid using it as part of a much larger program, but it's useful for code golf in small scripts.

It also loses the nice property of having a fully written file in place at all times, since it does the rename at the start of processing, rather than after processing, but if you don't anticipate concurrent access then it's probably ok.

# ? Dec 6, 2014 13:15

Dominoes: Sep 20, 2007

Coin posted:

I'm just finishing up my first-ever programming class. I couldn't find anything specifically forbidding homework help requests...if I am doing wrong, I will delete this with my apologies.

Assume there is a variable , mp_affiliation , that is associated with a dictionary that maps the names of parliament members to party affiliations, associate with the variable party_size a dictionary that maps party names to the number of members they have.

Some of the test data MPL uses:
mp_affiliation={'Dodds, Mr Nigel':'Democratic Unionist', 'Davies, Dai':'Independent', 'Robinson, Mrs Iris':'Democratic Unionist', 'Bacon, Mr Richard':'Conservative', 'Binley, Mr Brian':'Conservative'}

The code I have so far (probably too many comments, but my instructor loves these)
pre:
#Initializing the party_size dictionary
party_size={}
#Looking at each entry in the mp_affiliation dictionary.
for n in mp_affiliation:
	#If the party isn't in the party_size dictionary yet, I want to add it as a key with a value of one.
	if n not in party_size:
		party_size[mp_affiliation[n]]=1
	#If it is, then I want to increase that value by one. I'm pretty sure this is the part that doesn't work.
	else:
		party_size[mp_affiliation[n]]=(party_size[mp_affiliation[n]]+1)

My current code's output:
 party_size={'Independent': 1, 'Conservative':1, 'Democratic Unionist': 1}
edit: I'm sure you're all capable of the math, but to save you a few seconds the values for "Conservative" and "Democratic Unionist" should be 2.

This should do it.

Python code:

parties = mp_affiliation.values()
party_size = {party: list(parties).count(party) for party in set(parties)}


Out: {'Conservative': 2, 'Democratic Unionist': 2, 'Independent': 1}

The parties = assignment makes the code easier to read. The party_size = line is a dictionary comprehension, which is a way of making new dictionaries from simple rules. count() returns the number of times the item in parenthesis shows up in the list before it's dot. If you're using Python 2, you can skip the list() step, because .values() returns a list in 2, instead of an iterator, which count() doesn't work on. set() removes duplicates from a list, so you'd only look at each party once, instead of once per member.

The problem with your code is that you're looping through ministers instead of parties. If you replace the original for loop with for n in mp_affiliation.values():, and every instance of [mp_affiliation[n]] with [n], your original code will work.

Note: you can use the [code=python] tag instead of [pre] to get some syntax highlighting in your posts.

Dominoes fucked around with this message at 14:04 on Dec 6, 2014

# ? Dec 6, 2014 13:36

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

You should at least take the repetitive list creation out of the comprehension.

# ? Dec 6, 2014 14:42

Dominoes: Sep 20, 2007

What do you mean?

# ? Dec 6, 2014 14:47

EAT THE EGGS RICOLA: May 29, 2008

Coin posted:

I'm just finishing up my first-ever programming class. I couldn't find anything specifically forbidding homework help requests...if I am doing wrong, I will delete this with my apologies.

Assume there is a variable , mp_affiliation , that is associated with a dictionary that maps the names of parliament members to party affiliations, associate with the variable party_size a dictionary that maps party names to the number of members they have.

Some of the test data MPL uses:
mp_affiliation={'Dodds, Mr Nigel':'Democratic Unionist', 'Davies, Dai':'Independent', 'Robinson, Mrs Iris':'Democratic Unionist', 'Bacon, Mr Richard':'Conservative', 'Binley, Mr Brian':'Conservative'}

The code I have so far (probably too many comments, but my instructor loves these)
pre:
#Initializing the party_size dictionary
party_size={}
#Looking at each entry in the mp_affiliation dictionary.
for n in mp_affiliation:
	#If the party isn't in the party_size dictionary yet, I want to add it as a key with a value of one.
	if n not in party_size:
		party_size[mp_affiliation[n]]=1
	#If it is, then I want to increase that value by one. I'm pretty sure this is the part that doesn't work.
	else:
		party_size[mp_affiliation[n]]=(party_size[mp_affiliation[n]]+1)

My current code's output:
 party_size={'Independent': 1, 'Conservative':1, 'Democratic Unionist': 1}
edit: I'm sure you're all capable of the math, but to save you a few seconds the values for "Conservative" and "Democratic Unionist" should be 2.

Aag those comments.

# ? Dec 6, 2014 14:50

SurgicalOntologist: Jun 17, 2004

Python code:

from collections import Counter

party_size = Counter(mp_affiliation.values())

Cheating?

# ? Dec 6, 2014 17:09

Dominoes: Sep 20, 2007

SurgicalOntologist posted:

Python code:

from collections import Counter

party_size = Counter(mp_affiliation.values())

Cheating?

That's awesome. Not cheating, not sorcery; python.

Going to have to look through that module. functools and itertools are neat too.

# ? Dec 6, 2014 17:17

QuarkJets: Sep 8, 2008

Dominoes posted:

What do you mean?

parties is already a list, so list(parties) in the dictionary comprehension doesn't do anything

# ? Dec 6, 2014 20:01

Dominoes: Sep 20, 2007

QuarkJets posted:

parties is already a list, so list(parties) in the dictionary comprehension doesn't do anything

It's a dict_values view object. Correction: dict_view objects are not iterators as I described:

Python code:

In [13]: next(my_dict.values())
Out[13]: TypeError: 'dict_values' object is not an iterator

You'd have to call iter() on them.

vv yea that works better.

Dominoes fucked around with this message at 20:19 on Dec 6, 2014

# ? Dec 6, 2014 20:10

Telarra: Oct 9, 2012

QuarkJets posted:

parties is already a list, so list(parties) in the dictionary comprehension doesn't do anything

Dominoes posted:

If you're using Python 2, you can skip the list() step, because .values() returns a list in 2, instead of an iterator, which count() doesn't work on.

But it still has a problem in Python 3 in that it recreates the list once for every party. Just do it once on the first line:

Python code:

parties = list(mp_affiliation.values())

# ? Dec 6, 2014 20:11

Coin: Jan 9, 2006; I'm no shitposter; I always know how I'm posting is wrong. I'm just a guy that doesn't like reading the thread, effortposting, and respecting the mods. So if you think about it, I'm the best poster here.

EAT THE EGGS RICOLA posted:

Aag those comments.

Are they bad for reasons other than being unnecessary?

Dominoes posted:

The problem with your code is that you're looping through ministers instead of parties. If you replace the original for loop with for n in mp_affiliation.values():, and every instance of [mp_affiliation[n]] with [n], your original code will work.

I did this and it worked (unsurprisingly) Thanks!

SurgicalOntologist posted:

Python code:

from collections import Counter

party_size = Counter(mp_affiliation.values())

Cheating?

I guess not technically, but I feel like correcting my code and sticking with my original algorithm is probably better academic practice. I'll remember that for a personal project/pipe dream I have planned for break, though!

# ? Dec 7, 2014 01:20

Telarra: Oct 9, 2012

Coin posted:

Are they bad for reasons other than being unnecessary?

Comments that state what code does are bad. They're unnecessary, they can be misleading when you change the code but forget to update the comments, and they clutter the code with redundancy. Aim instead to write code that is itself readable (Python's PEP 8 provides a helpful guide here) and use comments to clarify intent when it's not clear from the code itself.

# ? Dec 7, 2014 01:49

Coin: Jan 9, 2006; I'm no shitposter; I always know how I'm posting is wrong. I'm just a guy that doesn't like reading the thread, effortposting, and respecting the mods. So if you think about it, I'm the best poster here.

Moddington posted:

Comments that state what code does are bad. They're unnecessary, they can be misleading when you change the code but forget to update the comments, and they clutter the code with redundancy. Aim instead to write code that is itself readable (Python's PEP 8 provides a helpful guide here) and use comments to clarify intent when it's not clear from the code itself.

Yeah, I've heard this before. However, our instructor teaches us to "code our comments": basically we write everything out in plain English before actually coding it. I can see why this might not be helpful in a production environment, but I'd be marked down for not having that stuff.

# ? Dec 7, 2014 01:57

Telarra: Oct 9, 2012

It does at least makes some sense in an academic setting, especially when you're learning how to read the code in the first place.

# ? Dec 7, 2014 02:09

Dominoes: Sep 20, 2007

Coin posted:

I did this and it worked (unsurprisingly) Thanks!

Btw, put a space on each end of you = signs, per PEP 8.

# ? Dec 7, 2014 15:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

hosed this post up, wrong thread

# ? Dec 7, 2014 17:32

Adbot: ADBOT LOVES YOU

# ? Jun 2, 2024 22:59

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Edison was a dick posted:

You could edit it in-place, but you may wish you didn't, as if your new line is longer or shorter than the old one, you need to rewrite everything after it too. Plus it can easily get corrupted by concurrent access.

With the "write new copy" then "rename into place" one of the writes will be successful.

If you have a posix compliant filesystem, you can rename a file on top of another, so you don't ever have a missing file there.

If you want to do this with a bit less code, you can use the fileinput module from the standard library.
code:
import fileinput, sys
perm_file = 'C:/Python27/Scripts/data/testdata.txt'

for line in fileinput.input(files=[perm_file], inplace=True):
    if 'OIS=53212^^^' in line:
	    sys.stdout.write('EDIT,LOAD_ORDER,,,,"@LOAD_ORDER=OIS=12345^^^OOS=0^^^"\n')
    else:
	    sys.stdout.write(line)
fileinput is old and crufty, and full of global state, so avoid using it as part of a much larger program, but it's useful for code golf in small scripts.

It also loses the nice property of having a fully written file in place at all times, since it does the rename at the start of processing, rather than after processing, but if you don't anticipate concurrent access then it's probably ok.

Thanks for this. I'm not knowledgeable enough about the filesystem to say how it would behave with concurrent access. I think I'll stick with what I have since its functional, if ugly.

# ? Dec 8, 2014 01:31

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »