Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Emmideer: Oct 20, 2011; Lovely night, no?; Grimey Drawer

Can dict.fromkeys() accept different values for different keys through some method? Ultimately I'm trying to get each key to have the same list as a value, but I want each list to be unique, such that can be edited separately according to which key I'm using.

# ? Jun 25, 2016 00:00

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 16:32

OnceIWasAnOstrich: Jul 22, 2006

jon joe posted:

Can dict.fromkeys() accept different values for different keys through some method? Ultimately I'm trying to get each key to have the same list as a value, but I want each list to be unique, such that can be edited separately according to which key I'm using.

Not fromkeys() but :

Python code:

d = {k: list(l) for k in keys}

# ? Jun 25, 2016 00:25

Emmideer: Oct 20, 2011; Lovely night, no?; Grimey Drawer

OnceIWasAnOstrich posted:

Not fromkeys() but :
Python code:
d = {k: list(l) for k in keys}

Woah I wish I knew this earlier. So much easier than the route I was going.

edit: Can someone tell me the difference between this written as a function (first example) vs written explicitly (second example), such that test_dict_5["Bob"]["round_1"] points to a single round_1 shared by all players, but test_dict_6["Bob"]["round_1"] is a separate round_1 like I want?

code:

##first example
def diff_dict(dy_el, st_el_a="", st_el_p="", insert_value=None):
	dict_pass = {indiv_key : insert_value for indiv_key in rename_elements(dy_el, st_el_a, st_el_p)}
	return dict_pass

test_dict_5 = diff_dict(player_list, "", "", round_names())
test_dict_5["Bob"]["round_1"] = "Rock"

##second example
test_dict_6 = {indiv_key : round_names() for indiv_key in rename_elements(player_list, "", "")}
test_dict_6["Bob"]["round_1"] = "Rock"

editedit: Best I can tell, the round_names() function (which just creates a dictionary of empty keys, one key for each round) is evaluating in the diff_dict function only once, at the time it's inserted, prior to it being used in the dict_pass. How do I prevent this unintended behavior?

Triple edit: Here's my lovely work-around that works how I want:

code:

def diff_dict(dy_el, st_el_a="", st_el_p="", insert_value=lambda : None)):
	dict_pass = {indiv_key : insert_value() for indiv_key in rename_elements(dy_el, st_el_a, st_el_p)}
	return dict_pass

test_dict_5 = diff_dict(player_list, "", "", round_names)
test_dict_5["Bob"]["round_1"] = "Rock"

It means I can only insert callables into insert_value, but it works! Not that I would want to insert non-callables anyway, as they have the same behavior as pre-evaluating as above, anyway. The only other thing I can think to do is use copying, but that seems like taking a hammer to a pillow party. Any other suggestions, or am I stuck in callables-only?

Emmideer fucked around with this message at 06:50 on Jun 25, 2016

# ? Jun 25, 2016 00:48

Dominoes: Sep 20, 2007

Looking for advice on ODE. Let's say I'm trying to model the first-order differential equation dy/dx = y. Manual code using euler's method, ie a good approximation of e**x:

Python code:

def euler(f, dx, x_init, y_init, end):
    x, y = x_init, y_init

    for x in np.arange(x_init, end, dx):
        yield x, y
        dy_dx = f(x, y)
        y+= dy_dx * dx

f = lambda x, y: y
list(euler(f, 1, 0, 1, 10))

[(0, 1),
 (1, 2),
 (2, 4),
 (3, 8),
 (4, 16),
 (5, 32),
 (6, 64),
 (7, 128),
 (8, 256),
 (9, 512)]

Odeint result:

Python code:

f = lambda x, y: y
scipy.integrate.odeint(f, y0=1, t=np.arange(0, 10))
 
array([[  1.        ],
       [  1.50000001],
       [  3.00000001],
       [  5.50000001],
       [  9.00000001],
       [ 13.50000001],
       [ 19.00000001],
       [ 25.50000001],
       [ 33.00000001],
       [ 41.50000001]])

Dominoes fucked around with this message at 00:44 on Jun 27, 2016

# ? Jun 27, 2016 00:36

SurgicalOntologist: Jun 17, 2004

The callable for odeint is supposed to have the signature func(y, t, *args), where y is the state vector and t is the time. Using y for state and t for time, you are actually integrating dy/dt = t (that is, the rate of change of y is the current time) but I think you want to be integrating dy/dt = y, at least if you expect to get e**x. So, your function should be lambda y, t: y. I think your mistake stems from using x and y rather than y and t (the roles of x and y being easier to mix up).

# ? Jun 27, 2016 01:10

SurgicalOntologist: Jun 17, 2004

Sorry for the doublepost, but I have my own question. I want to make a combination of collections.deque and asyncio.Queue. That is, I want a deque that I can do await deque.popleft with. To put it another way, I want a Queue that I can treat as a collection. Supposedly, deque is threadsafe and Queue is built on top of a deque, so is there an easier way than making my own class? If there is I can't figure it out.

Edit: maybe this is an XY problem? I have a data stream that writes to a Queue. I want to be able to keep some elements around, for example to always be able to see the previous element. Programming synchronously I'd use a deque but I need the consumer to wait without blocking until there's new data.

SurgicalOntologist fucked around with this message at 01:17 on Jun 27, 2016

# ? Jun 27, 2016 01:12

hooah: Feb 6, 2006; WTF?

I've found that there is evidently some difference between list() and [] when creating lists. For example, the following code doesn't work (a card is an object which knows its suit, rank, and value (value goes from 1-10), but nothing else):

Python code:

for card in player1_list:
	card_rank = card.rank()
	if card_rank in player1_dict:
		player1_dict[card_rank].append(card)
	else:
		player1_dict[card_rank] = list(card)

I get a TypeError 'Card' object is not iterable. However, the following code, which I thought would be equivalent, works fine:

Python code:

for card in player1_list:
	card_rank = card.rank()
	if card_rank in player1_dict:
		player1_dict[card_rank].append(card)
	else:
		player1_dict[card_rank] = [card]

Why is there a difference?

# ? Jun 27, 2016 01:57

KICK BAMA KICK: Mar 2, 2009

A working form of your first example would be, I think:

code:

player1_dict[card_rank] = list([card])

list() is a constructor which expects an iterable as its sole argument -- it takes something that is already an iterable and returns a list containing the elements of the iterable you gave it. It doesn't take a single item and return a one-element list containing that. The [element] literal syntax just creates a list out of whatever comma-separated elements are inside it, even if that's only a single element. They don't work the same cause they just don't have the same semantics.

# ? Jun 27, 2016 02:07

SurgicalOntologist: Jun 17, 2004

hooah posted:

I've found that there is evidently some difference between list() and [] when creating lists. For example, the following code doesn't work (a card is an object which knows its suit, rank, and value (value goes from 1-10), but nothing else):
Python code:
for card in player1_list:
	card_rank = card.rank()
	if card_rank in player1_dict:
		player1_dict[card_rank].append(card)
	else:
		player1_dict[card_rank] = list(card)
I get a TypeError 'Card' object is not iterable. However, the following code, which I thought would be equivalent, works fine:
Python code:
for card in player1_list:
	card_rank = card.rank()
	if card_rank in player1_dict:
		player1_dict[card_rank].append(card)
	else:
		player1_dict[card_rank] = [card]
Why is there a difference?

KICK BAMA KICK is right. But I thought I'd share something you might be interested in. There's an object designed for this situation: defaultdict. If you used defaultdict for player1_dict your code would be:

Python code:

from collections import defaultdict

player1_dict = defaultdict(list)

...

for card in player1_list:
    player1_dict[card.rank()].append(card)

That is, defaultdict(list) allows you to just append to an item of your dict without worrying if that item exists or not. If it doesn't, an empty list will be created.

# ? Jun 27, 2016 02:51

GameCube: Nov 21, 2006

Is there an accepted Good Way to do Python on OS X? I've screwed around with different combinations of homebrew, pyenv, and the official installers and I keep loving up my environment.

# ? Jun 27, 2016 02:53

SurgicalOntologist: Jun 17, 2004

Definitely conda.

# ? Jun 27, 2016 02:55

Dominoes: Sep 20, 2007

SurgicalOntologist posted:

The callable for odeint is supposed to have the signature func(y, t, *args), where y is the state vector and t is the time. Using y for state and t for time, you are actually integrating dy/dt = t (that is, the rate of change of y is the current time) but I think you want to be integrating dy/dt = y, at least if you expect to get e**x. So, your function should be lambda y, t: y. I think your mistake stems from using x and y rather than y and t (the roles of x and y being easier to mix up).

Thanks. Reversing the function argument order worked.

Dominoes fucked around with this message at 10:14 on Jun 27, 2016

# ? Jun 27, 2016 10:11

OnceIWasAnOstrich: Jul 22, 2006

SurgicalOntologist posted:

Definitely conda.

Conda is a good answer, although I'd have to give the advice that you shouldn't install anything in your default environment at all if you care about using homebrew or something like it. If you segregate Python into conda-only and then conda-environment only, then the only homebrew stuff you gently caress up is Python, which you wouldn't be using. If you start installing other stuff into your default conda environment then you start accumulating non-Python libraries and things in your default environment which will start to mess up Homebrew formulas (and brew doctor will warn you about this, and it is correct) when you get linking of homebrew builds to conda libraries.

The best solution is to us conda like a virtualenv-only environment if you are used to that. Install conda. Create new environments, and pretend you always have to source activate before you use Python. This has a ton of side benefits anyway, and no downsides other than the obvious.

# ? Jun 27, 2016 13:27

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I've been using virtual environments by default for everything for so long I get a little confused when helping someone out and they have stuff installed in or are trying to work in the default installation.

IMO, it's just a best practice.

# ? Jun 27, 2016 14:41

OnceIWasAnOstrich: Jul 22, 2006

Thermopyle posted:

IMO, it's just a best practice.

Definitely, but I've had to fix so many peoples completely busted Macs (and several HPC user profiles) by saving the installed packages list, wiping /usr/local and ~/anaconda and reinstalling everything with a conda environment that it's worth mentioning again and again.

# ? Jun 27, 2016 15:25

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Agreed.

# ? Jun 27, 2016 16:08

more like dICK: Feb 15, 2010; This is inevitable.

I still keep requests installed globally since its so handy in the repl

# ? Jun 27, 2016 17:40

OnceIWasAnOstrich: Jul 22, 2006

If you know what you are doing I'm pretty sure that any module that is pure Python and has no dependencies on native libraries will not pollute anything if you install it in your default environment. I know requests is pure Python but it has a long list of dependencies and I'm not sure if any of those depend on native libraries conda will install.

I don't have anything in my default environment but I've also aliased ipython to run from my "default" non-default environment (with full Anaconda distro and just about every package I've ever used) so I don't have to activate anything and quick REPL sessions are just as quick and I can have a whole mess of stuff ready by "default".

# ? Jun 27, 2016 17:51

SurgicalOntologist: Jun 17, 2004

Same. If I'm installing an "app" that's a PyPI package, I just make a new env, install it, and symlink the bin into ~/.local/bin. Similarly I have jupyter symlinked from my default non-default env.

That said, even if a package is pure Python, I wouldn't be pip installing things into your system Python. Especially if doing so requires sudo. On install, and on import, a package can run arbitrary code. Including transparently changing the language itself, which I shouldn't have to tell you can really screw up your system. It doesn't really matter if it's Python-only or not. It's one thing if you trust the package author, but even still there's nothing stopping someone from uploading a malicious pacakge as requestss and then you're one fat-finger away from causing serious damage.

Granted I've never heard of this actually causing a problem. But at the same time I've never heard of someone installing a fake OS on someone else's Chromebook for malicious purposes, and yet I continue to have the OS verified every boot-up.

SurgicalOntologist fucked around with this message at 18:01 on Jun 27, 2016

# ? Jun 27, 2016 17:58

BigRedDot: Mar 6, 2008

SurgicalOntologist posted:

Definitely conda.

I second this vote (but I'm also biased)

# ? Jun 27, 2016 22:30

Plasmafountain: Jun 17, 2008

I know its bad form, but I urgently need a debug since I cant make this equation match the example in the book.

I have an equation:

X4 = ((L2 - b) - SQRT((L2-b)^2 - 4*c*(a-Y2 + L2*X2))) / (2*c)

Its bugging the gently caress out of me since its a fairly straightforward, slightly-modified quadratic equation formula, but my answer doesnt match the result in the book.

code:




implemented in python as:

X2 = 60.48

Y2 = 59.625

a = 22.1852

b = 0.71568

c = 0.0010787

L2 = 1.2187

Xwall = ((lambda2-b)-(np.sqrt((lambda2-b)**2 - (4*c*(a-Y2+lambda2*X2))))/(2*c))

I should be getting Xwall = 63.46, but instead I'm getting anything but.

If anyone could tell me where I'm going wrong here I'd greatly appreciate it.

(PS I know lambda is a reserved term in python programming but its handy to keep track of equation terms in comparison to book formulae).

Plasmafountain fucked around with this message at 23:35 on Jun 27, 2016

# ? Jun 27, 2016 23:27

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

So is lambda2 == L2 here or what as you didn't define it in your code snippet? :confused:

# ? Jun 27, 2016 23:50

Plasmafountain: Jun 17, 2008

Yes, L2 = lambda2. = 1.2187

# ? Jun 27, 2016 23:54

Jose Cuervo: Aug 25, 2004

This

code:

Xwall = ((lambda2-b) - np.sqrt((lambda2-b)**2 - 4*c*(a-Y2+lambda2*X2))) / (2*c)

gets me Xwall = 89.1376 (I moved the last parentheses so that it closes the numerator).

# ? Jun 28, 2016 02:42

StormyDragon: Jun 3, 2011

Zero Gravitas posted:

I know its bad form, but I urgently need a debug since I cant make this equation match the example in the book.

I have an equation:

X4 = ((L2 - b) - SQRT((L2-b)^2 - 4*c*(a-Y2 + L2*X2))) / (2*c)

Zero Gravitas posted:

Python code:

Xwall = ((lambda2-b)-(np.sqrt((lambda2-b)**2 - (4*c*(a-Y2+lambda2*X2))))/(2*c))

The two equations do not match up with their parenthesis, so you are going to have to provide more information. A photo of the equation and the terms as written in the book.
You say this is a modified version of the quadratic formula, so perhaps it helps to put the terms into the actual formula as defined.

Python code:

a_ = c
b_ = L2 - b
c_ = a - Y2 + L2 * X2

Xlow = (-b_ - sqrt(b_**2 - 4*a_*c_))/(2*a_)
Xhi = (-b_ + sqrt(b_**2 - 4*a_*c_))/(2*a_)

print(Xlow, Xhi)
# -377.1829583283631 -89.13761272939158

Not that either result matches the expected one.

# ? Jun 28, 2016 02:48

Eela6: May 25, 2007; Shredded Hen

I've been reading Fluent Python by Luciano Ramalho and really enjoying it. The section on operator overloading has really let me get a lot more out of my custom classes (I.e, overloading __len__ to have len(foo) return something sensible) . Just wanted to give it a plug.

# ? Jun 28, 2016 03:05

Plasmafountain: Jun 17, 2008

Thanks for the attempts, gents - I was getting values of ~89, ~134 and some others depending on the exact position of parentheses, but nothing approaching the book value.

Ok, I've found a copy of the book on Scribd.

I'm trying to find X4 and Y4 in part E, lines k & l :

Theres a snippet of equivalent Fortran code a couple of pages earlier which form the same thing, here in the line marked 30:

Its this line that I thought I had transcribed into python.

If theres an alternate function for finding X4 and Y4 I'm all ears - I've previously used np.linalg.solve for a system of equations in terms of x and y, but im not sure how to formulate a solution array consisting of y, x and x squared terms.

Plasmafountain fucked around with this message at 09:16 on Jun 28, 2016

# ? Jun 28, 2016 08:39

Jose Cuervo: Aug 25, 2004

I believe that c = -0.0010787 from what you posted, and this should give you the right answer.

# ? Jun 28, 2016 15:56

Plasmafountain: Jun 17, 2008

Jose Cuervo posted:

I believe that c = -0.0010787 from what you posted, and this should give you the right answer.

It is, and it does, thankyou.:facepalm:

Is there a way of solving that kind of system to find x and y given x and x^2 terms?

Or will I need to repeat this process with the derivation and rearranging to isolate my term of interest?

# ? Jun 28, 2016 17:09

Jose Cuervo: Aug 25, 2004

Zero Gravitas posted:

It is, and it does, thankyou.:facepalm:

Is there a way of solving that kind of system to find x and y given x and x^2 terms?

Or will I need to repeat this process with the derivation and rearranging to isolate my term of interest?

I have no idea what kind of system you are talking about. I was only able to help because you posted the scan of the page and I realised you had not got the sign of c correct.

# ? Jun 28, 2016 17:19

Eela6: May 25, 2007; Shredded Hen

Zero Gravitas posted:

It is, and it does, thankyou.:facepalm:

Is there a way of solving that kind of system to find x and y given x and x^2 terms?

Or will I need to repeat this process with the derivation and rearranging to isolate my term of interest?

Hard to tell from the snippet of mathematics you've posted, but this looks like partial differential equations and they are pretty much always a nightmare.

# ? Jun 28, 2016 17:20

BigRedDot: Mar 6, 2008

Hey we did a thing: https://bokeh.github.io/blog/2016/6/28/release-0-12/

Now, excuse me while go collapse in a heap.

# ? Jun 28, 2016 19:42

Tigren: Oct 3, 2003

BigRedDot posted:

Hey we did a thing: https://bokeh.github.io/blog/2016/6/28/release-0-12/

Now, excuse me while go collapse in a heap.

Congrats BigRedDot, that's awesome. Bokeh is such a fun library.

# ? Jun 28, 2016 22:34

Baby Babbeh: Aug 2, 2005; It's hard to soar with the eagles when you work with Turkeys!!

Dumb newbie question about threading(which I feel I don't fully understand yet):

Say I have a thread that's continually monitoring a file for changes. When there's a change, I want it to take the thing that was changed and pass that data to another thread which does stuff with it and then keep monitoring. What's the best way to do this? is it Queue? Is this even a good idea or would it be better to just have the first thread do the stuff without involving another thread at all?

In this instance what's being done is taking a list of hashtags and querying the Twitter API with them. I have a vague sense that this would be faster if I created worker threads for each hashtag and had them make concurrent queries rather than executing them one by one, but maybe I'm just creating too many threads for no reason.

# ? Jun 29, 2016 00:01

SurgicalOntologist: Jun 17, 2004

Well, first I was going to share this: https://github.com/gorakhargosh/watchdog#example-api-usage

Then I read the rest of your post and realized that you're doing the same thing I am.

I'm monitoring tweets that match any of a number of hashtags and just saving them to a file. There are no docs yet (it's dissertation crunch time) but you can see the CLI here. It runs off a config folder containing two files: api.auth and hashtags.list. The former contains the three API keys (one per line, in the order tweepy requires them which I forget exactly) and the latter contains hashtags, one per line.

Instead of threading I just combine the hashtags into a giant OR term. There's also a unified API for searching and listening in case you want to look backwards as well.

If you want to use it, I'm happy to help. It is kind of rough around the edges, but I've had it going for a few weeks now without any hitches.

Back to your project though, I don't quite understand why you need to monitor a file for changes? Is the hashtag list changing often, or without a user around to restart the process?

# ? Jun 29, 2016 00:57

Baby Babbeh: Aug 2, 2005; It's hard to soar with the eagles when you work with Turkeys!!

Wow, that is remarkably similar to what I'm trying to do, down to using tweepy to interface with the Twitter API. Small world. I'll definitely take a look at how you're doing this.

The file monitor is part contrivance and part design -- I'm doing this as a learn-to-code-python-type project and I thought it'd be a good arbitrary challenge. What I'm trying to do is build a twitter bot that generates tweets for hashtags in a list, and this bit is for pulling tweets with those hashtags into training sets for a yet-undefined machine learning model. The idea is for the list to be updated regularly so that the hashtags stay current, like if it notices new hashtags in the mix when it pulls tweets it adds them to the list so that training sets can be generated from them. The monitor is so that the sets get created whenever new hashtags get added to the list, and so I can just open the file and throw new hashtags in when I run across them without having to muck around much with it.

I don't know if any of this is a good idea. Honestly I'm pretty over my head with all this, but the way I approach learning new skills is to get in over my head and then try to dig myself out.

# ? Jun 29, 2016 06:58

BigRedDot: Mar 6, 2008

Tigren posted:

Congrats BigRedDot, that's awesome. Bokeh is such a fun library.

Thanks, this made me smile

# ? Jun 29, 2016 14:46

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

I've got a question about modules in Python 2 vs Python 3. Namely, I've got this structure:

code:

 - server.py (flask app)
 - module_1/
    - __init__.py
    - code1.py
    - module_2/
         - __init__.py
         - code2.py
         - code3.py

Where code1 imports code2 through "from module_2.code2 import Class" and then code2 imports code3 through "import code3". I can then run any file in module_2 individually fine in Python 2 and Python 3, but then running code1 in python2 works but in python3 fails with message "No module named code3" in code2.py

Since I'm sure that explanation made no sense, the repo is here with failing travis build at https://travis-ci.org/RAIRLab/Talos (which is the first submodule)

# ? Jun 29, 2016 18:32

SurgicalOntologist: Jun 17, 2004

Baby Babbeh posted:

I don't know if any of this is a good idea. Honestly I'm pretty over my head with all this, but the way I approach learning new skills is to get in over my head and then try to dig myself out.

I took the same approach and I think it worked out well. Sounds like you're well on your way.

Baby Babbeh posted:

What I'm trying to do is build a twitter bot that generates tweets for hashtags in a list, and this bit is for pulling tweets with those hashtags into training sets for a yet-undefined machine learning model. The idea is for the list to be updated regularly so that the hashtags stay current, like if it notices new hashtags in the mix when it pulls tweets it adds them to the list so that training sets can be generated from them. The monitor is so that the sets get created whenever new hashtags get added to the list, and so I can just open the file and throw new hashtags in when I run across them without having to muck around much with it.

Sounds reasonable. One thread is updating the list of hashtags, and another thread is searching for tweets. The issue is you need to communicate between them, right? No need to bring the file system into it. Use a Queue to send a message whenever the hashtags change.

# ? Jun 29, 2016 19:07

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 16:32

SurgicalOntologist: Jun 17, 2004

Master_Odin posted:

I've got a question about modules in Python 2 vs Python 3. Namely, I've got this structure:
code:
 - server.py (flask app)
 - module_1/
    - __init__.py
    - code1.py
    - module_2/
         - __init__.py
         - code2.py
         - code3.py
Where code1 imports code2 through "from module_2.code2 import Class" and then code2 imports code3 through "import code3". I can then run any file in module_2 individually fine in Python 2 and Python 3, but then running code1 in python2 works but in python3 fails with message "No module named code3" in code2.py

Since I'm sure that explanation made no sense, the repo is here with failing travis build at https://travis-ci.org/RAIRLab/Talos (which is the first submodule)

First of all, modules are .py files. The things you're calling modules are packages.

I've never put together a flask app, but what do you mean "run any file individually?" Are you trying to have them all work when run as scripts?

Use fully qualified imports everywhere. Replace import code3 with from module_1.module_2 import code3. If you want to make shortcuts, import stuff in the __init__.py files. This should work for every use case but may require you to make a setup.py file and pip install the distribution.

# ? Jun 29, 2016 19:10

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »