Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

BigRedDot: Mar 6, 2008

OK, it's only been a few week since 0.4, but for those interested, we released Bokeh 0.4.1. For a point release it has some pretty awesome additions: improved matplotlib compat layer, categorical axes and plots, flexible hover tool, and preliminary Scala bindings. Not bad for a point release.

Lot's of great new kinds of plots you can make.

These are all from a really huge tutorial I just made to present at PyData London. They aren't in the docs yet but they will be very soon!

Coming in the next release is some abstract rendering and dynamic downsampling. There's a branch that can currently interactively explore GBs of data and it should scale easily to hundreds of GBs. As always anyone interested in finding a project to contribute to, we are always interested in new contributors. Check out https://github.com/ContinuumIO/bokeh

# ? Feb 21, 2014 11:05

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 23:03

My Rhythmic Crotch: Jan 13, 2011

^ Bokeh is looking pretty awesome, and may find great use at work for me. We have some ancient java servlet that renders some horrific graphs, and I'd love to replace it.

Does anyone have any experience with Python LDAP? I'm looking for a way to send the password already encrypted. I know LDAPS can use LDAP over SSL, and that's great, but what I want is the ability to accept user credentials without my application ever seeing someone's password in plaintext. Even using LDAPS would allow my application to see a password in plaintext before it's sent out over SSL.

# ? Feb 21, 2014 20:20

Pollyanna: Mar 5, 2005; Milk's on them.

I'm trying to implement a linked list, and I'm running into a problem. I had an issue where taking the next node, then trying to find the previous of that node would give me a NoneType error, and I tried to fix that. Unfortunately, I can't tell if it works because Python hangs for (as far as I can tell) forever if I add the fix in. Why does my attempted fix cause this, and what's even going on?

I commented out the lines I added.

Python code:

class Node:
	prev = None
	next = None
	data = 0

	def __init__(self, input):
		self.data = input

	def append_to_tail(self, input):
		end_node = Node(input)
		current_node = self
		while current_node.next != None:
			# current_node.prev = current_node
			current_node = current_node.next
		current_node.next = end_node

	def append_to_head(self, input):
		start_node = Node(input)
		current_node = self
		while current_node.prev != None:
			# current_node.next = current_node
			current_node = current_node.prev
		current_node.prev = start_node

	def find_tail(self):
		current_node = self
		while current_node.next != None:
			current_node = current_node.next
		return current_node

	def find_head(self):
		current_node = self
		while current_node.prev != None:
			current_node = current_node.prev
		return current_node

def ll_test():
	test_node = Node(10)
	for i in range(20, 60, 10):
		test_node.append_to_tail(i)
	for i in range(20, 60, 10):
		test_node.append_to_head(i)

	assert test_node.data == 10, 'Initial node data does not equal 10'
	assert test_node.next.data == 20, 'Next node data does not equal 20'
	assert test_node.find_tail().data == 50, 'Tail node data does not equal 50'
	assert test_node.find_head().data == 50, 'Head node data does not equal 10'
	assert test_node.prev.prev.prev.prev.data == test_node.find_head().data, 'Head node data is inconsistent'

	print test_node.prev.next.data


ll_test()

BRD, I'm still working through the tutorial! Entirely cause I've been busy and distracted, sorry...

# ? Feb 21, 2014 23:24

accipter: Sep 12, 2003

Pollyanna posted:

Python code:

class Node:
	prev = None
	next = None
	data = 0

	def __init__(self, input):
		self.data = input

	def append_to_tail(self, input):
		end_node = Node(input)
		current_node = self
		while current_node.next != None:
			# current_node.prev = current_node
			current_node = current_node.next
		current_node.next = end_node

	def append_to_head(self, input):
		start_node = Node(input)
		current_node = self
		while current_node.prev != None:
			# current_node.next = current_node
			current_node = current_node.prev
		current_node.prev = start_node

	def find_tail(self):
		current_node = self
		while current_node.next != None:
			current_node = current_node.next
		return current_node

	def find_head(self):
		current_node = self
		while current_node.prev != None:
			current_node = current_node.prev
		return current_node

def ll_test():
	test_node = Node(10)
	for i in range(20, 60, 10):
		test_node.append_to_tail(i)
	for i in range(20, 60, 10):
		test_node.append_to_head(i)

	assert test_node.data == 10, 'Initial node data does not equal 10'
	assert test_node.next.data == 20, 'Next node data does not equal 20'
	assert test_node.find_tail().data == 50, 'Tail node data does not equal 50'
	assert test_node.find_head().data == 50, 'Head node data does not equal 10'
	assert test_node.prev.prev.prev.prev.data == test_node.find_head().data, 'Head node data is inconsistent'

	print test_node.prev.next.data


ll_test()

BRD, I'm still working through the tutorial! Entirely cause I've been busy and distracted, sorry...

The problem is because you aren't forming bi-directional connections.

Python code:

In [2]: n1 = Node(1)
In [3]: n2 = Node(2)
In [4]: n1.append_to_tail(n2)
In [5]: n1.prev

In [6]: n1.next
Out[6]: <__main__.Node at 0x5843210>
In [7]: n2.prev

In [8]: n2.next

Edit: Also, you want prev/next to be instance variables.

I haven't fully tested this, but it seems like it is working. Let me know if you have any questions.

Python code:

class Node:
    def __init__(self, data, prev=None, next=None):
        self.data = data
        self._prev = prev
        self._next = next

    # Use properties to update prev/next when they get changed
    @property
    def prev(self):
        return self._prev

    @prev.setter
    def prev(self, node):
        # Only updated if need to prevent circular references
        if self._prev != node:
            self._prev = node
            node.next = self

    @property
    def next(self):
        return self._next

    @next.setter
    def next(self, node):
        # Only updated if need to prevent circular references
        if self._next != node:
            self._next = node
            node.prev = self

    def append_to_tail(self, data):
        self.find_tail().next = Node(data)

    def append_to_head(self, data):
        self.find_head().prev = Node(data)

    def find_tail(self):
        current_node = self
        while current_node.next != None:
            current_node = current_node.next

        return current_node

    def find_head(self):
        current_node = self
        while current_node.prev != None:
            current_node = current_node.prev

        return current_node

n1 = Node(1)
n2 = Node(2)
n3 = Node(3)

n1.next = n2
n2.next = n3
print(n2.prev.data, n2.data, n2.next.data)

node = Node(0)
for i in range(3):
    node.append_to_tail(i + 1)

print(node.data, node.next.data, node.next.next.prev.data)

accipter fucked around with this message at 00:15 on Feb 22, 2014

# ? Feb 21, 2014 23:47

emoji: Jun 4, 2004

BigRedDot posted:

OK, it's only been a few week since 0.4, but for those interested, we released Bokeh 0.4.1. For a point release it has some pretty awesome additions: improved matplotlib compat layer, categorical axes and plots, flexible hover tool, and preliminary Scala bindings. Not bad for a point release. Lot's of great new kinds of plots you can make.

These are all from a really huge tutorial I just made to present at PyData London. They aren't in the docs yet but they will be very soon!

Coming in the next release is some abstract rendering and dynamic downsampling. There's a branch that can currently interactively explore GBs of data and it should scale easily to hundreds of GBs. As always anyone interested in finding a project to contribute to, we are always interested in new contributors. Check out https://github.com/ContinuumIO/bokeh

Please tell Ilan that the 64-bit Linux Anaconda installer fails like 10-15% of the time (RHEL-like, different machines). This is for 1.8 and 1.9. There will be a random decompression error during extraction of any of the packages. IMO if it fails it should clean up the directory it created.

# ? Feb 22, 2014 00:47

BigRedDot: Mar 6, 2008

kraftwerk singles posted:

Please tell Ilan that the 64-bit Linux Anaconda installer fails like 10-15% of the time (RHEL-like, different machines). This is for 1.8 and 1.9. There will be a random decompression error during extraction of any of the packages. IMO if it fails it should clean up the directory it created.

I can certainly pass this on but if you send it to anaconda@continuum.io then you could be kept in the discussion, and also provide any additional diagnostic info.

# ? Feb 22, 2014 03:17

Dominoes: Sep 20, 2007

How can I get timezone-aware time objects? Per the docs, they exist: "For applications requiring aware objects, datetime and time objects have an optional time zone information attribute, tzinfo"

However, I can find fuckall about them. Everything's about datetime objects.

pytz.timezone('Europe/Athens').localize(dt.time(9,30)) results in TypeError: unsupported operand type(s) for +: 'datetime.time' and 'datetime.timedelta'

I'm trying to create time objects that are the UTC time when a stock market opens and closes. Their local time zones are fixed, but flex with DST. Ie I set a local timezone object of 0930, localized to EST. I then find and use its UTC time.

This appears to get the proper time, but as a naieve time object:

Python code:

MARKET_OPEN = dt.datetime.combine(dt.date.today(), dt.time(9, 30))
MARKET_OPEN = pytz.timezone('US/Eastern').localize(MARKET_OPEN)
MARKET_OPEN = MARKET_OPEN.astimezone(pytz.UTC).time()

Dominoes fucked around with this message at 00:14 on Feb 23, 2014

# ? Feb 22, 2014 23:55

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

Try arrow.

# ? Feb 23, 2014 05:28

Luigi Thirty: Apr 30, 2006; Emergency confection port.

Python's built-in support for timezone-aware time objects sucks. Use arrow instead.

# ? Feb 23, 2014 08:34

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

So there's this module for generating random fake test data that I really like using in test automation, called faker (though on pypi it's fake-factory).

Anyway, I'm transitioning all of the test automation at my work to Robot Framework, and RF has a keyword-based test DSL you use to write tests, but you can import Python as keywords. I wanted to import faker, but it wouldn't work because faker is like a few layers of generators, and RF does introspection to find keywords within files and classes. So, I decided to write a wrapper, but it turned out a little trickier than I thought. My solution works really, really well (RF's auto-documentation generator actually finds all the "keywords" I've pulled out of faker, including their docstrings and args). I normally hate doing tricky magical poo poo involving hacking the internals of classes, but in this case I think it's warranted and fairly well encapsulated.

If anyone wants to tell me how I could have done this better, take a look:

Python code:

import sys

import faker

"""

This is a very thin wrapper for faker. You can access all of faker's usual methods
via FakerLibrary calls in Robot Framework.

"""

# create our faker
_fake = faker.Factory.create()


class FakerKeywords(object):
    ROBOT_LIBRARY_SCOPE = 'Global'


# set all of the faker's public methods to be our methods
for method_name, method in _fake.__dict__.items():
    try:
        if not method_name[0] == '_':
            setattr(sys.modules[__name__].FakerKeywords, method_name, method)
    except (IndexError, AttributeError):
        import traceback
        traceback.print_exc()

https://github.com/guykisel/robotframework-faker/blob/master/robotframework-faker/FakerLibrary/keywords.py

Also this is my first time packaging Python and actually uploading it to pypi. It didn't go very smoothly at first.

Also, if anyone is still reading this, I feel like this is a fairly generic solution, and therefore might be worth either trying to turn into a generic package for helping people wrap stuff for Robot Framework, or maybe even a patch for the Robot Framework core for quick and dirty imports of arbitrary modules.

BeefofAges fucked around with this message at 16:53 on Feb 23, 2014

# ? Feb 23, 2014 09:06

Dominoes: Sep 20, 2007

BeefofAges posted:

Try arrow.

Luigi Thirty posted:

Python's built-in support for timezone-aware time objects sucks. Use arrow instead.

Looks solid. I replaced most of my datetime objects with arrow objects. Much cleaner syntax.

Per the arrow docs, Arrow is a "Fully implemented, drop-in replacement for datetime" that aims to fix problems including "Too many types: date, time, datetime, tzinfo, timedelta, relativedelta, etc."

That said, I can't figure out how to replace this code with arrow, as it doesn't have a timedelta eqivolent I've found:

Python code:

raw_time = end - start
return raw_time - dt.timedelta(days=days_closed)

There appear to be few-no resources on arrow outside its official page.

# ? Feb 23, 2014 15:28

Crosscontaminant: Jan 18, 2007

The equivalent to that appears to be this.

Python code:

raw_time = end - start
return raw_time.replace(days=+days_closed)

I'm not sure, though - the documentation uses simplistic examples (arw.replace(hour=4, minute=40) and arw.replace(weeks=+3)) and it's not clear how it's supposed to know you used a unary plus/minus operator.

# ? Feb 23, 2014 16:53

Dominoes: Sep 20, 2007

Crosscontaminant posted:

The equivalent to that appears to be this.
Python code:
raw_time = end - start
return raw_time.replace(days=+days_closed)
I'm not sure, though - the documentation uses simplistic examples (arw.replace(hour=4, minute=40) and arw.replace(weeks=+3)) and it's not clear how it's supposed to know you used a unary plus/minus operator.

code:

AttributeError: 'datetime.timedelta' object has no attribute 'replace'

Subtracting arrows yields a datetime.timedelta object, not an arrow equivalent.

# ? Feb 23, 2014 17:15

Crosscontaminant: Jan 18, 2007

Oh, I guess I completely misunderstood what you were doing.

What manner of beast is raw_time when using the datetime module?

# ? Feb 23, 2014 18:09

Dominoes: Sep 20, 2007

Crosscontaminant posted:

Oh, I guess I completely misunderstood what you were doing.

What manner of beast is raw_time when using the datetime module?

datetime.datetime

# ? Feb 23, 2014 18:14

onionradish: Jul 6, 2006; That's spicy.

I recently split a single script into separate files with grouped related functions so the project would be easier to manage.

After I split the script, there were some global constants defined in the parent script that the imported scripts could no longer see. I was able to move the constants to the appropriate files, or just add them as passed function parameters -- so Python may be helping enforce good coding practices -- but it got me curious to understand the scope of variables across imported files better.

As an example, if I wanted a "parent" script to set global constants or config that imported scripts might use, like the path to a master directory, is there an appropriate way to access those parent variables from an imported script? Is that what __init__.py is for, or is the idea bad practice in general, and an indication that they should be passed parameters?

# ? Feb 23, 2014 21:32

QuarkJets: Sep 8, 2008

For constant parameters used in more than one place I create a py file that just defines those values, and then whenever I need those constants they're just an import away.

# ? Feb 23, 2014 22:40

onionradish: Jul 6, 2006; That's spicy.

QuarkJets posted:

For constant parameters used in more than one place I create a py file that just defines those values, and then whenever I need those constants they're just an import away.

Holy crap... that's such an obvious solution! Thanks!

# ? Feb 23, 2014 23:12

Dominoes: Sep 20, 2007

QuarkJets posted:

For constant parameters used in more than one place I create a py file that just defines those values, and then whenever I need those constants they're just an import away.

Same. I also apply this to functions shared between multiple files.

Dominoes fucked around with this message at 07:22 on Feb 24, 2014

# ? Feb 24, 2014 06:53

semicolonsrock: Aug 26, 2009; chugga chugga chugga

Question: where is a good, scrape-able database of things like stock values, shorts as a % of float, etc?

# ? Feb 24, 2014 19:53

Dominoes: Sep 20, 2007

semicolonsrock posted:

Question: where is a good, scrape-able database of things like stock values, shorts as a % of float, etc?

Yahoo Finance, or a broker API like Tradeking's.

# ? Feb 24, 2014 20:13

KICK BAMA KICK: Mar 2, 2009

PyCharm (CE): changing "from package.module import Class" to "from package import module" and changing all references to "Class" to "module.Class" -- I figured they had a thing for this but I'm not seeing it under Refactor. There's a context-sensitive lightbulb that offers to change it to "import package.module" and handle references accordingly but not the exact intermediate step I want.

# ? Feb 24, 2014 20:20

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Is there an easy way to see which packages are depending on package <x> in a virtualenv?

# ? Feb 24, 2014 22:23

accipter: Sep 12, 2003

fletcher posted:

Is there an easy way to see which packages are depending on package <x> in a virtualenv?

You could try this:

code:

pip install --no-install <PACKAGE NAME HERE>

# ? Feb 24, 2014 23:37

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

accipter posted:

You could try this:
code:
pip install --no-install <PACKAGE NAME HERE>

That seems to work but only going the other direction:

code:

$ pip install --no-install Fabric==1.8.1
...
Requirement already satisfied (use --upgrade to upgrade): paramiko>=1.10.0 in /home/fletcher/.virtualenvs/tmp_env/lib/python2.7/site-packages (from Fabric==1.8.1)
...

But let's say I didn't know why paramiko got installed and I wanted to find out. Fabric doesn't get mentioned in the output (which makes sense):

code:

$ pip install --no-install paramiko==1.10.0 | grep -i fabric

# ? Feb 25, 2014 00:04

Pie Colony: Dec 8, 2006; I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

KICK BAMA KICK posted:

PyCharm (CE): changing "from package.module import Class" to "from package import module" and changing all references to "Class" to "module.Class" -- I figured they had a thing for this but I'm not seeing it under Refactor. There's a context-sensitive lightbulb that offers to change it to "import package.module" and handle references accordingly but not the exact intermediate step I want.

why not just do something like sed 's/from package\.module import .*/from package import module/g' file.py | sed 's/Class/module\.Class/g'

there's probably some fancier way but i'm no sed expert and i would just do that

Pie Colony fucked around with this message at 03:36 on Feb 25, 2014

# ? Feb 25, 2014 03:30

KICK BAMA KICK: Mar 2, 2009

Pie Colony posted:

why not just do something like sed 's/from package\.module import .*/from package import module/g' file.py | sed 's/Class/module\.Class/g'

there's probably some fancier way but i'm no sed expert and i would just do that

Thanks, I'll keep that in mind if I ever need it. I was mostly asking cause PyCharm seems to have thought of everything and this was the one case I couldn't find a feature I was looking for. File that prompted the question wasn't big anyway.

# ? Feb 25, 2014 03:44

SurgicalOntologist: Jun 17, 2004

I'm annoyed by Pycharm's inspection of example code in docstrings (detected by >>>). For example, I have this docstring segment:

Python code:

    """
    Examples
    --------
    Assuming an `Experiment` named ``exp`` with levels ``['participant', 'session', 'block', 'trial']``:

    >>>some_block = exp.section(participant=2, session=1, block=3)
    """

Pycharm highlights exp and gives a "no reference found" warning. Now I understand that the proper convention would be to actually instantiate exp instead of describing it in a sentence, but in this case that would require 3-4 lines of code and it would be unwieldy to put that in every docstring example.

So- am I doing it wrong? If so, any suggestions?

If not, is there a way to turn this inspection off? I can't find it. It wasn't a big deal until I made a change and wanted to check that it didn't produce any extra warnings. Since my file is littered with these warnings I can't tell quickly.

# ? Feb 25, 2014 18:04

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

SurgicalOntologist posted:

I'm annoyed by Pycharm's inspection of example code in docstrings (detected by >>>). For example, I have this docstring segment:
Python code:
    """
    Examples
    --------
    Assuming an `Experiment` named ``exp`` with levels ``['participant', 'session', 'block', 'trial']``:

    >>>some_block = exp.section(participant=2, session=1, block=3)
    """
Pycharm highlights exp and gives a "no reference found" warning. Now I understand that the proper convention would be to actually instantiate exp instead of describing it in a sentence, but in this case that would require 3-4 lines of code and it would be unwieldy to put that in every docstring example.

So- am I doing it wrong? If so, any suggestions?

If not, is there a way to turn this inspection off? I can't find it. It wasn't a big deal until I made a change and wanted to check that it didn't produce any extra warnings. Since my file is littered with these warnings I can't tell quickly.

Is that using the latest version of PyCharm? I do see several bug reports that sound like what you are describing. This one was closed as a wontfix - is your docstring the first statement in the function?

# ? Feb 25, 2014 18:15

SurgicalOntologist: Jun 17, 2004

Yes, my docstring really is the docstring, it's completely different from that guy's mistake (thinking triple-quoted strings in any arbitrary place are comments).

And what I'm talking about is not really a bug in any case--that is, Pycharm is correct to note that exp is an unresolved reference. I would just prefer it not to check for that particular warning in docstrings. I can turn off that warning globally, but that would be overkill.

# ? Feb 25, 2014 18:57

Razzled: Feb 3, 2011; MY HARLEY IS COOL

Anyone know if there is a way to "explain" an object? Basically, an easy to way to see what functions and attributes are callable for a particular object? Without having documentation?

# ? Feb 25, 2014 20:44

qntm: Jun 17, 2009

There's dir().

# ? Feb 25, 2014 20:48

Dren: Jan 5, 2001; Pillbug

qntm posted:

There's dir().

And help()

# ? Feb 26, 2014 01:12

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

SurgicalOntologist posted:

Yes, my docstring really is the docstring, it's completely different from that guy's mistake (thinking triple-quoted strings in any arbitrary place are comments).

And what I'm talking about is not really a bug in any case--that is, Pycharm is correct to note that exp is an unresolved reference. I would just prefer it not to check for that particular warning in docstrings. I can turn off that warning globally, but that would be overkill.

It's probably assuming you put the >>> in for a doctest.

# ? Feb 26, 2014 05:59

Lurchington: Jan 2, 2003; Forums Dragoon

BeefofAges posted:

It's probably assuming you put the >>> in for a doctest.

yeah, this is why. Removing the >>> or marking "plain" docstrings in integrated tools seems to get you to where you want to go. For what it's worth, if there was a global thing named "exp" in scope of the function this is for, it'd also work

# ? Feb 26, 2014 06:10

SurgicalOntologist: Jun 17, 2004

Yes, I understand why Pycharm is doing this. I'd get rid of the prompt but I'm using Sphinx so I do want it formatted as a code example even though I'm not using doctest.

Interesting idea about the global scope. Probably not worth it but good to know for temporarily getting rid of the warnings in order to scan for other ones.

# ? Feb 26, 2014 06:26

BeefofAges: Jun 5, 2004; Cry 'Havoc!', and let slip the cows of war.

I think you can disable it under Settings | Project Settings | Inspections

# ? Feb 26, 2014 07:17

SurgicalOntologist: Jun 17, 2004

I can't find it there. I can disable the "unresolved reference" inspection entirely, but I don't see a way to disable doctest inspections.

# ? Feb 26, 2014 08:59

Dren: Jan 5, 2001; Pillbug

How many lines would it be for an example with no unresolved references? Maybe you should write a full, functional example.

# ? Feb 26, 2014 16:22

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 23:03

SurgicalOntologist: Jun 17, 2004

Yeah I probably should. In this case I can cram it onto 2-3 lines, but in some other cases three lines could fix the unresolved reference, but the example wouldn't make sense. For example, a resume method only makes sense if you've previously started running. But running is accomplished from a shell command. And you can't run without setting a callback, that's like 4 more lines just to define an empty function and pass it. That's just one example. For the case I posted maybe I should write a full doctest, but I don't think I can do that across the board.

# ? Feb 26, 2014 16:40

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »