Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

Proteus Jones: Feb 28, 2013

Fergus Mac Roich posted:

So you're finding the smallest difference between any two values in the list, right? As viking_helmet was getting at, your algorithm needs to minimize the number of times it needs to "touch" its elements. Now, your new solution is better than your old one, but step back from it and think: what if you could just know where the num2 value that's most similar to num1 is in the list? There's actually a way to do it, and it's right under your nose. I'll just stick the answer in spoiler tags, because I know I didn't find the fact that this method is faster to be intuitive at first, but you might be smarter than me so make sure to think about it for a bit before looking.

So if your original algorithm takes time equivalent to N^2, most practical sorting algorithms are closer to N * log N, which is way, way faster for a large N. So much faster that you can just call the list's sort() method, THEN loop through the entire list, checking each value against only the next value in the list. Be careful you don't run off the end of the list.

I think this is probably a good place to say, you can always check out what you have at your disposal in the objects you're using. Do that in the console using dir(object).

So for list type you would do dir(list) and see that sort is one of the list methods. help(list.sort) will pull up the docstring

# ¿ Sep 3, 2015 01:25

Adbot: ADBOT LOVES YOU

# ¿ Apr 27, 2024 08:48

Proteus Jones: Feb 28, 2013

Gothmog1065 posted:

Thanks guys! I was overthinking it pretty badly, and after reading flosofl and fergus's hints (I had to cheat), I figured it out pretty drat easily after that. Ended up with this, though I"m sure there's always the way to make it even better (I could probably write it into the first for loop thinking about it just reversing the math signs a bit to save even more time), but this finally got it to work:
Python code:
horses.sort()
for i in range(N):
	if i != N-1:
		diff = abs(horses[i] - horses[i+1])
		if diff < smallest or smallest < 0:
			smallest = diff
I will look into that because I'm loving terrible at word problems and math, always have been.

This means I've done all the "basic" stuff at codingame except for the skynet one, which I can't wrap my loving head around for some reason, but that's a different problem for a different day.

This is more of a style thing, but I hate having more conditionals than I really need. I'd probably redo it like this.

Python code:

horses.sort()
for i in range(len(horses) - 1):
	diff = horses[i+1] - horses[i]  # it's sorted in ascending order, why do more work than you need to?
	if diff < smallest or smallest < 0:
		smallest = diff

Proteus Jones fucked around with this message at 02:14 on Sep 3, 2015

# ¿ Sep 3, 2015 02:11

Proteus Jones: Feb 28, 2013

Hughmoris posted:

Any recommended resources for learning OO programming, in Python 3? I've been getting by using basic flow control and conditional statements, and I feel like my development has stalled.

If you're willing to spend money, I like Team Treehouse. ($25/mo. or $199/yr) You get access to all the training, so not just Python.

Even though I considered myself "intermediate" with the automation I'd do I thought I'd give it a spin for the Data Science course they had. Turns out my poo poo worked out sheer stubbornness. I was a monkey banging the ground with a wrench. I ended up doing their whole Learn Python and I definitely learned I had Been Doing It Wrong. In fact it made crystal clear subjects that had confused and frightened me before like Classes, Methods and Inheritance (I typically wrote brute-force, bulldozer code before).

I'm in a much better place programming wise and more confident with my knowledge. In fact with the solid foundation, I find it much easier get up to speed on newer topics.

I definitely wouldn't call them advanced courses or really applicable to someone with a really solid mastery in Python, but they're definitely good for the beginning to intermediate programmer.

Be aware it is Python 3 they use.

I'm a fan and I'm going to be starting the Swift track when I wrap up a project I'm eyeball deep in at work.

# ¿ Sep 5, 2015 23:44

Proteus Jones: Feb 28, 2013

hooah posted:

Any suggestions for poorer/cheaper folk? I've done a little codeingame, but it doesn't really require any particular quality of code. I've considered translating some of my C++ projects from school; would this be a decent way to get more familiar with Python's classes?

There's this:
http://www.diveintopython3.net

or this:
http://www.python-course.eu/python3_course.php

Gothmog1065 posted:

Is there another site as good as stack overflow for programming questions? Our work content filter is set up to block access to it right now as "Social Media". Gonna talk to the boss about seeing if networking will allow an exemption to it but it's annoying as poo poo to try to learn crap on my downtime.

I'm going to lean toward no. Whenever I google for a coding issue or clarification on an run error usually the 1st page of results is almost all Stack Overflow. The quality of the results taper off sharply after the Stack Overflow.

Get them to allow Stack Exchange sites in general ASAP.

Proteus Jones fucked around with this message at 21:16 on Sep 7, 2015

# ¿ Sep 7, 2015 21:09

Proteus Jones: Feb 28, 2013

Let's talk VCS.

Here's my situation. It's just me and all my projects are local. They're mostly network automation to do same task or tasks thousands of times given a csv file or files of system information. I develop my utilities on a rMBP using PyCharm and I'm using anaconda (python 3.4.3).

As background I've just taken a look at one of the first network utilities I've made and I've decided to do a major update to it to make it more flexible and easier on the eyes. What I have been doing is create a new project directory, copying over the old files, and opening the new directory in PyCharm.

PyCharm itself tracks a very tiny amount of changes, which has served me OK so far, but if I wander down a wrong path for too long, I realized I have no real recourse other than doing a "poor mans" VCS. i.e. Each time I do a change (other than fixing syntax) I put a version number on the file. This is getting out of control for me.

So... what's the recommended method? GitHub is out for now, since I don't want to pay to have "private" projects and there's some stuff that could get me fired if I put it in a public one. But I do have git installed from when I was looking at GitHub. I can also probably get mercurial or others on my box via homebrew if necessary.

As an addendum to that question, are there any good tutorials on getting up to speed on VCS? I've never used any before as I'm not really a programmer, but I've started writing more and more Python as part of an automation push that's basically my 20% project at work.

# ¿ Sep 8, 2015 03:41

Proteus Jones: Feb 28, 2013

Fergus Mac Roich posted:

You don't need GitHub, you can use Git locally. It sounds like it would pretty much cover your needs with a simple local repository. With a single user, it's surprisingly easy to get everything set up. If you want remote code hosting, try Bitbucket.org, which is free for up to five users for a private repo.

You can get started here:
https://git-scm.com/book/en/v1/Git-Basics-Getting-a-Git-Repository

Thanks!

# ¿ Sep 8, 2015 05:17

Proteus Jones: Feb 28, 2013

baka kaba posted:

PyCharm has Git integration so you get a nice UI and tools right in the IDE

https://www.jetbrains.com/pycharm/help/using-git-integration.html

It doesn't handle everything (I've had to crack open the command line to stop it tracking a previously tracked file) but for your general day-to-day version control it's great

I already got it up and running with PyCharm. It's pretty drat pain-free so far.

I use time machine so poo poo is backed up, but they all live on a directory mapped to a cloud drive, so I can offload having to worry about both my machine and my local backup being destroyed by catastrophe. Of course, if some happens at that level, I'm sure my utility scripts will be the last thing on my mind.

Proteus Jones fucked around with this message at 22:37 on Sep 8, 2015

# ¿ Sep 8, 2015 22:33

Proteus Jones: Feb 28, 2013

Gothmog1065 posted:

Do I have to do something more than defining it outside the for loop (Adding it the line above?). That doesn't seem to be fixing anything (even setting the variable outside the while loop), nor does putting the removal into the for loop (which doesn't really do what I want).

print(disconnect) directly above that line and it outputs [(x,y)], print(disconnect[0]) gives just (x,y) (Which seems to be what I want), but link_list.remove(disconnect[0]) throws an out of range area. Am I moving the wrong direction?

Put break points in and step through the code to where the value goes wonky. I'd probably set a break point every time disconnect should receive a value and validate it's an expected value. Also stop right before everything blows up and look at what things are vs. what you expect.

Also, I'd probably do a

code:
disconnect = []
before entering the loop.

Actually, I'd probably put all my list initializations at the start of the function.

But definitely run it through a debugger.

Sorry I misread, you're removing the value of disconnect from link_list.

What you're doing is trying to remove a list of tuples from link_list instead of the tuple itself. That will break because link_list[x] is not equal to disconnect. Disconnect is a list of one tuple, so you need to give disconnect the index of that tuple. In this case: [0]

try:

code:

link_list.remove(disconnect[0])

That remove a specific tuple from a list of tuples.

Proteus Jones fucked around with this message at 23:23 on Sep 8, 2015

# ¿ Sep 8, 2015 22:50

Proteus Jones: Feb 28, 2013

Gothmog1065 posted:

STUFF

I thought I was going cross-eyed, but I think I found your error.

First off I was wrong. The remove is definitely:

Python code:

    link_list.remove(disconnect)
    print(str(disconnect[0]) + " " + str(disconnect[1]))

Secondly, you reversed the elements in the tuple when assigning them to disconnect, so remove method will *never* match the tuple.

Here's the working if, elif, else:

Python code:

    for links in link_list:
        #print(links,file=sys.stderr)
        print(links[0],links[1],enemy,file=sys.stderr)
        if (links[0] == enemy) and (links[1] in gateways):
            disconnect = ( links[1], enemy)
            print("One", disconnect, file=sys.stderr)
            break
        elif (links[1] == enemy) and (links[0] in gateways):
            disconnect = ( links[0], enemy)
            print("Two", disconnect, file = sys.stderr)
            break
        else:
            disconnect = link_list[-1:]
            print("Three",disconnect, file=sys.stderr)

# ¿ Sep 9, 2015 04:28

Proteus Jones: Feb 28, 2013

How do I get 3.5.0 in anaconda. I mean I see it there, taunting me, when I do a 'conda search python', but 3.4.3 is the "latest" version when I do a 'conda update python'.

I've just started using anaconda, and I'm still fumbling around in it.

# ¿ Sep 14, 2015 00:14

Proteus Jones: Feb 28, 2013

Dominoes posted:

Wait.

Fair enough. I just wish it wouldn't list it as a result in the conda search like it could actually be installed.

# ¿ Sep 14, 2015 00:26

Proteus Jones: Feb 28, 2013

Dominoes posted:

http://continuum.io/blog/python35

-To use Python 3.5 in Anaconda, use a conda environment:
code:
conda create -n py35 python=3.5
-3.5 will be included in an official Anaconda release at the end of the October.

Kick rear end, thanks. I think setting up an environment is the better way to go anyway, until I'm sure I want 3.5 as the default.

# ¿ Sep 14, 2015 21:35

Proteus Jones: Feb 28, 2013

Emacs Headroom posted:

As my analysis / modeling evolves, I end up moving code from inside the notebook to a library in Python that gets imported into the notebook (and re-used by other notebooks). The library is the time to add unit tests, write good docstrings, etc.

If you're doing data engineering in industry, the library can also be a good reference to base your streaming / hadoop / spark / whatever version on as well.

I don't use notebooks (really I do different stuff with my programs), but I've found creating libraries has been very useful to me, since there have been some classes I've used over and over again in different projects. I've found that while it adds a little more time to planning and a bit more effort programming and documenting when creating it for the first time, it's saved me oh so much time as opposed to cutting and pasting code and hammering it to fit.

**kwargs are my bestest friends.

# ¿ Sep 24, 2015 16:31

Proteus Jones: Feb 28, 2013

OK, I'm running into an issue I'm just not getting. pexpect is dumping out on me because of a weird error regarding the timeout value. It dumps out every time at 'i = device.expect(self.password_prompt, self.en_prompt)' with the following error:

code:

  File "/Users/philf/anaconda/lib/python3.4/site-packages/pexpect/__init__.py", line 1514, in expect_loop
    end_time = time.time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'

The thing is, when I put a break point right before that line, the device object has timeout = {int} 30 when I inspect it in the pycharm debugger. This is driving me crazy. I've tried manually setting

Python code:


    def check_ssh_logon(self, device, results, pull_results):
        ssh_logon_success = False
        ssh_logon_return = [SSH_FIRST_TIME,
                            self.password_prompt,
                            pexpect.TIMEOUT,
                            pexpect.EOF
                            ]
        ssh_attempt = device.expect(ssh_logon_return)
        if ssh_attempt == 0:
            device.sendline('yes')
            device.expect(self.password_prompt)
            device.sendline(self.password)
            ssh_logon_success = True
        elif ssh_attempt == 1:
            ssh_logon_success = True
            device.sendline(self.password)
        elif ssh_attempt == 2:
            ssh_logon_success = False
            print('!!!SSH TIMEOUT!!!\r\n')
            results.write('ssh,N,NETWORK TIMEOUT\r\n')
            pull_results.write('ssh,N,NETWORK TIMEOUT\r\n')
        elif ssh_attempt == 3:
            ssh_logon_success = False
            print('!!!SSH session refused!!!\r\n')
            results.write('ssh,N,SESSION NOT ALLOWED\r\n')
            pull_results.write('ssh,N,NETWORK TIMEOUT\r\n')
        return ssh_logon_success

    def ssh_try(self, results, pull_results):
        credential_error = '\r\n!!BAD CREDENTIALS(ssh)' \
                           '\r\n*****************\r\n\r\n'
        success_string = '\r\n *** Success (ssh) ***\r\n\r\n'
        device = pexpect.spawnu('ssh ' + self.username + '@' + self.ip)
        ssh_success = self.check_ssh_logon(device, results, pull_results)
        if ssh_success:
            i = device.expect(self.password_prompt, self.en_prompt)
            if i == 0:
                print(credential_error)
                results.write('ssh,N,BAD CREDENTIALS\r\n')
                pull_results.write('ssh,N,BAD CREDENTIALS\r\n')
                device.terminate()
                return
            elif i == 1:
                print(success_string)
                self.process_commands(results, pull_results, device)
        else:
            device.terminate()
            return

EDIT: NVM. As soon as I hit submit, I figured it out. I forgot to make the device.expect into a list.

Python code:

i = device.expect([self.password_prompt, self.en_prompt])

# ¿ Sep 26, 2015 21:43

Proteus Jones: Feb 28, 2013

Munkeymon posted:

QJ was arguing against sharing code at all, not about how it's shared and that bugs me purely from a consumer of scientific output standpoint because it's a great way to accidentally hide errors that affect results.

But that's *not* what he's saying? Unless I completely missed something elsewhere, he even mentioned having the code available for review to make sure the methodologies are appropriate and correct. He's just saying he doesn't see the point of a Notebook at all and it's sloppier than using a formally coded program.

# ¿ Sep 28, 2015 15:52

Proteus Jones: Feb 28, 2013

QuarkJets posted:

Sounds pretty straightforward to me. Check out docx2txt for dumping the raw text to an ascii format. From there it's all a matter of intelligently extracting the pieces that you want, but it sounds like you have some good ideas of how to do this. Once you've identified what everything is, dumping it to a CSV is extremely easy

re: CSV. There the builtin 'csv'. I use it all the time and it's very easy to use. Or there's always the option to write it out as a standard file just separating the elements with a ',' and manually adding a '\r\n' at the end of each row. But that seems like a lot of work compared to just using csv.

# ¿ Oct 3, 2015 08:10

Proteus Jones: Feb 28, 2013

Superdawg posted:

Makes sense. Thanks for the help.

I have made the adjustment and confirmed that the issue does not show anymore.

I've always used this to prune items from my list:

Python code:

>>> some_list = [1,2,5,3,4,5,6,5,7,8,5,9,10]
>>> print(some_list)
[1, 2, 5, 3, 4, 5, 6, 5, 7, 8, 5, 9, 10]
>>> some_list = [x for x in some_list if x != 5]
>>> print(some_list)
[1, 2, 3, 4, 6, 7, 8, 9, 10]

I've fallen into using that to get rid of duplicate values. If there's a better way to do this, I'd appreciate someone letting me know. Always looking to improve.

# ¿ Oct 4, 2015 21:32

Proteus Jones: Feb 28, 2013

Microsoft OneDrive dev team just released this today:

https://github.com/OneDrive/onedrive-sdk-python

# ¿ Oct 9, 2015 03:24

Proteus Jones: Feb 28, 2013

spiralbrain posted:

This helps. Thank you. I want the virtualenv to be used for Python 3.X (forget which version is loaded on my other laptop) so yeah, currently its not pointed at that and I could see it was giving me a path error for the Flask import when I tried running the basic Flask hello world app so I figured something in the virtualenv wasn't setup correctly. Thanks for that command.

I'm still not exactly sure how the virtual workspace works but it seems as though it makes it easier to switch between 2.7 and 3.x depending on which version you want to work with...also something with dependencies, but thats about all I know. Def need to read up more on this.

You may want to look into Anaconda Python. I really like conda for creating environments.

code:

conda create -n my_flask
source activate my_flask
conda install flask [additional package] [additional package]

This will create a conda environment with only the packages you need. Conda will figure out all the dependancies flask needs and install those as well. Another nice thing is I can keep the environment using the versions I need for my package dependancies. I'm still playing around with "conda build" so I can bundle up my environments and easily get them set up on the systems I need them.

Another nice thing is I can point PyCharm to the environment directory, and it will take it from there automatically using the conda environment on a project by project basis.

# ¿ Oct 15, 2015 04:05

Proteus Jones: Feb 28, 2013

Cingulate posted:

Is there a painless way of moving my primary (3.4) anaconda environment to 3.5? I have a huge bunch of conda packages installed, some from binstar.

won't conda update --all do it?

EDIT: No it won't. Sorry I thought they had released 3.5 and that's what you were asking.

Proteus Jones fucked around with this message at 02:06 on Oct 26, 2015

# ¿ Oct 26, 2015 02:04

Proteus Jones: Feb 28, 2013

https://www.continuum.io/blog/company-blog/anaconda-24-release

PyCharm 5 was released this week as well.

It's like Christmas

# ¿ Nov 4, 2015 16:31

Proteus Jones: Feb 28, 2013

QuarkJets posted:

I personally can't do it, because _ looks awful to me, but it's fine if others want to do that.

Same. And it always throws me for a second when I see it others' code.

# ¿ Apr 11, 2016 13:13

Proteus Jones: Feb 28, 2013

Loving Africa Chaps posted:

Just doing tabs but for some reason the code tags make it look weird

Here's the error:

Traceback (most recent call last):
File "test.py", line 12, in <module>
cursor2.execute(query, name)
File "/home/#/foambot/venv/lib/python2.7/site-packages/mysql/connector/cursor.py", line 515, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/home/#/foambot/venv/lib/python2.7/site-packages/mysql/connector/connection.py", line 488, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/home/#/foambot/venv/lib/python2.7/site-packages/mysql/connector/connection.py", line 395, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '%s' at line 1

Edit i should clarify that i'm using mysql.connector which didn't want to install via pip until i specified the version number so maybe that's got something to do with it?

Don't SQL queries end with a ';' ? It's been years since I've interacted with a database using SQL. Most of my python scripts just use flat files for input and results.

# ¿ May 5, 2016 18:41

Proteus Jones: Feb 28, 2013

The Gasmask posted:

Sorry, I didn't phrase it the best, but I figured it out. That part you quoted was incorrect of course, it's a variable regardless, but I just wasn't sure if I could reuse those shortened variable names for different blocks of code.

Technically yes. But in practice, it's not recommended.

# ¿ Aug 12, 2016 17:10

Proteus Jones: Feb 28, 2013

I'm in a weird situation where I'm developing a python script on one machine in a conda environment, but I need to run it on a Windows box. Normally, not a big deal since I have Anaconda installed and up to date on both.

However, this script relies on pexpect, which has some unix lib dependancies. So I'm limited to a Cygwin instance here. I tried to install the linux Anaconda, but it dumps out when it's trying to install conda. So my question is this:

pexpect is currently the only thing other std libs that I'm using for this and it's already installed in Cygwin via pip.. Python3 on Cygwin is 3.4.3. So far, I've only specified my conda envs with "python3". Can I specify a specific point release of python for my env? If not, what should I look out for in my source to make sure it will execute in the Cygwin environment?

# ¿ Sep 19, 2016 18:09

Proteus Jones: Feb 28, 2013

BigRedDot posted:

Yes, you can specify python versions like any other conda package:

code:


[bryan:...e-datavis-with-bokeh-python]$ conda create -n foo python=3.4.1                                 (master)
Fetching package metadata ...........
Solving package specifications: ..........

Package plan for installation in environment /Users/bryan/anaconda/envs/foo:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    setuptools-27.2.0          |           py34_0         528 KB

The following NEW packages will be INSTALLED:

    openssl:    1.0.1k-1
    pip:        8.1.2-py34_0
    python:     3.4.1-4
    readline:   6.2-2
    setuptools: 27.2.0-py34_0
    sqlite:     3.13.0-0
    tk:         8.5.18-0
    wheel:      0.29.0-py34_0
    xz:         5.0.5-1
    zlib:       1.2.8-3

Proceed ([y]/n)?

Thanks!

# ¿ Sep 19, 2016 21:39

Proteus Jones: Feb 28, 2013

Martytoof posted:

Man there are like 900 ways to install Py3 on OSX. I'm just going to brew install python3 and forget about it before I start to obsess about which is the best down the road. Never going to actually learn if I keep nitpicking my infrastructure :|

Anaconda will never let you down.

# ¿ Sep 21, 2016 12:59

Proteus Jones: Feb 28, 2013

accipter posted:

What are peoples thoughts on unicode variable names (e.g., Φ)? I do a lot of programming of equations that might have variables without physical meaning (no proper name) and are just a greek symbol. I will usually use an ascii representation of that variable because it is simpler to enter, but it is a little removed from the original reference.

If you're working in Python3 you should be fine, since source file encoding is UTF-8.

If you're using 2.x, I think you need to declare the encoding like this (not 100%, I had to look it up)

Python code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

However, it's on you to make sure your editor supports it and is not inserting extra crap.

Proteus Jones fucked around with this message at 18:52 on Feb 1, 2017

# ¿ Feb 1, 2017 18:50

Proteus Jones: Feb 28, 2013

Eela6 posted:

As of yesterday, Anaconda released version 4.3 - which has Python 3.6 as a default! Now is a great time to start using Python 3.6.

Yay!

# ¿ Feb 2, 2017 04:59

Adbot: ADBOT LOVES YOU

# ¿ Apr 27, 2024 08:48

Proteus Jones: Feb 28, 2013

baka kaba posted:

I've been using Visual Studio Code a bunch for things lately, might be worth putting that in the IDEs section since it's the hotness and people might already be using it? They have a Python page and that Code Runner extension is real nice too

It's nice and I use it for quick and dirty edits when I don't want to load PyCharm, or I'm on a computer I don't have access to PyCharm.

I would probably not call it an IDE (especially for macOS). It's really more of a language aware editor.

# ¿ Mar 6, 2017 13:30

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.