Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›2 »

huhu: Feb 24, 2006

Jose Cuervo posted:

I wrote a small data processing script a while back and since then I have updated pandas. Now the script does not work. Short of creating a venv for a single script, is there any way of telling my system to use a specific version of pandas when running the script?

Why not just use a virtualenv? virtualenvwrapper makes it a lot easier to work with virtualenv. I imagine this isn't going to be the last time you're going to have version issues so I'd suggest just making a virtualenv and setup a requirements.txt.

# ¿ Nov 20, 2017 20:38

Adbot: ADBOT LOVES YOU

# ¿ May 16, 2024 20:01

huhu: Feb 24, 2006

Boris Galerkin posted:

Are there any tutorials out there that's mostly dealing with PyCharm and some of the more useful things it can do? I've been using it for a while now and it's pretty great, but I can't help but feel like I'm missing out on a lot of its functionality that I either don't know exists or don't really know how to use to full advantage.

I'd say let's just share here the cool features we know.

- Right Click on variable/filename/etc -> Refactor -> Rename - finds any instance of that variable and renames it. Even able to look inside strings
- Navigate -> File / Class - Just learned this this week. Open any file within your project. So much quicker than navigating the file structure
- Click on any part of the file path at the top of the screen and you can browse relative to the file you're currently viewing.
- View -> Tool Window -> Todo - Write TODO in any comment and you can keep track of all the things you need to do. Useful when wiring up a project initially.
- Debug - So glad I don't have to throw print statements everywhere now
- Integration with GIT is pretty cool. Especially color changes for lines/files within your project.

# ¿ Dec 10, 2017 21:33

huhu: Feb 24, 2006

outlier posted:

Python-adjacent question: Recommendations for a rich ui / internet application frontend to a Python-backed webapp?

No more constraints to give, because this is strictly in the early / idle speculation stage for a possible analysis / data management platform. But I'd like the keep the backend in Python and while there was a burst of RIA toolkits some year ago, it seems to have settled down.

Are you building it in Django or Flask or ... hopefully you're not creating your own? Django comes with templates (https://docs.djangoproject.com/en/2.0/topics/templates/) and Flask works well with Jinja 2(http://jinja.pocoo.org/docs/2.10/). If you're using Django Rest Framwork it pairs pretty nicely with React, and I'm guessing every other front end framework out there these days (I only know React).

# ¿ Dec 12, 2017 16:20

huhu: Feb 24, 2006

outlier posted:

I don't have any thoughts about the backend at the moment other than I'd like it to be a Python-based one (which I am not going to build myself). I figured the choice of frontend might constraint the backend. Will chase up Django REST.

Something like React + DRF is what I do for work and it's a pretty solid pairing. There's quite a lot you'll need to read up on though. At the very least probably React, Redux, Webpack, ES6, Axios, Django, and Django Rest Framework. If you don't need a fancy front end and your back end isn't complicated I'd suggest going the Flask + Jinja 2 route.

# ¿ Dec 12, 2017 21:26

huhu: Feb 24, 2006

JVNO posted:

In list logic, .remove will remove the first item in a given list that matches the query...

How do you delete the last item in a list that matches a particular query?

Because the best I can come up with is to reverse the list, apply the remove function, then reverse the list again. And that strikes me as terribly inefficient

You could iterate in reverse.

# ¿ Mar 2, 2018 00:26

huhu: Feb 24, 2006

I constantly finding myself going what is that path to the thing I need to import and I start guessing is it "Foo" or "Foo.SubFoo" or "Foo.Sub.Bar"? Is there some easy way with Python, I use Pycharm specifically, to not have to Google where to import what I need from?

# ¿ Mar 7, 2018 03:02

huhu: Feb 24, 2006

I don't know much about sorting but I do know things tend to run a lot faster when you're not printing a bunch of stuff out.

# ¿ Mar 18, 2018 01:47

huhu: Feb 24, 2006

Anyone have any recommendations for resources for learning MicroPython (with the ESP2866)? I see a few books on Amazon but none look promising and I can't really find good Youtube videos or other courses online.

# ¿ Mar 26, 2018 21:10

huhu: Feb 24, 2006

Todo apps are like the most popular way of learning backend. Google "flask/Django todo app tutorial".

# ¿ Apr 25, 2018 14:09

huhu: Feb 24, 2006

I don't really understand what's going on with your code, there are a few errors. I think what you're looking for though is something like (I'm phone posting)

code:


for p in ports: 
  is_port_closed = False
  while not is_port_closed:
    is_port_closed = closePort(p)
    # assuming is_port_closed returns true or false

You could also throw in a try/except in the while loop instead and toggle is_port_closed in the try block after the closePort function is called.

Errors in your code include a try without an except, you never use port variable, and an else without a for.

huhu fucked around with this message at 10:17 on Apr 27, 2018

# ¿ Apr 27, 2018 10:14

huhu: Feb 24, 2006

Anyone else observe PyCharm coming to a standstill for even the most basic of operations? I started a new project and was working in Sublime. I opened the same directory in Pycharm and it's taking me like ~1-2 seconds to open a directory to see what's inside. I don't have any other processes running within it and the Activity Monitor for mac is reporting up to 300% CPU usage.

# ¿ May 30, 2018 22:56

huhu: Feb 24, 2006

What the hell is happening here? I add a closing bracket and PyCharm just shits all over the formatting.

# ¿ Jun 9, 2018 16:39

huhu: Feb 24, 2006

CarForumPoster posted:

I have a dumb newbie question but I feel like I'm missing something obvious.

I have a function to make QRcodes:
code:
def genqr(url="defaulturl", fname="defaultfname"):
    img = qrcode.make(url)
    img.save(fname)
    print('Generating QR code from URL: ', url, " Saved to", fname)
I have some excel data to pass to it. My intuition is to load this in a dataframe with:
df = pd.read_excel("data.xlsx"), giving:
code:
                     urls filenames  number
0  [url]http://url.com/?n=1001[/url]  1001.jpg    1001
1  [url]http://url.com/?n=1002[/url]  1002.jpg    1002
2  [url]http://url.com/?n=1003[/url]  1003.jpg    1003
3  [url]http://url.com/?n=1004[/url]  1004.jpg    1004
I want to loop through the urls generating QR codes and saving them with those file names. I think I am missing something obvious. I thought df.apply(qrgen.genqr, axis=1, urls, filenames) or something like that might work.

What do I need to be doing?

I don't see a loop anywhere? I can't be sure because I've not used the functions you're discussing but I think you're assuming something is iterating when it's not.

You might want to do something like (phone posting):

code:


for row in data: 
  genQR(row)

huhu fucked around with this message at 00:59 on Jun 10, 2018

# ¿ Jun 10, 2018 00:56

huhu: Feb 24, 2006

Not trying to start some argument, genuinely curious... The constructs I use exclusively are for item in array, for item in enumerate (array), and for item in range. Are there any good uses of the basic for loop with i=0, i++?

# ¿ Jun 10, 2018 16:24

huhu: Feb 24, 2006

pmchem posted:

I need to mentor people of wildly different ages, skills and backgrounds on a language new to them (in this case, Python).

I know most of what I'm going to do, but I am looking for a free "coding test" style website for them to run exercises against. For privacy reasons I won't go into, I'd prefer if the site does not require a login (one strike against HackerRank). http://codingbat.com/python seems like what I want: it's lightweight, does the checking of answers on the fly in the browser, is free, and doesn't require a login. But on the downside, its examples are not very extensive.

Does anyone know a better resource / website for what I need?

Even better, perhaps, would be a set of ipython notebooks that could be run without an internet connection. So, they'd need to set up the problems, and have the answer available (but hidden), so the user could check that their code is producing the correct answer.

Project Euler is pretty good. Only need a username and password to keep track of progress. Not sure if that's quite what you're looking for. Questions increase in difficulty so people could start at different points.

# ¿ Jun 13, 2018 02:43

huhu: Feb 24, 2006

Sad Panda posted:

Is there a way to bulk import a whole subdirectory? I ask because I've got 150ish files in books/, import them all manually and then call them for example using that.
Python code:
    balance = eval(script_name).scrape_balance(driver)
One idea I had is to move all those import statements into their own file and then import that just to make my main file that much shorter.

Maybe I'm missing something but you could just use the os package and get all the files in a directory and then iterate over them.

# ¿ Jul 2, 2018 03:56

huhu: Feb 24, 2006

Jose posted:

Whats the best library for map data visualisations? Particularly if my datasets have country codes

Google Maps and Flask?

Edit: you could probably just write a script to spit out an HTML file.

# ¿ Jul 20, 2018 14:27

huhu: Feb 24, 2006

Could someone help me out with what I'm missing here? Just switched over two VSCode from Pycharm which used to handle all this kind of stuff for me. My issue is that I have a course I'm taking that has weekly assignments. I'd like to have a directory per week, and then a collection of utilities, such as timing functions to measure various solutions I come up with. Here is my directory structure.

code:

c1_algorithmic_toolbox/
__init__.py
    week1_programming_challenges/
        __init__.py
        maximum_pairwise_product.py
    utilities/
        timing.py
        __init__.py

timing.py consists of some code, which I don't think is relevant to my current problem

code:

def timing(method):
    def timed(*args, **kw):
        ts = time.time()
        result = method(*args, **kw)
        te = time.time()
        if 'log_time' in kw:
            name = kw.get('log_name', method.__name__.upper())
            kw['log_time'][name] = int((te - ts) * 1000)
        else:
            print '%r  %2.2f ms' % \
                  (method.__name__, (te - ts) * 1000)
        return result
    return timed

Inside of maximum_pairwise_product.py, when I try to run the file from the command line, I get:

code:

(3336) >> python week1_programming_challenges/maximum_pairwise_product.py
Traceback (most recent call last):
  File "week1_programming_challenges/maximum_pairwise_product.py", line 6, in <module>
    from utilities.timing import timing
ModuleNotFoundError: No module named 'utilities'

It seems to be getting tripped up on the line

code:

from utilities.timing import timing

# ¿ Sep 1, 2018 01:06

huhu: Feb 24, 2006

Dr Subterfuge posted:

pretty sure this would work
Python code:
from c1_algorithmic_toolbox.utilities.timing import timing

That results in:

code:

(3337) >> python week1_programming_challenges/maximum_pairwise_product.py
Traceback (most recent call last):
  File "week1_programming_challenges/maximum_pairwise_product.py", line 6, in <module>
    from c1_algorithmic_toolbox.utilities.timing import timing
ModuleNotFoundError: No module named 'c1_algorithmic_toolbox'

Edit: If I open this in Pycharm and do

code:

from utilities.timing import timing

It works. What magic is happening!?

huhu fucked around with this message at 02:02 on Sep 1, 2018

# ¿ Sep 1, 2018 01:56

huhu: Feb 24, 2006

Dr Subterfuge posted:

And I just realized the print statement isn't a function, so this is python 2. I know some import behavior changed between 2 and 3, but I don't know if that's relevant here.

That's why I tried to highlight that " some code, which I don't think is relevant to my current problem " because I couldn't even get anything to yell about it being Python 2 code.

# ¿ Sep 1, 2018 03:00

huhu: Feb 24, 2006

SurgicalOntologist posted:

Edit: Oh, you're using Python 3 but have some old Python 2 code you want to run? Well, fix the import problem first, and then yes, you'll have to update timing.py.

I'm leading a class at work because I'd like to learn new things. I might have spent too long as a React/Node developer that I forget the Python way of doing things or I'm just trying to force the JS way on Python. Eeeesh...

# ¿ Sep 1, 2018 03:41

huhu: Feb 24, 2006

Does anyone know of a good technique for extracting XMP Data from Photos? I used Lightroom to add a bunch of data to Photos and I'm trying to extract it all in a batch script for Django. Currently I'm using libxmp which doesn't seem to want to install on my hosting due to a bunch of non Python dependencies.

Edit: Ended up editing the library and hardcoding a path to the dependency.

huhu fucked around with this message at 23:23 on Oct 1, 2018

# ¿ Oct 1, 2018 21:49

huhu: Feb 24, 2006

CarForumPoster posted:

I have a dumb question, at some point in developing stuff does it become more natural to read and work with json? Like are there some of you who can deal with json as intuitively as you would data in a table/dataframe?

Having a way to view it can helpful. Something like this https://chrome.google.com/webstore/detail/json-formatter/bcjindcccaagfpapjjmafapmmgkkhgoa?hl=en or a plugin for your text editor make it easier to read. I like being able to collapse sections of a JSON doc to focus on what I need.

There's also not a native way to do this, but I really like JavaScript's destructuring and found a lambda function in Python that can do it that makes it easier to split apart huge json blobs into more manageable chunks.

code:

>>> import json
>>> json_string = '{"first_name": "Guido", "last_name":"Rossum"}'
>>> parsed_json = json.loads(json_string)
>>> pluck = lambda dict, *args: (dict[arg] for arg in args)
>>> first_name, last_name = pluck(parsed_json, 'first_name', 'last_name')
>>> first_name
'Guido'

huhu fucked around with this message at 20:03 on Nov 22, 2018

# ¿ Nov 22, 2018 19:57

huhu: Feb 24, 2006

code:

def map_color(value):
    gameboy = [
        np.array([15,56,15]), #'0f380f' Black
        np.array([48,246,35]), #'30f6230' Dark Gray
        np.array([139,172,15]), #'8bac0f' Light Gray
        np.array([155,188,15]) # '9bbc0f' White
    ] 
    return gameboy[value]

buckets = bucketColors(img, 3)

mod_bucket = []
for row in buckets:
    mod_row = []
    for cell in row:
        mod_row.append(map_color(cell))
    mod_bucket.append(np.array(mod_row))

I've got this code with numpy and cv2. bucketColors takes a photo and outputs 0,1,2 or 3 per pixel value into a new 2D array. I then want to map those 4 values to colors listed in map_color. Is there a way to not do the nested for loops here? I'd like to be able to perform this calculation for webcam streaming and nested for loops is too slow.

# ¿ Dec 17, 2019 15:05

huhu: Feb 24, 2006

DarthRoblox posted:

Yes - you're already using numpy, so sticking to numpy array functions will likely be your best bet. They're vectorized, so the entire operation happens at once instead of having to iterate cell by cell, row by row.

np.take() should do what you want. Here's a basic example, note the axis=0 argument or this won't work (it'll just take the first value from each) https://repl.it/repls/FrillyFragrantParameter

Thanks, this looks awesome! I continued to do research and came across doing it this way:

code:

    color_buckets = grayscale_buckets[:, :, np.newaxis] * np.ones((HEIGHT, WIDTH,3)).astype(np.uint8)

    color_buckets = np.where(color_buckets == [0,0,0], np.array([155,188,15]),color_buckets)
    color_buckets = np.where(color_buckets == [1,1,1], np.array([48,246,35]),color_buckets)
    color_buckets = np.where(color_buckets == [2,2,2], np.array([139,172,15]),color_buckets)
    color_buckets = np.where(color_buckets == [3,3,3], np.array([15,56,15]),color_buckets)
    color_buckets = color_buckets.astype(np.uint8)

I added some timers to my code and your solution got this:

code:

60 frames took 1.1479568481445312s, or 52.26677300368856 FPS
60 frames took 0.9086148738861084s, or 66.03457826238576 FPS
60 frames took 0.8385148048400879s, or 71.55508722525481 FPS
60 frames took 0.80253005027771s, or 74.76355555687593 FPS
60 frames took 0.7379393577575684s, or 81.30749413112548 FPS

Whereas mine that I listed above got:

code:

60 frames took 0.8082489967346191s, or 74.23454930646876 FPS
60 frames took 0.8121190071105957s, or 73.88079761052693 FPS
60 frames took 1.1538612842559814s, or 51.99931813180512 FPS
60 frames took 0.919954776763916s, or 65.22059726789976 FPS
60 frames took 1.0144267082214355s, or 59.14670770567174 FPS

I would have thought np.take would be faster than 4x np.where. But it doesn't seem to be the case. Thoughts?

# ¿ Dec 18, 2019 16:21

huhu: Feb 24, 2006

And a follow up question. I've got a script running that can take in video from the webcam and output it to the screen. How might I be able to turn that output into a virtual webcam so that I could use my code for Zoom or Google Hangouts? I'm not quite sure what to search for here or if it's even possible.

# ¿ Dec 18, 2019 16:23

huhu: Feb 24, 2006

I'm so lost. I've got a file uploader in Flask that's intermittently uploading the file. I can't tell if this is one bug or multiple.

I've got a form:

code:

        <form method="post" novalidate>
            {{ form.hidden_tag() }}
            <div id="form_elements_wrapper">
                <div>
                    {{ form.photo.label }}
                    {{ form.photo }}
                </div>

		<!-- more code here -->
        </form>

Which goes to a route like so:

code:

@app.route('/', methods=['GET', 'POST'])
def stitch(): 
    form = StitchForm(request.form, meta={'csrf': False}) 
    if form.validate_on_submit():
        stitched_image = stitch_image(
            image=form.photo.data,
        )
    # ... more code

And the form looks like:

code:

class StitchForm(FlaskForm):
    photo = FileField(
        'Photo',
        validators=[
            DataRequired('The field photo is required')
        ]
    )

When I finally get that to the stitch_image function:

code:

def stitch_image(image):
    img = cv2.imread(image)

img ends up being None.

Further super weird behavior - I have an image in a directory and I can upload it just fine. However if I copy and paste it then try to upload the pasted image, it doesn't work.

# ¿ Mar 10, 2020 22:28

huhu: Feb 24, 2006

Edit: I'm doing a bunch of dumb crap. Taking a break. Carrry on.

huhu fucked around with this message at 23:15 on Mar 10, 2020

# ¿ Mar 10, 2020 22:58

huhu: Feb 24, 2006

https://micropython.org/ ?

# ¿ Apr 1, 2020 00:51

huhu: Feb 24, 2006

I have two classes Plotter and Layer. A Plotter consists of an array of layers. Each layer is an array of instructions.

code:

class Plotter:
    def add_layer(self, name: str):
        self._layers[name] = Layer(self)

    def update_layer(self, name: str) -> Layer:
        return self._layers[name]

code:

class Layer:
    def add_line(self, x1, y1, x2, y2):
        self.add_pen_down_point(x1, y1)
        self.add_point(x2, y2)

I've removed a bunch of code for brevity.

If I want to update a layer, I currently do the following:

code:

plotter.update_layer(RED_LAYER).add_line(0, 0, 50, -50)

This feels bad, is there a more pythonic way?

And one more question. I've got a directory structure

code:

project
    main.py
    __init__.py
demos
    demo1.py
    __init__.py

I want to import project into demos. However, when I cd into the demos directory and try and run any variation of import statements, I get various errors. I've Googled a bunch and I cannot figure out what's wrong with my import. Or is there a better way to make a demos directory for a package I'm building? One of the stack overflow answers says

code:

If the directories aren't Python packages, you can do this by messing around with sys.path, but you shouldn't.

So I feel like I'm missing something here.

huhu fucked around with this message at 18:53 on Sep 25, 2023

# ¿ Sep 25, 2023 18:11

huhu: Feb 24, 2006

I added this gross code:

code:

sys.path.append(os.path.abspath('../project'))
from Project import Project

Which now works except VSCode now says `Import "Project" could not be resolvedPylance` but the code still excutes.

I swear, between installing a virtual environment and getting imports, I end up just abandoning Python if my code grows out of a single file with basic imports.

# ¿ Sep 25, 2023 19:42

huhu: Feb 24, 2006

Zoracle Zed posted:

You'll probably want to follow a guide like this. This will allow you to install the project path permanently to the python path. In short:

1. Create a 'src' subfolder in the root folder and move the project folder there
2. Create a pyproject.toml file in the root folder and add the appropriate contents
3. In the root folder, run 'pip install --editable .'

Then 'python demos/demo1.py' should work as expected. (The --editable part means you don't have to reinstall everything everytime you modify the code)

Ahhhhh, this was what I was looking for. Thank you!

# ¿ Sep 25, 2023 19:54

huhu: Feb 24, 2006

I feel like every definition I�ve had of imports paints them nicely. Then I ask one follow up question and it falls apart.

# ¿ Sep 26, 2023 07:44

huhu: Feb 24, 2006

I'm writing Python to control a 2d Plotter and make art. I've been wanting to explore randomness more in depth. Currently, I've just been doing stuff like plotting a series of points with random() as in the code below. (Code might not actually work, was just trying to give a simplified idea of what I've been working on)

code:

path = []

x_location = 0
y_location = 0

while True:
  x_direction = 1 if random() <= 0.5 else -1
  y_direction = 1 if random () <= 0.5 else -1

  x_magnitude = randint(0,10)
  y_magnitude = randint(0,10)

  x_location += x_direction * x_magnitude
  y_location += y_direction * y_magnitude

  points.append((x_location, y_location))

  # at some point break

plotter.plot(points)

I'm wondering if there are more interesting explorations I could do than this very simplistic approach I have in my code?

# ¿ Nov 14, 2023 05:56

Adbot: ADBOT LOVES YOU

# ¿ May 16, 2024 20:01

huhu: Feb 24, 2006

Jabor posted:

One possibility is to pick a random direction and distance, instead of doing X and Y separately.

Another possibility is to draw Bezier curves with randomly generated control points.

You could bias your diffs in some direction - so that it looks generally random at the small scale, but slowly tracks across the plot when you look at it overall.

--

Really there are lots of fun things you can do with this.

How do I begin to think like this is really my question. Should I go find a book about math and art or something�

# ¿ Nov 14, 2023 21:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›2 »