Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

C2C - 2.0 posted:

I just finished building a project for my class; it's essentially a Tkinter interface with a bunch of inputs for my music collection that saves the inputs to a .json file.

I'm having a go at using PyVis to map out some of the relationships of the items in the collection. Here's my code:

Here's the error I'm getting:
code:
AttributeError: 'NoneType' object has no attribute 'render'
music.html

I can�t tell exactly what the issue is from what you�ve posted, but the line

code:

g.show(�music.html�)

Isn�t calling your show() function, it�s calling a class method on the object g (an instance of class Network) that also has the name show(). That might or might not be the method you want to call, but assuming it is it looks like you�re not calling it correctly - either an argument is missing or is of the wrong type. You�ll need to look more into the Network class and its show() method to figure out what you�re doing wrong here.

The error message tells you that at some point in the invocation it tried to call a render() method on some object, but that object turned out to be None instead of what it was expecting so it couldn�t continue. Unfortunately that�s not enough information to solve the issue - python is sadly notorious for unhelpful error messages like this.

Also the show() function in that snippet looks like it was originally a member function that you copy pasted from some class defined elsewhere. That is almost certainly the wrong thing to do, though as I mentioned above that function isn�t currently doing anything in your code.

# ? Mar 1, 2023 00:02

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 09:00

Zoracle Zed: Jul 10, 2001

C2C - 2.0 posted:

Here's the error I'm getting:
code:
AttributeError: 'NoneType' object has no attribute 'render'
music.html
Prior to this, I didn't have the def show(): function written and was receiving the same error. The function was lifted straight from the documents and I'm still getting the same error. The only thing I came across while searching is that the notebook defaults to True unless it's passed a False argument which seems to be rendered moot by the function itself.

big picture suggestion here for when you're asking for troubleshooting help: try first to simplify the problem as much as possible. here's a heavily simplified version that exhibits the same error:

notice how this no longer requires us to understand what's in your 'music.json' (that would've been my first guess for a NoneType attribute error), and including the stack trace helps narrow down the problem

anyway, after poking around a bit, I think

code:

g.show(�music.html�)

can be changed to

code:

g.show(�music.html�, notebook=False)

using notebook=True (the default) pyvis is attempting to construct some HTML representation for inline viewing in a jupyter notebook. With it off, it just writes the requested html file.

if you do want the inline plot, it seems like you need to specify that at Network initialization for the template to be set properly?

code:

from pyvis.network import Network
net = Network(notebook=True)
net.add_node('fart')
net.show('music.html')

I'm not a pyvis user but imo this would warrant a nice, polite issue submitting to their github or whatever, seems like an unnecessary beginner trap

# ? Mar 1, 2023 00:22

QuarkJets: Sep 8, 2008

DoctorTristan posted:

python is sadly notorious for unhelpful error messages like this.

Whoa, that is not my experience at all. The problem is not that the message is vague, in fact the message states exactly what the problem was and the specific line that led to the problem is in the printed traceback but was just left out of the post. This is way more useful than the C standard of simply seg faulting. The problem here is that the OP left out most of the traceback

# ? Mar 1, 2023 01:56

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

The error messages are not very helpful, if you don't look at them.

# ? Mar 1, 2023 01:58

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

QuarkJets posted:

Whoa, that is not my experience at all. The problem is not that the message is vague, in fact the message states exactly what the problem was and the specific line that led to the problem is in the printed traceback but was just left out of the post. This is way more useful than the C standard of simply seg faulting. The problem here is that the OP left out most of the traceback

Yeah, 100% agreed; Python's error messages aren't Rust-level, but they're still pretty drat good and the traceback is almost always enough to figure out what's going on. This one immediately tells you that there's an issue where it's attempting to call a render method on something it expects to be there, but is a None instead. The library you're using shouldn't let a NoneType get that far without throwing a more specific error first, but that's not Python's fault.

While I'm soapboxing a little bit; this is also an excellent example of when a debugger would be super handy. Whatever IDE you're using should have one in this day and age (and if you're going to skip using an IDE, then you better learn to love PDB for debugging) and should call up a debug session exactly where the error occurs, allowing you to look over the stack and see what is assigned to what.

Zoracle Zed posted:

if you do want the inline plot, it seems like you need to specify that at Network initialization for the template to be set properly?
code:
from pyvis.network import Network
net = Network(notebook=True)
net.add_node('fart')
net.show('music.html')
I'm not a pyvis user but imo this would warrant a nice, polite issue submitting to their github or whatever, seems like an unnecessary beginner trap

Yeah this should 100% be caught by PyVis and throw an error. I might even look into raising a PR for this issue since I'm trying to get more into OSS.

Falcon2001 fucked around with this message at 02:17 on Mar 1, 2023

# ? Mar 1, 2023 02:06

pmchem: Jan 22, 2010

Jabor posted:

The error messages are not very helpful, if you don't look at them.

# ? Mar 1, 2023 03:23

Jose Cuervo: Aug 25, 2004

Seventh Arrow posted:

Not even the SQL thread?

Sorry I missed that but will ask in that thread in the future. I had previously asked SQLite questions in here and been answered, so more I meant my googling of search terms etc had failed.

Zoracle Zed posted:

just guessing but I bet your SQL driver is (correctly, imo) not interpolating the tss variable here:
code:
datetime(:start_dt, '+:tss hours')
because it's inside a string literal. try it something like this, (modulo whatever the appropriate string concatenation operator is for your db)
code:
datetime(:start_dt, STR_CONCAT('+', :tss, ' hours'))

String concatenation is the search term I needed to look for. In SQLite the concatenation operator is ||, so the following code works:

Python code:

c.execute("""
             SELECT dt_index, bg_mg_per_dL from blood_glucose
             WHERE SID == :SID
             AND datetime(dt_index) >= :start_dt
             AND datetime(dt_index) < datetime(:start_dt, '+' || :tss || ' hours')
             """,
             {'SID': SID,
              'start_dt': str(start_dt),
              'tss': tss_length_hrs}).fetchall()

Thanks!

# ? Mar 1, 2023 03:51

duck monster: Dec 15, 2004

I've been using python since the 1990s.

And the thing still finds new ways to delight me in the "how did I somehow not know about this amazing thing?" way.

code:

a = ['hey','there','dude']
b = [1,2,3]
zip
<class 'zip'>
zip(a,b)
<zip object at 0x100ae5c00>
dict(zip(a,b))
{'hey': 1, 'there': 2, 'dude': 3}
list(zip(a,b))
[('hey', 1), ('there', 2), ('dude', 3)]

Granted when I did my main education as a coder Java hadnt even been invented yet, I missed the transition of functional from "something you learn to pass the lisp unit" into "we actually do this IRL now" but..... neato

# ? Mar 1, 2023 07:45

12 rats tied together: Sep 7, 2006

Everything in itertools is delightful. Basically a standard library module tailor-made for trivializing coding interview screens.

# ? Mar 1, 2023 07:52

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

QuarkJets posted:

Whoa, that is not my experience at all. The problem is not that the message is vague, in fact the message states exactly what the problem was and the specific line that led to the problem is in the printed traceback but was just left out of the post. This is way more useful than the C standard of simply seg faulting. The problem here is that the OP left out most of the traceback

Maybe I�m projecting my own experience of using pandas and its truly incomprehensible traces, but I frequently find that the combination of dUcK tYpInG plus a general design approach in third-party libraries of throwing as late as possible means that the trace points to something long after the bug that actually caused the error. Here I agree it�s fairly easy for an experienced python person to figure out, though I really wish argument validation in top-level functions were more widespread than it currently is.

# ? Mar 1, 2023 13:54

Data Graham: Dec 28, 2009; 📈📊🍪😋

It took me a long time to realize that the whole stack trace is useful, not just the last few lines.

Like you'll have 30 lines in a trace; the first 14 are nested calls in native code, the last 14 are library code, and the one right in the middle is YOUR code where the issue is.

e: Python 3.11 now does highlighting with ^^^^^^^^^^ marks to show extremely clearly where it's breaking

Data Graham fucked around with this message at 14:12 on Mar 1, 2023

# ? Mar 1, 2023 14:06

C2C - 2.0: May 14, 2006; Dubs In The Key Of Life; Lipstick Apathy

Hey y'all, sorry for the previous mystery post. The instructor in the class really hammered away at looking at the actual error instead of the entire traceback. I was looking thru the PyVis documents yesterday evening and came across their "Getting Started" example which is rather simple:

Python code:

from pyvis.network import Network

g = Network()
g.add_node(0)
g.add_node(1)
g.add_edge(0, 1)
g.show("basic.html")

As expected, based on the replies to my previous post, the console output was:

code:

Traceback (most recent call last):
  File "/Users/rhc.iv/Development/Personal Projects/PyViz/PyVis Official/main.py", line 7, in <module>
    g.show("basic.html")
  File "/Users/rhc.iv/Development/Personal Projects/PyViz/PyVis Official/venv/lib/python3.11/site-packages/pyvis/network.py", line 546, in show
    self.write_html(name, open_browser=False,notebook=True)
  File "/Users/rhc.iv/Development/Personal Projects/PyViz/PyVis Official/venv/lib/python3.11/site-packages/pyvis/network.py", line 515, in write_html
    self.html = self.generate_html(notebook=notebook)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rhc.iv/Development/Personal Projects/PyViz/PyVis Official/venv/lib/python3.11/site-packages/pyvis/network.py", line 479, in generate_html
    self.html = template.render(height=height,
                ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'render'

Adding "notebook=False" to the .show() method fixed it. As an aside, I was looking at the official PyVis docs and perusing the individual methods in the library & there's a link to the source code in the library for each one. Here's the source code for .show():

Python code:

]
    def show(self, name, local=True,notebook=True):
        """
        Writes a static HTML file and saves it locally before opening.

        :param: name: the name of the html file to save as
        :type name: str
        """
        print(name)
        if notebook:
            self.write_html(name, open_browser=False,notebook=True)
        else:
            self.write_html(name, open_browser=True)
        if notebook:
            return IFrame(name, width=self.width, height=self.height)

It seems kinda' clunky to me, but I'm new to all this. Now that I've got it working in PyCharm, I tried to drop the basic Python code above into both Jetrbrains Datalore and Google Collab to just take a look at how PyVis looks/works in Jupyter but either using no parameter/arguments for .show() or trying to set them when calling the method gives errors on both platforms. I'm starting to realize that debugging is very much a part of the process of writing code, but this is the first time I've worked with a library where the tutorial section of their docs is spitting errors immediately barring installation issues. I think I'll just stick to using the library in my IDE since I'm trying to (ultimately) connect it to either a Flask or Streamlit app.

I also found a single mention of this issue on StackOverflow that indicated the show function located in the package's network.py file can be edited to change the defaults, but I think I'm gonna' keep my hands off of it for now and just pass the arguments in the script.

Thanks for everyone's help and constructive criticism!

# ? Mar 1, 2023 14:35

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

duck monster posted:

I've been using python since the 1990s.

And the thing still finds new ways to delight me in the "how did I somehow not know about this amazing thing?" way.
code:
a = ['hey','there','dude']
b = [1,2,3]
zip
<class 'zip'>
zip(a,b)
<zip object at 0x100ae5c00>
dict(zip(a,b))
{'hey': 1, 'there': 2, 'dude': 3}
list(zip(a,b))
[('hey', 1), ('there', 2), ('dude', 3)]

zip() is great. Python�s built-ins do so many cool things that aren�t always obvious.

Like, I recently discovered (through Trey Hunner�s newsletter) a way of avoiding a common for/break pattern. Let�s say you want to assign foo to the first item in an iterable that meets some criterion (such as the first even number), and if you don�t find it, then assign it to None. The verbose implementation is:

code:

numbers = [1, 3, 5, 8]
foo = None
for num in numbers:
    if num % 2 == 0:
        foo = num
        break

foo will be 8 here. And if there were no evens in the list, it�d stay as None. But the code is�not elegant to say the least. You could instead do this as a one-liner:

code:

foo = next((num for num in numbers if num % 2 == 0), None)

Basically next() will run through your generator expression until a value is yielded. If one is, it gets assigned to foo. If not, foo gets assigned to the default argument you supplied, None.

Seems like Python always has a way of simplifying really ugly code.

# ? Mar 1, 2023 14:56

QuarkJets: Sep 8, 2008

C2C - 2.0 posted:

Hey y'all, sorry for the previous mystery post. The instructor in the class really hammered away at looking at the actual error instead of the entire traceback.

Sometimes you have to look at both, they give you different information; the error tells you what went wrong (a None was encountered) and the traceback tells you where

# ? Mar 1, 2023 17:54

QuarkJets: Sep 8, 2008

C2C - 2.0 posted:

Adding "notebook=False" to the .show() method fixed it. As an aside, I was looking at the official PyVis docs and perusing the individual methods in the library & there's a link to the source code in the library for each one. Here's the source code for .show():
Python code:
]
    def show(self, name, local=True,notebook=True):
        """
        Writes a static HTML file and saves it locally before opening.

        :param: name: the name of the html file to save as
        :type name: str
        """
        print(name)
        if notebook:
            self.write_html(name, open_browser=False,notebook=True)
        else:
            self.write_html(name, open_browser=True)
        if notebook:
            return IFrame(name, width=self.width, height=self.height)
It seems kinda' clunky to me, but I'm new to all this. Now that I've got it working in PyCharm, I tried to drop the basic Python code above into both Jetrbrains Datalore and Google Collab to just take a look at how PyVis looks/works in Jupyter but either using no parameter/arguments for .show() or trying to set them when calling the method gives errors on both platforms. I'm starting to realize that debugging is very much a part of the process of writing code, but this is the first time I've worked with a library where the tutorial section of their docs is spitting errors immediately barring installation issues. I think I'll just stick to using the library in my IDE since I'm trying to (ultimately) connect it to either a Flask or Streamlit app.

I also found a single mention of this issue on StackOverflow that indicated the show function located in the package's network.py file can be edited to change the defaults, but I think I'm gonna' keep my hands off of it for now and just pass the arguments in the script.

Thanks for everyone's help and constructive criticism!

Yeah, that really sucks; their tutorial assumes the user is running in a notebook without stating that assumption. Yuck, you should send them a note about this; it'd be so easy to update the tutorial a little.

This is another major part of software development: documentation is sometimes deceptive

# ? Mar 1, 2023 18:10

C2C - 2.0: May 14, 2006; Dubs In The Key Of Life; Lipstick Apathy

More questions:

I used Pandas to create a DataFrame from my .json file and am working on learning MatplotLib to create graphs from the .json data. Some of the fields in the .json (like 'bpm', 'year', et. al.) are integers but they're saved as strings in the .json file. Short of going back and editing the .json file itself, is there a way to order the strings that are actually integers in either ascending or descending order? Right now, my x- and y-axes are displaying the values in the order (I assume) they're appearing in the DataFrame. Would using int() on those strings then creating a function to order them be possible? Here's one of the graphs as-is with the DataFrame ordering (again, I assume):

(Some of the .json entries don't have BPM values yet as I haven't calculated all of them; thus the initial empty BPM value on the x-axis)

# ? Mar 1, 2023 20:56

Jose Cuervo: Aug 25, 2004

I am trying to parallelize (using the joblib module) some code which involves querying a database (the query I had issues with earlier).

Python code:

conn = sqlite3.connect('study_data.db')
c = conn.cursor()

parallel_ts3 = joblib.Parallel(n_jobs=n_cpus)(
                        joblib.delayed(diag.parallel_tss_entry)(c, sub_SID_SDTs, tss_length_hrs)
                        for sub_SID_SDTs in chunked_iterable(SID_SDTs, chunk_size))

The above code produced the following error:

code:

"""
Traceback (most recent call last):
  File "/home/bjl2n/.conda/envs/jdrf/lib/python3.9/site-packages/joblib/externals/loky/backend/queues.py", line 153, in _feed
    obj_ = dumps(obj, reducers=reducers)
  File "/home/bjl2n/.conda/envs/jdrf/lib/python3.9/site-packages/joblib/externals/loky/backend/reduction.py", line 271, in dumps
    dump(obj, buf, reducers=reducers, protocol=protocol)
  File "/home/bjl2n/.conda/envs/jdrf/lib/python3.9/site-packages/joblib/externals/loky/backend/reduction.py", line 264, in dump
    _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
  File "/home/bjl2n/.conda/envs/jdrf/lib/python3.9/site-packages/joblib/externals/cloudpickle/cloudpickle_fast.py", line 563, in dump
    return Pickler.dump(self, obj)
TypeError: cannot pickle 'sqlite3.Cursor' object

...

PicklingError: Could not pickle the task to send it to the workers.
"""

Is there something I can do differently to run the code in parallel?

# ? Mar 1, 2023 22:07

Jose Cuervo: Aug 25, 2004

C2C - 2.0 posted:

More questions:

I used Pandas to create a DataFrame from my .json file and am working on learning MatplotLib to create graphs from the .json data. Some of the fields in the .json (like 'bpm', 'year', et. al.) are integers but they're saved as strings in the .json file. Short of going back and editing the .json file itself, is there a way to order the strings that are actually integers in either ascending or descending order? Right now, my x- and y-axes are displaying the values in the order (I assume) they're appearing in the DataFrame. Would using int() on those strings then creating a function to order them be possible? Here's one of the graphs as-is with the DataFrame ordering (again, I assume):

(Some of the .json entries don't have BPM values yet as I haven't calculated all of them; thus the initial empty BPM value on the x-axis)

You can use df['column_name'].astype(int) to convert the entries in a column to integers (note this will throw an error if any row entry in that column cannot be converted into an integer, e.g., if the entry is 'dog').

# ? Mar 1, 2023 22:08

Zoracle Zed: Jul 10, 2001

Jose Cuervo posted:

I am trying to parallelize (using the joblib module) some code which involves querying a database (the query I had issues with earlier).

first: sqlite (afaik) isn't great for parallel writing, so this is only worth doing if each task is only reading from the database.

second: the error you have there says "TypeError: cannot pickle 'sqlite3.Cursor' object". Can you see how to rearrange some things so the cursor object doesn't need to be shared between the jobs running in parallel?

# ? Mar 1, 2023 22:21

Jose Cuervo: Aug 25, 2004

Zoracle Zed posted:

first: sqlite (afaik) isn't great for parallel writing, so this is only worth doing if each task is only reading from the database.

second: the error you have there says "TypeError: cannot pickle 'sqlite3.Cursor' object". Can you see how to rearrange some things so the cursor object doesn't need to be shared between the jobs running in parallel?

The queries are only reading from the database.

And yes, I actually do not need to share the cursor but I did not realize that until you pointed it out, thanks!

# ? Mar 1, 2023 22:35

Oysters Autobio: Mar 13, 2017

C2C - 2.0 posted:

More questions:

I used Pandas to create a DataFrame from my .json file and am working on learning MatplotLib to create graphs from the .json data. Some of the fields in the .json (like 'bpm', 'year', et. al.) are integers but they're saved as strings in the .json file. Short of going back and editing the .json file itself, is there a way to order the strings that are actually integers in either ascending or descending order? Right now, my x- and y-axes are displaying the values in the order (I assume) they're appearing in the DataFrame. Would using int() on those strings then creating a function to order them be possible? Here's one of the graphs as-is with the DataFrame ordering (again, I assume):

(Some of the .json entries don't have BPM values yet as I haven't calculated all of them; thus the initial empty BPM value on the x-axis)

Gonna build a question on your question because I realized that I have no idea why I don't use matplotlib.

Is there a reason specifically why matplotlib is your choice for viz? My go to has always been more high level packages, and I'm particularly fond of Altair or plotly or seaborne. Throw your JSON into a pandas dataframe and then just declare your chart type and which columns to encode as x and y markers is just so easy. Though I guess if your JSON is heavily nested then you'd have to flatten it or remodel it to work easily in a pandas df.

Is there more interesting control that comes from matplotlib that makes it worth it? I'm really into data viz, soI've been debating to learn d3js or maybe it's higher level bindings in vega, but man I really wish I didn't have to get out of Python to do interactive and dynamic viz stuff so always on the look out for something better.

Oysters Autobio fucked around with this message at 03:45 on Mar 2, 2023

# ? Mar 2, 2023 03:41

C2C - 2.0: May 14, 2006; Dubs In The Key Of Life; Lipstick Apathy

Jose Cuervo posted:

You can use df['column_name'].astype(int) to convert the entries in a column to integers (note this will throw an error if any row entry in that column cannot be converted into an integer, e.g., if the entry is 'dog').

Ended up fixing it at the root of my app pipeline. My cataloging app was calling get() on all the .json fields for my Add Entry and Update Entry functions, so I just wrapped the necessary values as int(some_value.get()). Now I don't have to do anything with the .json other than convert it to a DataFrame.

Your suggestion did lead me to looking further into the Pandas docs and that's a ton of interesting reading since my class lightly touched on DataFrame manipulation and I reckon we didn't cover even 1/3 of just the DataFrame methods.

# ? Mar 2, 2023 03:52

C2C - 2.0: May 14, 2006; Dubs In The Key Of Life; Lipstick Apathy

Oysters Autobio posted:

Gonna build a question on your question because I realized that I have no idea why I don't use matplotlib.

Is there a reason specifically why matplotlib is your choice for viz? My go to has always been more high level packages, and I'm particularly fond of Altair or plotly or seaborne. Throw your JSON into a pandas dataframe and then just declare your chart type and which columns to encode as x and y markers is just so easy. Though I guess if your JSON is heavily nested then you'd have to flatten it or remodel it to work easily in a pandas df.

Is there more interesting control that comes from matplotlib that makes it worth it? I'm really into data viz, soI've been debating to learn d3js or maybe it's higher level bindings in vega, but man I really wish I didn't have to get out of Python to do interactive and dynamic viz stuff so always on the look out for something better.

I'm not the one who could answer this as I have zero programming background and just started learning Python a few months ago.

For my part, MatplotLib (specifically matplotlib.pyplot) was the first visualization library that was touched on in my class. Else that, we also did a handful of visualization sections using Plotly. Any other viz libraries (NetworkX, PyVis,) have just been me exploring. I'm in the final projects phase of the class, but other than core Python concepts, we only lightly touched on APIs, scraping, visualization, Numpy, et. al. While I'm doing my projects for the class, I'm going back to certain sections and doing a deeper dive into stuff that really interested me. I'm attempting to change careers after decades & it's somewhat disheartening that Python is everywhere & tons of folks are amazing at it. So I'm trying to shore up as much knowledge as I can before being brave enough to fill out job applications that are just going to be deleted :lol:

# ? Mar 2, 2023 03:59

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

Zugzwang posted:

You could instead do this as a one-liner:
code:
foo = next((num for num in numbers if num % 2 == 0), None)
Basically next() will run through your generator expression until a value is yielded. If one is, it gets assigned to foo. If not, foo gets assigned to the default argument you supplied, None.

Seems like Python always has a way of simplifying really ugly code.

Thank you for this. What I'd been doing in the past (when I was quite sure that the element I wanted would be in the list, and only exactly once) was use list comprehension and just tack a [0] at the end, but this is a bit more elegant. I just used it to filter through a list of GitHub release assets to find the right file to install during a pipeline run.

# ? Mar 3, 2023 19:09

Data Graham: Dec 28, 2009; 📈📊🍪😋

Yeah, I hear the Kill Bill klaxon in my head whenever I see [0]

IndexError waiting to happen.

# ? Mar 3, 2023 19:14

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

A friend pointed out to me that I don't need the default value in next, especially when the next step in my code (loading the URL I found) would fail if the default value happened, so I just took that out. It'll cause a StopIteration exception which I'm not handling, but what it really means is something is messed up with the project I'm reading from and I'll need to fix my code regardless.

# ? Mar 3, 2023 20:07

Precambrian Video Games: Aug 19, 2002

Oysters Autobio posted:

Gonna build a question on your question because I realized that I have no idea why I don't use matplotlib.

Is there a reason specifically why matplotlib is your choice for viz? My go to has always been more high level packages, and I'm particularly fond of Altair or plotly or seaborne. Throw your JSON into a pandas dataframe and then just declare your chart type and which columns to encode as x and y markers is just so easy. Though I guess if your JSON is heavily nested then you'd have to flatten it or remodel it to work easily in a pandas df.

Is there more interesting control that comes from matplotlib that makes it worth it? I'm really into data viz, soI've been debating to learn d3js or maybe it's higher level bindings in vega, but man I really wish I didn't have to get out of Python to do interactive and dynamic viz stuff so always on the look out for something better.

Seaborn is mostly a convenient wrapper around matplotlib with a less obtuse/ambiguous interface. It's quite handy but sometimes you end up needing to fiddle directly with matplotlib objects, change rcparams, etc. beyond what seaborn does. Also, you might find yourself needing to use a library that calls matplotlib directly so it doesn't hurt to get familiar with it.

# ? Mar 4, 2023 04:52

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

FISHMANPET posted:

A friend pointed out to me that I don't need the default value in next, especially when the next step in my code (loading the URL I found) would fail if the default value happened, so I just took that out. It'll cause a StopIteration exception which I'm not handling, but what it really means is something is messed up with the project I'm reading from and I'll need to fix my code regardless.

Yeah, I guess the one to pick depends on the context of the rest of your code and how you want to handle it. "Is there an even number in this collection? If not, assign variable to None" vs "roll through all these values and raise StopIteration if you hit the end" both make sense.

Data Graham posted:

Yeah, I hear the Kill Bill klaxon in my head whenever I see [0]

IndexError waiting to happen.

Sometimes it's impossible to avoid if you're calling certain functions that you know will return at least one unpackable value. Some NumPy functions are like this (at least, to my non-NumPy-expert knowledge). But yeah I hate it.

Zugzwang fucked around with this message at 06:17 on Mar 4, 2023

# ? Mar 4, 2023 06:08

Oysters Autobio: Mar 13, 2017

Thanks for the answers about matplotlib.

Would love any suggestions folks might have on courses, books, videos or other learning content where I can start learning a bit more about actual Python development beyond the basics of understanding dicts, and loops and etc. and something thats focused in the data analytics, engineering or science domain.

I'm very interested in learning more design patterns, but particularily what I would love is to find tutorials that teach a given design pattern but with tutorials and examples that are more align with data related domains like data engineering, data analytics or data science. A lot of the stuff I've been able to find sort of seem more angled towards building a web app for like, a company or inventory system or something, but I'm interested in learning how I could apply these patterns for building out applications that fit my day to day.

I'm a data analyst (though more like BI analyst in terms of actual job functions) who's main deliverable is basically tableau dashboards and maybe other ad-hoc visualisations or reports in Jupyter. But, like many places, my infrastructure is a shitshow/joke so its just a bunch of csv files being emailed around. Would love to learn how to actually build applications that could make my life or my teams life a bit better. Like, there's only like maybe 3-4 different csv structures we get in each dashboard product we make, so sure I could bring each one in and use pandas to clean it up each and every time, but it seems like it'd be worth the effort (and more interesting frankly) to make an actual tool rather than only learning python for one-off stuff.

I'd love to do a tutorial that taught some design patterns that had you build one program end-to-end along with tests and the like.

I've seen some that are for like, making a to-do list, but im having trouble finding examples that are like, okay we're gonna build a small app to automate cleaning a csv, or more advanced like, building a pipeline that takes data from these databases/APIs, does some things to it, then sends it to this database.

Oysters Autobio fucked around with this message at 22:21 on Mar 4, 2023

# ? Mar 4, 2023 22:16

QuarkJets: Sep 8, 2008

I don't have suggestions for books but if you start working through one and have any questions at all feel free to ask them here

# ? Mar 4, 2023 23:06

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

�Beyond the Boring Stuff with Python� and �Practices of the Python Pro� are both great books that are about how to write good Python in general.

Wes McKinney�s (creator of pandas) book is online for free too: https://wesmckinney.com/book/

Also, check out polars (the package) for data analysis. It�s a newer DataFrame library written in Rust. It isn�t a full replacement for everything pandas does, but in general, it�s comically faster.

Zugzwang fucked around with this message at 23:26 on Mar 4, 2023

# ? Mar 4, 2023 23:13

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

FWIW it sounds like you're looking more for courses than books since you want like a tutorial thing. I can't recommend any in particular but Udemy has a bunch. I once had access to PluralSight through my employer and those were all pretty great too, but I wouldn't pay for it out of pocket.

# ? Mar 6, 2023 07:53

Oysters Autobio: Mar 13, 2017

Thanks, yeah I think my ask is a bit too hyper specific so I'll widen my question just a bit better.

Other than SA, what other social media or platforms are people using for learning python and programming in general?

I've been really frustrated lately that my go to learning strategy is still opening up Google first.

With Google I realized that I just keep finding the same SEO-padding blogs that actually say next to nothing in terms of actually useful tutorials. Just feels like they're all either banal generalities about why x is better than y. And because of the SEO algo, every single startup, open source project or even indie devs have to min-max social media marketing with just banal blog posts about how "LoFi-SQL is totally the best new approach. LoFi-SQL is a database approach adopted in repos like <OPs_Project>".

Reddit and HN feel the same way because the useful threads are buried deep in scattered posts. Say what you want about megathreads but at least they're oriented around categories and topics. Reddit et al just feels like having to sort through endless small megathreads generated by the same repos, or on the other hand lurkers who are just trying to figure out wtf is LoFi-SQL and why it's part of the neomodern data stack.

Any good subreddits, forums, bulletin boards, or hell I'll get into listserv or mailing lists or w/e but just some places where people actually discuss how they do stuff in python.

Can't believe this dumb web 1.0 comedy forum has some of the best technical resources I've been able to find, so I'll take any suggestion even if it's a sub forum for New York renters that happens to have a decent tech community.

Oysters Autobio fucked around with this message at 17:10 on Mar 6, 2023

# ? Mar 6, 2023 16:56

The Fool: Oct 16, 2003

I did freecodecamp when it was just javascript and it was well designed, they have a bunch of other stuff now including some python courses:

https://www.freecodecamp.org/learn

# ? Mar 6, 2023 17:00

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

Oysters Autobio posted:

Thanks, yeah I think my ask is a bit too hyper specific so I'll widen my question just a bit better.

Other than SA, what other social media or platforms are people using for learning python and programming in general?

I've been really frustrated lately that my go to learning strategy is still opening up Google first.

With Google I realized that I just keep finding the same SEO-padding blogs that actually say next to nothing in terms of actually useful tutorials. Just feels like they're all either banal generalities about why x is better than y. And because of the SEO algo, every single startup, open source project or even indie devs have to min-max social media marketing with just banal blog posts about how "LoFi-SQL is totally the best new approach. LoFi-SQL is a database approach adopted in repos like <OPs_Project>".

Reddit and HN feel the same way because the useful threads are buried deep in scattered posts. Say what you want about megathreads but at least they're oriented around categories and topics. Reddit et al just feels like having to sort through endless small megathreads generated by random thought.

Any good subreddits, forums, bulletin boards, or hell I'll get into listserv or mailing lists or w/e but just some places where people actually discuss how they do stuff in python.

Can't believe this dumb web 1.0 comedy forum has some of the best technical resources I've been able to find, so I'll take any suggestion even if it's a sub forum for New York renters that happens to have a decent tech community.

I've also hung out in the Python discord community which seems to be pretty reasonable in terms of people asking questions and walking through stuff.

For just learning algorithms / etc, I'm a fan of code kata style approaches (such as leetcode or codewars), but it sounds like those are pretty low-level compared to what you're asking about.

# ? Mar 6, 2023 17:14

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

I really like Al Sweigart�s book Automate the Boring Stuff with Python. It�s �here are the basics of Python, now here are a bunch of very useful things you can do with it.� It�s available for free at https://automatetheboringstuff.com

For further education, the YouTube channel ArjanCodes is great.

# ? Mar 6, 2023 17:15

Zoracle Zed: Jul 10, 2001

my #1 suggestion would be to start contributing to open source projects you're already using.

1) If it's a project you use and like, presumably the people who maintain it know a whole bunch of stuff about python you could learn
2) Having a pre-existing project to contribute to removes a lot of "where do I even begin?" blank-page syndrome
3) In addition to language-specific skills, getting better at source control and collaborative programming is useful everywhere
4) It can genuinely feel good to contribute -- you might make the world an infinitesimally better place!

Oysters Autobio posted:

With Google I realized that I just keep finding the same SEO-padding blogs that actually say next to nothing in terms of actually useful tutorials.

this poo poo sucks so bad and it's depressing how it's never going to get better

# ? Mar 6, 2023 17:21

Oysters Autobio: Mar 13, 2017

Falcon2001 posted:

I've also hung out in the Python discord community which seems to be pretty reasonable in terms of people asking questions and walking through stuff.

For just learning algorithms / etc, I'm a fan of code kata style approaches (such as leetcode or codewars), but it sounds like those are pretty low-level compared to what you're asking about.

Oh hey both of these are awesome resources too, thanks for sharing. Someone should setup an open source project that assigns issue tickets and WIPs as code kata challenges that independently look random but actually combine to build a codebase, lol.

Definitely very useful for reusability and practice.

Zoracle Zed posted:

my #1 suggestion would be to start contributing to open source projects you're already using.

1) If it's a project you use and like, presumably the people who maintain it know a whole bunch of stuff about python you could learn
2) Having a pre-existing project to contribute to removes a lot of "where do I even begin?" blank-page syndrome
3) In addition to language-specific skills, getting better at source control and collaborative programming is useful everywhere
4) It can genuinely feel good to contribute -- you might make the world an infinitesimally better place!

this poo poo sucks so bad and it's depressing how it's never going to get better

I've wanted to do this but feeling a bit intimidated and overly anxious (which is silly now that I say it outloud I know) about looking like an idiot when submitting a MR. Worried my Python abilities aren't "meta" enough for actually contributing it somewhere. But, you're right actually. Plus if its something I think I could use at work then it actually might be significantly easier to try and build inu a couple features I might find useful into an existing codebase rather than trying to re-make whatever edge-case and being intimidated by the whole initial setup.

And yeah SEO really is sad. Like, the "dead internet" theory as a literal actual existing thing is paranoid conspiracy theory (everyone's a bot!! 🤖) but as a way to look at Google advertising-driven content its pretty much allegorical to websites creating this padded blog poo poo to game the SEO algo. I'm sure in the last few years some of these are either sped up or assisted by NLP generation of some kind. It's sadly only going to get worse I imagine.

Oysters Autobio fucked around with this message at 17:52 on Mar 6, 2023

# ? Mar 6, 2023 17:37

Zoracle Zed: Jul 10, 2001

Oysters Autobio posted:

I've wanted to do this but feeling a bit intimidated and overly anxious (which is silly now that I say it outloud I know) about looking like an idiot when submitting a MR. Worried my Python abilities aren't "meta" enough for actually contributing it somewhere. But, you're right actually. Plus if its something I think I could use at work then it actually might be significantly easier to try and build inu a couple features I might find useful into an existing codebase rather than trying to re-make whatever edge-case and being intimidated by the whole initial setup.

yeah! don't forget you can dip your toes in the water by adding or improving documentation, something that pretty much every project ever will welcome with open arms. I like to start contributing to a new project with small PRs anyway, it's nice to feel out the maintainers a bit -- how fast they respond, how detailed their feedback, how stringent they are on formatting issues, etc.

# ? Mar 6, 2023 17:57

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 09:00

ynohtna: Feb 16, 2007; backwoods compatible; Illegal Hen

When researching and learning new areas for which I have few other sources, I like to search for "<topic> best practices"

It filters out lots of article spam in favour of material with lessons drawn from experience and expertise. Could be something to consider.

# ? Mar 6, 2023 18:18

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »