|
nullfunction posted:Since you're offering it up for a roast, here are some things to consider: Do you have any good resources that discuss the proper way to deal with error handling?
|
# ? Apr 30, 2024 21:53 |
|
|
# ? May 4, 2024 09:28 |
|
Fender posted:There is some good feedback in here. Some of it I feel would be appropriate for a conversation. Like, why I check for None a lot. It's just a habit I learned in my last role where we parsed a lot of unreliable data sourced from scrapers. I would happily explain my position there. One problem I see is that the None-checking is everywhere. Since you need to filter out empty strings anyway, it'd be better if you translated "None" to an empty string - then all of the code that's downstream from the file parsers doesn't have to do any None checking. Basically you have these 3 interface functions that are letting None values filter into the rest of your functions, so you need to handle the possibility of None everywhere, but if you handled the None values in the interface functions then the rest of your code would be a bit nicer. Another problem is that the None checking has become boilerplate; it'd be better to use some small utility functions to simplify the code. This also gets back to the issue I talked about where the street values have the possibility of raising an exception if any of them happen to be None; you did the none-checking for each of the street components in the XML function, but then you forgot that any of them could be None when you concatenated them together. If you had an interface function that returned only strings instead, the code looks nicer and this bug goes away. For example: Python code:
Python code:
quote:And stuff like the Address class having that weird view dict method, would love to chat about that. That is like that bc while they never said it, the examples both showed the printed json ignoring any keys without values, so I went out of my way to provide that since all the example data had different fields present/missing depending on the format. I was really getting in my own head by hour number 4. I got that it was necessary to filter keys without values; I was saying that the view_dict method could have been a lot more concise. If you'd used a dataclass: Python code:
Python code:
|
# ? May 1, 2024 03:37 |
|
Chillmatic posted:I wrote a script to get all the xml files from the show in question, and then iterate through them using the intro marker data in those xml files to decide where to make the edits using ffmpeg, except now I'm running into losing the subtitles from the original files, and converting them is a whole thing because they're PGS format. Comedy option: Send the edited files to OpenAIs whisper API and let them create new transcripts for you!
|
# ? May 1, 2024 04:14 |
|
Chillmatic posted:I wrote a script to get all the xml files from the show in question, and then iterate through them using the intro marker data in those xml files to decide where to make the edits using ffmpeg, except now I'm running into losing the subtitles from the original files, and converting them is a whole thing because they're PGS format. You "simply" need to discover the correct invocation Obi-Wan. (I do not know it, sorry...)
|
# ? May 1, 2024 04:45 |
|
I ended up doing the most hacky bullshit ever, but finally I will never have to listen to the Office theme song ever again. gently caress that piano. gently caress that doo doo rear end elementary school recorder instrument or whatever the gently caress it is. Boom bitch, bye.
|
# ? May 1, 2024 15:03 |
|
QuarkJets posted:More stuff. My entire thought process for that Address class was basically, "this data is crap, I'm just gonna pass None all the way to the constructor and not worry about it." Which sounds so obviously pointless after reading your advice. It's rough getting your code roasted, but drat if this isn't good poo poo.
|
# ? May 1, 2024 21:37 |
|
Thank you! Yeah that data sucks for sure, what a pain. And irl someone is going to come to you with some stupid bullshit like "Oops between dates X and Y the last and first names are swapped and in the wrong columns, can you fix that" and depending on the request you have to know how to ride the line between "no that's an unreasonable request" and "this is important, I will help you with that".
|
# ? May 1, 2024 22:27 |
|
Cyril Sneer posted:Do you have any good resources that discuss the proper way to deal with error handling? Afraid not. My knowledge is built up over many years and many versions and there's no one place I can point to that has everything you would conceivably want to know. I can generalize a bit but you're going to need to get your hands dirty if you really want to understand. Always catch the most specific type you can in any given situation. Overly-broad exception handling is definitely a smell and you can configure your linter to warn you about this if you're not already being warned. There are a couple of patterns you can use to help manage complex exception possibilities: Python code:
Python code:
Python code:
Python code:
It's a good practice to put the absolute minimum you can get away with inside each try block, often this is a single line of code. It's much nicer to reason about lots of try/except blocks that catch the same error from different calls than to have all the calls in a single try/except and not have a clear indication of which call raised the exception without digging through the exception's guts. If you intend to handle an exception from either of those calls in the same way and it makes no difference to you where that exception originated, feel free to use the more concise form. Python code:
Consider a script that munges CSV files from a list of provided directories and outputs the result of some calculation as a different CSV file. How many failure modes can you think of that might need to be handled differently? What if we were provided an empty list of directories? What if we can't access one of those directories? What if we can't access any of the directories? What if we can't access a single file in one of the directories but can access the rest? What happens if a provided directory doesn't contain any CSV data? What happens if we load something that is named CSV but doesn't contain CSV data, just plain text or a blank file? What happens if it contains binary data? What happens if the CSV headers don't include the field(s) we're after? What happens if the data itself is not valid? What if we can't write the output file? That's a lot of failures and I'm sure you could come up with many more in this vein! What's not great is that these are largely technical concerns which naturally lend themselves to error handling conversations. The other side of the coin is how the script fits into the larger picture. If your script is run nightly on a server using automation, your error handling story will be completely different than if Bob in Accounting has to run this script ad-hoc at the end of the month to come up with the numbers needed to close the books.
|
# ? May 2, 2024 01:38 |
|
I have a docker container that just runs a python script that does a TCP server I built. It's worked great for the last 3 years. I was changing some logging in the code and in the meantime upgraded python and the underlying linux. Python 3.9 -> 3.12 is no problem but bullseye -> bookworm seems to break it. Python code:
pre:[2024-05-02 01:23:14,025] - ERROR - Unhandled exception in client_connected_cb handle_traceback: Handle created at (most recent call last): File "/usr/local/app/server.py", line 442, in <module> asyncio.run(main(bind_address, bind_port), debug=True) File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1979, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>> Traceback (most recent call last): File "/usr/local/app/server.py", line 285, in client_connected fwd_reader, fwd_writer = await asyncio.open_connection(host_ip, port, family=AF_INET, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/streams.py", line 48, in open_connection transport, _ = await loop.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1080, in create_connection infos = await self._ensure_resolved( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1456, in _ensure_resolved return await loop.getaddrinfo(host, port, family=family, type=type, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 901, in getaddrinfo return await self.run_in_executor( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 863, in run_in_executor executor.submit(func, *args), loop=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 179, in submit self._adjust_thread_count() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 202, in _adjust_thread_count t.start() File "/usr/local/lib/python3.12/threading.py", line 992, in start _start_new_thread(self._bootstrap, ()) RuntimeError: can't start new thread It works fine if I run Python 3.12 in bullseye, just not bookworm, so not sure what changed in bookworm. Running ulimit and ulimit -u inside the container result in "unlimited" and the host has thousands of spare threads. Any idea what it could be? Hed fucked around with this message at 13:55 on May 3, 2024 |
# ? May 2, 2024 02:35 |
|
I found this thread on docker forums that mentions the —privileged flag with docker resolved it. https://forums.docker.com/t/runtimeerror-cant-start-new-thread/138142/4 Upgrading both the OS and Python at the same time can make it difficult to nail down exactly what side changed, but my hunch is something related to Linux and containers and security contexts. There may be a more limited seccomp change you can make to allow threading without the privileged flag. https://docs.docker.com/engine/security/seccomp/
|
# ? May 2, 2024 02:53 |
|
|
# ? May 4, 2024 09:28 |
|
Thanks, I’ll give that a shot. Great find. And I posted in error, I was upgrading all the way from buster. So everything but the latest Linux release seems to work. I’ll report back.
|
# ? May 2, 2024 02:57 |