C2C - 2.0 posted:

Is there a detailed explainer somewhere about imports/packages?

I'm on an M1 MacBook Pro & have also installed 3.10 via Homebrew. I've been back & forth between the "Non-Programmers Guide to Python 3" and "The Hitchhiker's Guide to Python" working through the chapters. The past couple of days, I've also been perusing "project"-type repos on Github and have found myself at odds with imports, packages, & things like 'requirements.txt'. As the screenshot below illustrates, I'm unable to run the programs but also unable (to reconcile?) the packages required for the programs. My guess is that there's a disparity between the version of Python pre-installed on the laptop (3.9?) and the version I have installed via Homebrew.

I'm not looking for a complete solution here on the forums & don't wanna' bog down the thread with baby's-first-python; any suggestions for another site that explains this process in detail would be fine.

C2C - 2.0 posted:

Is there a detailed explainer somewhere about imports/packages?

I'm on an M1 MacBook Pro & have also installed 3.10 via Homebrew. I've been back & forth between the "Non-Programmers Guide to Python 3" and "The Hitchhiker's Guide to Python" working through the chapters. The past couple of days, I've also been perusing "project"-type repos on Github and have found myself at odds with imports, packages, & things like 'requirements.txt'. As the screenshot below illustrates, I'm unable to run the programs but also unable (to reconcile?) the packages required for the programs. My guess is that there's a disparity between the version of Python pre-installed on the laptop (3.9?) and the version I have installed via Homebrew.

I'm not looking for a complete solution here on the forums & don't wanna' bog down the thread with baby's-first-python; any suggestions for another site that explains this process in detail would be fine.

SurgicalOntologist posted:

I would fork remotezip and modify it to support passing a session optionally. Looks like it's less than 200 lines so it should be pretty easy and is a worthwhile contribution to give back to the project

Bad Munki posted:

Yeah, a PR was actually one of our considered options. Pretty sure the required changes would only amount to a few lines.

SurgicalOntologist posted:

Nice! I didn't know about remotezip before you mentioned it, I'm going to keep it in mind in case the need arises.

FredMSloniker posted:

After spending a couple of days wrestling with an unrelated problem, I'm back to my project to create a game book solver, using "Impudent Peasant!" as a test case. My code's getting pretty long, so I won't paste the whole thing, just the bits I have questions or comments about.

f-strings are my new best friend, by the way, QuarkJets. I've discovered their limitations, though (they don't always handle backslashes well), and in the process learned that implicit string joining only works on literal strings, so a number of f-strings became parenthesized items became "".join() instances. (flake8 has been helpful for error-checking, so thanks for that too.)

FredMSloniker posted:

f-strings are my new best friend, by the way, QuarkJets. I've discovered their limitations, though (they don't always handle backslashes well), and in the process learned that implicit string joining only works on literal strings, so a number of f-strings became parenthesized items became "".join() instances. (flake8 has been helpful for error-checking, so thanks for that too.)

CarForumPoster posted:

IDK if anyone needs this but I've wanted a simple to use and install way to OCR PDFs that might be a mix of already OCR'd, scanned documents with automated text added or just images for a while.

Here is a better-than-pytesseract way to take in PDFs, flatten to an image then OCR. Very little testing but its the most promising and easiest to set up of any of the solutions I've tried. Assumes Anaconda which made this easier.

conda install pytorch using the proper lines for your OS/CUDA version from their website

code:

conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
pip install easyocr
conda install -c conda-forge poppler
pip install pdf2image

code:

import easyocr
reader = easyocr.Reader(['en'])

from pdf2image import convert_from_path
# pages is a list of PIL objects
pages = convert_from_path('file.pdf', 250)

import io
img_byte_arr = io.BytesIO()
pages[1].save(img_byte_arr, format='PNG')
img_byte_arr = img_byte_arr.getvalue()

text_output = reader.readtext(img_byte_arr, detail=0, paragraph=True)
for text in text_output:
    print(text)

QuarkJets posted:

Can you provide examples of these? I haven't experienced this but maybe that's just a fluke in how I write f-strings

Jigsaw posted:

Do you know how this compares to ocrmypdf? I’ve been using that and found it simple to use in the cases you describe and that it works pretty well.

FredMSloniker posted:

As I recall, the problem I ran into was something like this:

And I take a quick break to amend my code appropriately.

FredMSloniker posted:

Zephirus posted:

While you can't use \n in a f-string

FredMSloniker posted:

my plan for storing the section function itself in the state string won't work,

FredMSloniker posted:

I need the serialized version of a dictionary to be identical no matter what order the keys are defined in, because I don't care in what order I wrote stuff down on the character sheet, just what I wrote down.

FredMSloniker posted:

Some further testing of my code has discovered a problem - namely, my plan for storing the section function itself in the state string won't work, or at least won't work the way I had planned. The error in question is:

TypeError: Object of type function is not JSON serializable

So that's a problem. Fortunately, it looks like I just need to use pickle instead of json. Unfortunately, I ran into this problem again:

Python 3.10.1 (tags/v3.10.1:2cd268a, Dec 6 2021, 19:10:37) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pickle import dumps, loads
>>> a = {"foo": 1, "bar": 2}
>>> b = {"bar": 2, "foo": 1}
>>> a == b
True
>>> dumps(a) == dumps(b)
False

And unlike json, there's no sort_keys option. I need the serialized version of a dictionary to be identical no matter what order the keys are defined in, because I don't care in what order I wrote stuff down on the character sheet, just what I wrote down. What would you suggest? Do I need to pickle.dumps the section function and json.dumps the section data?

Also, on the topic of falsiness, since Zephirus brought it up: one thing that's annoyed me about Python versus Lua is that, in Lua, something being falsy means it's false or nil and truthy means it's anything else. In Python, though, some other things are falsy, including the number 0 and the string "", which keeps tripping me up. If I want to write a pluralizing function, for instance, (n == 1) and "" or "s" always returns "s"; I have to write (n != 1) and "s" or "" for it to work right. And I can't test if something is exactly True or False with a simple ==; I also have to check that it's the correct type.

That (condition) if (true result) else (false result) construction isn't something I've seen before, and I'll keep it in mind.

Foxfire_ posted:

If it isn't from you copying to forum, part of your first error is because you have some screwed up braces.

Foxfire_ posted:

There's nothing wrong with using a \n in a f-string:

Python code:

In [62]: f"this is fine: {some_number}\n"
Out[62]: 'this is fine: 1\n'

Foxfire_ posted:

This seems like a very bad idea. You are on-purpose trying to store a snapshot of the program in your database state?

QuarkJets posted:

What if you just write the function name instead of the function? That should give you a json-serializable dict. I would avoid using pickle at all

When you write the dictionary, convert functions to function names (you should be able to use the function's __name__ attribute)

When you load the dictionary remap function names to functions. You're using a specific key for this already so these cases should be easy to find.

FredMSloniker posted:

return vars().get(section, lambda a: None)(state_string)

QuarkJets posted:

I don't think that this does what you want it to do, and it'd be bad code even if it did work. Don't do this. If you absolutely need to be able to map function string names to functions, create a dict that contains those mappings and then refer to that.

It'd be better still if you didn't need to refer to a lookup table of function names at all and instead just called the functions directly.

QuarkJets posted:

On truthiness, you've got the construction mixed up. It should be
(true result) if (condition) else (false result)
That may be why you're seeing unexpected results. For instance
x = "" if n == 1 else "s"
This will set x to "" if n equals 1, otherwise x is set to "s"

FredMSloniker posted:

f-strings

FredMSloniker posted:

So now I'm kind of lost as to what you think I should actually do.

quote:

My example was actually x = (n == 1) and "" or "s", but thanks for the correction on the if-else construction.

SirPablo posted:

Since this thread seems more active than the the AI thread and Data Science thread, figured I would ask here. I have a science background (Master's degree) and have been working with python for 10 years now. I'm trying to get more into ML, maybe even to switch careers. Are there any certification programs that employers actually value? I found some options listed here and here, but not sure if any of these are considered worthwhile. I'm not completely opposed to a graduate certificate from a university, but is that even necessary or just throwing money away. Any thoughts or recommendations?

QuarkJets posted:

f-strings don't handle backslashes inside of f-string expressions, that's the problem that you're seeing. You can solve this by just assigning that part of the expression to its own variable (e.g. the expression inside of the expression). In general you should try to avoid writing expressions with any amount of complexity in f-strings; you can do it but it makes your code way harder to read

QuarkJets posted:

This is not if-else assignment. What you're doing here is truthy assignment, which is why it's giving you results that you don't expect.

QuarkJets posted:

I think that you should make that mapping yourself rather than directly looking up functions out of locals().

QuarkJets posted:

Usually you shouldn't be using locals() at all,

SirPablo posted:

Since this thread seems more active than the the AI thread and Data Science thread, figured I would ask here. I have a science background (Master's degree) and have been working with python for 10 years now. I'm trying to get more into ML, maybe even to switch careers. Are there any certification programs that employers actually value? I found some options listed here and here, but not sure if any of these are considered worthwhile. I'm not completely opposed to a graduate certificate from a university, but is that even necessary or just throwing money away. Any thoughts or recommendations?

SirPablo posted:

Since this thread seems more active than the the AI thread and Data Science thread, figured I would ask here. I have a science background (Master's degree) and have been working with python for 10 years now. I'm trying to get more into ML, maybe even to switch careers. Are there any certification programs that employers actually value? I found some options listed here and here, but not sure if any of these are considered worthwhile. I'm not completely opposed to a graduate certificate from a university, but is that even necessary or just throwing money away. Any thoughts or recommendations?

FredMSloniker posted:

So you think I should do something like:

Python code:

def section_1(section_data):
    # stuff
def section_2(section_data):
    # stuff
def section_3(section_data):
    # stuff
# and so on until
def section_1000(section_data):
    # stuff

section_dictionary = {
    "section_1": section_1,
    "section_2": section_2,
    "section_3": section_3,
    # and so on until
    "section_1000": section_1000
}

def process_state(game_state):
    section_dictionary[game_state["section_number"]](game_state["section_data"])

Or am I misunderstanding you? Because I'm really not seeing how a long list of functions followed by a long dictionary I have to roll by hand is better than:

Python code:

def section_1(section_data):
    # stuff
def section_2(section_data):
    # stuff
def section_3(section_data):
    # stuff
# and so on until
def section_1000(section_data):
    # stuff

section_dictionary = {name: value for name, value in locals().items() if callable(value) and name.startswith("section")}

def process_state(game_state):
    section_dictionary[game_state["section_number"]](game_state["section_data"])

quote:

Why not? What are the hazards of using it, and why does it exist if I shouldn't use it?

QuarkJets posted:

The reason that most projects shouldn't use locals() is that it's not usually not necessary, so it's a code smell. But a code smell does not mean that you should never use it - it means that you should carefully assess whether it's necessary. This is a case where I think it makes sense to use it

FredMSloniker posted:

That's an interesting term, code smell. I can glark it, though. At any rate, I'll be back to the thread once I have more to show. Thanks for the help!

e: quick question. It's okay if I define function_dictionary after the functions that rely on it as long as I don't actually call any of those functions before then, right? I'm under the impression that Python isn't picky about that sort of thing.

QuarkJets posted:

The code inside of those functions won't execute until you actually call them, that's correct; not sure if that's also what you're asking.

SirPablo posted:

Thanks for the recommendations, I'll focus on improving my GitHub presentation (which is zero!). My experience is certainly more hacky I feel, being one of the few coders in a scientific office where most of the software used by the staff is handed down from a contractor. Most of my work is quality-of-life type projects, somewhat small and only beneficial to what my office does.