Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
credburn
Jun 22, 2016
A tangled skein of bad opinions, the hottest takes, and the the world's most misinformed nonsense. Do not engage with me, it's useless, and better yet, put me on ignore.

QuarkJets posted:

You've nailed the cause; the last column is actually "boy\n" rather than "boy", so those are identified as distinct. That's why the last line looks like that: print is seeing that newline character and actually printing a new line.

You can very simply remove newline characters (as well as white space) using strip():

Python code:
fileb = filea.read().strip()

Oh, wow, that took care of it, thanks!

Is that generally the case, then, that a csv file formatted in such a way always has a newline at the end?

I seem to use .strip() a lot in these assignments; as one does a lot of coding, is this just a real common cleanup tool, or are you always using it intentionally?

Adbot
ADBOT LOVES YOU

lazerwolf
Dec 22, 2009

Orange and Black

credburn posted:

Hey, gang! Can you guys help identify where the problem lies in my code? I've almost got it down.

In this assignment, there is a file called input1.csv -- the online lab thing will input the appropriate name, as well as bunch of other files for the sake of testing. Each one I run into the same problem, which I'll explain in a second.

the contents of input1.csv are:

code:
hello,cat,man,hey,dog,boy,Hello,man,cat,woman,dog,Cat,hey,boy
The assignment wants me to write some code that takes an input and returns the words and their frequencies like so:

code:
hello 1
cat 2
man 2
hey 2
dog 2
boy 2
Hello 1
woman 1
Cat 1
And so here is my code:

code:
import csv

filea = open(input())
fileb = filea.read()
filec = fileb.split(',')
filea.close # I know there are better ways of doing this, but I don't have the with / as thing memorized and I'm trying not to just look this stuff up
bark = {}
for i in filec:
    if i not in bark:
        bark[i] = 1 # goes through the words, adds each new one to the dictionary and gives them a value of 1.
    else:
        bark[i] += 1 # if the word already exists as a key in the dictionary, just add 1 to its value
for k, v in bark.items():
    print(k, v)
But my output looks like this:

code:
hello 1
cat 2
man 2
hey 2
dog 2
boy 1
Hello 1
woman 1
Cat 1
boy
 1
and I can't figure out why. It's always the last one, but I don't know why it wouldn't have appropriately applied added to the the dictionary {bark} in the first for loop. :confused:

I think possibly the very last line of the csv may be interpreted as a newline character or something...? But in the example provided, there is no indication thereof; it just terminates at boy. This being an online environment, it's not without a couple bugs, which makes me really kind of annoyed and always cautious about whether the problem is my code, the software, some limitation of the virtual web-based environment, or even just a flaw in the presentation. But I guess I should err that the error is in my code; it usually is.

Your guess of a newline character is probably the case.

I’m not going to refactor the entire code but modifying this line to strip new lines before splitting should do it

filec = fileb.strip().split(',')

QuarkJets
Sep 8, 2008

credburn posted:

Oh, wow, that took care of it, thanks!

Is that generally the case, then, that a csv file formatted in such a way always has a newline at the end?

I seem to use .strip() a lot in these assignments; as one does a lot of coding, is this just a real common cleanup tool, or are you always using it intentionally?

Each line of a CSV is usually going to have a newline character at the end of it so it's a pretty safe assumption, yeah. I'm sure there are madmen out there that use some other character for some reason but that's unusual

It depends on what you're doing; if you're manually reading a lot of CSV files then you're going to be using strip() a lot, assuming you actually need that last column. A lot of other data formats don't require strip() at all, and libraries like pandas can take care of the newline characters for you. In my professional life I work with numerical data and never need strip()

DoctorTristan
Mar 11, 2006

I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

punk rebel ecks posted:

You lost me here. The tutorial has no "requirements.txt".

Also how do I run my python file from the virtual environment? Navigate to it's folder via the terminal?

I wish there was a way I could do all this from VSC. :(

Whoever wrote the tutorial might have skipped creating a requirements.txt because the project doesn’t have many dependencies. Try just running pip install flask inside the virtual env - that may be all you need.

To run a python file in a virtual environment, you launch a terminal, activate the environment in that terminal (as described by posters above), then do run the file as you normally would (python -m path_to_file if you’re on windows)

You absolutely can use virtual environments within VSCode - it wouldn’t be much use as a python dev environment if you couldn’t. Do ctrl-shift-P and search for the Python: Select Interpreter command - that will allow you to select the virtual environment. Now whenever you create a new terminal window within VSCode it will automatically activate the virtual environment in that window, so code running in that window will run in the virtual environment. Alternatively if you created the virtual environment within the workspace folder (which is what I do), VSCode will automatically detect it and ask if you want to use it.

DoctorTristan
Mar 11, 2006

I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

QuarkJets posted:

Each line of a CSV is usually going to have a newline character at the end of it so it's a pretty safe assumption, yeah. I'm sure there are madmen out there that use some other character for some reason but that's unusual


This is not a safe assumption at all for the last (data) line in the file - it’s only true there if the creator of the file ended with a blank line (which you’re supposed to do, but people frequently don’t bother).

ExcessBLarg!
Sep 1, 2001

credburn posted:

I seem to use .strip() a lot in these assignments; as one does a lot of coding, is this just a real common cleanup tool, or are you always using it intentionally?
In most environments when you do line-by-line processing of a text file, the string containing each line will end with a newline "\n", or sometimes even a "\r\n", which you usually want to drop while processing.

In some other languages (notably Perl & Ruby) there's a function/method called chomp that will strip a "\n" or "\r\n" from the end a string if present and do nothing else. So, yes, you very commonly call chomp on your input, and the resulting behavior is the same whether the end of the file has a newline or not.

There's no exact chomp equivalent in Python (str.removesuffix("\n") in 3.9 maybe?). The closest is str.rstrip() which will remove any trailing whitespace from a string. When text processing though, most people would argue that any whitespace at the beginning or end of a line is probably unintentional and so just use str.strip() to clean up the whole thing.

Let's jump back a bit though. Outside the scope of a homework assignment, if you have a data file of a known format you're generally best off using the most abstracted means to read it. For example, if you have a ".csv" file (and you don't want to read it as a DataFrame), you're probably best using csv.reader since it will handle newlines, delimiters, quoted strings, etc., for you. If you have text that's not quite csv then maybe you're better off manually parsing it.

Regarding context managers, I'll just throw out there that my preferred way of reading data in simple programs like this is fileinput.input. It lets you read input from stdin or a file specified on the command line, and you don't have to worry about using a context manager or closing it (although both are OK too).

worms butthole guy
Jan 29, 2021

by Fluffdaddy
Hey all, I have another simple question that i'm hoping someone can point me in the right direction.

How do you test if a JSON key exists in Python and if not, move on?


So I am using Flask Mail to send out a email. It looks like this:


code:
f"""
Reservation Process: {HubspotData['properties']['reservation_process']['value']}
Pool: {HubspotData['properties']['pool']['value']}
"""
The problem is, if someone didn't fill out that field, the app will crash instead of just continuing on. Is there a way to rectify this? With PHP and Javascript I can use something like:

code:
"Reservation Process:{HubspotData['properties']['reservation_process'] ? {HubspotData['properties']['reservation_process']['value']} : null}"
Thanks!

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Without having access to the rest of the code, I would do something like this, using comprehensions. Put the if/else on whatever

Python code:
f"""
Reservation Process: {HubspotData['properties']['reservation_process']['value'] if HubspotData['properties']['reservation_process']['value'] else "Not Selected"}
Pool: {HubspotData['properties']['pool']['value'] if HubspotData['properties']['pool']['value'] else "Not Selected"}
"""

OR

reservation_process_selection = bool(HubspotData['properties']['reservation_process']['value'])
pool_selection = bool(HubspotData['properties']['pool']['value'] if HubspotData['properties']['pool']['value'])

f"""
Reservation Process: {HubspotData['properties']['reservation_process']['value'] if reservation_process_selection else "Not Selected"}
Pool: {HubspotData['properties']['pool']['value'] if pool_selection else "Not Selected"}
"""
It won't crash, but it also won't throw an error if that's what you're after. It looks like the PHP/JS thing you're talking about is similar to an if/else comprehension.

Can you tell I love comprehensions? :awesome:

worms butthole guy
Jan 29, 2021

by Fluffdaddy
Ternarys are very fun in javascript and php so I don't blame you! That worked though so thank you :)

QuarkJets
Sep 8, 2008

DoctorTristan posted:

This is not a safe assumption at all for the last (data) line in the file - it’s only true there if the creator of the file ended with a blank line (which you’re supposed to do, but people frequently don’t bother).

I'm talking about the last character of every line, not the last line of every file

QuarkJets
Sep 8, 2008

worms butthole guy posted:

Hey all, I have another simple question that i'm hoping someone can point me in the right direction.

How do you test if a JSON key exists in Python and if not, move on?


So I am using Flask Mail to send out a email. It looks like this:


code:
f"""
Reservation Process: {HubspotData['properties']['reservation_process']['value']}
Pool: {HubspotData['properties']['pool']['value']}
"""
The problem is, if someone didn't fill out that field, the app will crash instead of just continuing on. Is there a way to rectify this? With PHP and Javascript I can use something like:

code:
"Reservation Process:{HubspotData['properties']['reservation_process'] ? {HubspotData['properties']['reservation_process']['value']} : null}"
Thanks!

You can always suppress an exception:

Python code:
from contextlib import suppress

with suppress(KeyError):
    # Do something
Or if you still want the code to execute, but you want an invalid key to return a default value, then you can use .get():

code:
"Reservation Process:{HubspotData['properties']['reservation_process'].get('value')}"
(dict.get() will return None if the provided key does not exist, or it will return whatever you specify as the second argument. This means you can chain together nested dictionary access to always return something: some_dict.get(some_key1, {}).get(some_key2, {}).get(some_key3) will return None if any of the keys don't exist)

QuarkJets fucked around with this message at 18:03 on Apr 27, 2022

Armitag3
Mar 15, 2020

Forget it Jake, it's cybertown.


worms butthole guy posted:

Hey all, I have another simple question that i'm hoping someone can point me in the right direction.

How do you test if a JSON key exists in Python and if not, move on?


So I am using Flask Mail to send out a email. It looks like this:


code:
f"""
Reservation Process: {HubspotData['properties']['reservation_process']['value']}
Pool: {HubspotData['properties']['pool']['value']}
"""
The problem is, if someone didn't fill out that field, the app will crash instead of just continuing on. Is there a way to rectify this? With PHP and Javascript I can use something like:

code:
"Reservation Process:{HubspotData['properties']['reservation_process'] ? {HubspotData['properties']['reservation_process']['value']} : null}"
Thanks!

When you say a JSON key in python do you mean a dict? Assuming HubspotData is a dict (or at least easily convertible to one) the preferred way to look-before-getting a key is HubspotData.get(“key”, default_value).

If expressions (the ternary operator equivalent in python) will also work with a caveat: if the value is falsy you won’t get what you expect.

ef;b

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Yeah, I always forget about dict.get(). Looking at it again, it's a much simpler way to do it than ternary operators, and what Armitag3 means by "falsy" is that if your dict value exists but is a value that bool() would interpret as False, like 0, it'll break.

Python code:
f"""
Reservation Process: {HubspotData['properties']['reservation_process'].get('value', 'Not Selected')}
Pool: {HubspotData['properties']['pool'].get('value', 'Not Selected')}
"""

DoctorTristan
Mar 11, 2006

I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

QuarkJets posted:

I'm talking about the last character of every line, not the last line of every file

Let me rephrase.

If the file ends with a blank line then every line ends with a newline character.

If the file does not end with a blank line, then the last line does not end with a newline character, so the assumption is false.

ExcessBLarg!
Sep 1, 2001

worms butthole guy posted:

How do you test if a JSON key exists in Python and if not, move on?
What do you want to do when you come across an invalid form? Send the email but with the fields replaced with a default value, or not send an email since the underlying data is malformed?

If the latter, I'd put the entire process-email loop in a try/except statement so that if composing the email blows up for any reason, you move on to the next email.

If the former, I'd write a dig function (inspired by Ruby's dig methods) to return a default value when any part of your data structure is missing during traversal:
Python code:
def dig(data, identifiers, default):
   try:
        for i in identifiers:
            data = data[i]
        return data
   except LookupError:
       return default
From there you can do a lookup like:
Python code:
dig(HubspotData, ('properties','reservation_process','value'), "No reservation process.")
glom is an existing module that provides similar functionality that you can probably make do the same thing.


D34THROW posted:

It looks like the PHP/JS thing you're talking about is similar to an if/else comprehension.
Just a heads up, these are called conditional expressions which are distinct from the sub-expressions used in comprehensions.

12 rats tied together
Sep 7, 2006

If this logic will result in "the body of an email" I would probably use a jinja2 template for it, which would let you more cleanly express "this line of the email should contain this text, or be empty".

Using a templating tool to render a document, like an email, tends to result in more maintainable and community-understandable code in the long term, IME.

Data Graham
Dec 28, 2009

📈📊🍪😋



punk rebel ecks posted:

You lost me here. The tutorial has no "requirements.txt".

Also how do I run my python file from the virtual environment? Navigate to it's folder via the terminal?

I wish there was a way I could do all this from VSC. :(

Sorry, I made a bad assumption. Not all projects have a requirements.txt, and a tutorial like this won't unless it shows you how to create one. It's just a way of managing your dependencies. I was using it as a way to illustrate that if you install any python libraries using pip (like flask for example), they'll be installed into your virtualenv, and they won't affect your OS Python or any other virtualenvs you might have created (i.e. for other projects; for context I have about 80 virtualenvs in my dev folder, all with different versions of Python and different sets of libs installed).

As for running your .py file -- once you've activated the virtualenv, just running "python" automatically uses the virtualenv's isolated python. So assuming flask_tutorial.py is in your current directory (your project dir, the one that has the venv in it), just go like

code:
$ python flask_tutorial.py
And it should work. You don't need to navigate into or interact with the venv directory at all except to activate it.

QuarkJets
Sep 8, 2008

DoctorTristan posted:

Let me rephrase.

If the file ends with a blank line then every line ends with a newline character.

If the file does not end with a blank line, then the last line does not end with a newline character, so the assumption is false.

Then it's a good thing that I used words like "usually" and "pretty safe assumption" to describe something that I already acknowledged is usually but not always true :shrug:

12 rats tied together
Sep 7, 2006

punk rebel ecks posted:

You lost me here. The tutorial has no "requirements.txt".

Also how do I run my python file from the virtual environment? Navigate to it's folder via the terminal?

I wish there was a way I could do all this from VSC. :(

Other people have been doing a good job of fielding these questions, but I wanted to try and provide a zoomed-out explanation for you real quick because virtualenvs are one of those things that are easiest to understand if you start at the bottom (IMO), but it's hard to find resources online that actually start at the bottom and don't presume some foreknowledge on your part.
  • In order to run python stuff, your computer needs to be able to know where "python.exe" is (assuming you're on windows).
  • Your python.exe is preconfigured to look for python-related files inside a certain set of directories, some more info on this can be found here, but you don't have to fully understand it.
  • One of the folders your python is configured to look in is called "site-packages". Some info here, but it's not all the info and you don't have to fully understand it either.
  • When you "pip install something", a folder named "something" will be downloaded to your site-packages folder.
  • When you "import something", python scans your site-packages folder for a folder named "something", and then tries to run an __init__.py that exists inside that folder.
A virtualenv is like if you made isolated copies of these site-packages folders and swapped them around behind the scenes. In order to "use" a virtualenv, then, you need to tell Python about it in some way, so it knows to look in one site-packages instead of another. On the shell you do this by "activating" it.

VS Code is slightly more complicated and your best bet will be to read the documentaton for the python extension, which explains how VS Code finds your environments and how it decides which one to activate for you.

You could also do it by hand instead, and move these directories around yourself, but that would suck and you'd probably want to script it.

12 rats tied together fucked around with this message at 20:22 on Apr 27, 2022

Armitag3
Mar 15, 2020

Forget it Jake, it's cybertown.


12 rats tied together posted:

Other people have been doing a good job of fielding these questions, but I wanted to try and provide a zoomed-out explanation for you real quick because virtualenvs are one of those things that are easiest to understand if you start at the bottom (IMO), but it's hard to find resources online that actually start at the bottom and don't presume some foreknowledge on your part.
  • In order to run python stuff, your computer needs to be able to know where "python.exe" is (assuming you're on windows).
  • Your python.exe is preconfigured to look for python-related files inside a certain set of directories, some more info on this can be found here, but you don't have to fully understand it.
  • One of the folders your python is configured to look in is called "site-packages". Some info here, but it's not all the info and you don't have to fully understand it either.
  • When you "pip install something", a folder named "something" will be downloaded to your site-packages folder.
  • When you "import something", python scans your site-packages folder for a folder named "something", and then tries to run an __init__.py that exists inside that folder.
A virtualenv is like if you made isolated copies of these site-packages folders and swapped them around behind the scenes. In order to "use" a virtualenv, then, you need to tell Python about it in some way, so it knows to look in one site-packages instead of another. On the shell you do this by "activating" it.

VS Code is slightly more complicated and your best bet will be to read the documentaton for the python extension, which explains how VS Code finds your environments and how it decides which one to activate for you.

You could also do it by hand instead, and move these directories around yourself, but that would suck and you'd probably want to script it.

Good info here.

I use VSCode personally and can expand a bit here. To activate a virtual environment in vscode you need the python extension installed. Then CMD/Ctrl + Shift + P to pull up the command palette and select Python: Select Interpreter… then open your virtual environment which should be listed there. If it isn’t try the Developer: Reload Window command first from the command palette.

Your vscode should now recognise your virtual env and give you nice autocomplete etc for your locally installed packages.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Always fun to be reminded that str.center() exists :shepicide:

Version A
Python code:
def center_text_on_terminal(text: str, padding: str = " ") -> str:
    # Get the terminal size.
    term_size = get_terminal_size().columns
    # Pad the text and determine the width.
    text = f" {text} "
    width = len(text)
    is_even = width % 2 == 0
    if is_even:
        pad_left = pad_right = int((term_size - width) / 2)
    else:
        pad_left = int((term_size - (width - 1)) / 2)
        pad_right = int((term_size - width) / 2)

    return f"{padding * pad_left}{text}{padding * pad_right}"
Version B
Python code:
def centered_on_terminal(text: str, padding: str = " ") -> str:
    return f" {text} ".center(get_terminal_size().columns, padding)
I've removed the docstrings for brevity.

As an aside, from you more experienced day-to-day Python dev out there, how readable is my code that you've seen? I run black and flake8 and try to be as descriptive as I can; I'm not comfortable sharing the github quite yet but I'm sure I will at some point.

worms butthole guy
Jan 29, 2021

by Fluffdaddy
Wow thanks everyone for the tips. I ended up making a function that runs a dict.get on each property and if it finds it it returns the property otherwise it returns empty spaces. Which I believe should work fin!

Thanks all

lazerwolf
Dec 22, 2009

Orange and Black

D34THROW posted:

Always fun to be reminded that str.center() exists :shepicide:

Version A
Python code:
def center_text_on_terminal(text: str, padding: str = " ") -> str:
    # Get the terminal size.
    term_size = get_terminal_size().columns
    # Pad the text and determine the width.
    text = f" {text} "
    width = len(text)
    is_even = width % 2 == 0
    if is_even:
        pad_left = pad_right = int((term_size - width) / 2)
    else:
        pad_left = int((term_size - (width - 1)) / 2)
        pad_right = int((term_size - width) / 2)

    return f"{padding * pad_left}{text}{padding * pad_right}"
Version B
Python code:
def centered_on_terminal(text: str, padding: str = " ") -> str:
    return f" {text} ".center(get_terminal_size().columns, padding)
I've removed the docstrings for brevity.

As an aside, from you more experienced day-to-day Python dev out there, how readable is my code that you've seen? I run black and flake8 and try to be as descriptive as I can; I'm not comfortable sharing the github quite yet but I'm sure I will at some point.

I’ll be honest, reading code posted here isn’t the best. GitHub would be the best because of proper formatting line numbers syntax highlighting etc.

necrotic
Aug 2, 2005
I owe my brother big time for this!
Python code:
def post(good):
    return False
You can syntax highlight on the forums I think? No line numbers though

edit: okay maybe not, or awful app doesn’t do it

New Zealand can eat me
Aug 29, 2008

:matters:


D34THROW posted:

As an aside, from you more experienced day-to-day Python dev out there, how readable is my code that you've seen?

As someone who used to write a lot of python back when django was new, did other things for a great while, and is now back to writing python sometimes: your code is perfectly fine dude

I went back through the posts here to try and find something to nitpick and everything seems fine. FWIW I find myself doing the same thing pretty often, writing things 'the long way' and then finally finding the one function that does everything and replacing it all

Sucks that phone apps don't syntax highlight, mobile web might do the trick in a pinch.

Fender
Oct 9, 2000
Mechanical Bunny Rabbits!
Dinosaur Gum

D34THROW posted:

As an aside, from you more experienced day-to-day Python dev out there, how readable is my code that you've seen? I run black and flake8 and try to be as descriptive as I can; I'm not comfortable sharing the github quite yet but I'm sure I will at some point.

This is probs more specific to my team/company, but we'd want more types explicitly stated. Most of your stuff in that example is pretty obvious, but something like the following could be an integer, but it could easily be a tuple (x,y) or a list (the plural makes it seem possible). The general rule here is that if you can't tell what something is on the line where it is declared, then you should add a type. You can still pretty easily suss out that it's an integer by reading on for a bit, but it's nicer for other devs to just have it stated explicitly.

Python code:
term_size = get_terminal_size().columns
We're hyper-aware of this sort of stuff because we write a lot of fragile/tightly-coupled integration code that breaks easily when other platforms change. So hotfixing, and thus reading someone else's code, is a constant duty. But it motivates us to write code that is as easy as possible for other devs to sort out as quickly as possible.

worms butthole guy
Jan 29, 2021

by Fluffdaddy
Is Selenium the best web automation package? I'm looking at making a little thing that lets me log into a site, grab a generated report as a excel sheet and then use that data to send to a API?

credburn
Jun 22, 2016
A tangled skein of bad opinions, the hottest takes, and the the world's most misinformed nonsense. Do not engage with me, it's useless, and better yet, put me on ignore.
EDIT:

Sorry, I figured it out, I think! I don't want to leave this question and a big block of code up, so as not to confuse anyone passing by...

credburn fucked around with this message at 22:24 on Apr 29, 2022

ExcessBLarg!
Sep 1, 2001
No, the print function takes an optional "file" keyword argument. You can pass a file object (from open) to this and reuse your code.

credburn
Jun 22, 2016
A tangled skein of bad opinions, the hottest takes, and the the world's most misinformed nonsense. Do not engage with me, it's useless, and better yet, put me on ignore.

ExcessBLarg! posted:

No, the print function takes an optional "file" keyword argument. You can pass a file object (from open) to this and reuse your code.

Oh, thanks!

Dangit, I was hoping to edit my post before anyone responded to it. I just changed the print() functions to file.open() and .write() to accomplish effectively the same thing as just printing it. I wasn't aware of what you were talking about though, which seems like a more efficient way of doing it.

ExcessBLarg!
Sep 1, 2001
You can post code in the thread, that's the point of it.

ExcessBLarg! fucked around with this message at 12:12 on Apr 30, 2022

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
poo poo, half of my posts are screen length blocks of lovely code

worms butthole guy
Jan 29, 2021

by Fluffdaddy
I have a question about what the best "stack" would be to do this stupidly simple task.

I want to write a little program that reads from an API and then shows that data, refreshing itself every few minutes. The catch is during this refresh period, I want to play a video. I guess it's basically a digital sign, but I want to make it by hand.

I figured since i'm not doing anything super advanced i'd just use Flask for the backend to retrieve the data, but being "new" to Python (but not programming; I usually program in PHP), what should I use for the front end to accomplish this? I know React and Vue and can use those but those seem maybe a bit excessive.

Anyways I appreciate the help. Thank you!

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug

worms butthole guy posted:

I have a question about what the best "stack" would be to do this stupidly simple task.

I want to write a little program that reads from an API and then shows that data, refreshing itself every few minutes. The catch is during this refresh period, I want to play a video. I guess it's basically a digital sign, but I want to make it by hand.

I figured since i'm not doing anything super advanced i'd just use Flask for the backend to retrieve the data, but being "new" to Python (but not programming; I usually program in PHP), what should I use for the front end to accomplish this? I know React and Vue and can use those but those seem maybe a bit excessive.

Anyways I appreciate the help. Thank you!

Depending on how much you want to try a new tech, I've been seeing HTMX a lot lately as a JS-less (but built in JS) library that seems to be a nice lightweight alternative to vue/react/etc. I haven't tried it myself though, and I'm curious how other view it, but the videos/etc seem pretty promising.

edit: here's a video example that seems to at least be somewhat near what you're looking at :
https://www.youtube.com/watch?v=kNXVFB5eQfo&t=1079s

Falcon2001 fucked around with this message at 23:40 on May 2, 2022

worms butthole guy
Jan 29, 2021

by Fluffdaddy

Falcon2001 posted:

Depending on how much you want to try a new tech, I've been seeing HTMX a lot lately as a JS-less (but built in JS) library that seems to be a nice lightweight alternative to vue/react/etc. I haven't tried it myself though, and I'm curious how other view it, but the videos/etc seem pretty promising.

edit: here's a video example that seems to at least be somewhat near what you're looking at :
https://www.youtube.com/watch?v=kNXVFB5eQfo&t=1079s

Huh this actually looks pretty rad. Thank you, i'll check this out.

A Strange Aeon
Mar 26, 2010

You are now a slimy little toad
The Great Twist
I am extremely new to coding as will no doubt be evident, but I had a conceptual question for a project I'm working on.

Project: There used to be a wiki site that you could create random generators on, and it was very WYSIWYG, super easy to translate your idea into a random table, and link other pages to your page, so if someone had already made a "Weird Smells" random generator and you wanted to include an adjective from that page in one of your generators, it was super easy.

Then I guess an updated version of PHP and/or Mediawiki broke the site owner's custom extension that powered the random generator stuff, and the site has been pretty much broken ever since. The pages are all still there, but it doesn't work how it's supposed to. Link to site: http://www.random-generator.com/index.php?title=Main_Page

So, I had put a bunch of content into one of the pages several years ago, and now want to salvage it somehow. I'd read Python is a good beginner language, so I've made a bit of progress, but running into a snag.

The page was originally built with high level categories like Urban, Wilds, Dungeon, etc., which then called sub tables which called other sub tables, etc.

I took the source code from my page on the wiki site, which has a layout like this:

;famous_goons #name of the 'table'; called by [famous_goons]
5,Lowtax #the first number is the weight, followed by the entry itself
1,Fifty Foot Ant

Some are nested, though, so picture:

;origin_builder
1, [heroes_and_villains]
1, the [ancientpeople]
1, [urbn_subjectplural]
1, a fabulously wealthy [profession]

And then the high level ones can be even more nested, like:

;Urban
20,there's [urbn_place_adj] [urbn_place] [%sign] [%smellslike] [urbn_locn]. It's said that here, [urbn_subjectverb]

What I've managed to do is build a dictionary from a text file of the wiki page content that has each 'table' name as the key, followed by a list of lists, each containing the weight and then the entry. I'm not even worrying about the weights yet, but using this dictionary, I can sort of see how I could recreate this. But it seems like it will be an insane amount of work so I was curious if there was a better way or I'm just doing this really stupidly.

Code:

foreign_adj1 = dict["%foreign_adj"]
foreign_adj2 = random.choice(foreign_adj1) # random unweighted value from foreign_adj key

I have a similar set up for the ~250 keys in the dictionary.

Here's where I need to know if I'm even on the right track or not:

if '[%foreign_adj]' in foreign_phrase2[1]:# membership check, if exact string is in the phrase value
foreign_adj3 = random.choice(foreign_adj1) # random unweighted value from foreign_adj key, different random from adj2
foreign_phrase3 = foreign_phrase2[1].replace('[%foreign_adj]',foreign_adj3[1]) # set variable to hold new replaced str
inflector_point2 = "Quick test for " + inflector.a(foreign_adj2[1]) + " Hanover" + foreign_phrase3
print(inflector_point2)
elif '[%herdwith]' in foreign_phrase2[1]:
herdwith3 = random.choice(herdwith1)
foreign_phrase3 = foreign_phrase2[1].replace('[%herdwith]',herdwith3[1])
inflector_point2 = "Quick test for " + inflector.a(foreign_adj2[1]) + " Hanover" + foreign_phrase3
print(inflector_point2)
elif '[origin_builder]' in foreign_phrase2[1]:
origin_builder3 = random.choice(origin_builder1)
foreign_phrase3 = foreign_phrase2[1].replace('[origin_builder]',origin_builder3[1])
inflector_point2 = "Quick test for " + inflector.a(foreign_adj2[1]) + " Hanover" + foreign_phrase3
print(inflector_point2)
else:
inflector_point1 = "Quick test for " + inflector.a(foreign_adj2[1]) + " Hanover" + foreign_phrase2[1]
print(inflector_point1)

This will mostly do the output I want--printing stuff like:

"Quick test for an eerie Hanover where the birds do not sing" -- no nested replacements from foreign_phrase

"Quick test for a low Hanover where women with eyes strange and full of power are revered" -- replaces "where women[%herdwith] are revered" from the foreign_phrase table with "where women with eyes strange and full of power are revered", pulling from the herdwith table.

"Quick test for a bleak Hanover beyond the unhappy sea" -- replaces "beyond the [foreign_adj] sea" from foreign_phrase table with "beyond the unhappy sea" and has a distinct result from the first foreign_adj (bleak)

and I think I could possibly manage to figure out this one too, though I haven't yet:

"Quick test for a bleak Hanover where the [ancientpeople] dwelt in former days" -- replaces "where [origin_builder] dwelt in former days" with a valid value from the origin_builder table, "the [ancientpeople]", but that value happens to include a call to the ancientpeople table, which I didn't include in the if statement.

But confronted with this, and knowing that some of these have 6+ different tables being called, which may include strings with calls to yet other tables, I can't figure out if this is the right approach or not--it seems like it would require a TON of busy work entering in the different tables and then including dozens of clauses in if statements, and the more complicated ones, I'm not even sure how to structure exactly.

To get the basic variables for each key, I did something simple:

for x in get_keys:
print(x + '1 = dict["'+ x +'"]')
print(x + '2 = random.choice(' + x + '1)')

That spit out

foreign_adj1 = dict["%foreign_adj"]
foreign_adj2 = random.choice(foreign_adj1)

for every key.

Is it even possible to use that same idea to cycle through the dictionary and end up printing the complete if and elif statements I'd need, for every key and searching for substrings that contain calls to other keys? Or is this just completely the wrong approach here?

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Rather than writing separate code for every possible replacement, you should aim to write one bit of code that works for everything.

A good place to start would be writing code that works out if there is any [text in brackets] in the string so far.

Then you can write code that expands the [text in brackets] based on what you've defined in your input dictionary. Note that this doesn't need any replacement-specific code - you're just snipping out one bit of text and putting a new one in its place.

Then you can repeat that whole process until there isn't any more [text in brackets] left.

QuarkJets
Sep 8, 2008

A Strange Aeon posted:

Is it even possible to use that same idea to cycle through the dictionary and end up printing the complete if and elif statements I'd need, for every key and searching for substrings that contain calls to other keys? Or is this just completely the wrong approach here?

You're on the right track, in that you've identified that A) this is tedious and B) it can be fixed. You can probably just define a function that can be invoked recursively to figure out and perform all of the necessary replacements

Let's first talk about your data structures. Your approach is sound and very beginner-intuitive; group lists of data as lists, mappings of data as dicts, etc. This is extremely flexible, so it's very easy to just build something that works. Your code is revealing some of the flaws in this approach though; it's hard to read and debug, there are magic numbers all over the place, and it's kind of brittle. As a first step, let's learn about dataclasses by converting one of these lists into one:

Python code:
from dataclasses import dataclass
...  # I'm just using these dots to denote space; put your imports at the top of your code

@dataclass
class Entry:
    weight: int
    name: str

all_tables = {'famous_goons': [Entry(5, 'Some Goon'), Entry(10, 'Some Other Goon')]}
for goons in all_tables['famous_goons']:
     print(goons.name, goons.weight)
This should illustrate what I'm talking about; instead of a dict of lists of lists, this is now a dict of lists of Entries. Now instead of accessing the name of an entry with [1] and having to know what that index corresponds to for that object, you'd use .name. This illustrates a good concept to learn: try to avoid directly indexing into stuff (e.g. some_variable[1]), use iterators or attributes instead. Sometimes it's unavoidable, but it's a good thing to check for

Now you're putting all of these entry lists into a dictionary, right? And you want to iterate through some string, and perform all of the necessary replacements, including nested replacements? Maybe something like this would do:

Python code:
from dataclasses import dataclass
import random
import re


@dataclass
class Entry:
    weight: int
    name: str


def return_new_random_phrase(table_map, table_requested):
    """Return a random phrase from a specific table.

    Args:
        table_name (dict): Mapping of table names to lists of entries
        table_requested (str): A specific table name

    Returns:
        phrase (str): A random phrase from the requested table
    """
    # I am going to add overly verbose comments from here.
    # Comments should be much rarer than this; try to let your code describe itself.
    # Start with an empty string, which will serve as our default return value.
    # We don't have to do it this way, we could just use multiple return statements. That's fine too.
    phrase = ''
    # Is the table name in the requested dictionary? If not, we'll hit "return phrase". Otherwise...
    if table_requested in table_map: 
        # Grab the list of entries corresponding to the requested table name
        all_entries = table_map[table_requested]
        # Grab a random entry from the list of entries.
        # I was going to use the same function that you chose, but noticed it's not using the weights.
        # random.choices lets you provide weights, you'd just need to extract them first.
        # You could do that with a list comprehension:
        entry_weights = [entry.weight for entry in all_entries]
        phrase = random.choices(all_entries, weights=entry_weights)[0].name
        # Now we should to see if any further replacements are needed.
        # Find all words that are surrounded by square brackets
        bracketed_phrases = re.findall('\[.*?\]', phrase )  # SCARY REGEX
        for sub_phrase in bracketed_phrases:
            # Each discovered word is just another reference to our tables, yes?
            # That means we can just call this same recursively:
            replacement_phrase = return_new_random_phrase(table_map, sub_phrase.replace('[', '').replace(']', ''))
            phrase = phrase.replace(sub_phrase, replacement_phrase)
    # All random phrase requests should be satisfied now; return the result
    return phrase


# Let's try it out
# some data I made up:
all_tables = {'famous_goons': [Entry(5, 'Some Goon'), Entry(10, 'Some [evil adjectives] poster')],
              'evil adjectives': [Entry(100, 'rear end in a top hat'), Entry(200, 'fuckwad'), Entry(300, 'misbegotten [idiot names]')],
              'idiot names': [Entry(5, 'idiot'), Entry(10, 'fool'), Entry(20, '[piss drinkers]-like piss drinker')],
              'piss drinkers': [Entry(100, 'Lowtax'), Entry(200, 'Trump'), Entry(500, 'Rush Limbaugh')]
}

# Let's get a random famous goon. 
print(return_new_random_phrase(all_tables, 'famous_goons'))
# Here's the results from a few trials:
# 'Some misbegotten Rush Limbaugh-like piss drinker poster'
# 'Some misbegotten Lowtax-like piss drinker poster'
# 'Some Goon'
# Seems to work
That's like 50% comments and it's the entirety of the code needed to actually do all of the replacements, but it only covers square bracket cases. Maybe your data will need further modifications, but you get the idea; we've defined a function that replace certain strings with random selections of other strings based on the contents of some dictionary, and it does so recursively such that all square-bracketed phrases eventually get replaced (possibly with blank strings, if no such table name is in the dictionary). We cleaned up the data schema a little bit by incorporating iterators and a dataclass.

I also used an f-string, which is something you should try reading about. This is the sexy new (new-ish) way to do string concatenation
Python code:
# Old and busted, don't do this:
new_string = string1 + 'foo bar' + str(some_int)
# Newer and better, but don't do this either:
new_string = '{} foo bar {}'.format(string1, some_int)
# Always use f-strings:
new_string = f'{string1} foo bar {some_int}'

QuarkJets fucked around with this message at 00:47 on May 7, 2022

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
I adore f strings and use them whereever possible. The lone exception is if I have something where I'd want a preformatted string with placeholders, to use in several places, with .format().

Adbot
ADBOT LOVES YOU

lazerwolf
Dec 22, 2009

Orange and Black

D34THROW posted:

I adore f strings and use them whereever possible. The lone exception is if I have something where I'd want a preformatted string with placeholders, to use in several places, with .format().

I feel it’s still more readable to just repeat the variable in the f string than to have to look at which variable in format()

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply