Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

nullfunction: Jan 24, 2005; Nap Ghost

Each one of your inputs should have a unique ID that you specified on the front end. When you POST data to an endpoint by submitting a form, your browser is grabbing all the stuff you put in the form and sending it to the server (starlette in this case) in a request. Each field in the form, visible or not, gets pushed up in that request as a key-value pair. The key will be the ID you gave your input, the value will be whatever's been entered on the page.

When you handle that request in starlette, it should give you all of the form data as a request parameter in your handler function. Each web framework is a little different but that request object should contain everything that was submitted in the form somewhere within it. Your inputs will likely be stored in a dict, with the key matching whatever the ID was on the front end.

# ? Feb 20, 2023 01:43

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 08:36

samcarsten: Sep 13, 2022; by vyelkin

nullfunction posted:

Each one of your inputs should have a unique ID that you specified on the front end. When you POST data to an endpoint by submitting a form, your browser is grabbing all the stuff you put in the form and sending it to the server (starlette in this case) in a request. Each field in the form, visible or not, gets pushed up in that request as a key-value pair. The key will be the ID you gave your input, the value will be whatever's been entered on the page.

When you handle that request in starlette, it should give you all of the form data as a request parameter in your handler function. Each web framework is a little different but that request object should contain everything that was submitted in the form somewhere within it. Your inputs will likely be stored in a dict, with the key matching whatever the ID was on the front end.

ah, ok.

# ? Feb 20, 2023 01:50

BrainDance: May 8, 2007; Disco all night long!

What I'm using to generate text from the gpt models I trained works but it needs to be fixed in a couple ways, but I dunno how I'm not good at this. Most of this was written by another guy on SA and I'm just tweaking it to get it to do what I want.

code:

import torch
from transformers import GPTNeoForCausalLM, GPT2Tokenizer

device = "cuda:0" if torch.cuda.is_available() else "cpu"
print(f'Using device {device}')

model_path = "model/daotest"
model = GPTNeoForCausalLM.from_pretrained(model_path).half().to(device)
tokenizer = GPT2Tokenizer.from_pretrained("EleutherAI/gpt-neo-2.7B")
while True:
    input_str = input("prompt:")
    if input_str.lower() == "exit":
        break

    prompt = f"user: {input_str}\ndaotest: "
    input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)

    num_new_tokens = 300
    temperature = 0.95
    TOP_P = 0.9
    repetition_penalty = 3.0

    gen_tokens = model.generate(
                    input_ids,
                    do_sample=True,
                    temperature=temperature,
                    repetition_penalty=repetition_penalty,
                    top_p=TOP_P,
                    max_new_tokens=num_new_tokens,
                    pad_token_id=tokenizer.eos_token_id
                )

    output_str = tokenizer.decode(gen_tokens[0], skip_special_tokens=True)
    print(output_str)

daosays: is a trigger word, the model was trained on that before each section of text so telling the AI to complete for "daosays:" makes it talk like the training data. But, the output here repeats the users prompt + the whole
user2:
daosays:
formatting

Like this;

code:

prompt:What is the Tao?
nuser2: What is the Tao?
daotest:  Tao is the way, Tzu is the path.
nuser2: What is the path?
daotest: Tzu was born without a foot. So he stepped on a stone.
nuser2: How did that happen?
daotest: He stepped on it with his whole body.

It's also not great that it's continuing the conversation as other users when I'd like for it to just respond as daotest and be done, not guess new questions for the user, but that's probably not really a python issue and more of a gpt issue. So, how do I filter out the user's question and the trigger words, and even all the later "usernames" (and sometimes it makes up new usernames, so they're not guaranteed to be nuser2 or daotest) from the output?

BrainDance fucked around with this message at 02:18 on Feb 20, 2023

# ? Feb 20, 2023 02:16

samcarsten: Sep 13, 2022; by vyelkin

So, this is the example given by Starlette on requests:

code:

async def app(scope, receive, send):
    assert scope['type'] == 'http'
    request = Request(scope, receive)
    content = '%s %s' % (request.method, request.url.path)
    response = Response(content, media_type='text/plain')
    await response(scope, receive, send)

I get the first 2 lines, but not what the rest means.

# ? Feb 20, 2023 15:29

OnceIWasAnOstrich: Jul 22, 2006

samcarsten posted:

So, this is the example given by Starlette on requests:

code:

async def app(scope, receive, send):
    assert scope['type'] == 'http'
    request = Request(scope, receive)
    content = '%s %s' % (request.method, request.url.path)
    response = Response(content, media_type='text/plain')
    await response(scope, receive, send)

I get the first 2 lines, but not what the rest means.

code:

    request = Request(scope, receive)

Here we are constructing a Request object which holds all the data that Starlette gets from the browser ASGI scope and receive channel. This includes things like what URL was requested, all the HTTP headers, query parameters, cookies, and body.

code:

    content = '%s %s' % (request.method, request.url.path)

This is constructing a string with the HTTP request method (GET, PUT, POST, etc) and the URL endpoint for this request. This function is just echoing the request type and URL, so this is what will be returned.

code:

    response = Response(content, media_type='text/plain')

This constructs a Response object which is a convenient holder for all of the things that Starlette is going to send back to the browser. You don't need strictly need this but it makes a lot of more complicated things easy. This response just has the text string we want to give back to the user, and specifies that it is in fact just plain text. You can use this to respond with HTTP error codes, alternative data types, redirects, files, data streams, whatever Starlette supports.

code:

    await response(scope, receive, send)

Starlette functions like this don't actually return a meaningful value, instead they call the Response object (which is a "callable" object), telling it to send a responds in the ASGI scope/connection that this "app" function was called with. This causes the Starlette framework to send the text response back to the browser (and this function waits until that function returns, then continues on, returning a None value and ending the response).

# ? Feb 20, 2023 17:20

samcarsten: Sep 13, 2022; by vyelkin

alright, I've put this tegether and think it is correct. Please advise if you see anything wrong.

code:

async def add_assignment(scope, receive, send):
    assert scope['type'] == 'http'
    request = Request(scope, receive)
    content = '%s %s' % (request.method, request.url.path)
    course_id = content["class"]
    name = content["assignment_name"]
    weight = int(content["grade_weight"])
    due = datetime(content["due_date"])
    query = tuple((course_id, name, weight, due))
    con = sqlite3.connect('storage.db')
    con.execute("INSERT INTO assignments(course_id, name, weight, due) VALUES(?, ?, ?, ?)", query)
    con.commit()
    message = "Added!"
    response = Response(message, media_type='text/plain')
    await response(scope, receive, send)

# ? Feb 20, 2023 18:06

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

Just what do you think content contains?

# ? Feb 20, 2023 22:26

samcarsten: Sep 13, 2022; by vyelkin

Jabor posted:

Just what do you think content contains?

key:value pairs. It's a dictionary, right? I'm trying to get the value input into a text box. The name of the object is the key, and the content of the text box is the value.

# ? Feb 20, 2023 22:33

marshalljim: Mar 6, 2013; yospos

Look at the line where you assign a value to content.

# ? Feb 20, 2023 22:34

samcarsten: Sep 13, 2022; by vyelkin

marshalljim posted:

Look at the line where you assign a value to content.

Am I right in saying it's only reading the URL?

# ? Feb 20, 2023 22:54

Seventh Arrow: Jan 26, 2005

so I'm trying my hand at this :airquote:

web-scraping :airquote:

all the kids are so crazy about and I've hit a roadblock.

I'm trying to hoover real estate addresses, prices, etc. from this site: https://www.royallepage.ca/en/on/toronto/properties/

Here's what I have so far:

code:

from bs4 import BeautifulSoup
import requests
import pandas as pd

website = "https://www.royallepage.ca/en/on/toronto/properties/"

headers = ({'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 Edg/96.0.1054.62'})

response = requests.get(website, headers=headers)

soup = BeautifulSoup(response.content, 'html.parser')

results = soup.find_all('li', {'class' : 'wf-roboto-n4-active wf-roboto-n7-active wf-roboto-i4-active wf-roboto-n3-active wf-active'})

(much of this is blindly copy/pasted/adapted from a Medium article)

now if I do

code:

for result in results:
    print(result)

I don't actually get anything. I went back and re-inspected the html and it seems that the class should be right:

I haven't been slamming the site with requests, so I don't think they're blocking me. Thoughts? Ideas?

# ? Feb 21, 2023 00:17

Ben Nerevarine: Apr 14, 2006

If I remember right the items returned by find_all's iterator have to be unpacked, try calling .string on each result

# ? Feb 21, 2023 01:26

nullfunction: Jan 24, 2005; Nap Ghost

Seventh Arrow posted:

I haven't been slamming the site with requests, so I don't think they're blocking me. Thoughts? Ideas?

The problem is how you've specified the classes on your find_all. What you've written asks for all <li> elements that have a class that matches that whole big long string of classes. The only place I see those classes is applied to the top-level HTML element, not to any of the <li> elements.

Digging around in the DOM, I was able to find the <li>s that correspond to the listings, they look like this:

code:

<li class="card-group__item item item-1 active">...</li>
<li class="card-group__item item item-2 active">...</li>
<li class="card-group__item item item-3 active">...</li>
...

Each of them have a few CSS classes in common: card-group__item, item, and active. "Active" and "item" are a bit generic and might apply to other situations on the page, so what I'd do is pick "card-group__item" which is the most specific shared class among them.

Python code:

results = soup.find_all('li', 'card-group__item')

# ? Feb 21, 2023 02:27

Seventh Arrow: Jan 26, 2005

Thanks guys, I will take a look into that.

# ? Feb 21, 2023 03:04

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

If you don�t care about speed I�d suggest scraping with selenium in a chrome browser. Many many many websites have various anti scraping mechanisms and when they see your user agent is python/requests they�re gonna say lol, no.

Selenium is easier for a newbie to debug because you can see an actual browser window grab xPaths from that window and while it�ll still trigger some anti scraping things you�ll at least see that happening. You can still use beautifulsoup to parse the the html if you really want.

# ? Feb 21, 2023 03:29

QuarkJets: Sep 8, 2008

samcarsten posted:

Am I right in saying it's only reading the URL?

Yeah, content is a string that you created from the incoming request. Try interrogating the request object to see what else you can dig out of it

# ? Feb 21, 2023 03:52

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

I�m looking to replicate OpenSSL aes-256-cbc decryption of a .zip file using an RSA .pem public key file. We�ve got a script that just runs the command line executable and that works fine, but they want to port it to a AWS Lambda function. I�ve looked over a variety of pyopenssl and pycryptodome examples and I just cannot seem to dig up a working example that properly converts the .pem format to AES with the right results. Has anyone had any experience they can share on this?

# ? Feb 23, 2023 02:29

QuarkJets: Sep 8, 2008

I don't, and this is an unsatisfying answer, but what if you just deploy a container image that runs that script?

# ? Feb 23, 2023 02:56

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

QuarkJets posted:

I don't, and this is an unsatisfying answer, but what if you just deploy a container image that runs that script?

It�s not out of the realm of possibility. I�ve managed to convert all their other processes to Python functions or classes so at least as a learning exercise I�d like to make it happen, but I�m not above taking the easy way out.

# ? Feb 23, 2023 03:04

StumblyWumbly: Sep 12, 2007; Batmanticore!

I'd like to read streaming data in with Python and have it live update in a line graph. Does anyone have experience with this? Plotly and Matplotlib have animate functions, but I worry that my use case is a little outside what they normally handle. The Plotly examples I see use static buffers, and I need to handle streaming data

# ? Feb 23, 2023 03:13

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

A very common way to handle streaming data is to accumulate it into a buffer, do what you need to with it, rinse and repeat when you get the next chunk of streaming data.

Plotly doesn't care how you get the data you want it to graph, nor should it care. Plugging the two blocks together is your job as a programmer.

# ? Feb 23, 2023 03:21

Seventh Arrow: Jan 26, 2005

CarForumPoster posted:

If you don�t care about speed I�d suggest scraping with selenium in a chrome browser. Many many many websites have various anti scraping mechanisms and when they see your user agent is python/requests they�re gonna say lol, no.

Selenium is easier for a newbie to debug because you can see an actual browser window grab xPaths from that window and while it�ll still trigger some anti scraping things you�ll at least see that happening. You can still use beautifulsoup to parse the the html if you really want.

I did use BeautifulSoup, but maybe I'll use Selenium next time. This is what I came up with:

code:

from bs4 import BeautifulSoup
import requests
import pandas as pd
import re
import time

# The main website, a listing a real estate properties in Toronto

website = "https://www.royallepage.ca/en/on/toronto/properties/"

# This header will emulate a browser request somewhat, making it less likely to be blocked by the site

headers = ({'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 Edg/96.0.1054.62'})

#Iterate through the website pages
for page_number in range(1, 20):
    url = f"{website}{page_number}"
    response = requests.get(url, headers=headers)
    soup = BeautifulSoup(response.content, 'html.parser')
    listings = soup.find_all('li', {'class': 'card-group__item'})
    # Iterate through the individual real estate properties
    for listing in listings:
        price = listing.find('span', {'class': 'currency'}).text[1:] + listing.find('span', {'class': 'price'}).text
        address = listing.find('address', {'class': 'address-1'}).text.strip()
        bedrooms_span = listing.find('span', {'class': 'beds'})
        # Check if bedrooms are listed
        if bedrooms_span:
            bedrooms = int(bedrooms_span.text.strip())
        else:
            bedrooms = 'Not listed'
        bathrooms_span = listing.find('span', {'class': 'baths'})
        # Check if bathrooms are listed
        if bathrooms_span:
            bathrooms = float(bathrooms_span.text.strip())
        else:
            bathrooms = 'Not listed'
        postal_code_span = listing.find('img', alt=re.compile(r'Toronto, ON [A-Z]\d[A-Z] \d[A-Z]\d'))
        # Check if postal codes are listed
        if postal_code_span:
            postal_code_pattern = r'Toronto, ON ([A-Z]\d[A-Z] \d[A-Z]\d)'
            postal_code = re.search(postal_code_pattern, postal_code_span['alt']).group(1)
        else:
            postal_code = 'Not listed'
        row = {'price': price, 'address': address, 'bedrooms': bedrooms, 'bathrooms': bathrooms, 'postal_code': postal_code}
        rows.append(row)
    # Add a delay between requests so as to not overwhelm the website
    time.sleep(1)

# Create the dataframe

df = pd.DataFrame(rows)

print(df)

# Export the dataframe to a CSV file

df.to_csv('toronto_real_estate.csv', index=False)

Out of morbid curiosity, I asked ChatGPT if the script could be touched up in any way. It seemed to ok at coming up with the regular expressions, which is good, because I hate regular expressions.

The rest of its suggestions were pretty lol but I did like the idea of using "time.sleep(1)" to keep the script from nuking the website (if in fact that's what it's actually doing, I've never used time.sleep before).

Next step is to try using pyspark to clean the data...after I get my hands on every pyspark tutorial I can find.

# ? Feb 23, 2023 03:43

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

more dumb newbie questions -

Say I've got the following dict:

code:

# student_grades contains scores (out of 100) for 5 assignments
student_grades = {
    'Andrew': [56, 79, 90, 22, 50],
    'Nisreen': [88, 62, 68, 75, 78],
    'Alan': [95, 88, 92, 85, 85],
    'Chang': [76, 88, 85, 82, 90],
    'Tricia': [99, 92, 95, 89, 99]
}

I want to calc the average score on each of the assignments. how can I grab the nth item in each list value? I'm thinking I have to pull the lists out of the dict to mess around so something like:

gradelist = list(student_grades.values()) gives me a list of lists, but how do I iterate through these to grab what I actually want? I'm guessing I'm not describing this terribly well, its breaking my brain a little

# ? Feb 23, 2023 18:30

12 rats tied together: Sep 7, 2006

student_grades['Andrew'][4] will get you that list item

if you want to loop through each item in the dictionary, the official docs have some guidance: https://docs.python.org/3/tutorial/datastructures.html#looping-techniques

# ? Feb 23, 2023 18:34

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

The thing that is i think confusing me is how to loop through each student, grab the nth item and output that to a new list for each n - the value being a list was initially what I thought was throwing me but I think I'm misunderstanding something more basic.

Soylent Majority fucked around with this message at 18:53 on Feb 23, 2023

# ? Feb 23, 2023 18:39

Zugzwang: Jan 2, 2005; You have a kind of sick desperation in your laugh.; Ramrod XTreme

Do you mean you want to make a list containing grade #n from every list?

That can just be

code:

nth_grades = [grades[n] for grades in student_grades.values()]

# ? Feb 23, 2023 19:00

QuarkJets: Sep 8, 2008

Try writing it out as a simple loop first:

Python code:

# How many grades are there? Assume all entries are the same length
num_grades = len(next(iter(student_grades.values())))
num_students = len(student_grades)
# Create a list of averages to accumulate over
average_grades = [0]*num_grades
# Iterate over pairs of (student name, grades list)
for student, grades in student_grades.items():
    # Iterate over that student's grades
    for i, grade in enumerate(grades):
        # Accumulate
        average_grades[i] += grade / num_students

This is a biased estimation of the mean and you should instead use Welford's Online Algorithm, but for a sample size of 5 positive integers <100 this naive approach should be fine.

If this data was ingested into a pandas dataframe it'd be even easier, but this implementation requires no new dependencies. There are more succinct implementations, but this should help you figure out how to cleanly iterate over these objects

# ? Feb 23, 2023 19:07

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

kinda yeah - i want to grab each students grades for the first exam and find the average grade for that exam, then do that for each exam. I'm just trying to figure out in general how I could grab the first item from multiple lists and create a new list with that data, and do that for however many items exist in the list - it seems like once I've got that data out and in their own lists I can mutilate it however I need in order to get something out, in this case averages. I've come up with:

code:

grade1 =[]
grade2 =[]
grade3 =[]
grade4 =[]
grade5 =[]
for student, grades in student_grades.items():
    grade1.append(grades[0])
    grade2.append(grades[1])...

this seems like it'll work, but its so loving ugly

Soylent Majority fucked around with this message at 19:20 on Feb 23, 2023

# ? Feb 23, 2023 19:09

Apex Rogers: Jun 12, 2006; disturbingly functional

Soylent Majority posted:

kinda yeah - i want to grab each students grades for the first exam and find the average grade for that exam, then do that for each exam. I'm just trying to figure out in general how I could grab the first item from multiple lists and create a new list with that data, and do that for however many items exist in the list - it seems like once I've got that data out and in their own lists I can mutilate it however I need in order to get something out, in this case averages

Would something like this work? I've just typed it up here, haven't run it myself.

Python code:

# Assume all entries have all the exams, find number using the first one
num_exams = len(student_grades['Andrew'])
exam_avgs = []
for exam_num in range(0, num_exams):
    # Running sum for this exam number
    exam_sum = 0
    # Iterate through students and grab this exam's grade
    for student,grades in student_grades:
        exam_sum += grades[exam_num]
    avg = exam_sum/num_exams
    exam_avgs.append(avg)
print(exam_avgs)

# ? Feb 23, 2023 19:29

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

well gently caress that doesn't work after all, those last 4 averages cant be right
nope i'm an idiot who didn't finish c/ping properly

Write a program that uses the keys(), values(), and/or items() dict methods to find statistics about the student_grades dictionary. Find the following:

Print the name and grade percentage of the student with the highest total of points.
Find the average score of each assignment.
Find and apply a curve to each student's total score, such that the best student has 100% of the total points.

code:

# student_grades contains scores (out of 100) for 5 assignments
student_grades = {
    'Andrew': [56, 79, 90, 22, 50],
    'Nisreen': [88, 62, 68, 75, 78],
    'Alan': [95, 88, 92, 85, 85],
    'Chang': [76, 88, 85, 82, 90],
    'Tricia': [99, 92, 95, 89, 99]
}

avg = {}
total = {}
for student, grades in student_grades.items():
    #print(f'{student} ave = {sum(grades)/len(grades)}')
    avg[student] = sum(grades)/len(grades)
    total[student] = sum(grades)
curvebasis = avg[max(avg)]
print(f'Highscore is {max(avg)}: {avg[max(avg)]}')

grade1 =[]
grade2 =[]
grade3 =[]
grade4 =[]
grade5 =[]
for student, grades in student_grades.items():
    grade1.append(grades[0])
    grade2.append(grades[1])
    grade3.append(grades[2])
    grade4.append(grades[3])
    grade5.append(grades[4])
print(f'Average score for grade1: {sum(grade1)/len(grade1)}')
print(f'Average score for grade2: {sum(grade5)/len(grade1)}')
print(f'Average score for grade3: {sum(grade5)/len(grade1)}')   
print(f'Average score for grade4: {sum(grade5)/len(grade1)}')
print(f'Average score for grade5: {sum(grade5)/len(grade1)}') 

curvebasis = avg[max(avg)]
curvedscores = {}
for student, grades in student_grades.items():
    curvedscores[student] = total[student] / total[max(total)]
#print(avg)
#print(total)
print(curvedscores)

outputs:
Highscore is Tricia: 94.8
Average score for grade1: 82.8
Average score for grade2: 80.4
Average score for grade3: 80.4
Average score for grade4: 80.4
Average score for grade5: 80.4
{'Andrew': 0.6265822784810127, 'Nisreen': 0.7827004219409283, 'Alan': 0.9388185654008439, 'Chang': 0.8881856540084389, 'Tricia': 1.0}

Soylent Majority fucked around with this message at 19:48 on Feb 23, 2023

# ? Feb 23, 2023 19:41

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

OK, here's the slightly unfucked version that I think meets the requirements as written, but its hardcoded to only work with up to 5 grades. Still trying to figure out how I'd allow it to be more flexible, somehow create the gradex lists based off the count of grades in one of the dict values?

Python code:


# student_grades contains scores (out of 100) for 5 assignments
student_grades = {
    'Andrew': [56, 79, 90, 22, 50],
    'Nisreen': [88, 62, 68, 75, 78],
    'Alan': [95, 88, 92, 85, 85],
    'Chang': [76, 88, 85, 82, 90],
    'Tricia': [99, 92, 95, 89, 99]
}

avg = {}
total = {}
for student, grades in student_grades.items():
    #print(f'{student} ave = {sum(grades)/len(grades)}')
    avg[student] = sum(grades)/len(grades)
    total[student] = sum(grades)
curvebasis = avg[max(avg)]
print(f'Highscore is {max(avg)}: {avg[max(avg)]}')

grade1 =[]
grade2 =[]
grade3 =[]
grade4 =[]
grade5 =[]
for student, grades in student_grades.items():
    grade1.append(grades[0])
    grade2.append(grades[1])
    grade3.append(grades[2])
    grade4.append(grades[3])
    grade5.append(grades[4])
print(f'Average score for grade1: {sum(grade1)/len(grade1)}')
print(f'Average score for grade2: {sum(grade2)/len(grade1)}')
print(f'Average score for grade3: {sum(grade3)/len(grade1)}')   
print(f'Average score for grade4: {sum(grade4)/len(grade1)}')
print(f'Average score for grade5: {sum(grade5)/len(grade1)}') 

curvebasis = avg[max(avg)]
curvedscores = {}
for student, grades in student_grades.items():
    curvedscores[student] = round((total[student] / total[max(total)] *100),2)
for student, grades in curvedscores.items():
    print(f'{student}\'s curved score is {grades}%')

output:
Highscore is Tricia: 94.8
Average score for grade1: 82.8
Average score for grade2: 81.8
Average score for grade3: 86.0
Average score for grade4: 70.6
Average score for grade5: 80.4
Andrew's curved score is 62.66%
Nisreen's curved score is 78.27%
Alan's curved score is 93.88%
Chang's curved score is 88.82%
Tricia's curved score is 100.0%

Soylent Majority fucked around with this message at 19:58 on Feb 23, 2023

# ? Feb 23, 2023 19:56

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

Soylent Majority posted:

OK, here's the slightly unfucked version that I think meets the requirements as written, but its hardcoded to only work with up to 5 grades. Still trying to figure out how I'd allow it to be more flexible, somehow create the gradex lists based off the count of grades in one of the dict values?

If you can trust that all the lists are the same length, you can do something such as enumerate over the first student, using that index as the key to handle all the others.

Python code:


first_student = student_grades.values()[0]

for idx in range(len(first_student)):
	scores_for_this_test = []
	for student_name, student_scores in student_grades.items():
		scores_for_this_test.append(student_scores[idx])
	# Do your math for the averages for a given test here

The trick for approaches like this is to ask 'what if there were a hundred tests? What about a million?' and figure out how to accomplish it via iteration instead of manually creating a list for each test.

Edit: vvvv - Gah, I knew there was a simpler tool for this; just don't have to do this often enough to remember.

Falcon2001 fucked around with this message at 21:04 on Feb 23, 2023

# ? Feb 23, 2023 20:07

nullfunction: Jan 24, 2005; Nap Ghost

No mention of zip() as a way to grab items in order from a collection of iterables?

Python code:

>>> student_grades = {
...     'Andrew': [56, 79, 90, 22, 50],
...     'Nisreen': [88, 62, 68, 75, 78],
...     'Alan': [95, 88, 92, 85, 85],
...     'Chang': [76, 88, 85, 82, 90],
...     'Tricia': [99, 92, 95, 89, 100]
... }
>>> 
>>> list(zip(*[semester_grades for student, semester_grades in student_grades.items()]))
[(56, 88, 95, 76, 99), (79, 62, 88, 88, 92), (90, 68, 92, 85, 95), (22, 75, 85, 82, 89), (50, 78, 85, 90, 100)]

From there it's easy enough to sum() and divide by the len() of each of those.

You can also use itertools.zip_longest() if you wanted to handle replacing missing grades with zeroes, for example:

Python code:

>>> import itertools
>>> student_grades = {
...     'Andrew': [56, 79, 90, 22], # length is only 4!
...     'Nisreen': [88, 62, 68, 75, 78],
...     'Alan': [95, 88, 92, 85, 85],
...     'Chang': [76, 88, 85, 82, 90],
...     'Tricia': [99, 92, 95, 89, 100]
... }
>>> 
>>> list(zip_longest(*[semester_grades for student, semester_grades in student_grades.items()], fillvalue=0))
[(56, 88, 95, 76, 99), (79, 62, 88, 88, 92), (90, 68, 92, 85, 95), (22, 75, 85, 82, 89), (0, 78, 85, 90, 100)]

# ? Feb 23, 2023 20:35

Soylent Majority: Jul 13, 2020; Dune 2: Chicks At The Same Time

nullfunction posted:

No mention of zip() as a way to grab items in order from a collection of iterables?

hot drat, zip() does exactly what I was aiming for there, thanks

# ? Feb 24, 2023 01:18

Oysters Autobio: Mar 13, 2017

I use a jupyterhub environment with my team members, but some of them are just learning python and coding in general. I'd like to make the onboarding a bit easier / automated, so for example, them needing to use the terminal and bash to create SSH profiles so they can connect to gitlab, and having a curated conda env kernel that has all the stuff they need installed.

What's the best way to do this? Can I save a conda env config in a central place in jupyterhub so others can use it?

I've seen some onboarding type .sh bash scripts that look interesting, anyone got any good tips on this? I'm also fairly new to python and coding world.

# ? Feb 26, 2023 18:47

Falcon2001: Oct 10, 2004; Eat your hamburgers, Apollo.; Pillbug

Oysters Autobio posted:

I use a jupyterhub environment with my team members, but some of them are just learning python and coding in general. I'd like to make the onboarding a bit easier / automated, so for example, them needing to use the terminal and bash to create SSH profiles so they can connect to gitlab, and having a curated conda env kernel that has all the stuff they need installed.

What's the best way to do this? Can I save a conda env config in a central place in jupyterhub so others can use it?

I've seen some onboarding type .sh bash scripts that look interesting, anyone got any good tips on this? I'm also fairly new to python and coding world.

FWIW: Another way to tackle this if automating it isn't simple, is to try and write clear, easily followable documentation for how to perform those tasks. If they need to do similar tasks in the future, this isn't a bad way to be like 'hey here's some stuff and why we're doing it'.

# ? Feb 27, 2023 08:28

Jose Cuervo: Aug 25, 2004

I keep having SQL questions which I cannot seem to locate the answer to:

This code works just fine and returns the blood glucose values which fall in the 6 hour interval

Python code:

c.execute("""
             SELECT dt_index, bg_mg_per_dL from blood_glucose
             WHERE SID == :SID
             AND datetime(dt_index) >= :start_dt
             AND datetime(dt_index) < datetime(:start_dt, '+6 hours')
             """,
             {'SID': SID,
              'start_dt': str(start_dt)}).fetchall()

Now I want to generalize the code and be able to specify a different time interval length:

Python code:

c.execute("""
             SELECT dt_index, bg_mg_per_dL from blood_glucose
             WHERE SID == :SID
             AND datetime(dt_index) >= :start_dt
             AND datetime(dt_index) < datetime(:start_dt, '+:tss hours')
             """,
             {'SID': SID,
              'start_dt': str(start_dt),
              'tss': str(tss_length_hrs)}).fetchall()

but the above code returns an empty list.

I know I can write this query using fstrings (because I am the one writing the query so there is not any user input to worry about sanitizing):

Python code:

query = f"SELECT dt_index, bg_mg_per_dL from blood_glucose " \
        f"WHERE SID == '{SID}' " \
        f"AND datetime(dt_index) >= '{start_dt}' " \
        f"AND datetime(dt_index) < datetime('{start_dt}', '+{tss_length_hrs} hours')"

but I would like to know how to write this query "properly".

# ? Feb 28, 2023 02:15

Seventh Arrow: Jan 26, 2005

Jose Cuervo posted:

I keep having SQL questions which I cannot seem to locate the answer to:

Not even the SQL thread?

# ? Feb 28, 2023 02:27

C2C - 2.0: May 14, 2006; Dubs In The Key Of Life; Lipstick Apathy

I just finished building a project for my class; it's essentially a Tkinter interface with a bunch of inputs for my music collection that saves the inputs to a .json file.

I'm having a go at using PyVis to map out some of the relationships of the items in the collection. Here's my code:

Python code:

# Import statements:
import json

from IPython.lib.display import IFrame
from pyvis.network import Network

# Create path to data document:
file = 'music.json'


# Create a function to write a static HTML file:
def show(self, name, local, notebook):
    if notebook:
        self.write_html(
            name='music.html',
            open_browser=False,
            local=True,
            notebook=True)
    else:
        self.write_html(
            name='music.html',
            open_browser=True,
            local=True,
            notebook=False)
    if notebook:
        return IFrame(name, width=self.width, height=self.height)


# Create a function to get data from JSON file:
def get_data():
    with open(file, 'r') as json_file:
        data = json.load(json_file)
        return data


# Create a function to map data from the JSON file:
def map_data(release_data):
    g = Network(
        height='940px',
        width='100%',
        bgcolor='#222222',
        font_color='white',
    )
    for release in release_data:
        artist = (release['artist'])
        label = (release['rec_label'])
        g.add_node(artist)
        g.add_node(label)
        g.add_edge(artist, label)

    g.show('music.html')


# Save loaded data to a variable:
music_data = get_data()

# Send loaded data to be mapped:
map_data(music_data)

Here's the error I'm getting:

code:

AttributeError: 'NoneType' object has no attribute 'render'
music.html

Prior to this, I didn't have the def show(): function written and was receiving the same error. The function was lifted straight from the documents and I'm still getting the same error. The only thing I came across while searching is that the notebook defaults to True unless it's passed a False argument which seems to be rendered moot by the function itself.

# ? Feb 28, 2023 20:12

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 08:36

Zoracle Zed: Jul 10, 2001

Jose Cuervo posted:

I keep having SQL questions which I cannot seem to locate the answer to:

just guessing but I bet your SQL driver is (correctly, imo) not interpolating the tss variable here:

code:

datetime(:start_dt, '+:tss hours')

because it's inside a string literal. try it something like this, (modulo whatever the appropriate string concatenation operator is for your db)

code:

datetime(:start_dt, STR_CONCAT('+', :tss, ' hours'))

# ? Feb 28, 2023 23:41

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »