hbag's blog thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > hbag's blog thread

hbag: Feb 13, 2021

hmmmm
app is saying it cant find a file:

pre:

PIL.UnidentifiedImageError: cannot identify image file '227624av.png'

and yet when i put a debug print in before the line that throws that error, its right there

# ? Jun 2, 2021 23:50

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 13:25

hbag: Feb 13, 2021

no wait it can find it but not identify it
hmm

this is how the image is being created btw

pre:

		avatarImage = userProfile.select_one('.userinfo .title').find('img')
		avatarURL = (avatarImage.attrs['src'])
		avatarFile = f'{horribleJerk}av.{avatarURL.split(".")[-1]}'
		avatar = requests.get(avatarURL, cookies=authCookies, headers=headers)
		print(str(avatar.status_code))
		if avatar.status_code == 200:
			with open(avatarFile, 'wb') as avFile:
				avatar.raw.decode_content = True
				shutil.copyfileobj(avatar.raw, avFile)

# ? Jun 2, 2021 23:59

mystes: May 31, 2006

Did you write to the file and not close it or something?

# ? Jun 3, 2021 00:02

hbag: Feb 13, 2021

mystes posted:

Did you write to the file and not close it or something?

no, with open automatically closes the file when it unindents

# ? Jun 3, 2021 00:03

mystes: May 31, 2006

Yeah I didn't actually read your code. Maybe print the file size or otherwise make sure it's writing a valid image though.

# ? Jun 3, 2021 00:04

mystes: May 31, 2006

Also are you really supposed to use response.raw? I think it might break in some situations.

# ? Jun 3, 2021 00:08

hbag: Feb 13, 2021

mystes posted:

Also are you really supposed to use requests.raw? I think it might break in some situations.

i have no idea how else to download an image when all i have is the image link
via a script

# ? Jun 3, 2021 00:09

mystes: May 31, 2006

code:

if avatar.status_code == 200:
    with open(avatarFile, 'wb') as avFile:
         avFile.write(avatar.content)

# ? Jun 3, 2021 00:16

hbag: Feb 13, 2021

mystes posted:

code:

if avatar.status_code == 200:
    with open(avatarFile, 'wb') as avFile:
         avFile.write(avatar.content)

eh ill give it a shot

# ? Jun 3, 2021 00:17

Cybernetic Vermin: Apr 18, 2005

a quick thing to do is just less the supposed png file and discover that the contents is actually a 404 page or some such.

# ? Jun 3, 2021 00:19

mystes: May 31, 2006

You don't even need to be logged in to access avatars, it seems, so you can just do something like this to test it:

code:

import requests
import shutil
avatarURL = "https://fi.somethingawful.com/safs/titles/05/1a/00227624.0029.png"
avatarFile = "asdf.png"
avatar = requests.get(avatarURL)
print(str(avatar.status_code))
if avatar.status_code == 200:
    with open(avatarFile, 'wb') as avFile:
        avatar.raw.decode_content = True
        shutil.copyfileobj(avatar.raw, avFile)

And you can confirm that indeed the original code produces a 0 byte file.

# ? Jun 3, 2021 00:20

hbag: Feb 13, 2021

alright well im out of ideas for how to download an image from the link
cos i can get the link, ive tested that, but i dont know how to download the image

Cybernetic Vermin posted:

a quick thing to do is just less the supposed png file and discover that the contents is actually a 404 page or some such.

considering it only writes a file if its 200 then i dont think thats the case

# ? Jun 3, 2021 00:22

mystes: May 31, 2006

hbag posted:

alright well im out of ideas for how to download an image from the link
cos i can get the link, ive tested that, but i dont know how to download the image
considering it only writes a file if its 200 then i dont think thats the case

Did the thing I suggested a minute ago not work? Because it works for me with a hardcoded url. I'm guessing it's a problem with compression or something even though I guess response.raw.decode_content = True is allegedly supposed to fix that.

# ? Jun 3, 2021 00:23

hbag: Feb 13, 2021

mystes posted:

Did the thing I suggested a minute ago not work? Because it works for me with a hardcoded url.

testing that now, think it might

# ? Jun 3, 2021 00:23

hbag: Feb 13, 2021

alright i think that worked
now to figure out how to use pillow to paste one image ontop of another and then save it

# ? Jun 3, 2021 00:31

akadajet: Sep 14, 2003

Asleep Style posted:

cisco stu here with some good news about VOIP adoption trends

lol

# ? Jun 3, 2021 01:00

akadajet: Sep 14, 2003

Smythe posted:

update this: https://chrome.google.com/webstore/detail/sa-needful/bipfbhacdgoojjfblifgjadhabepbekm

please!

# ? Jun 3, 2021 01:01

hbag: Feb 13, 2021

i uh

what
how

# ? Jun 3, 2021 01:14

Sagebrush: Feb 26, 2012; ERM... Actually I have stellar scores on the surveys, and every year students tell me that my classes are the best ones they’ve ever taken.

hbag posted:

i uh

what
how

wherever you are, there is the worst programmer

# ? Jun 3, 2021 01:15

fart simpson: Jul 2, 2005; DEATH TO AMERICA

rotor posted:

hbag, are you familiar with nntp

lol

# ? Jun 3, 2021 01:51

hbag: Feb 13, 2021

you can tell from my commit messages alone exactly where i figured out a thing i was trying to do

# ? Jun 3, 2021 02:00

hbag: Feb 13, 2021

euuuuurrrghh alright im still trying to figure out how to test if the entire image is just transparent and nothing else

# ? Jun 3, 2021 02:23

Jonny 290: May 5, 2005; [ASK] me about OS/2 Warp

on: too many redirects: make sure you're consistently sticking with https for all requests. you can snowball a bad http request into too many redirects if, like, somethingawful, you use a lovely garbage CDN like NaziCache-sorry, Cloudflare

# ? Jun 3, 2021 02:35

hbag: Feb 13, 2021

Jonny 290 posted:

on: too many redirects: make sure you're consistently sticking with https for all requests. you can snowball a bad http request into too many redirects if, like, somethingawful, you use a lovely garbage CDN like NaziCache-sorry, Cloudflare

yea i figured it out
i was accidentally sending it auth cookies that were None so the php was making GBS threads itself
fixed that poo poo now though

NOW what need to do is figure out how to detect if an entire image is just one big transparent space
my reasoning for this is so i can detect if someone's doing a tall gangtag av (like mine) and then, if they are, just use the first image gangtag in place of their avatar

# ? Jun 3, 2021 02:38

mystes: May 31, 2006

hbag posted:

NOW what need to do is figure out how to detect if an entire image is just one big transparent space
my reasoning for this is so i can detect if someone's doing a tall gangtag av (like mine) and then, if they are, just use the first image gangtag in place of their avatar

You can just iterate through the pixels and check the alpha value, but unless people are using giant transparent avatars maybe you should just check whether the avatar is like 5x5 or smaller?

# ? Jun 3, 2021 02:42

hbag: Feb 13, 2021

mystes posted:

You can just iterate through the pixels and check the alpha value, but unless people are using giant transparent avatars maybe you should just check whether the avatar is like 5x5 or smaller?

i thought putting loops inside loops was a bad idea
this part of the code is already in a while loop, sticking a for loop in it is gonna be 2 loops

# ? Jun 3, 2021 02:51

fart simpson: Jul 2, 2005; DEATH TO AMERICA

mystes posted:

You can just iterate through the pixels and check the alpha value, but unless people are using giant transparent avatars maybe you should just check whether the avatar is like 5x5 or smaller?

yeah and because this problem is embarrassingly parallel you can parallelize it for efficiency, each �process� only needs to check one pixel each and your result is still accurate

code:

from multiprocessing import Pool

def is_transparent(alpha):
  return alpha == 0

# imagine this is a sequence of every pixel's alpha value
alphas = range(32400)

if __name__ == '__main__':
  with Pool(processes=32400) as p:
    x = p.map(is_transparent, alphas)

  print(all(x))

# ? Jun 3, 2021 02:52

hbag: Feb 13, 2021

fart simpson posted:

yeah and because this problem is embarrassingly parallel you can parallelize it for efficiency, each �process� only needs to check one pixel each and your result is still accurate
code:
from multiprocessing import Pool

def is_transparent(alpha):
  return alpha == 0

# imagine this is a sequence of every pixel's alpha value
alphas = range(32400)

if __name__ == '__main__':
  with Pool(processes=32400) as p:
    x = p.map(is_transparent, alphas)

  print(all(x))

having a hard time breaking down your snippet into "what bit is doing what" but that might be because its 3 am and im tired

obviously im still not gonna say no to a more basic explanation because i will take that every time
i apparently need things explained to me with small words because my brain is a collander

# ? Jun 3, 2021 02:55

Fuzzy Mammal: Aug 15, 2001; Lipstick Apathy

hbag posted:

i thought putting loops inside loops was a bad idea
this part of the code is already in a while loop, sticking a for loop in it is gonna be 2 loops

it's not slow, it's readable

# ? Jun 3, 2021 03:01

mystes: May 31, 2006

hbag posted:

i thought putting loops inside loops was a bad idea
this part of the code is already in a while loop, sticking a for loop in it is gonna be 2 loops

Probably whoever said nesting loops was bad was talking about stuff like looping over a huge list in the inner loop of a nested loop which will be executed a zillion times, when you could just be using a hashtable or something, and not looping over the x and y coordinates of a tiny image.

Fart simpson is just trying to make everything complicated.

You can probably just do something like:

code:

def has_opaque_pixel(im):
    if im.mode != "RGBA":
        return True
    for x in range(im.width):
        for y in range(im.height):
            alpha = im.getpixel((x,y))[3]
            if alpha != 0:
                return True
    return False

Although again I'm not sure this is really the best way to decide if an avatar is just an empty one pixel image in general.

Also this might not handle gifs if people still use gifs, because they're indexed color and I don't care enough to check whether they are treated as a different color mode by pillow.

# ? Jun 3, 2021 03:06

hbag: Feb 13, 2021

mystes posted:

Probably whoever said nesting loops was bad was talking about stuff like looping over a huge list in the inner loop of a nested loop which will be executed a zillion times, when you could just be using a hashtable or something, and not looping over the x and y coordinates of a tiny image.

Fart simpson is just trying to make everything complicated.

You can probably just do something like:
code:
def has_opaque_pixel(im):
    if im.mode != "RGBA":
        return True
    for x in range(im.width):
        for y in range(im.height):
            alpha = im.getpixel((x,y))[3]
            if alpha != 0:
                return True
    return False
Although again I'm not sure this is really the best way to decide if an avatar is just an empty one pixel image in general.

Also this might not handle gifs if people still use gifs, because they're indexed color and I don't care enough to check whether they are treated as a different color mode by pillow.

alright, ill give this a try in the morning
rn its 3 am so im going to bed lol

# ? Jun 3, 2021 03:12

fart simpson: Jul 2, 2005; DEATH TO AMERICA

mystes posted:

Fart simpson is just trying to make everything complicated.

i already posted a link to the pillow docs for the built in method that tells you the min and max pixel values for an entire band of the whole image :shrug:

seems pretty simple and useful here to me

# ? Jun 3, 2021 03:16

hbag: Feb 13, 2021

fart simpson posted:

i already posted a link to the pillow docs for the built in method that tells you the min and max pixel values for an entire band of the whole image seems pretty simple and useful here to me

i couldn't find the post when i was scrolling through em which is why i asked again tbh

# ? Jun 3, 2021 03:19

mystes: May 31, 2006

fart simpson posted:

i already posted a link to the pillow docs for the built in method that tells you the min and max pixel values for an entire band of the whole image seems pretty simple and useful here to me

Ah yeah I guess you did somewhere in there.

# ? Jun 3, 2021 03:23

Eeyo: Aug 29, 2004

i've never bought one of those big avatars. are you saying it's required to upload a tiny one as your "real" avatar? i always thought you just put bbcode into the text and didn't have a regular avatar.

# ? Jun 3, 2021 04:10

Jonny 290: May 5, 2005; [ASK] me about OS/2 Warp

fart simpson posted:

yeah and because this problem is embarrassingly parallel you can parallelize it for efficiency, each �process� only needs to check one pixel each and your result is still accurate
code:
from multiprocessing import Pool

def is_transparent(alpha):
  return alpha == 0

# imagine this is a sequence of every pixel's alpha value
alphas = range(32400)

if __name__ == '__main__':
  with Pool(processes=32400) as p:
    x = p.map(is_transparent, alphas)

  print(all(x))

this poo poo is why i gave up trying to learn Coding. what the gently caress is this

# ? Jun 3, 2021 04:42

Bloody: Mar 3, 2013

Jonny 290 posted:

this poo poo is why i gave up trying to learn Coding. what the gently caress is this

an exciting way to turn a simple problem into a worse, more complex, problem

# ? Jun 3, 2021 05:25

Storysmith: Dec 31, 2006

Jonny 290 posted:

this poo poo is why i gave up trying to learn Coding. what the gently caress is this

okay so the code assumes you have a 1-dimensional list that has all the pixel values of your av image.
it defines a function that tells you if the image is transparent or not, and then starts a (lightweight thread, i think is the default?) pool of executors that each take a single entry in that list and calculate if that pixel is opaque or transparent, and then the all() (iirc) tells you if any of them are opaque

the use of the map function there is some unintuitive stuff to newcomers � you�re not calling the is_transparent function in that map line. you�re passing the function itself in as an object that then gets called by map() with the proper alpha array entry for each one in the list.

i can never remember what the technical cs term for it is but it�s one of those things that broke my mind when i learned ruby from a codebase full of people who loved map and filter and the like.