Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
lord fifth
Dec 26, 2019

LUCK ???
it was a great six months, gentlemen. proud to report that i spent pretty much all my forums time in yospos, reading

i hope things don't go under, i want to keep reading

Adbot
ADBOT LOVES YOU

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.

TerminalRaptor posted:

Why roll it back? Just show how much user engagement is rising (and not mention why)

there were like 2500 people in gbs yesterday. even today it's only down to 1700

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)

SoundMonkey posted:

look for a forums announcement some time tonight

:ohdear:

TerminalRaptor
Nov 6, 2012

Mostly Harmless

infernal machines posted:

there were like 2500 people in gbs yesterday. even today it's only down to 1700

By those metrics if lowtax were to punch more women the forums would return to their posting heyday

SRQ
Nov 9, 2009

just pull a tumblr and sell it and then the new owners will carpet bomb it with bans and nuke the entire userbase

TerminalRaptor
Nov 6, 2012

Mostly Harmless
Whatever happened to those internet jpg spaceships he was supposed to draw for supporters? Did he welch on even doing the bars minimum for that?

corgski
Feb 6, 2007

Silly goose, you're here forever.

SRQ posted:

just pull a tumblr and sell it and then the new owners will carpet bomb it with bans and nuke the entire userbase

what would the SA equivalent of tumblr banning porn be? banning swear words?

SRQ
Nov 9, 2009

TerminalRaptor posted:

Whatever happened to those internet jpg spaceships he was supposed to draw for supporters? Did he welch on even doing the bars minimum for that?

he's competing with chris roberts?

SRQ
Nov 9, 2009

corgski posted:

what would the SA equivalent of tumblr banning porn be? banning swear words?

banning goatse

pram
Jun 10, 2001
no. hes literally never done or delivered anything and yet idiots still kept handing money over. the ONLY significant change over the past 20 years was the elasticsearch backed search replacement that some grad student did for free(?)

that donation pill is a badge of shame lol

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano

corgski posted:

what would the SA equivalent of tumblr banning porn be? banning swear words?

Banning the word "retarded", which they did last year after a QCS struggle session

Lain Iwakura
Aug 5, 2004

The body exists only to verify one's own existence.

Taco Defender

pram posted:

that donation pill is a badge of shame lol

agreed. i'll live though

anyway, if you want to archive the forums for yourself, here's some garbage python code

Python code:
from bs4 import BeautifulSoup
import json
from requests import session
import os
from concurrent.futures import ThreadPoolExecutor
import sys

threadRange = [int(x) for x in (sys.argv[1], sys.argv[2])]
threadWorkers = 5
threadStorage = '' # Place path here
username = '' # Your username
password = '' # Your password
debug = False

class saRipper(object):
    def __init__(self, username=None, password=None):
        self.username = username
        self.password = password
        self.loginURL = 'https://forums.somethingawful.com/account.php?action=loginform'
        self.firstPage = 'https://forums.somethingawful.com/showthread.php?threadid={}'
        self.newPage = 'https://forums.somethingawful.com/showthread.php?threadid={}&perpage=40&pagenumber={}'
        self.session = session()
        self.debug = debug
        self.errors = None
        if None not in [username, password]:
            self.generateSession()

    def generateSession(self):
        data = {'username': self.username, 'password': self.password, 'action': 'login', 'next': '/'}
        self.session.post(self.loginURL, data=data)

    def pullRaw(self, url):
        r = self.session.get(url, headers={'User-Agent': 'Forum Cloner (User: {})'.format(self.username)})
        return r.text

    def startParse(self, data):
        return BeautifulSoup(data, 'html.parser')

    def parsePages(self, data):
        out = list(dict.fromkeys([int(x['value']) for x in self.startParse(data).find_all('option') if x['value'].isdigit()]))
        if len(out) is 0:
            out = [1]
        return out

    def successPage(self, data):
        i = [
                'The page number you requested does not exist in this thread.', 
                'You do not have permission to access this page.',
            ]
        self.errors = [x for x in i if x in data]
        return len(self.errors) is 0

    def pullPage(self, threadID, page=1):
        self.errors = []
        if page is 1:
            u = self.firstPage.format(threadID)
        else:
            u = self.newPage.format(threadID, page)
        if self.debug:
            print('URL: {}'.format(u))
        return self.pullRaw(url=u)

def writeRaw(path, threadID, data, page=1):
    if not checkRaw(path=path, threadID=threadID, page=page):
        with open(pathRaw(path=path, threadID=threadID, page=page), 'w') as f:
            f.write(data)

def checkRaw(path, threadID, page=1):
    return os.path.exists(pathRaw(path=path, threadID=threadID, page=page))

def pathRaw(path, threadID, page=1):
    fname = '{}.{}.html'.format(threadID, page)
    return os.path.join(path, fname)

def workerTask(threadID):
    sa = saRipper(username=username, password=password)
    print('Thread ID: {} (starting worker process)'.format(threadID))
    data = sa.pullPage(threadID=threadID)
    if sa.successPage(data=data):
        pages = sa.parsePages(data)
        print('Thread ID: {} (pages to consider: {})'.format(threadID, len(pages)))
        if len(pages) is not 0:
            for page in pages:
                if not checkRaw(path=threadStorage, threadID=threadID, page=page):
                    if page is not 1:
                        data = sa.pullPage(threadID=threadID, page=page)
                    if sa.successPage(data=data):
                        writeRaw(path=threadStorage, threadID=threadID, page=page, data=data)
                        print('Thread ID: {} (page {}) (written)'.format(threadID, page))
                    else:
                        print('Thread ID: {} (page {}) (error encountered from site)'.format(threadID, page))
                        print('Thread ID: {} ({})'.format(threadID, ', '.join(sa.errors)))
                        break
                else:
                    print('Thread ID: {} (page {}) (skipped)'.format(threadID, page))
            print('ThreadID: {} (completed pulling all pages)'.format(threadID))
        else:
            print('Thread ID: {} (pages: 1 or inaccessible)'.format(threadID))
            writeFirstPage()
    else:
        print('Thread ID: {} (encountered an error from site)'.format(threadID))
        print('Thread ID: {} ({})'.format(threadID, ', '.join(sa.errors)))
    print('Thread ID: {} (finished worker process)'.format(threadID))

if __name__ == '__main__':
    s = saRipper(username=username, password=password)
    processes = []
    threadIDs = list(range(threadRange[0], threadRange[1] + 1))
    if not debug:
        def run(d):
            workerTask(threadID=d['threadID'])
        with ThreadPoolExecutor(max_workers=threadWorkers) as executor:
            f = [executor.submit(run, {'threadID': threadID}) for threadID in threadIDs]
    if debug:
        for threadID in threadIDs:
            workerTask(threadID=threadID)
i am sure that this will raise some eyebrows but whatever. i don't want to deal with someone who abuses women and i furthermore don't want him to get another penny

it just writes the thread pages to disk and it doesn't do it in the most efficient manner but whatever it's threaded and works

code:
$ python3 script.py threadidstart threadidend
that will scrape the forums based on threadid. it will only read five pages at a time

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.

SRQ posted:

he's competing with chris roberts?

yes, it was a joke about people spending money on starship jpegs, so naturally lowtax took money for starship jpegs and didn't even ship those

frankly, it's loving hilarious

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.
oh cool, jeffrey of yospos is taking over control of the site's owning legal entity, voluntarily, from lowtax

this is not in any way suspicious

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.
he also removed the looney tunes goatse animation from the announcement.

i will not stand for this cowardice from the admins

Maximo Roboto
Feb 4, 2012

who is Jeffrey and what OS does he use

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)
it’s jeffk

Vintersorg
Mar 3, 2004

President of
the Brendan Fraser
Fan Club



yospos birch

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)
does anyone know if jeffrey is indeed of yospos?

pram
Jun 10, 2001
no hes of fyad

Saoshyant
Oct 26, 2010

:hmmorks: :orks:


yospos greatest son

(who no one remembers)

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)
hmm sounds like Stolen Valor to me

Shaggar
Apr 26, 2006
hey lurkers, you should post more

Amethyst
Mar 28, 2004

I CANNOT HELP BUT MAKE THE DCSS THREAD A FETID SWAMP OF UNFUN POSTING
plz notice me trunk-senpai

infernal machines posted:

oh cool, jeffrey of yospos is taking over control of the site's owning legal entity, voluntarily, from lowtax

this is not in any way suspicious

i'm fine with this jeffery is cool

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)

Shaggar posted:

hey lurkers, you should post more

this is what i did and look what happened

Maximo Roboto
Feb 4, 2012

why does Jeffrey have hundreds of thousands of dollars with which to buy the forums from Lowtax

Soviet
Jul 17, 2003

stick it in me
bring back ycs

pram
Jun 10, 2001

infernal machines posted:

this is not in any way suspicious

Sagebrush
Feb 26, 2012

ERM... Actually I have stellar scores on the surveys, and every year students tell me that my classes are the best ones they’ve ever taken.
i just hope they don't bring back fyad.

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)

Sagebrush posted:

i just hope they don't bring back fyad.

jesus i’m pretty sure that would kill this place off.

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.

Amethyst posted:

i'm fine with this jeffery is cool

i've talked with him a bit about technical issues and he seems pretty alright.

Maximo Roboto
Feb 4, 2012

https://twitter.com/angryaboutbikes/status/938794023510298624

oh I get it now

Scud Hansen
Dec 13, 2015

Darkness and Evil
whoever this jeffrey is he seems to regard something awful as "incredible" which is highly suspicious to me

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.
my comment was solely WRT "lowtax is no longer a named director of the llc, so everything is fine now" part, which is, uh, something

DELETE CASCADE
Oct 25, 2017

i haven't washed my penis since i jerked it to a phtotograph of george w. bush in 2003

Maximo Roboto posted:

why does Jeffrey have hundreds of thousands of dollars with which to buy the forums from Lowtax

why don't you?

Luigi Thirty
Apr 30, 2006

Emergency confection port.

Maximo Roboto posted:

why does Jeffrey have hundreds of dollars with which to buy the forums from Lowtax

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano
hearing credible rumors that Jeffrey Of Yospos has just beaten the absolute poo poo out of his wife

KaosMachina
Oct 9, 2012

There's nothing special about me.

President Beep posted:

this is what i did and look what happened

yeah posting is overrated, stare at the pretty pictures

Aeryk
Aug 31, 2006

Ah. It must have been when I was younger.
Fun Shoe
if i stopped lurking i might have to tell you if i did amberpos or greenpos and i don't think i'm ready for that kind of commitment

Adbot
ADBOT LOVES YOU

President Beep
Apr 30, 2009





i have to have a car because otherwise i cant drive around the country solving mysteries while being doggedly pursued by federal marshals for a crime i did not commit (9/11)

Aeryk posted:

if i stopped lurking i might have to tell you if i did amberpos or greenpos and i don't think i'm ready for that kind of commitment

hold up motherfucker your not going anywhere

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply