Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Sagebrush
Feb 26, 2012

can't they just write a script that like

10 > loads first thread in archive
20 > goes to first page
30 > goes to first post
40 > clicks "quote" button on this post
50 > gets the returned form content
60 > adds to a database with the quoted userid
70 > moves to next post
80 > goto 40

with 63 million posts and doing that like, say, i dunno, 25 times a second they could be done in less than a month. 25hz is probably super slow estimate too but i don't wanna put tooooo much strain on the web server, we know how that goes

Adbot
ADBOT LOVES YOU

Notorious b.s.d.
Jan 25, 2003

by Reene

Sagebrush posted:

can't they just write a script that like

10 > loads first thread in archive
20 > goes to first page
30 > goes to first post
40 > clicks "quote" button on this post
50 > gets the returned form content
60 > adds to a database with the quoted userid
70 > moves to next post
80 > goto 40

with 63 million posts and doing that like, say, i dunno, 25 times a second they could be done in less than a month. 25hz is probably super slow estimate too but i don't wanna put tooooo much strain on the web server, we know how that goes

something like this might be exactly what xenforo proposes

it's not like they do this for a living or anything..

CrazyLittle
Sep 11, 2001





Clapping Larry
can't "quote" archived posts that are stored on a separate db system that doesn't support data extraction into the live forums db

The Management
Jan 2, 2010

sup, bitch?
jfc, how hard is it to do a sql dump?

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

The Management posted:

jfc, how hard is it to do a sql dump?

it ain't but translating similar schema from one app's to another can actually be a huge mission

The Management
Jan 2, 2010

sup, bitch?

Cocoa Crispies posted:

it ain't but translating similar schema from one app's to another can actually be a huge mission

that's certainly true but you would have to do that whether you're going from a sql dump or a scraped rendered output.

infernal machines
Oct 11, 2012

we monitor many frequencies. we listen always. came a voice, out of the babel of tongues, speaking to us. it played us a mighty dub.

The Management posted:

jfc, how hard is it to do a sql dump?

bitfields

The Management
Jan 2, 2010

sup, bitch?

gently caress, :boom:

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

The Management posted:

that's certainly true but you would have to do that whether you're going from a sql dump or a scraped rendered output.

doing it from live sql tables is going to be easiest, but not easy

trilljester
Dec 7, 2004

The People's Tight End.

Cocoa Crispies posted:

doing it from live sql tables is going to be easiest, but not easy

I imagine the data mapping between source and destination DB would be the hardest part. Then again, Xenforo guys should be able to do that if Lowtax forks over some cash.

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

trilljester posted:

I imagine the data mapping between source and destination DB would be the hardest part. Then again, Xenforo guys should be able to do that if Lowtax forks over some cash.

it's the only part, lmao

Linguica
Jul 13, 2000
You're already dead

When I recently moved forum software I had to write my own bespoke forum converter to move everything over and there is nothing hard about it, it's just tedious to figure out all the settings and idiosyncracies and how to convert them to the new database properly. I say this because I was able to port over basically 100% of all settings and I am a dipshit with no formal training in programming

the only "hard" part is doing it on 250 million posts without extended downtime

Linguica fucked around with this message at 21:14 on Mar 29, 2017

The Management
Jan 2, 2010

sup, bitch?

Linguica posted:

When I recently moved forum software I had to write my own bespoke forum converter to move everything over and there is nothing hard about it, it's just tedious to figure out all the settings and idiosyncracies and how to convert them to the new database properly. I say this because I was able to port over basically 100% of all settings and I am a dipshit with no formal training in programming

the only "hard" part is doing it on 250 million posts without extended downtime

this. I mean yeah, bitfields, but it's a matter of grinding it out. you don't need downtime, it's not like most of those posts are changing. you can lock down editing on anything you're working on, shouldn't take long

pram
Jun 10, 2001
no he can only find programmers who will write new forums, not basic ETL

akadajet
Sep 14, 2003

I think it would be a "fun" exercise, but I hate myself so...

The Management
Jan 2, 2010

sup, bitch?
richard publish the schema populated with like one thread and the associated users

Sagebrush
Feb 26, 2012

The Management posted:

richard publish the schema populated with like one thread and the associated users

yeah no kidding. if he'd post a dump of one small representative segment of the db I bet someone in here or shsc would have a parser written by that evening

and a table-dropping exploit by the next morning

Improbable Lobster
Jan 6, 2012

"From each according to his ability" said Ares. It sounded like a quotation.
Buglord
good op

Fergus Mac Roich
Nov 5, 2008

Soiled Meat

The Management posted:

richard publish the schema populated with like one thread and the associated users

AWWNAW
Dec 30, 2008

why has no one realized this is a perfect use for the block chain????

we need an immutable and cryptographically expensive ledger of all posts

Doom Mathematic
Sep 2, 2008
Don't cryptographically sign your posts.

burning swine
May 26, 2004



Doom Mathematic posted:

Don't cryptographically sign your posts.

Sham bam bamina!
Nov 6, 2012

ƨtupid cat
i have like 3k unread posts in the bit coin thread, have i missed any notable developments

pram
Jun 10, 2001
esoteric programming

Seat Safety Switch
May 27, 2008

MY RELIGION IS THE SMALL BLOCK V8 AND COMMANDMENTS ONE THROUGH TEN ARE NEVER LIFT.

Pillbug
it's like jazz. you gotta appreciate the blobflags he isn't writing.

Fergus Mac Roich
Nov 5, 2008

Soiled Meat
perhaps it is for us to discover the thread states which were conceived of, but never implemented.

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
if you use save corruption to get a thread state of 7, you can trigger an easter egg that jumps you straight to the end credits

Asymmetric POSTer
Aug 17, 2005

we should crowdsource the money to get radium to come back and finish titian

Sham bam bamina!
Nov 6, 2012

ƨtupid cat

zen death robot posted:

did you know an active thread as the "threadoptions" field with a 0 in it but one flagged for archival has a 16?

1-15 do nothing so it appears to be some random bullshit radium came up with. Or he thought hexadecimal was numbers 0-16.
wow

mod saas
May 4, 2004

Grimey Drawer

keep reading this as shorthand for a secret adopt-a-thread program

Endless Mike
Aug 13, 2003



i'm claiming threadoptions=8

Stymie
Jan 9, 2001

by LITERALLY AN ADMIN

mishaq posted:

we should crowdsource the money to get radium to come back and finish titian

:ssh:

SmokaDustbowl
Feb 12, 2001

by vyelkin
Fun Shoe
clean up the smilies lol

Sagebrush
Feb 26, 2012

Fergus Mac Roich posted:

perhaps it is for us to discover the thread states which were conceived of, but never implemented.

I have discovered a truly remarkable proof of the existence of alternate thread states which this post is too small to contain.

jony ive aces
Jun 14, 2012

designer of the lomarf car


Buglord

Sham bam bamina! posted:

i have like 3k unread posts in the bit coin thread, have i missed any notable developments
yeah everyone getting fishmeched

burning swine
May 26, 2004



Sagebrush posted:

I have discovered a truly remarkable proof of the existence of alternate thread states which this post is too small to contain.

radium's last theorem

Linguica
Jul 13, 2000
You're already dead

jony ive aces posted:

yeah everyone getting fishmeched
he said "notable"

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde

zen death robot posted:

did you know an active thread as the "threadoptions" field with a 0 in it but one flagged for archival has a 16?

1-15 do nothing so it appears to be some random bullshit radium came up with. Or he thought hexadecimal was numbers 0-16.
he was using nibbles to store & manipulate bit flags

i think back to my early pro years, I did some really dumb things under the impression that they were clever. radium was stuck at that stage, I can only hope he's moved on and also found a manager who can mentor him the way lowtax never could

skimothy milkerson
Nov 19, 2006

COACHS SPORT BAR posted:

radium's last theorem

Adbot
ADBOT LOVES YOU

N.Z.'s Champion
Jun 8, 2003

Yam Slacker
if you donate $66.6 then on the checkout it'll only be $66.59 what a bargain

  • Locked thread