i actually have reasons to post here unlike half of the people with impostor syndrome

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > i actually have reasons to post here unlike half of the people with impostor syndrome

«‹›239 »

Shaggar: Apr 26, 2006

MALE SHOEGAZE posted:

yeah i thought about it a lot while reading house of suns and it's basically an impossible problem.

just use eastern like you're supposed to on earth

# ? Nov 10, 2017 02:31

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 21:44

tef: May 30, 2004; -> some l-system crap ->

the iss uses utc

# ? Nov 10, 2017 02:40

JawnV6: Jul 4, 2004; So hot ...

milliseconds since the epoch combined with an inertial log

# ? Nov 10, 2017 03:05

tef: May 30, 2004; -> some l-system crap ->

a vector clock of pulsars :2bong:

# ? Nov 10, 2017 03:15

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

Ciaphas posted:

actual question, does anyone know if any implementations of EXI for C/C++ or C#/.NET are any good

guy at work is proposing that for long term storage format for data we're working with and it looks like a good fit in theory

how much data we talkin about, and how flat is it? Also, what is long-term for you?

If you want your data to really live for a long time (10+ years) and if it's a big giant table, you should probably just store it as CSV text files. The format sucks but it's pretty gaddam feature-proof and simple to read into whatever you want

# ? Nov 10, 2017 03:21

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

Poopernickel posted:

how much data we talkin about, and how flat is it? Also, what is long-term for you?

If you want your data to really live for a long time (10+ years) and if it's a big giant table, you should probably just store it as CSV text files. The format sucks but it's pretty gaddam feature-proof and simple to read into whatever you want

maximally right now is 5gb+ per file and growing (average is closer to 100MB per, and there are thousands of the things), and long term is at least 20 years, probably more. currently it's in a binary format that got homerolled 20 years ago for our tools, and we're finally taking the time for a format upgrade/migration

basically this decision is miles above my paygrade and i don't really have a stake or call in it but i'm curious and thinking about it anyway

our tools can already read and write flat csv files ok but it's pretty slow to process and makes for larger files than i feel they have to be, even gzipped

(e) it's probably cos i'm tired but i have no idea what you mean by 'flat' for some reason. it's not relational db data if that's what you mean, just giant single tables of timestamped data

Ciaphas fucked around with this message at 03:30 on Nov 10, 2017

# ? Nov 10, 2017 03:28

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

Ciaphas posted:

maximally right now is 5gb+ per file and growing (average is closer to 100MB per, and there are thousands of the things), and long term is at least 20 years, probably more. currently it's in a binary format that got homerolled 20 years ago for our tools, and we're finally taking the time for a format upgrade/migration

basically this decision is miles above my paygrade and i don't really have a stake or call in it but i'm curious and thinking about it anyway

our tools can already read and write flat csv files ok but it's pretty slow to process and makes for larger files than i feel they have to be, even gzipped

(e) it's probably cos i'm tired but i have no idea what you mean by 'flat' for some reason. it's not relational db data if that's what you mean, just giant single tables of timestamped data

'flat' in that context is exactly what you're describing, as opposed to some kind of database with a bunch of nested record or something.

So CSVs deffo aren't anywhere near as fast or compact as binary data, that's true! But they're super flexible, and most importantly human readable.

Guaranteed that somebody's life is gonna be easier in 10 years when they need to dig up some old data - they don't need to worry about what happens if Carl forgets to document his snowflake binary format, and they don't need to worry about whether there will be library support in Visual FutureLangGnu++2069

Even if the files take twice the space and load at half the speed, IMO that's a small price to pay for a future-proof format

it's also nice to be able to use grep and friends on your data

Poopernickel fucked around with this message at 04:04 on Nov 10, 2017

# ? Nov 10, 2017 04:02

silvergoose: Mar 18, 2006; IT IS SAID THE TEARS OF THE BWEENIX CAN HEAL ALL WOUNDS

note: please please do not make human readable files tab separated

because invariably the next human to edit it will do so with spaces, and then the file format will be broken

# ? Nov 10, 2017 04:04

Shaggar: Apr 26, 2006

finding reliable media for 20 years of storage will probably be more of a challenge than space requirements.

# ? Nov 10, 2017 04:08

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

silvergoose posted:

note: please please do not make human readable files tab separated

because invariably the next human to edit it will do so with spaces, and then the file format will be broken

iawtp

# ? Nov 10, 2017 04:09

silvergoose: Mar 18, 2006; IT IS SAID THE TEARS OF THE BWEENIX CAN HEAL ALL WOUNDS

Poopernickel posted:

iawtp

followon note: importing files into hive tends to mean tab separated files

# ? Nov 10, 2017 04:11

Sapozhnik: Jan 2, 2005; Nap Ghost

gonadic io posted:

https://twitter.com/joe_jag/status/510048646482894848?lang=en

# ? Nov 10, 2017 04:12

mystes: May 31, 2006

Shaggar posted:

finding reliable media for 20 years of storage will probably be more of a challenge than space requirements.

There's that new archival dvd/blu-ray format m-disc which supposedly will last an extremely long time, but obviously hasn't actually been tested in real use for that period so YMMV.

# ? Nov 10, 2017 04:12

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

store your data ~*iN tHe cLoUd*~

what could possibly go wrong

# ? Nov 10, 2017 04:17

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

Ciaphas posted:

actual question, does anyone know if any implementations of EXI for C/C++ or C#/.NET are any good

guy at work is proposing that for long term storage format for data we're working with and it looks like a good fit in theory

the problem with EXI is who knows whether or not that poo poo will take off, or whether it'll be the minidisk of data

if its the latter, somebody's choices will eventually be:
- get stuck with whatever language you're using now (.NET/C# in 2037, anybody??)
- write a complicated rear end parser themselves
- bitch about the data format and then convert it all to the flavor of the week data format 20 GOTO 10

# ? Nov 10, 2017 04:28

Luigi Thirty: Apr 30, 2006; Emergency confection port.

Oooookay yospos let's give this a go again. This time I have a way better ability to debug and get the intermediate results than last time I tried this.

I have a fixed-point multiplication routine that works for positive numbers but gives incorrect answers when I'm trying to multiply two negative 16.16 numbers together. The multiply instruction is 16-bit and produces a 32-bit result.

ex: A=0xFFFFFB88, B=0xFFFFFB88 . A * B = 0x00000013.
This routine says the result is 0xF7100013.

The logic and intermediate results I have (with assembly code in case it's helpful at all) are...

code:

	;; A.i * B.f
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4 			; Fetch B
	shrq	#16,FIXED_PRODUCT_TEMP3				; Shift A right 16 bits to get the integer portion. 0 is shifted in
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP4			; Mask off the integer portion of B
	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0xFB870478)

	;; A.f * B.i
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP3			; Mask off the integer portion of A
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift B right, 0 is shifted in
	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0xFB870478)

	;; (A.i * B.i) << 16
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	shrq	#16,FIXED_PRODUCT_TEMP3				; Shift A right to get the integer portion
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift B right to get the integer portion

	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	shlq	#16,FIXED_PRODUCT_TEMP4				; Shift the result left 16 bits from fraction to integer position. 0 is shifted in
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0x00010000)

	;; (A.f * B.f) >> 16
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP3			; Mask off the integer portion of A
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP4			; Mask off the integer portion of B

	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift the result right 16 bits into the fraction position, 0 is shifted in
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0x0000F723)

What should I be looking at for figuring out how to calculate the correct product with negative numbers?

# ? Nov 10, 2017 04:37

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

thanks for all the feedback. kind of an interesting/mildly complicated problem that i'm glad i don't have to make decisions on (but will probably have a hand in implementing because i am a cog in the machine~)

(e) also i didn't even think about the storage media. i wonder what they've been using up to now, i'm not privy to that

probably a shitload of 9tracks or something :\

# ? Nov 10, 2017 04:37

mod saas: May 4, 2004; Grimey Drawer

Poopernickel posted:

rear end parser

isn't facebook already working on this

# ? Nov 10, 2017 04:47

Poopernickel: Oct 28, 2005; electricity bad; Fun Shoe

mod saas posted:

isn't facebook already working on this

i heard about that yeah, facebook's working on rear end recognition for hands free sign-in

jam that phone down yo pants

# ? Nov 10, 2017 05:01

ComradeCosmobot: Dec 4, 2004; USPOL July

gonadic io posted:

https://twitter.com/joe_jag/status/510048646482894848?lang=en

this looks easy and fun and definitely not hell to implement correctly

# ? Nov 10, 2017 05:28

Doc Block: Apr 15, 2003; Fun Shoe

Luigi Thirty posted:

code:

	;; A.i * B.f
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4 			; Fetch B
	shrq	#16,FIXED_PRODUCT_TEMP3				; Shift A right 16 bits to get the integer portion. 0 is shifted in
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP4			; Mask off the integer portion of B
	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0xFB870478)

	;; A.f * B.i
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP3			; Mask off the integer portion of A
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift B right, 0 is shifted in
	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0xFB870478)

	;; (A.i * B.i) << 16
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	shrq	#16,FIXED_PRODUCT_TEMP3				; Shift A right to get the integer portion
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift B right to get the integer portion

	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	shlq	#16,FIXED_PRODUCT_TEMP4				; Shift the result left 16 bits from fraction to integer position. 0 is shifted in
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0x00010000)

	;; (A.f * B.f) >> 16
	movefa	TEMP1,FIXED_PRODUCT_TEMP3			; Fetch A
	movefa	TEMP2,FIXED_PRODUCT_TEMP4			; Fetch B
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP3			; Mask off the integer portion of A
	and	LOWORD_MASK,FIXED_PRODUCT_TEMP4			; Mask off the integer portion of B

	mult	FIXED_PRODUCT_TEMP3,FIXED_PRODUCT_TEMP4		; Multiply A and B
	shrq	#16,FIXED_PRODUCT_TEMP4				; Shift the result right 16 bits into the fraction position, 0 is shifted in
	add	FIXED_PRODUCT_TEMP4,FIXED_PRODUCT_RESULT	; accumulate. (result += 0x0000F723)

What should I be looking at for figuring out how to calculate the correct product with negative numbers?

Could the problem be that you're shifting over the sign bit when you shift right to get the integer portion? Looking at the docs, it looks like the SHA and SHARQ instructions preserve the sign bit when shifting right.

# ? Nov 10, 2017 05:29

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

So here's why it's hard to figure out: your routine is working perfectly. 0x00000000FFFFFB88 * 0x00000000FFFFFB88 does in fact equal 0xFFFFF7100013F840.

If you did a piecewise multiply of 0xFFFFFFFFFFFFFB88 * 0xFFFFFFFFFFFFFB88, it'd work. And the bits you're missing are two copies of (0xFFFF * 0xFB88) << 16.

Easy fix is to, if A is negative, add (-B.f) << 16, and vice versa if B is negative. (Essentially, you'd be sign-extending the inputs without doing as much work). Alternatively, only multiply positive numbers and track the sign of the result separately.

# ? Nov 10, 2017 05:30

carry on then: Jul 10, 2010; by VideoGames
(and can't post for 10 years!)

Doc Block posted:

Could the problem be that you're shifting over the sign bit when you shift right to get the integer portion? Looking at the docs, it looks like the SHA and SHARQ instructions preserve the sign bit when shifting right.

drat, so close to SHAQ or SHART

# ? Nov 10, 2017 05:57

Luigi Thirty: Apr 30, 2006; Emergency confection port.

Jabor posted:

So here's why it's hard to figure out: your routine is working perfectly. 0x00000000FFFFFB88 * 0x00000000FFFFFB88 does in fact equal 0xFFFFF7100013F840.

If you did a piecewise multiply of 0xFFFFFFFFFFFFFB88 * 0xFFFFFFFFFFFFFB88, it'd work. And the bits you're missing are two copies of (0xFFFF * 0xFB88) << 16.

Easy fix is to, if A is negative, add (-B.f) << 16, and vice versa if B is negative. (Essentially, you'd be sign-extending the inputs without doing as much work). Alternatively, only multiply positive numbers and track the sign of the result separately.

okay sweet this fixed it. i moved my rotation matrix calculation over to the DSP and got a bit of a speed increase. nothing major until I move the actual Matrix4x4_Multiply function from the CPU to the DSP...

the 68000 unsigned multiplication instruction requires 70 cycles in the worst case and the CPU runs at 13.295MHz, so worst case ~5.265uS per multiplication instruction
the DSP's unsigned multiplication instruction requires 3 cycles in all cases and the CPU runs at 26.56MHz, so ~0.112uS per multiplication instruction. wow! that should be enough to... spin my goddamn cube

carry on then posted:

drat, so close to SHAQ or SHART

there's an IMACN instruction if you've got a racist iMac

# ? Nov 10, 2017 06:24

Doc Block: Apr 15, 2003; Fun Shoe

carry on then posted:

drat, so close to SHAQ or SHART

sorry, but the docs say SHARQ is the best

Doc Block fucked around with this message at 07:05 on Nov 10, 2017

# ? Nov 10, 2017 06:41

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

Luigi Thirty posted:

spin my goddamn cube

Luigi make the wild demo of cubes spinning on legacy hardware set to techno music

# ? Nov 10, 2017 11:04

fart simpson: Jul 2, 2005; DEATH TO AMERICA

ComradeCosmobot posted:

this looks easy and fun and definitely not hell to implement correctly

# ? Nov 10, 2017 11:41

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

terraform is really good

way better than than our home built dsl that wraps cloudformation i tell you what

# ? Nov 10, 2017 15:31

Mao Zedong Thot: Oct 16, 2008

Blinkz0rz posted:

terraform is really good

way better than than our home built dsl that wraps cloudformation i tell you what

My dude wait until you start forking terraform providers

# ? Nov 10, 2017 16:15

Mao Zedong Thot: Oct 16, 2008

Nah we just unforked thank god, but it was pretty fun for a bit.

Terraform rules except for random resources that prevent you doing (reasonable) cyclical stuff

# ? Nov 10, 2017 16:17

my homie dhall: Dec 9, 2010; honey, oh please, it's just a machine

I wish there was a sane infrastructure tool that didn�t require me to write either json or hcl

# ? Nov 10, 2017 16:26

Schadenboner: Aug 15, 2011; by Shine

Poopernickel posted:

store your data ~*iN tHa cLuRb*~

what could possibly go wrong

# ? Nov 10, 2017 16:33

cinci zoo sniper: Mar 15, 2013

pandas feels fairly awkward after r

# ? Nov 10, 2017 16:39

HoboMan: Nov 4, 2010

ComradeCosmobot posted:

this looks easy and fun and definitely not hell to implement correctly

how is NASA's site a laggy mess when there's nothing fancy happening in the layout?

# ? Nov 10, 2017 17:00

HoboMan: Nov 4, 2010

about a week after switching to git from tfsvc i am kinda under the impression that shaggar, might indeed, have been right

# ? Nov 10, 2017 17:27

Shaggar: Apr 26, 2006

tfsvc is bad and so is git but they are bad in totally unique ways

# ? Nov 10, 2017 17:35

Xarn: Jun 26, 2015

HoboMan posted:

about a week after switching to git from tfsvc i am kinda under the impression that shaggar, might indeed, have been right

Spent about half a year working with TFS and sometimes I kinda miss easy shelvesets and gated check-in integration, but the way branches work in TFS mean I never, ever want to use it again.

# ? Nov 10, 2017 17:36

Luigi Thirty: Apr 30, 2006; Emergency confection port.

HoboMan posted:

how is NASA's site a laggy mess when there's nothing fancy happening in the layout?

it�s probably hosted on surplus 286s

# ? Nov 10, 2017 17:47

JawnV6: Jul 4, 2004; So hot ...

plus it has to come all the way from space

# ? Nov 10, 2017 17:54

Adbot: ADBOT LOVES YOU

# ? Jun 1, 2024 21:44

Ellie Crabcakes: Feb 1, 2008; Stop emailing my boyfriend Gay Crungus

There is a ~2 second lag between the secret moon base where they faked the Mars rover and the earth, so that makes sense.

# ? Nov 10, 2017 18:14

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > i actually have reasons to post here unlike half of the people with impostor syndrome

«‹›239 »