|
Solution: don't use unreliable links.
|
#
?
Jun 25, 2013 18:20
|
|
|
|
| # ? May 26, 2024 17:27 |
|
|
Shaggar posted:Solution: don't use unreliable links. you first
|
|
#
?
Jun 25, 2013 18:24
|
|
|
Shaggar posted:Solution: don't use unreliable links. Next problem: timeouts. They leave you uncertain on whether the problem is (to name a few that can still happen with 'reliable' links):
And each of them can fail in the same way as above. Moreover, what can fail is your 'rollback' RPC call, letting you unsure whether it did roll back or not. This is disregarding bugs of any kind. If your application is in any way dealing with side-effects following any single decision made where the failure or success assumption was wrong, you may end up doing the equivalent of duplicating messages even over reliable links, negating a chunk of all that investment in hardware just to get more reliable links  So it turns out you need to actually think about your things and do things like have pre/post conditions, have idempotence, or other similar properties that you have to think of ahead of time for these things and not just assume they don't happen. RPC is the p-lang of distributed systems. MononcQc fucked around with this message at 18:56 on Jun 25, 2013 |
|
#
?
Jun 25, 2013 18:38
|
|
|
MononcQc posted:You are retrying a call that actually succeeded my favorite
|
|
#
?
Jun 25, 2013 18:54
|
|
|
tef posted:i know this is all a bit cjs but i've quit my captialist life and i'm gonna work for a non-proft and help teach kids to play and invent with code. oh so you're gonna start postin in here full time
|
|
#
?
Jun 25, 2013 18:57
|
|
|
MononcQc posted:Next problem: timeouts. They leave you uncertain on whether the problem is (to name a few that can still happen with 'reliable' links): Wrong. The far end should accept the retry and return either the status if the new attempt if it previously failed or the status of the old attempt if it previously succeeded or failed permanently. Both sides are then reconciled and everyone is happy. Rpc is fine and is not actually different than your overly complicated rpc-but-not-called-rpc systems.
|
|
#
?
Jun 25, 2013 18:58
|
|
|
MononcQc posted:laying the cold hold truth I actually implemented a monitoring/stress testing tool to help catch parts of our application that weren't idempotent. It handles and retries based on the type of errors it knows about. It's really hard to draw conclusions of where the problems occur, but it's fun stuff throwing 30,000 clients at our servers to see how they handle.
|
|
#
?
Jun 25, 2013 19:05
|
|
|
Condiv posted:the sbt izpack plugin uses yaml as its config file. it takes this yaml file and converts it into xml which it hands to the izpack thing. i had my own xml config file for izpack but the plugin demands i make a yaml one so it can covert it into xml. sbt is horrible andd the #1 reason i don't use scala more
|
|
#
?
Jun 25, 2013 19:07
|
|
|
your project definition is scala code! your project definition definition is also scala code! want to know how to do anything? use the source, luke!
|
|
#
?
Jun 25, 2013 19:09
|
|
|
gradle seemed decent
|
|
#
?
Jun 25, 2013 19:12
|
|
|
Nomnom Cookie posted:sbt is horrible andd the #1 reason i don't use scala more nah, sbt is generally pretty nice. the documentation is much better nowadays.
|
|
#
?
Jun 25, 2013 19:16
|
|
|
Can you build scala w/ maven?
|
|
#
?
Jun 25, 2013 19:18
|
|
|
Shaggar posted:Can you build scala w/ maven? yeah, and we used to. sbt builds our code faster and is more flexible though, so we switched.
|
|
#
?
Jun 25, 2013 19:20
|
|
|
Sounds like you weren't using maven right.
|
|
#
?
Jun 25, 2013 19:23
|
|
|
"More flexible" aka "lets us do terrible hacks in the build"
|
|
#
?
Jun 25, 2013 19:24
|
|
|
Cocoa Crispies posted:gradle seemed decent i'm very intrigued by gradle; the people who wrote "continuous delivery" make it sound a bit like "maven plus"
|
|
#
?
Jun 25, 2013 19:26
|
|
|
Shaggar posted:Wrong. The far end should accept the retry and return either the status if the new attempt if it previously failed or the status of the old attempt if it previously succeeded or failed permanently. Both sides are then reconciled and everyone is happy. This depends on how your RPC calls for transactions work and how many hosts are involved. What you suggest is safe only if: - Your operations are side-effect free, outside of commit logs - You have ACID distributed commits over the entire service-side - You want the entire system to lock up when there is a part of the service that becomes unavailable to others (which is usually indistinguishable from high latency or a timeout) - Keep your transaction/commit logs around for a relatively long time Otherwise, you will have problems. Side-effects in the operations of your RPC calls means a retry is not purely a retry and other entities may receive duplicated or unordered messages about events that happened. If you 'undo' the side-effects by sending a new one, you have to be aware that this 'undo' can fail in a similar way. If you don't have ACID commits over the cluster for the service (or have more than one service node), a retry can land on a different node and run at the same time as the other one that was still running, with either of them not knowing that the other was done, breaking your "return the previous result" scheme. This can be avoided if you make sure a client always contacts the same RPC endpoint, but unless you agree to have the system lock up, it means you may retry on another node when failing over and that you will break your scheme the same way. Also other clients on other nodes may call different RPC endpoints with similar operations and still break your assumptions. If you garbage-collect your transaction logs from time to time (or compact them, or drop them to favor a snapshot of the system state), a prolonged delay or timeout (say your operation is stuck in a queue, or your Java servers locked up because of a leap second), the previous accepted value for that transaction can be lost or impossible to retrieve, leading to a duplicated effort. Nevermind that your scheme basically reduces RPC to a database operation in some SQL store given what you need to make it safe for real.
|
|
#
?
Jun 25, 2013 19:29
|
|
|
Shaggar posted:"More flexible" aka "lets us do terrible hacks in the build" only if you consider any functionality not offered by maven a terrible hack. which i'm sure you do.
|
|
#
?
Jun 25, 2013 19:31
|
|
|
Shaggar posted:"More flexible" aka "lets us do terrible hacks in the build" can maven build x++?
|
|
#
?
Jun 25, 2013 19:38
|
|
|
tef posted:i know this is all a bit cjs but i've quit my captialist life and i'm gonna work for a non-proft and help teach kids to play and invent with code. code club is legit cool congrats
|
|
#
?
Jun 25, 2013 19:45
|
|
|
Cocoa Crispies posted:gradle seemed decent gradle to the grave
|
|
#
?
Jun 25, 2013 20:10
|
|
|
is gradle any good for building c++? i'm using maven currently to build a bunch of java code + produce some assets tbz2s and we're adding c++. i can write some makefiles or whatever but i'd prefer to keep One Build to Rule Them All if i can
|
|
#
?
Jun 25, 2013 21:15
|
|
|
Shaggar posted:Solution: don't use unreliable links. Solution: don't use Microsoft software.
|
|
#
?
Jun 25, 2013 21:16
|
|
|
Microsoft and java are the best.
|
|
#
?
Jun 25, 2013 21:18
|
|
|
tbf shaggar probably just has to deal with 1 app server + 1 db server so he doesn't need to worry about resilience
|
|
#
?
Jun 25, 2013 21:20
|
|
|
I have multiple servers but one db, but its mssql and not faddb so ur right I do t have to worry about failures.
|
|
#
?
Jun 25, 2013 21:22
|
|
|
MononcQc posted:This depends on how your RPC calls for transactions work and how many hosts are involved. What you suggest is safe only if: just have each RPC call return a CRDT with each partial success/failure state and the options available to the client at each forming a monotonic semilattice that converges to a failure state if the client doesn't keep retrying, ease peace
|
|
#
?
Jun 25, 2013 21:30
|
|
|
Max Facetime posted:just have each RPC call return a CRDT with each partial success/failure state and the options available to the client at each forming a monotonic semilattice that converges to a failure state if the client doesn't keep retrying, ease peace corba 3.1
|
|
#
?
Jun 25, 2013 21:32
|
|
|
Max Facetime posted:just have each RPC call return a CRDT with each partial success/failure state and the options available to the client at each forming a monotonic semilattice that converges to a failure state if the client doesn't keep retrying, ease peace They would only form a semilattice if the operations were idempotent, not commutative. ps. idempotence is the best. idempote every day
|
|
#
?
Jun 25, 2013 21:43
|
|
|
Max Facetime posted:just have each RPC call return a CRDT with each partial success/failure state and the options available to the client at each forming a monotonic semilattice that converges to a failure state if the client doesn't keep retrying, ease peace This is the super ideal "here's what you want to do and be rock solid" (that or idempotence if you want to keep retrying) but usually it just means being aware of things that may or may not fail, and designing around that (somewhat equivalent to not pushing out a git clone that believes merge conflicts will never happen for example), possibly reminding yourself that side-effects should be limited or maybe grouped together, and that if you can easily get idempotent operations, it might be decent to aim for that.
|
|
#
?
Jun 25, 2013 21:52
|
|
|
Beat the CAP theorem erryday by using infinite timeouts and never giving up on an operation, just believing it is very very late
|
|
#
?
Jun 25, 2013 21:59
|
|
|
MononcQc posted:Beat the CAP theorem erryday by using infinite timeouts and never giving up on an operation, just believing it is very very late The I2C library provided with the uC on my main project at work does this. You can probably imagine the results. (we eventually wrote our own for a variety of reasons, but this was the biggest)
|
|
#
?
Jun 25, 2013 22:09
|
|
|
OBAMA BIN LinkedIn posted:so basically anything that involves that involves things being related being related to other things
|
|
#
?
Jun 26, 2013 04:37
|
|
|
MononcQc posted:rpc is p-lang depends if you mean synchronous rpc, or async rpc using futures/callbacks/promises. i mean finagle is pretty neat i mean syncrhonous rpc, i.e function call that is mapped, isn't /all/ that bad for very simple interfaces if you don't really worry about side-effects, or retries, or timeouts. if something goes wrong, you throw an exception and let the client code handle it. i use sync rpc a lot but pretty much everything is idempotent, and timeouts are fixed but overridable. it isn't the great satan steve vinowski makes it out to be and for async stuff, although some advocate raw message passing as the true and future king of protocols, it's actually pretty conveient to be able to compose the result of rpc operations. now with futures, the code is a little more clunky but as a result it's very easy to compose operations that can operate concurrently. then you have the church-of-fielding style criticism that argues that network services should be more like filesystems, rather than a precompiled list of function stubs. the rationale being that it's easier to compose these systems, use uniform middleware and components, and it's incredibly flexible. but it's actually possible to layer one over the other, they aren't so contradictory. it's just a whole bunch of tradeoffs that people gloss over.
|
|
#
?
Jun 26, 2013 14:34
|
|
|
The last place I worked at used a combination of IBM MQSeries and ICE as their middleware They used MQ as a parallel scatter-gather RPC and they took ICE's super stateful remote object protocol and used it to build something stateless. In both cases the official API was a COM DLL that used the MQSeries and ICE libs/stubs as implementation details. I'm not actually criticising either of these things per se because honestly compared to the poo poo that we had to interface with their code was solid gold. The early 2000s vintage shambling C++ code base at the center of it though...  I still have nightmares about Boost mixed with ATL.Otto Skorzeny posted:The I2C library provided with the uC on my main project at work does this. You can probably imagine the results. (we eventually wrote our own for a variety of reasons, but this was the biggest) Have you ever worked with Freescale's Cortex-M parts by any chance? A hardware guy I talk to a lot spent like a solid week bitching about one that he was forced to use in one of his projects. Fun fact about these chips: there's a flash configuration page at loving 0x400 in ROM. Not at the end, or at the beginning (vector table lives there) or even immediately after the vector table which would kind of almost have made sense, just randomly at 0x400, because why the gently caress not. Write something unlucky to that page and you'll permanently lock out the ISP connections and then it's just liek "sorry abut ur bricked mcu mang D:" So anyway a friend of mine is working on a hobby project and decided to put a Teensy3 inside it and welp, guess what MCU that piece of poo poo uses. There's some mickey mouse Arduino dev environment for it but I need to turn it into a custom HID, obviously there's no LUFA support...
|
|
#
?
Jun 26, 2013 15:28
|
|
|
Only Freescale parts I've worked with are the S08s, which are pretty good in their niche although somewhat more obnoxious to use from a hw designer perspective than more "modern" uCs (I don't think the line has been really worked on since the Motorola days). A product we shipped about a year ago uses one because it was available in a package with 42 dedicated LCD pins to drive custom glass, and its low power modes were good. Some of them have banked memory though which is  My current project uses a Cypress chip built around a Cortex M3 and it's terribad. The whole analog section is thunderfucked, we've found multiple bugs that they were unaware of which their FAEs can confirm and send up the chain to design but don't know any workarounds to, etc. We like the CortexM in general though, so we're looking at STM and a couple other vendors for the projects we have in the pipeline.
|
|
#
?
Jun 26, 2013 15:41
|
|
|
Look at EFM32 if you're doing anything low-power. They're a bit on the pricey side but they're just dreamlike to program for. The only major drawback is that they don't have JTAG, only SWD (so no OpenOCD for you and their licensed J-Link on the dev/eval boards sucks out loud) and the USB core in the bigger parts is rather lovely IP core licensed from Synopsys that doesn't integrate well with the other stuff on that chip in the slightest. I kinda want to work in embedded but from what I hear a lot of HW guys will force you to use some terrible 8-bit crap instead of a nice Cortex. That and writing firmware for deep fat fryers or whatever doesn't exactly set my world on fire. Other than that embdev seems pretty fun
|
|
#
?
Jun 26, 2013 16:04
|
|
|
Mr Dog posted:they don't have JTAG NOPE Mr Dog posted:Synopsys although to be honest those wonder geckos look pretty sweet if my project continues on i may actually have to check those out because id kinda like a very low power arm at this point just looked at the datasheet hot drat that's low power that's almost msp430 territory Bloody fucked around with this message at 16:10 on Jun 26, 2013 |
|
#
?
Jun 26, 2013 16:06
|
|
|
dear yospos i'm gonna be doing a new jerb where I sorta you know write code, give talks and generally do a whole bunch of poo poo to mean that people can help kids learn through play please give me a job title
|
|
#
?
Jun 26, 2013 17:12
|
|
|
|
| # ? May 26, 2024 17:27 |
|
|
tef posted:dear yospos autism specialist
|
|
#
?
Jun 26, 2013 17:13
|
|