Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Corla Plankun
May 8, 2007

improve the lives of everyone

Hed posted:

So if you're sending logs to a managed provider like Datadog, but some fraction of the logs contain transactions that you want to hang on to for billing questions, do you just have a cron job pull out the info from Datadog and put it into your own holdings?

I see most providers center around 15-day stuff, so I like I'm thinking about it wrong if I want to hold on to some things for 30-90 days.

definitely don't rely on logs for core business functions but furthermore definitely don't have datadog in-the-loop for anything important. We used it at my last job and ddog was great for general stuff (looking to see if a thing was done restarting, counting the lower bound of errors, etc) but it would routinely have outages/delays/lost logs to the point that if something was missing in datadog we just assumed it was fine

Adbot
ADBOT LOVES YOU

Corla Plankun
May 8, 2007

improve the lives of everyone
you need an append-only table in a database that keeps track of the transaction event stream, and you need to write to it in a more robust way than just using stdout or ddog's possibly inconsistent log sending

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply