Wachter posted:An announcement of the death of a colleague came in... send condolences in Jokerman
|
|
#
?
Jan 26, 2016 17:06
|
|
|
|
| # ? Jun 3, 2024 22:36 |
|
|
We already backup to a removable disk and ... sometimes... take off-site.. but they are adamant about wanting "cloud" backup. But of course they also don't want to pay for the additional upload. I've not found any backup software that will do proper differentials on the compressed full SQL backups. It always ends up copying the entire file :/. We don't have control over the backup configuration for the SQL instances. We are given the nightly full backup files and that's it. Trying to explain all of this falls on deaf ears most of the time. The CEOs response is always "With today's technology I can't believe you cant make it happen!" Well I could make it happen, if they where willing to pay! I am just fed up with them bugging me about it! They are just going to have to come up with better arrangement for taking the backup disk out of the building on a daily basis. Sorry, turned into a rant.
|
|
#
?
Jan 26, 2016 17:23
|
|
|
You can't request that the ERP company provides differential backups instead? Otherwise there's really no way to solve it. Maths says no, and if you present the calculation that you'd tie up your internet connection for 20 hours every day it's pretty hard to argue against. Collateral Damage fucked around with this message at 17:41 on Jan 26, 2016 |
|
#
?
Jan 26, 2016 17:37
|
|
|
Collateral Damage posted:Otherwise there's really no way to solve it. Maths says no, and if you present the calculation that you'd tie up your internet connection for 20 hours every day it's pretty hard to argue against. Only 20 hours? Heh. I had a job where I was mandated to pull backups from our data center to local storage for ~reasons~. when I told them that pulling down the database snapshots alone would take 26 hours a day and the only way to work around this would be to ship tapes or increase the size of our pipe, I was told "you're the infrastructure guy. Figure it out." My response was that I HAD figured it out and the solution is either ship tapes or increase the pipe. We ended up not doing any backups at all. :woot:
|
|
#
?
Jan 26, 2016 18:14
|
|
|
The vendor we're working with for this install put the vertical PDUs on the same side that the cables are run on. This is making it harder to make everything look pretty. Oh and they plugged the UPS units into the PDUs...instead of plugging the PDUs into the UPS units. Pretty sure that does jack squat in terms of protection. Are you loving kidding me. Oh look, none of their other equipment is protected by UPS units! They better hope the generator kicks on immediately. pr0digal fucked around with this message at 19:08 on Jan 26, 2016 |
|
#
?
Jan 26, 2016 18:23
|
|
|
Carbonite server backup does what you want.
|
|
#
?
Jan 26, 2016 19:15
|
|
|
ConfusedUs posted:Carbonite server backup does what you want. Yes but that costs money
|
|
#
?
Jan 26, 2016 19:17
|
|
|
pr0digal posted:The vendor we're working with for this install put the vertical PDUs on the same side that the cables are run on. This is making it harder to make everything look pretty. LOL. They probably don't know what a UPS does, just that it needs to plug in, and things plug into the PDUs.
|
|
#
?
Jan 26, 2016 19:21
|
|
|
KillHour posted:LOL. They probably don't know what a UPS does, just that it needs to plug in, and things plug into the PDUs. He's like "oh it's got a battery, it'll kick on if the PDU dies" Yes, that's nice. I had to really baby step him through the steps of "what plugs into what" to give protection.
|
|
#
?
Jan 26, 2016 19:24
|
|
|
go3 posted:Yes but that costs money I missed the part where there was zero budget, I guess. CSB would be cheaper by far than an upgraded connection. It'll compress the backups (not uncommon to see 85% compression on databases or more). And it will do true diffs or point in time incrementals on a schedule of your choosing, up to every fifteen minutes.
|
|
#
?
Jan 26, 2016 19:40
|
|
|
ConfusedUs posted:I missed the part where there was zero budget, I guess. I'll give it a try... I had previously trialed Crashplan /Code42, and 2 others (names escape me at the moment), but they all ended up wanting to transfer almost the entire file after the initial upload.
|
|
#
?
Jan 26, 2016 20:11
|
|
|
I got asked to fix a row of computers in one of the Customer Service Rep bays. They had turned off, and weren't coming back on. On top of that, the peeps were made because the their UPS didn't seem to be working. They had the UPS plugged into wall, and the power strips were also plugged into the wall. It was really hard not to laugh as I explained they had to plug the power strip into the UPS in order for UPS to do ANYTHING. It had been like that for 2 years apparently.  e: Oh yeah, and the Power Strip was just turned off 
Turtlicious fucked around with this message at 20:54 on Jan 26, 2016 |
|
#
?
Jan 26, 2016 20:48
|
|
|
stevewm posted:I'll give it a try... I had previously trialed Crashplan /Code42, and 2 others (names escape me at the moment), but they all ended up wanting to transfer almost the entire file after the initial upload. Check your backup compression options. I had this problem at one point with a PostgreSQL DB on my home machine and Crashplan. Compression shrank the dump from roughly 3TB to about 1.8T, but it meant that adding a few records resulted in so many changed blocks in the dump file that it was effectively a new file. Turning off compression made it a bigger backup but the changes were all localized and the upload bandwidth went to almost nothing. Politically speaking, reach out to sales at some of those providers and ask them for a recommendation. If you can get them to come back with the same conclusions, then it shows your management that 1) it's not just you and 2) you took the initiative to bring in resources to meet the challenge they set you. If they come back with an idea that makes sense, you can try it. If it works, you were still the guy who found a vendor who could get it done. If it fails, you blame the vendor. It's wins for you all around if you play the cards right.
|
|
#
?
Jan 26, 2016 21:18
|
|
|
While I don't work in IT, I do have a wonderful UPS story that your gentlemen may enjoy. I used to work at a mid sized independent retailer. It had a single location and was an extremely seasonal business; 85% of the years business was conducted in 4 weeks. It was also run by technical illiterates when I started; transactions were completed on 70's era registers and they had only moved to electronic credit card transactions a couple years before I started there (2003). This state of affairs continued for a few years until finally, the new office manager managed to finally convince the owners that there would be value in a new POS system that would also finally (gasp) actually have a barcode database. So off they went, the owners and new office manager. A room was retrofitted in the office for two server racks, air conditoning was installed and ten new touch screen POS systems with bar code scanners were installed! Also provisioned for each till and the server was sufficient battery capacity to allow for a 30 minute power outage. Installations went well, the change over was smooth and the retraining needed was minimal. Much rejoicing was had by all. Two years later, at 930 one fine sunny morning, the power goes out on the single busiest day of the year. No problem, the UPS on all the tills and server kick in, life goes one, everyone has a laugh. Until the 30 minutes are up. The power hasn't come back on. All of a sudden the hundreds of people trying to purchase their items can't. The tills don't work. Hundreds of thousands of dollars of purchases are abandoned as the black out stretches for hours. 9 hours in this case. The owners are furious. Something had to be done! A week later I find out what something is. Not a generator backup; apparently you need permits for that. Nope, apparently the decision was made that we were insufficiently provisioned for battery back up. How much back up did the owners decide was necessary? Why, enough to have lasted through the 9 hours that we lost! Plus an extra couple hours of course. For every single till, server, switch, modem, whatever. Which is why I found myself using the forklift to unload skid after skid of immense UPS units, each weighing 5 or 600 pounds. So on the bright side, the entire store is now essentially able to be battery powered, on the downside I done even want to know how much each of the units cost. I also can't look at a UPS without immediate back spasms. After I left I found out that they had another day long black out a few years after this. Apparently the batteries didn't last the full expected time. After this second loss they actually went through the whole permitting process to get a backup generator, which ended up costing far less then they spend on the new batteries in the first place. Moral of story: do it right the first loving time, it's cheaper.
|
|
#
?
Jan 26, 2016 23:37
|
|
|
Gorau posted:While I don't work in IT, I do have a wonderful UPS story that your gentlemen may enjoy.
|
|
#
?
Jan 26, 2016 23:40
|
|
|
stevewm posted:I'll give it a try... I had previously trialed Crashplan /Code42, and 2 others (names escape me at the moment), but they all ended up wanting to transfer almost the entire file after the initial upload. There's a free trial, at least. Just a general PSA for if you start shopping around vendors for MSSQL backups. (Also Sharepoint and Exchange) MSSQL behaves very, very poorly when it comes to multiple backup sources acting on the same database, if you try to do incremental or differential backups with one or more of those sources. If all your sources are doing full backups only, you're fine. But if you have more than one source, any incs/diffs from any source are gonna get hosed up. It will happen. The basic idea is that MSSQL (through volume shadowcopy services) is what serves up the backup data. The backup applications take whatever MSSQL gives them. When there are multiple sources, MSSQL may be giving out bad data, and no one will be the wiser. Essentially, MSSQL doesn't keep track of what application it's talking to, just that a backup occurred. This can mean that one application gets some portion of the data. The other application gets another portion. If you're lucky, MSSQL realizes something isn't right when source A asks for something that was already given to source B and fails the backup process. If you're unlucky, MSSQL won't realize anything's wrong, and it gives source B whatever changed since source A took the backup. Without including whatever was in Source A's backup. In this situation, you end up with gaps in your backup data. Gaps render everything after the gap non-restorable. This is super bad because, in most cases, the backup applications can't realize anything is wrong, because they just take whatever MSSQL gives them. They happily report everything is successful. Sharepoint behaves just like MSSQL, because it has a MSSQL backend. Exchange is smarter; it ALMOST always realizes something is wrong, and will fail the VSS Writer involved. I made a lovely flowchart: https://dl.dropboxusercontent.com/u/27127/MultipleBackupSources.pdf
|
|
#
?
Jan 26, 2016 23:42
|
|
|
ConfusedUs posted:MSSQL Backup Advice Is this assuming everything is stored in the same instance, or the same database? I am assuming database since that is what you said, so if App1 needed backup data to Database 1 and App2 requested for Database 2 it would be fine even if App2 was incremental correct?
|
|
#
?
Jan 26, 2016 23:55
|
|
|
Alighieri posted:Is this assuming everything is stored in the same instance, or the same database? I am assuming database since that is what you said, so if App1 needed backup data to Database 1 and App2 requested for Database 2 it would be fine even if App2 was incremental correct? Yeah, it's per database. It's no biggie if DB1 is backed up by app 1 and DB2 is backed up by app 2. It's only when DB1 is backed up by apps 1&2 that it's a problem.
|
|
#
?
Jan 27, 2016 00:06
|
|
|
How about logshipping? You just need to ship out a single full backup once to start, and then you can ship over logbackups as you get them. You can then run integrity checks on the off-site copy or back it up again to have more full backups under your own control. Also it can serve as a "luke-warm-standby" in case poo poo hits the fan (provided your pipe can handle the normal transacion volume and the latency doesn't hurt too bad).
|
|
#
?
Jan 27, 2016 00:17
|
|
|
pr0digal posted:The vendor we're working with for this install put the vertical PDUs on the same side that the cables are run on. This is making it harder to make everything look pretty. Ugh... sometime I need to come in after hours and properly hook our stuff up and refit things, there are two PDUs with one covering the square holes at the bottom of the rack which can be removed entirely, and only the router and three servers are connected to UPS's (one of which has an incomplete rack kit thus sits on the floor). But that means coming in after hours, which means getting home later at night.
|
|
#
?
Jan 27, 2016 00:28
|
|
|
ConfusedUs posted:I made a lovely flowchart: Holy poo poo, I think you just solved an ongoing problem we've been having at a client, thanks buddy!
|
|
#
?
Jan 27, 2016 00:44
|
|
|
Segmentation Fault posted:send condolences in Jokerman Please, it's a solemn occasion. Diablo font.
|
|
#
?
Jan 27, 2016 01:50
|
|
|
HELVETICA
|
|
#
?
Jan 27, 2016 02:11
|
|
|
KoRMaK posted:Hey! We have some ladies in this thread too ya know! Apologies on his behalf for the offense, miss  Entropic posted:Please, it's a solemn occasion. Diablo font. That's a terrible idea you insensitive jerk  Always use Chiller font for bereavements and condolences
|
|
#
?
Jan 27, 2016 02:11
|
|
|
Ozz81 posted:Apologies on his behalf for the offense, miss
|
|
#
?
Jan 27, 2016 02:57
|
|
|
KoRMaK posted:I didn't say I was one of 'em!
|
|
#
?
Jan 27, 2016 04:01
|
|
|
MF_James posted:Holy poo poo, I think you just solved an ongoing problem we've been having at a client, thanks buddy! Glad to be of assistance. I seriously see this poo poo daily. At least. And you wouldn't believe how many of the people who end up in that bad situation had no idea they even had more than one backup source in play.  This behavior matches what I'd expect if you had more than one backup operating against the same databases. Do you have another backup program? No, of course not! Let's see what event logs say. Hey, what do you know, MSSQL logged a backup event at <datetime>, and there was no backup from my program at that time. I have no idea what that it could be! I'm serious! Well you should find out. Here's some knowledgebase articles about why doing this is bad. Your backups will continue to break on a regular basis until you resolve the conflict.
|
|
#
?
Jan 27, 2016 05:29
|
|
|
Yuup. Conflicting backups and running full recovery databases with no log backups are the two most common misconfigurations I run into with SQL Server.
|
|
#
?
Jan 27, 2016 10:30
|
|
|
Entropic posted:Please, it's a solemn occasion. Diablo font. Fraktur is suitable for any occasion 
|
|
#
?
Jan 27, 2016 11:50
|
|
|
ConfusedUs posted:Glad to be of assistance. Would something that takes a whole server backup, like say veeam, also cause this error if you then had separate DB backups?
|
|
#
?
Jan 27, 2016 13:45
|
|
|
As far as I know Veeam uses VSS to get the application-aware backups that allow you to perform individual restores of Exchange objects etc., so I can't see why it wouldn't affect that. If you're just taking a storage snapshot and not doing any guest processing then you should be OK.
|
|
#
?
Jan 27, 2016 14:14
|
|
|
ConfusedUs posted:.......MSSQL backups. (Also Sharepoint and Exchange).... Yeah, all of this is exactly why our ERP company has it setup the way it is. They have the SQL instance configured to make a log backup every hour, and then a full backup nightly. SQL is configured to keep the last 2 full backup files. Any backup app we use does not actually access or touch the SQL instance itself, we simply copy the backup files that SQL itself generated.
|
|
#
?
Jan 27, 2016 14:25
|
|
|
stevewm posted:Yeah, all of this is exactly why our ERP company has it setup the way it is. I wish more people understood and did this.
|
|
#
?
Jan 27, 2016 16:02
|
|
|
LordVorbis posted:Would something that takes a whole server backup, like say veeam, also cause this error if you then had separate DB backups? Yes. In fact it was veeam that lead me down the path to discovering this, some years ago.
|
|
#
?
Jan 27, 2016 16:03
|
|
|
ConfusedUs posted:Yes. In fact it was veeam that lead me down the path to discovering this, some years ago. Well gently caress poo poo and double gently caress.
|
|
#
?
Jan 27, 2016 16:24
|
|
|
Two young adults came in. "I know this is a vape, and not a computer, but..."
|
|
|
#
?
Jan 27, 2016 16:28
|
|
|
"adults"
|
|
#
?
Jan 27, 2016 16:31
|
|
|
LordVorbis posted:Would something that takes a whole server backup, like say veeam, also cause this error if you then had separate DB backups? Veeam needs you to explicitly enable application-aware VSS snapshots, it's not enabled by default in a new job. Thanks for making me double-check all my jobs, thread. On the upside I did discover a recently created MSSQL db with a Full recovery model that nobody had scheduled log backups on.
|
|
#
?
Jan 27, 2016 16:51
|
|
|
Segmentation Fault posted:Two young adults came in. I bet it had a USB port on it.
|
|
#
?
Jan 27, 2016 16:57
|
|
|
|
| # ? Jun 3, 2024 22:36 |
|
spog posted:I bet it had a USB port on it. It did! A standard Micro B connector, the type you use on cell phones. The vape pen's screen didn't work, guy reckoned the ribbon cable inside got loose after taking a fall. I have no interest in an ad-hoc interactive learning experience with vape pens and I have other things to be doing, so I declined and referred him to a vape shop, failing that, the manufacturer, failing that Google.
|
|
|
#
?
Jan 27, 2016 17:17
|
|