Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Knormal
Nov 11, 2001

Visio is our $SOFTWARE that I keep having to explain why we can't just put on everyone's computer, no matter how good your business justification is. I even got asked by my boss' boss why this one unit doesn't have Visio on all their PCs when she's the one who turned down our attempt to buy more Visio licenses earlier this year because it was too expensive and "not mission critical".

Adbot
ADBOT LOVES YOU

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

Visio and Project. I loooove getting emails from managers asking to put one or both on a users machine. My go to response is "Sure, a license is $cost, what account number should I charge it to?"

Then never getting a reply.

At least Visio has a loving free reader. MS is dicks about Project.

JohnnyCanuck
May 28, 2004

Strong And/Or Free
A Professional Services Request came in!

quote:

"Hi! We're a large client with multiple sites who's currently in the middle of a massive XP --> Win7 upgrade! Everybody decided it would be too tough to actually record and manage the huge amounts of turnover and change in the CMDB, so we did the next best thing!

WE SET UP A COMPLETELY UN-VALIDATED EXCEL SPREADSHEET! :D :D :D

Now, the spreadsheet gets updated every day, and none of the data is actually validated, as we mentioned, and sometimes vital fields (like, say, ASSET TAG or SERIAL NUMBER) just go reported as "N/A", and we're using weird location-based filters on the spreadsheet, and half the time there's no actual matching between the old CI Configuration and the new one, and there's lots of entry points that use a single entry to record two additional monitors or something, but...

Can you create an ODBC-based integration to perform an automatic daily import into our CMDB?

Also, each import will need to be multi-step to ensure that we get each new Configuration set up, and then the PC or laptop assigned (which we still call a CPU here), and then any additional monitors or PERSONAL PRINTERS assigned, and also unassign all the old stuff while you're at it

Also, we need a brand new error-reporting module to spit back each item that didn't get imported into an easy-to-read format with a clear explanation of why that record failed

Also to minimize errors we need YOU to massage the data

Also we need this yesterday

Also we don't really want to pay too much for this

Also we still haven't paid for a two day job that ballooned out into a two-week job TWO YEARS AGO because we keep on finding new things to nitpick or new functionality that absolutely needed to be in there but wasn't on the initial scope

Also we feel the job can be done in about two days so that's all we want to pay for

Also we want a discount

So when can you get started?"

Inspector_666
Oct 7, 2003

benny with the good hair

JohnnyCanuck posted:

A Professional Services Request came in!

This sounds remarkably like a request one of our dudes just got.

rolleyes
Nov 16, 2006

Sometimes you have to roll the hard... two?
I would say the correct answer to "so when can you get started?" should be "when you've paid your outstanding invoice".

After that, start nitpicking their proposal :v:

Powerful Two-Hander
Mar 10, 2004

Mods please change my name to "Tooter Skeleton" TIA.


Knormal posted:

Visio is our $SOFTWARE that I keep having to explain why we can't just put on everyone's computer, no matter how good your business justification is. I even got asked by my boss' boss why this one unit doesn't have Visio on all their PCs when she's the one who turned down our attempt to buy more Visio licenses earlier this year because it was too expensive and "not mission critical".

It really annoys me when people take my nice tidy Visio diagrams then because they don't have a Visio license, screenshot it and MS paint additions on to it then circulate that. I need to start watermarking my stuff or something.

Second place for OCD irritation is when people send me diagrams where the boxes don't line up or the lines aren't straight. Use the construction lines you idiots :mad:

JohnnyCanuck
May 28, 2004

Strong And/Or Free

rolleyes posted:

I would say the correct answer to "so when can you get started?" should be "when you've paid your outstanding invoice".

After that, start nitpicking their proposal :v:

Oh, that was my reaction.
But guess who gets to choose what jobs we accept?

Oswald Kesselpot
Jan 14, 2008

HONK HONK HONK
Anyone have links to a couple of good articles on why end users should not have full Admin rights in an ERP system?

Lum
Aug 13, 2003

KillHour posted:

It's really... odd actually. The cameras are pointed at digital signage. The company needs a way to prove that the displays are showing what they're supposed to when they're supposed to, so they are recording all 10,000 displays they have, and having the video compared against what SHOULD be displayed there. They want to store the data all at the same place so they can have people "forensically analyze" the feeds (their words, not mine) in their NOC to make sure everything's correct. It's basically one of the craziest requests I've ever had.

I realise that this is from several pages back, but there are better ways of doing this.

For example you can run a length of fibre from each individual LED in the sign and hook it up to a sensor and record that. Your video is now 300x50 in 1-bit colour.

The request isn't as crazy as you might think. I have to be very vague here, but situations exist where the failure of even a single LED in the sign can affect the outcome of a court case.

Fake edit: Just read what this is actually for!

Gerdalti
May 24, 2003

SPOON!
A little off topic, but this thread goes this way every page. What do you like for enterprise backup? Mostly Windows servers, a couple linux servers, and a single Mac server. I've got about 11 TB of data.
I'd prefer a centralized console to manage the backups/send notifications. The ability to push backups to Amazon's S3 (or something) is drat near a must. I don't want to go back to Iron Mountain for offsite backups.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
I fixed it, by downloading the cloud!

I love the people I work with

Galler
Jan 28, 2008


Powerful Two-Hander posted:

Second place for OCD irritation is when people send me diagrams where the boxes don't line up or the lines aren't straight. Use the construction lines you idiots :mad:

Good news! Visio 2013 doesn't give a gently caress about how you want your boxes aligned. I've turned every snap to option on and off and a whole bunch of permutations and it didn't make any difference.

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

Holy poo poo, one of these latest Outlook 2013 updates fixed one of my biggest gripes about the product. I have Outlook showing my calendar on the right side and forever it would only show my meetings for today - not like Outlook 2007/2010 that would show meetings for today plus the next couple of days. Now 2013 finally got fixed. :dance:

Zamboni Apocalypse
Dec 29, 2009

Sham I Am posted:

Anyone have links to a couple of good articles on why end users should not have full Admin rights in an ERP system?

You misspelled "some documented disasters caused by end users with full Admin rights in an ERP system".

thebigcow
Jan 3, 2001

Bully!

Sham I Am posted:

Anyone have links to a couple of good articles on why end users should not have full Admin rights in an ERP system?

You could calculate the cost of restoring everything from backup including lost hours of business while you do the restoration and then have people re-enter missing data. It won't get you anywhere but planning how you're going to fix the problem now is worth it on its own.

rolleyes
Nov 16, 2006

Sometimes you have to roll the hard... two?

Sham I Am posted:

Anyone have links to a couple of good articles on why end users should not have full Admin rights in an ERP system?

Would you like your inventory to match reality, your schedule to match the capabilities of your staff, and your projects' bills of materials to match your inventory? No? In that case, go right ahead!

Scikar
Nov 20, 2005

5? Seriously?

Lord Dudeguy posted:

So how the heck do you do it? Do you basically have to pre-emptively robocopy the entire volume again before re-introducing the node? If that's the case, then I'd need to set DFSR to manual service startup and not automatic, which is the default.

:edit: ^^^ Dear christ. I'm going to start shopping for a better method. poo poo, SAMBA w/ rsync has better failure recovery features than that.

You don't robocopy it. As soon as you remove the server from the replication group and the others in the group pick up the changes from AD, they will refuse to replicate with the problem server whatever happens. When you add the server back in it is in read-only state until it is caught up with whichever server you picked as authoritative when you added it. DFSR itself is usually way quicker than robocopy unless you have a short fat pipe between the two servers, and even if you pre-seed the data it still has to process all of the files to check they match and build the DB anyway. In 2012 R2 you can export the DB from another server and import it to a new one if you're cloning them side-by-side as well but I haven't had chance to do that yet.

FlyingCowOfDoom
Aug 1, 2003

let the beat drop
I posted this in the Exchange thread as well because I'm not sure if Outlook or Exchange is the issue here.

My CIO, using Office 2013, let me know when he puts name in the To/CC/BCC field they show up as Brad Smith, like they should, but sometimes it will go Brad Smith, Amy Smith, John Smith <jsmith@companyname.com>, Hank Smith. I'm trying to figure out why it randomly adds in the bracket with the email address cause it doesn't do it all the time and it doesn't matter how you add them to the field from the GAL.

If this was a normal user I'd tell them to deal with it cause its a non issue but since its coming from on high I'm the one who has to deal with it and Dr. Google doesn't know what I'm talking about.

thebigcow
Jan 3, 2001

Bully!
Are some address book and some .nk2 entries?

EoRaptor
Sep 13, 2003

by Fluffdaddy

FlyingCowOfDoom posted:

I posted this in the Exchange thread as well because I'm not sure if Outlook or Exchange is the issue here.

My CIO, using Office 2013, let me know when he puts name in the To/CC/BCC field they show up as Brad Smith, like they should, but sometimes it will go Brad Smith, Amy Smith, John Smith <jsmith@companyname.com>, Hank Smith. I'm trying to figure out why it randomly adds in the bracket with the email address cause it doesn't do it all the time and it doesn't matter how you add them to the field from the GAL.

If this was a normal user I'd tell them to deal with it cause its a non issue but since its coming from on high I'm the one who has to deal with it and Dr. Google doesn't know what I'm talking about.

They have 'John Smith <jsmith@companyname.com>' as the persons name in their contact for that person, as well as having the email in the email field in the contact.

This can 'come and go' as a problem because outlook will only sometimes look in the contacts. If John Smith is also a cached name or a GAL name, outlook might look there first.

This can be cleaned up by editing the contact, and force deleting the name out of the completion cache.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!
Regarding DFS: How the gently caress do 3 nodes losing 1 get split-brain? Isn't that the entire loving reason to go with 3+ nodes so they have quorum? How do the two remaining nodes not become the source of truth for an automatic recovery?

I mean if you had to do manual recovery and hosed it up it's one thing for not fencing it correctly but if it's supposed to be automated the 2 nodes left up should be considered canonical by an automatic recovery system. If it decides it doesn't want to use the 2 nodes as masters then it should fail (limit access (fence) to the split-off partition without quorum) and require human intervention to unfence the split partition.

deimos fucked around with this message at 21:47 on Nov 13, 2013

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


You can get partitioned clusters for a lot of reasons. It's possible that the one node dropping out was a symptom of something else, and that something else could also have been responsible for both nodes losing the other and thinking it was time to be primary.

I'm not familiar with DFS itself, but any kind of HA configuration has the potential for split brain/partitioning in the right circumstances. A 3-node cluster doesn't completely negate the possibility.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

AlexDeGruven posted:

You can get partitioned clusters for a lot of reasons. It's possible that the one node dropping out was a symptom of something else, and that something else could also have been responsible for both nodes losing the other and thinking it was time to be primary.

I'm not familiar with DFS itself, but any kind of HA configuration has the potential for split brain/partitioning in the right circumstances. A 3-node cluster doesn't completely negate the possibility.

Agreed, but assuming that the split brain is severe enough and quorum can't be achieved should fence every node and require user interaction. If we assume that the 2 node minimum quorum existed the entire time then the event should've never happened where the split partition became the source of truth. In no way should that be able to happen, an automatic quorum like that should have fencing mechanics with human intervention before destroying data.

3+ nodes significantly reduce the chance (if we're talking same-DC different physical machines) of not having quorum at any given time, whereas split-braining 2 nodes is the guaranteed effect.

evol262
Nov 30, 2010
#!/usr/bin/perl

AlexDeGruven posted:

You can get partitioned clusters for a lot of reasons. It's possible that the one node dropping out was a symptom of something else, and that something else could also have been responsible for both nodes losing the other and thinking it was time to be primary.

I'm not familiar with DFS itself, but any kind of HA configuration has the potential for split brain/partitioning in the right circumstances. A 3-node cluster doesn't completely negate the possibility.

I find it surprising that DFS won't take a quorum vote in case of split brain and fence off the other nodes, if so.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

evol262 posted:

I find it surprising that DFS won't take a quorum vote in case of split brain and fence off the other nodes, if so.

I think he's arguing worst-case scenario where all 3 nodes stopped seeing each other at some point (to which I'd say the same thing, why didn't the split partitions self-fence because they can't reach quorum).

Edit: Even more strange is the fact that (IIRC) DFS nodes don't rely on each other for quorum, they rely on the AD DCs for it, so what the gently caress is going on.

ElGroucho
Nov 1, 2005

We already - What about sticking our middle fingers up... That was insane
Fun Shoe

GreenNight posted:

Visio and Project. I loooove getting emails from managers asking to put one or both on a users machine. My go to response is "Sure, a license is $cost, what account number should I charge it to?"

I hate that most people at my company use Visio like a suped-up PowerPoint and still think they deserve a license.

Oswald Kesselpot
Jan 14, 2008

HONK HONK HONK

Zamboni Apocalypse posted:

You misspelled "some documented disasters caused by end users with full Admin rights in an ERP system".
Oh, no. We have already had our ERP crashed by 3 separate people with admin rights. Unfortunately, those incidents occurred under different management, so it is as if they never actually happened.

Today I got the "Your department reports to me" speech when I said no to the request to give admin rights to a Jr. accountant. I wonder how many times in my career I will have to lose this argument before I finally give up the ghost and just let the auditors take care of it for me.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

deimos posted:

Regarding DFS: How the gently caress do 3 nodes losing 1 get split-brain?

That's what I want to loving know.

Fortunately the re-sync of the down node restored a good portion of data once it finished (overnight). Almost like it was replaying transactional logs as it went from volume to volume.

For those who still had missing files, I was able to dive into the share's DFSRPrivate folder and restore the... what are they, snapshots? Fragments? of the files.

We're trying to establish some sort of way to psychically establish when to consider a DFS node "down" and pull it out of DFSR and Namespace. It's 100% judgement call. If the server's up, says it's up, and the logs are clean, it might still be "down", but HEY WHO KNOWS, RIGHT?

rolleyes
Nov 16, 2006

Sometimes you have to roll the hard... two?
I have no experience with DFS other than what you guys have said, but it sounds like it's what :psyduck: was invented for. Add me to the list of people wondering how a system with 3 votes ends up treating the one vote which disagrees with the other two as authoritative.

The Space Shuttle used triple redundancy with voting for a number of critical systems; could've been a much shorter era of spaceflight if it used DFS voting (TM).

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

Lord Dudeguy posted:

That's what I want to loving know.

Fortunately the re-sync of the down node restored a good portion of data once it finished (overnight). Almost like it was replaying transactional logs as it went from volume to volume.

For those who still had missing files, I was able to dive into the share's DFSRPrivate folder and restore the... what are they, snapshots? Fragments? of the files.

We're trying to establish some sort of way to psychically establish when to consider a DFS node "down" and pull it out of DFSR and Namespace. It's 100% judgement call. If the server's up, says it's up, and the logs are clean, it might still be "down", but HEY WHO KNOWS, RIGHT?

Not sure if sarcastic or just sleep deprived. Either way :bravo:.

Spudalicious
Dec 24, 2003

I <3 Alton Brown.
Some senior citizen scientist just tied up 2 IT staff for 2 loving hours while they were trying to troubleshoot his email. I saw it and left - he had obviously sent all of his email to junk for some reason. But NOOOOOO, it wasn't him, it was us. So now my guys spend forever trying to explain how junk email works, how his account interacts with our servers, and how spam filtering works. So after 1.5 hours apparently they got it working and just turned off the spam filter and lied to him to shut him up, and he asks "Can you guys help me set up my iPad?".

loving. Christ.

Have you ever had to watch someone very old enter a 12 character password thats alphanumericsymbolic, on an iPad keyboard? If not I encourage you to set your iPad password and have the kids make Grandpa do it every time they want to use it. They will learn patience. I would normally call this a great training opportunity for a new student tech...but we are all senior techs. Time to hire a student I guess.

One of my guys counted 22 times he had to re-enter his password for imap...then another 14 for smtp. ARGGHH

e: Just FYI, I'm giving age as a reference point. I don't assume anything about people until I see them in action.

porktree
Mar 23, 2002

You just fucked with the wrong Mexican.

Sham I Am posted:

Oh, no. We have already had our ERP crashed by 3 separate people with admin rights. Unfortunately, those incidents occurred under different management, so it is as if they never actually happened.

Today I got the "Your department reports to me" speech when I said no to the request to give admin rights to a Jr. accountant. I wonder how many times in my career I will have to lose this argument before I finally give up the ghost and just let the auditors take care of it for me.

Even before we had to limit the ERP admin rights to one person because of SOX, the private auditors would have flagged anyone having that kind of access to the ERP system as serious security issue. I mean, hey, let me create a bill and pay it, or order goods and mark it paid or any other thing. Let alone doing some admin crap with the concurrent managers or fuxoring with the value sets.

nitrogen
May 21, 2004

Oh, what's a 217°C difference between friends?
I don't know much about DFS, but having dealt with database clusters and other types (veritas, etc) clusters, a main cause of issues like this is if the time was incorrect on one or all of the nodes.

It is imperative that time be synchronized on all nodes, or weird issues like this can occur.

If the time was incorrect when you brought the other node up, either too fast or slow, BAM.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

nitrogen posted:

I don't know much about DFS, but having dealt with database clusters and other types (veritas, etc) clusters, a main cause of issues like this is if the time was incorrect on one or all of the nodes.

It is imperative that time be synchronized on all nodes, or weird issues like this can occur.

If the time was incorrect when you brought the other node up, either too fast or slow, BAM.

Our network admin bought a GPS/Network/Atomic/Magic based time server and used me as an excuse because I wanted to open ports to access well known public NTP servers so I could synchronise some poo poo.

I am not even loving sure how you authorise spending ~$30K instead of opening ports to a public service and setting up an internal NTP server.

Mind you this is for <500 servers.

nitrogen
May 21, 2004

Oh, what's a 217°C difference between friends?

deimos posted:

Our network admin bought a GPS/Network/Atomic/Magic based time server and used me as an excuse because I wanted to open ports to access well known public NTP servers so I could synchronise some poo poo.

I am not even loving sure how you authorise spending ~$30K instead of opening ports to a public service and setting up an internal NTP server.

Mind you this is for <500 servers.

My employer refuses to do this for any of our datacenters, and we have over 100x that number of servers. And we also block outgoing udp/123 by default, too. I've commonly noticed our inhouse stratum 2 servers poo poo themselves for 5-10 days when the connection to one of our upstream time sources ate itself or changed, and nobody noticed until we build something that won't sync.

EDIT: I also love your avatar.

nitrogen fucked around with this message at 00:04 on Nov 14, 2013

Lamar Smith R-TX
Feb 23, 2012

This was pretty fun

Inspector_666
Oct 7, 2003

benny with the good hair

Lamar Smith R-TX posted:

This was pretty fun



I was going to try and run a sideline business "ballistically retiring" hard drives, but the range nearest to me wasn't having any of it.

Paladine_PSoT
Jan 2, 2010

If you have a problem Yo, I'll solve it

Lamar Smith R-TX posted:

This was pretty fun



Retire a printer that way. Videotape it. Post the video.

linusBorlaug
Aug 1, 2013
A ticket came in for my coworker. Our warehouse manager unplugged the server that holds the security camera recordings because it was "making weird sounds". It was rebuilding a disk array.

Adbot
ADBOT LOVES YOU

KillHour
Oct 28, 2007


Inspector_666 posted:

I was going to try and run a sideline business "ballistically retiring" hard drives, but the range nearest to me wasn't having any of it.

Were you going to shoot the drive, or shoot at the drive? Please say the former. :allears:

  • Locked thread